UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Characteristics of variable error and their effects on the type I error rate 1981

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
UBC_1981_A7_5 G48.pdf
UBC_1981_A7_5 G48.pdf [ 4.2MB ]
UBC_1981_A7_5 G48.pdf
Metadata
JSON: 1.0077122.json
JSON-LD: 1.0077122+ld.json
RDF/XML (Pretty): 1.0077122.xml
RDF/JSON: 1.0077122+rdf.json
Turtle: 1.0077122+rdf-turtle.txt
N-Triples: 1.0077122+rdf-ntriples.txt
Citation
1.0077122.ris

Full Text

CHARACTERISTICS OF VARIABLE ERROR AND THEIR EFFECTS ON THE TYPE I ERROR RATE by MARC ELIE GESSAROLI B.P.E., The U n i v e r s i t y of B r i t i s h Columbia, 1978 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTERS OF PHYSICAL EDUCATION in THE FACULTY OF GRADUATE STUDIES (PHYSICAL EDUCATION) We accept t h i s t h e s i s as conforming to the r e q u i r e d standard THE UNIVERSITY OF BRITISH COLUMBIA JUNE 1981 © M a r c E l i e G e s s a r o l i , 1981 In p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t of the requirements f o r an advanced degree a t the U n i v e r s i t y o f B r i t i s h Columbia, I agree t h a t the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r r e f e r e n c e and study. I f u r t h e r agree t h a t p e r m i s s i o n f o r e x t e n s i v e copying of t h i s t h e s i s f o r s c h o l a r l y purposes may be granted by the head of my department or by h i s o r her r e p r e s e n t a t i v e s . I t i s understood t h a t copying or p u b l i c a t i o n of t h i s t h e s i s f o r f i n a n c i a l g a i n s h a l l not be allowed without my w r i t t e n p e r m i s s i o n . Department of PHYSICAL EDUCATION The U n i v e r s i t y o f B r i t i s h Columbia 2075 Wesbrook P l a c e Vancouver, Canada V6T 1W5 Date AUGUST 4. 1981 DE-6 (2/79) ABSTRACT A common p r a c t i c e i n motor behavior research i s to analyze V a r i a b l e Error data with a repeated measures a n a l y s i s of variance. The purpose of t h i s study was to examine the degree to which blocked (VE) data s a t i s f i e s the assumptions underlying a repeated measures ANOVA. Of p a r t i c u l a r i n t e r e s t was whether the assumption of covariance homogeneity - both w i t h i n and between experimental groups - i s s a t i s f i e d i n a c t u a l experimental data. Monte Carlo procedures were used to study the e f f e c t of varying degrees cf v i o l a t i o n s of these assumptions on the Type I er r o r r a t e . The means and ranges of the c o r r e l a t i o n matrices of eight experimental data sets were studied for both raw and VE scores based upon d i f f e r e n t block s i z e s . In every s i t u a t i o n where the experimental groups were comprised of feedback and no feedback c o n d i t i o n s , the c o r r e l a t i o n matrix for the no feedback group displayed c o r r e l a t i o n s of greater magnitudes and consistency r e l a t i v e to those of the feedback c o n d i t i o n . The next phase involved using the underlying variance-covariance matrices for three of these data sets to simulate raw and VE data based on various block s i z e s . Raw data were simulated for each of four covariance heterogeneity c o n d i t i o n s : ( 1 ) e q u a l i t y w i t h i n and between the variance-covariance matrices; ( 2 ) i n e q u a l i t y w i t h i n the matrices but e q u a l i t y between the matrices; ( 3 ) e q u a l i t y w i t h i n each variance-covariance matrix but i n e q u a l i t y between the matrices; ( 4 ) i n e q u a l i t y both w i t h i n and between the two variance-covariance matrices. Populations of 10,000 subjects for each of two groups, the underlying variance-covariance matrices being dependent upon the homogeneity of covariance c o n d i t i o n being s t u d i e d , were generated based on each of three a c t u a l experimental data se t s . The data were blocked i n various ways depending on the o r i g i n a l number of t r i a l s i n the experiment (36, 24 or 18) with VE being the dependent v a r i a b l e . An experiment c o n s i s t e d of randomly s e l e c t i n g 20 subjects for each of the two groups, b l o c k i n g the t r i a l s based on s p e c i f i c block s i z e s and a n a l y z i n g the raw and VE data by a repeated measures ANOVA. The e f f e c t of i n t e r e s t was the Groups by Blocks i n t e r a c t i o n . The complete process was r e p l i c a t e d for the four covariance homogeneity c o n d i t i o n s for each of the three data s e t s , r e s u l t i n g i n a t o t a l of 22,000 simulated experiments. Results i n d i c a t e d that the Type I e r r o r rate increases as the degree of heterogeneity w i t h i n the variance-covariance matrices increases when raw (unblocked) data i s analyzed. With VE, the e f f e c t s of w i t h i n - m a t r i x heterogeneity on the Type I e r r o r rate are i n c o n c l u s i v e . However, block s i z e does seem to a f f e c t the p r o b a b i l i t y of o b t a i n i n g a s i g n i f i c a n t i n t e r a c t i o n , but the nature of t h i s r e l a t i o n s h i p i s not c l e a r as there does not appear to be any c o n s i s t e n t r e l a t i o n s h i p between the s i z e of the block and the p r o b a b i l i t y of o b t a i n i n g s i g n i f i c a n c e . For both raw and VE data there was no i n f l a t i o n in the number of Type I e r r o r s when the covariances w i t h i n a given matrix were homogeneous, regardless of the d i f f e r e n c e s between the group variance-covariance matrices. iv ACKNOWLEDGEMENTS I wish to thank the members of my t h e s i s committee, Dr. A.. John Petkau and Dr. A. Ralph Hakstian f o r t h e i r time taken i n a i d i n g me with the v a r i o u s , and o f t e n n o n - t r i v i a l , s t a t i s t i c a l problems a s s o c i a t e d with t h i s study and Dr. Gordon L. Diewert whose enthusiam and i n t e r e s t i n the study was g r e a t l y a p p r e c i a t e d . T h e i r many comments and suggestions were i n v a l u a b l e to the progress of the t h e s i s . Many thanks are a l s o given to Dr. D. Gordon E. Robertson whose knowledge of computer a p p l i c a t i o n s s i m p l i f i e d many f a c e t s of my data a n a l y s e s . S p e c i a l thanks and a l a r g e debt i s owed to my t h e s i s chairman and a d v i s o r , Dr. Robert W. Schutz, f o r h i s c o n t i n u a l support, enthusiasm and l e a d e r s h i p i n a l l aspects of my graduate l i f e , at U.B.C. Due to Dr. Schutz, my work was not only e d u c a t i o n a l , but a l s o e n j o y a b l e . The extent of h i s c o n t r i b u t i o n cannot be measured. F i n a l l y , a s p e c i a l type of g r a t i t u d e must go to my parents, whose constant support and understanding towards a seemingly endless process was, and i s , very important to me. TABLE,'OF - CONTENTS INTRODUCTION 1. METHODOLOGY 6 Phase 1 6 Phase 2 8 Pr e l i m i n a r y Analyses 9 Homogeneity Conditions 10 S e l e c t i o n C r i t e r i a 12 Simulation Procedures 13 E f f e c t Of The Number Of T r i a l s 16 RESULTS AND DISCUSSION 17 Structure Of The C o r r e l a t i o n Matrices 17 Raw Scores 17 V a r i a b l e E r r o r 19 D i s t r i b u t i o n Of Raw And VE Scores ...20 Raw Scores 20 V a r i a b l e Error 20 V i o l a t i o n s Of Covariance Homogeneity 21 Condition 1 21 Condition 2 23 Condition 3 29 Condition 4 30 E f f e c t Of Block Size ...32 CONCLUSIONS 35 REFERENCES 37 APPENDIX A - LETTER REQUESTING EXPERIMENTAL DATA 39 APPENDIX B - PROGRAM TO.CALCULATE VE AND ANOVA ....43 APPENDIX C - REVIEW OF LITERATURE 47 Overview Of Chapter 47 The S t a t i s t i c a l Model 48 Assumptions Of Repeated Measures ANOVA ...49 Assumption Of Normality ......49. Homogeneity Of Variances .....51 Homogeneity Of Covariances 52 Heterogeneity Of Covariances: D e f i n i t i o n And Measurement 53 A Measure Of Covariance Heterogeneity 53 Compound Symmetry And C i r c u l a r i t y 54 Covariance Heterogeneity And Type I Error Rates 57 Evidence Of Type I E r r o r I n f l a t i o n 57 T r a d i t i o n a l Adjustments In The Degrees Of Freedom 59 Studies Using 60 . M o d i f i c a t i o n s Of e: i And I 61 GA And IGA Tests . . .62 M u l t i v a r i a t e Techniques 63 Overview Of U n i v a r i a t e Vs M u l t i v a r i a t e Tests On Power .65 Summary 67 AE-CE-VE Debate 68 Summary 72 REFERENCES 74 LIST OF TABLES Table I - C h a r a c t e r i s t i c s Of The Raw Experimental Data Sets Received From Motor Behaviour Researchers 14 Table II - P r o p o r t i o n Of S i g n i f i c a n t G X B I n t e r a c t i o n s For Unblocked Data 22 Table III - P r o p o r t i o n Of S i g n i f i c a n t G X B I n t e r a c t i o n s For VE Data - 24 Table IV - E f f e c t Of The Number Of T r i a l s On The Type I E r r o r Rate For Raw And VE Data Based On Data Set 7 25 Table V - The Mean And Ranges Of The C o r r e l a t i o n C o e f f i c i e n t s For Various Block S i z e s 28 1 INTRODUCTION Motor performance data i s unique i n , that a. subject i s measured over numerous t r i a l s under r e l a t i v e l y constant c o n d i t i o n s . This large number of t r i a l s i s needed due to the large i n t r a - s u b j e c t v a r i a b i l i t y c h a r a c t e r i s t i c of most motor performance tasks. The reduction and s t a t i s t i c a l a n a l y s i s of these data possesses problems not encountered i n most f i e l d s of study ( p h y s i o l o g i c a l and b i o l o g i c a l measures are u s u a l l y h i g h l y r e l i a b l e and, t h e r e f o r e , often need only one or two t r i a l s ; s o c i a l psychology t e s t c o n d i t i o n s can often not be repeated without changing the c o n d i t i o n i t s e l f ) . Therefore, the purpose of t h i s study i s to examine s e l e c t e d problems a s s o c i a t e d with the a n a l y s i s of the h i g h l y interdependent repeated measures frequently encountered i n motor behaviour research. T y p i c a l motor l e a r n i n g experiments require a subject to perform a number (p) of t r i a l s on a motor task, the nature of the i n v e s t i g a t i o n being to compare the subject's performance on that task to a predetermined target score. The d i f f e r e n c e i n these two scores i s c a l l e d the subject's performance e r r o r for that t r i a l . In most instances the subjects are d i v i d e d i n t o q groups based on v a r i a b l e s such as teaching method, experimental c o n d i t i o n , previous p r a c t i c e or some other f a c t o r , r e s u l t i n g i n a q X p f a c t o r i a l experiment with repeated measures on the second f a c t o r . A technique known as " b l o c k i n g " i s often employed in an attempt to : (a) obtain a measure of i n t r a - s u b j e c t v a r i a b i l i t y (VE) or; (b) smooth the data i f the subjects' i n t e r t r i a l v a r i a b i l i t y i s l a r g e . Here, the p o r i g i n a l t r i a l s are d i v i d e d i n t o c "blocks" with each block comprised of p/c 2 o r i g i n a l t r i a l s . Any or a l l of three performance e r r o r scores are then c a l c u l a t e d for each of the new b l o c k s ; Absolute E r r o r (AE) - the mean absolute d e v i a t i o n from the ta r g e t score over the p/c t r i a l s ; Constant E r r o r (CE) - the subject's mean al g e b r a i c e r r o r over the t r i a l s and, V a r i a b l e E r r o r (VE) - the square root of the w i t h i n - s u b j e c t variance over the t r i a l s . This b l o c k i n g procedure reduces the design to a q X c f a c t o r i a l experiment with repeated measures on the second f a c t o r . S t a t i s t i c a l analyses, u s u a l l y a n a l y s i s of varian c e , are then performed on each of these dependent v a r i a b l e s with the r e s u l t of i n t e r e s t being the Groups by Blocks i n t e r a c t i o n . The use of these three e r r o r scores and the c h a r a c t e r i s t i c s t y p i c a l to most st u d i e s r e s u l t i n a number of p o s s i b l e problems. Absolute e r r o r , because i t i s an absolute value, probably has a non-normal d i s t r i b u t i o n and, t h e r e f o r e , there may be problems when using ANOVA since the assumption of normality may be v i o l a t e d ( S a f r i t , Spray & Diewert, 1980). This i n i t s e l f may not be too serious since ANOVA i s robust to non-normality i f the number of subjects i n e,ach group i s large and the population variances are equal (Boneau, 1960). However, u n t i l the d i s t r i b u t i o n of AE scores, along with the underlying variance- covariance s t r u c t u r e , has been determined, and t h e i r e f f e c t s on the Type I e r r o r rate s t u d i e d , the v a l i d i t y and i n t e r p r e t a t i o n of research using AE i s questionable. One of the assumptions of ANOVA i s that the t r i a l s have equal variances and that a l l the covariances be equal to zero. F a i l u r e to adhere to t h i s assumption r e s u l t s i n a p r o b a b i l i t y of f a l s e l y r e j e c t i n g the n u l l hypothesis greater than the set l e v e l 3 of s i g n i f i c a n c e (Box, 1954). However, i t was l a t e r shown by Lana and Lubin (1963) that the Type I e r r o r rate i s not i n f l a t e d i f the covariances are equal though not n e c e s s a r i l y equal to zero. Constant e r r o r scores are assumed to have the c h a r a c t e r i s t i c of adjacent t r i a l s being h i g h l y c o r r e l a t e d with the c o r r e l a t i o n s , decreasing as the t r i a l s become.farther apart (Gaito, 1973; Lana & Lubin, 1963). Schutz and G e s s a r o l i (1980), i n a Monte Carlo study using data based on such a variance-covariance matrix, reported many more Type I e r r o r s than i n s i m i l a r s t u d i e s i n which fewer t r i a l s were incorporated ( C o l l i e r , Baker, Mandeville & Hayes, 1967). This brings f o r t h many questions, namely: (1) Are the number of t r i a l s under which a subject i s t e s t e d r e l a t e d to the p r o b a b i l i t y of making a Type I e r r o r ? (2) How does the range of covariances a f f e c t the Type I e r r o r rate? (3) Are the magnitudes of the covariances important when using ANOVA in t e s t i n g hypotheses with CE as the dependent v a r i a b l e ? That i s , does a range of covariances from 0.6 to 0.1 r e s u l t i n the same degree of Type I e r r o r s as covariances spanning 0.9 to 0.4? Another p o t e n t i a l problem a l s o a s s o c i a t e d with b l o c k i n g i s whether varying the block s i z e d i f f e r e n t i a l l y a f f e c t s the Type I e r r o r r a t e . I t has been shown that when using CE data the s i z e of the block i s of no consequence i n the degree of i n f l a t i o n of Type I e r r o r s (Schutz & G e s s a r o l i , 1980). V a r i a b l e e r r o r i s u n l i k e e i t h e r CE or AE because i t i s a variance and,' consequently, probably has a non-normal d i s t r i b u t i o n ( S a f r i t , Spray & Diewert, 1980). However, as with AE, t h i s may not be serious depending upon the sample s i z e s and the s t r u c t u r e of the variance-covariance matrix. In t h e i r Monte 4 Carlo study Schutz and G e s s a r o l i (19"80) found no " i n f l a t i o n i n the Type I er r o r rate when VE scores were c a l c u l a t e d from raw score matrices with unequal covariances. However., the data s i m u l a t i o n procedures c a l c u l a t e d VE scores which were uncor r e l a t e d across blocks. Schutz and G e s s a r o l i used raw score covariances among t r i a l s which decreased i n a l i n e a r fashion as the t r i a l s became f a r t h e r apart. Mathematically, such a raw score covariance s t r u c t u r e a l w a y s ^ w i l l r e s u l t i n uncorrelated VE scores. I f , i n r e a l data, the VE scores are c o r r e l a t e d and these c o r r e l a t i o n s are unequal, then problems a r i s e when using ANOVA since the assumption of homogeneous covariances has been v i o l a t e d . The variance-covariance s t r u c t u r e of e m p i r i c a l VE data must be studied before i t can be s a i d with any c e r t a i n t y i f heterogeneous covariances i n the raw data a f f e c t the Type I err o r r a t e . A second part of the assumption of homogeneous covariances deals with the s t r u c t u r e of the covariance matrices between experimental c o n d i t i o n s . Not only do the covariances w i t h i n each group have to be equal, but the magnitudes of the covariances i n one matrix need to be equal to those of the variance-covariance matrices f o r the other group. This i s r e f e r r e d to as "compound symmetry'. As of now i t i s not c l e a r i f raw experimental data have covariance matrices of t h i s type. Extending t h i s concept to AE, CE and VE data i t i s a l s o not known i f t h e i r underlying covariance matrices s a t i s f y t h i s assumption. I t i s obvious that the r e s u l t s obtained when an a l y z i n g AE, CE or VE by an ANOVA are, at best, i n c o n c l u s i v e . Over the l a s t eight years there has been extensive debate 5 as to the v a l i d i t y of using these measures i n the a n a l y s i s - and i n t e r p r e t a t i o n of motor performance (e.g., Laabs, 1.973; Newell, 1976; Schmidt, 1975; Schutz, 1979). Schutz and Roy (1973) i n i t i a t e d t h i s debate when they provided a mathematical proof that AE could be w r i t t e n as a composite score of CE and VE, but in d i f f e r e n t proportions depending upon the r e l a t i v e magnitude of the CE and VE scores. Absolute e r r o r i s ther e f o r e redundant, and furthermore, i t i t i s used, i t can be prop e r l y i n t e r p r e t e d : only when the CE and VE components are known. For t h i s reason, the problems a s s o c i a t e d with the a n a l y s i s of AE data w i l l not be de a l t with i n t h i s study. When analyz i n g CE data no "absolute answers" are a v a i l a b l e i n de a l i n g with a l l the p o t e n t i a l problems but, i n general, the most common d i f f i c u l t i e s have been adequately resolved by Schutz and G e s s a r o l i (1980). Although p o t e n t i a l problems may e x i s t with the s t a t i s t i c a l analyses of a l l three e r r o r measures, AE, CE and VE, VE appears to be the l e a s t understood. Thus, t h i s study w i l l focus p r i m a r i l y on the analyses of raw and VE data. Therefore, the purpose of t h i s study i s t o : (a) discover the s t r u c t u r e of the variance-covariance matrices a s s o c i a t e d with e m p i r i c a l raw and VE data; (b) determine the d i s t r i b u t i o n s of the raw and VE data? (c) study the e f f e c t of the number of t r i a l s on Type I e r r o r s when using VE as the dependent v a r i a b l e ; (d) study the e f f e c t of the block s i z e on the Type I e r r o r rate when VE i s used as the measure of performance e r r o r ; (e) study the e f f e c t of the degree of heterogeneity of the covariances on the p r o b a b i l i t y of making a Type I e r r o r ; ( f ) study the e f f e c t of heterogeneity of the covariance matrices between the various 6 groups i n the experimental design on the Type I e r r o r r a t e . METHODOLOGY This study c o n s i s t e d of two phases -• the f i r s t d e a l i n g with the a n a l y s i s of e m p i r i c a l data and the second being a Monte Carl o study of VE data. Phase 1 The d i s t r i b u t i o n and general p a t t e r n of the variance- covariance matrices of raw and VE data were studied through the f o l l o w i n g steps: 1. L e t t e r s were sent to approximately 2 0 motor performance researchers requesting that they supply some of t h e i r a c t u a l experimental data from which VE was even t u a l l y c a l c u l a t e d and analyzed (a copy of the covering l e t t e r i s i n Appendix A). The experimental data d e s i r e d could have been l e a r n i n g or performance data but i t had to s a t i s f y two c o n d i t i o n s : (a) each subject had to perform a minimum of twelve t r i a l s on a given task and; (b) each experimental c o n d i t i o n (group) had at l e a s t tw-elve s u b j e c t s . Upon r e c e i p t of the data sets (eight were received) they were c a t e g o r i z e d by the type of experimental task (e.g., movement reproduction, r e a c t i o n time), the l e v e l of task f a m i l i a r i t y ( l e a r n i n g or performance) and the experimental c o n d i t i o n s involved (e.g., feedback, no feedback). 2 . The next step involved studying the variance-covariance 7 s t r u c t u r e of the raw e m p i r i c a l d a t a / C o r r e l a t i o n m a t rices f o r each data set were obtained v i a the s t a t i s t i c a l computer package MIDAS (Michigan I n t e r a c t i v e Data A n a l y s i s System). Separate c o r r e l a t i o n and c o v a r i a n c e matrices were c a l c u l a t e d f o r every experimental c o n d i t i o n w i t h i n a data s e t . The s t r u c t u r e of the c o r r e l a t i o n m a t rices was s t u d i e d i n the f o l l o w i n g ways: (a) the mean c o r r e l a t i o n c o e f f i c i e n t i n each matrix was c a l c u l a t e d ; (b) the maximum and minimum c o r r e l a t i o n c o e f f i c i e n t s i n each matrix were noted and; (c) an i n s p e c t i o n was conducted to see i f there was a d i f f e r e n c e i n the magnitude of the c o r r e l a t i o n c o e f f i c i e n t s of t r i a l s c l o s e together as compared to those f a r t h e r a p a r t . T h i s was done by t a k i n g the mean of a l l c o r r e l a t i o n s one t r i a l a p a r t , three t r i a l s a p a r t , f i v e t r i a l s a p a r t , e t c . In the cases where the number of t r i a l s i n the c o r r e l a t i o n matrix numbered gre a t e r than f o r t y , the mean of the c o r r e l a t i o n s one, s i x , eleven, e t c . t r i a l s apart were c a l c u l a t e d . 3. I t was imperative to d i s c o v e r the e m p i r i c a l d i s t r i b u t i o n , of the raw scores as t h i s d i s t r i b u t i o n would d i c t a t e the type of data to be simulated i n Part 2. Histograms of the frequency d i s t r i b u t i o n s of every t r i a l were obtained using the MIDAS s t a t i s t i c a l package. The main problem was to d i s c o v e r i f the data were d i s t r i b u t e d as m u l t i v a r i a t e normal. As there i s p r e s e n t l y no easy method of t e s t i n g f o r m u l t i v a r i a t e n o r m a l i t y , an examination of the marginal d i s t r i b u t i o n s was done. Although the d i s t r i b u t i o n s of the marginals would not i n d i c a t e m u l t i v a r i a t e n o r m a l i t y , a departure from u n i v a r i a t e n o r m a l i t y would c l e a r l y make the assumption of m u l t i v a r i a t e n o r m a l i t y 8 tenuous. For the purpose of t h i s , paper, data whose marginal d i s t r i b u t i o n s e x h i b i t e d u n i v a r i a t e normality were considered to be m u l t i v a r i a t e l y normally d i s t r i b u t e d . 4. The raw data were then reduced to VE scores (the s i z e of the blocks dependent upon the number of o r i g i n a l t r i a l s ) using the F o r t r a n computer program DATASNIFF (Goodman & Schutz, 1975). The data received from the researchers c o n s i s t e d of experiments having 12, 18, 20, 24, 30, 36 and 50 t r i a l s . These t r i a l s were blocked i n the f o l l o w i n g manners ( 3 X 6 defines three blocks of s i x t r i a l s / b l o c k ) : (a) Data set 1 - 50 t r i a l s : 10. X 5, 5 X 10 (b) Data set 2 - 24 t r i a l s : 8 X 3 , 6 X 4 , 3 X 8 (c) Data set 3 - 20 t r i a l s : 5 X 4 , 4 X 5 (d) Data set 4 - 1 2 t r i a l s : 4 X 3 , 3 X 4 (e) Data set 5 - 18 t r i a l s : 6 X 3 , 3 X 6 (f) Data set 6 - 30 t r i a l s : 1 0 X 3 , 6 X 5 , 5 X 6 , 3 X 1 0 (g) Data set 7 - 36 t r i a l s : 9 X 4 , 6 X 6 , 3 X 1 2 (h) Data set 8 - 20 t r i a l s : 5 X 4 , 4 X 5 5. The s t r u c t u r e of the VE variance-covariance matrices studied for each group as o u t l i n e d i n step 2 above. 6. The e m p i r i c a l d i s t r i b u t i o n of VE scores was a l s o examined by the use of histograms as explained i n step 3 above. Phase 2 This part of the study d e a l t with the a c t u a l Monte Carlo procedures used t o i n v e s t i g a t e the c h a r a c t e r i s t i c s of VE and t h e i r e f f e c t s on the Type I e r r o r r a t e . This phase c o n s i s t e d of generating data representing 500 experiments (two groups, 20 subjects/group; v a r i a b l e number of t r i a l s ) for each of four 9 variance-covariance c o n d i t i o n s , d e r i v i n g VE scores f o r various block s i z e s , and examining Type I e r r o r rates for the Groups by Blocks i n t e r a c t i o n . P r e l i m i n a r y analyses. There were two primary concerns before s i m u l a t i n g the data. F i r s t l y , as a computer program to simulate m u l t i v a r i a t e l y normal data was r e a d i l y a v a i l a b l e , i t was necessary to discover i f the data were normally d i s t r i b u t e d . A f t e r studying the histograms of the marginal d i s t r i b u t i o n s of the raw scores and VE scores i t was concluded that both sets of scores e x h i b i t e d u n i v a r i a t e normality based on -t h e i r sample s i z e s . That i s , from the shapes of the histograms any t e s t of s i g n i f i c a n c e f or normality of the marginals c l e a r l y would have f a i l e d to r e j e c t the n u l l hypothesis. The next step involved determining the procedure for simu l a t i n g VE scores. I t was e s s e n t i a l to determine i f the c o r r e l a t i o n s among VE scores for the generated data would mirror those of the o r i g i n a l experimental VE data. That i s , i t was e s s e n t i a l to determine that the generated raw data be based on covariance matrices d e p i c t i n g a c t u a l experimental data and have c o r r e l a t i o n s between blocks of VE scores s i m i l a r to the a c t u a l c o r r e l a t i o n matrices of VE scores. To examine t h i s , the var iance-covar iance matrix and vector of means fo.r the raw data were s p e c i f i e d to be e x a c t l y equal to those of an o r i g i n a l data set (Data set 3) having 20 t r i a l s . Raw data for 20 subjects were generated. One hundred of these data s e t s , each having the same covariance matrix, were generated. Each data set had d i f f e r e n t raw scores due to a d i f f e r e n t " s t a r t i n g p o i n t " being used to i n i t i a l i z e the data generation. The r e s u l t a n t data sets were 1 0 blocked and VE scores c a l c u l a t e d i n two ways: f i v e blocks of four t r i a l s / b l o c k and four blocks of f i v e t r i a l s / b l o c k . The net r e s u l t was 100 four-by-four and f i v e - b y - f i v e matrices of VE scores. To compare the c o r r e l a t i o n c o e f f i c i e n t s of Data set 3 with the c o r r e l a t i o n c o e f f i c i e n t s of the generated data, a "mean c o r r e l a t i o n matrix" was c a l c u l a t e d . This matrix was obtained by c a l c u l a t i n g the mean (across the 100 matrices) of every c o r r e l a t i o n c o e f f i c i e n t i n the same p o s i t i o n in the c o r r e l a t i o n matrix. The "mean c o r r e l a t i o n matrix" d i s p l a y e d c o e f f i c i e n t s of the same magnitude and range as the a c t u a l c o r r e l a t i o n matrix under both b l o c k i n g c o n d i t i o n s . Tests for d i f f e r e n c e s between c o r r e l a t i o n c o e f f i c i e n t s in equivalent l o c a t i o n s i n the two matrices . f a i l e d to produce s i g n i f i c a n c e at the .05 l e v e l . Based on these r e s u l t s i t was concluded that generation of raw data ( e x h i b i t i n g m u l t i v a r i a t e normality) using an e m p i r i c a l c o r r e l a t i o n matrix produces c o r r e l a t i o n s among VE scores which adequately r e f l e c t those i n the o r i g i n a l data. Homogeneity c o n d i t i o n s . The question remained as to which variance-covariance matrices to use f o r each group as the b a s i s for the data generation. As these matrices are user s p e c i f i e d , well-chosen matrices could simulate data which s a t i s f i e d or v i o l a t e d the various assumptions involved in a n a l y z i n g repeated measures data by an ANOVA. The assumptions of ANOVA req u i r e both homogeneity of the covariances w i t h i n a variance-covariance matrix as w e l l as e q u a l i t y between the variance-covariance matrices d e p i c t i n g the d i f f e r e n t experimental c o n d i t i o n s i n the design. By s p e c i f y i n g the nature of the matrix for each of two groups i t was hoped 11 that 'the . e f f e c t of v i o l a t i n g none, one. or both of these c o n d i t i o n s could be determined when VE was .the dependent v a r i a b l e . Therefore, four s t a t i s t i c a l c o n d i t i o n s which span a l l p o s s i b i l i t i e s of adherence or v i o l a t i o n of the two varian c e - covariance assumptions were used as bases f o r the.generation of raw data. The nature of the "within-group" and "between-group" covariances i n each of the s t a t i s t i c a l c o n d i t i o n s f o l l o w : . , . 1. Condition 1 ( e q u a l i t y w i t h i n ; e q u a l i t y between). The magnitude of the covariances w i t h i n each group were equal and the magnitude of the covariances between each group were a l s o equal. In order to obtain a variance-covariance matrix s a t i s f y i n g the assumption of symmetry yet r e f l e c t i n g the magnitude of the variances and covariances of the a c t u a l matrix the f o l l o w i n g procedures were employed: (a) the mean of the variances (diagonals) i n the a c t u a l variance-covariance matrix was c a l c u l a t e d . This value was used for a l l the variances i n the new homogeneous matrix and; (b) the mean of the covariances ( o f f - d i a g o n a l values) i n the a c t u a l variance-covariance matrix was c a l c u l a t e d and was used as the value to which a l l the new covariances were equal. Homogeneous matrices of t h i s type were c a l c u l a t e d based on both Group 1 and Group 2 a c t u a l variance-covariance matrices. They were used as needed to t e s t the e f f e c t of the v i o l a t i o n of the two assumptions. Generated data for the two groups i n Condition 1 r e s u l t e d i n the f o l l o w i n g variance-covariance matr i c e s : Group 1: The homogeneous matrix derived from the a c t u a l variance-covariance matrix of Group 1. 12 Group 2 : Same matrix as Group 1. 2 . C o n d i t i o n 2 ( i n e q u a l i t y w i t h i n ; e q u a l i t y between). The magnitude of the c o v a r i a n c e s w i t h i n each group were heterogeneous and the magnitude of the c o v a r i a n c e s between each group were equal. The v a r i a n c e - c o v a r i a n c e m a t r i c e s used to generate such data were: Group 1: The o r i g i n a l v a r i a n c e - c o v a r i a n c e matrix of Group 1. Group 2 : Same matrix as Group 1. 3. C o n d i t i o n 3 ( e q u a l i t y w i t h i n ; i n e q u a l i t y between). The magnitude of the c o v a r i a n c e s w i t h i n each group were homogeneous and the magnitude of the c o v a r i a n c e s between each group were heterogeneous. The v a r i a n c e - c o v a r i a n c e matrices used to generate such data were: Group 1: The homogeneous matrix d e r i v e d from the a c t u a l v a r i a n c e - c o v a r i a n c e matrix of Group 1 ( i . e . , as used i n C o n d i t i o n 1 ) . Group 2 : The homogeneous matrix based on the a c t u a l v a r i a n c e - c o v a r i a n c e matrix of Group 2 . 4. C o n d i t i o n 4 ( i n e q u a l i t y w i t h i n ; i n e q u a l i t y between). The magnitude of the c o v a r i a n c e s w i t h i n each group were heterogeneous and the magnitude of the c o v a r i a n c e s between each group were heterogeneous. Generated data f o r the two groups r e s u l t e d i n the f o l l o w i n g v a r i a n c e - c o v a r i a n c e m a t r i c e s : Group 1: The o r i g i n a l v a r i a n c e - c o v a r i a n c e matrix of Group 1. Group 2 : The o r i g i n a l v a r i a n c e - c o v a r i a n c e matrix of Group 2 . S e l e c t i o n C r i t e r i a . The primary concern of t h i s study was to i n v e s t i g a t e the Type I e r r o r r a t e (using VE as the dependent v a r i a b l e ) when the raw data had v a r y i n g degrees of h e t e r o g e n e i t y 1 3 w i t h i n and between the groups i n the design. I t was a l s o d e s i r a b l e to study the e f f e c t s of the number of t r i a l s before b l o c k i n g occurred on the subsequent number of Type I e r r o r s when VE was c a l c u l a t e d . Therefore, three sets of a c t u a l experimental data (Data sets 2, 5 and 7) were used as the bases of the s i m u l a t i o n procedures. S p e c i f i c a l l y , they were chosen based on the f o l l o w i n g design and data c h a r a c t e r i s t i c s : (a) the range of the c o r r e l a t i o n c o e f f i c i e n t s ; (b) the mean of the c o r r e l a t i o n c o e f f i c i e n t s ; (c) d i f f e r e n c e s i n the c o r r e l a t i o n matrices between the two groups; (d) the number of t r i a l s i n the experiment. A l l data were from l e a r n i n g experiments. The s p e c i f i c a t t r i b u t e s of these three data sets are shown i n Table I. S imulation procedures. Five hundred two-way experiments were simulated for each of the three data s e t s . The number of Type I e r r o r s for the Group by Block i n t e r a c t i o n , when using raw scores and VE scores as the dependent v a r i a b l e s , were analyzed v i a 500 ANOVAs for each of a number of d i f f e r e n t b l o c k i n g c o n d i t i o n s i n each case. S p e c i f i c procedures for each of these processes f o l l o w . Raw scores for a population of 10000 observations having a variance-covariance matrix and vector of t r i a l means e x a c t l y as s p e c i f i e d by the user were generated for each group. The data were produced using the computer program UBC NORMAL (Halm, 1970). The net r e s u l t was 10000 observations in each of two groups having raw scores for a s p e c i f i c number of t r i a l s . Samples of s i z e 20 per group were subsequently drawn from t h i s p o pulation. Thus the sampling was not based on an i n f i n i t e Table I C h a r a c t e r i s t i c s of the Raw Experimental Data Sets Received from Motor Behaviour Researchers Raw VE Data Set No. of t r i a l s No. of S/Group mean r range of r's mean r range of r's 1 8 KR 50 30 .10 - .65 to .90 .78 .44 to .94 > KR 24 29 .00 .62 to .67 .20 -.21 to .41 No KR 24 29 .83 - .46 to .95 .20 -.26 to .58 3 KR 20 13 .05 .57 to .78 . 14 -.11 to .29 No KR 20 13 .50 - .32 to .85 .20 -.54 to .77 No KR 20 13 .20 - .54 to .77 .25 .05 to .53 [ No KR 12 24 .20 -. .66 to .70 .20 -.50 to .50 KR 18 40 .00 .57 to .60 .20 -.12 to .31 No KR 18 40 .45 .25 to .86 .25 .02 to .44 KR 30 40 .10 -. .50 to .60 .10 -.50 to .60 f No KR 36 48 .30 ,22 to .50 .20 -.19 to .54 No KR 36 48 .15 .36 to .48 .20 -.21 to .52 i KR 20 10 .35 ,65 to .88 .00 -.53 to .80 KR 20 10 .55 -, 06 to .92 .70 .23 to .92 No KR 20 10 .30 -. 67 to .91 .45 -.21 to .90 15 popu l a t i o n . However, the sample-to-population r a t i o (20 : 1 0000) i s s u f f i c i e n t l y small to negate the need to incorporate any f i n i t e population c o r r e c t i o n f a c t o r s i n t o the analyses. The data were blocked and the VE scores were c a l c u l a t e d by a Fortran computer program (see Appendix B); the s i z e of the block being, dependent upon the number of o r i g i n a l t r i a l s and as defined i n step 4 of Phase 1 of the previous s e c t i o n . The v a l i d i t y of the c a l c u l a t i o n s was t e s t e d by comparing the c a l c u l a t e d VE scores with those produced by a program known to c a l c u l a t e accurate VE scores, DATASNIFF (Goodman & Schutz, 1975). The scores were accurate to the f o u r t h decimal p l a c e . Each experiment was analyzed by an a n a l y s i s of variance on the data of twenty subjects i n each of two groups. A computer program (Appendix B) read 40 subjects at a time (20 from Group 1 and 20 from Group 2) and c a l c u l a t e d the Sum of Squares and Mean Square E r r o r terms for a l l the e f f e c t s , and the subsequent F value for the Groups by Blocks i n t e r a c t i o n . This c a l c u l a t e d F value was compared to the c r i t i c a l F value at the .10, .05 and .01 l e v e l s of s i g n i f i c a n c e . The c r i t i c a l F values were obtained v i a the f u n c t i o n subroutine UBC FVALUE which gives the F value of a u s e r - s p e c i f i e d l e v e l of s i g n i f i c a n c e based on user- s p e c i f i e d degress of freedom. The r e s u l t s were stored by the computer where, a f t e r a l l 500 experiments had been analyzed, a ta b l e showing the number of s i g n i f i c a n t i n t e r a c t i o n s at the .10, .05 and .01 l e v e l s of s i g n i f i c a n c e was p r i n t e d . The t a b l e a l s o d i s p l a y e d the mean F value c a l c u l a t e d in the 500 ANOVAs and the average Mean Square Error for each of the Groups, Subjects w i t h i n Groups, T r i a l s , Groups by T r i a l s and Subjects w i t h i n 16 Groups by T r i a l s e f f e c t s . The a c t u a l Type I e r r o r rate was compared to the nominal l e v e l of s i g n i f i c a n c e by the standard e r r o r of a proportion as given by [p(l-p)/500] . A d i f f e r e n c e of more than two standard e r r o r s of a pr o p o r t i o n between the a c t u a l number of Type I e r r o r s committed and the nominal l e v e l of s i g n i f i c a n c e was considered s i g n i f i c a n t . The net r e s u l t was. raw and VE scores being generated for 10000 observations i n each of two groups for each of four sets of underlying v a r i a n c e - covariance matrices. This was done separately for each of the three data sets for each b l o c k i n g c o n d i t i o n r e s u l t i n g i n a t o t a l of twenty-two thousand experiments being analyzed. E f f e c t of the number of t r i a l s . In order to study the e f f e c t of the number of i n i t i a l t r i a l s on the Type I e r r o r rate when VE scores are e v e n t u a l l y c a l c u l a t e d , a d d i t i o n a l s i m u l a t i o n s were performed on Data set 7. Conditions 1 and 3 were studied using 12 and 24 t r i a l s as w e l l as the a c t u a l 36 t r i a l s . I t was p o s s i b l e to use only these two c o n d i t i o n s since they both e x h i b i t e d homogeneity of the covariances w i t h i n a variance- covariance matrix. When using a s p e c i f i c heterogeneous variance- covariance matrix as the base i t i s d i f f i c u l t to obtain an equivalent heterogeneous matrix having fewer t r i a l s . The magnitude of the covariances were equal to those i n the "mean covariance matrix" based on 36 t r i a l s . The f i r s t 12 and 24 t r i a l means of the o r i g i n a l data were used as the t r i a l means for both groups i n the 12 and 24- t r i a l c o n d i t i o n s , r e s p e c t i v e l y . The twelve t r i a l s had VE c a l c u l a t e d based upon three blocks of four t r i a l s / b l o c k and four blocks of three t r i a l s / b l o c k while the 24 t r i a l s were c o l l a p s e d i n t o data sets of three, s i x and 17 eight b l ocks. Five hundred ANOVAs were performed on each of the blocking; c o n d i t i o n s as w e l l as the o r i g i n a l (unblocked) number of t r i a l s . RESULTS AND DISCUSSION Str u c t u r e of the C o r r e l a t i o n M a trices Raw scores. I t was of i n t e r e s t to^study the patterns of the c o r r e l a t i o n matrices of the raw data and the subsequently blocked VE data for each experimental data set rece i v e d . Of p a r t i c u l a r i n t e r e s t was whether the raw data e x h i b i t e d decreasing magnitudes of the c o r r e l a t i o n c o e f f i c i e n t s as t r i a l s become f a r t h e r apart as hypothesized by Gaito (1973), Lana and Lubin (1963) and others. This pattern was common to only one of the eight data sets (Data Set 1) studied. This observation i s f u r t h e r weakened by noting that t h i s decreasing pattern occurred only for c o r r e l a t i o n s among the f i r s t s i x of the f i f t y t r i a l s i n t o t a l . The remaining c o r r e l a t i o n s seemed to be randomly v a r i a b l e i n t h e i r magnitudes. A more common occurrence (though c l e a r l y not the r u l e ) was the magnitude of l a t e r t r i a l s being g e n e r a l l y greater than that of e a r l i e r t r i a l s (Data Sets 2 and 5). These c o r r e l a t i o n s , however, d i d not e x h i b i t any p a r t i c u l a r p a t t e r n . More s t r i k i n g i s the d i f f e r e n c e i n magnitudes of the c o r r e l a t i o n s between groups of subjects who received feedback and those who d i d not. In almost every case where an experiment c o n s i s t e d of two groups, KR and no-KR, the c o r r e l a t i o n among the raw scores i n the no-KR groups was much greater than for the 18 subjects which obtained feedback (see Table I ) . The sole exception, Data set. 8, was based on only 10 subjects/group ( l e s s than the minimum c r i t e r i o n of 12 subjects/group) and, t h e r e f o r e , any conclusions based on t h i s Data set are tenuous. This i s most c l e a r l y e x e m p l i f i e d by Data Set 2 . Here the mean c o r r e l a t i o n i n the feedback group was approximately equal to zero while the no- KR group had an average c o r r e l a t i o n of about .83 (see Table I ) . Although d i f f e r e n c e s between the two groups were not as extreme in the other data s e t s , d i f f e r e n c e s s t i l l e x i s t e d and were co n s i s t e n t regardless of the type of task ( l i n e a r s l i d e , etc.) performed. These d i f f e r e n c e s can be l o g i c a l l y explained. Subjects given feedback a l t e r t h e i r motor program a f t e r each t r i a l , r e s u l t i n g in r e l a t i v e l y v a r i a b l e performances from t r i a l to t r i a l . However, these t r i a l - t o - t r i a l f l u c t u a t i o n s are not constant across s u b j e c t s , thus r e s u l t i n g i n very low c o r r e l a t i o n s between p a i r s of t r i a l s . The no KR s u b j e c t s , conversely, receive no information on which to change t h e i r responses. This r e s u l t s i n a more c o n s i s t e n t performance over the repeated measures. Although the average c o r r e l a t i o n i n the no KR groups i s greater than for KR groups, the upper l i m i t s of the c o r r e l a t i o n s are approximately equal (see Table I ) . In most cases, however, the lower bound of the c o r r e l a t i o n s i n the no feedback c o n d i t i o n s i s s l i g h t l y greater than for feedback (Data Sets 2,3,5). I t appears that one p o s s i b l e explanation for the g e n e r a l l y higher c o r r e l a t i o n s i n the no KR groups i s the greater c o r r e l a t i o n s between i n i t i a l t r i a l s . Again, t h i s i s expected since response s t r a t e g i e s vary l i t t l e i n t h i s group. Subjects, 19 during i n i t i a l t r i a l s , probably perform with a l a r g e r degree of e r r o r . The r e c e i p t of feedback may d r a s t i c a l l y a l t e r the response s t r a t e g i e s and, t h e r e f o r e , l a r g e negative c o r r e l a t i o n s between these t r i a l s r e s u l t . V a r i a b l e e r r o r . When the raw data are blocked and VEs c a l c u l a t e d the nature of the c o r r e l a t i o n matrices change. Table I d i s p l a y s the d i f f e r e n c e s i n the ranges and magnitudes of the c o r r e l a t i o n s when using VE instead of the raw scores. I t i s obvious that there i s no set pattern as to what happens to the c o r r e l a t i o n s when the raw scores are. blocked i n d i f f e r e n t ways. Data set 1 shows a larg e increase i n the magnitude of the c o r r e l a t i o n s a f t e r VE i s c a l c u l a t e d while both Data set 5 (group 2) and Data set 3 (group 2) react o p p o s i t e l y . The c a l c u l a t i o n of VE seems to increase the lower bound of the range of c o r r e l a t i o n s when compared to the raw data. With the exception of Data set 3 (group 2), every data set analyzed d i s p l a y e d t h i s f a c t . However, the opposite cannot be s a i d f o r the upper l i m i t of the c o r r e l a t i o n s . Some data sets (2,3,4,5) i n d i c a t e a decrease i n magnitude of the upper l i m i t of the c o r r e l a t i o n s while others (Data sets 6 and 8) remain unchanged. In general, though, the e f f e c t of c a l c u l a t i n g VE i s to decrease the degree of heterogeneity in the c o r r e l a t i o n matrix. Study of the c o r r e l a t i o n matrices based on VE data revealed no s p e c i f i c pattern i n the c o r r e l a t i o n c o e f f i c i e n t s . There appeared to be no d i f f e r e n c e in the strength of the c o r r e l a t i o n between adjacent t r i a l s compared to those f a r t h e r apart. Data sets 3 and 4 had lower adjacent t r i a l c o r r e l a t i o n s than those a greater distance apart, while Data set 7, d i s p l a y e d the opposite 20 ef f e c t . D i f f e r e n t block s i z e s r e s u l t e d i n v a r y i n g degrees of h e t e r o g e n e i t y i n the c o r r e l a t i o n m a t r i c e s . Table I d i s p l a y s the ranges of the c o r r e l a t i o n m a t r i c e s f o r those b l o c k i n g c o n d i t i o n s which have the l a r g e s t degree of h e t e r o g e n e i t y . I n v a r i a b l y , those c o r r e l a t i o n matrices corresponded to the experimental design having the l a r g e s t number of blocks ( i . e . , the s m a l l e s t block s i z e ) . A general c h a r a c t e r i s t i c of VE data i s that the range of the c o r r e l a t i o n s i n c r e a s e d i n v e r s e l y to the block s i z e . The e f f e c t of these heterogeneous matrices on the p r o b a b i l i t y of committing a Type I e r r o r i s d i s c u s s e d i n C o n d i t i o n 2 below. D i s t r i b u t i o n of Raw and VE scores Raw s c o r e s . Histograms p l o t t i n g the raw scores f o r each t r i a l i n each data set suggested that raw data c o u l d be assumed to be normally d i s t r i b u t e d . That i s , based on the r e l a t i v e l y small sample s i z e s in each experimental group i t was obvious that any t e s t f o r n o r m a l i t y (e.g.; Kolmogorov-Smirnov, C h i - square Goodness of F i t ) would have f a i l e d to r e j e c t the i n i t i a l assumption of n o r m a l i t y . I t i s acknowledged that the small sample s i z e s of these data s e t s would.result i n r e l a t i v e l y low power on any such d i s t r i b u t i o n a l t e s t . However, o b s e r v a t i o n s of the histograms f a i l e d to r e v e a l any obvious departures from n o r m a l i t y . V a r i a b l e e r r o r . V a r i a b l e E r r o r scores are v a r i a n c e s and, t h e r e f o r e , one would expect that they are d i s t r i b u t e d as C h i - square with the a p p r o p r i a t e degrees of freedom. S a f r i t et a l . , showed that the d i s t r i b u t i o n of VE scores i s dependent on 21 d i f f e r e n t e f f e c t s under various experimental designs. These authors s t a t e d that a non-normal d i s t r i b u t i o n , may r e s u l t , but they f e l l short of saying that the d i s t r i b u t i o n was Chi-square. I t i s w e l l known, however, that one method of making a C h i - square d i s t r i b u t i o n more normal i s by t a k i n g the square root of the raw scores. The VE score used in t h i s study was the square root of the i n t r a - s u b j e c t v a r i a b i l i t y w i t h i n a block. The histograms showed that VE was a l s o d i s t r i b u t e d as u n i v a j r i a t e l y normal. This i s understandable c o n s i d e r i n g the s i z e of the sample and the f a c t that the VE scores have been transformed by the square root function.. S a f r i t e_t a_l. , may be c o r r e c t i n s t a t i n g that the t h e o r e t i c a l d i s t r i b u t i o n of VE i s non-normal. However, l a r g e r samples and untransformed VE scores would be necessary to r e f l e c t t h i s . V i o l a t i o n s of Covariance Homogeneity Of major importance, s t a t i s t i c a l l y , i s whether the a n a l y s i s of data v i a ANOVA i s v a l i d when VE i s the dependent v a r i a b l e . This question was studied under various degrees of v i o l a t i o n of the homogeneity of covariances assumptions. Condition 1 ( e q u a l i t y w i t h i n ; e q u a l i t y between). In t h i s c o n d i t i o n the covariance assumptions are adhered to and, t h e r e f o r e , Type I e r r o r rates equal to the nominally set alphas are expected when a n a l y z i n g the raw data. Table II shows that the a c t u a l number of Type I e r r o r s d i d not s i g n i f i c a n t l y d i f f e r from the nominal rate for any of the three alpha l e v e l s examined. This was c o n s i s t e n t for a l l the data s e t s . Actual alphas which d i f f e r e d by more than two standard e r r o r s of a p r o p o r t i o n from the nominal l e v e l of s i g n i f i c a n c e were Table II P r o p o r t i o n o f S i g n i f i c a n t G X B I n t e r a c t i o n s f o r Unblocked (Raw) data Homogeneity Data s e t 2 Data s e t 5 Data s e t 7 C o n d i t i o n Nominal a 24 t r i a l s 18 t r i a l s 36 t r i a l s 1 =within .10 .118 .100 . 112 .05 .066 .058 .062 =between .01 .014 .012 .010 .10 .200* .138* . 190* 2 ^ w i t h i n .05 .164* .076* . 144* =between .01 .100* .026* .068* .10 .118 .102 . 116 3 =within .05 .056 .058 .066 ^between .01 .014 " .010 .018 .10 .178* .110 .176* 4 ^ w i t h i n .05 .136* .068 .118* ^between .01 ' .076* .018 .050* a c t u a l number o f s i g n i f i c a n t i n t e r a c t i o n s which d i f f e r by more than two s t a n d a r d e r r o r s o f a p r o p o r t i o n from the nominal l e v e l o f s i g n i f i cance 23 c l a s s i f i e d as biased. The corresponding confidence i n t e r v a l s are: f o r a= . 1 0 , (.073<.10<.127 ); for a=.05, (.031<.05<.069); and l a s t l y , for C=.01, (.001<.01<.019) . The analyses of VE scores d i s p l a y s i m i l a r r e s u l t s (see Table I I I ) . Although, i n s e v e r a l data s e t s , using VE as the dependent v a r i a b l e seem to decrease the a c t u a l number of Type I e r r o r s , the d i f f e r e n c e s are not s i g n i f i c a n t . The sole exception i s i n Data set 7 (12 t r i a l s ) where the four blocks of three t r i a l s / b l o c k d i s p l a y s h i g h l y i n f l a t e d Type I e r r o r s (see Table IV). No l o g i c a l explanation for t h i s i s apparent. The e f f e c t of the number of o r i g i n a l t r i a l s on the p r o b a b i l i t y of committing a Type I er r o r i s explained i n more d e t a i l i n the d i s c u s s i o n of Condition 3. Therefore, as a general r u l e , i t appears that the a n a l y s i s of VE data c a l c u l a t e d from raw data s a t i s f y i n g the covariance assumptions does not cause a greater number of f a l s e r e j e c t i o n of the n u l l hypothesis than i s expected. Condition 2 ( i n e q u a l i t y w i t h i n ; e q u a l i t y between). Table II shows that the v i o l a t i o n of the assumption of symmetry has the e f f e c t of i n c r e a s i n g the p r o b a b i l i t y of committing a Type I er r o r when raw data i s used. Data set 2 e x h i b i t e d the greatest degree of i n f l a t i o n with the .01 l e v e l of s i g n i f i c a n c e having the l a r g e s t percentage d i f f e r e n c e from the nominal alpha. At the .01 l e v e l of s i g n i f i c a n c e the a c t u a l proportion of Type I e r r o r s was as high as .10. The increases i n the Type I e r r o r rates were 100% and 325% at nominal alphas of .10 and .05, r e s p e c t i v e l y . I t was expected that the raw data based on the most heterogeneous variance-covariance matrix wouldhave the maximum l e v e l of T a b l e I I I P r o p o r t i o n o f S i g n i f i c a n t G X B I n t e r a c t i o n s f o r VE data Homogenei t y Data s e t 2 Data s e t 5 . Data s e t 7 C o n d i t i o n Nominal a 3X8 6X4 8X3 3X6 6X3 3X12 6X6 9X4 i .10 .096 .078 .098 .082 .096 . 110 . 100 .086 i •-within .05 .046 .038 .044 .048 .038 .058 .036 .036 ^between .01 .002 .006 .006 .006 .016 .014 .006 .000 o .10 .088 .080 .122 .084 .094 . 128* .140* .150* c. ^ w i t h i n .05 .040 .034 .054 ..044 .048 .066 .084* .100* =between .01 .006 .010 .012 .002 .012 .026* .042 .034 o . 10 .098 .086 .096 .094 .082 . 128* .092 .076 5 ^ w i t h i n .05 .040 .036 .040 .042 .048 .062 .036 .034 ^between .01 .008 .008 ' .008 .012 .006 .014 .008 .002 * * * * * A .10 .762 .922 .520 .252 .496 1 .000 1.000 1 .000 4 ^ w i t h i n .05 .628* .850* * .400 .154* .356* 1 .000* 1.000* .998* ^between .01 .•380* .658* .218* .048* .172* .998* 1.000* .992* * a c t u a l number o f s i g n i f i c a n t i n t e r a c t i o n s whi ch d i f f e r by more than tWO ! standard e r r o r s o f a p r o p o r t i o n from the nominal l e v e l o f s i g n i f i cance Table IV Ef fect of the Number of T r ia l s on the Type I Error Rate for Raw and VE data Based on Data set 7 Homogeneity Condition Nominal a 36 t r i a l s 24 t r i a l s 12 t r i a l s Unblocked (Raw) Blocked (VE) Unblocked (Raw) Blocked (VE) Unblocked (Raw) Blocked (VE) 3X12 6X6 9X4 3X8 6X4 8X3 3X4 4X3 i .10 .112 .110 .100 .086 .120 .096 .098 .094 .098 .092 . , 138* l =within .05 .062 .058 .036 .036 .068 .054 .038 .044 .050 .046 . 076* =between .01 .010 .014 .006 .000* .014 .006 .008 .004 .010 .004 . 024* o .10 .116 .128* .092 .076 .120 .108 .094 .092 .096 .082 . 140* J =within .05 .066 .062 .036 .034 .060 .042 .046 .040 .048 .042 . ,070* ^between .01 .018 .014 .008 .002 .014 .010 .008 .004 .010 .004 . 022* *actual number of s i gn i f i can t in teract ions which d i f f e r by more than two standard errors of a proportion from the nominal level of s ign i f icance ro 26 i n f l a t i o n , which, i n f a c t , d i d occur. The raw data which had the highest Type I e r r o r rate (Data set 2) was based upon a mean c o r r e l a t i o n of about 0.0 and l i m i t s of -.62 to .67. This matrix was more heterogeneous than those of both Data sets 5 and 7. However, when comparing Data sets 5 and 7 t h i s l i n e of reasoning was not v a l i d . Data set 5 which r e s u l t e d i n the smallest increase i n Type I e r r o r s had a mean c o r r e l a t i o n of approximately zero and a range from -.57 to .60. A greater number of s i g n i f i c a n t F values were obtained from Data set 7, where the mean r equalled .30 and whose c o r r e l a t i o n s l a y between -.22 and .50. Conventional t h i n k i n g would assume that the variance-covariance matrix underlying Data set 7 was l e s s heterogeneous than that for Data set 5 and, t h e r e f o r e , greater i n f l a t i o n would occur using Data set 5. In f a c t , the opposite was t r u e . Several researchers (Box, 1954b; G a i t o , 1973) have i n d i c a t e d that the degree of i n f l a t i o n increases as the number of repeated measures becomes l a r g e r . In comparing the number of Type I e r r o r s committed when using raw scores as the dependent v a r i a b l e for the d i f f e r e n t Data s e t s , i t i s c l e a r that t h i s d i d not always occur (Table I I ) . However, .the degree of heterogeneity of the variance-covariance matrices were not equal e i t h e r . I t does seem very p o s s i b l e that the number of Type I e r r o r s i s r e l a t e d to the i n t e r a c t i o n of the number of t r i a l s and the heterogeneity of the underlying -matrix. For example, Data set 2, which c o n s i s t e d of 24 t r i a l s , y i e l d e d a greater number of Type I e r r o r s than d i d Data set 7 which had 36 t r i a l s . However, Table I i n d i c a t e s that the degree of heterogeneity of the 27 covariances was greater i n Data set 2 than i n Data set 7. Therefore, i t appears that the increase in covariance heterogeneity i n Data set 2 more than compensates for the fewer number of repeated measures i n the design and thus, more Type I e r r o r s were found with Data set 2. The number of Type I e r r o r s found for the 36 t r i a l s of the raw scores i n Data set 7 i s s l i g h t l y higher than i n the study by Schutz and G e s s a r o l i (1980) which employed an equal number of t r i a l s . Using a c o r r e l a t i o n matrix ranging from .54 to .95 »they found an e m p i r i c a l Type I e r r o r rate of about .16, .12 and .05 for the .10, .05 and .01 l e v e l s of s i g n i f i c a n c e , r e s p e c t i v e l y , as compared to these r e s u l t s of .190, .144 and .068 for the same nominal alphas. The increase i s probably due to the greater heterogeneity i n the c o r r e l a t i o n matrix used as the b a s i s for the generation of raw data in t h i s study. A n a l y s i s of the VE scores showed no s i g n i f i c a n t i n f l a t i o n i n Type I e r r o r s for Data Sets 2 and 5, but d i d d i s p l a y an i n f l a t e d number of Type I e r r o r s for Data Set 7. While i t i s obvious that the e m p i r i c a l Type I e r r o r rate for a l l bloc k i n g c o n d i t i o n s i s w e l l w i t h i n two standard e r r o r s of a proportion for Data sets 2 and 5, c e r t a i n b l o c k i n g c o n d i t i o n s i n Data set 7 d i s p l a y a c t u a l o's outside t h i s range. A l l b l o c k i n g c o n d i t i o n s d i s p l a y e d an increase i n the number of s i g n i f i c a n t Groups by Blocks i n t e r a c t i o n s with the degree of i n f l a t i o n being greatest for the nine blocks case. The sole exception was the .05 l e v e l of s i g n i f i c a n c e for the 3 X 1 2 case- where the a c t u a l Type I e r r o r rate d i d not d i f f e r from the nominal rate by more than two standard e r r o r s of a p r o p o r t i o n . Table V shows the patte r n of T a b l e V The Mean and Ranges o f the C o r r e l a t i o n C o e f f i c i e n t s f o r V a r i o u s B l o c k S i z e s Data S e t B l o c k i n g P a t t e r n mean r range o f r ' s 3 X 8 Group 1 Group 2 6 X 4 Group 1 Group 2 8 X 3 Group 1 Group 2 .35 .28 .15 20 18 18 ,27 t o .45 .14 t o .40 14 t o .41 16 t o .44 .21 t o .41 26 t o .58 3 X 6 Group 1 Group 2 6 X 3 Group 1 Group 2 .27 30 .12 ,20 .19 t o .36 24 to .40 .12 t o .31 .02 t o .44 3 X 12 Group 1 Group 2 6 X 6 Group 1 Group 2 9 X 4 Group 1 Group 2 26 26 20 17 ,20 20 17 t o .40 15 to .41 11 t o .44 06 t o .51 10 t o .54 21 t o .52 29 the c o r r e l a t i o n matrices of the VE scores for the three block s i z e s of Data set 7. VE simulated with 12 t r i a l s / b l o c k had the smallest range of c o r r e l a t i o n s while the VE based on four t r i a l s / b l o c k d i s p l a y e d the greatest heterogeneity i n the c o r r e l a t i o n matrix. These r e s u l t s agree with previous research (Rogan, Reselman &Mendoza, 1979) i n that the Type I eror rate increases as the degree of heterogeneity w i t h i n a matrix increases. However, Table V shows s i m i l a r degrees of heterogeneity f o r Data set 2, yet no i n f l a t i o n i n the number of Type I e r r o r s occurs. A l s o , as the degree of heterogeneity increases as the s i z e of the block decreases, a corresponding i n f l a t i o n i n the Type I e r r o r rate does not occur. No v i a b l e r a t i o n a l e i s apparent to e x p l a i n these c o n f l i c t i n g r e s u l t s obtained for the d i f f e r e n t data s e t s . Condition 3 ( e q u a l i t y w i t h i n ; i n e q u a l i t y between). S i m i l a r to Condition 1, the covariances w i t h i n each matrix are equal, however they d i f f e r i n t h e i r magnitudes between the two groups. I t seems that the assumption of e q u a l i t y between the covariance matrices of the d i f f e r e n t experimental c o n d i t i o n s i s q u i t e robust i f the second assumption of homogeneity w i t h i n the covariance matrices i s s a t i s f i e d . With one exception, the e m p i r i c a l Type I e r r o r rate d i d not exceed the nominal value for any of the data s e t s . This held regardless whether raw scores or VE was the dependent v a r i a b l e . The d i f f e r e n c e s i n the magnitudes of the c o r r e l a t i o n s between the two groups a l s o had no e f f e c t on the Type I e r r o r r a t e . Small (.15 vs .30, Data Set 7), moderate(0 vs .45, Data 30 Set 5) and large (0 vs .83, Data Set 2) d i f f e r e n c e s i'n the mean c o r r e l a t i o n s between groups were used with the same net r e s u l t i n each case - no bias in the e m p i r i c a l Type I e r r o r rate.. The f a c t that the covariances w i t h i n the matrices were homogeneous allowed for an attempt to i s o l a t e the e f f e c t of the number of repeated measures, and subsequent block s i z e on the p r o b a b i l i t y of f a l s e l y r e j e c t i n g the n u l l hypothesis. This was done by s i m u l a t i n g raw data for designs having e i t h e r 36, 24 or 12 repeated measures where the underlying variance-covariance matrices were equal i n each case. The variance-covariance. matrices s a t i s f i e d the "within-group" homogeneity assumption but f a i l e d to adhere to the "between-group" assumption. D i f f e r e n c e s i n the number of t r i a l s had no s i g n i f i c a n t e f f e c t under t h i s c o n d i t i o n (Table V). Again, i t seems as i f the number of repeated measures i s only important when the assumption of compound symmetry i s v i o l a t e d . The only case where the number of Type I e r r o r s committed was greater than expected was when the 12 t r i a l s c o n d i t i o n of Data set 7 produced VE scores based on three t r i a l s per block. Here, the percentage of Type I e r r o r s found was .140 f o r o=.10, .070 for a=.05 and .022 for C=.01. C a l c u l a t i n g VE using four t r i a l s / b l o c k found the number of corresponding e r r o r s to be .082, .042 and .004 - a l l w i t h i n two standard e r r o r s of a proportion of the nominal l e v e l of s i g n i f i c a n c e . No l o g i c a l explanation f o r t h i s i s apparent. Condition 4 ( i n e q u a l i t y w i t h i n ; i n e q u a l i t y between). The a c t u a l experimental variance-covariance matrices for each group were used to simulate the data for t h i s c o n d i t i o n . When the raw 31 data was analyzed the r e s u l t s ranged from no i n f l a t i o n i n the number of Type I e r r o r s (Data Set 5) to serious departures for the pre-set alpha (Data Set 2). Numerous researchers, s t a r t i n g with Box (1954b), have shown that the p r o b a b i l i t y of making a Type I e r r o r increases when the two covariance assumptions are not met. As expected, .Data set 2, having the greatest degree of heterogeneity w i t h i n the matrices f o r the two groups as w e l l as the l a r g e s t descrepancy between the matrices, has the greatest Type I e r r o r r a t e . However, with Data set 5, which has moderate heterogeneity both w i t h i n and between the c o r r e l a t i o n matrices, the e m p i r i c a l l e v e l of s i g n i f i c a n c e f a i l e d to increases a p p r e c i a b l y . Data set 7, having the l e a s t degree of heterogeneity both w i t h i n and between the matrices, produced the second highest e m p i r i c a l Type I e r r o r rate (see Table I I ) . While the l a s t two f i n d i n g s c o n t r a d i c t previous research, i t must be remembered that Data set 7 had twice the number of t r i a l s (36) as d i d Data set 5 (18). Therefore, i t again appears that when the raw data i s analyzed, the degree of heterogeneity combined with the number of repeated measurements i s r e l a t e d to the p r o b a b i l i t y of committing a Type I e r r o r . The.results of the a n a l y s i s of the VE data i n i t i a l l y appear to be overwhelming because of the number of s i g n i f i c a n t i n t e r a c t i o n s obtained (Table I I I ) . However, t h i s does not n e c e s s a r i l y imply that a number of Type I e r r o r s were committed, but may r e f l e c t the fact that the VE scores between the two groups are, i n f a c t , d i f f e r e n t . This i s q u i t e p o s s i b l e since subjects r e c e i v i n g feedback supposedly have d i f f e r e n t underlying processes on which to base t h e i r responses than do subjects who 32 receive no information regarding t h e i r previous response. For Data set 7, which produced almost 100% s i g n i f i c a n t interactions., the a c t u a l experimental data was blocked i n the same way as i n the s i m u l a t i o n procedures. Analyses of variance conducted on these o r i g i n a l VE scores show that the two groups d i d i n f a c t change d i f f e r e n t l y over the blocks of t r i a l s . The c a l c u l a t e d F for the Groups by Blocks i n t e r a c t i o n s for the three blocks was 16.67, 10.51 for the s i x blocks and 6.60 for nine blocks. C l e a r l y , these are a l l s i g n i f i c a n t values. The Monte Carlo procedures produced corresponding mean F values of 20.90, 13.01 and 7.50. Although the simulated data r e s u l t e d i n higher F values i t i s q u i t e conceivable that the a c t u a l experimental data are samples from the population on which the simulated data are based. E f f e c t of Block Size The r a t i o n a l e for the choice of the s i z e of the block i n c a l c u l a t i n g VE scores i s commonly based on p r a c t i c a l c o n s i d e r a t i o n s , not s t a t i s t i c a l ones. The r e s u l t s of t h i s study i n d i c a t e , however, that the choice of the block s i z e may be a f a c t o r i n the subsequent s t a t i s t i c a l a n a l y s i s . The most l u c i d example of t h i s i s for the 12 t r i a l s of Data set 7 based on the variance-covariance matrices for Conditions 1 and 3. In Condition 1 the p r o b a b i l i t y of committing a Type I e r r o r d i f f e r e d s i g n i f i c a n t l y depending upon the block s i z e chosen. At the .10 l e v e l of s i g n i f i c a n c e , 9.2% of the experiments had s i g n i f i c a n t i n t e r a c t i o n s when VE was based on four t r i a l s / b l o c k , but jumped to 13.8% when three t r i a l s / b l o c k 33 were used. A nominal alpha equal to .05 d i s p l a y e d an increase from 4.6% to 7.6% while a s i x - f o l d increase occurred (.40%^ to 2.4%) at the .01 l e v e l of s i g n i f i c a n c e . S i m i l a r changes i n the number of Type I e r r o r s were found under Condition 3 f o r t h i s data. More i n t e r e s t i n g are the r e s u l t s of the s i m u l a t i o n s based on the a c t u a l variance-covariance matrices for each group (Condition 4). T h i s , of course, i s the one which an a c t u a l researcher would analyze. Data sets 2 and 5 both show n o t i c e a b l e d i f f e r e n c e s i n the number of Type I e r r o r s depending upon the block s i z e used to c a l c u l a t e VE. In Data set 5, c o n d i t i o n 4, the three t r i a l s / b l o c k p a t t e r n r e s u l t e d i n almost double the number of Type I e r r o r s found for s i x t r i a l s / b l o c k . The corresponding p r o b a b i l i t i e s are .496 vs .252 for a=.10, .356 vs .154 for o=.05 and .172 vs .048 at the .01 l e v e l of s i g n i f i c a n c e . In l o o k i n g at Data Set 2 (Table I I I ) i t i s obvious that using four t r i a l s / b l o c k instead of three t r i a l s / b l o c k r e s u l t s i n almost twice the number of s i g n i f i c a n t i n t e r a c t i o n s at the .10 l e v e l of s i g n i f i c a n c e and more than three times at the .01 l e v e l . The question remains as to the nature of the r e l a t i o n s h i p between the s i z e of the block and the p r o b a b i l i t y of committing a Type I e r r o r . The number of Type I e r r o r s increase i n v e r s e l y to the s i z e of the block for Data set 7 (12 t r i a l s ) under Conditions 1 and 3 and for Data set 7 under Condition 4. I t appeared that t h i s a l s o was true for Data set 2 (Condition 4) since the percentage of s i g n i f i c a n t i n t e r a c t i o n s increased from .762 to .922 (at a=.l0.) as the s i z e of the block decreased from eight t r i a l s / b l o c k to four t r i a l s / b l o c k . However, when the block 34 s i z e was f u r t h e r reduced to three t r i a l s / b l o c k the Type I er r o r rate decreased to .520. Therefore, the obtained r e s u l t s are in c o n c l u s i v e as to whether there i s a d i r e c t r e l a t i o n s h i p between block s i z e and the p r o b a b i l i t y of o b t a i n i n g a s i g n f i c a n t i n t e r a c t i o n when analyzing VE data with an a n a l y s i s of variance. Although block s i z e i s not d i r e c t l y r e l a t e d to the p r o b a b i l i t y of obta i n i n g s i g n i f i c a n c e f or VE data, i t appears that the proper choice of the s i z e of the block may d r a s t i c a l l y a f f e c t the researcher's p r o b a b i l i t y of r e j e c t i n g the n u l l hypothesis. Examining the number of s i g n i f i c a n t Groups by Blocks i n t e r a c t i o n s for Data set 5 under c o n d i t i o n 4, i t i s apparent that the p r o b a b i l i t y of ob t a i n i n g s i g n i f i c a n c e was greater when three t r i a l s / b l o c k were used i n c a l c u l a t i n g VE (Table I I I ) . In f a c t , at the .01 l e v e l of s i g n i f i c a n c e , the 6 X 3 case produced 3.6 times as many s i g n i f i c a n t i n t e r a c t i o n s as d i d the 3 X 6 bloc k i n g p a t t e r n . While the percent d i f f e r e n c e i n the number of s i g n i f i c a n t i n t e r a c t i o n s decreases as the l e v e l of s i g n i f i c a n c e increases, at c=.10, the s i x blocks case r e s u l t e d i n 1.97 times the number of s i g n i f i c a n t i n t e r a c t i o n s as when three blocks of VE were analyzed. S i m i l a r , though not as extreme, values are apparent for the r e s u l t s of Data set 2, c o n d i t i o n 4 (Table I I I ) . The f a c t that the s i z e of the blocks used to c a l c u l a t e VE may d i f f e r e n t i a l l y a f f e c t the p r o b a b i l i t y of achieving a s i g n i f i c a n t i n t e r a c t i o n undermines the r e l i a b i l i t y of VE when i t i s analyzed by an ANOVA. 35 CONCLUSIONS Based on the a n a l y s i s of the s t r u c t u r e of c o r r e l a t i o n matrices for raw and VE data of eight a c t u a l experimental data s e t s , and on Monte Carlo analyses of three of these experiments, the f o l l o w i n g conclusions can be made: 1. A " t y p i c a l " c o r r e l a t i o n p a t t e r n does not e x i s t for e i t h e r the raw data or the VE scores. 2. C o r r e l a t i o n s between raw scores for subjects r e c e i v i n g no feedback are g e n e r a l l y l e s s v a r i a b l e and 'greater i n magnitude than for those subjects who were given feedback. 3. The c o r r e l a t i o n matrix among VE scores i s u s u a l l y more homogeneous, than for unblocked data. 4. E m p i r i c a l performance e r r o r scores are marg i n a l l y normally d i s t r i b u t e d . VE scores (the square root of the w i t h i n - s u b j e c t variance) a l s o appear to have normal d i s t r i b u t i o n s . However, these r e s u l t s are based on small sample s i z e s (max=48) and, the r e f o r e , s t u d i e s with l a r g e r samples are needed to confirm t h i s . „5. Most e m p i r i c a l data sets v i o l a t e both the w i t h i n and between matrix homogeneity assumptions. 6. I f the raw data s a t i s f i e s the covariance homogeneity assumptions, then the subsequent analyses of VE scores by an a n a l y s i s of variance does not i n f l a t e the Type I e r r o r r a t e . 7. In a n a l y z i n g experiments with repeated measurements by an a n a l y s i s of variance the within-group homogeneity of covariance assumption i s more important than the between-group assumption. V i o l a t i o n of the former assumption r e s u l t s i n an increase' i n the Type I e r r o r rate when raw data i s analyzed but r e s u l t s are 36 i n c o n c l u s i v e with VE data. However, when the within-group assumption i s s a t i s f i e d and the between-group assumption i s v i o l a t e d no i n f l a t i o n i n the number of Type I e r r o r s occurs. 8. The s i z e of the block used to c a l c u l a t e VE a f f e c t s the p r o b a b i l i t y of achieving s i g n i f i c a n c e . Such a f i n d i n g questions the r e l i a b i l i t y of using VE as a dependent measure i n an ANOVA. 9. When analyz i n g raw data the number of t r i a l s i n the design does not d i f f e r e n t i a l l y a f f e c t the Type I e r r o r rate i f the within-group c o r r e l a t i o n matrices are homogeneous. I f these matrices are heterogeneous the degree of i n f l a t i o n of Type I er r o r s appears to be r e l a t e d to an i n t e r a c t i v e e f f e c t between the number of t r i a l s and the degree of heterogeneity w i t h i n the ma t r i c e s . 37 REFERENCES Boneau, C. A. The e f f e c t s of v i o l a t i o n s of assumptions u n d e r l y i n g the t t e s t . P s y c h o l o g i c a l B u l l e t i n , 1960, 57, 49- 64. Box, G. E. P. Some theorems on q u a d r a t i c forms a p p l i e d i n the study of a n a l y s i s of v a r i a n c e problems, I I . E f f e c t s of i n e q u a l i t y of v a r i a n c e and of c o r r e l a t i o n between e r r o r s i n the two-way c l a s s i f i c a t i o n . Annals of Mathematical S t a t i s t i c s , 1 954b, 25j_ 484-498. C o l l i e r , R. G., Baker, F. B., M a n d e v i l i e , G. K., & Hayes, T. F. Estimates of t e s t s i z e f o r s e v e r a l t e s t procedures based on c o n v e n t i o n a l v a r i a n c e r a t i o s in the repeated measures desi g n . Psychometrica, 1967, 32, 339-353. Davidson, M. L. U n i v a r i a t e versus m u l t i v a r i a t e t e s t s i n repeated measures experiments. P s y c h o l o g i c a l B u l l e t i n , 1972, 77, 446-452. G a i t o , J . Repeated measurements designs and t e s t s of n u l l hypotheses. E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 1973, 33, 69-75. Goodman, D. & Schutz, R. W. DATASNIFF: A program to check data and compute summary d e s c r i p t i v e s t a t i s t i c s and c o r r e l a t i o n s . Developed at the P h y s i c a l Education Q u a n t i f i c a t i o n Laboratory, U n i v e r s i t y of B r i t i s h Columbia, 1975. Halm, J . UBC NORMAL. U n i v e r s i t y of B r i t i s h Columbia Computing Center, 1970. Laabs, G. J . Retention c h a r a c t e r i s t i c s of d i f f e r e n t r e p r o d u c t i o n cues in motor short-term memory. J o u r n a l of Experimental Psychology, 1973, 100, 168-177. Lana, R. E., & Lubin, A. The e f f e c t of c o r r e l a t i o n on the repeated measures d e s i g n . E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 1963, 23^ 729-739. Newell, K. M. More on a b s o l u t e e r r o r , e t c . J o u r n a l of Motor Behavior, 1976, 8^ 139-142. Rogan, J . C. , Keselman, H. J . & Mendoza, J . L. A n a l y s i s of 38 repeated measurements. B r i t i s h J ournal of Mathematical and S t a t i s t i c a l Psychology, 1979, 32, 269-286. S a f r i t , M. J . , Spray, A. J . & Diewert, G. L. Methodological issues i n short-term motor memory research. Journal of Motor Behavior, 1980, 12, 13-28. Schmidt, R. A. A schema theory of d i s c r e t e motor l e a r n i n g . P s y c h o l o g i c a l Review, 1975, 82, 225-260. Schutz, R. W. Absolute, Constant, and V a r i a b l e E r r o r : Problems and s o l u t i o n s . In D. Mood (Ed.), Proceedings of the Colorado Measurement Symposium. U n i v e r s i t y of Colorado Press: Boulder, Colorado, 1979, 82-100. , Schutz, R. W. & G e s s a r o l i , M. E. The e f f e c t s of block s i z e and heterogeneity of covariance on Type I e r r o r rates with Constant e r r o r and V a r i a b l e e r r o r data, i n Psychology of Motor Behavior and Sport - 1979. Champaign, I l l i n o i s : Human K i n e t i c s , 1980, 633-642. Schutz, R. W., & Roy, E. A. Absolute e r r o r : The d e v i l i n d i s g u i s e . Journal of Motor Behavior, 1973, 5^ 141-153. Appendix A LETTER REQUESTING EXPERIMENTAL DATA 40 Appendix A LETTER REQUESTING EXPERIMENTAL DATA 24 June, 1980 Dear As a follow-up to our T r o i s - R i v i e r e s paper on heterogeneity of covariance and block s i z e , Marc G e s s a r o l i and I are embarking on a research p r o j e c t on VE. Very b r i e f l y , our research purposes are as f o l l o w s : 1) determine the d i s t r i b u t i o n of VE (as a variance i t i s probably d i s t r i b u t e d as chi-square) and a s c e r t a i n how t h i s a f f e c t s the d i s t r i b u t i o n of F i n a t y p i c a l repeated measures ANOVA; 2) examine t h i s e f f e c t under d i f f e r e n t c o n d i t i o n s of number of t r i a l s , b l o c k i n g parameters, and variance-covariance s t r u c t u r e s . To accomplish t h i s we plan on c o l l e c t i n g e m p i r i c a l data from researchers in the f i e l d i n order to determine the a c t u a l d i s t r i b u t i o n of VE under various experimental c o n d i t i o n s and for a v a r i e t y of dependent v a r i a b l e s . Based on the f i n d i n g s , Monte Carlo procedures w i l l be followed to simulate r e a l i t y while varying the parameters of number of t r i a l s , block s i z e and variance-covariance s t r u c t u r e . As you have probably summized by now, I would l i k e to get some of you data! We are p r i m a r i l y i n t e r e s t e d i n l e a r n i n g data, but may ( i f not enough l e a r n i n g data i s a v a i l a b l e ) a l s o look at performance data. What we would l i k e i s raw data (data sheets, computer l i s t i n g , cards, or whatever) which has been used to r e f l e c t performance e r r o r , i . e . , from l i n e a r p o s i t i o n i n g tasks, temporal accuracy, e t c . We are r e s t r i c t i n g our e m p i r i c a l samples to data sets which meet the f o l l o w i n g requirements: 1) at l e a s t 12 t r i a l s per experimental c o n d i t i o n , and 2) at lease 12 subjects per group (one or more groups). I f you have such data s e t ( s ) a v a i l a b l e I would be most a p p r e c i a t i v e i f you would send i t to us. A d e s c r i p t i o n of the experimental design and data format, an i n d i c a t i o n of what ( i f any) b l o c k i n g was performed, and, i f p o s s i b l e , a copy of any published or unpublished reports of the s t u d i e s would be necessary i n order for us to i n t e r p r e t and analyze your data. Please note - we are not conducting a review of the appropriateness of s t a t i s t i c a l analyses done i n our f i e l d , and w i l l not be r e - a n a l y z i n g your data (but j u s t l o o k i n g at the d i s t r i b u t i o n of the raw data and the VE s c o r e s ) . 41 Marc w i l l be using these data sets i n h i s Master's t h e s i s . Included on h i s committee are Dr. Ralph Hakstian, a noted psychometrician, Dr. John Petkau, a b r i l l i a n t young mathematical s t a t i s t i c i a n , and Dr. Gordon Diewert from Simon Fraser U n i v e r s i t y . They a l l view t h i s study as a c h a l l e n g i n g and worthy study. I b e l i e v e that with t h e i r help we can make a valuable c o n t r i b u t i o n to an important m e a s u r e m e n t / s t a t i s t i c a l problem i n motor behavior research. Your a s s i s t a n c e w i l l enable us to accomplish t h i s . We w i l l be glad to send you a copy of our f i n d i n g s , and reimburse you for any c o s t s a s s o c i a t e d with sending and d u p l i c a t i n g m a t e r i a l s . Thank-you i n a n t i c i p a t i o n . Yours s i n c e r e l y , R.W. Schutz Professor A p p e n d i x B P R O G R A M TO C A L C U L A T E V E AND A N O V A 43 Appendix B ' PROGRAM TO CALCULATE VE AND ANOVA DOUBLE PRECISION X(200,18),SUBSUM(200),SUBSM2(200),TR1(100) DOUBLE PRECISION.X2(200,100),TX2(200),BE(200),BE2(200),AE(200). DOUBLE PRECISION DSWGT(500),DTRIAL(500),DGRPS(500),DSWG(500) DOUBLE PRECISION TRT2(100), VE(200,60),TR2(100),DGXT(500),AE2(200) DOUBLE PRECISION TRT(100),F(500),SX,SX2,V(200,60) DOUBLE PRECISION FTOT,TDGXT,TDSWGT, TDTR,TDGRPS,TDSWG,SUBJ DOUBLE PRECISION TOTAL2,BE2G1,BE2G2,SSTR,XGT,XGTB,XTOTAL DOUBLE PRECISION SUBJCT,XTOT2,TOTAL,BETW,TRIALS,SUBTR,XGROUP DOUBLE PRECISION XGRTR,SWGT,SWG READ(8,16) NT,NSG,NTB,NREP,IB,FVAL10,FVAL5,FVAL1 16 FORMAT(4(1X,I3),1X,I1,3(1X,F5.3)) NS=NSG*2 NTOT=NS*NREP NB=NT/NTB K= 1 L=NSG SX=0. SX2 = 0. XNB=NB XNSG=NSG XNT=NT XNS=NS L2=L/2 K2=K+NSG FTOT=0. TDGXT= 0. TDSWGT=0. TDTR=0. TDGRPS=0. TDSWG=0. IT10=0. IT5=0 IT1=0 DO 105 NR=1,NREP K=1 L=NSG READ(4,1) ((X(I,J),J=1,NT),I=K,L) K=K+NSG L=L+NSG READ(5,1) ((X(I,J),J=1,NT),I=K,L) 1 FORMAT(12(1X,F10.5)/12(1X,F10.5)) IF(IB.EQ.1 ) GO TO 71 NT=NB GO TO 106 71 DO 11 I=1,NS M=0 DO 10 J=1,NT,NTB J2=J+(NTB-1) DO 9 K=J,J2 SX=SX+X(I,K) SX2=SX2+(X(I,K)**2) 9 CONTINUE 44 M=M+1 VE(I,M)=((SX2-(SX**2)/NTB)/NTB)**.5 SX=0. SX2 = 0. 10 CONTINUE 11 CONTINUE GO TO 201 106 DO 199 1=1,NS DO 198 J=1,NT VE(I,J)=X(I,J) 198 CONTINUE 199 CONTINUE 201 XTOTAL=0. K=1 L=NS SUBJ=0. TOTAL2=0. BE2G1=0. BE2G2=0. SSTR=0. XGT=0. XGTB=0. K2=K+NSG L2=L/2 DO 99 I=K,L SUBSUM(I )=0. TX2(I)=0. DO 98 M=1,NB XTOTAL=XTOTAL+VE(I,M) SUBSUM(I)=SUBSUM(I)+VE(I,M) TX2(I)=TX2(I)+(VE(l,M)**2) 98 CONTINUE SUBJ=SUBJ+(SUBSUM(I)**2) TOTAL 2=TOTAL 2+TX2(I) 99 CONTINUE XTOT2=(XTOTAL**2)/(XNB*XNS) SUBJCT=(SUBJ/XNB)-XTOT2 TOTAL=TOTAL2-XTOT2 DO 89 M=1, NB TR1(M)=0. BE(M)=0. DO 88 I=K,L2 TR1(M)=TR1(M)+VE(I,M) 88 CONTINUE BE2G1=BE2G1+(TR1(M)**2) 89 CONTINUE DO 79 M=1,NB AE(M)=0. TR2(M)=0. DO 78 I=K2,L TR2(M)=TR2(M)+VE(I,M) 78 CONTINUE SSTR=SSTR+((TR1(M)+TR2(M))**2) BE2G2=BE2G2+(TR2(M)**2) 79 CONTINUE BETW=((BE2G2+BE2G1)/XNSG)-XTOT2 45 TRIALS=(SSTR/NS)-XT0T2 SUBTR=TOTAL-SUBJCT-TRIALS DO 69 I=K,L2 DO 68 M=1,NB XGT=XGT+VE(I,M) 68 CONTINUE 69 CONTINUE DO 59 I=K2,L DO 58 M=1,NB XGTB=XGTB+VE(I,M) 58 CONTINUE 59 CONTINUE XGROUP=(((XGT**2)+(XGTB**2))/(XNB*XNSG))-XTOT2 XGRTR=BETW-TRIALS-XGROUP SWGT=TOTAL-SUBJCT-TRIALS-XGRTR SWG=SUBJCT-XGROUP DGXT(NR)=XGRTR/(XNB-1) DSWGT(NR)=SWGT/((2*(XNSG-1))*(XNB-1)) DTRIAL(NR)=TRIALS/(XNB-1) DGRPS(NR)=XGROUP DSWG(NR)=SWG/(2*(XNSG-1)) F(NR)=DGXT(NR)/DSWGT(NR) IF(F(NR).GE.FVAL10) IT10=IT10+1 IF(F(NR).GE.FVAL 5) IT5 = IT5+1 IF(F(NR).GE.FVAL1) IT1=IT1+1 FTOT=FTOT+F(NR) TDGXT=TDGXT+DGXT(NR) TDSWGT=TDSWGT+DSWGT(NR) TDTR=TDTR+DTRIAL(NR) TDGRPS=TDGRPS+DGRPS(NR) TDSWG=TDSWG+DSWG(NR) 105 CONTINUE FMEAN=FTOT/NREP TDGXTM=TDGXT/NREP TSWGTM=TDSWGT/NREP TDTRM=TDTR/NREP TDGRPM=TDGRPS/NREP TDSWGM=TDSWG/NREP WRITE(6,2) IT10,IT5, IT1, FMEAN,TDGRPM,TDSWGM,TDTRM,TDGXTM,TSWGTM 2 FORMAT('THE # OF PS LESS THAN 10=',13,/, *'THE NUMBER OF PS LESS THAN 05=',13,/, *'THE NUMBER OF PS LESS THAN 01 = ',13,/, *'THE MEAN F VALUE WAS = ',F10.5,/, *'THE MEAN FOR MS GROUPS =',F13.4,/, *'THE MEAN FOR SUB WITHIN GROUPS =',F12.4,/, *'THE MEAN FOR MS TRIALS = ',F12.4,/, *'THE MEAN FOR MS GROUPS BY TRIALS = ',F10.4,/, *'THE MEAN FOR MS SUB WITHIN GROUPS BY TRIALS = ',F10.4) STOP END Appendix C REVIEW OF LITERATURE 47 Appendix. C REVIEW OF LITERATURE Overview of Chapter The most common experiment i n motor behaviour research involves each subject performing s e v e r a l t r i a l s of a p a r t i c u l a r task. Repeated measures designs are i n v a r i a b l y used since i t i s the researcher's goal to study how the subject performs over a period of time. In t h i s way, some knowledge as to how a subject l e a r n s , f o r gets or r e t a i n s may be examined. U s u a l l y , there are at l e a s t two experimental c o n d i t i o n s i n the design, thereby a l l o w i n g f o r comparisons between various groups or treatment c o n d i t i o n s . The data are g e n e r a l l y analyzed by an a n a l y s i s of variance. The proper a n a l y s i s of repeated measures data v i a ANOVA i s dependent upon the data s a t i s f y i n g various assumptions. While the assumptions of normality and e q u a l i t y of variances are important and should be checked, the most common assumptions which are v i o l a t e d with motor l e a r n i n g data are those d e a l i n g with the heterogeneity of covariances. In f a c t , Lana and Lubin (1963) and others s t a t e d that c o r r e l a t i o n s among t r i a l s c l o s e r together are la r g e r than for those f a r t h e r apart. A l s o , because the experimental groups are g e n e r a l l y q u i t e d i f f e r e n t , the covariance matrices between the various groups are probably unequal - a v i o l a t i o n of an assumption of ANOVA. Therefore, although previous research i n t o the e f f e c t s of v i o l a t i n g the 48 assumptions of normality and e q u a l i t y of variances w i l l be summarized, the emphasis w i l l be on reviewing l i t e r a t u r e concerned with the assumption of compound symmetry (homogeneity of the covariances w i t h i n each group and between the varian c e - covariance matrices of each group). The e f f e c t s of v i o l a t i n g these assumptions on the Type I er r o r rate and methods for compensating for covariance heterogeneity w i l l be the main focus of t h i s l i t e r a t u r e review. There has been much debate i n the l i t e r a t u r e over the l a s t eleven years as to the proper choice of a dependent v a r i a b l e i n motor behaviour s t u d i e s . Some of the arguements have been made on a purely t h e o r e t i c a l b a s i s while others have considered the s t a t i s t i c a l p r o p e r t i e s of the dependent measures. As t h i s study concerns i t s e l f with the a n a l y s i s of one of these dependent v a r i a b l e s (VE) a review of the ensuing debate seems ap p r o p r i a t e . The S t a t i s t i c a l Model As mentioned p r e v i o u s l y , the common motor l e a r n i n g experiment c o n s i s t s of each subject performing s e v e r a l t r i a l s (q) of a ..specific task. U s u a l l y the subjects are d i v i d e d i n t o £ experimental groups, the r e s u l t a n t design being a p X q experimental design with repeated measures on the l a s t f a c t o r . This data i s subsequently analyzed by an a n a l y s i s of varia n c e . The model underlying a repeated measures ANOVA of t h i s type i s l i n e a r i n nature and defined by: x i j k = " + °j+<3k+»i(j) + 0 ' j k + " k i ( j ) + £ i j k where xij|< defines the score for the i t h subject i n the j t h group on the kth t r i a l ; pis the o v e r a l population mean; o- and 0k are the e f f e c t s of the j t h treatment and the kth occasion, 49 r e s p e c t i v e l y ; ^ s a constant r e l a t i n g the i t h subject with the j t h treatment group; i s t n e i n t e r a c t i o n of the j t h group with the kth occasion; ^ ^ - ( j ) * s t n e i n t e r a c t i o n of occasion k and subject i w i t h i n j ; and t - j j ^ i s the random e r r o r in the system. Furthermore, these parameters are subject to the f o l l o w i n g c o n s t r a i n t s : )°i = = J°PJk = ^ j k = ^ " k i ( j ) = °' where i = 1,..'.,N; j = 1,...,P; k=1 , . . . ,Q Assumptions of Repeated Measures ANOVA The s p e c i f i c assumptions underlying the a n a l y s i s of repeated measures data by a n a l y s i s of variance are as f o l l o w s : 1. The populations must be m u l t i v a r i a t e l y normally d i s t r i b u t e d . 2. The population variances must be equal. 3. (a) The magnitudes of the covariances w i t h i n a group must be equal. (b) The magnitudes of the covariances between each grouping f a c t o r must be equal. Assumption of normality. The f i r s t assumption underlying an a n a l y s i s of variance i s that the populations must be d i s t r i b u t e d as m u l t i v a r i a t e l y normal. However, as t e s t s f or m u l t i v a r i a t e normality are few and somewhat complex i n nature (see Gnanadesikan, p. 151-195), the l e s s s t r i n g e n t assumption of u n i v a r i a t e normality between the marginal d i s t r i b u t i o n s has been accepted as a s a t i s f a c t o r y c o n d i t i o n for a v a l i d F t e s t . Several e a r l y pieces of research have been done studying the e f f e c t s of non-normality on the p r o b a b i l i t y of committing a Type I e r r o r . Although a multitude of research regarding the e f f e c t s of non- normality e x i s t s , only a summary of the conclusions w i l l be 50 presented here. Boneau ( i 9 6 0 ) , using equal sample s i z e s and equal variances, found that an ANOVA. i s q u i t e robust to varying l e v e l s of non- normality. In f a c t , i n f l a t i o n i n the number of Type I e r r o r s was found only when one or more of the populations were non-normal ( i . e . , exponential or rectangular) and the sample s i z e s were very small ( f i v e subjects/group). As the sample s i z e s increases to 15 subjects/group, the a c t u a l number of, Type I e r r o r s was only s l i g h t l y higher than the nominal value. Scheffe (1959) has proven mathematically that the robustness of the ANOVA F t e s t increases as N becomes large with F t e s t s being p e r f e c t l y robust with i n f i n i t e sample s i z e s . Therefore, i t appears that i f the sample s i z e s and variances are equal, the F t e s t i s q u i t e robust to v i o l a t i o n s of the normality assumption with the robustness i n c r e a s i n g as N increases. When non-normality i s combined with other f a c t o r s such as unequal variances and/or covariances the r e s u l t s are d i f f e r e n t . Several i n v e s t i g a t o r s have stated that ANOVA i s f a i r l y robust to departures from normality and e q u a l i t y of variances (e.g., G a i t o , 1973; Wilson & Lange, 1972), but Bradley (1980) showed that the combination of these two f a c t o r s severely a f f e c t s the Type I er r o r r a t e . Bradley, attempting to simulate r e a l - l i f e data, found that under varying l e v e l s of unequal sample s i z e s , non-normality and variance r a t i o s , 25% of the s i t u a t i o n s f a i l e d to produce a reasonable F l e v e l when N was l e s s than 100. He found that the sample s i z e needed for robustness increased as the l e v e l of s i g n i f i c a n c e decreases. More s p e c i f i c c o n d i t i o n s and t h e i r e f f e c t s on the robustness of the t e s t are discussed i n 51 the a r t i c l e . Non-normality when combined with covariance heterogeneity has the e f f e c t of i n f l a t i n g the Type I e r r o r rate of the w i t h i n - subjects main e f f e c t , e s p e c i a l l y when using m u l t i v a r i a t e t e s t s (Mendoza, Toothaker & Nicewander, 1974; Rogan, Keselman . & Mendoza, 1979). However, when the e f f e c t of i n t e r e s t was the w i t h i n - s u b j e c t s i n t e r a c t i o n , the a c t u a l Type I e r r o r rate underestimated the nominal l e v e l of s i g n i f i c a n c e . Thus, when anal y z i n g w i t h i n - s u b j e c t s e f f e c t s from non-normal data d i s p l a y i n g heterogeneous covariances, the e f f e c t being t e s t e d must be considered. Homogeneity of variances. An e a r l y study by Hsu (1938) showed that the t - t e s t i s robust to i n e q u a l i t y of variance i f the sample s i z e s are equal. However, the a c t u a l p r o b a b i l i t y of committing a Type I e r r o r moves away from the nominal l e v e l of s i g n i f i c a n c e as the r a t i o between the variances and/or the degree of i n e q u a l i t y between sample s i z e s increase (Hsu, 1938; Scheffe, 1959). More s p e c i f i c a l l y , when the smaller variance i s a s s o c i a t e d with the l a r g e r p o p u l a t i o n , an i n f l a t i o n i n the Type I e r r o r rate occurs while i n the s i t u a t i o n where the l a r g e r population has the l a r g e r v a r i a n c e , the a c t u a l alpha underestimates the nominal l e v e l . C o l l i e r , Baker, Mandeville and Hayes (1967), i n a Monte Car l o study, found that there were no extreme departures from the nominal alpha l e v e l s i f the covariances and sample s i z e s were equal and any i n f l a t i o n which d i d occur decreased as the sample s i z e increased. As with the assumption of n o r m a l i t y , the F t e s t i s q u i t e robust to v i o l a t i o n s of the homogeneous variances assumption 52 with the degree of robustness i n c r e a s i n g with increases i n the sample , s i z e . However, as Bradley (1980) d i s p l a y e d , the i n t e r a c t i v e e f f e c t s of the v i o l a t i o n s of the various assumptions can have severe e f f e c t s on the Type I e r r o r r a t e , and the f a c t that the sample s i z e s are equal i s not s u f f i c i e n t reason to assume robustness of the ANOVA. Homogeneity of covariances. The f i n a l two assumptions can be represented by the Q X Q population variance-covariance matrix of the form: a2 pa2 p e r c r • per • p a 2 p e r p e r • • • c r = ^2 '1 P P 1 P P ' D e f i n i n g the population variance-covariance matrix f o r each l e v e l of P as Ij , the above matrix must be common to a l l l e v e l s of P ( i . e . , Z.= Z, j=1,...,P) i n the p x q design. A matrix of the above form i s s a i d to have the p r o p e r t i e s of "compound symmetry" or "u n i f o r m i t y " (Geisser, 1963) or "multisample s p h e r i c i t y " (Huynh, 1978). Studies i n motor l e a r n i n g i n which a subject i s te s t e d on many t r i a l s over time on a task, i n most cases, do not adhere to the e q u a l i t y of covariance assumptions. I t i s not u n l i k e l y to have higher c o r r e l a t i o n s between adjacent t r i a l s with the magnitude of the c o r r e l a t i o n s decreasing as the t r i a l s become f a r t h e r apart (Davidson, 1972; Greenwald, 1976; Lana & Lubin, 1963; Wilson, 1975). The question remains as to the e f f e c t on the v a l i d i t y of the ANOVA when one or more of the above assumptions are v i o l a t e d . This study i s p r i m a r i l y concerned with the e f f e c t of the the v i o l a t i o n s of these 53 assumptions and, t h e r e f o r e , the remainder of the l i t e r a t u r e review deals almost e x c l u s i v e l y with variance-covariance heterogeneity problems. Heterogenity of Covariances: D e f i n i t i o n and Measurement. A measure of covariance heterogeneity. Box (1954b) i n studying, the e f f e c t s of unequal covariances on a one-way t e s t for d i f f e r e n c e s i n treatments found that the r a t i o , SST/ S JWITHIN ' has an approximate F d i s t r i b u t i o n with degrees of freedom equal to ( q - l ) e and (q-1)(p-1)e, where € i s defined as e = q 2 («' -c. . ) V(q-1 ) [He2. . -2kl<r.2 +k 2 t f 2. ], and e i s the mean of the -column variances, c- i s the mean of the i t h row and cm. i s the mean of a l l the elements i n the population covariance matrix. Geisser and Greenhouse ( 1 9 5 8 ) , i n extending Box's f i n d i n g s , showed that e must l i e between 1 / ( q - l ) and 1. I f the variances are homogeneous and the covariances are homogeneous, e=1. Extreme degrees of heterogeneity r e s u l t i n € having a value of l / ( q - 1 ) . Under the c o n d i t i o n of complete homogeneity amongst the variances and covariances (e=1) the degree of freedom for the c r i t i c a l F are (q-1),(n-1)(q-1), while when €=1/(q-l) the t e s t s t a t i s t i c f or s i g n i f i c a n c e i s F [ l , ( n - 1 ) ] . As i s obvious, the former F value i s l e s s s t r i n g e n t than the l a t t e r , t h e r e f o r e , i t i s c a l l e d a " l i b e r a l " t e s t while the l a t t e r c r i t i c a l F value i s greater r e s u l t i n g i n a "conservative" t e s t . Applying Box's r e s u l t s from a one-way c l a s s i f i c a t i o n to the 54 two-way c l a s s i f i c a t i o n ( i . e . , a grouping f a c t o r e x i s t s ) Geisser and Greenhouse (1958) found that the adjusted degrees of freedom corresponding to the t e s t for s i g n i f i c a n c e between treatments (MS /MSC r_ ) and for i n t e r a c t i o n s (MS„ /MS ^ ) to be (q-1)e, T SwGT GxT SwGT p ( n - l ) ( q - l ) e and ( p - l ) ( q - 1 ) e , p(n-1)(q-1)c, r e s p e c t i v e l y . The upper and lower bounds for t i n the two way c l a s s i f i c a t i o n remain at 1 and l / ( q - 1 ) , thus f a c i l i t a t i n g the c a l c u l a t i o n of the l i b e r a l and conservative c r i t i c a l F values. Compound symmetry and c i r c u l a r i t y . In a Groups by T r i a l s repeated mesures design three t e s t s t a t i s t i c s (F r a t i o s ) are c a l c u l a t e d by. the ANOVA: E, =MSQ/MS^wg , a t e s t for d i f f e r e n c e s i n groups; F-j-=MSy/MSgwg-p, a t e s t f o r d i f f e r e n c e s between t r i a l s ; and FQJ =MSgx-j. /MS^Q-J- , a t e s t f o r i n t e r a c t i o n . A l l these r a t i o s are d i s t r i b u t e d as F with appropriate degrees of freedom i f compound symmetry e x i s t s in the variance-covariance matrix (with the exception of MS^/MS^g which i s not dependent upon such a r e s t r i c t i o n ) . S i m i l a r l y , i f there i s no grouping f a c t o r ( i . e . , a one-way c l a s s i f i c a t i o n ) , the t e s t s t a t i s t i c f o r d i f f e r e n c e s i n treatments i s given by F s M S ^/MS W I T H I N . F has an F d i s t r i b u t i o n i f the compund symmetry assumption i s s a t i s f i e d . Work by Rouanet and Lepine ('1970) and Huynh and F e l d t (1970) has shown that the assumption of u n i f o r m i t y or symmetry of the yariance-covariance matrices need not n e c e s s a r i l y be met for the F r a t i o to be l e g i t i m a t e . Given g t r i a l s in a one-way c l a s s i f i c a t i o n , a s u f f i c i e n t c o n d i t i o n for an exact F t e s t i s when C'IC=<72I^ ^ , where I i s the population variance- covariance matrix, I i s the i d e n t i t y matrix and C i s a (q-1)- dimensional orthonormal contrast matrix (Huynh & F e l d t , 1970; 55 Rouanet and Lepine, 1970). Both sets of authors, by d i f f e r e n t methods, show that i f t h i s c o n d i t i o n i s met (the c o n d i t i o n i s defined as " c i r c u l a r i t y " ) the Box-Geisser-Greenhouse c o r r e c t i o n f a c t o r e i s equal to one. Extending t h i s i d e a , i t f o l l o w s that i f the symmetry assumption i s s a t i s f i e d then so i s the c i r c u l a r i t y assumption. However, i t does not n e c e s s a r i l y f o l l o w that c i r c u l a r i t y i m plies symmetry of the variance-covariance matrix (Rouanet & Lepine, 1970). I t i s obvious that c i r c u l a r i t y i s a l e s s s t r i n g e n t requirement necessary to obtain v a l i d F r a t i o s by a n a l y s i s of variance. In a two-way c l a s s i f i c a t i o n (a between-groups f a c t o r e x i s t s ) the c o n d i t i o n which must be s a t i s f i e d i s : C ' l C=«r2I, p=1,...P (Huynh & F e l d t , 1970). This i m p l i e s that the c o n d i t i o n of c i r c u l a r i t y , as described above, e x i s t s for each of the P groups and that the value of C EC r e s u l t s in the same value of the s c a l a r , a2 , for each group. The primary d i f f e r e n c e between the r e s u l t s of Huynh and F e l d t and those of Rouanet and Lepine i s that the f i r s t set of authors deal only with the c i r c u l a r i t y c o n d i t i o n s for the o v e r a l l F t e s t while Rouanet and Lepine consider both o v e r a l l and p a r t i a l F t e s t s . Rouanet and Lepine showed that c e r t a i n p a r t i a l comparisons are v a l i d even i f the o v e r a l l c i r c u l a r i t y c o n d i t i o n i s not s a t i s f i e d . The example given by Rouanet and Lepine i s based upon a four by two c l a s s i f i c a t i o n with repeated measures on both f a c t o r s ( i . e . , eight treatments). They define the o v e r a l l comparison (7 df) as w e l l as three p o s s i b l e p a r t i a l comparisons based upon the two f a c t o r s (3 and 1 df) and the i n t e r a c t i o n (3 d f ) . 56 Two methods have been suggested for t e s t i n g . p a r t i a l comparisons: (1) using an e r r o r term , based upon the corresponding sum of squares as the e f f e c t being t e s t e d . For example, the denominator for the t e s t of a comparison based on f a c t o r A would be Subjects w i t h i n A. The corresponding F r a t i o i s designated as F'. (2) Using an e r r o r term based upon the o v e r a l l sum of squares ( i . e . , the sum of the three sum , of squares, S S $ W A , SS5Wg , SS5 w^g). This r a t i o i s c a l l e d F". Authors d i f f e r i n t h e i r o pinion as to which i s the proper er r o r term to use. Many t e x t s favor the us of F" only while others s t a t e that F' should be used i n a l l cases (e.g., Gaito & Turner, 1963). Since the degrees of freedom are l a r g e r i n the er r o r term for F" than for F', i t would seem that F" y i e l d s a more powerful t e s t . However, s a t i s f y i n g the c i r c u l a r i t y assumption for F" i s more d i f f i c u l t than for F'. I f the o v e r a l l c i r c u l a r i t y assumption i s s a t i s f i e d (F" i s v a l i d ) , then any of the p a r t i a l comparisons (F') are a l s o v a l i d . However, the opposite does not apply. The assumption for F' i s l e s s s t r i n g e n t than for F" and becomes weaker as the degrees of freedom i n the er r o r term decrease. Furthermore, even the s t r i c t e r c o n d i t i o n of o v e r a l l c i r c u l a r i t y i s l e s s rigorous than the c l a s s i c a l symmetry assumption. As the F t e s t i s not v a l i d i f c i r c u l a r i t y assumptions are not met, i t i s necessary to be able to t e s t f o r c i r c u l a r i t y . Huynh and F e l d t (1970) provide a t e s t for o v e r a l l c i r c u l a r i t y based on the Box t e s t (1950) and Mauchly's c r i t e r i o n W (1940). The s t a t i s t i c s c a l c u l a t e d are s i m i l a r to.those i n t e s t i n g for symmetry i n the variance-covariance matrices. Rouanet and Lepine 57 (1970) adopt a multidimensional approach i n t e s t i n g f or c i r c u l a r i t y based upon an adaptation of.Anderson's ' s p h e r i c i t y t e s t ' (1958, p.263). Although Rouanet and Lepine do not give a t e s t when there are p between-level f a c t o r s or groups, Box's (1950) t e s t could be used to t e s t C'E C=V 2I, p=1,...,P. I f the n u l l hypothesis i s not r e j e c t e d , Andersons's t e s t (1958) could subsequently be employed. Covariance Heterogeneity and Type I Err o r Rates Evidence of Type I e r r o r i n f l a t i o n . Several i n v e s t i g a t o r s (e.g., Box, 1954a,b; C o l l i e r , Baker, Mandeville & Hayes, 1967; Gaito , 1961; Geisser and Greenhouse, 1958; Lana & Lubin, 1963) have discussed the e f f e c t of covariance heterogeneity upon the Type I e r r o r r a t e . Kogan (1948) was the f i r s t to p o s t u l a t e that when the t r i a l s were p o s i t i v e l y i n t e r c o r r e l a t e d the subsequent F t e s t f or d i f f e r e n c e s i n the t r i a l s would be l i b e r a l . Box (1954b) i n v e s t i g a t e d the s i t u a t i o n where adjacent t r i a l s had c o r r e l a t i o n s equal to zero. He found that the p r o b a b i l i t y of obt a i n i n g a s i g n i f i c a n t p-value increased as the c o r r e l a t i o n s increased from 0 to ±.40. As the magnitude of the c o r r e l a t i o n increases the value of € decreases. When r=0, e=1; with l i t t l e c o r r e l a t i o n (r=.20), €=.9507 and a c o r r e l a t i o n of .40 r e s u l t e d in c e q u a l l i n g .8033. The corresponding negative c o r r e l a t i o n s r e s u l t e d i n e p s i l o n values of .9640 and .8862. Negative c o r r e l a t i o n s have l e s s of an e f f e c t on the Type I e r r o r rate than do t h e i r p o s i t i v e counterparts. Box concluded that as the value of e decreased the p r o b a b i l i t y of f a l s e l y r e j e c t i n g the n u l l hypothesis increased. Gaito (1973) c a l c u l a t e d e p s i l o n 58 values for c o r r e l a t i o n s greater than .40 for a covariance s t r u c t u r e s i m i l a r to that of Box (1954b) and found that e p s i l o n decreased q u i t e r a p i d l y as the c o r r e l a t i o n increased (e.g., r=.60, €=.5977? r=.80, e=.4009? r=.90, €=.3189). He found the Type I erro r rate increased s i m i l a r l y , with a c o r r e l a t i o n of +.90 r e s u l t i n g i n an a c t u a l p r o b a b i l i t y of making a Type I e r r o r of .16 at the .05 l e v e l of s i g n i f i c a n c e . C o l l i e r , Baker, Mandeville and Hayes (1967) stud i e d s e v e r a l very simple covariance matrices having high adjacent t r i a l c o r r e l a t i o n s with the magnitudes of the c o r r e l a t i o n s decreasing as the t r i a l s become f a r t h e r apart. Using only four t r i a l s and c o r r e l a t i o n s ranging from .80 to .20, they found the p - l e v e l s to be about twice as large as the expected .05 and three to f i v e times as large at the .01 l e v e l of s i g n i f i c a n c e . I t i s q u i t e p o s s i b l e that many studies have more than four t r i a l s and the subsequent e r r o r rate could be much higher than those reported by C o l l i e r et a l . , (1967). Schutz and G e s s a r o l i (1980) used a c o r r e l a t i o n matrix with a s i m i l a r magnitude and p a t t e r n of c o r r e l a t i o n s but had data for each of 36 t r i a l s . Their Monte Carlo study r e s u l t e d i n a Type I e r r o r rate of .17 at the .10 l e v e l , .12 at an alpha of .05 and .05 at the .01 l e v e l •- a degree of i n f l a t i o n greater than that of C o l l i e r et a_l. , (1967). This i s c o n s i s t e n t with r e s u l t s of Box (1954b) who discovered that the value of e p s i l o n decreases i n v e r s e l y with the number of t r i a l s . Wilson (1975), i n a s i m u l a t i o n study based on each "subject" having 10 t r i a l s , used an a r b i t r a r y c o r r e l a t i o n matrix with the c o r r e l a t i o n s ranging from 0 to 0.98. The Type I e r r o r 59 rate was c o n s i s t e n t with the high degree of covariance heterogeneity and moderate number of t r i a l s . At the 5% l e v e l of s i g n i f i c a n c e the a c t u a l Type I e r r o r rate was over 20% and at the 1% l e v e l i t was about 13%. T r a d i t i o n a l adjustments i n the degrees of freedom. Several methods have been suggested to deal with the problems produced by covariance heterogeneity; some are m e t h o d o l i g i c a l ; some focus on the choice of s t a t i s t i c a l t e s t , and others t r y and reduce the bias in the F r a t i o by a l t e r i n g the degrees of freedom. Greenhouse and Geisser (1959) based on the previous work of Geisser and Greenhouse (1958) and Box (1954b) proposed a three step procedure i n a n a l y z i n g repeated measures experiments. They suggested f i r s t doing a conservative F t e s t . This involves using the lower bound of e p s i l o n , 1 / ( q - l ) , where q i s the number of t r i a l s , thereby making the adjusted degrees of freedom 1 and (N- 1) d.f. for the t e s t of a t r i a l s e f f e c t . In the groups by t r i a l s design the conservative t e s t for an i n t e r a c t i o n e f f e c t would be d i s t r i b u t e d as F with 1 and p ( n - l ) degrees of freedom, where p i s the number of groups and n i s the number of subjects under each l e v e l of p. If t h i s proved s i g n i f i c a n t , the t e s t would be f i n i s h e d . I f , however, the n u l l hypothesis was not r e j e c t e d , then an F t e s t based on the conventional degrees of freedom (e=1) should be done. Here the degrees of freedom corresponding to the t e s t s for a t r i a l s e f f e c t and group by t r i a l s i n t e r a c t i o n would be (q-1 ) , (q-1 ) (n-1 ) and (q-1 •) ,p(q-1) (n-1 ) , r e s p e c t i v e l y . I f the F r a t i o i s n o n - s i g n i f i c a n t the. t e s t i n g i s f i n i s h e d . I f the s i t u a t i o n a r i s e s where the conservative t e s t proves non- 60 s i g n i f i c a n t and the conventional t e s t s i g n i f i c a n t , then an attempt must be made to estimate €. The exact value of € would give the a c t u a l d i s t r i b u t i o n of the F r a t i o . Studies using e As i s obvious from the e a r l i e r equation d e f i n i n g e p s i l o n , e can be c a l c u l a t e d only i f the population variance-covariance matrix i s known. In a c t u a l experimental data the population values are never known. Geisser and Greenhouse c a l c u l a t e d the sample estimate (e) of c i n the same manner, as the o r i g i n a l equation, with,, the population variances and covariances being s u b s t i t u t e d by the corresponding sample s t a t i s t i c s . The degrees of freedom of the c r i t i c a l F are then reduced using Z rather than €. Several studies have i n v e s t i g a t e d the e f f e c t of using e i nstead of € i n c o n t r o l l i n g for Type I e r r o r s . C o l l i e r et al., (1967) found t h a t , i n general, e was a good estimate of e p s i l o n . However, I i s a conservative estimate of e when the population value i s near one r e s u l t i n g i n a somewhat conservative t e s t of the n u l l hypothesis. The sampling d i s t r i b u t i o n of e i s n e g a t i v e l y skewed at i t s upper l i m i t but becomes l e s s v a r i a b l e and l e s s biased as the population value decreases ( C o l l i e r , Baker, Mandeville & Hayes, 1967.; Mendoza, Toothaker & Nicewander, 1974; Rogan, Keselman and Mendoza, 1979; S t o l o f f , 1970; Wilson, 1975). S t o l o f f (1970) reported data which i n d i c a t e d t h a t , as the sample s i z e i n c r e a s e s , the t e s t using e to adjust the degrees of freedom r e s u l t s i n the e m p i r i c a l Type I e r r o r rate i s c l o s e r to the nominal rate when e i s approximately one. The d i f f e r e n c e i n Type I e r r o r s using c and t. decreases as the sample s i z e increases and 61 as € decreases ( C o l l i e r et a l . , 1967; S t o l o f f , 1970). An i n t e r e s t i n g aspect of S t o l o f f s study i s how.e and i react when the number of t r i a l s increased. He found that as the t r i a l s increased, the magnitude of the Type I e r r o r s increased when t was used as the c o r r e c t i o n f a c t o r . However, when the degrees of freedom were reduced by c , the p r o b a b i l i t y of making Type I e r r o r s decreased. This was c o n s i s t e n t under varying l e v e l s of e. I t appears that the sample estimate of e p s i l o n c o n t r o l s the Type I e r r o r rate b e t t e r than the population value as the number of t r i a l s increase. As the maximum number of t r i a l s used was f i v e , f u r t h e r i n v e s t i g a t i o n should be undertaken to see how conservative the t e s t using e becomes as the l e v e l s of the repeated f a c t o r increase to a much higher degree. Modif i c a t i o n s of e j_ e and e ̂  The f a c t that the value of e p s i l o n based upon sample data i s n e g a t i v e l y biased at high l e v e l s of € caused Huynh and F e l d t (1976) to develop a new s t a t i s t i c to adjust the degrees of freedom i n the F r a t i o . This estimator, I , e l i m i n a t e s most of the negative bias i n the t e s t fo r s i g n i f i c a n c e wjien € i s used. They define I as: I = [n(k-1)€-2]/(k-1)[n-1-(k-1 ) l ] for the one-way c l a s s i f i c a t i o n with k t r i a l s and, for the groups by t r i a l s design: e = [N(k-1)€-2]/(k-1)[N-g-(k-1)e], where N i s the t o t a l number of subjects and g i s the number of groups. In the l a t t e r design, € i s c a l c u l a t e d by using the pooled estimates of the sample variance-covariance matrices f o r each of the g groups. T h i s , of course, assumes that a l l the i n d i v i d u a l population variance-covariance matrices are equal for 62 a l l the groups. Huynh (1978) deals with the case when t h i s i s not t r u e . Huynh and F e l d t (1976) note that for any values of n and k, e i s always greater than €, with t h i s d i f f e r e n c e decreasing as n increases. This formula for € allows i t to have a value greater than one when there i s a high degree of homogeneity i n the matrix. In t h i s case, the upper l i m i t i s exceeded. Therefore, e i s equated to one i f the a c t u a l c a l c u l a t i o n of e i s greater than one. Huynh and F e l d t (1976), i n a Monte C a r l o study .comparing \ and e i n c o n t r o l l i n g for Type I e r r o r s under varying l e v e l s of c (.363<c^1.000) found t h a t , i n general, i i s the b e t t e r estimator when € i s greater than 0.75 while c i s superior at higher degrees of heterogeneity. They a l s o discovered that both t e s t s behave d i f f e r e n t l y depending upon the number of groups and s u b j e c t s . They s t a t e , " I t can be seen that the t e s t based on e i s more s a t i s f a c t o r y when the parameter i s r e l a t i v e l y low or when the number of blocks or subjects i s f a i r l y l a r g e . The t e s t based on I, on the other hand, behaves very w e l l at the nominal ten or f i v e per cent l e v e l s i n a l l of the s i t u a t i o n s considered. At the nominal 2.5 and 1 percent l e v e l s i t gives somewhat more rel a x e d , but reasonably adequate, c o n t r o l over Type I e r r o r whenever the covariance matrix i s not extremely heterogeneous. This t e s t i s l e s s dependent on the number of b l o c k s , and i s f a i r l y good even with a block s i z e as small as twice the number of treatment l e v e l s . " (p. 80) GA and IGA t e s t s . Huynh (1978) extended the work of Huynh and F e l d t (1976) to consider the case when the various population matrices are heterogeneous. Two t e s t s , the General 63 Approximate t e s t (GA t e s t ) 'and the Improved General Approximate t e s t (IGA t e s t ) were developed to deal with t h i s s i t u a t i o n . The GA and IGA a l s o have the added f l e x i b i l i t y of being s u i t a b l e for t e s t s with unequal sample s i z e s . Huynh (1978), comparing a l l four t e s t s (e, c, GA and IGA) i n a s i t u a t i o n where the matrices almost e x h i b i t e d multisample s p h e r i c i t y found that the GA and e approximate t e s t s always e r r on the l i b e r a l s i d e . However, the IGA and e t e s t s y i e l d e d b e t t e r o v e r a l l c o n t r o l of the Type I e r r o r r a t e . Huynh then compared the IGA and e t e s t s under eleven d i f f e r e n t heterogeneity c o n d i t i o n s with the r e s u l t that the IGA t e s t tended to f u n c t i o n b e t t e r than the approximation, although both were s l i g h t l y l i b e r a l . However, most d i f f e r e n c e s were at smaller l e v e l s of s i g n i f i c a n c e or when the sample s i z e s were q u i t e large (N=30). Huynh concludes that although the IGA t e s t i s more accurate and f l e x i b l e , i t i s computationally more complex and, i n many s i t u a t i o n s , the e approximate procedure f u n c t i o n s as w e l l as the IGA t e s t and, therefore i s more d e s i r a b l e . M u l t i v a r i a t e techniques. An a l t e r n a t i v e to the various c o r r e c t i o n techniques a p p l i e d when repeated measures data i s anlayzed by an a n a l y s i s of variance i s a m u l t i v a r i a t e a n a l y s i s of variance (MANOVA). M u l t i v a r i a t e a n a l y s i s of v a r i a n c e , which req u i r e s no assumptions of within-group variance or covariance homogeneity, has been fre q u e n t l y recommended as the appropriate technique for a l l repeated measures designs (Davidson, 1972; Morrow & Frankiewicz, 1979; Schutz, 1978). Among the basic assumptions i n m u l t i v a r i a t e a n a l y s i s of variance are: (a) the data are d i s t r i b u t e d as m u l t i v a r i a t e 64 normal, and ..'(b) the group covariance matrices all-come from, a s i n g l e population covariance matrix. However, while MANOVA has l e s s s t r i n g e n t assumptions, v i o l a t i o n s may have serious consequences on the Type I e r r o r r a t e . The e f f e c t s of v i o l a t i n g the assumption of normality are g e n e r a l l y not severe. Mardia (1971) and I t o (1969) found that the m u l t i v a r i a t e t e s t s are q u i t e robust to departures from m u l t i v a r i a t e normality, e s p e c i a l l y i f the sample s i z e s are equal. Studies i n v e s t i g a t i n g the assumption of equal covariance matrices between groups found that the Type I e r r o r rate i s c o n t r o l l e d under moderate degrees of heterogeneity i f the sample s i z e s are equal (Holloway & Dunn, 1967; Hakstian, Roed & L i n d , 1979; I t o fc.Schull, 1964; Rogan, Keselman & Mendoza, 1979). Holloway and Dunn, however, found that sample s i z e e q u a l i t y does not n e c e s s a r i l y ensure c o n t r o l of the number of Type I e r r o r s committed as the r a t i o of the sample s i z e to the number of dependent v a r i a b l e s and the degree of covariance heterogeneity are a l s o important. Using a r a t i o of 10:1 between the variances i n the two covariance matrices, Holloway and Dunn discovered that equal sample s i z e s of 25 were s u f f i c i e n t when only two or three v a r i a t e s were used but, for 10 v a r i a t e s , the m u l t i v a r i a t e t e s t , H o t e l l i n g ' s T 2, was not robust u n t i l the sample reached 100. In r e l a t i n g these r e s u l t s to a c t u a l b e h a v i o r a l data, i t must be remembered that a r e a l i s t i c extreme for the r a t i o between population variances i s only 2.5 (Hakstian, Roed & L i n d , 1979). Hakstian et a l . , (1979), using variance scale f a c t o r s up to 2.5 showed that the T 2 procedure was r e l a t i v e l y robust to v i o l a t i o n s i n the covariance assumption, even when the r a t i o 65 between subjects and dependent v a r i a b l e s was as low as 3:1. While the t e s t of main e f f e c t s appear to be r e l a t i v e l y robust, other m u l t i v a r i a t e procedures t e s t i n g for s i g n i f i c a n t i n t e r a c t i o n s d i d not show the same r e s u l t s . In studying the e f f e c t s of covariance heterogeneity (with equal sample s i z e , r a t i o of subjects to v a r i a t e s approximately 4:1) on the t e s t s for s i g n i f i c a n t i n t e r a c t i o n s , Rogan, Keselman and Mendoza (1979) discovered an i n f l a t i o n i n the number of Type I e r r o r s . These increases were s l i g h t for the P i l l a i - B a r t l e t t trace c r i t e r i o n , and Wilk's l i k e l i h o o d r a t i o c r i t e r i o n , but were much l a r g e r (as high as .070 at alpha equal to .05) when Roy's l a r g e s t root c r i t e r i o n was used. When unequal sample s i z e s e x i s t , the Type I . e r r o r rate f l u c t u a t e s g r e a t l y , with the Type I e r r o r r a t e s i n c r e a s i n g q u i c k l y to very unacceptable l e v e l s as the degree of heterogeneity increases, even at small sample s i z e r a t i o s as low as 2:1. In the most extreme case s t u d i e d , with 10 v a r i a t e s , 50 subjects i n one group compared to 10 i n the other, and the variances in one group scaled at 2.5 times the magnitude Of the other group, the Type I e r r o r rates were: for O=.01, .152; for a=.05, .337 and; for C=.10, .473 (Hakstian et a l . , 1979). C l e a r l y , as the authors point out, "the T 2 procedure i s not robust i n the face of covariance matrix heterogeneity coupled with unequal n's, even for r e l a t i v e l y minor departures from e q u a l i t y of the covariance matrices, sample s i z e s or both." (p. 1261) Overview of u n i v a r i a t e vs m u l t i v a r i a t e t e s t s on power. In general, when the u n i v a r i a t e assumptions regarding the 66 covariance matrices are met, the conventional u n i v a r i a t e ANOVA i s more powerful than m u l t i v a r i a t e techniques (Mendoza, Toothaker & Nicewander, 1974; Rogan et a l . , 1979). Of i n t e r e s t i s the comparison between the power of the adjusted u n i v a r i a t e t e s t s (e.g., £, e), the conventional u n i v a r i a t e t e s t and m u l t i v a r i a t e t e s t s under various l e v e l s of within-group and between-group covariance matrix heterogeneity. When a l l covariance assumptions are met the conventional u n i v a r i a t e t e s t i s more powerful than e i t h e r the adjusted u n i v a r i a t e t e s t s or the m u l t i v a r i a t e t e s t s . However, as the degree of within-group matrix heterogeneity increases the m u l t i v a r i a t e t e s t s become more powerful i n d e t e c t i n g s i g n i f i c a n c e for d i f f e r e n c e s i n the main e f f e c t s . Rogan e_t a l . , (1979), found that as the value of £ decreased the power of a l l the t e s t s decreased, but the m u l t i v a r i a t e t e s t s decreased at a slower r a t e . As the degree of covariance heterogeneity increases the power of the adjusted u n i v a r i a t e t e s t s are of concern since they are the t e s t of s i g n i f i c a n c e . I t appears that when e p s i l o n dips below .75 the m u l t i v a r i a t e t e s t s more often detect the d i f f e r e n c e s i n the means (Mendoza et a l . , 1974; Rogan et a l . , 1979). When €^.75 the adjusted u n i v a r i a t e t e s t s are more powerful than t h e i r m u l t i v a r i a t e counterparts. Mendoza et a l . , (1974), found that the power of d e t e c t i n g small i n t e r a c t i o n s was greatest for Roy's l a r g e s t root c r i t e r i o n but i n d e t e c t i n g large d i f f e r e n c e s , the adjusted u n i v a r i a t e t e s t s were more powerful (for e<.75). Rogan et a l . , examined the power of three m u l t i v a r i a t e t e s t s for i n t e r a c t i o n and reported s i m i l a r r e s u l t s as i n the t e s t for main e f f e c t s , that being that 67 the m u l t i v a r i a t e t e s t s were more powerful that the u n i v a r i a t e t e s t s . I t should be noted, however, that Roy's l a r g e s t root c r i t e r i o n had the greatest Type I e r r o r rate under covariance heterogeneity and, caution must be employed i f i t i s to be used. Summarizing, i f a l l the covariance assumptions are met, the conventional u n i v a r i a t e t e s t i s the best to use in t e s t i n g for both i n t e r a c t i o n s or main e f f e c t s . With moderate l e v e l s of heterogeneity i n the covariance matrices (e^.75) the adjusted u n i v a r i a t e t e s t s are best and, g e n e r a l l y , when €<.75 the m u l t i v a r i a t e t e s t s are the most powerful. Summary. When de a l i n g with data which e x h i b i t s heterogeneity of covariances (as i s common i n repeated measures behavioral data) the e a s i e s t , and often s u f f i c i e n t method of c o r r e c t i n g f or t h i s heterogeneity i s to use the three-step procedure as o u t l i n e d by Geisser and Greenhouse (1959). However, i f a sample estimate of e need be c a l c u l a t e d to adjust the degrees of freedom there are s e v e r a l choices. I f t i s l e s s than .75 the best u n i v a r i a t e s t a t i s t i c i s e , but i f e p s i l o n i s greater than .75 e i t h e r i or IGA approximate t e s t s are the most powerful yet c o n t r o l f or the Type I e r r o r r a t e . Of the l a t t e r two, the e i s much e a s i e r to c a l c u l a t e and i s q u i t e often as good i n c o n t r o l l i n g f o r Type I e r r o r s as the IGA t e s t . M u l t i v a r i a t e t e s t s do not depend upon the assumption of w i t h i n - group covariance homogeneity and, as such, may often be the pr e f e r r e d method of a n a l y s i s . They prove to be more powerful than t h e i r u n i v a r i a t e counterparts when e<.75 but are weaker above t h i s l e v e l . M u l t i v a r i a t e t e s t s , however, do require that the covariance matrices between groups come from a common 68 population matrix, an assumption which may not be often s a t i s f i e d i n motor behavior research. AE-CE-VE Debate A considerable controversy has developed i n the past ten years regarding which s t a t i s t i c s (AE, CE or VE) should be used as mesures of a subject's performance on some motor performance task. The debate has been p r i m a r i l y between those researchers who are concerned with the s t a t i s t i c a l and mathematical p r o p e r t i e s of AE, CE and VE and those i n v e s t i g a t o r s who are more i n t e r e s t e d i n the conceptual i n t e r p r e t a t i o n of these scores. An e x c e l l e n t review of t h i s debate i s given by Schutz (1979). While seve r a l researchers had p r e v i o u s l y commented on the appropriateness of these performance measures (Burdick, 1972; Laabs, 1973; Schmidt, 1970; Underwood, 1957; Woodworth, 1938) the problem received s e r i o u s c o n s i d e r a t i o n a f t e r a paper by Schutz and Roy (1973) proved mathematically that AE i s d i r e c t l y r e l a t e d to CE and VE and, as such, can only be i n t e r p r e t e d i n l i g h t of the l a t t e r two measures. They stated that a l l the information of AE i s found i n CE when the r a t i o of CE//VE i s greater than 2.0 or i s i n VE when CE i s approximately equal to zero. AE i s a weighted combination of CE and VE when 0<CE//VE<2.0. As the mathematical .derivations, discussed above, were based on the assumption that the raw performance scores are normally d i s t r i b u t e d , the v a l i d i t y of t h e i r conclusions decreases as the departure from normality increases. The use of V a r i a b l e e r r o r as the optimal measure of w i t h i n - subject v a r i a b l i t y was questioned by Burdick (1972) and Schutz, Roy and Goodman (1973) because i t d i d not r e f l e c t the temporal 69 dimension of performance e r r o r s . An a l t e r n a t e choice of measures such as the Mean Square Successive D i f f e r e n c e , the A u t o c o r r e l a t i o n and the C o e f f i c i e n t of Temporal V a r i a b i l i t y have been suggested by Burdick (1972). Schutz et a l . , (1973) suggested that the non-normal d i s t r i b u t i o n of a variance r e s u l t s i n a l o s s of power when VE i s analyzed by an ANOVA and i n d i c a t e d that the a u t o c o r r e l a t i o n c o e f f i c i e n t be used as an a d d i t i o n a l measure of i n t r a - s u b j e c t v a r i a b l i t y . S a f r i t , Spray and Diewert (1980), in examining the t h e o r e t i c a l d i s t r i b u t i o n of VE, stated that VE may not be normally d i s t r i b u t e d , but f a i l e d to conclude that the d i s t r i b u t i o n was d e f i n i t e l y non-normal. One of the purposes of t h i s study i s to determine i f the d i s t r i b u t i o n of VE scores c a l c u l a t e d from a c t u a l raw scores i s non-normally d i s t r i b u t e d . I f the e m p i r i c a l d i s t r i b u t i o n i s normal, many of the concerns of Schutz et a l . , (1973) and S a f r i t et a l . , (1980) w i l l not be v i t a l i n an a l y z i n g VE data by an ANOVA. Henry (1974) agreed with Schutz and Roy (1973) on the inadequacy of AE. While s t a t i n g that CE and VE must always be looked at when i n t e r p r e t i n g performance e r r o r , he s a i d t h a t , at times, i t may be necessary to use a composite score. Henry suggested using E 2 (where E 2=CE 2+VE 2) to which Schutz (1974) r e p l i e d that E 2 i s s t i l l a composite score and must be i n t e r p r e t e d from CE and VE scores. Henry (1975), using m u l t i p l e c o r r e l a t i o n s , showed that E 2 was bet t e r than AE since the e f f e c t of VE i s never excluded i n E 2 while i t may be i n AE (when CE//VE>2.0). Schutz (1979) conceded t h a t , i f a composite measure had to be used, then E 2 i s p r e f e r a b l e to AE but i t s t i l l must be i n t e r p r e t e d with respect to CE and VE. 70 Jones (1974) suggested that AE, not VE i s the appropriate e r r o r score when the c r i t e r i o n i s changed for each t r i a l of a s i m i l a r task. Roy (1974), r e p l i e d that since KR i s not given on every t r i a l , the t y p i c a l movement reproduction experiment i s not a l e a r n i n g s i t u a t i o n but a f o r g e t t i n g one. Roy argued that VE i s a measure of f o r g e t t i n g and lack of consistency i n performance which does not require the c r i t e r i o n , for each t r i a l to be s i m i l a r i n order to be i n t e r p r e t e d . Schmidt (1975) favored the use of AE c l a i m i n g that for motor r e c a l l s t u d i e s i t i s the p r e f e r a b l e dependent measure for the f o l l o w i n g reasons: (a) the use of two dependent v a r i a b l e s (CE and VE) may y i e l d d i f f e r e n t r e s u l t s , thereby confusing any i n t e r p r e t a t i o n of r e s u l t s ; (b) AE i s the t r a d i t i o n a l measure, and (c) since the subject i s required to minimize h i s e r r o r on each t r i a l , AE i s what should be measured. Schutz (1979) responded to each of these arguements, r e s p e c t i v e l y , as such: (a) any theory should s a t i s f y both performance dimensions as suggested by the CE and VE scores; (b) the fa c t that AE has been the t r a d i t i o n a l measure i s s u f f i c i e n t reason to continue using i t ; and (c) since the purpose of the researcher i s to e x p l a i n performance, not only to measure i t , CE and VE must be used i n the i n t e r p r e t a t i o n . In 1976, Newell stated that when one h a l f of the subjects have p o s i t i v e CE's while the other h a l f have negative CE's, the use of an average CE i s inap p r o p r i a t e and AE should be used. In t h i s s i t u a t i o n Schutz (1979) agreed with Henry (1975) i n that the absolute value of CE, |CE|, i s the best measure. The AE-CE-VE controversy then s h i f t e d from the t h e o r e t i c a l 71 i n t e r p r e t a t i o n s of these measures to more s t a t i s t i c a l ones.Roy (1976) sta t e d that a good method of re p o r t i n g a l l three e r r o r terms (AE or E, CE and VE) i n studie s i s to analyze a l l three measures by a MANOVA since i t c o n t r o l s f or the Type I e r r o r r a t e . Roy provided a footnote which i n d i c a t e d t h a t , based on work by Schutz and Roy, AE may be a l i n e a r composite of CE and VE and, t h e r e f o r e , a MANOVA could not be c a l c u l a t e d . However, he state d that t h i s r a r e l y occurs across a l l s u b j e c t s . Thomas (1977) r e p l i e d that even though an absolute l i n e a r r e l a t i o n s h i p between the three dependent v a r i a b l e s may not e x i s t , the problem of m u l t i c o l l i n e a r i t y does. M u l t i c o l l i n e a r i t y has the e f f e c t of in c r e a s i n g the Type I e r r o r rate (Press, 1972). Thomas suggested an a l y z i n g VE and CE with a MANOVA and doing a separate ANOVA f o r AE or E. In r e p l y i n g to Thomas (1977), Roy (1977) agreed with the concept of m u l t i c o l l i n e a r i t y but fu r t h e r complicated the issue by i n d i c a t i n g that a high c o r r e l a t i o n may e x i s t between CE and VE, thereby making a t e s t of these v a r i a b l e s by a MANOVA subject to the e f f e c t s of m u l t i c o l l i n e a r i t y . S a f r i t , Spray and Diewert (1980) caution against the use of a l l AE, CE and VE i n a MANOVA for d i f f e r e n t reasons. An assumption i n MANOVA designs i s that the j o i n t p r o b a b i l i t y vector of the random vector be m u l t i v a r i a t e l y normally d i s t r i b u t e d . S a f r i t et §_1. , showed that CE i s m a r g i n a l l y normal, but both VE and AE may be ma r g i n a l l y non-normal, and concluded that u n t i l future e m p i r i c a l work shows that the v i o l a t i o n s of these assumptions are not s e r i o u s , a n a l y z i n g AE, CE and VE by a MANOVA should be avoided. E a r l i e r work, however, has shown that the T 2 procedure i s r e l a t i v e l y robust to m u l t i v a r i a t e non-normality (Mardia, 1971). 72 E a r l i e r , Thomas and Moon (1976) found AE scores to have higher r e l i a b i l i t i e s than VE and a greater number of s i g n i f i c a n t d i f f e r e n c e s were obtained with AE. These f a c t s along with t h e i r f i n d i n g that AE appeared.to be more normally d i s t r i b u t e d about the target than VE allowed them to conclude that AE i s the best dependent measure when conducting motor rhythm experiments. S a f r i t e_t a l . , (1980) i n s t a t i n g that the d i s t r i b u t i o n s of AE and VE may be non-normal caution i n v e s t i g a t o r s i n an a l y z i n g these dependent measures by an ANOVA. However, the v i o l a t i o n of the normality assumption by i t s e l f i s not serious (Boneau, 1960), but when i n t e r a c t i v e with v i o l a t i o n s of other assumptions, the Type I e r r o r rate i s a f f e c t e d (Bradley, 1980). Therefore, i f the researcher h e s i t a t e s i n using an ANOVA due s o l e l y to non-normality, he should check the other assumptions to see i f they are s a t i s f i e d . While the area of which dependent measure i s proper to use and report i s obviously confusing, the f o l l o w i n g r u l e of thumb i s g e n e r a l l y accepted. Any i n v e s t i g a t o r who can provide a l o g i c a l explanation as to what information AE provides i s j u s t i f i e d i n r e p o r t i n g i t ( S a f r i t et a l . , 1980). Summary. As the wealth of l i t e r a t u r e has i n d i c a t e d , the choice of the dependent measure to be analyzed and i n t e r p r e t e d i s a subject of great controversy. Much of the debate deals with the conceptual i n t e r p r e t a t i o n of these measures, and thus i s out of the range of the s t a t i s t i c i a n , but a great deal of uncer t a i n t y surrounds the d i s t r i b u t i o n s and e f f e c t s of using these dependent v a r i a b l e s i n an a n a l y s i s of var i a n c e . Although many of the present problems w i l l s t i l l e x i s t , h o p e f u l l y , the 73 question of the e m p i r i c a l d i s t r i b u t i o n of VE and i t s subsequent e f f e c t on the Type I e r r o r rate w i l l be adequately resolved at the c o n c l u s i o n of t h i s study. 74 REFERENCES Anderson, T. W. An In t r o d u c t i o n to M u l t i v a r i a t e A n a l y s i s . New York: Wiley, 1958. Boneau, C. A. The e f f e c t s of v i o l a t i o n s of. assumptions underlying the t t e s t . P s y c h o l o g i c a l B u l l e t i n , 1960, 57, 49- "64. Box, G. E. P. Problems i n the a n a l y s i s of growth and wear curves. B i o m e t r i c s , 1950, 6, 362-389. Box, G. E. P. Some theorems on quadratic forms a p p l i e d in the study of a n a l y s i s of variance problems. I . E f f e c t of i n e q u a l i t y of variance i n the one-way c l a s s i f i c a t i o n . Annals of Mathematical S t a t i s t i c s , 1954a, 25, 290-302. Box, G. E. P. Some theorems on quadratic forms a p p l i e d i n the study of a n a l y s i s of variance problems, I I . E f f e c t s of i n e q u a l i t y of variance and of c o r r e l a t i o n between e r r o r s i n the two-way c l a s s i f i c a t i o n . Annals of Mathematical S t a t i s t i c s , 1954b, 25j_ 484-498. Bradley, J . V. Nonrobustness in Z, t , and F t e s t s at large sample s i z e s . B u l l e t i n of the Psychonomic S o c i e t y , 1980, 16, 333-336. Burdick, J . A. Measurements of " V a r i a b i l i t y " . Journal of General Psychology, 1972, 86, 201-206. C o l l i e r , R. 0., Baker, F. B., Mande v i l l e , G. R., & Hayes, T. F. Estimates of t e s t s i z e for s e v e r a l t e s t procedures based on conventional variance r a t i o s i n the repeated measures design. Psychometrica, 1967, 32j_ 339-353. Davidson, M. L. U n i v a r i a t e versus m u l t i v a r i a t e t e s t s i n repeated measures experiments. P s y c h o l o g i c a l B u l l e t i n , 1972, 77, 446-452. Gait o , J . Repeated measurements designs and counterbalancing. P s y c h o l o g i c a l B u l l e t i n , 1961, 58, 46-54. Ga i t o , J . Repeated measurements .designs and t e s t s of n u l l hypotheses. Educational and P s y c h o l o g i c a l Measurement, 1973, 33, 69-75. 75 G a i t o , J . & Turner, E. D. Error terms i n trend a n a l y s i s . P s y c h o l o g i c a l B u l l e t i n , 1963, 60, 464-474. Gei s s e r , S. M u l t i v a r i a t e a n a l y s i s of variance for a s p e c i a l covariance case. Journal of the American S t a t i s t i c a l A s s o c i a t i o n , 1963, 58, 600-669. Ge i s s e r , S. & Greenhouse, S. W. An extension of Box's r e s u l t s on the use of the F d i s t r i b u t i o n , i n m u l t i v a r i a t e a n a l y s i s . Annals of Mathematical S t a t i s t i c s , 1958, 29, 855-891. Gnanadesikan, R. Methods for s t a t i s t i c a l data a n a l y s i s of m u l t i v a r i a t e observations. New York: Wiley, 1977. Goodman, D. & Schutz, R. W. DATASNIFF: A program to check data and compute summary d e s c r i p t i v e s t a t i s t i c s and c o r r e l a t i o n s . Developed at the P h y s i c a l Education Q u a n t i f i c a t i o n Laboratory, U n i v e r s i t y of B r i t i s h Columbia, 1975. Greenwald, A. G. Within-subjects designs: To use or not to use? P s y c h o l o g i c a l B u l l e t i n , 1976, 83, 314-320. Greenhouse, S. W. & G e i s s e r , S. On methods i n the a n a l y s i s of p r o f i l e data. Psychometrika, 1959, 24, 95-112. Hakstian, A. R., Roed, J . C. & L i n d , J . C. Two-sample T 2 procedure and the assumption of homogeneous covariance matrices. P s y c h o l o g i c a l B u l l e t i n , 1979, 86, 1255-1263. Halm, J . UBC NORMAL. U n i v e r s i t y of B r i t i s h Columbia Computing Center, 1970. Henry, F. M. V a r i a b l e and constant performance e r r o r s w i t h i n a group of i n d i v i d u a l s . Journal of Motor Behavior, 1974, 6, 149- 154. Henry, F. M. Absolute e r r o r vs. "E" i n target accuracy. Journal of Motor Behavior, 1975, 7, 227-228. Holloway, L. N. & Dunn, 0. J . The robustness of H o t e l l i n g ' s T 2. Journal of the American S t a t i s t i c a l A s s o c i a t i o n , 1967, 62, 124-136. Hsu, P. L. C o n t r i b u t i o n s to the theory of Student's t - t e s t as 76 a p p l i e d to the problem of two samples. S t a t i s t i c a l Research Methods, 1938, 2, 1-24. Huynh, H. Some approximate t e s t s for. repeated measurements designs. Psychometrika,.1978, 48, 161-175. Huynh, H. & F e l d t , L. Conditions under which mean square r a t i o s i n repeated measurements designs have exact F - d i s t r i b u t i o n s . Journal of the American S t a t i s t i c a l A s s o c i a t i o n , 1970, 65, 1582-1585. Huynh, H. & F e l d t , L. S. Est i m a t i o n of the Box c o r r e c t i o n for degrees of freedom from sample data i n randomized block and s p l i t - p l o t designs. Journal of Educational S t a t i s t i c s , 1976, 1, 69-82. I t o , K. On the e f f e c t of h e t e r o s c e d a s t i c i t y and non-normality upon some . m u l t i v a r i a t e t e s t procedures. In P. R. Krishn a i a h (Ed.), M u l t i v a r i a t e a n a l y s i s - I I . New York: Academic Press, 1969. I t o , K. & S c h u l l , W. J . On the robustness of the T 2 t e s t i n m u l t i v a r i a t e a n a l y s i s of variance when variance-covariance matrices are not equal. Biometr i k a , 1964, 5J_, 71-82. Jones, B. "What i s the best measure of accuracy of movement d u p l i c a t i o n ? " Unpublished paper, U n i v e r s i t y of Waterloo, 1974. Kogan, L. S. A n a l y s i s of va r i a n c e : Repeated measurements. P s y c h o l o g i c a l B u l l e t i n , 1948, 45, 131-143. Laabs, G. J . Retention c h a r a c t e r i s t i c s of d i f f e r e n t reproduction cues i n motor short-term memory. Journal of Experimental Psychology, 1973, 100, 168-177. Lana, R. E., & Lubin, A. The e f f e c t of c o r r e l a t i o n on the repeated measures design. Educational and P s y c h o l o g i c a l Measurement, 1963, 23j_ 729-739. Mardia, K. V. The e f f e c t of nonnormality on some m u l t i v a r i a t e t e s t s and robustness to nonnormality i n the l i n e a r model. Biometrika, 1971, 58, 105-121. Mauchly, J . W. S i g n i f i c a n c e t e s t for s p h e r i c i t y of a normal n- v a r i a t e d i s t r i b u t i o n . The Annals of Mathematical S t a t i s t i c s , 77 1940, 29, 204-209. Mendoza, J . L., Toothaker, L.E. & Nicewander, W.A. A Monte Carlo comparison of the u n i v a r i a t e and . m u l t i v a r i a t e methods for the groups by repeated measures design. M u l t i v a r i a t e Behavioral Research, 1974, 9, 165-178. Morrow, J . R. & Frankiewicz, R. G. S t r a t e g i e s for the a n a l y s i s of repeated and m u l t i p l e measures designs. Research Q u a r t e r l y , 1979, 50, 297-304. Newell, K. M. More on absolute e r r o r , e t c . Journal of Motor Behavior, 1976, 8^ 139-142. Press, S. J . Applied m u l t i v a r i a t e a n a l y s i s . New York: H o l t , Rinehart, and Winston, 1972. Rogan, J . C. , Keselman, H. J . & Mendoza, J . L. A n a l y s i s of repeated measurements. B r i t i s h Journal of Mathematical and S t a t i s t i c a l Psychology, 1979, 32, 269-286. Rouanet, H. & Lepine, D. Comparison between treatments i n a repeated-measures design: ANOVA and m u l t i v a r i a t e methods. The B r i t i s h Journal of Mathematical and S t a t i s t i c a l Psychology, 1970, 23j_147-163. Roy, E. A. "Measures of accuracy of movement d u p l i c a t i o n - A reply to Jones." Unpublished paper, U n i v e r s i t y of Waterloo, 1974. Roy, E. A. Measuring change i n motor memory. Journal of Motor Behavior, 1976, 8, 283-287. Roy, E. A. Measuring change: A r e p l y to Thomas. Journal of Motor Behavior, 1977, 9, 255-256. Schmidt, R. A. Critque of Henry's paper. In L. E. Smith (Ed.), Psychology of motor l e a r n i n g . Chicago: A t h l e t i c I n s t i t u t e , 1 970. Schmidt, R. A. A schema theory of d i s c r e t e motor l e a r n i n g . P s y c h o l o g i c a l Review, 1975, 82, 225-260. S a f r i t , M. J . , Spray, A. J . & Diewert, G. L. Methodological 78 issues i n short-term motor memory research. Journal of Motor Behavior, 1980, 12, 13-28. Scheffe, H. The A n a l y s i s of Variance. New York: Wiley, 1959. Schutz, R. W. Absolute e r r o r . J o u r n a l of Motor Behavior, 1974, 6, 299-301. Schutz, R. W. S p e c i f i c problems i n the measurement of change: L o n g i t u d i n a l s t u d i e s , d i f f e r e n c e scores, and m u l t i v a r i a t e analyses. In D. Landers & R. C h r i s t i n a (Eds.), Psychology of Motor Behavior and Sport - 1977, Champaign, I l l i n o i s : Human K i n e t i c s , 1978. Schutz, R. W. Absolute, Constant, and V a r i a b l e E r r o r : Problems and s o l u t i o n s . In D. Mood (Ed.), Proceedings of the Colorado Measurement Symposium. U n i v e r s i t y of Colorado Press: Boulder, Colorado, 1979, 82-100. Schutz, R. W. & G e s s a r o l i , M. E. The e f f e c t s of block s i z e and heterogeneity of covariance on Type I e r r o r rates with Constant e r r o r and V a r i a b l e e r r o r data, i n Psychology of Motor Behavior and Sport ^ 1979. Champaign, I l l i n o i s : Human K i n e t i c s , 1980, 633-642. Schutz, R. W., Goodman, D. & Roy, E. A. "More on e r r o r . " Paper presented at the F i r s t Canadian Congress for the M u l t i d i s c i p l i n a r y Study of Sport and P h y s i c a l A c t i v i t y , Montreal, Quebec, October, 1973. Schutz, R. W., & Roy, E. A. Absolute e r r o r : The d e v i l i n d i s g u i s e . Journal of Motor Behavior/ 1973, 5j_ 141-153. S t o l o f f , P. H. C o r r e c t i n g for heterogeneity of covariance for repeated measures designs of the a n a l y s i s of variance. Educational and P s y c h o l o g i c a l Measurement, 1970, 3_0, 909-924. Thomas, J . R. & Moon, D. H. Measuring motor rhythmic a b i l i t y . Research Q u a r t e r l y , 1976, 47, 20-32. Thomas, J . R. A note concerning a n a l y s i s of e r r o r scores from motor memory research. Journal of Motor Behavior, 1977, 9, 251-253. Wilson, A. & Lange, D. E. U n i v a r i a t e a n a l y s i s of variance for 79 repeated measures. The Journa l of Experimental Education, 1972, 40, 83-85. Wilson, K. The sampling d i s t r i b u t i o n s of c o n v e n t i o n a l , conservative and c o r r e c t e d F - r a t i o s i n repeated measurement designs with heterogeneity of covariance. J o u r n a l of S t a t i s t i c a l Computing and S i m u l a t i o n , 1975, 3_, 201-215. Woodworth, R. S. Experimental psychology. New York: H o l t , 1938.

Cite

Citation Scheme:

    

Usage Statistics

Country Views Downloads
United States 2 0
China 1 23
Australia 1 0
Japan 1 0
City Views Downloads
Tokyo 1 0
Unknown 1 0
Las Vegas 1 0
Beijing 1 0
Ashburn 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}

Share

Share to:

Comment

Related Items