UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Characteristics of variable error and their effects on the type I error rate Gessaroli, Marc Elie 1981

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
[if-you-see-this-DO-NOT-CLICK]
UBC_1981_A7_5 G48.pdf [ 4.2MB ]
[if-you-see-this-DO-NOT-CLICK]
Metadata
JSON: 1.0077122.json
JSON-LD: 1.0077122+ld.json
RDF/XML (Pretty): 1.0077122.xml
RDF/JSON: 1.0077122+rdf.json
Turtle: 1.0077122+rdf-turtle.txt
N-Triples: 1.0077122+rdf-ntriples.txt
Original Record: 1.0077122 +original-record.json
Full Text
1.0077122.txt
Citation
1.0077122.ris

Full Text

CHARACTERISTICS OF VARIABLE ERROR AND THEIR EFFECTS ON THE TYPE I ERROR RATE by MARC ELIE GESSAROLI B.P.E., The U n i v e r s i t y of B r i t i s h Columbia, 1978 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTERS OF PHYSICAL EDUCATION in THE FACULTY OF GRADUATE STUDIES (PHYSICAL EDUCATION) We accept t h i s t h e s i s as conforming to the r e q u i r e d standard THE UNIVERSITY OF BRITISH COLUMBIA JUNE 1981 © M a r c E l i e G e s s a r o l i , 1981 In p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t of the requirements f o r an advanced degree a t the U n i v e r s i t y o f B r i t i s h Columbia, I agree t h a t the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r r e f e r e n c e and study. I f u r t h e r agree t h a t p e r m i s s i o n f o r e x t e n s i v e copying of t h i s t h e s i s f o r s c h o l a r l y purposes may be granted by the head of my department or by h i s o r her r e p r e s e n t a t i v e s . I t i s understood t h a t copying or p u b l i c a t i o n of t h i s t h e s i s f o r f i n a n c i a l g a i n s h a l l not be allowed without my w r i t t e n p e r m i s s i o n . Department of PHYSICAL EDUCATION The U n i v e r s i t y o f B r i t i s h Columbia 2075 Wesbrook P l a c e Vancouver, Canada V6T 1W5 Date AUGUST 4. 1981 DE-6 (2/79) ABSTRACT A common p r a c t i c e i n motor behavior research i s to analyze V a r i a b l e Error data with a repeated measures a n a l y s i s of variance. The purpose of t h i s study was to examine the degree to which blocked (VE) data s a t i s f i e s the assumptions underlying a repeated measures ANOVA. Of p a r t i c u l a r i n t e r e s t was whether the assumption of covariance homogeneity - both w i t h i n and between experimental groups - i s s a t i s f i e d i n a c t u a l experimental data. Monte Carlo procedures were used to study the e f f e c t of varying degrees cf v i o l a t i o n s of these assumptions on the Type I er r o r r a t e . The means and ranges of the c o r r e l a t i o n matrices of eight experimental data sets were studied for both raw and VE scores based upon d i f f e r e n t block s i z e s . In every s i t u a t i o n where the experimental groups were comprised of feedback and no feedback c o n d i t i o n s , the c o r r e l a t i o n matrix for the no feedback group displayed c o r r e l a t i o n s of greater magnitudes and consistency r e l a t i v e to those of the feedback c o n d i t i o n . The next phase involved using the underlying variance-covariance matrices for three of these data sets to simulate raw and VE data based on various block s i z e s . Raw data were simulated for each of four covariance heterogeneity c o n d i t i o n s : ( 1 ) e q u a l i t y w i t h i n and between the variance-covariance matrices; ( 2 ) i n e q u a l i t y w i t h i n the matrices but e q u a l i t y between the matrices; ( 3 ) e q u a l i t y w i t h i n each variance-covariance matrix but i n e q u a l i t y between the matrices; ( 4 ) i n e q u a l i t y both w i t h i n and between the two variance-covariance matrices. Populations of 10,000 subjects for each of two groups, the underlying variance-covariance matrices being dependent upon the homogeneity of covariance c o n d i t i o n being s t u d i e d , were generated based on each of three a c t u a l experimental data se t s . The data were blocked i n various ways depending on the o r i g i n a l number of t r i a l s i n the experiment (36, 24 or 18) with VE being the dependent v a r i a b l e . An experiment c o n s i s t e d of randomly s e l e c t i n g 20 subjects for each of the two groups, b l o c k i n g the t r i a l s based on s p e c i f i c block s i z e s and a n a l y z i n g the raw and VE data by a repeated measures ANOVA. The e f f e c t of i n t e r e s t was the Groups by Blocks i n t e r a c t i o n . The complete process was r e p l i c a t e d for the four covariance homogeneity c o n d i t i o n s for each of the three data s e t s , r e s u l t i n g i n a t o t a l of 22,000 simulated experiments. Results i n d i c a t e d that the Type I e r r o r rate increases as the degree of heterogeneity w i t h i n the variance-covariance matrices increases when raw (unblocked) data i s analyzed. With VE, the e f f e c t s of w i t h i n - m a t r i x heterogeneity on the Type I e r r o r rate are i n c o n c l u s i v e . However, block s i z e does seem to a f f e c t the p r o b a b i l i t y of o b t a i n i n g a s i g n i f i c a n t i n t e r a c t i o n , but the nature of t h i s r e l a t i o n s h i p i s not c l e a r as there does not appear to be any c o n s i s t e n t r e l a t i o n s h i p between the s i z e of the block and the p r o b a b i l i t y of o b t a i n i n g s i g n i f i c a n c e . For both raw and VE data there was no i n f l a t i o n in the number of Type I e r r o r s when the covariances w i t h i n a given matrix were homogeneous, regardless of the d i f f e r e n c e s between the group variance-covariance matrices. iv ACKNOWLEDGEMENTS I wish to thank the members of my t h e s i s committee, Dr. A.. John Petkau and Dr. A. Ralph Hakstian f o r t h e i r time taken i n a i d i n g me with the v a r i o u s , and o f t e n n o n - t r i v i a l , s t a t i s t i c a l problems a s s o c i a t e d with t h i s study and Dr. Gordon L. Diewert whose enthusiam and i n t e r e s t i n the study was g r e a t l y a p p r e c i a t e d . T h e i r many comments and suggestions were i n v a l u a b l e to the progress of the t h e s i s . Many thanks are a l s o given to Dr. D. Gordon E. Robertson whose knowledge of computer a p p l i c a t i o n s s i m p l i f i e d many f a c e t s of my data a n a l y s e s . S p e c i a l thanks and a l a r g e debt i s owed to my t h e s i s chairman and a d v i s o r , Dr. Robert W. Schutz, f o r h i s c o n t i n u a l support, enthusiasm and l e a d e r s h i p i n a l l aspects of my graduate l i f e , at U.B.C. Due to Dr. Schutz, my work was not only e d u c a t i o n a l , but a l s o e n j o y a b l e . The extent of h i s c o n t r i b u t i o n cannot be measured. F i n a l l y , a s p e c i a l type of g r a t i t u d e must go to my parents, whose constant support and understanding towards a seemingly endless process was, and i s , very important to me. TABLE,'OF - CONTENTS INTRODUCTION 1. METHODOLOGY 6 Phase 1 6 Phase 2 8 Pr e l i m i n a r y Analyses 9 Homogeneity Conditions 10 S e l e c t i o n C r i t e r i a 12 Simulation Procedures 13 E f f e c t Of The Number Of T r i a l s 16 RESULTS AND DISCUSSION 17 Structure Of The C o r r e l a t i o n Matrices 17 Raw Scores 17 V a r i a b l e E r r o r 19 D i s t r i b u t i o n Of Raw And VE Scores ...20 Raw Scores 20 V a r i a b l e Error 20 V i o l a t i o n s Of Covariance Homogeneity 21 Condition 1 21 Condition 2 23 Condition 3 29 Condition 4 30 E f f e c t Of Block Size ...32 CONCLUSIONS 35 REFERENCES 37 APPENDIX A - LETTER REQUESTING EXPERIMENTAL DATA 39 APPENDIX B - PROGRAM TO.CALCULATE VE AND ANOVA ....43 APPENDIX C - REVIEW OF LITERATURE 47 Overview Of Chapter 47 The S t a t i s t i c a l Model 48 Assumptions Of Repeated Measures ANOVA ...49 Assumption Of Normality ......49. Homogeneity Of Variances .....51 Homogeneity Of Covariances 52 Heterogeneity Of Covariances: D e f i n i t i o n And Measurement 53 A Measure Of Covariance Heterogeneity 53 Compound Symmetry And C i r c u l a r i t y 54 Covariance Heterogeneity And Type I Error Rates 57 Evidence Of Type I E r r o r I n f l a t i o n 57 T r a d i t i o n a l Adjustments In The Degrees Of Freedom 59 Studies Using 60 . M o d i f i c a t i o n s Of e: i And I 61 GA And IGA Tests . . .62 M u l t i v a r i a t e Techniques 63 Overview Of U n i v a r i a t e Vs M u l t i v a r i a t e Tests On Power .65 Summary 67 AE-CE-VE Debate 68 Summary 72 REFERENCES 74 LIST OF TABLES Table I - C h a r a c t e r i s t i c s Of The Raw Experimental Data Sets Received From Motor Behaviour Researchers 14 Table II - P r o p o r t i o n Of S i g n i f i c a n t G X B I n t e r a c t i o n s For Unblocked Data 22 Table III - P r o p o r t i o n Of S i g n i f i c a n t G X B I n t e r a c t i o n s For VE Data - 24 Table IV - E f f e c t Of The Number Of T r i a l s On The Type I E r r o r Rate For Raw And VE Data Based On Data Set 7 25 Table V - The Mean And Ranges Of The C o r r e l a t i o n C o e f f i c i e n t s For Various Block S i z e s 28 1 INTRODUCTION Motor performance data i s unique i n , that a. subject i s measured over numerous t r i a l s under r e l a t i v e l y constant c o n d i t i o n s . This large number of t r i a l s i s needed due to the large i n t r a - s u b j e c t v a r i a b i l i t y c h a r a c t e r i s t i c of most motor performance tasks. The reduction and s t a t i s t i c a l a n a l y s i s of these data possesses problems not encountered i n most f i e l d s of study ( p h y s i o l o g i c a l and b i o l o g i c a l measures are u s u a l l y h i g h l y r e l i a b l e and, t h e r e f o r e , often need only one or two t r i a l s ; s o c i a l psychology t e s t c o n d i t i o n s can often not be repeated without changing the c o n d i t i o n i t s e l f ) . Therefore, the purpose of t h i s study i s to examine s e l e c t e d problems a s s o c i a t e d with the a n a l y s i s of the h i g h l y interdependent repeated measures frequently encountered i n motor behaviour research. T y p i c a l motor l e a r n i n g experiments require a subject to perform a number (p) of t r i a l s on a motor task, the nature of the i n v e s t i g a t i o n being to compare the subject's performance on that task to a predetermined target score. The d i f f e r e n c e i n these two scores i s c a l l e d the subject's performance e r r o r for that t r i a l . In most instances the subjects are d i v i d e d i n t o q groups based on v a r i a b l e s such as teaching method, experimental c o n d i t i o n , previous p r a c t i c e or some other f a c t o r , r e s u l t i n g i n a q X p f a c t o r i a l experiment with repeated measures on the second f a c t o r . A technique known as " b l o c k i n g " i s often employed in an attempt to : (a) obtain a measure of i n t r a - s u b j e c t v a r i a b i l i t y (VE) or; (b) smooth the data i f the subjects' i n t e r t r i a l v a r i a b i l i t y i s l a r g e . Here, the p o r i g i n a l t r i a l s are d i v i d e d i n t o c "blocks" with each block comprised of p/c 2 o r i g i n a l t r i a l s . Any or a l l of three performance e r r o r scores are then c a l c u l a t e d for each of the new b l o c k s ; Absolute E r r o r (AE) - the mean absolute d e v i a t i o n from the ta r g e t score over the p/c t r i a l s ; Constant E r r o r (CE) - the subject's mean al g e b r a i c e r r o r over the t r i a l s and, V a r i a b l e E r r o r (VE) - the square root of the w i t h i n - s u b j e c t variance over the t r i a l s . This b l o c k i n g procedure reduces the design to a q X c f a c t o r i a l experiment with repeated measures on the second f a c t o r . S t a t i s t i c a l analyses, u s u a l l y a n a l y s i s of varian c e , are then performed on each of these dependent v a r i a b l e s with the r e s u l t of i n t e r e s t being the Groups by Blocks i n t e r a c t i o n . The use of these three e r r o r scores and the c h a r a c t e r i s t i c s t y p i c a l to most st u d i e s r e s u l t i n a number of p o s s i b l e problems. Absolute e r r o r , because i t i s an absolute value, probably has a non-normal d i s t r i b u t i o n and, t h e r e f o r e , there may be problems when using ANOVA since the assumption of normality may be v i o l a t e d ( S a f r i t , Spray & Diewert, 1980). This i n i t s e l f may not be too serious since ANOVA i s robust to non-normality i f the number of subjects i n e,ach group i s large and the population variances are equal (Boneau, 1960). However, u n t i l the d i s t r i b u t i o n of AE scores, along with the underlying variance- covariance s t r u c t u r e , has been determined, and t h e i r e f f e c t s on the Type I e r r o r rate s t u d i e d , the v a l i d i t y and i n t e r p r e t a t i o n of research using AE i s questionable. One of the assumptions of ANOVA i s that the t r i a l s have equal variances and that a l l the covariances be equal to zero. F a i l u r e to adhere to t h i s assumption r e s u l t s i n a p r o b a b i l i t y of f a l s e l y r e j e c t i n g the n u l l hypothesis greater than the set l e v e l 3 of s i g n i f i c a n c e (Box, 1954). However, i t was l a t e r shown by Lana and Lubin (1963) that the Type I e r r o r rate i s not i n f l a t e d i f the covariances are equal though not n e c e s s a r i l y equal to zero. Constant e r r o r scores are assumed to have the c h a r a c t e r i s t i c of adjacent t r i a l s being h i g h l y c o r r e l a t e d with the c o r r e l a t i o n s , decreasing as the t r i a l s become.farther apart (Gaito, 1973; Lana & Lubin, 1963). Schutz and G e s s a r o l i (1980), i n a Monte Carlo study using data based on such a variance-covariance matrix, reported many more Type I e r r o r s than i n s i m i l a r s t u d i e s i n which fewer t r i a l s were incorporated ( C o l l i e r , Baker, Mandeville & Hayes, 1967). This brings f o r t h many questions, namely: (1) Are the number of t r i a l s under which a subject i s t e s t e d r e l a t e d to the p r o b a b i l i t y of making a Type I e r r o r ? (2) How does the range of covariances a f f e c t the Type I e r r o r rate? (3) Are the magnitudes of the covariances important when using ANOVA in t e s t i n g hypotheses with CE as the dependent v a r i a b l e ? That i s , does a range of covariances from 0.6 to 0.1 r e s u l t i n the same degree of Type I e r r o r s as covariances spanning 0.9 to 0.4? Another p o t e n t i a l problem a l s o a s s o c i a t e d with b l o c k i n g i s whether varying the block s i z e d i f f e r e n t i a l l y a f f e c t s the Type I e r r o r r a t e . I t has been shown that when using CE data the s i z e of the block i s of no consequence i n the degree of i n f l a t i o n of Type I e r r o r s (Schutz & G e s s a r o l i , 1980). V a r i a b l e e r r o r i s u n l i k e e i t h e r CE or AE because i t i s a variance and,' consequently, probably has a non-normal d i s t r i b u t i o n ( S a f r i t , Spray & Diewert, 1980). However, as with AE, t h i s may not be serious depending upon the sample s i z e s and the s t r u c t u r e of the variance-covariance matrix. In t h e i r Monte 4 Carlo study Schutz and G e s s a r o l i (19"80) found no " i n f l a t i o n i n the Type I er r o r rate when VE scores were c a l c u l a t e d from raw score matrices with unequal covariances. However., the data s i m u l a t i o n procedures c a l c u l a t e d VE scores which were uncor r e l a t e d across blocks. Schutz and G e s s a r o l i used raw score covariances among t r i a l s which decreased i n a l i n e a r fashion as the t r i a l s became f a r t h e r apart. Mathematically, such a raw score covariance s t r u c t u r e a l w a y s ^ w i l l r e s u l t i n uncorrelated VE scores. I f , i n r e a l data, the VE scores are c o r r e l a t e d and these c o r r e l a t i o n s are unequal, then problems a r i s e when using ANOVA since the assumption of homogeneous covariances has been v i o l a t e d . The variance-covariance s t r u c t u r e of e m p i r i c a l VE data must be studied before i t can be s a i d with any c e r t a i n t y i f heterogeneous covariances i n the raw data a f f e c t the Type I err o r r a t e . A second part of the assumption of homogeneous covariances deals with the s t r u c t u r e of the covariance matrices between experimental c o n d i t i o n s . Not only do the covariances w i t h i n each group have to be equal, but the magnitudes of the covariances i n one matrix need to be equal to those of the variance-covariance matrices f o r the other group. This i s r e f e r r e d to as "compound symmetry'. As of now i t i s not c l e a r i f raw experimental data have covariance matrices of t h i s type. Extending t h i s concept to AE, CE and VE data i t i s a l s o not known i f t h e i r underlying covariance matrices s a t i s f y t h i s assumption. I t i s obvious that the r e s u l t s obtained when an a l y z i n g AE, CE or VE by an ANOVA are, at best, i n c o n c l u s i v e . Over the l a s t eight years there has been extensive debate 5 as to the v a l i d i t y of using these measures i n the a n a l y s i s - and i n t e r p r e t a t i o n of motor performance (e.g., Laabs, 1.973; Newell, 1976; Schmidt, 1975; Schutz, 1979). Schutz and Roy (1973) i n i t i a t e d t h i s debate when they provided a mathematical proof that AE could be w r i t t e n as a composite score of CE and VE, but in d i f f e r e n t proportions depending upon the r e l a t i v e magnitude of the CE and VE scores. Absolute e r r o r i s ther e f o r e redundant, and furthermore, i t i t i s used, i t can be prop e r l y i n t e r p r e t e d : only when the CE and VE components are known. For t h i s reason, the problems a s s o c i a t e d with the a n a l y s i s of AE data w i l l not be de a l t with i n t h i s study. When analyz i n g CE data no "absolute answers" are a v a i l a b l e i n de a l i n g with a l l the p o t e n t i a l problems but, i n general, the most common d i f f i c u l t i e s have been adequately resolved by Schutz and G e s s a r o l i (1980). Although p o t e n t i a l problems may e x i s t with the s t a t i s t i c a l analyses of a l l three e r r o r measures, AE, CE and VE, VE appears to be the l e a s t understood. Thus, t h i s study w i l l focus p r i m a r i l y on the analyses of raw and VE data. Therefore, the purpose of t h i s study i s t o : (a) discover the s t r u c t u r e of the variance-covariance matrices a s s o c i a t e d with e m p i r i c a l raw and VE data; (b) determine the d i s t r i b u t i o n s of the raw and VE data? (c) study the e f f e c t of the number of t r i a l s on Type I e r r o r s when using VE as the dependent v a r i a b l e ; (d) study the e f f e c t of the block s i z e on the Type I e r r o r rate when VE i s used as the measure of performance e r r o r ; (e) study the e f f e c t of the degree of heterogeneity of the covariances on the p r o b a b i l i t y of making a Type I e r r o r ; ( f ) study the e f f e c t of heterogeneity of the covariance matrices between the various 6 groups i n the experimental design on the Type I e r r o r r a t e . METHODOLOGY This study c o n s i s t e d of two phases -• the f i r s t d e a l i n g with the a n a l y s i s of e m p i r i c a l data and the second being a Monte Carl o study of VE data. Phase 1 The d i s t r i b u t i o n and general p a t t e r n of the variance- covariance matrices of raw and VE data were studied through the f o l l o w i n g steps: 1. L e t t e r s were sent to approximately 2 0 motor performance researchers requesting that they supply some of t h e i r a c t u a l experimental data from which VE was even t u a l l y c a l c u l a t e d and analyzed (a copy of the covering l e t t e r i s i n Appendix A). The experimental data d e s i r e d could have been l e a r n i n g or performance data but i t had to s a t i s f y two c o n d i t i o n s : (a) each subject had to perform a minimum of twelve t r i a l s on a given task and; (b) each experimental c o n d i t i o n (group) had at l e a s t tw-elve s u b j e c t s . Upon r e c e i p t of the data sets (eight were received) they were c a t e g o r i z e d by the type of experimental task (e.g., movement reproduction, r e a c t i o n time), the l e v e l of task f a m i l i a r i t y ( l e a r n i n g or performance) and the experimental c o n d i t i o n s involved (e.g., feedback, no feedback). 2 . The next step involved studying the variance-covariance 7 s t r u c t u r e of the raw e m p i r i c a l d a t a / C o r r e l a t i o n m a t rices f o r each data set were obtained v i a the s t a t i s t i c a l computer package MIDAS (Michigan I n t e r a c t i v e Data A n a l y s i s System). Separate c o r r e l a t i o n and c o v a r i a n c e matrices were c a l c u l a t e d f o r every experimental c o n d i t i o n w i t h i n a data s e t . The s t r u c t u r e of the c o r r e l a t i o n m a t rices was s t u d i e d i n the f o l l o w i n g ways: (a) the mean c o r r e l a t i o n c o e f f i c i e n t i n each matrix was c a l c u l a t e d ; (b) the maximum and minimum c o r r e l a t i o n c o e f f i c i e n t s i n each matrix were noted and; (c) an i n s p e c t i o n was conducted to see i f there was a d i f f e r e n c e i n the magnitude of the c o r r e l a t i o n c o e f f i c i e n t s of t r i a l s c l o s e together as compared to those f a r t h e r a p a r t . T h i s was done by t a k i n g the mean of a l l c o r r e l a t i o n s one t r i a l a p a r t , three t r i a l s a p a r t , f i v e t r i a l s a p a r t , e t c . In the cases where the number of t r i a l s i n the c o r r e l a t i o n matrix numbered gre a t e r than f o r t y , the mean of the c o r r e l a t i o n s one, s i x , eleven, e t c . t r i a l s apart were c a l c u l a t e d . 3. I t was imperative to d i s c o v e r the e m p i r i c a l d i s t r i b u t i o n , of the raw scores as t h i s d i s t r i b u t i o n would d i c t a t e the type of data to be simulated i n Part 2. Histograms of the frequency d i s t r i b u t i o n s of every t r i a l were obtained using the MIDAS s t a t i s t i c a l package. The main problem was to d i s c o v e r i f the data were d i s t r i b u t e d as m u l t i v a r i a t e normal. As there i s p r e s e n t l y no easy method of t e s t i n g f o r m u l t i v a r i a t e n o r m a l i t y , an examination of the marginal d i s t r i b u t i o n s was done. Although the d i s t r i b u t i o n s of the marginals would not i n d i c a t e m u l t i v a r i a t e n o r m a l i t y , a departure from u n i v a r i a t e n o r m a l i t y would c l e a r l y make the assumption of m u l t i v a r i a t e n o r m a l i t y 8 tenuous. For the purpose of t h i s , paper, data whose marginal d i s t r i b u t i o n s e x h i b i t e d u n i v a r i a t e normality were considered to be m u l t i v a r i a t e l y normally d i s t r i b u t e d . 4. The raw data were then reduced to VE scores (the s i z e of the blocks dependent upon the number of o r i g i n a l t r i a l s ) using the F o r t r a n computer program DATASNIFF (Goodman & Schutz, 1975). The data received from the researchers c o n s i s t e d of experiments having 12, 18, 20, 24, 30, 36 and 50 t r i a l s . These t r i a l s were blocked i n the f o l l o w i n g manners ( 3 X 6 defines three blocks of s i x t r i a l s / b l o c k ) : (a) Data set 1 - 50 t r i a l s : 10. X 5, 5 X 10 (b) Data set 2 - 24 t r i a l s : 8 X 3 , 6 X 4 , 3 X 8 (c) Data set 3 - 20 t r i a l s : 5 X 4 , 4 X 5 (d) Data set 4 - 1 2 t r i a l s : 4 X 3 , 3 X 4 (e) Data set 5 - 18 t r i a l s : 6 X 3 , 3 X 6 (f) Data set 6 - 30 t r i a l s : 1 0 X 3 , 6 X 5 , 5 X 6 , 3 X 1 0 (g) Data set 7 - 36 t r i a l s : 9 X 4 , 6 X 6 , 3 X 1 2 (h) Data set 8 - 20 t r i a l s : 5 X 4 , 4 X 5 5. The s t r u c t u r e of the VE variance-covariance matrices studied for each group as o u t l i n e d i n step 2 above. 6. The e m p i r i c a l d i s t r i b u t i o n of VE scores was a l s o examined by the use of histograms as explained i n step 3 above. Phase 2 This part of the study d e a l t with the a c t u a l Monte Carlo procedures used t o i n v e s t i g a t e the c h a r a c t e r i s t i c s of VE and t h e i r e f f e c t s on the Type I e r r o r r a t e . This phase c o n s i s t e d of generating data representing 500 experiments (two groups, 20 subjects/group; v a r i a b l e number of t r i a l s ) for each of four 9 variance-covariance c o n d i t i o n s , d e r i v i n g VE scores f o r various block s i z e s , and examining Type I e r r o r rates for the Groups by Blocks i n t e r a c t i o n . P r e l i m i n a r y analyses. There were two primary concerns before s i m u l a t i n g the data. F i r s t l y , as a computer program to simulate m u l t i v a r i a t e l y normal data was r e a d i l y a v a i l a b l e , i t was necessary to discover i f the data were normally d i s t r i b u t e d . A f t e r studying the histograms of the marginal d i s t r i b u t i o n s of the raw scores and VE scores i t was concluded that both sets of scores e x h i b i t e d u n i v a r i a t e normality based on -t h e i r sample s i z e s . That i s , from the shapes of the histograms any t e s t of s i g n i f i c a n c e f or normality of the marginals c l e a r l y would have f a i l e d to r e j e c t the n u l l hypothesis. The next step involved determining the procedure for simu l a t i n g VE scores. I t was e s s e n t i a l to determine i f the c o r r e l a t i o n s among VE scores for the generated data would mirror those of the o r i g i n a l experimental VE data. That i s , i t was e s s e n t i a l to determine that the generated raw data be based on covariance matrices d e p i c t i n g a c t u a l experimental data and have c o r r e l a t i o n s between blocks of VE scores s i m i l a r to the a c t u a l c o r r e l a t i o n matrices of VE scores. To examine t h i s , the var iance-covar iance matrix and vector of means fo.r the raw data were s p e c i f i e d to be e x a c t l y equal to those of an o r i g i n a l data set (Data set 3) having 20 t r i a l s . Raw data for 20 subjects were generated. One hundred of these data s e t s , each having the same covariance matrix, were generated. Each data set had d i f f e r e n t raw scores due to a d i f f e r e n t " s t a r t i n g p o i n t " being used to i n i t i a l i z e the data generation. The r e s u l t a n t data sets were 1 0 blocked and VE scores c a l c u l a t e d i n two ways: f i v e blocks of four t r i a l s / b l o c k and four blocks of f i v e t r i a l s / b l o c k . The net r e s u l t was 100 four-by-four and f i v e - b y - f i v e matrices of VE scores. To compare the c o r r e l a t i o n c o e f f i c i e n t s of Data set 3 with the c o r r e l a t i o n c o e f f i c i e n t s of the generated data, a "mean c o r r e l a t i o n matrix" was c a l c u l a t e d . This matrix was obtained by c a l c u l a t i n g the mean (across the 100 matrices) of every c o r r e l a t i o n c o e f f i c i e n t i n the same p o s i t i o n in the c o r r e l a t i o n matrix. The "mean c o r r e l a t i o n matrix" d i s p l a y e d c o e f f i c i e n t s of the same magnitude and range as the a c t u a l c o r r e l a t i o n matrix under both b l o c k i n g c o n d i t i o n s . Tests for d i f f e r e n c e s between c o r r e l a t i o n c o e f f i c i e n t s in equivalent l o c a t i o n s i n the two matrices . f a i l e d to produce s i g n i f i c a n c e at the .05 l e v e l . Based on these r e s u l t s i t was concluded that generation of raw data ( e x h i b i t i n g m u l t i v a r i a t e normality) using an e m p i r i c a l c o r r e l a t i o n matrix produces c o r r e l a t i o n s among VE scores which adequately r e f l e c t those i n the o r i g i n a l data. Homogeneity c o n d i t i o n s . The question remained as to which variance-covariance matrices to use f o r each group as the b a s i s for the data generation. As these matrices are user s p e c i f i e d , well-chosen matrices could simulate data which s a t i s f i e d or v i o l a t e d the various assumptions involved in a n a l y z i n g repeated measures data by an ANOVA. The assumptions of ANOVA req u i r e both homogeneity of the covariances w i t h i n a variance-covariance matrix as w e l l as e q u a l i t y between the variance-covariance matrices d e p i c t i n g the d i f f e r e n t experimental c o n d i t i o n s i n the design. By s p e c i f y i n g the nature of the matrix for each of two groups i t was hoped 11 that 'the . e f f e c t of v i o l a t i n g none, one. or both of these c o n d i t i o n s could be determined when VE was .the dependent v a r i a b l e . Therefore, four s t a t i s t i c a l c o n d i t i o n s which span a l l p o s s i b i l i t i e s of adherence or v i o l a t i o n of the two varian c e - covariance assumptions were used as bases f o r the.generation of raw data. The nature of the "within-group" and "between-group" covariances i n each of the s t a t i s t i c a l c o n d i t i o n s f o l l o w : . , . 1. Condition 1 ( e q u a l i t y w i t h i n ; e q u a l i t y between). The magnitude of the covariances w i t h i n each group were equal and the magnitude of the covariances between each group were a l s o equal. In order to obtain a variance-covariance matrix s a t i s f y i n g the assumption of symmetry yet r e f l e c t i n g the magnitude of the variances and covariances of the a c t u a l matrix the f o l l o w i n g procedures were employed: (a) the mean of the variances (diagonals) i n the a c t u a l variance-covariance matrix was c a l c u l a t e d . This value was used for a l l the variances i n the new homogeneous matrix and; (b) the mean of the covariances ( o f f - d i a g o n a l values) i n the a c t u a l variance-covariance matrix was c a l c u l a t e d and was used as the value to which a l l the new covariances were equal. Homogeneous matrices of t h i s type were c a l c u l a t e d based on both Group 1 and Group 2 a c t u a l variance-covariance matrices. They were used as needed to t e s t the e f f e c t of the v i o l a t i o n of the two assumptions. Generated data for the two groups i n Condition 1 r e s u l t e d i n the f o l l o w i n g variance-covariance matr i c e s : Group 1: The homogeneous matrix derived from the a c t u a l variance-covariance matrix of Group 1. 12 Group 2 : Same matrix as Group 1. 2 . C o n d i t i o n 2 ( i n e q u a l i t y w i t h i n ; e q u a l i t y between). The magnitude of the c o v a r i a n c e s w i t h i n each group were heterogeneous and the magnitude of the c o v a r i a n c e s between each group were equal. The v a r i a n c e - c o v a r i a n c e m a t r i c e s used to generate such data were: Group 1: The o r i g i n a l v a r i a n c e - c o v a r i a n c e matrix of Group 1. Group 2 : Same matrix as Group 1. 3. C o n d i t i o n 3 ( e q u a l i t y w i t h i n ; i n e q u a l i t y between). The magnitude of the c o v a r i a n c e s w i t h i n each group were homogeneous and the magnitude of the c o v a r i a n c e s between each group were heterogeneous. The v a r i a n c e - c o v a r i a n c e matrices used to generate such data were: Group 1: The homogeneous matrix d e r i v e d from the a c t u a l v a r i a n c e - c o v a r i a n c e matrix of Group 1 ( i . e . , as used i n C o n d i t i o n 1 ) . Group 2 : The homogeneous matrix based on the a c t u a l v a r i a n c e - c o v a r i a n c e matrix of Group 2 . 4. C o n d i t i o n 4 ( i n e q u a l i t y w i t h i n ; i n e q u a l i t y between). The magnitude of the c o v a r i a n c e s w i t h i n each group were heterogeneous and the magnitude of the c o v a r i a n c e s between each group were heterogeneous. Generated data f o r the two groups r e s u l t e d i n the f o l l o w i n g v a r i a n c e - c o v a r i a n c e m a t r i c e s : Group 1: The o r i g i n a l v a r i a n c e - c o v a r i a n c e matrix of Group 1. Group 2 : The o r i g i n a l v a r i a n c e - c o v a r i a n c e matrix of Group 2 . S e l e c t i o n C r i t e r i a . The primary concern of t h i s study was to i n v e s t i g a t e the Type I e r r o r r a t e (using VE as the dependent v a r i a b l e ) when the raw data had v a r y i n g degrees of h e t e r o g e n e i t y 1 3 w i t h i n and between the groups i n the design. I t was a l s o d e s i r a b l e to study the e f f e c t s of the number of t r i a l s before b l o c k i n g occurred on the subsequent number of Type I e r r o r s when VE was c a l c u l a t e d . Therefore, three sets of a c t u a l experimental data (Data sets 2, 5 and 7) were used as the bases of the s i m u l a t i o n procedures. S p e c i f i c a l l y , they were chosen based on the f o l l o w i n g design and data c h a r a c t e r i s t i c s : (a) the range of the c o r r e l a t i o n c o e f f i c i e n t s ; (b) the mean of the c o r r e l a t i o n c o e f f i c i e n t s ; (c) d i f f e r e n c e s i n the c o r r e l a t i o n matrices between the two groups; (d) the number of t r i a l s i n the experiment. A l l data were from l e a r n i n g experiments. The s p e c i f i c a t t r i b u t e s of these three data sets are shown i n Table I. S imulation procedures. Five hundred two-way experiments were simulated for each of the three data s e t s . The number of Type I e r r o r s for the Group by Block i n t e r a c t i o n , when using raw scores and VE scores as the dependent v a r i a b l e s , were analyzed v i a 500 ANOVAs for each of a number of d i f f e r e n t b l o c k i n g c o n d i t i o n s i n each case. S p e c i f i c procedures for each of these processes f o l l o w . Raw scores for a population of 10000 observations having a variance-covariance matrix and vector of t r i a l means e x a c t l y as s p e c i f i e d by the user were generated for each group. The data were produced using the computer program UBC NORMAL (Halm, 1970). The net r e s u l t was 10000 observations in each of two groups having raw scores for a s p e c i f i c number of t r i a l s . Samples of s i z e 20 per group were subsequently drawn from t h i s p o pulation. Thus the sampling was not based on an i n f i n i t e Table I C h a r a c t e r i s t i c s of the Raw Experimental Data Sets Received from Motor Behaviour Researchers Raw VE Data Set No. of t r i a l s No. of S/Group mean r range of r's mean r range of r's 1 8 KR 50 30 .10 - .65 to .90 .78 .44 to .94 > KR 24 29 .00 .62 to .67 .20 -.21 to .41 No KR 24 29 .83 - .46 to .95 .20 -.26 to .58 3 KR 20 13 .05 .57 to .78 . 14 -.11 to .29 No KR 20 13 .50 - .32 to .85 .20 -.54 to .77 No KR 20 13 .20 - .54 to .77 .25 .05 to .53 [ No KR 12 24 .20 -. .66 to .70 .20 -.50 to .50 KR 18 40 .00 .57 to .60 .20 -.12 to .31 No KR 18 40 .45 .25 to .86 .25 .02 to .44 KR 30 40 .10 -. .50 to .60 .10 -.50 to .60 f No KR 36 48 .30 ,22 to .50 .20 -.19 to .54 No KR 36 48 .15 .36 to .48 .20 -.21 to .52 i KR 20 10 .35 ,65 to .88 .00 -.53 to .80 KR 20 10 .55 -, 06 to .92 .70 .23 to .92 No KR 20 10 .30 -. 67 to .91 .45 -.21 to .90 15 popu l a t i o n . However, the sample-to-population r a t i o (20 : 1 0000) i s s u f f i c i e n t l y small to negate the need to incorporate any f i n i t e population c o r r e c t i o n f a c t o r s i n t o the analyses. The data were blocked and the VE scores were c a l c u l a t e d by a Fortran computer program (see Appendix B); the s i z e of the block being, dependent upon the number of o r i g i n a l t r i a l s and as defined i n step 4 of Phase 1 of the previous s e c t i o n . The v a l i d i t y of the c a l c u l a t i o n s was t e s t e d by comparing the c a l c u l a t e d VE scores with those produced by a program known to c a l c u l a t e accurate VE scores, DATASNIFF (Goodman & Schutz, 1975). The scores were accurate to the f o u r t h decimal p l a c e . Each experiment was analyzed by an a n a l y s i s of variance on the data of twenty subjects i n each of two groups. A computer program (Appendix B) read 40 subjects at a time (20 from Group 1 and 20 from Group 2) and c a l c u l a t e d the Sum of Squares and Mean Square E r r o r terms for a l l the e f f e c t s , and the subsequent F value for the Groups by Blocks i n t e r a c t i o n . This c a l c u l a t e d F value was compared to the c r i t i c a l F value at the .10, .05 and .01 l e v e l s of s i g n i f i c a n c e . The c r i t i c a l F values were obtained v i a the f u n c t i o n subroutine UBC FVALUE which gives the F value of a u s e r - s p e c i f i e d l e v e l of s i g n i f i c a n c e based on user- s p e c i f i e d degress of freedom. The r e s u l t s were stored by the computer where, a f t e r a l l 500 experiments had been analyzed, a ta b l e showing the number of s i g n i f i c a n t i n t e r a c t i o n s at the .10, .05 and .01 l e v e l s of s i g n i f i c a n c e was p r i n t e d . The t a b l e a l s o d i s p l a y e d the mean F value c a l c u l a t e d in the 500 ANOVAs and the average Mean Square Error for each of the Groups, Subjects w i t h i n Groups, T r i a l s , Groups by T r i a l s and Subjects w i t h i n 16 Groups by T r i a l s e f f e c t s . The a c t u a l Type I e r r o r rate was compared to the nominal l e v e l of s i g n i f i c a n c e by the standard e r r o r of a proportion as given by [p(l-p)/500] . A d i f f e r e n c e of more than two standard e r r o r s of a pr o p o r t i o n between the a c t u a l number of Type I e r r o r s committed and the nominal l e v e l of s i g n i f i c a n c e was considered s i g n i f i c a n t . The net r e s u l t was. raw and VE scores being generated for 10000 observations i n each of two groups for each of four sets of underlying v a r i a n c e - covariance matrices. This was done separately for each of the three data sets for each b l o c k i n g c o n d i t i o n r e s u l t i n g i n a t o t a l of twenty-two thousand experiments being analyzed. E f f e c t of the number of t r i a l s . In order to study the e f f e c t of the number of i n i t i a l t r i a l s on the Type I e r r o r rate when VE scores are e v e n t u a l l y c a l c u l a t e d , a d d i t i o n a l s i m u l a t i o n s were performed on Data set 7. Conditions 1 and 3 were studied using 12 and 24 t r i a l s as w e l l as the a c t u a l 36 t r i a l s . I t was p o s s i b l e to use only these two c o n d i t i o n s since they both e x h i b i t e d homogeneity of the covariances w i t h i n a variance- covariance matrix. When using a s p e c i f i c heterogeneous variance- covariance matrix as the base i t i s d i f f i c u l t to obtain an equivalent heterogeneous matrix having fewer t r i a l s . The magnitude of the covariances were equal to those i n the "mean covariance matrix" based on 36 t r i a l s . The f i r s t 12 and 24 t r i a l means of the o r i g i n a l data were used as the t r i a l means for both groups i n the 12 and 24- t r i a l c o n d i t i o n s , r e s p e c t i v e l y . The twelve t r i a l s had VE c a l c u l a t e d based upon three blocks of four t r i a l s / b l o c k and four blocks of three t r i a l s / b l o c k while the 24 t r i a l s were c o l l a p s e d i n t o data sets of three, s i x and 17 eight b l ocks. Five hundred ANOVAs were performed on each of the blocking; c o n d i t i o n s as w e l l as the o r i g i n a l (unblocked) number of t r i a l s . RESULTS AND DISCUSSION Str u c t u r e of the C o r r e l a t i o n M a trices Raw scores. I t was of i n t e r e s t to^study the patterns of the c o r r e l a t i o n matrices of the raw data and the subsequently blocked VE data for each experimental data set rece i v e d . Of p a r t i c u l a r i n t e r e s t was whether the raw data e x h i b i t e d decreasing magnitudes of the c o r r e l a t i o n c o e f f i c i e n t s as t r i a l s become f a r t h e r apart as hypothesized by Gaito (1973), Lana and Lubin (1963) and others. This pattern was common to only one of the eight data sets (Data Set 1) studied. This observation i s f u r t h e r weakened by noting that t h i s decreasing pattern occurred only for c o r r e l a t i o n s among the f i r s t s i x of the f i f t y t r i a l s i n t o t a l . The remaining c o r r e l a t i o n s seemed to be randomly v a r i a b l e i n t h e i r magnitudes. A more common occurrence (though c l e a r l y not the r u l e ) was the magnitude of l a t e r t r i a l s being g e n e r a l l y greater than that of e a r l i e r t r i a l s (Data Sets 2 and 5). These c o r r e l a t i o n s , however, d i d not e x h i b i t any p a r t i c u l a r p a t t e r n . More s t r i k i n g i s the d i f f e r e n c e i n magnitudes of the c o r r e l a t i o n s between groups of subjects who received feedback and those who d i d not. In almost every case where an experiment c o n s i s t e d of two groups, KR and no-KR, the c o r r e l a t i o n among the raw scores i n the no-KR groups was much greater than for the 18 subjects which obtained feedback (see Table I ) . The sole exception, Data set. 8, was based on only 10 subjects/group ( l e s s than the minimum c r i t e r i o n of 12 subjects/group) and, t h e r e f o r e , any conclusions based on t h i s Data set are tenuous. This i s most c l e a r l y e x e m p l i f i e d by Data Set 2 . Here the mean c o r r e l a t i o n i n the feedback group was approximately equal to zero while the no- KR group had an average c o r r e l a t i o n of about .83 (see Table I ) . Although d i f f e r e n c e s between the two groups were not as extreme in the other data s e t s , d i f f e r e n c e s s t i l l e x i s t e d and were co n s i s t e n t regardless of the type of task ( l i n e a r s l i d e , etc.) performed. These d i f f e r e n c e s can be l o g i c a l l y explained. Subjects given feedback a l t e r t h e i r motor program a f t e r each t r i a l , r e s u l t i n g in r e l a t i v e l y v a r i a b l e performances from t r i a l to t r i a l . However, these t r i a l - t o - t r i a l f l u c t u a t i o n s are not constant across s u b j e c t s , thus r e s u l t i n g i n very low c o r r e l a t i o n s between p a i r s of t r i a l s . The no KR s u b j e c t s , conversely, receive no information on which to change t h e i r responses. This r e s u l t s i n a more c o n s i s t e n t performance over the repeated measures. Although the average c o r r e l a t i o n i n the no KR groups i s greater than for KR groups, the upper l i m i t s of the c o r r e l a t i o n s are approximately equal (see Table I ) . In most cases, however, the lower bound of the c o r r e l a t i o n s i n the no feedback c o n d i t i o n s i s s l i g h t l y greater than for feedback (Data Sets 2,3,5). I t appears that one p o s s i b l e explanation for the g e n e r a l l y higher c o r r e l a t i o n s i n the no KR groups i s the greater c o r r e l a t i o n s between i n i t i a l t r i a l s . Again, t h i s i s expected since response s t r a t e g i e s vary l i t t l e i n t h i s group. Subjects, 19 during i n i t i a l t r i a l s , probably perform with a l a r g e r degree of e r r o r . The r e c e i p t of feedback may d r a s t i c a l l y a l t e r the response s t r a t e g i e s and, t h e r e f o r e , l a r g e negative c o r r e l a t i o n s between these t r i a l s r e s u l t . V a r i a b l e e r r o r . When the raw data are blocked and VEs c a l c u l a t e d the nature of the c o r r e l a t i o n matrices change. Table I d i s p l a y s the d i f f e r e n c e s i n the ranges and magnitudes of the c o r r e l a t i o n s when using VE instead of the raw scores. I t i s obvious that there i s no set pattern as to what happens to the c o r r e l a t i o n s when the raw scores are. blocked i n d i f f e r e n t ways. Data set 1 shows a larg e increase i n the magnitude of the c o r r e l a t i o n s a f t e r VE i s c a l c u l a t e d while both Data set 5 (group 2) and Data set 3 (group 2) react o p p o s i t e l y . The c a l c u l a t i o n of VE seems to increase the lower bound of the range of c o r r e l a t i o n s when compared to the raw data. With the exception of Data set 3 (group 2), every data set analyzed d i s p l a y e d t h i s f a c t . However, the opposite cannot be s a i d f o r the upper l i m i t of the c o r r e l a t i o n s . Some data sets (2,3,4,5) i n d i c a t e a decrease i n magnitude of the upper l i m i t of the c o r r e l a t i o n s while others (Data sets 6 and 8) remain unchanged. In general, though, the e f f e c t of c a l c u l a t i n g VE i s to decrease the degree of heterogeneity in the c o r r e l a t i o n matrix. Study of the c o r r e l a t i o n matrices based on VE data revealed no s p e c i f i c pattern i n the c o r r e l a t i o n c o e f f i c i e n t s . There appeared to be no d i f f e r e n c e in the strength of the c o r r e l a t i o n between adjacent t r i a l s compared to those f a r t h e r apart. Data sets 3 and 4 had lower adjacent t r i a l c o r r e l a t i o n s than those a greater distance apart, while Data set 7, d i s p l a y e d the opposite 20 ef f e c t . D i f f e r e n t block s i z e s r e s u l t e d i n v a r y i n g degrees of h e t e r o g e n e i t y i n the c o r r e l a t i o n m a t r i c e s . Table I d i s p l a y s the ranges of the c o r r e l a t i o n m a t r i c e s f o r those b l o c k i n g c o n d i t i o n s which have the l a r g e s t degree of h e t e r o g e n e i t y . I n v a r i a b l y , those c o r r e l a t i o n matrices corresponded to the experimental design having the l a r g e s t number of blocks ( i . e . , the s m a l l e s t block s i z e ) . A general c h a r a c t e r i s t i c of VE data i s that the range of the c o r r e l a t i o n s i n c r e a s e d i n v e r s e l y to the block s i z e . The e f f e c t of these heterogeneous matrices on the p r o b a b i l i t y of committing a Type I e r r o r i s d i s c u s s e d i n C o n d i t i o n 2 below. D i s t r i b u t i o n of Raw and VE scores Raw s c o r e s . Histograms p l o t t i n g the raw scores f o r each t r i a l i n each data set suggested that raw data c o u l d be assumed to be normally d i s t r i b u t e d . That i s , based on the r e l a t i v e l y small sample s i z e s in each experimental group i t was obvious that any t e s t f o r n o r m a l i t y (e.g.; Kolmogorov-Smirnov, C h i - square Goodness of F i t ) would have f a i l e d to r e j e c t the i n i t i a l assumption of n o r m a l i t y . I t i s acknowledged that the small sample s i z e s of these data s e t s would.result i n r e l a t i v e l y low power on any such d i s t r i b u t i o n a l t e s t . However, o b s e r v a t i o n s of the histograms f a i l e d to r e v e a l any obvious departures from n o r m a l i t y . V a r i a b l e e r r o r . V a r i a b l e E r r o r scores are v a r i a n c e s and, t h e r e f o r e , one would expect that they are d i s t r i b u t e d as C h i - square with the a p p r o p r i a t e degrees of freedom. S a f r i t et a l . , showed that the d i s t r i b u t i o n of VE scores i s dependent on 21 d i f f e r e n t e f f e c t s under various experimental designs. These authors s t a t e d that a non-normal d i s t r i b u t i o n , may r e s u l t , but they f e l l short of saying that the d i s t r i b u t i o n was Chi-square. I t i s w e l l known, however, that one method of making a C h i - square d i s t r i b u t i o n more normal i s by t a k i n g the square root of the raw scores. The VE score used in t h i s study was the square root of the i n t r a - s u b j e c t v a r i a b i l i t y w i t h i n a block. The histograms showed that VE was a l s o d i s t r i b u t e d as u n i v a j r i a t e l y normal. This i s understandable c o n s i d e r i n g the s i z e of the sample and the f a c t that the VE scores have been transformed by the square root function.. S a f r i t e_t a_l. , may be c o r r e c t i n s t a t i n g that the t h e o r e t i c a l d i s t r i b u t i o n of VE i s non-normal. However, l a r g e r samples and untransformed VE scores would be necessary to r e f l e c t t h i s . V i o l a t i o n s of Covariance Homogeneity Of major importance, s t a t i s t i c a l l y , i s whether the a n a l y s i s of data v i a ANOVA i s v a l i d when VE i s the dependent v a r i a b l e . This question was studied under various degrees of v i o l a t i o n of the homogeneity of covariances assumptions. Condition 1 ( e q u a l i t y w i t h i n ; e q u a l i t y between). In t h i s c o n d i t i o n the covariance assumptions are adhered to and, t h e r e f o r e , Type I e r r o r rates equal to the nominally set alphas are expected when a n a l y z i n g the raw data. Table II shows that the a c t u a l number of Type I e r r o r s d i d not s i g n i f i c a n t l y d i f f e r from the nominal rate for any of the three alpha l e v e l s examined. This was c o n s i s t e n t for a l l the data s e t s . Actual alphas which d i f f e r e d by more than two standard e r r o r s of a p r o p o r t i o n from the nominal l e v e l of s i g n i f i c a n c e were Table II P r o p o r t i o n o f S i g n i f i c a n t G X B I n t e r a c t i o n s f o r Unblocked (Raw) data Homogeneity Data s e t 2 Data s e t 5 Data s e t 7 C o n d i t i o n Nominal a 24 t r i a l s 18 t r i a l s 36 t r i a l s 1 =within .10 .118 .100 . 112 .05 .066 .058 .062 =between .01 .014 .012 .010 .10 .200* .138* . 190* 2 ^ w i t h i n .05 .164* .076* . 144* =between .01 .100* .026* .068* .10 .118 .102 . 116 3 =within .05 .056 .058 .066 ^between .01 .014 " .010 .018 .10 .178* .110 .176* 4 ^ w i t h i n .05 .136* .068 .118* ^between .01 ' .076* .018 .050* a c t u a l number o f s i g n i f i c a n t i n t e r a c t i o n s which d i f f e r by more than two s t a n d a r d e r r o r s o f a p r o p o r t i o n from the nominal l e v e l o f s i g n i f i cance 23 c l a s s i f i e d as biased. The corresponding confidence i n t e r v a l s are: f o r a= . 1 0 , (.073<.10<.127 ); for a=.05, (.031<.05<.069); and l a s t l y , for C=.01, (.001<.01<.019) . The analyses of VE scores d i s p l a y s i m i l a r r e s u l t s (see Table I I I ) . Although, i n s e v e r a l data s e t s , using VE as the dependent v a r i a b l e seem to decrease the a c t u a l number of Type I e r r o r s , the d i f f e r e n c e s are not s i g n i f i c a n t . The sole exception i s i n Data set 7 (12 t r i a l s ) where the four blocks of three t r i a l s / b l o c k d i s p l a y s h i g h l y i n f l a t e d Type I e r r o r s (see Table IV). No l o g i c a l explanation for t h i s i s apparent. The e f f e c t of the number of o r i g i n a l t r i a l s on the p r o b a b i l i t y of committing a Type I er r o r i s explained i n more d e t a i l i n the d i s c u s s i o n of Condition 3. Therefore, as a general r u l e , i t appears that the a n a l y s i s of VE data c a l c u l a t e d from raw data s a t i s f y i n g the covariance assumptions does not cause a greater number of f a l s e r e j e c t i o n of the n u l l hypothesis than i s expected. Condition 2 ( i n e q u a l i t y w i t h i n ; e q u a l i t y between). Table II shows that the v i o l a t i o n of the assumption of symmetry has the e f f e c t of i n c r e a s i n g the p r o b a b i l i t y of committing a Type I er r o r when raw data i s used. Data set 2 e x h i b i t e d the greatest degree of i n f l a t i o n with the .01 l e v e l of s i g n i f i c a n c e having the l a r g e s t percentage d i f f e r e n c e from the nominal alpha. At the .01 l e v e l of s i g n i f i c a n c e the a c t u a l proportion of Type I e r r o r s was as high as .10. The increases i n the Type I e r r o r rates were 100% and 325% at nominal alphas of .10 and .05, r e s p e c t i v e l y . I t was expected that the raw data based on the most heterogeneous variance-covariance matrix wouldhave the maximum l e v e l of T a b l e I I I P r o p o r t i o n o f S i g n i f i c a n t G X B I n t e r a c t i o n s f o r VE data Homogenei t y Data s e t 2 Data s e t 5 . Data s e t 7 C o n d i t i o n Nominal a 3X8 6X4 8X3 3X6 6X3 3X12 6X6 9X4 i .10 .096 .078 .098 .082 .096 . 110 . 100 .086 i •-within .05 .046 .038 .044 .048 .038 .058 .036 .036 ^between .01 .002 .006 .006 .006 .016 .014 .006 .000 o .10 .088 .080 .122 .084 .094 . 128* .140* .150* c. ^ w i t h i n .05 .040 .034 .054 ..044 .048 .066 .084* .100* =between .01 .006 .010 .012 .002 .012 .026* .042 .034 o . 10 .098 .086 .096 .094 .082 . 128* .092 .076 5 ^ w i t h i n .05 .040 .036 .040 .042 .048 .062 .036 .034 ^between .01 .008 .008 ' .008 .012 .006 .014 .008 .002 * * * * * A .10 .762 .922 .520 .252 .496 1 .000 1.000 1 .000 4 ^ w i t h i n .05 .628* .850* * .400 .154* .356* 1 .000* 1.000* .998* ^between .01 .•380* .658* .218* .048* .172* .998* 1.000* .992* * a c t u a l number o f s i g n i f i c a n t i n t e r a c t i o n s whi ch d i f f e r by more than tWO ! standard e r r o r s o f a p r o p o r t i o n from the nominal l e v e l o f s i g n i f i cance Table IV Ef fect of the Number of T r ia l s on the Type I Error Rate for Raw and VE data Based on Data set 7 Homogeneity Condition Nominal a 36 t r i a l s 24 t r i a l s 12 t r i a l s Unblocked (Raw) Blocked (VE) Unblocked (Raw) Blocked (VE) Unblocked (Raw) Blocked (VE) 3X12 6X6 9X4 3X8 6X4 8X3 3X4 4X3 i .10 .112 .110 .100 .086 .120 .096 .098 .094 .098 .092 . , 138* l =within .05 .062 .058 .036 .036 .068 .054 .038 .044 .050 .046 . 076* =between .01 .010 .014 .006 .000* .014 .006 .008 .004 .010 .004 . 024* o .10 .116 .128* .092 .076 .120 .108 .094 .092 .096 .082 . 140* J =within .05 .066 .062 .036 .034 .060 .042 .046 .040 .048 .042 . ,070* ^between .01 .018 .014 .008 .002 .014 .010 .008 .004 .010 .004 . 022* *actual number of s i gn i f i can t in teract ions which d i f f e r by more than two standard errors of a proportion from the nominal level of s ign i f icance ro 26 i n f l a t i o n , which, i n f a c t , d i d occur. The raw data which had the highest Type I e r r o r rate (Data set 2) was based upon a mean c o r r e l a t i o n of about 0.0 and l i m i t s of -.62 to .67. This matrix was more heterogeneous than those of both Data sets 5 and 7. However, when comparing Data sets 5 and 7 t h i s l i n e of reasoning was not v a l i d . Data set 5 which r e s u l t e d i n the smallest increase i n Type I e r r o r s had a mean c o r r e l a t i o n of approximately zero and a range from -.57 to .60. A greater number of s i g n i f i c a n t F values were obtained from Data set 7, where the mean r equalled .30 and whose c o r r e l a t i o n s l a y between -.22 and .50. Conventional t h i n k i n g would assume that the variance-covariance matrix underlying Data set 7 was l e s s heterogeneous than that for Data set 5 and, t h e r e f o r e , greater i n f l a t i o n would occur using Data set 5. In f a c t , the opposite was t r u e . Several researchers (Box, 1954b; G a i t o , 1973) have i n d i c a t e d that the degree of i n f l a t i o n increases as the number of repeated measures becomes l a r g e r . In comparing the number of Type I e r r o r s committed when using raw scores as the dependent v a r i a b l e for the d i f f e r e n t Data s e t s , i t i s c l e a r that t h i s d i d not always occur (Table I I ) . However, .the degree of heterogeneity of the variance-covariance matrices were not equal e i t h e r . I t does seem very p o s s i b l e that the number of Type I e r r o r s i s r e l a t e d to the i n t e r a c t i o n of the number of t r i a l s and the heterogeneity of the underlying -matrix. For example, Data set 2, which c o n s i s t e d of 24 t r i a l s , y i e l d e d a greater number of Type I e r r o r s than d i d Data set 7 which had 36 t r i a l s . However, Table I i n d i c a t e s that the degree of heterogeneity of the 27 covariances was greater i n Data set 2 than i n Data set 7. Therefore, i t appears that the increase in covariance heterogeneity i n Data set 2 more than compensates for the fewer number of repeated measures i n the design and thus, more Type I e r r o r s were found with Data set 2. The number of Type I e r r o r s found for the 36 t r i a l s of the raw scores i n Data set 7 i s s l i g h t l y higher than i n the study by Schutz and G e s s a r o l i (1980) which employed an equal number of t r i a l s . Using a c o r r e l a t i o n matrix ranging from .54 to .95 »they found an e m p i r i c a l Type I e r r o r rate of about .16, .12 and .05 for the .10, .05 and .01 l e v e l s of s i g n i f i c a n c e , r e s p e c t i v e l y , as compared to these r e s u l t s of .190, .144 and .068 for the same nominal alphas. The increase i s probably due to the greater heterogeneity i n the c o r r e l a t i o n matrix used as the b a s i s for the generation of raw data in t h i s study. A n a l y s i s of the VE scores showed no s i g n i f i c a n t i n f l a t i o n i n Type I e r r o r s for Data Sets 2 and 5, but d i d d i s p l a y an i n f l a t e d number of Type I e r r o r s for Data Set 7. While i t i s obvious that the e m p i r i c a l Type I e r r o r rate for a l l bloc k i n g c o n d i t i o n s i s w e l l w i t h i n two standard e r r o r s of a proportion for Data sets 2 and 5, c e r t a i n b l o c k i n g c o n d i t i o n s i n Data set 7 d i s p l a y a c t u a l o's outside t h i s range. A l l b l o c k i n g c o n d i t i o n s d i s p l a y e d an increase i n the number of s i g n i f i c a n t Groups by Blocks i n t e r a c t i o n s with the degree of i n f l a t i o n being greatest for the nine blocks case. The sole exception was the .05 l e v e l of s i g n i f i c a n c e for the 3 X 1 2 case- where the a c t u a l Type I e r r o r rate d i d not d i f f e r from the nominal rate by more than two standard e r r o r s of a p r o p o r t i o n . Table V shows the patte r n of T a b l e V The Mean and Ranges o f the C o r r e l a t i o n C o e f f i c i e n t s f o r V a r i o u s B l o c k S i z e s Data S e t B l o c k i n g P a t t e r n mean r range o f r ' s 3 X 8 Group 1 Group 2 6 X 4 Group 1 Group 2 8 X 3 Group 1 Group 2 .35 .28 .15 20 18 18 ,27 t o .45 .14 t o .40 14 t o .41 16 t o .44 .21 t o .41 26 t o .58 3 X 6 Group 1 Group 2 6 X 3 Group 1 Group 2 .27 30 .12 ,20 .19 t o .36 24 to .40 .12 t o .31 .02 t o .44 3 X 12 Group 1 Group 2 6 X 6 Group 1 Group 2 9 X 4 Group 1 Group 2 26 26 20 17 ,20 20 17 t o .40 15 to .41 11 t o .44 06 t o .51 10 t o .54 21 t o .52 29 the c o r r e l a t i o n matrices of the VE scores for the three block s i z e s of Data set 7. VE simulated with 12 t r i a l s / b l o c k had the smallest range of c o r r e l a t i o n s while the VE based on four t r i a l s / b l o c k d i s p l a y e d the greatest heterogeneity i n the c o r r e l a t i o n matrix. These r e s u l t s agree with previous research (Rogan, Reselman &Mendoza, 1979) i n that the Type I eror rate increases as the degree of heterogeneity w i t h i n a matrix increases. However, Table V shows s i m i l a r degrees of heterogeneity f o r Data set 2, yet no i n f l a t i o n i n the number of Type I e r r o r s occurs. A l s o , as the degree of heterogeneity increases as the s i z e of the block decreases, a corresponding i n f l a t i o n i n the Type I e r r o r rate does not occur. No v i a b l e r a t i o n a l e i s apparent to e x p l a i n these c o n f l i c t i n g r e s u l t s obtained for the d i f f e r e n t data s e t s . Condition 3 ( e q u a l i t y w i t h i n ; i n e q u a l i t y between). S i m i l a r to Condition 1, the covariances w i t h i n each matrix are equal, however they d i f f e r i n t h e i r magnitudes between the two groups. I t seems that the assumption of e q u a l i t y between the covariance matrices of the d i f f e r e n t experimental c o n d i t i o n s i s q u i t e robust i f the second assumption of homogeneity w i t h i n the covariance matrices i s s a t i s f i e d . With one exception, the e m p i r i c a l Type I e r r o r rate d i d not exceed the nominal value for any of the data s e t s . This held regardless whether raw scores or VE was the dependent v a r i a b l e . The d i f f e r e n c e s i n the magnitudes of the c o r r e l a t i o n s between the two groups a l s o had no e f f e c t on the Type I e r r o r r a t e . Small (.15 vs .30, Data Set 7), moderate(0 vs .45, Data 30 Set 5) and large (0 vs .83, Data Set 2) d i f f e r e n c e s i'n the mean c o r r e l a t i o n s between groups were used with the same net r e s u l t i n each case - no bias in the e m p i r i c a l Type I e r r o r rate.. The f a c t that the covariances w i t h i n the matrices were homogeneous allowed for an attempt to i s o l a t e the e f f e c t of the number of repeated measures, and subsequent block s i z e on the p r o b a b i l i t y of f a l s e l y r e j e c t i n g the n u l l hypothesis. This was done by s i m u l a t i n g raw data for designs having e i t h e r 36, 24 or 12 repeated measures where the underlying variance-covariance matrices were equal i n each case. The variance-covariance. matrices s a t i s f i e d the "within-group" homogeneity assumption but f a i l e d to adhere to the "between-group" assumption. D i f f e r e n c e s i n the number of t r i a l s had no s i g n i f i c a n t e f f e c t under t h i s c o n d i t i o n (Table V). Again, i t seems as i f the number of repeated measures i s only important when the assumption of compound symmetry i s v i o l a t e d . The only case where the number of Type I e r r o r s committed was greater than expected was when the 12 t r i a l s c o n d i t i o n of Data set 7 produced VE scores based on three t r i a l s per block. Here, the percentage of Type I e r r o r s found was .140 f o r o=.10, .070 for a=.05 and .022 for C=.01. C a l c u l a t i n g VE using four t r i a l s / b l o c k found the number of corresponding e r r o r s to be .082, .042 and .004 - a l l w i t h i n two standard e r r o r s of a proportion of the nominal l e v e l of s i g n i f i c a n c e . No l o g i c a l explanation f o r t h i s i s apparent. Condition 4 ( i n e q u a l i t y w i t h i n ; i n e q u a l i t y between). The a c t u a l experimental variance-covariance matrices for each group were used to simulate the data for t h i s c o n d i t i o n . When the raw 31 data was analyzed the r e s u l t s ranged from no i n f l a t i o n i n the number of Type I e r r o r s (Data Set 5) to serious departures for the pre-set alpha (Data Set 2). Numerous researchers, s t a r t i n g with Box (1954b), have shown that the p r o b a b i l i t y of making a Type I e r r o r increases when the two covariance assumptions are not met. As expected, .Data set 2, having the greatest degree of heterogeneity w i t h i n the matrices f o r the two groups as w e l l as the l a r g e s t descrepancy between the matrices, has the greatest Type I e r r o r r a t e . However, with Data set 5, which has moderate heterogeneity both w i t h i n and between the c o r r e l a t i o n matrices, the e m p i r i c a l l e v e l of s i g n i f i c a n c e f a i l e d to increases a p p r e c i a b l y . Data set 7, having the l e a s t degree of heterogeneity both w i t h i n and between the matrices, produced the second highest e m p i r i c a l Type I e r r o r rate (see Table I I ) . While the l a s t two f i n d i n g s c o n t r a d i c t previous research, i t must be remembered that Data set 7 had twice the number of t r i a l s (36) as d i d Data set 5 (18). Therefore, i t again appears that when the raw data i s analyzed, the degree of heterogeneity combined with the number of repeated measurements i s r e l a t e d to the p r o b a b i l i t y of committing a Type I e r r o r . The.results of the a n a l y s i s of the VE data i n i t i a l l y appear to be overwhelming because of the number of s i g n i f i c a n t i n t e r a c t i o n s obtained (Table I I I ) . However, t h i s does not n e c e s s a r i l y imply that a number of Type I e r r o r s were committed, but may r e f l e c t the fact that the VE scores between the two groups are, i n f a c t , d i f f e r e n t . This i s q u i t e p o s s i b l e since subjects r e c e i v i n g feedback supposedly have d i f f e r e n t underlying processes on which to base t h e i r responses than do subjects who 32 receive no information regarding t h e i r previous response. For Data set 7, which produced almost 100% s i g n i f i c a n t interactions., the a c t u a l experimental data was blocked i n the same way as i n the s i m u l a t i o n procedures. Analyses of variance conducted on these o r i g i n a l VE scores show that the two groups d i d i n f a c t change d i f f e r e n t l y over the blocks of t r i a l s . The c a l c u l a t e d F for the Groups by Blocks i n t e r a c t i o n s for the three blocks was 16.67, 10.51 for the s i x blocks and 6.60 for nine blocks. C l e a r l y , these are a l l s i g n i f i c a n t values. The Monte Carlo procedures produced corresponding mean F values of 20.90, 13.01 and 7.50. Although the simulated data r e s u l t e d i n higher F values i t i s q u i t e conceivable that the a c t u a l experimental data are samples from the population on which the simulated data are based. E f f e c t of Block Size The r a t i o n a l e for the choice of the s i z e of the block i n c a l c u l a t i n g VE scores i s commonly based on p r a c t i c a l c o n s i d e r a t i o n s , not s t a t i s t i c a l ones. The r e s u l t s of t h i s study i n d i c a t e , however, that the choice of the block s i z e may be a f a c t o r i n the subsequent s t a t i s t i c a l a n a l y s i s . The most l u c i d example of t h i s i s for the 12 t r i a l s of Data set 7 based on the variance-covariance matrices for Conditions 1 and 3. In Condition 1 the p r o b a b i l i t y of committing a Type I e r r o r d i f f e r e d s i g n i f i c a n t l y depending upon the block s i z e chosen. At the .10 l e v e l of s i g n i f i c a n c e , 9.2% of the experiments had s i g n i f i c a n t i n t e r a c t i o n s when VE was based on four t r i a l s / b l o c k , but jumped to 13.8% when three t r i a l s / b l o c k 33 were used. A nominal alpha equal to .05 d i s p l a y e d an increase from 4.6% to 7.6% while a s i x - f o l d increase occurred (.40%^ to 2.4%) at the .01 l e v e l of s i g n i f i c a n c e . S i m i l a r changes i n the number of Type I e r r o r s were found under Condition 3 f o r t h i s data. More i n t e r e s t i n g are the r e s u l t s of the s i m u l a t i o n s based on the a c t u a l variance-covariance matrices for each group (Condition 4). T h i s , of course, i s the one which an a c t u a l researcher would analyze. Data sets 2 and 5 both show n o t i c e a b l e d i f f e r e n c e s i n the number of Type I e r r o r s depending upon the block s i z e used to c a l c u l a t e VE. In Data set 5, c o n d i t i o n 4, the three t r i a l s / b l o c k p a t t e r n r e s u l t e d i n almost double the number of Type I e r r o r s found for s i x t r i a l s / b l o c k . The corresponding p r o b a b i l i t i e s are .496 vs .252 for a=.10, .356 vs .154 for o=.05 and .172 vs .048 at the .01 l e v e l of s i g n i f i c a n c e . In l o o k i n g at Data Set 2 (Table I I I ) i t i s obvious that using four t r i a l s / b l o c k instead of three t r i a l s / b l o c k r e s u l t s i n almost twice the number of s i g n i f i c a n t i n t e r a c t i o n s at the .10 l e v e l of s i g n i f i c a n c e and more than three times at the .01 l e v e l . The question remains as to the nature of the r e l a t i o n s h i p between the s i z e of the block and the p r o b a b i l i t y of committing a Type I e r r o r . The number of Type I e r r o r s increase i n v e r s e l y to the s i z e of the block for Data set 7 (12 t r i a l s ) under Conditions 1 and 3 and for Data set 7 under Condition 4. I t appeared that t h i s a l s o was true for Data set 2 (Condition 4) since the percentage of s i g n i f i c a n t i n t e r a c t i o n s increased from .762 to .922 (at a=.l0.) as the s i z e of the block decreased from eight t r i a l s / b l o c k to four t r i a l s / b l o c k . However, when the block 34 s i z e was f u r t h e r reduced to three t r i a l s / b l o c k the Type I er r o r rate decreased to .520. Therefore, the obtained r e s u l t s are in c o n c l u s i v e as to whether there i s a d i r e c t r e l a t i o n s h i p between block s i z e and the p r o b a b i l i t y of o b t a i n i n g a s i g n f i c a n t i n t e r a c t i o n when analyzing VE data with an a n a l y s i s of variance. Although block s i z e i s not d i r e c t l y r e l a t e d to the p r o b a b i l i t y of obta i n i n g s i g n i f i c a n c e f or VE data, i t appears that the proper choice of the s i z e of the block may d r a s t i c a l l y a f f e c t the researcher's p r o b a b i l i t y of r e j e c t i n g the n u l l hypothesis. Examining the number of s i g n i f i c a n t Groups by Blocks i n t e r a c t i o n s for Data set 5 under c o n d i t i o n 4, i t i s apparent that the p r o b a b i l i t y of ob t a i n i n g s i g n i f i c a n c e was greater when three t r i a l s / b l o c k were used i n c a l c u l a t i n g VE (Table I I I ) . In f a c t , at the .01 l e v e l of s i g n i f i c a n c e , the 6 X 3 case produced 3.6 times as many s i g n i f i c a n t i n t e r a c t i o n s as d i d the 3 X 6 bloc k i n g p a t t e r n . While the percent d i f f e r e n c e i n the number of s i g n i f i c a n t i n t e r a c t i o n s decreases as the l e v e l of s i g n i f i c a n c e increases, at c=.10, the s i x blocks case r e s u l t e d i n 1.97 times the number of s i g n i f i c a n t i n t e r a c t i o n s as when three blocks of VE were analyzed. S i m i l a r , though not as extreme, values are apparent for the r e s u l t s of Data set 2, c o n d i t i o n 4 (Table I I I ) . The f a c t that the s i z e of the blocks used to c a l c u l a t e VE may d i f f e r e n t i a l l y a f f e c t the p r o b a b i l i t y of achieving a s i g n i f i c a n t i n t e r a c t i o n undermines the r e l i a b i l i t y of VE when i t i s analyzed by an ANOVA. 35 CONCLUSIONS Based on the a n a l y s i s of the s t r u c t u r e of c o r r e l a t i o n matrices for raw and VE data of eight a c t u a l experimental data s e t s , and on Monte Carlo analyses of three of these experiments, the f o l l o w i n g conclusions can be made: 1. A " t y p i c a l " c o r r e l a t i o n p a t t e r n does not e x i s t for e i t h e r the raw data or the VE scores. 2. C o r r e l a t i o n s between raw scores for subjects r e c e i v i n g no feedback are g e n e r a l l y l e s s v a r i a b l e and 'greater i n magnitude than for those subjects who were given feedback. 3. The c o r r e l a t i o n matrix among VE scores i s u s u a l l y more homogeneous, than for unblocked data. 4. E m p i r i c a l performance e r r o r scores are marg i n a l l y normally d i s t r i b u t e d . VE scores (the square root of the w i t h i n - s u b j e c t variance) a l s o appear to have normal d i s t r i b u t i o n s . However, these r e s u l t s are based on small sample s i z e s (max=48) and, the r e f o r e , s t u d i e s with l a r g e r samples are needed to confirm t h i s . „5. Most e m p i r i c a l data sets v i o l a t e both the w i t h i n and between matrix homogeneity assumptions. 6. I f the raw data s a t i s f i e s the covariance homogeneity assumptions, then the subsequent analyses of VE scores by an a n a l y s i s of variance does not i n f l a t e the Type I e r r o r r a t e . 7. In a n a l y z i n g experiments with repeated measurements by an a n a l y s i s of variance the within-group homogeneity of covariance assumption i s more important than the between-group assumption. V i o l a t i o n of the former assumption r e s u l t s i n an increase' i n the Type I e r r o r rate when raw data i s analyzed but r e s u l t s are 36 i n c o n c l u s i v e with VE data. However, when the within-group assumption i s s a t i s f i e d and the between-group assumption i s v i o l a t e d no i n f l a t i o n i n the number of Type I e r r o r s occurs. 8. The s i z e of the block used to c a l c u l a t e VE a f f e c t s the p r o b a b i l i t y of achieving s i g n i f i c a n c e . Such a f i n d i n g questions the r e l i a b i l i t y of using VE as a dependent measure i n an ANOVA. 9. When analyz i n g raw data the number of t r i a l s i n the design does not d i f f e r e n t i a l l y a f f e c t the Type I e r r o r rate i f the within-group c o r r e l a t i o n matrices are homogeneous. I f these matrices are heterogeneous the degree of i n f l a t i o n of Type I er r o r s appears to be r e l a t e d to an i n t e r a c t i v e e f f e c t between the number of t r i a l s and the degree of heterogeneity w i t h i n the ma t r i c e s . 37 REFERENCES Boneau, C. A. The e f f e c t s of v i o l a t i o n s of assumptions u n d e r l y i n g the t t e s t . P s y c h o l o g i c a l B u l l e t i n , 1960, 57, 49- 64. Box, G. E. P. Some theorems on q u a d r a t i c forms a p p l i e d i n the study of a n a l y s i s of v a r i a n c e problems, I I . E f f e c t s of i n e q u a l i t y of v a r i a n c e and of c o r r e l a t i o n between e r r o r s i n the two-way c l a s s i f i c a t i o n . Annals of Mathematical S t a t i s t i c s , 1 954b, 25j_ 484-498. C o l l i e r , R. G., Baker, F. B., M a n d e v i l i e , G. K., & Hayes, T. F. Estimates of t e s t s i z e f o r s e v e r a l t e s t procedures based on c o n v e n t i o n a l v a r i a n c e r a t i o s in the repeated measures desi g n . Psychometrica, 1967, 32, 339-353. Davidson, M. L. U n i v a r i a t e versus m u l t i v a r i a t e t e s t s i n repeated measures experiments. P s y c h o l o g i c a l B u l l e t i n , 1972, 77, 446-452. G a i t o , J . Repeated measurements designs and t e s t s of n u l l hypotheses. E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 1973, 33, 69-75. Goodman, D. & Schutz, R. W. DATASNIFF: A program to check data and compute summary d e s c r i p t i v e s t a t i s t i c s and c o r r e l a t i o n s . Developed at the P h y s i c a l Education Q u a n t i f i c a t i o n Laboratory, U n i v e r s i t y of B r i t i s h Columbia, 1975. Halm, J . UBC NORMAL. U n i v e r s i t y of B r i t i s h Columbia Computing Center, 1970. Laabs, G. J . Retention c h a r a c t e r i s t i c s of d i f f e r e n t r e p r o d u c t i o n cues in motor short-term memory. J o u r n a l of Experimental Psychology, 1973, 100, 168-177. Lana, R. E., & Lubin, A. The e f f e c t of c o r r e l a t i o n on the repeated measures d e s i g n . E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 1963, 23^ 729-739. Newell, K. M. More on a b s o l u t e e r r o r , e t c . J o u r n a l of Motor Behavior, 1976, 8^ 139-142. Rogan, J . C. , Keselman, H. J . & Mendoza, J . L. A n a l y s i s of 38 repeated measurements. B r i t i s h J ournal of Mathematical and S t a t i s t i c a l Psychology, 1979, 32, 269-286. S a f r i t , M. J . , Spray, A. J . & Diewert, G. L. Methodological issues i n short-term motor memory research. Journal of Motor Behavior, 1980, 12, 13-28. Schmidt, R. A. A schema theory of d i s c r e t e motor l e a r n i n g . P s y c h o l o g i c a l Review, 1975, 82, 225-260. Schutz, R. W. Absolute, Constant, and V a r i a b l e E r r o r : Problems and s o l u t i o n s . In D. Mood (Ed.), Proceedings of the Colorado Measurement Symposium. U n i v e r s i t y of Colorado Press: Boulder, Colorado, 1979, 82-100. , Schutz, R. W. & G e s s a r o l i , M. E. The e f f e c t s of block s i z e and heterogeneity of covariance on Type I e r r o r rates with Constant e r r o r and V a r i a b l e e r r o r data, i n Psychology of Motor Behavior and Sport - 1979. Champaign, I l l i n o i s : Human K i n e t i c s , 1980, 633-642. Schutz, R. W., & Roy, E. A. Absolute e r r o r : The d e v i l i n d i s g u i s e . Journal of Motor Behavior, 1973, 5^ 141-153. Appendix A LETTER REQUESTING EXPERIMENTAL DATA 40 Appendix A LETTER REQUESTING EXPERIMENTAL DATA 24 June, 1980 Dear As a follow-up to our T r o i s - R i v i e r e s paper on heterogeneity of covariance and block s i z e , Marc G e s s a r o l i and I are embarking on a research p r o j e c t on VE. Very b r i e f l y , our research purposes are as f o l l o w s : 1) determine the d i s t r i b u t i o n of VE (as a variance i t i s probably d i s t r i b u t e d as chi-square) and a s c e r t a i n how t h i s a f f e c t s the d i s t r i b u t i o n of F i n a t y p i c a l repeated measures ANOVA; 2) examine t h i s e f f e c t under d i f f e r e n t c o n d i t i o n s of number of t r i a l s , b l o c k i n g parameters, and variance-covariance s t r u c t u r e s . To accomplish t h i s we plan on c o l l e c t i n g e m p i r i c a l data from researchers in the f i e l d i n order to determine the a c t u a l d i s t r i b u t i o n of VE under various experimental c o n d i t i o n s and for a v a r i e t y of dependent v a r i a b l e s . Based on the f i n d i n g s , Monte Carlo procedures w i l l be followed to simulate r e a l i t y while varying the parameters of number of t r i a l s , block s i z e and variance-covariance s t r u c t u r e . As you have probably summized by now, I would l i k e to get some of you data! We are p r i m a r i l y i n t e r e s t e d i n l e a r n i n g data, but may ( i f not enough l e a r n i n g data i s a v a i l a b l e ) a l s o look at performance data. What we would l i k e i s raw data (data sheets, computer l i s t i n g , cards, or whatever) which has been used to r e f l e c t performance e r r o r , i . e . , from l i n e a r p o s i t i o n i n g tasks, temporal accuracy, e t c . We are r e s t r i c t i n g our e m p i r i c a l samples to data sets which meet the f o l l o w i n g requirements: 1) at l e a s t 12 t r i a l s per experimental c o n d i t i o n , and 2) at lease 12 subjects per group (one or more groups). I f you have such data s e t ( s ) a v a i l a b l e I would be most a p p r e c i a t i v e i f you would send i t to us. A d e s c r i p t i o n of the experimental design and data format, an i n d i c a t i o n of what ( i f any) b l o c k i n g was performed, and, i f p o s s i b l e , a copy of any published or unpublished reports of the s t u d i e s would be necessary i n order for us to i n t e r p r e t and analyze your data. Please note - we are not conducting a review of the appropriateness of s t a t i s t i c a l analyses done i n our f i e l d , and w i l l not be r e - a n a l y z i n g your data (but j u s t l o o k i n g at the d i s t r i b u t i o n of the raw data and the VE s c o r e s ) . 41 Marc w i l l be using these data sets i n h i s Master's t h e s i s . Included on h i s committee are Dr. Ralph Hakstian, a noted psychometrician, Dr. John Petkau, a b r i l l i a n t young mathematical s t a t i s t i c i a n , and Dr. Gordon Diewert from Simon Fraser U n i v e r s i t y . They a l l view t h i s study as a c h a l l e n g i n g and worthy study. I b e l i e v e that with t h e i r help we can make a valuable c o n t r i b u t i o n to an important m e a s u r e m e n t / s t a t i s t i c a l problem i n motor behavior research. Your a s s i s t a n c e w i l l enable us to accomplish t h i s . We w i l l be glad to send you a copy of our f i n d i n g s , and reimburse you for any c o s t s a s s o c i a t e d with sending and d u p l i c a t i n g m a t e r i a l s . Thank-you i n a n t i c i p a t i o n . Yours s i n c e r e l y , R.W. Schutz Professor A p p e n d i x B P R O G R A M TO C A L C U L A T E V E AND A N O V A 43 Appendix B ' PROGRAM TO CALCULATE VE AND ANOVA DOUBLE PRECISION X(200,18),SUBSUM(200),SUBSM2(200),TR1(100) DOUBLE PRECISION.X2(200,100),TX2(200),BE(200),BE2(200),AE(200). DOUBLE PRECISION DSWGT(500),DTRIAL(500),DGRPS(500),DSWG(500) DOUBLE PRECISION TRT2(100), VE(200,60),TR2(100),DGXT(500),AE2(200) DOUBLE PRECISION TRT(100),F(500),SX,SX2,V(200,60) DOUBLE PRECISION FTOT,TDGXT,TDSWGT, TDTR,TDGRPS,TDSWG,SUBJ DOUBLE PRECISION TOTAL2,BE2G1,BE2G2,SSTR,XGT,XGTB,XTOTAL DOUBLE PRECISION SUBJCT,XTOT2,TOTAL,BETW,TRIALS,SUBTR,XGROUP DOUBLE PRECISION XGRTR,SWGT,SWG READ(8,16) NT,NSG,NTB,NREP,IB,FVAL10,FVAL5,FVAL1 16 FORMAT(4(1X,I3),1X,I1,3(1X,F5.3)) NS=NSG*2 NTOT=NS*NREP NB=NT/NTB K= 1 L=NSG SX=0. SX2 = 0. XNB=NB XNSG=NSG XNT=NT XNS=NS L2=L/2 K2=K+NSG FTOT=0. TDGXT= 0. TDSWGT=0. TDTR=0. TDGRPS=0. TDSWG=0. IT10=0. IT5=0 IT1=0 DO 105 NR=1,NREP K=1 L=NSG READ(4,1) ((X(I,J),J=1,NT),I=K,L) K=K+NSG L=L+NSG READ(5,1) ((X(I,J),J=1,NT),I=K,L) 1 FORMAT(12(1X,F10.5)/12(1X,F10.5)) IF(IB.EQ.1 ) GO TO 71 NT=NB GO TO 106 71 DO 11 I=1,NS M=0 DO 10 J=1,NT,NTB J2=J+(NTB-1) DO 9 K=J,J2 SX=SX+X(I,K) SX2=SX2+(X(I,K)**2) 9 CONTINUE 44 M=M+1 VE(I,M)=((SX2-(SX**2)/NTB)/NTB)**.5 SX=0. SX2 = 0. 10 CONTINUE 11 CONTINUE GO TO 201 106 DO 199 1=1,NS DO 198 J=1,NT VE(I,J)=X(I,J) 198 CONTINUE 199 CONTINUE 201 XTOTAL=0. K=1 L=NS SUBJ=0. TOTAL2=0. BE2G1=0. BE2G2=0. SSTR=0. XGT=0. XGTB=0. K2=K+NSG L2=L/2 DO 99 I=K,L SUBSUM(I )=0. TX2(I)=0. DO 98 M=1,NB XTOTAL=XTOTAL+VE(I,M) SUBSUM(I)=SUBSUM(I)+VE(I,M) TX2(I)=TX2(I)+(VE(l,M)**2) 98 CONTINUE SUBJ=SUBJ+(SUBSUM(I)**2) TOTAL 2=TOTAL 2+TX2(I) 99 CONTINUE XTOT2=(XTOTAL**2)/(XNB*XNS) SUBJCT=(SUBJ/XNB)-XTOT2 TOTAL=TOTAL2-XTOT2 DO 89 M=1, NB TR1(M)=0. BE(M)=0. DO 88 I=K,L2 TR1(M)=TR1(M)+VE(I,M) 88 CONTINUE BE2G1=BE2G1+(TR1(M)**2) 89 CONTINUE DO 79 M=1,NB AE(M)=0. TR2(M)=0. DO 78 I=K2,L TR2(M)=TR2(M)+VE(I,M) 78 CONTINUE SSTR=SSTR+((TR1(M)+TR2(M))**2) BE2G2=BE2G2+(TR2(M)**2) 79 CONTINUE BETW=((BE2G2+BE2G1)/XNSG)-XTOT2 45 TRIALS=(SSTR/NS)-XT0T2 SUBTR=TOTAL-SUBJCT-TRIALS DO 69 I=K,L2 DO 68 M=1,NB XGT=XGT+VE(I,M) 68 CONTINUE 69 CONTINUE DO 59 I=K2,L DO 58 M=1,NB XGTB=XGTB+VE(I,M) 58 CONTINUE 59 CONTINUE XGROUP=(((XGT**2)+(XGTB**2))/(XNB*XNSG))-XTOT2 XGRTR=BETW-TRIALS-XGROUP SWGT=TOTAL-SUBJCT-TRIALS-XGRTR SWG=SUBJCT-XGROUP DGXT(NR)=XGRTR/(XNB-1) DSWGT(NR)=SWGT/((2*(XNSG-1))*(XNB-1)) DTRIAL(NR)=TRIALS/(XNB-1) DGRPS(NR)=XGROUP DSWG(NR)=SWG/(2*(XNSG-1)) F(NR)=DGXT(NR)/DSWGT(NR) IF(F(NR).GE.FVAL10) IT10=IT10+1 IF(F(NR).GE.FVAL 5) IT5 = IT5+1 IF(F(NR).GE.FVAL1) IT1=IT1+1 FTOT=FTOT+F(NR) TDGXT=TDGXT+DGXT(NR) TDSWGT=TDSWGT+DSWGT(NR) TDTR=TDTR+DTRIAL(NR) TDGRPS=TDGRPS+DGRPS(NR) TDSWG=TDSWG+DSWG(NR) 105 CONTINUE FMEAN=FTOT/NREP TDGXTM=TDGXT/NREP TSWGTM=TDSWGT/NREP TDTRM=TDTR/NREP TDGRPM=TDGRPS/NREP TDSWGM=TDSWG/NREP WRITE(6,2) IT10,IT5, IT1, FMEAN,TDGRPM,TDSWGM,TDTRM,TDGXTM,TSWGTM 2 FORMAT('THE # OF PS LESS THAN 10=',13,/, *'THE NUMBER OF PS LESS THAN 05=',13,/, *'THE NUMBER OF PS LESS THAN 01 = ',13,/, *'THE MEAN F VALUE WAS = ',F10.5,/, *'THE MEAN FOR MS GROUPS =',F13.4,/, *'THE MEAN FOR SUB WITHIN GROUPS =',F12.4,/, *'THE MEAN FOR MS TRIALS = ',F12.4,/, *'THE MEAN FOR MS GROUPS BY TRIALS = ',F10.4,/, *'THE MEAN FOR MS SUB WITHIN GROUPS BY TRIALS = ',F10.4) STOP END Appendix C REVIEW OF LITERATURE 47 Appendix. C REVIEW OF LITERATURE Overview of Chapter The most common experiment i n motor behaviour research involves each subject performing s e v e r a l t r i a l s of a p a r t i c u l a r task. Repeated measures designs are i n v a r i a b l y used since i t i s the researcher's goal to study how the subject performs over a period of time. In t h i s way, some knowledge as to how a subject l e a r n s , f o r gets or r e t a i n s may be examined. U s u a l l y , there are at l e a s t two experimental c o n d i t i o n s i n the design, thereby a l l o w i n g f o r comparisons between various groups or treatment c o n d i t i o n s . The data are g e n e r a l l y analyzed by an a n a l y s i s of variance. The proper a n a l y s i s of repeated measures data v i a ANOVA i s dependent upon the data s a t i s f y i n g various assumptions. While the assumptions of normality and e q u a l i t y of variances are important and should be checked, the most common assumptions which are v i o l a t e d with motor l e a r n i n g data are those d e a l i n g with the heterogeneity of covariances. In f a c t , Lana and Lubin (1963) and others s t a t e d that c o r r e l a t i o n s among t r i a l s c l o s e r together are la r g e r than for those f a r t h e r apart. A l s o , because the experimental groups are g e n e r a l l y q u i t e d i f f e r e n t , the covariance matrices between the various groups are probably unequal - a v i o l a t i o n of an assumption of ANOVA. Therefore, although previous research i n t o the e f f e c t s of v i o l a t i n g the 48 assumptions of normality and e q u a l i t y of variances w i l l be summarized, the emphasis w i l l be on reviewing l i t e r a t u r e concerned with the assumption of compound symmetry (homogeneity of the covariances w i t h i n each group and between the varian c e - covariance matrices of each group). The e f f e c t s of v i o l a t i n g these assumptions on the Type I er r o r rate and methods for compensating for covariance heterogeneity w i l l be the main focus of t h i s l i t e r a t u r e review. There has been much debate i n the l i t e r a t u r e over the l a s t eleven years as to the proper choice of a dependent v a r i a b l e i n motor behaviour s t u d i e s . Some of the arguements have been made on a purely t h e o r e t i c a l b a s i s while others have considered the s t a t i s t i c a l p r o p e r t i e s of the dependent measures. As t h i s study concerns i t s e l f with the a n a l y s i s of one of these dependent v a r i a b l e s (VE) a review of the ensuing debate seems ap p r o p r i a t e . The S t a t i s t i c a l Model As mentioned p r e v i o u s l y , the common motor l e a r n i n g experiment c o n s i s t s of each subject performing s e v e r a l t r i a l s (q) of a ..specific task. U s u a l l y the subjects are d i v i d e d i n t o £ experimental groups, the r e s u l t a n t design being a p X q experimental design with repeated measures on the l a s t f a c t o r . This data i s subsequently analyzed by an a n a l y s i s of varia n c e . The model underlying a repeated measures ANOVA of t h i s type i s l i n e a r i n nature and defined by: x i j k = " + °j+<3k+»i(j) + 0 ' j k + " k i ( j ) + £ i j k where xij|< defines the score for the i t h subject i n the j t h group on the kth t r i a l ; pis the o v e r a l population mean; o- and 0k are the e f f e c t s of the j t h treatment and the kth occasion, 49 r e s p e c t i v e l y ; ^ s a constant r e l a t i n g the i t h subject with the j t h treatment group; i s t n e i n t e r a c t i o n of the j t h group with the kth occasion; ^ ^ - ( j ) * s t n e i n t e r a c t i o n of occasion k and subject i w i t h i n j ; and t - j j ^ i s the random e r r o r in the system. Furthermore, these parameters are subject to the f o l l o w i n g c o n s t r a i n t s : )°i = = J°PJk = ^ j k = ^ " k i ( j ) = °' where i = 1,..'.,N; j = 1,...,P; k=1 , . . . ,Q Assumptions of Repeated Measures ANOVA The s p e c i f i c assumptions underlying the a n a l y s i s of repeated measures data by a n a l y s i s of variance are as f o l l o w s : 1. The populations must be m u l t i v a r i a t e l y normally d i s t r i b u t e d . 2. The population variances must be equal. 3. (a) The magnitudes of the covariances w i t h i n a group must be equal. (b) The magnitudes of the covariances between each grouping f a c t o r must be equal. Assumption of normality. The f i r s t assumption underlying an a n a l y s i s of variance i s that the populations must be d i s t r i b u t e d as m u l t i v a r i a t e l y normal. However, as t e s t s f or m u l t i v a r i a t e normality are few and somewhat complex i n nature (see Gnanadesikan, p. 151-195), the l e s s s t r i n g e n t assumption of u n i v a r i a t e normality between the marginal d i s t r i b u t i o n s has been accepted as a s a t i s f a c t o r y c o n d i t i o n for a v a l i d F t e s t . Several e a r l y pieces of research have been done studying the e f f e c t s of non-normality on the p r o b a b i l i t y of committing a Type I e r r o r . Although a multitude of research regarding the e f f e c t s of non- normality e x i s t s , only a summary of the conclusions w i l l be 50 presented here. Boneau ( i 9 6 0 ) , using equal sample s i z e s and equal variances, found that an ANOVA. i s q u i t e robust to varying l e v e l s of non- normality. In f a c t , i n f l a t i o n i n the number of Type I e r r o r s was found only when one or more of the populations were non-normal ( i . e . , exponential or rectangular) and the sample s i z e s were very small ( f i v e subjects/group). As the sample s i z e s increases to 15 subjects/group, the a c t u a l number of, Type I e r r o r s was only s l i g h t l y higher than the nominal value. Scheffe (1959) has proven mathematically that the robustness of the ANOVA F t e s t increases as N becomes large with F t e s t s being p e r f e c t l y robust with i n f i n i t e sample s i z e s . Therefore, i t appears that i f the sample s i z e s and variances are equal, the F t e s t i s q u i t e robust to v i o l a t i o n s of the normality assumption with the robustness i n c r e a s i n g as N increases. When non-normality i s combined with other f a c t o r s such as unequal variances and/or covariances the r e s u l t s are d i f f e r e n t . Several i n v e s t i g a t o r s have stated that ANOVA i s f a i r l y robust to departures from normality and e q u a l i t y of variances (e.g., G a i t o , 1973; Wilson & Lange, 1972), but Bradley (1980) showed that the combination of these two f a c t o r s severely a f f e c t s the Type I er r o r r a t e . Bradley, attempting to simulate r e a l - l i f e data, found that under varying l e v e l s of unequal sample s i z e s , non-normality and variance r a t i o s , 25% of the s i t u a t i o n s f a i l e d to produce a reasonable F l e v e l when N was l e s s than 100. He found that the sample s i z e needed for robustness increased as the l e v e l of s i g n i f i c a n c e decreases. More s p e c i f i c c o n d i t i o n s and t h e i r e f f e c t s on the robustness of the t e s t are discussed i n 51 the a r t i c l e . Non-normality when combined with covariance heterogeneity has the e f f e c t of i n f l a t i n g the Type I e r r o r rate of the w i t h i n - subjects main e f f e c t , e s p e c i a l l y when using m u l t i v a r i a t e t e s t s (Mendoza, Toothaker & Nicewander, 1974; Rogan, Keselman . & Mendoza, 1979). However, when the e f f e c t of i n t e r e s t was the w i t h i n - s u b j e c t s i n t e r a c t i o n , the a c t u a l Type I e r r o r rate underestimated the nominal l e v e l of s i g n i f i c a n c e . Thus, when anal y z i n g w i t h i n - s u b j e c t s e f f e c t s from non-normal data d i s p l a y i n g heterogeneous covariances, the e f f e c t being t e s t e d must be considered. Homogeneity of variances. An e a r l y study by Hsu (1938) showed that the t - t e s t i s robust to i n e q u a l i t y of variance i f the sample s i z e s are equal. However, the a c t u a l p r o b a b i l i t y of committing a Type I e r r o r moves away from the nominal l e v e l of s i g n i f i c a n c e as the r a t i o between the variances and/or the degree of i n e q u a l i t y between sample s i z e s increase (Hsu, 1938; Scheffe, 1959). More s p e c i f i c a l l y , when the smaller variance i s a s s o c i a t e d with the l a r g e r p o p u l a t i o n , an i n f l a t i o n i n the Type I e r r o r rate occurs while i n the s i t u a t i o n where the l a r g e r population has the l a r g e r v a r i a n c e , the a c t u a l alpha underestimates the nominal l e v e l . C o l l i e r , Baker, Mandeville and Hayes (1967), i n a Monte Car l o study, found that there were no extreme departures from the nominal alpha l e v e l s i f the covariances and sample s i z e s were equal and any i n f l a t i o n which d i d occur decreased as the sample s i z e increased. As with the assumption of n o r m a l i t y , the F t e s t i s q u i t e robust to v i o l a t i o n s of the homogeneous variances assumption 52 with the degree of robustness i n c r e a s i n g with increases i n the sample , s i z e . However, as Bradley (1980) d i s p l a y e d , the i n t e r a c t i v e e f f e c t s of the v i o l a t i o n s of the various assumptions can have severe e f f e c t s on the Type I e r r o r r a t e , and the f a c t that the sample s i z e s are equal i s not s u f f i c i e n t reason to assume robustness of the ANOVA. Homogeneity of covariances. The f i n a l two assumptions can be represented by the Q X Q population variance-covariance matrix of the form: a2 pa2 p e r c r • per • p a 2 p e r p e r • • • c r = ^2 '1 P P 1 P P ' D e f i n i n g the population variance-covariance matrix f o r each l e v e l of P as Ij , the above matrix must be common to a l l l e v e l s of P ( i . e . , Z.= Z, j=1,...,P) i n the p x q design. A matrix of the above form i s s a i d to have the p r o p e r t i e s of "compound symmetry" or "u n i f o r m i t y " (Geisser, 1963) or "multisample s p h e r i c i t y " (Huynh, 1978). Studies i n motor l e a r n i n g i n which a subject i s te s t e d on many t r i a l s over time on a task, i n most cases, do not adhere to the e q u a l i t y of covariance assumptions. I t i s not u n l i k e l y to have higher c o r r e l a t i o n s between adjacent t r i a l s with the magnitude of the c o r r e l a t i o n s decreasing as the t r i a l s become f a r t h e r apart (Davidson, 1972; Greenwald, 1976; Lana & Lubin, 1963; Wilson, 1975). The question remains as to the e f f e c t on the v a l i d i t y of the ANOVA when one or more of the above assumptions are v i o l a t e d . This study i s p r i m a r i l y concerned with the e f f e c t of the the v i o l a t i o n s of these 53 assumptions and, t h e r e f o r e , the remainder of the l i t e r a t u r e review deals almost e x c l u s i v e l y with variance-covariance heterogeneity problems. Heterogenity of Covariances: D e f i n i t i o n and Measurement. A measure of covariance heterogeneity. Box (1954b) i n studying, the e f f e c t s of unequal covariances on a one-way t e s t for d i f f e r e n c e s i n treatments found that the r a t i o , SST/ S JWITHIN ' has an approximate F d i s t r i b u t i o n with degrees of freedom equal to ( q - l ) e and (q-1)(p-1)e, where € i s defined as e = q 2 («' -c. . ) V(q-1 ) [He2. . -2kl<r.2 +k 2 t f 2. ], and e i s the mean of the -column variances, c- i s the mean of the i t h row and cm. i s the mean of a l l the elements i n the population covariance matrix. Geisser and Greenhouse ( 1 9 5 8 ) , i n extending Box's f i n d i n g s , showed that e must l i e between 1 / ( q - l ) and 1. I f the variances are homogeneous and the covariances are homogeneous, e=1. Extreme degrees of heterogeneity r e s u l t i n € having a value of l / ( q - 1 ) . Under the c o n d i t i o n of complete homogeneity amongst the variances and covariances (e=1) the degree of freedom for the c r i t i c a l F are (q-1),(n-1)(q-1), while when €=1/(q-l) the t e s t s t a t i s t i c f or s i g n i f i c a n c e i s F [ l , ( n - 1 ) ] . As i s obvious, the former F value i s l e s s s t r i n g e n t than the l a t t e r , t h e r e f o r e , i t i s c a l l e d a " l i b e r a l " t e s t while the l a t t e r c r i t i c a l F value i s greater r e s u l t i n g i n a "conservative" t e s t . Applying Box's r e s u l t s from a one-way c l a s s i f i c a t i o n to the 54 two-way c l a s s i f i c a t i o n ( i . e . , a grouping f a c t o r e x i s t s ) Geisser and Greenhouse (1958) found that the adjusted degrees of freedom corresponding to the t e s t for s i g n i f i c a n c e between treatments (MS /MSC r_ ) and for i n t e r a c t i o n s (MS„ /MS ^ ) to be (q-1)e, T SwGT GxT SwGT p ( n - l ) ( q - l ) e and ( p - l ) ( q - 1 ) e , p(n-1)(q-1)c, r e s p e c t i v e l y . The upper and lower bounds for t i n the two way c l a s s i f i c a t i o n remain at 1 and l / ( q - 1 ) , thus f a c i l i t a t i n g the c a l c u l a t i o n of the l i b e r a l and conservative c r i t i c a l F values. Compound symmetry and c i r c u l a r i t y . In a Groups by T r i a l s repeated mesures design three t e s t s t a t i s t i c s (F r a t i o s ) are c a l c u l a t e d by. the ANOVA: E, =MSQ/MS^wg , a t e s t for d i f f e r e n c e s i n groups; F-j-=MSy/MSgwg-p, a t e s t f o r d i f f e r e n c e s between t r i a l s ; and FQJ =MSgx-j. /MS^Q-J- , a t e s t f o r i n t e r a c t i o n . A l l these r a t i o s are d i s t r i b u t e d as F with appropriate degrees of freedom i f compound symmetry e x i s t s in the variance-covariance matrix (with the exception of MS^/MS^g which i s not dependent upon such a r e s t r i c t i o n ) . S i m i l a r l y , i f there i s no grouping f a c t o r ( i . e . , a one-way c l a s s i f i c a t i o n ) , the t e s t s t a t i s t i c f o r d i f f e r e n c e s i n treatments i s given by F s M S ^/MS W I T H I N . F has an F d i s t r i b u t i o n i f the compund symmetry assumption i s s a t i s f i e d . Work by Rouanet and Lepine ('1970) and Huynh and F e l d t (1970) has shown that the assumption of u n i f o r m i t y or symmetry of the yariance-covariance matrices need not n e c e s s a r i l y be met for the F r a t i o to be l e g i t i m a t e . Given g t r i a l s in a one-way c l a s s i f i c a t i o n , a s u f f i c i e n t c o n d i t i o n for an exact F t e s t i s when C'IC=<72I^ ^ , where I i s the population variance- covariance matrix, I i s the i d e n t i t y matrix and C i s a (q-1)- dimensional orthonormal contrast matrix (Huynh & F e l d t , 1970; 55 Rouanet and Lepine, 1970). Both sets of authors, by d i f f e r e n t methods, show that i f t h i s c o n d i t i o n i s met (the c o n d i t i o n i s defined as " c i r c u l a r i t y " ) the Box-Geisser-Greenhouse c o r r e c t i o n f a c t o r e i s equal to one. Extending t h i s i d e a , i t f o l l o w s that i f the symmetry assumption i s s a t i s f i e d then so i s the c i r c u l a r i t y assumption. However, i t does not n e c e s s a r i l y f o l l o w that c i r c u l a r i t y i m plies symmetry of the variance-covariance matrix (Rouanet & Lepine, 1970). I t i s obvious that c i r c u l a r i t y i s a l e s s s t r i n g e n t requirement necessary to obtain v a l i d F r a t i o s by a n a l y s i s of variance. In a two-way c l a s s i f i c a t i o n (a between-groups f a c t o r e x i s t s ) the c o n d i t i o n which must be s a t i s f i e d i s : C ' l C=«r2I, p=1,...P (Huynh & F e l d t , 1970). This i m p l i e s that the c o n d i t i o n of c i r c u l a r i t y , as described above, e x i s t s for each of the P groups and that the value of C EC r e s u l t s in the same value of the s c a l a r , a2 , for each group. The primary d i f f e r e n c e between the r e s u l t s of Huynh and F e l d t and those of Rouanet and Lepine i s that the f i r s t set of authors deal only with the c i r c u l a r i t y c o n d i t i o n s for the o v e r a l l F t e s t while Rouanet and Lepine consider both o v e r a l l and p a r t i a l F t e s t s . Rouanet and Lepine showed that c e r t a i n p a r t i a l comparisons are v a l i d even i f the o v e r a l l c i r c u l a r i t y c o n d i t i o n i s not s a t i s f i e d . The example given by Rouanet and Lepine i s based upon a four by two c l a s s i f i c a t i o n with repeated measures on both f a c t o r s ( i . e . , eight treatments). They define the o v e r a l l comparison (7 df) as w e l l as three p o s s i b l e p a r t i a l comparisons based upon the two f a c t o r s (3 and 1 df) and the i n t e r a c t i o n (3 d f ) . 56 Two methods have been suggested for t e s t i n g . p a r t i a l comparisons: (1) using an e r r o r term , based upon the corresponding sum of squares as the e f f e c t being t e s t e d . For example, the denominator for the t e s t of a comparison based on f a c t o r A would be Subjects w i t h i n A. The corresponding F r a t i o i s designated as F'. (2) Using an e r r o r term based upon the o v e r a l l sum of squares ( i . e . , the sum of the three sum , of squares, S S $ W A , SS5Wg , SS5 w^g). This r a t i o i s c a l l e d F". Authors d i f f e r i n t h e i r o pinion as to which i s the proper er r o r term to use. Many t e x t s favor the us of F" only while others s t a t e that F' should be used i n a l l cases (e.g., Gaito & Turner, 1963). Since the degrees of freedom are l a r g e r i n the er r o r term for F" than for F', i t would seem that F" y i e l d s a more powerful t e s t . However, s a t i s f y i n g the c i r c u l a r i t y assumption for F" i s more d i f f i c u l t than for F'. I f the o v e r a l l c i r c u l a r i t y assumption i s s a t i s f i e d (F" i s v a l i d ) , then any of the p a r t i a l comparisons (F') are a l s o v a l i d . However, the opposite does not apply. The assumption for F' i s l e s s s t r i n g e n t than for F" and becomes weaker as the degrees of freedom i n the er r o r term decrease. Furthermore, even the s t r i c t e r c o n d i t i o n of o v e r a l l c i r c u l a r i t y i s l e s s rigorous than the c l a s s i c a l symmetry assumption. As the F t e s t i s not v a l i d i f c i r c u l a r i t y assumptions are not met, i t i s necessary to be able to t e s t f o r c i r c u l a r i t y . Huynh and F e l d t (1970) provide a t e s t for o v e r a l l c i r c u l a r i t y based on the Box t e s t (1950) and Mauchly's c r i t e r i o n W (1940). The s t a t i s t i c s c a l c u l a t e d are s i m i l a r to.those i n t e s t i n g for symmetry i n the variance-covariance matrices. Rouanet and Lepine 57 (1970) adopt a multidimensional approach i n t e s t i n g f or c i r c u l a r i t y based upon an adaptation of.Anderson's ' s p h e r i c i t y t e s t ' (1958, p.263). Although Rouanet and Lepine do not give a t e s t when there are p between-level f a c t o r s or groups, Box's (1950) t e s t could be used to t e s t C'E C=V 2I, p=1,...,P. I f the n u l l hypothesis i s not r e j e c t e d , Andersons's t e s t (1958) could subsequently be employed. Covariance Heterogeneity and Type I Err o r Rates Evidence of Type I e r r o r i n f l a t i o n . Several i n v e s t i g a t o r s (e.g., Box, 1954a,b; C o l l i e r , Baker, Mandeville & Hayes, 1967; Gaito , 1961; Geisser and Greenhouse, 1958; Lana & Lubin, 1963) have discussed the e f f e c t of covariance heterogeneity upon the Type I e r r o r r a t e . Kogan (1948) was the f i r s t to p o s t u l a t e that when the t r i a l s were p o s i t i v e l y i n t e r c o r r e l a t e d the subsequent F t e s t f or d i f f e r e n c e s i n the t r i a l s would be l i b e r a l . Box (1954b) i n v e s t i g a t e d the s i t u a t i o n where adjacent t r i a l s had c o r r e l a t i o n s equal to zero. He found that the p r o b a b i l i t y of obt a i n i n g a s i g n i f i c a n t p-value increased as the c o r r e l a t i o n s increased from 0 to ±.40. As the magnitude of the c o r r e l a t i o n increases the value of € decreases. When r=0, e=1; with l i t t l e c o r r e l a t i o n (r=.20), €=.9507 and a c o r r e l a t i o n of .40 r e s u l t e d in c e q u a l l i n g .8033. The corresponding negative c o r r e l a t i o n s r e s u l t e d i n e p s i l o n values of .9640 and .8862. Negative c o r r e l a t i o n s have l e s s of an e f f e c t on the Type I e r r o r rate than do t h e i r p o s i t i v e counterparts. Box concluded that as the value of e decreased the p r o b a b i l i t y of f a l s e l y r e j e c t i n g the n u l l hypothesis increased. Gaito (1973) c a l c u l a t e d e p s i l o n 58 values for c o r r e l a t i o n s greater than .40 for a covariance s t r u c t u r e s i m i l a r to that of Box (1954b) and found that e p s i l o n decreased q u i t e r a p i d l y as the c o r r e l a t i o n increased (e.g., r=.60, €=.5977? r=.80, e=.4009? r=.90, €=.3189). He found the Type I erro r rate increased s i m i l a r l y , with a c o r r e l a t i o n of +.90 r e s u l t i n g i n an a c t u a l p r o b a b i l i t y of making a Type I e r r o r of .16 at the .05 l e v e l of s i g n i f i c a n c e . C o l l i e r , Baker, Mandeville and Hayes (1967) stud i e d s e v e r a l very simple covariance matrices having high adjacent t r i a l c o r r e l a t i o n s with the magnitudes of the c o r r e l a t i o n s decreasing as the t r i a l s become f a r t h e r apart. Using only four t r i a l s and c o r r e l a t i o n s ranging from .80 to .20, they found the p - l e v e l s to be about twice as large as the expected .05 and three to f i v e times as large at the .01 l e v e l of s i g n i f i c a n c e . I t i s q u i t e p o s s i b l e that many studies have more than four t r i a l s and the subsequent e r r o r rate could be much higher than those reported by C o l l i e r et a l . , (1967). Schutz and G e s s a r o l i (1980) used a c o r r e l a t i o n matrix with a s i m i l a r magnitude and p a t t e r n of c o r r e l a t i o n s but had data for each of 36 t r i a l s . Their Monte Carlo study r e s u l t e d i n a Type I e r r o r rate of .17 at the .10 l e v e l , .12 at an alpha of .05 and .05 at the .01 l e v e l •- a degree of i n f l a t i o n greater than that of C o l l i e r et a_l. , (1967). This i s c o n s i s t e n t with r e s u l t s of Box (1954b) who discovered that the value of e p s i l o n decreases i n v e r s e l y with the number of t r i a l s . Wilson (1975), i n a s i m u l a t i o n study based on each "subject" having 10 t r i a l s , used an a r b i t r a r y c o r r e l a t i o n matrix with the c o r r e l a t i o n s ranging from 0 to 0.98. The Type I e r r o r 59 rate was c o n s i s t e n t with the high degree of covariance heterogeneity and moderate number of t r i a l s . At the 5% l e v e l of s i g n i f i c a n c e the a c t u a l Type I e r r o r rate was over 20% and at the 1% l e v e l i t was about 13%. T r a d i t i o n a l adjustments i n the degrees of freedom. Several methods have been suggested to deal with the problems produced by covariance heterogeneity; some are m e t h o d o l i g i c a l ; some focus on the choice of s t a t i s t i c a l t e s t , and others t r y and reduce the bias in the F r a t i o by a l t e r i n g the degrees of freedom. Greenhouse and Geisser (1959) based on the previous work of Geisser and Greenhouse (1958) and Box (1954b) proposed a three step procedure i n a n a l y z i n g repeated measures experiments. They suggested f i r s t doing a conservative F t e s t . This involves using the lower bound of e p s i l o n , 1 / ( q - l ) , where q i s the number of t r i a l s , thereby making the adjusted degrees of freedom 1 and (N- 1) d.f. for the t e s t of a t r i a l s e f f e c t . In the groups by t r i a l s design the conservative t e s t for an i n t e r a c t i o n e f f e c t would be d i s t r i b u t e d as F with 1 and p ( n - l ) degrees of freedom, where p i s the number of groups and n i s the number of subjects under each l e v e l of p. If t h i s proved s i g n i f i c a n t , the t e s t would be f i n i s h e d . I f , however, the n u l l hypothesis was not r e j e c t e d , then an F t e s t based on the conventional degrees of freedom (e=1) should be done. Here the degrees of freedom corresponding to the t e s t s for a t r i a l s e f f e c t and group by t r i a l s i n t e r a c t i o n would be (q-1 ) , (q-1 ) (n-1 ) and (q-1 •) ,p(q-1) (n-1 ) , r e s p e c t i v e l y . I f the F r a t i o i s n o n - s i g n i f i c a n t the. t e s t i n g i s f i n i s h e d . I f the s i t u a t i o n a r i s e s where the conservative t e s t proves non- 60 s i g n i f i c a n t and the conventional t e s t s i g n i f i c a n t , then an attempt must be made to estimate €. The exact value of € would give the a c t u a l d i s t r i b u t i o n of the F r a t i o . Studies using e As i s obvious from the e a r l i e r equation d e f i n i n g e p s i l o n , e can be c a l c u l a t e d only i f the population variance-covariance matrix i s known. In a c t u a l experimental data the population values are never known. Geisser and Greenhouse c a l c u l a t e d the sample estimate (e) of c i n the same manner, as the o r i g i n a l equation, with,, the population variances and covariances being s u b s t i t u t e d by the corresponding sample s t a t i s t i c s . The degrees of freedom of the c r i t i c a l F are then reduced using Z rather than €. Several studies have i n v e s t i g a t e d the e f f e c t of using e i nstead of € i n c o n t r o l l i n g for Type I e r r o r s . C o l l i e r et al., (1967) found t h a t , i n general, e was a good estimate of e p s i l o n . However, I i s a conservative estimate of e when the population value i s near one r e s u l t i n g i n a somewhat conservative t e s t of the n u l l hypothesis. The sampling d i s t r i b u t i o n of e i s n e g a t i v e l y skewed at i t s upper l i m i t but becomes l e s s v a r i a b l e and l e s s biased as the population value decreases ( C o l l i e r , Baker, Mandeville & Hayes, 1967.; Mendoza, Toothaker & Nicewander, 1974; Rogan, Keselman and Mendoza, 1979; S t o l o f f , 1970; Wilson, 1975). S t o l o f f (1970) reported data which i n d i c a t e d t h a t , as the sample s i z e i n c r e a s e s , the t e s t using e to adjust the degrees of freedom r e s u l t s i n the e m p i r i c a l Type I e r r o r rate i s c l o s e r to the nominal rate when e i s approximately one. The d i f f e r e n c e i n Type I e r r o r s using c and t. decreases as the sample s i z e increases and 61 as € decreases ( C o l l i e r et a l . , 1967; S t o l o f f , 1970). An i n t e r e s t i n g aspect of S t o l o f f s study i s how.e and i react when the number of t r i a l s increased. He found that as the t r i a l s increased, the magnitude of the Type I e r r o r s increased when t was used as the c o r r e c t i o n f a c t o r . However, when the degrees of freedom were reduced by c , the p r o b a b i l i t y of making Type I e r r o r s decreased. This was c o n s i s t e n t under varying l e v e l s of e. I t appears that the sample estimate of e p s i l o n c o n t r o l s the Type I e r r o r rate b e t t e r than the population value as the number of t r i a l s increase. As the maximum number of t r i a l s used was f i v e , f u r t h e r i n v e s t i g a t i o n should be undertaken to see how conservative the t e s t using e becomes as the l e v e l s of the repeated f a c t o r increase to a much higher degree. Modif i c a t i o n s of e j_ e and e ̂  The f a c t that the value of e p s i l o n based upon sample data i s n e g a t i v e l y biased at high l e v e l s of € caused Huynh and F e l d t (1976) to develop a new s t a t i s t i c to adjust the degrees of freedom i n the F r a t i o . This estimator, I , e l i m i n a t e s most of the negative bias i n the t e s t fo r s i g n i f i c a n c e wjien € i s used. They define I as: I = [n(k-1)€-2]/(k-1)[n-1-(k-1 ) l ] for the one-way c l a s s i f i c a t i o n with k t r i a l s and, for the groups by t r i a l s design: e = [N(k-1)€-2]/(k-1)[N-g-(k-1)e], where N i s the t o t a l number of subjects and g i s the number of groups. In the l a t t e r design, € i s c a l c u l a t e d by using the pooled estimates of the sample variance-covariance matrices f o r each of the g groups. T h i s , of course, assumes that a l l the i n d i v i d u a l population variance-covariance matrices are equal for 62 a l l the groups. Huynh (1978) deals with the case when t h i s i s not t r u e . Huynh and F e l d t (1976) note that for any values of n and k, e i s always greater than €, with t h i s d i f f e r e n c e decreasing as n increases. This formula for € allows i t to have a value greater than one when there i s a high degree of homogeneity i n the matrix. In t h i s case, the upper l i m i t i s exceeded. Therefore, e i s equated to one i f the a c t u a l c a l c u l a t i o n of e i s greater than one. Huynh and F e l d t (1976), i n a Monte C a r l o study .comparing \ and e i n c o n t r o l l i n g for Type I e r r o r s under varying l e v e l s of c (.363<c^1.000) found t h a t , i n general, i i s the b e t t e r estimator when € i s greater than 0.75 while c i s superior at higher degrees of heterogeneity. They a l s o discovered that both t e s t s behave d i f f e r e n t l y depending upon the number of groups and s u b j e c t s . They s t a t e , " I t can be seen that the t e s t based on e i s more s a t i s f a c t o r y when the parameter i s r e l a t i v e l y low or when the number of blocks or subjects i s f a i r l y l a r g e . The t e s t based on I, on the other hand, behaves very w e l l at the nominal ten or f i v e per cent l e v e l s i n a l l of the s i t u a t i o n s considered. At the nominal 2.5 and 1 percent l e v e l s i t gives somewhat more rel a x e d , but reasonably adequate, c o n t r o l over Type I e r r o r whenever the covariance matrix i s not extremely heterogeneous. This t e s t i s l e s s dependent on the number of b l o c k s , and i s f a i r l y good even with a block s i z e as small as twice the number of treatment l e v e l s . " (p. 80) GA and IGA t e s t s . Huynh (1978) extended the work of Huynh and F e l d t (1976) to consider the case when the various population matrices are heterogeneous. Two t e s t s , the General 63 Approximate t e s t (GA t e s t ) 'and the Improved General Approximate t e s t (IGA t e s t ) were developed to deal with t h i s s i t u a t i o n . The GA and IGA a l s o have the added f l e x i b i l i t y of being s u i t a b l e for t e s t s with unequal sample s i z e s . Huynh (1978), comparing a l l four t e s t s (e, c, GA and IGA) i n a s i t u a t i o n where the matrices almost e x h i b i t e d multisample s p h e r i c i t y found that the GA and e approximate t e s t s always e r r on the l i b e r a l s i d e . However, the IGA and e t e s t s y i e l d e d b e t t e r o v e r a l l c o n t r o l of the Type I e r r o r r a t e . Huynh then compared the IGA and e t e s t s under eleven d i f f e r e n t heterogeneity c o n d i t i o n s with the r e s u l t that the IGA t e s t tended to f u n c t i o n b e t t e r than the approximation, although both were s l i g h t l y l i b e r a l . However, most d i f f e r e n c e s were at smaller l e v e l s of s i g n i f i c a n c e or when the sample s i z e s were q u i t e large (N=30). Huynh concludes that although the IGA t e s t i s more accurate and f l e x i b l e , i t i s computationally more complex and, i n many s i t u a t i o n s , the e approximate procedure f u n c t i o n s as w e l l as the IGA t e s t and, therefore i s more d e s i r a b l e . M u l t i v a r i a t e techniques. An a l t e r n a t i v e to the various c o r r e c t i o n techniques a p p l i e d when repeated measures data i s anlayzed by an a n a l y s i s of variance i s a m u l t i v a r i a t e a n a l y s i s of variance (MANOVA). M u l t i v a r i a t e a n a l y s i s of v a r i a n c e , which req u i r e s no assumptions of within-group variance or covariance homogeneity, has been fre q u e n t l y recommended as the appropriate technique for a l l repeated measures designs (Davidson, 1972; Morrow & Frankiewicz, 1979; Schutz, 1978). Among the basic assumptions i n m u l t i v a r i a t e a n a l y s i s of variance are: (a) the data are d i s t r i b u t e d as m u l t i v a r i a t e 64 normal, and ..'(b) the group covariance matrices all-come from, a s i n g l e population covariance matrix. However, while MANOVA has l e s s s t r i n g e n t assumptions, v i o l a t i o n s may have serious consequences on the Type I e r r o r r a t e . The e f f e c t s of v i o l a t i n g the assumption of normality are g e n e r a l l y not severe. Mardia (1971) and I t o (1969) found that the m u l t i v a r i a t e t e s t s are q u i t e robust to departures from m u l t i v a r i a t e normality, e s p e c i a l l y i f the sample s i z e s are equal. Studies i n v e s t i g a t i n g the assumption of equal covariance matrices between groups found that the Type I e r r o r rate i s c o n t r o l l e d under moderate degrees of heterogeneity i f the sample s i z e s are equal (Holloway & Dunn, 1967; Hakstian, Roed & L i n d , 1979; I t o fc.Schull, 1964; Rogan, Keselman & Mendoza, 1979). Holloway and Dunn, however, found that sample s i z e e q u a l i t y does not n e c e s s a r i l y ensure c o n t r o l of the number of Type I e r r o r s committed as the r a t i o of the sample s i z e to the number of dependent v a r i a b l e s and the degree of covariance heterogeneity are a l s o important. Using a r a t i o of 10:1 between the variances i n the two covariance matrices, Holloway and Dunn discovered that equal sample s i z e s of 25 were s u f f i c i e n t when only two or three v a r i a t e s were used but, for 10 v a r i a t e s , the m u l t i v a r i a t e t e s t , H o t e l l i n g ' s T 2, was not robust u n t i l the sample reached 100. In r e l a t i n g these r e s u l t s to a c t u a l b e h a v i o r a l data, i t must be remembered that a r e a l i s t i c extreme for the r a t i o between population variances i s only 2.5 (Hakstian, Roed & L i n d , 1979). Hakstian et a l . , (1979), using variance scale f a c t o r s up to 2.5 showed that the T 2 procedure was r e l a t i v e l y robust to v i o l a t i o n s i n the covariance assumption, even when the r a t i o 65 between subjects and dependent v a r i a b l e s was as low as 3:1. While the t e s t of main e f f e c t s appear to be r e l a t i v e l y robust, other m u l t i v a r i a t e procedures t e s t i n g for s i g n i f i c a n t i n t e r a c t i o n s d i d not show the same r e s u l t s . In studying the e f f e c t s of covariance heterogeneity (with equal sample s i z e , r a t i o of subjects to v a r i a t e s approximately 4:1) on the t e s t s for s i g n i f i c a n t i n t e r a c t i o n s , Rogan, Keselman and Mendoza (1979) discovered an i n f l a t i o n i n the number of Type I e r r o r s . These increases were s l i g h t for the P i l l a i - B a r t l e t t trace c r i t e r i o n , and Wilk's l i k e l i h o o d r a t i o c r i t e r i o n , but were much l a r g e r (as high as .070 at alpha equal to .05) when Roy's l a r g e s t root c r i t e r i o n was used. When unequal sample s i z e s e x i s t , the Type I . e r r o r rate f l u c t u a t e s g r e a t l y , with the Type I e r r o r r a t e s i n c r e a s i n g q u i c k l y to very unacceptable l e v e l s as the degree of heterogeneity increases, even at small sample s i z e r a t i o s as low as 2:1. In the most extreme case s t u d i e d , with 10 v a r i a t e s , 50 subjects i n one group compared to 10 i n the other, and the variances in one group scaled at 2.5 times the magnitude Of the other group, the Type I e r r o r rates were: for O=.01, .152; for a=.05, .337 and; for C=.10, .473 (Hakstian et a l . , 1979). C l e a r l y , as the authors point out, "the T 2 procedure i s not robust i n the face of covariance matrix heterogeneity coupled with unequal n's, even for r e l a t i v e l y minor departures from e q u a l i t y of the covariance matrices, sample s i z e s or both." (p. 1261) Overview of u n i v a r i a t e vs m u l t i v a r i a t e t e s t s on power. In general, when the u n i v a r i a t e assumptions regarding the 66 covariance matrices are met, the conventional u n i v a r i a t e ANOVA i s more powerful than m u l t i v a r i a t e techniques (Mendoza, Toothaker & Nicewander, 1974; Rogan et a l . , 1979). Of i n t e r e s t i s the comparison between the power of the adjusted u n i v a r i a t e t e s t s (e.g., £, e), the conventional u n i v a r i a t e t e s t and m u l t i v a r i a t e t e s t s under various l e v e l s of within-group and between-group covariance matrix heterogeneity. When a l l covariance assumptions are met the conventional u n i v a r i a t e t e s t i s more powerful than e i t h e r the adjusted u n i v a r i a t e t e s t s or the m u l t i v a r i a t e t e s t s . However, as the degree of within-group matrix heterogeneity increases the m u l t i v a r i a t e t e s t s become more powerful i n d e t e c t i n g s i g n i f i c a n c e for d i f f e r e n c e s i n the main e f f e c t s . Rogan e_t a l . , (1979), found that as the value of £ decreased the power of a l l the t e s t s decreased, but the m u l t i v a r i a t e t e s t s decreased at a slower r a t e . As the degree of covariance heterogeneity increases the power of the adjusted u n i v a r i a t e t e s t s are of concern since they are the t e s t of s i g n i f i c a n c e . I t appears that when e p s i l o n dips below .75 the m u l t i v a r i a t e t e s t s more often detect the d i f f e r e n c e s i n the means (Mendoza et a l . , 1974; Rogan et a l . , 1979). When €^.75 the adjusted u n i v a r i a t e t e s t s are more powerful than t h e i r m u l t i v a r i a t e counterparts. Mendoza et a l . , (1974), found that the power of d e t e c t i n g small i n t e r a c t i o n s was greatest for Roy's l a r g e s t root c r i t e r i o n but i n d e t e c t i n g large d i f f e r e n c e s , the adjusted u n i v a r i a t e t e s t s were more powerful (for e<.75). Rogan et a l . , examined the power of three m u l t i v a r i a t e t e s t s for i n t e r a c t i o n and reported s i m i l a r r e s u l t s as i n the t e s t for main e f f e c t s , that being that 67 the m u l t i v a r i a t e t e s t s were more powerful that the u n i v a r i a t e t e s t s . I t should be noted, however, that Roy's l a r g e s t root c r i t e r i o n had the greatest Type I e r r o r rate under covariance heterogeneity and, caution must be employed i f i t i s to be used. Summarizing, i f a l l the covariance assumptions are met, the conventional u n i v a r i a t e t e s t i s the best to use in t e s t i n g for both i n t e r a c t i o n s or main e f f e c t s . With moderate l e v e l s of heterogeneity i n the covariance matrices (e^.75) the adjusted u n i v a r i a t e t e s t s are best and, g e n e r a l l y , when €<.75 the m u l t i v a r i a t e t e s t s are the most powerful. Summary. When de a l i n g with data which e x h i b i t s heterogeneity of covariances (as i s common i n repeated measures behavioral data) the e a s i e s t , and often s u f f i c i e n t method of c o r r e c t i n g f or t h i s heterogeneity i s to use the three-step procedure as o u t l i n e d by Geisser and Greenhouse (1959). However, i f a sample estimate of e need be c a l c u l a t e d to adjust the degrees of freedom there are s e v e r a l choices. I f t i s l e s s than .75 the best u n i v a r i a t e s t a t i s t i c i s e , but i f e p s i l o n i s greater than .75 e i t h e r i or IGA approximate t e s t s are the most powerful yet c o n t r o l f or the Type I e r r o r r a t e . Of the l a t t e r two, the e i s much e a s i e r to c a l c u l a t e and i s q u i t e often as good i n c o n t r o l l i n g f o r Type I e r r o r s as the IGA t e s t . M u l t i v a r i a t e t e s t s do not depend upon the assumption of w i t h i n - group covariance homogeneity and, as such, may often be the pr e f e r r e d method of a n a l y s i s . They prove to be more powerful than t h e i r u n i v a r i a t e counterparts when e<.75 but are weaker above t h i s l e v e l . M u l t i v a r i a t e t e s t s , however, do require that the covariance matrices between groups come from a common 68 population matrix, an assumption which may not be often s a t i s f i e d i n motor behavior research. AE-CE-VE Debate A considerable controversy has developed i n the past ten years regarding which s t a t i s t i c s (AE, CE or VE) should be used as mesures of a subject's performance on some motor performance task. The debate has been p r i m a r i l y between those researchers who are concerned with the s t a t i s t i c a l and mathematical p r o p e r t i e s of AE, CE and VE and those i n v e s t i g a t o r s who are more i n t e r e s t e d i n the conceptual i n t e r p r e t a t i o n of these scores. An e x c e l l e n t review of t h i s debate i s given by Schutz (1979). While seve r a l researchers had p r e v i o u s l y commented on the appropriateness of these performance measures (Burdick, 1972; Laabs, 1973; Schmidt, 1970; Underwood, 1957; Woodworth, 1938) the problem received s e r i o u s c o n s i d e r a t i o n a f t e r a paper by Schutz and Roy (1973) proved mathematically that AE i s d i r e c t l y r e l a t e d to CE and VE and, as such, can only be i n t e r p r e t e d i n l i g h t of the l a t t e r two measures. They stated that a l l the information of AE i s found i n CE when the r a t i o of CE//VE i s greater than 2.0 or i s i n VE when CE i s approximately equal to zero. AE i s a weighted combination of CE and VE when 0<CE//VE<2.0. As the mathematical .derivations, discussed above, were based on the assumption that the raw performance scores are normally d i s t r i b u t e d , the v a l i d i t y of t h e i r conclusions decreases as the departure from normality increases. The use of V a r i a b l e e r r o r as the optimal measure of w i t h i n - subject v a r i a b l i t y was questioned by Burdick (1972) and Schutz, Roy and Goodman (1973) because i t d i d not r e f l e c t the temporal 69 dimension of performance e r r o r s . An a l t e r n a t e choice of measures such as the Mean Square Successive D i f f e r e n c e , the A u t o c o r r e l a t i o n and the C o e f f i c i e n t of Temporal V a r i a b i l i t y have been suggested by Burdick (1972). Schutz et a l . , (1973) suggested that the non-normal d i s t r i b u t i o n of a variance r e s u l t s i n a l o s s of power when VE i s analyzed by an ANOVA and i n d i c a t e d that the a u t o c o r r e l a t i o n c o e f f i c i e n t be used as an a d d i t i o n a l measure of i n t r a - s u b j e c t v a r i a b l i t y . S a f r i t , Spray and Diewert (1980), in examining the t h e o r e t i c a l d i s t r i b u t i o n of VE, stated that VE may not be normally d i s t r i b u t e d , but f a i l e d to conclude that the d i s t r i b u t i o n was d e f i n i t e l y non-normal. One of the purposes of t h i s study i s to determine i f the d i s t r i b u t i o n of VE scores c a l c u l a t e d from a c t u a l raw scores i s non-normally d i s t r i b u t e d . I f the e m p i r i c a l d i s t r i b u t i o n i s normal, many of the concerns of Schutz et a l . , (1973) and S a f r i t et a l . , (1980) w i l l not be v i t a l i n an a l y z i n g VE data by an ANOVA. Henry (1974) agreed with Schutz and Roy (1973) on the inadequacy of AE. While s t a t i n g that CE and VE must always be looked at when i n t e r p r e t i n g performance e r r o r , he s a i d t h a t , at times, i t may be necessary to use a composite score. Henry suggested using E 2 (where E 2=CE 2+VE 2) to which Schutz (1974) r e p l i e d that E 2 i s s t i l l a composite score and must be i n t e r p r e t e d from CE and VE scores. Henry (1975), using m u l t i p l e c o r r e l a t i o n s , showed that E 2 was bet t e r than AE since the e f f e c t of VE i s never excluded i n E 2 while i t may be i n AE (when CE//VE>2.0). Schutz (1979) conceded t h a t , i f a composite measure had to be used, then E 2 i s p r e f e r a b l e to AE but i t s t i l l must be i n t e r p r e t e d with respect to CE and VE. 70 Jones (1974) suggested that AE, not VE i s the appropriate e r r o r score when the c r i t e r i o n i s changed for each t r i a l of a s i m i l a r task. Roy (1974), r e p l i e d that since KR i s not given on every t r i a l , the t y p i c a l movement reproduction experiment i s not a l e a r n i n g s i t u a t i o n but a f o r g e t t i n g one. Roy argued that VE i s a measure of f o r g e t t i n g and lack of consistency i n performance which does not require the c r i t e r i o n , for each t r i a l to be s i m i l a r i n order to be i n t e r p r e t e d . Schmidt (1975) favored the use of AE c l a i m i n g that for motor r e c a l l s t u d i e s i t i s the p r e f e r a b l e dependent measure for the f o l l o w i n g reasons: (a) the use of two dependent v a r i a b l e s (CE and VE) may y i e l d d i f f e r e n t r e s u l t s , thereby confusing any i n t e r p r e t a t i o n of r e s u l t s ; (b) AE i s the t r a d i t i o n a l measure, and (c) since the subject i s required to minimize h i s e r r o r on each t r i a l , AE i s what should be measured. Schutz (1979) responded to each of these arguements, r e s p e c t i v e l y , as such: (a) any theory should s a t i s f y both performance dimensions as suggested by the CE and VE scores; (b) the fa c t that AE has been the t r a d i t i o n a l measure i s s u f f i c i e n t reason to continue using i t ; and (c) since the purpose of the researcher i s to e x p l a i n performance, not only to measure i t , CE and VE must be used i n the i n t e r p r e t a t i o n . In 1976, Newell stated that when one h a l f of the subjects have p o s i t i v e CE's while the other h a l f have negative CE's, the use of an average CE i s inap p r o p r i a t e and AE should be used. In t h i s s i t u a t i o n Schutz (1979) agreed with Henry (1975) i n that the absolute value of CE, |CE|, i s the best measure. The AE-CE-VE controversy then s h i f t e d from the t h e o r e t i c a l 71 i n t e r p r e t a t i o n s of these measures to more s t a t i s t i c a l ones.Roy (1976) sta t e d that a good method of re p o r t i n g a l l three e r r o r terms (AE or E, CE and VE) i n studie s i s to analyze a l l three measures by a MANOVA since i t c o n t r o l s f or the Type I e r r o r r a t e . Roy provided a footnote which i n d i c a t e d t h a t , based on work by Schutz and Roy, AE may be a l i n e a r composite of CE and VE and, t h e r e f o r e , a MANOVA could not be c a l c u l a t e d . However, he state d that t h i s r a r e l y occurs across a l l s u b j e c t s . Thomas (1977) r e p l i e d that even though an absolute l i n e a r r e l a t i o n s h i p between the three dependent v a r i a b l e s may not e x i s t , the problem of m u l t i c o l l i n e a r i t y does. M u l t i c o l l i n e a r i t y has the e f f e c t of in c r e a s i n g the Type I e r r o r rate (Press, 1972). Thomas suggested an a l y z i n g VE and CE with a MANOVA and doing a separate ANOVA f o r AE or E. In r e p l y i n g to Thomas (1977), Roy (1977) agreed with the concept of m u l t i c o l l i n e a r i t y but fu r t h e r complicated the issue by i n d i c a t i n g that a high c o r r e l a t i o n may e x i s t between CE and VE, thereby making a t e s t of these v a r i a b l e s by a MANOVA subject to the e f f e c t s of m u l t i c o l l i n e a r i t y . S a f r i t , Spray and Diewert (1980) caution against the use of a l l AE, CE and VE i n a MANOVA for d i f f e r e n t reasons. An assumption i n MANOVA designs i s that the j o i n t p r o b a b i l i t y vector of the random vector be m u l t i v a r i a t e l y normally d i s t r i b u t e d . S a f r i t et §_1. , showed that CE i s m a r g i n a l l y normal, but both VE and AE may be ma r g i n a l l y non-normal, and concluded that u n t i l future e m p i r i c a l work shows that the v i o l a t i o n s of these assumptions are not s e r i o u s , a n a l y z i n g AE, CE and VE by a MANOVA should be avoided. E a r l i e r work, however, has shown that the T 2 procedure i s r e l a t i v e l y robust to m u l t i v a r i a t e non-normality (Mardia, 1971). 72 E a r l i e r , Thomas and Moon (1976) found AE scores to have higher r e l i a b i l i t i e s than VE and a greater number of s i g n i f i c a n t d i f f e r e n c e s were obtained with AE. These f a c t s along with t h e i r f i n d i n g that AE appeared.to be more normally d i s t r i b u t e d about the target than VE allowed them to conclude that AE i s the best dependent measure when conducting motor rhythm experiments. S a f r i t e_t a l . , (1980) i n s t a t i n g that the d i s t r i b u t i o n s of AE and VE may be non-normal caution i n v e s t i g a t o r s i n an a l y z i n g these dependent measures by an ANOVA. However, the v i o l a t i o n of the normality assumption by i t s e l f i s not serious (Boneau, 1960), but when i n t e r a c t i v e with v i o l a t i o n s of other assumptions, the Type I e r r o r rate i s a f f e c t e d (Bradley, 1980). Therefore, i f the researcher h e s i t a t e s i n using an ANOVA due s o l e l y to non-normality, he should check the other assumptions to see i f they are s a t i s f i e d . While the area of which dependent measure i s proper to use and report i s obviously confusing, the f o l l o w i n g r u l e of thumb i s g e n e r a l l y accepted. Any i n v e s t i g a t o r who can provide a l o g i c a l explanation as to what information AE provides i s j u s t i f i e d i n r e p o r t i n g i t ( S a f r i t et a l . , 1980). Summary. As the wealth of l i t e r a t u r e has i n d i c a t e d , the choice of the dependent measure to be analyzed and i n t e r p r e t e d i s a subject of great controversy. Much of the debate deals with the conceptual i n t e r p r e t a t i o n of these measures, and thus i s out of the range of the s t a t i s t i c i a n , but a great deal of uncer t a i n t y surrounds the d i s t r i b u t i o n s and e f f e c t s of using these dependent v a r i a b l e s i n an a n a l y s i s of var i a n c e . Although many of the present problems w i l l s t i l l e x i s t , h o p e f u l l y , the 73 question of the e m p i r i c a l d i s t r i b u t i o n of VE and i t s subsequent e f f e c t on the Type I e r r o r rate w i l l be adequately resolved at the c o n c l u s i o n of t h i s study. 74 REFERENCES Anderson, T. W. An In t r o d u c t i o n to M u l t i v a r i a t e A n a l y s i s . New York: Wiley, 1958. Boneau, C. A. The e f f e c t s of v i o l a t i o n s of. assumptions underlying the t t e s t . P s y c h o l o g i c a l B u l l e t i n , 1960, 57, 49- "64. Box, G. E. P. Problems i n the a n a l y s i s of growth and wear curves. B i o m e t r i c s , 1950, 6, 362-389. Box, G. E. P. Some theorems on quadratic forms a p p l i e d in the study of a n a l y s i s of variance problems. I . E f f e c t of i n e q u a l i t y of variance i n the one-way c l a s s i f i c a t i o n . Annals of Mathematical S t a t i s t i c s , 1954a, 25, 290-302. Box, G. E. P. Some theorems on quadratic forms a p p l i e d i n the study of a n a l y s i s of variance problems, I I . E f f e c t s of i n e q u a l i t y of variance and of c o r r e l a t i o n between e r r o r s i n the two-way c l a s s i f i c a t i o n . Annals of Mathematical S t a t i s t i c s , 1954b, 25j_ 484-498. Bradley, J . V. Nonrobustness in Z, t , and F t e s t s at large sample s i z e s . B u l l e t i n of the Psychonomic S o c i e t y , 1980, 16, 333-336. Burdick, J . A. Measurements of " V a r i a b i l i t y " . Journal of General Psychology, 1972, 86, 201-206. C o l l i e r , R. 0., Baker, F. B., Mande v i l l e , G. R., & Hayes, T. F. Estimates of t e s t s i z e for s e v e r a l t e s t procedures based on conventional variance r a t i o s i n the repeated measures design. Psychometrica, 1967, 32j_ 339-353. Davidson, M. L. U n i v a r i a t e versus m u l t i v a r i a t e t e s t s i n repeated measures experiments. P s y c h o l o g i c a l B u l l e t i n , 1972, 77, 446-452. Gait o , J . Repeated measurements designs and counterbalancing. P s y c h o l o g i c a l B u l l e t i n , 1961, 58, 46-54. Ga i t o , J . Repeated measurements .designs and t e s t s of n u l l hypotheses. Educational and P s y c h o l o g i c a l Measurement, 1973, 33, 69-75. 75 G a i t o , J . & Turner, E. D. Error terms i n trend a n a l y s i s . P s y c h o l o g i c a l B u l l e t i n , 1963, 60, 464-474. Gei s s e r , S. M u l t i v a r i a t e a n a l y s i s of variance for a s p e c i a l covariance case. Journal of the American S t a t i s t i c a l A s s o c i a t i o n , 1963, 58, 600-669. Ge i s s e r , S. & Greenhouse, S. W. An extension of Box's r e s u l t s on the use of the F d i s t r i b u t i o n , i n m u l t i v a r i a t e a n a l y s i s . Annals of Mathematical S t a t i s t i c s , 1958, 29, 855-891. Gnanadesikan, R. Methods for s t a t i s t i c a l data a n a l y s i s of m u l t i v a r i a t e observations. New York: Wiley, 1977. Goodman, D. & Schutz, R. W. DATASNIFF: A program to check data and compute summary d e s c r i p t i v e s t a t i s t i c s and c o r r e l a t i o n s . Developed at the P h y s i c a l Education Q u a n t i f i c a t i o n Laboratory, U n i v e r s i t y of B r i t i s h Columbia, 1975. Greenwald, A. G. Within-subjects designs: To use or not to use? P s y c h o l o g i c a l B u l l e t i n , 1976, 83, 314-320. Greenhouse, S. W. & G e i s s e r , S. On methods i n the a n a l y s i s of p r o f i l e data. Psychometrika, 1959, 24, 95-112. Hakstian, A. R., Roed, J . C. & L i n d , J . C. Two-sample T 2 procedure and the assumption of homogeneous covariance matrices. P s y c h o l o g i c a l B u l l e t i n , 1979, 86, 1255-1263. Halm, J . UBC NORMAL. U n i v e r s i t y of B r i t i s h Columbia Computing Center, 1970. Henry, F. M. V a r i a b l e and constant performance e r r o r s w i t h i n a group of i n d i v i d u a l s . Journal of Motor Behavior, 1974, 6, 149- 154. Henry, F. M. Absolute e r r o r vs. "E" i n target accuracy. Journal of Motor Behavior, 1975, 7, 227-228. Holloway, L. N. & Dunn, 0. J . The robustness of H o t e l l i n g ' s T 2. Journal of the American S t a t i s t i c a l A s s o c i a t i o n , 1967, 62, 124-136. Hsu, P. L. C o n t r i b u t i o n s to the theory of Student's t - t e s t as 76 a p p l i e d to the problem of two samples. S t a t i s t i c a l Research Methods, 1938, 2, 1-24. Huynh, H. Some approximate t e s t s for. repeated measurements designs. Psychometrika,.1978, 48, 161-175. Huynh, H. & F e l d t , L. Conditions under which mean square r a t i o s i n repeated measurements designs have exact F - d i s t r i b u t i o n s . Journal of the American S t a t i s t i c a l A s s o c i a t i o n , 1970, 65, 1582-1585. Huynh, H. & F e l d t , L. S. Est i m a t i o n of the Box c o r r e c t i o n for degrees of freedom from sample data i n randomized block and s p l i t - p l o t designs. Journal of Educational S t a t i s t i c s , 1976, 1, 69-82. I t o , K. On the e f f e c t of h e t e r o s c e d a s t i c i t y and non-normality upon some . m u l t i v a r i a t e t e s t procedures. In P. R. Krishn a i a h (Ed.), M u l t i v a r i a t e a n a l y s i s - I I . New York: Academic Press, 1969. I t o , K. & S c h u l l , W. J . On the robustness of the T 2 t e s t i n m u l t i v a r i a t e a n a l y s i s of variance when variance-covariance matrices are not equal. Biometr i k a , 1964, 5J_, 71-82. Jones, B. "What i s the best measure of accuracy of movement d u p l i c a t i o n ? " Unpublished paper, U n i v e r s i t y of Waterloo, 1974. Kogan, L. S. A n a l y s i s of va r i a n c e : Repeated measurements. P s y c h o l o g i c a l B u l l e t i n , 1948, 45, 131-143. Laabs, G. J . Retention c h a r a c t e r i s t i c s of d i f f e r e n t reproduction cues i n motor short-term memory. Journal of Experimental Psychology, 1973, 100, 168-177. Lana, R. E., & Lubin, A. The e f f e c t of c o r r e l a t i o n on the repeated measures design. Educational and P s y c h o l o g i c a l Measurement, 1963, 23j_ 729-739. Mardia, K. V. The e f f e c t of nonnormality on some m u l t i v a r i a t e t e s t s and robustness to nonnormality i n the l i n e a r model. Biometrika, 1971, 58, 105-121. Mauchly, J . W. S i g n i f i c a n c e t e s t for s p h e r i c i t y of a normal n- v a r i a t e d i s t r i b u t i o n . The Annals of Mathematical S t a t i s t i c s , 77 1940, 29, 204-209. Mendoza, J . L., Toothaker, L.E. & Nicewander, W.A. A Monte Carlo comparison of the u n i v a r i a t e and . m u l t i v a r i a t e methods for the groups by repeated measures design. M u l t i v a r i a t e Behavioral Research, 1974, 9, 165-178. Morrow, J . R. & Frankiewicz, R. G. S t r a t e g i e s for the a n a l y s i s of repeated and m u l t i p l e measures designs. Research Q u a r t e r l y , 1979, 50, 297-304. Newell, K. M. More on absolute e r r o r , e t c . Journal of Motor Behavior, 1976, 8^ 139-142. Press, S. J . Applied m u l t i v a r i a t e a n a l y s i s . New York: H o l t , Rinehart, and Winston, 1972. Rogan, J . C. , Keselman, H. J . & Mendoza, J . L. A n a l y s i s of repeated measurements. B r i t i s h Journal of Mathematical and S t a t i s t i c a l Psychology, 1979, 32, 269-286. Rouanet, H. & Lepine, D. Comparison between treatments i n a repeated-measures design: ANOVA and m u l t i v a r i a t e methods. The B r i t i s h Journal of Mathematical and S t a t i s t i c a l Psychology, 1970, 23j_147-163. Roy, E. A. "Measures of accuracy of movement d u p l i c a t i o n - A reply to Jones." Unpublished paper, U n i v e r s i t y of Waterloo, 1974. Roy, E. A. Measuring change i n motor memory. Journal of Motor Behavior, 1976, 8, 283-287. Roy, E. A. Measuring change: A r e p l y to Thomas. Journal of Motor Behavior, 1977, 9, 255-256. Schmidt, R. A. Critque of Henry's paper. In L. E. Smith (Ed.), Psychology of motor l e a r n i n g . Chicago: A t h l e t i c I n s t i t u t e , 1 970. Schmidt, R. A. A schema theory of d i s c r e t e motor l e a r n i n g . P s y c h o l o g i c a l Review, 1975, 82, 225-260. S a f r i t , M. J . , Spray, A. J . & Diewert, G. L. Methodological 78 issues i n short-term motor memory research. Journal of Motor Behavior, 1980, 12, 13-28. Scheffe, H. The A n a l y s i s of Variance. New York: Wiley, 1959. Schutz, R. W. Absolute e r r o r . J o u r n a l of Motor Behavior, 1974, 6, 299-301. Schutz, R. W. S p e c i f i c problems i n the measurement of change: L o n g i t u d i n a l s t u d i e s , d i f f e r e n c e scores, and m u l t i v a r i a t e analyses. In D. Landers & R. C h r i s t i n a (Eds.), Psychology of Motor Behavior and Sport - 1977, Champaign, I l l i n o i s : Human K i n e t i c s , 1978. Schutz, R. W. Absolute, Constant, and V a r i a b l e E r r o r : Problems and s o l u t i o n s . In D. Mood (Ed.), Proceedings of the Colorado Measurement Symposium. U n i v e r s i t y of Colorado Press: Boulder, Colorado, 1979, 82-100. Schutz, R. W. & G e s s a r o l i , M. E. The e f f e c t s of block s i z e and heterogeneity of covariance on Type I e r r o r rates with Constant e r r o r and V a r i a b l e e r r o r data, i n Psychology of Motor Behavior and Sport ^ 1979. Champaign, I l l i n o i s : Human K i n e t i c s , 1980, 633-642. Schutz, R. W., Goodman, D. & Roy, E. A. "More on e r r o r . " Paper presented at the F i r s t Canadian Congress for the M u l t i d i s c i p l i n a r y Study of Sport and P h y s i c a l A c t i v i t y , Montreal, Quebec, October, 1973. Schutz, R. W., & Roy, E. A. Absolute e r r o r : The d e v i l i n d i s g u i s e . Journal of Motor Behavior/ 1973, 5j_ 141-153. S t o l o f f , P. H. C o r r e c t i n g for heterogeneity of covariance for repeated measures designs of the a n a l y s i s of variance. Educational and P s y c h o l o g i c a l Measurement, 1970, 3_0, 909-924. Thomas, J . R. & Moon, D. H. Measuring motor rhythmic a b i l i t y . Research Q u a r t e r l y , 1976, 47, 20-32. Thomas, J . R. A note concerning a n a l y s i s of e r r o r scores from motor memory research. Journal of Motor Behavior, 1977, 9, 251-253. Wilson, A. & Lange, D. E. U n i v a r i a t e a n a l y s i s of variance for 79 repeated measures. The Journa l of Experimental Education, 1972, 40, 83-85. Wilson, K. The sampling d i s t r i b u t i o n s of c o n v e n t i o n a l , conservative and c o r r e c t e d F - r a t i o s i n repeated measurement designs with heterogeneity of covariance. J o u r n a l of S t a t i s t i c a l Computing and S i m u l a t i o n , 1975, 3_, 201-215. Woodworth, R. S. Experimental psychology. New York: H o l t , 1938.

Cite

Citation Scheme:

    

Usage Statistics

Country Views Downloads
United States 3 0
Germany 2 0
China 1 23
Australia 1 0
Japan 1 0
City Views Downloads
Unknown 3 0
Ashburn 2 0
Beijing 1 0
Las Vegas 1 0
Tokyo 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}
Download Stats

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0077122/manifest

Comment

Related Items