Open Collections

Characteristics of variable error and their effects on the type I error rate

University of British Columbia

A common practice in motor behavior research is to analyze Variable Error data with a repeated measures analysis of variance. The purpose of this study was to examine the degree to which blocked (VE) data satisfies the assumptions underlying a repeated measures ANOVA. Of particular interest was whether the assumption of covariance homogeneity - both within and between experimental groups - is satisfied in actual experimental data. Monte Carlo procedures were used to study the effect of varying degrees of violations of these assumptions on the Type I error rate. The means and ranges of the correlation matrices of eight experimental data sets were studied for both raw and VE scores based upon different block sizes. In every situation where the experimental groups were comprised of feedback and no feedback conditions, the correlation matrix for the no feedback group displayed correlations of greater magnitudes and consistency relative to those of the feedback condition. The next phase involved using the underlying variance-covariance matrices for three of these data sets to simulate raw and VE data based on various block sizes. Raw data were simulated for each of four covariance heterogeneity conditions: (1) equality within and between the variance-covariance matrices; (2) inequality within the matrices but equality between the matrices; (3) equality within each variance-covariance matrix but inequality between the matrices; (4) inequality both within and between the two variance-covariance matrices. Populations of 10,000 subjects for each of two groups, the underlying variance-covariance matrices being dependent upon the homogeneity of covariance condition being studied, were generated based on each of three actual experimental data sets. The data were blocked in various ways depending on the original number of trials in the experiment (36, 24 or 18) with VE being the dependent variable. An experiment consisted of randomly selecting 20 subjects for each of the two groups, blocking the trials based on specific block sizes and analyzing the raw and VE data by a repeated measures ANOVA. The effect of interest was the Groups by Blocks interaction. The complete process was replicated for the four covariance homogeneity conditions for each of the three data sets, resulting in a total of 22,000 simulated experiments. Results indicated that the Type I error rate increases as the degree of heterogeneity within the variance-covariance matrices increases when raw (unblocked) data is analyzed. With VE, the effects of within-matrix heterogeneity on the Type I error rate are inconclusive. However, block size does seem to affect the probability of obtaining a significant interaction, but the nature of this relationship is not clear as there does not appear to be any consistent relationship between the size of the block and the probability of obtaining significance. For both raw and VE data there was no inflation in the number of Type I errors when the covariances within a given matrix were homogeneous, regardless of the differences between the group variance-covariance matrices.

Text

2010-03-29

For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.

http://hdl.handle.net/2429/22845

Physical Education

University of British Columbia

Graduate