Open Collections

Violations of unidimensionality and local independence in measures intended as unidimensional : assessing levels of violations and the accuracy in unidimensional IRT model estimates

University of British Columbia

The current study was motivated by psychological measures intended as unidimensional, in which violations of unidimensionality and local independence (LI) are present. Unidimensionality and LI are the assumptions of unidimensional IRT models, widely used in educational assessment and increasingly used in other fields of psychology and in social and health sciences. Providing more details about the relation between different levels of violations of the two assumptions and the accuracy in unidimensional IRT model estimates was the main goal of the current study. The second goal of the study was to investigate the utility of certain ways to provide recommendations and guidelines about the size of distortions in unidimensional model estimates at different levels of violations. Four indexes based on eigenvalues from exploratory factor analysis were examined for such a purpose. Deemed beneficial for the main purpose of the study, a multidimensional model consistent with a “locally dependent unidimensional model” and a particular research design, based on varying the strength of the relevant latent dimensions, were employed in the study to create conditions of violations of the two assumptions in measures intended as unidimensional. The results of the study demonstrated robustness of the unidimensional IRT model (i.e., 2PL IRT model) under a range of violations and provided more information about the conditions of robustness. A strong relation was demonstrated between the violations, as defined by the strength of local dependence (LD), and the size of the distortions in the item parameters estimation (e.g., correlation of 0.90 with bias in the item discrimination estimates). The item discrimination parameter was systematically overestimated. In relation to the person location parameter, overestimation at the low end of the latent trait and underestimation at the high end of the trait was found – with bias systematically increasing with decrease in the strength of the dominant dimension and with increase in the strength of LD. Regarding the use of eigenvalue-based indexes in predicting bias, mixed results were obtained (no value was demonstrated in relation to item parameters estimation and some value in relation to person location estimation). The utility of the model/design employed in the study was discussed.

Text

2019-07-30

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/71155

Measurement, Evaluation and Research Methodology

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/