Open Collections

On the impact of negatively keyed items on the assessment of the unidimensionality of psychological tests and measures

University of British Columbia

Evidence of test dimensionality supports test scoring, and it is essential to construct validity. Yet many issues remain unclear in assessing dimensionality, especially when the response data are collected through self-report Likert-type tests that include negatively keyed items. The emergence of additional factors that can be attributed to the mixed-keyed format is an issue that draws much attention in investigating the dimensionality of tests that are designed to be unidimensional. Common methods for assessing dimensionality can be categorized into two types: exploratory and confirmatory. Exploratory studies use many rules and criteria, such as the eigenvalues-greater-than-one (K-G) rule and parallel analysis (PA), along with exploratory factor analysis (EFA) to help researchers determine the number of factors (i.e., dimensions). Confirmatory factor analysis (CFA), on the other hand, is often employed to examine the fit of a hypothesized measurement model. A large number of fit indices, including the Chi-square test, the comparative fit index (CFI), and the root mean square error of approximation (RMSEA), have been proposed to evaluate a model’s overall fit. This dissertation investigated, via computer simulation, how these various procedures performed in assessing the dimensionality of item response data collected using tests with negatively keyed items. Factors in the simulation experiment included psychometric models (i.e., the simulation methods) of negatively keyed items, the number of negatively keyed items, the magnitude of item communality, the distribution of observed item response, the scoring methods of negatively keyed items, and the methods and rules used for the statistical judgment of dimensionality. This dissertation adopts the threshold model of item responses, which assumes a monotonic relationship among the latent variable, item thresholds, and observed item responses. The results indicate that the dimensionality of tests with mixed-keyed items is always correctly identified when the observed item response distribution is symmetric. When it is asymmetric, however, the methods and decision rules used in dimensionality assessment affect the statistical judgment of test dimensionality. The results highlight the benefit of using categorical data analytic methods in dealing with item responses obtained through Likert-type rating scales. Guidelines are provided to inform researchers when assessing the dimensionality of mixed-keyed tests.

Text

2017-10-23

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/63393

Measurement, Evaluation and Research Methodology

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/