Open Collections

Comparing Likert-type and forced-choice formats for assessing preference : a validation study on the College Major Preference Assessment (CMPA)

University of British Columbia

This study validated the College Major Preference Assessment (CMPA) by evaluating its effectiveness of using a combination of Likert-type rating scale and forced-choice formats. Following the argument-based approach to validation by Kane (1992), I articulated the interpretation use argument for CMPA and its corresponding warrants and assumptions. The argument was evaluated by the three research questions based on datasets for Education and Psychology: 1) Did the respondents display response set due to Likert-type format of self-assessment? 2) Can the Likert rounds effectively screen out individuals’ “not preferred” majors? 3) Can the forced-choice rounds effectively identify individuals’ favorite major? A two-dimensional latent trait model was tested using structural equation modeling. The results, by and large, confirmed my hypotheses. Both Likert and forced-choice assessments had a very high level of discrimination power for assessing the major trait Favorite (individuals’ inclination towards a major being their favorite). The Likert rounds were relatively less difficult than the forced-choice rounds. The Likert rounds were more reliable for individuals with a relatively lower level of Favorite, and the forced-choice rounds were more reliable for a relatively higher level of Favorite. Moreover, respondents exhibited a response set, shown by the secondary latent trait, which reflected individuals’ perception about the majors when answering Likert-type items. Nonetheless, the assessment of Favorite was not contaminated by perception. All the findings matched the assumptions and warrants for the argument, showing that CMPA worked in the way it was designed for achieving a high level of effectiveness.

Text

2021-04-28

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/78023

Measurement, Evaluation and Research Methodology

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/