Open Collections

Ji, Xuejun (Ryan)

University of British Columbia

Validity is the paramount consideration while developing and evaluating tests. To boost validation efforts, this dissertation provides a flexible and pragmatic validation method based on a validity concept with a focus on test score interpretation (Cizek, 2020). In more detail, validation is viewed as an exercise of identifying and explaining desired and undesired effects via a cross-classified mixed effects model (CCMEM). To investigate its potential usefulness, this proposed validation method is applied to two illustrative studies. The first study showcases how to assess reliability and different sources of validity evidence for the ePIRLS 2016 Reading Assessment by utilizing CCMEM. The second study illustrates how to identify potential rating biases and investigate their sources, as well as highlights the importance of a multi-segment (repeated) rating design for competency-based assessment using the Mathematical Quality of Instruction (MQI) instrument. This dissertation expands the utility of CCMEM as a validation tool, and the two application studies showcase the practical versatility of this validation method.

Text

2023-11-02

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/86429

Measurement, Evaluation and Research Methodology

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/