Open Collections

Kasana, Niyati

University of British Columbia

The kernel smoothing item response theory (IRT) is a nonparametric IRT approach based on nonparametric kernel density estimation (KDE) that predicts a test taker's expected response to an item using nonparametric item response functions (IRFs). The extant psychometric literature recommends using this technique for measurement scenarios with small sample sizes; what qualifies as small is seldom described. In recognition of this, two expository publications on this item analysis method suggest that it may be appropriate for about 300 to 400 test takers; however, the rationale for this recommendation needs to be further unpacked. Therefore, focusing on factors affecting the nonparametric item response generation process, the research presented in this dissertation aimed to systematically unpack the motivation and recommendation for using kernel smoothing IRT with small sample sizes. Three simulation studies were conducted based on actual (real) data from a validated education measure to document the robustness of the recommendation of using kernel smoothing IRT with small sample sizes for 11 study populations of test takers reflecting normal, uniform, asymmetric, and discontinuity or gaps in the total score distribution. For each study population, the selected bandwidth values and nonparametric IRFs were generated from a five-by-two factorial design simulation experiment with five levels of sample size (50, 100, 200, 300, and 500 test takers) and two bandwidth selectors: Silverman's Rule of Thumb (SROT) and Cross-Validation (CV). The two bandwidth selectors were compared with the selected bandwidth values and nonparametric IRFs generated for their respective populations. Unpacking the nuance of the recommendation to use kernel smoothing nonparametric IRT with small sample sizes, the overall results from the three simulation studies demonstrated that while the IRFs began to stabilize at a sample size of 300, a minimum sample size of 500 was required, especially under conditions of departures from normality and discontinuity of the test score distribution, to ensure robustness, replicability, and consistency of the conclusions drawn about the item characteristics. Furthermore, it was concluded that compared to the CV bandwidth selector, the Silverman bandwidth selector provided consistency, replicability, and ease of interpretation because the chosen bandwidth stayed the same across items and replications.

Text

2024-05-03

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/88189

Measurement, Evaluation and Research Methodology

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/