Open Collections

Abstract

This dissertation addresses the persistent challenge of incorporating study quality into meta-analysis. Although differences in study quality are widely recognized as problematic, guidance on how to handle them remains inconsistent and often misleading. To tackle this reality, this work pursues three objectives: to argue that quality score weighting is inherently unavoidable, to reconsider the value of quality score stratification (multiple restrictions) under the random effects framework, and to provide simulation-based evidence on when imperfectly reliable quality score weighting improves meta-analytic inferences. The first contribution demonstrates that any classical meta-analysis can be viewed as a quality score weighted meta-analysis; hence, there is no avoiding the problem of how to best incorporate study quality into a meta-analytic estimator. It is shown that even when a meta-analysis does not incorporate explicit quality score weights into the construction of a meta-analytic estimator, implicit weights are still present that imply all studies are of equal quality. The second contribution challenges the prevailing consensus against multiple restrictions by showing that, when random-effects models are used instead of fixed-effect models, restricting studies by quality scores can in fact yield more consistent and informative results. The third contribution advances the discussion through a simulation study that establishes clear conditions under which quality weighting is beneficial. Specifically, when the reliability of the quality scale exceeds 0.5, quality weighting reduces bias and improves confidence interval coverage. Together, these findings contest influential but flawed guidance and provide evidence-based recommendations for when and how quality weighting can meaningfully improve meta-analytic practice. The dissertation concludes by emphasizing transparent, rigorous, and practically applicable approaches for addressing study quality in evidence synthesis.