Open Collections

Grouping variables in bias and impact detection : an attributional stance for observational studies

University of British Columbia

Group disparity in the desired outcome (e.g., academic achievement) has always been a research emphasis. A group comparison is fair and meaningful only if the measured scores are comparable and free from bias. Measurement bias is closely connected to test fairness and has been investigated for decades. In the psychometric literature, the concepts of DIF, bias and impact are interconnected but different from one another. Wu et al.'s (2017) attributional framework, based on group comparison, explicates the differences among them and provides analytical procedures to detect bias and impact. Their attributional view makes the definition and interpretation of bias and impact sensible and meaningful even when groups are non-experimentally formed. Building on Wu et al.'s (2017) framework, the purpose of this thesis is to address the issues arising from group comparisons intended to test a causal hypothesis in observational studies. Specifically, this thesis focuses on the concerns with interpreting and communicating bias and impact detection based on comparisons between "pre-existing" groups (i.e., groups differ in their socio-demographic characteristics that exist before testing or research). To this end, Chapter 1 provides background knowledge of DIF, bias and impact, including their relationships with test fairness, measurement invariance, and validity. The terminology of DIF, bias, impact and the manners in which they are connected with and distinct from one another are discussed. The definition of bias and impact refined by Wu et al. are then reviewed. Chapter 2 distinguishes various types of grouping variables and proposes a typology for grouping variables resulting from different research designs in social and behavioral research. Based on the typology, difficulties in interpreting the results of applying Rubin's causal model (Rosenbaum & Rubin, 1983) are discussed. Chapter 3 presents relevant terminologies and techniques according to Wu et al.'s analytical procedures for studying bias and impact, including the propensity score, statistical matching, and conditional logistic regression. In Chapter 4, two data examples are illustrated to show how the adapted conceptual, methodological, and technical rhetoric helps to clarify the long-standing confusions in understanding and communicating bias and impact. Chapter 5 summarizes contributions and limitations of this thesis, and future work.

Text

2017-10-02

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/63188

Measurement, Evaluation and Research Methodology

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/