Impact of scoring and response format on inferences from electronic surveys for rating scales
department or school: Education, Faculty of
department or school: Educational and Counselling Psychology, and Special Education (ECPS), Department of
data provider: DSpace
degree campus: UBCV
creator: Coronel Villalobos, Mauricio
issued: 2020-09-24T16:55:46Z
issued: 2020
related degree: Doctor of Philosophy - PhD
degree grantor: University of British Columbia
description: In social sciences, the measurement of constructs requires decisions on how to quantify, record and analyse scores. The use of electronic surveys allows for additional measurement tools different from traditional paper surveys. However, the manner of how these choices affect the validity of scores has not been explored thoroughly. This dissertation uses the Center for Epidemiology Scale of Depression to explore the effects of scoring method and response formats. Study 1 compares four response formats commonly used in electronic surveys, and scores obtained using different electronic devices. Measurement invariance of the scale was examined using multi-group confirmatory factor analysis. Results showed that electronic response formats suggest response behaviours to participants. Furthermore, scores obtained using different response formats and electronic devices failed to show measurement invariance at a configural level. Study 2 compares changes in differential item functioning (DIF) using ordinal logistic regression based on the scoring method. Furthermore, DIF contrasts included binary sex, minority gender identity and sexual orientation. Results showed that changes in scoring methodology altered which items presented DIF. Moreover, contrasts with sexual and gender minority identity altered the presence of DIF. As such, treatment of scores and choice of focus for demographic characteristics impact the findings of psychometric properties of surveys. Overall, findings from both studies suggest that the structure of surveys can be fragile, and conclusions drawn from scores are sensitive to any changes to the measurement and analyses. Any such change, even if seemingly minor, may change the construct, and consequently affect the validity of scores. Researchers must be mindful of the specific needs of their target construct and use adequate forms of measurement and analyses to suit their studies, thus guiding more precise measurement. A fuller understanding of the fact…
aggregated CHO: https://circle.library.ubc.ca/rest/handle/2429/76151
note: IMPACT OF SCORING AND RESPONSE FORMAT ON INFERENCES FROM ELECTRONIC SURVEYS FOR RATING SCALES by MAURICIO CORONEL VILLALOBOS M.A. The University of British Columbia, 2015 A DISSERTATION SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Measurement, Evaluation and Research Methodology) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) SEPTEMBER 2020 © Mauricio Coronel Villalobos, 2020 ii The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the dissertation entitled: Impact of scoring and response format on inferences from electronic surveys for rating scales submitted by Mauricio Coronel Villalobos in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Measurement, Evaluation and Research Methodology Examining Committee: Dr. Bruno D. Zumbo, Professor, Educational and Counselling Psychology, and Special Education, UBC Supervisor Dr. Sheila Marshall, Professor, Department of Pediatrics, UBC Supervisory Committee Member Dr. Amery Wu, Professor, Educational and Counselling Psychology, and Special Education, UBC Supervisory Committee Member Dr. Kim T. Zebehazy, Professor, Educational and Counselling Psychology, and Special Education, UBC University Examiner Dr. Skye Barbic, Professor, Occupational Science & Occupational Therapy, UBC University Examiner Additional Supervisory Committee Members: Dr. Elizabeth Sanders, Professor, Educational Psychology, University of Washington External Examiner iii Abstract In social sciences, the measurement of constructs requires decisions on how to quantify, record and analyse scores. The use of electronic surveys allows for additional measurement tools different from traditional paper surveys. However, the manner of how these choices affect the validity of scores has not been explored thoroughly. This dissertation uses the Center for…
has type: Thesis/Dissertation
date issued: 2020-11
is shown at: 10.14288/1.0394498
language: eng
degree discipline: Measurement, Evaluation and Research Methodology
provider: Vancouver : University of British Columbia Library
publisher: University of British Columbia
rights: Attribution-ShareAlike 4.0 International
rights URI: http://creativecommons.org/licenses/by-sa/4.0/
scholar level: Graduate
type: Text
identifier URI: http://hdl.handle.net/2429/76151