UBC Theses and Dissertations
Cognitive strategies and heuristics underlying psychologists’ judgments on the WISE-R verbal scales : a protocol analysis Perot, Josette Anne-Marie
The primary purpose of this study was to investigate psychologists' natural, interactive decision-making behaviour while scoring difficult verbal responses on the Wechsler Intelligence Scale for Children-Revised. A total of 23 psychologists participated in the study. First of all, in order to obtain scoring information descriptive of the sample, psychologists scored a WISC-R protocol. This protocol comprised four verbal scale subtests: the Vocabulary, Similarities, Information, and Comprehension subtests. In order of difficulty, the Vocabulary, Comprehension, Similarities, and Information subtests were found to be most prone to scoring differences. The Verbal IQ was found to vary by 11 points. Differences in point assignment within subtests accounted for variance in scoring. Following the completion of the first measure, a sub-sample of 8 psychologists provided think-aloud protocols in a separate session while scoring a second fabricated Comprehension subtest. The complexity of the task involved the consideration of administration errors and response judgment while scoring. Rather than focus solely on quantitative analysis of error differences as has been done in prior research, this study conceptualized these sources by providing additional analysis of specific strategies psychologists used while making scoring decisions. The results of the verbal protocol analysis identified cognitive strategies inherent in the scoring of difficult type responses. The type and frequency of cognitive strategies identified in the study appear to be related to individual scoring accuracy. At the end of the session, psychologists were asked to identify strategies that were useful to them in difficult scoring situations. All psychologists identified the manual as the primary heuristic; however, percentage frequencies of verbalized strategies across subjects indicated that only four of the subjects used the manual as their primary aid on this task. These findings are further discussed, as well as their implications and inferences.
Item Citations and Data