Open Collections

A new method of scoring Torrance's Test of Creativity

University of British Columbia

This study proposed an alternative method of scoring the Torrance Test of Creative Thinking: Thinking Creatively with Words (TTCT: Verbal A). This scoring method was compared to the conventional scoring method. I first considered the current method of scoring, "the old method" and then proposed a "new method". The old and the new methods of scoring were compared on a subscale of the TTCT. Both the old and the new methods of scoring were then compared to the Canadian Test of Basic Skills (CTBS) total language scores to determine if a relationship existed between verbal academic achievement versus the creativity construct. The new scoring method of creativity showed an improvement over the old method of scoring on the TTCT: Verbal A subtest. The reliability coefficient, for internal consistency, declined from 0.83 to 0.54. This decline was interpreted as an improvement because the initial value of 0.83 was artificially high. This was due, to a very high intercorrelation between fluency and originality (inter-attribute correlation value =. 97). The new scoring method corrected this inflation. I found a veiy low, negative correlation between CTBS total language and both the old and the new methods of scoring for creativity. The correlation coefficients were -.18, and -.17, respectively. This low value negative correlation demonstrated that creativity was not connected to verbal academic achievement. Although divergent thinking is the most commonly used aspect of creative thinking, previous researchers (e.g., Runco, 1986) have suggested that the tests of creativity lack reliability. The issue of reliability of a test score has generally been defined in terms of variation of scores obtained by the individual on successive independent testing (Cronbach, 1947). Neither the assumption of constancy of true scores nor the assumptions of experimental independence can be realized in practice with most psychological variables. Therefore, the composite reliability of a test needs to be considered as a concept, which can not be directly measured. Torrance (1986) specifically discouraged the use of composite test scores and instead recommended interpretation of the subscale scores in relation to one another. The fundamental problem currently faced with the TTCT is that scores derived from the same response data cause potentially spuriously subscale correlation. This inflates the reliability coefficient and seriously jeopardizes score consistency and reliability. Thorndike (1972) has argued that fluency, flexibility, and originality scores tended to be highly correlated, since all are accumulated over the same set of responses given by the examinees. This seriously discounts the value of the old scoring formula recommended by the Technical Scoring Formula of the TTCT. This fundamental problem of scoring has been attributed to derivation of five separate scores in that the scores were derived from the same response data (Heasusler & Thompson, 1988). This type of score derivation introduced spuriously high scale correlation because the flexibility, fluency, and originality scores tended to be highly correlated. Through some corrective procedures, correcting fluency, Clark and Mirels (1970) recommended that correlation coefficient value from several measures of Torrance's Test of Creative Thinking (Figure Completion) would decrease. This procedure consisted of administering a revised form of the sub-test to 93 students. These were then scored, based on fluency, flexibility, originality, and elaboration. Corrected fluency scores led to lower reliability coefficient values. My study had a similar experience. The alpha coefficient value dropped to .54 from .83, with the new scoring method. My research dealt with the score inflation, with the old scoring method, scoring inconsistency, and, proposed a new method of scoring the TTCT. I also highlighted the issue of inter-attribute high correlation between fluency and originality attributes. These two attributes are the main ingredients in assessing creativity. My research focused on the three traits of creativity, namely: (1) the fluency, (2) the originality, (3) the flexibility. These were measured with the use of the TTCT: Verbal A: Activity 4: Product Improvement. The TTCT was administered to a sample of children (N=187) in grades4 to 8. The data was analyzed to assess score reliability. The existing CTBS total language scores available at the school were also utilized in order to consider any relationship between creativity and verbal academic achievement. The new scoring method enhanced the reliability coefficient and improved upon the scoring techniques. The new scoring approach adopted the same scoring techniques as the old method' with the measurement of fluency and flexibility attributes. However, with the measurement of originality attribute, a new approach was adopted. The new scoring method I have designed considered only the two score attributes; fluency and flexibility. The originality score, which is simply a "fall-out" from the fluency score, was not included. This exclusion avoided double counting between fluency and originality and dealt with the high correlation problem between these two attributes. With the old scoring method, the correlation between fluency and originality would be as high as .99 (Thorndike, 1972). This simply indicated that fluency and originality attributes were measuring the same creativity dimension. The new method of scoring rectified this problem. Hocevar (1979) recommended a method as a way of improving test reliability. Hocevar's method, however, recommended scoring for originality and flexibility and dividing the resulting score by the total number of responses. His recommendation was very valid in improving discriminant validity and did not address the score inflation and high inter-attribute correlation problem between fluency and, originality. I have somewhat revised his recommended method and proposed that e capture the flexibility and fluency responses and divide the resulting score by the total number of responses. This ensured consistency in measurement and corrected the high inter-correlation problem between the TTCT attributes. It also improved score reliability. Under the new method of scoring, each response for flexibility and fluency is adjusted by dividing the resulting scores by the total number of responses. Since fluency scores tended to be highly correlated with the originality scores, this new approach safeguarded against duplication in scoring also reduced spurious correlation between fluency and originality. Originality attribute was no longer required for scoring. Division of both flexibility and fluency by the total number of responses also streamlined the overall composite score. The composite score of the TTCT, under the new method contained fluency and flexibility scores. This also ensured that a number of response which would normally be scored with "0", would be considered as part of the fluency score, as originality attribute was no longer considered. The factor analytic results obtained by Heauslar and Thompson (1988) showed that the TTCT subscale yielded discrete scores. The general creativity factors were sufficient in assessing creative behavior. These factors included fluency, flexibility and originality. The total TTCT scores were not recommended (Torrance, 1974; Torrance & Ball, 1984) for assessment of creativity. He recommended using scores of fluency, flexibility and originality. This also jeopardized the integrity of the total creativity scores. The new method improved the reliability of the test scores of We TTCT by enhancing the existing scoring methodology. With the old scoring approach of originality, responses which did not meet the pre-determined fluency categories, were scored a "0", regardless of the relevancy of response. This diluted the originality scored, and distorted score reliability. The new method considered all the fluency responses and scrutinized all response for consistency under fluency category. Fluency attribute was not simply considered as the number of total ideas, regardless of the degree of relevancy of task on hand. Each response of fluency was evaluated for its relevancy, in view of the task, on hand. This streamlined the scoring approach with fluency. With the new method of scoring, the standardized alpha coefficient declined from .83 to .54. The new method of scoring improved the process and made it easy and more relevant to score for fluency and flexibility. A very low, negative linear relationship was found between the creativity construct and academic achievement, as represented CTBS total language test.

Text

2012-05-28

For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.

http://hdl.handle.net/2429/42400

Special Education

University of British Columbia

Graduate