Open Collections

Effects of order of judgment on subjective evaluation

University of British Columbia

It is commonly agreed that subjective evaluation tends to be less reliable than objective evaluation. This study represents an attempt to discover whether or not there are predictable characteristics of subjective evaluation which account for part of this unreliability. After logical analysis, three possible effects were proposed, (1) A contrast effect, in which the difference between a sample and the preceding sample or samples is minimized or exaggerated. (2) An experience effect, in which there is a long term shift in values. (3) An end effect, in which the last few samples in a series are judged according to different standards. Several possible test materials were considered. Finally, handwriting samples were chosen as satisfying the criteria of explicitness and necessity for subjective rather than disguised objective evaluation. Two experiments were devised; the first to test for a contrast effect, and the second to test for the experience and end effects. Evidence significant at the one percent level was presented for rejecting the hypothesis that there is no contrast effect. It was shown that there is a long term increase in the marks given, in which the increase was spread over the whole range of marks. There was evidence significant at the five percent level for rejecting the hypothesis that there is no end effect. It was shown that the last three samples of a long series tend to be downgraded.

Text

2011-12-15

For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.

http://hdl.handle.net/2429/39707

Education

University of British Columbia

Graduate