Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Use of the Cloze procedure in the assessment of schizophrenic thought disorder Huberman, Jack Jacob 1963

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


831-UBC_1963_A8 H85 U7.pdf [ 6.63MB ]
JSON: 831-1.0105626.json
JSON-LD: 831-1.0105626-ld.json
RDF/XML (Pretty): 831-1.0105626-rdf.xml
RDF/JSON: 831-1.0105626-rdf.json
Turtle: 831-1.0105626-turtle.txt
N-Triples: 831-1.0105626-rdf-ntriples.txt
Original Record: 831-1.0105626-source.json
Full Text

Full Text

USE OF THE CLOZE PROCEDURE IH THE ASSESSMENT OP SCHIZOPHRENIC THOUGHT DISORDER by JACK JACOB HUB1RMAH A Thesis Submitted i n P a r t i a l fulfilment of the Requirements for t h e Degree of Master of Arts i n the Department of Psychology We accept th i s thesis as conforming to the standard required from candidates for the degree of MASTER OP ARTS Members of the Department of Psychology The University of B r i t i s h Columbia September, 1963 I n p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t of the r e q u i r e m e n t s f o r an advanced degree at the U n i v e r s i t y of B r i t i s h Columbia, I agree t h a t the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r r e f e r e n c e and s t u d y . I f u r t h e r agree t h a t p e r -m i s s i o n f o r e x t e n s i v e c o p y i n g of t h i s t h e s i s f o r . s c h o l a r l y purposes may be granted by the Head of my Department or by h i s r e p r e s e n t a t i v e s , , I t i s understood t h a t copying, or p u b l i -c a t i o n o f t h i s t h e s i s f o r f i n a n c i a l g a i n s h a l l not be a l l o w e d w i t h o u t my w r i t t e n p e r m i s s i o n . Department of The U n i v e r s i t y of B r i t i s h Columbia,, Vancouver ST, Canada. Date ABSTRACT: The present investigation was an attempt to -determine the v a l i d i t y of the Cloze Procedure i n assessing differences between'schizophrenic patients i n the r e l a t i v e severity of thought disorganization. For th i s purpose, samples of schizophrenic, speech i n response to TAT cards were obtained from twelve male schizophrenic patients who had been rank ordered by two psychiatrists i n terms of disordered thought processes. Equal length schizophrenic language samples with every f i f t h word deleted and replaced by a standard length blank were mimeographed and presented to a group of upper l e v e l undergraduate psychology students who w e r e instructed to t r y to guess the missing words from the context of those that remained. These mutilated language samples constitute the Cloze forms. The scores on the Cloze Procedure•represented a•rank ordering by the students of the severity of schizophrenic thought disturbance i n terms of the number of co r r e c t l y replaced words i n the language samples e l i c i t e d from each patient. A substantial correlation c o e f f i c i e n t ("r" = .82) was found between the rank order on the Cloze Procedure and the rank order obtained i n the external c r i t e r i o n measure, that i s , the psychiatric ratings. The results of tests for internal consistency and r e l i a b i l i t y of the Cloze Procedure ("V.r" = .622) and for the r e l i a b i l i t y of the Close raters ("R" = ,792) were also substantial. A number of advantages and disadvantages of this method of investigation are discussed and contrasted with those of other somewhat similar procedures. Possible applications to psychiatric research are suggested. The results appear to be conclusive within the limitations cf the c r i t e r i o n measure employed i n t h i s study. More widespread generalizations await further research which i s invited i n l i g h t of the apparently promising findings of t h i s exploratory study. i i i ACKNOWLEDGEMENTS The author wishes to express his sincere apprecia-t i o n to Dr. E.I. Signori for his constant interest, i n s p i r a t i o n and guidance. He also wishes to thank Dr. D.L. ^ ampson for his help f u l comments and encourage-ment, and Dr. D.T. Kenny for his assistance i n formulat-ing the i n i t i a l plans f o r this research. The assistance of Dr. W.G. Mac Earland and Dr. ¥. Schlichter of the medical s t a f f at Crease C l i n i c i s also g r a t e f u l l y acknowledged. o i v TABLE OF CONTENTS Chapter . ', Page ABSTRACT . . . . . . i i ACKNOWLEDGEMENTS . iv I STATEMENT OF THE,PROBLEM 1 II REVIEW OF THE LITERATURE 4 Schizophrenic Thought Processes . . . . . . . 4 Theoretical Background . . . . . . . . . . . 4-Experimental Background . . . . . . . . . 10 Schizophrenic Language . . . . . . . . . . . 16, Cloze Procedure . . . . . . . . . . . . . 28 Reliability of Psychiatric Ratings . . . . . 34 III . PROCEDURE , . . , . . , , , . . . . . . • „*.... .. 38 El i c i t i n g Verbal Samples . . . . . . . . . . 39 Subjects . . • . . . . • .. • • .... . •, . . 39 Method . . . . . . . . . . . . . . . . . . 40 Establishment of the Criterion , . .•• . . . 42 Cloze Procedure . . . . . . * ... . . . . . 44 Cloze Raters . . , . . . . • . . . . . . * 44 Method . , . .* . . ., .. . ,. . , . . . . 44 IV RESULTS AND DISCUSSION , ,. . . . . . .. . . . * 47 V SUMMARY AND CONCLUSIONS . . . ,. . ,. , . . . ,. 57 APPENDICES • Page A Criterion Checklist for the Assessment of Thought Disorder • • • • • « • • o * * 61 B Cloze Forms . . . . . . 9 » • 63 G Cloze Answer Blank . . . . . . . . * • « 70 D Cloze Raw Scores • • • • • • » • * » • a 76 E Cloze Ratings on Three Subsections 9 9 77 F Cloze Rankings on the- First Third • « O 80 Q Cloze Rankings on the Middle Third • 9 o 82 H Cloze Rankings on the Last Third • • • • 84 I Cloze Rankings • • • • 86 J Statistical Formulae • • • 88 REFERENCES .*• . . . . . ... . 90 YX LIST OF TABLES Table Page 1 Psychiatric Rank Ordering of Patients i n Terms of Degree of Disturbed Thinking . . . . . . . 4 8 2 Total Cloze Scores and Their Corresponding ' Rank Ordering . . . . . . . . . . . . . . . . '50 3 Correlation Coefficients for Internal Consistency .... . . . ... • . . * . . . . . . 5 4 4 Average Cloze Rank Orderings with Different Groups of Raters . . . . . . . . . . . . . . '55 v i i CHAPTER I STATEMENT OF THE PROBLEM The language and verbal behaviour of the schizo-phrenic patient has been an area of spe c i a l interest and has commanded the attention of investigators as a source of information about the phenomenology and psyehopathology of schizophrenia* The types of research that have been concerned with schizophrenia and the language patterns of schizophrenic patients have taken several directions* In the f i r s t of these, some investigators have studied the nature of the conceptual and symbolic aspects of schizo-phrenic communication from the psychopathologieal point of view* Others have described ways i n which the form and structure of language behaviour are diagnostic of the schizophrenic syndrome. And more recently, investigators have studied schizophrenic verbal productions with the view to developing a means of quantifying the extent of d i s t u r -bance of the schizophrenic patient. The problem under consideration in t h i s study i s d i r e c t l y related to thi s most recent l i n e of investigation. Much of what i s known about schizophrenic thought disorder rests upon the assumption that a parallelism exists between speech and thought. Language represents the external manifestations of thought. The disturbances of language are presumably therefore, e s s e n t i a l l y evidence of 2 the disturbance of thought, a cardinal problem i n schizo-phrenia. The schizophrenic often uses words and sentences i n such a way that t h e i r meaning i s not fa m i l i a r to the average person. His .language i s often ambiguous and has a tendency to - d i f f u s i o n and generalization. His sentences may contain many, self-references as well as impersonal constructions,* His speech may have a formal and involved structure, or i t may be ;expressed i n a "word,salad" with no apparent, ar t i c u l a t e d structure, His communication i s therefore d i f f i c u l t to understand. The lack of continuity and coherance, i n parts, reveals ,the extent of mental d i s i n -tegration. This study i s concerned with the language of schizophrenic patients but not i n the sense of tryi n g to amplify further the already substantial knowledge and theo r e t i c a l considerations regarding the thinking and communication disturbances i n schizophrenia. Bather, an attempt i s made to u t i l i z e t h i s schizophrenic verbal communication as part of a-recently developed psycho-l i n g u i s t i c technique known as the Cloze Procedure (Taylor, 1953) in. attempting to assess, i n a quantitative manner, the r e l a t i v e severity of thought disorder i n schizophrenic patients. The Cloze Procedure was o r i g i n a l l y devised to assess r e a d a b i l i t y and comprehensability of prose material and l a t e r 3 extended to an assessment of the effectiveness of other communication media.- Studies of i t s application to the understanding of the verbal behaviour of mental patients ; has just begun (Taylor, 1963)• It w i l l be employed in.this study to assess the comprehensability of schizophrenic language, which is presumably a.reflection of the under-lying disordered schizophrenic thought processes. . The values of the present investigation are, generally speaking, two-fold? f i r s t , to work oh an experimental approach to a basic problem in psychology, namely, the problem of developing methods of quantifying psychological processes,and behavioural patterns; and second, to work towards developing a •useful-.tool, in psychiatric practise and research, a means, for example, of assessing the effectiveness of various therapies on the relative severity of disturbance in a number of different psychiatric dis-orders. CHAPTER II REVIEW OP THE LITERATURE Of the varied and multiple manifestations of the mental disorganization i n schizophrenia, those related to thought have attracted the most attention from psychologists and p s y c h i a t r i s t s . This i s so, perhaps, because formal thought disorder i s generally accepted as being one of the few unifying features of schizophrenia. "Almost a l l schizophrenics show formal thought disorder i f t h e i r i l l n e s s e s l a s t long enough" (Pish, 1962, p.* 18); Neverthe-le s s , an examination of the psychological research i n schizophrenic language and thought reveals that a consider-able d i v e r s i t y of opinion s t i l l exists with regard to the exact nature of the psychological disturbances i n schizo-^ phrenia* Below i s presented a b r i e f h i s t o r i c a l survey of the theore t i c a l and experimental background of the prob-lems i n t h i s area. The reader should keep i n mind that there may be considerable overlap i n time between di f f e r e n t phases of research on t h i s problem. Schizophrenic Thought Processes (1) Theoretical Background Psychological investigations of schizophrenia were o r i g i n a l l y oriented i n the concepts f i r s t outlined by Eraepelin and Bleuler. Kraepelin (1919) was the f i r s t 5 p s y c h i a t r i s t to i s o l a t e "dementia praecox" as a c l i n i c a l e n tity and describe i t s symptomatology i n great d e t a i l . He defined as "dementia praecox" a symptomatology consisting of auditory hallucinations, delusions of persecution, incongruous emotivity, impairment of attention, negativism and stereotyped behaviour * The symptoms were described and accepted 1 as they were, with no. attempt .being made to., interpret: them.1 " i Bleuler (1950) delimited the "dementia praecox" of Kraepelin to schizophrenia. He postulated a the associative processes as the basic disturbance i n schizb-phreniai He explained t h i s as ifollows t the1 absence.of a c e n t r a l l y determining thought leads to the sequence of thought being determined by incidental: associations,. so that the association between one idea and the next becomes haphazard; This disorder of association leads to a change-a b i l i t y ,and lack of c l a r i t y of the concepts which are f a l s e l y constructed by condensation, displacement and symbolism, : In condensation, ideas which have something i n common are lumped together regardless of l o g i c , whereas i n displacement, an associated idea i s used i n place of the correct one. Symbolism refers to the misuse of symbols, usually the use of symbols i n a concrete way* The weakness i n association allows the a f f e c t to dominate the t r a i n of thought, and i f severe enough, to the fragmentation of thought. Incoherence and i l l o g i c a l thinking r e s u l t . 6 Kraepel in and Bleulery, with accurate, de ta i l ed descr ipt ions of the behaviour of schizophrenics , revealed a f r u i t f u l f i e l d for subsequent invest igators who. have proceeded along a number of d i f f erent l ine s of approach, each based on some of the features of schizophrenic thought and.language. One of the e a r l i e s t formulations concerned i t s e l f with the "regression hypothesis". Vigotsky (1934), the f i r s t to put f o r t h th i s ideaj invest igated the speculations of Piaget (In Goodstein, 1951) who had re lated schizophrenic th inking to a hierarchy by p lac ing the c h i l d ' s th inking midway between the "aut i s t ic" thinking of the schizophrenic and the l o g i c a l th inking of the adult* He postulated that an essent ia l c h a r a c t e r i s t i c of schizophrenia was the loss of the a b i l i t y to think i n concepts and a regress ion to a more pr imi t ive l e v e l of th ink ing . A r i e t i (In Werner, 1955) suggests that the sch izo-phrenic "resorts to archaic ways of expressing f e e l i n g and thought, long ago discarded i n the course of evo lut ion . He undergoes "teleologlc regress ion." Regression because the patient reverts to more pr imi t ive or archaic forms of, integration;; t e l eo log ic because the process i s psycho-dynamically oriented" (p.54) . The l i n g u i s t i c thought of c e r t a i n stages of sch izo-phrenia have been interpreted by Yon Domarus' p r i n c i p l e (Von Domarus, In Kasanin, 1946) which states that while a normal person would accept iden t i ty only upon the basis of identical subjects, the schizophrenic assumes identity based identical predicates. For instance, he would think, "Eisenhower was a soldierj I am a soldier. Therefore, I am Eisenhower". This conclusion is reached because the identity of the predicates (being a soldier) makes the schizophrenic accept the identity of the subjeots (Eisenhower and himself*) This i l l o g i c a l use of identities cause the schizophrenic, the "paralogical thinker", to revert to a level of primitive mentality which corresponds to that of primitive people and children. Von Domarus (In Goodstein, 1951) uses this resemblance between the speech of schizo-phrenics and primitives and children in support of Vigotsky*s (1934) "regression hypothesis." However, the fact that schizophrenic disorders do appear in primitive c i v i l i z a -tions and among children of our own contemporary society would seem to contradict this "regression hypothesis". Cameron (1939a) observed that the child does not show the same structure of language as the schizophrenic, so that regression used in this sense becomes a very loose word, especially as the schizophrenic loses the adult organization of speech without reverting to a childlike level. These speculations into the regression of schizo-phrenic thinking have not been subjected to controlled study and consequently remain merely as descriptive theories. 8 It has long been recognized that the, conceptual thinking:of schizophrenics deviates from that of normals inmany respects>• However* the exact nature of the schizo-phrenic*s conceptual performance is not clearly under-stood.. Goldstein (In Hasanin, 1946) has attempted to ex- . plain schizophrenic thought disorder by means of his con-cepts of concrete and abstract attitudes. He considers that an impairment of the a b i l i t y to abstract i s partly responsible for the schizophrenic thought disorder since the concrete attitude causes the schizophrenic to be unduly influenced by external stimuli wh^ich arise at any moment* Words are,chosen according to the peculiar way in which the schizophrenic is experiencing a certain object or situation, in the environment. As a result of this concrete attitude, words cease to retain their usual meanings,; and tend to take on unusual ones. Words which indicate categories or classes are. thus used in a way which does not correspond to the generalizations which they normally signify. The general consensus in most of the earlier work arising from Goldstein's (In Kasanin, 1946) "abstract/ concrete- hypothesis" was that a d e f i c i t in the conceptual performance of schizophrenics did exist, and that this def i c i t could be expressed in terms of a less abstract and/or more concrete level of performing relative to normal expectancy. However, other investigators have proposed that conceptual abi l i t y may not be the primary source Of 9 error i n the conceptual performance of schizophrenics *.: They are suggesting that conceptual a b i l i t y , per se, merely; ; appears to, be impaired, because of some other d e f i c i t rr; producing factors involved i n the performance. . At.the present time there are several other ... hypotheses which have been proposed as alternatives to, the hypothesis that the deviant conceptual performance results from impaired a b i l i t y . One .alternative proposes that ''overinclusiveness" or the tendency to respond t o irrelevant stimuli: i s primarily responsible for;the d e f i c i t i n eon-, ceptual ;performance^ This d i f f e r s from the aforementioned theory since, as Cameron (In Kasanin,, 1946) points•...pui.y:.. the schizophrenic can .conceptualize,; but that he. does so i n such a fashion that :,is; not recognized'. as being abstract.. In overinclusion, the schizophrenic cannot maintain the boundaries of the problem* He cannot r e s t r i c t his thinking to the task in.hand, to eliminate a l l unnecessary material focus his thoughts on.any given problem^ Supporters of this.hypothesis have argued that conceptual a b i l i t y , merely seems impaired because adequate conceptual per'forr*-• ;;B®.^ oe i s prevented by the over-responsiveness to extraneous s t i m u l i , ... In another alternative, Cameron,(1939b) and ,. S u l l i v a n (In Kasanin, 1946) hypothesize that what appears as a d e f i c i t i n conceptual a b i l i t y i s primarily a r e f l e c t i o n of the;schizophrenic's s o c i a l d i s a r t i c u l a t i o n and autism».' Sulli v a n suggests that confusion of. certain c r i t i c a l '• i 10 faculties involved in the thought processes of the schizo-phrenic result because of recurrent severe disturbances in Ms relationship with other people, fhe theory .cohtends that conceptual deficit can best be characterized as a disturbance•of social .communication^ - Other theorists suggest that the conceptual p e r f o r -mance of schizophrenics-may vary as a function.of the content and structure of the task i t s e l f . Whiteman (1956) considers the apparent deficit to be related to the degree of inter-personal content of the conceptual task? Richmah (1957) and Wexler (In Lorthrop, 1961) believe that a reduction in schizophrenic abstract thinking may be associated with the presentation of emotionally-toned material as opposed to neutral material. They also•suggest that a deficit would -j result under conditions of threat Of failure, that i s , that anxiety impairs concept formation. (2) Experimental Background The earliest objective studies of schizophrenic thought were concerned, to a very large extent, with an analysis of-concept formation. Psychologists -have attempted, in their research in this areas (1) to describe and demonstrate a generalized conceptual impairment in schizo-phrenia; (2) to secure some measure of this impairment; (3) to describe and demonstrate the basic disturbances of some specific thought processes; and (4), to show that 11 certain characteristics of these thought processes are; unique:to schizophrenics* Ih i t s ; i n i t i a l stages, research interest was centered:upon the apparent concreteness of the schizo-phrenic's thinking and upon his d i f f i c u l t y in seeing abs-tract relationships "between things* Accordingly, a methodology was developed for studying conceptual thinking, the "sorting test n technique. The general principle of this method involves presenting the subject with a number;of stimulus objects and then requiring him-to sOrt them into different classes on the basis of their : various properties. Most of the early tests measured performance in terms of levels of conceptualization, running on a continuum from concrete through.functional to abstract, A variety of sorting tests have yielded evidence that some schizophrenics identify or construct fewer abs-tract relationships within the stimuli presented than normals (Feldman and Drawgow, 1951; Fey, 1951); corres-pondingly, they tend to give more concrete responses than normals, (Kasanin and Hanfmann, 1938; Bashkis,•1947)» In addition, there is some evidence that schizophrenics have d i f f i c u l t y in shifting from one concept to another. Chapman (1956) found that abstract thinking in schizo-phrenics i s influenced by an inability to maintain a con-sistent direction or set to their behaviour in the presence of inappropriate stimuli and that this, rather than an i n -a b i l i t y to form concepts may account for the poor conceptual 12 performance. of schizophrenics *-The analysis of vocabulary responses i s a methodological approach to schizophrenic thinking that i s more recent i n o r i g i n . Two studies have Indicated that i n defining words verbally, schisophrenics give s i g n i f i c a n t l y fewer abstract responses than normals * ( F e i f e l , 19491 Harrington and Ehrmann, 1954). £«»<T. Koran, F.A. Moran and Blake (1952) found s i g n i f i c a n t l y lower raw vocabulary scores f o r the schizophrenics as compared with normal© matched f o r age, sex and education. Several studies also reported that schizophrenics chose fewer abstract d e f i n i t i o n s than normals from the alternatives of multiple-choice vocabulary tests (Chordokoff and Mussen, 1952; F l a v e l l , 1956; Milgram, 1959). Moran (1953) compared a group of schizophrenics with a matched group of normals on vocabulary knowledge and usage. He found that the schizophrenics were less precise i n the understanding of word meanings, l e s s able to use words as conceptual instruments and l e s s able to integrate words into meaningful communication. Rabin, King and Ehrmann (1955) suggest that two factors must be taken into account i n the evaluation of 'vocabulary;: performance of schizophrenics, namely the s e n s i t i v i t y of the vocabulary measures and the nature of th© schizophrenic sample. In t h e i r study, using three vocabulary measures and two groups of schizophrenics d i f f e r -ing^ i n chronicity, they.found no difference between short* term schizophrenics and normals on any of the measures, but found that the long-term schizophrenics were s i g n i f i c a n t l y lower than both the normals and short-term schizophrenics on '• a l l measures* •,•: • Bpstein (1953) was one^of the f i r s t to invest igate Cameron's (In Sasanin, 1946) concept of over inc lus ion i n schizophrenic th ink ing . He constructed a test f or th i s ••• purpose which involved•having the subject underl ine words , esseiitial"'for descr ib ing a -given-, stimtilus•'.word,; --He "found • '• that schizophrenics , as a group, overincluded unessent ial words s i g n i f i c a n t l y more often than normals,.,. A ser ies of . experiments by Chapman (1956) and Chapman and Taylor (1957) offers a more d i r e c t t es t of the over inc lus ion hypothesis . They found that abstract th inking i n schisophrenics' was influenced by an i n a b i l i t y to maintain a consistent d i r e c -t i o n or set to the i r behaviour i n the presence of "dis tracter s t i m u l i " . In the presence of "dis tracter s t i m u l i " , the schizophrenics' performance on a p a r t i c u l a r card-sor t ing task diverged from that of the normals, and when the "distracters" were hot presentj the performance of the schizophrenics was close to that of the normals. These f indings were interpreted as i n d i c a t i n g that the schizophrenic d e f i c i t on conceptual tasks was, i n part , the r e s u l t of t h e i r overresponsiveness to d i s t r a c t i o n s , thus preventing correct performance on the task.^ A recent inves t igat ion by Payne, Mattusek and George (1959) gives further, evidence i n support of the over inc lus ion hypothesis; 14 : Be l lak (1958) suggests that one of the dominant or ientat ions i n schizophrenia i s a psychogenic one, the general view being held that b a s i c a l l y schizophrenia • represents a disturbance i n interpersonal r e l a t i o n s h i p s , This notion i s e spec ia l ly congruent with the t h e o r e t i c a l pos i t ions held by Cameron (1939b) and S u l l i v a n (In Kasanin, 1946). Studies demonstrating that conceptual d e f i c i t increases as the interpersonal content of the t a s k . i s < ; increased have been. interpreted-as lending i n d i r e c t support to the hypothesis ( F l a v e l l , 1956WhitemaiiV • 1956). ' A more . d i r e c t test of the hypothesis was made i n a study by • •• McGaughran and Moz^ ah (1956)• : : They did not. f ind any ; sign'i-f leant 1 :'differehq ;es :'between''their-schisophrenic-' sample' and. • controls when performance was scored only for conceptual l e v e l . On the other hand, they were able to demonstrate : : a s i g n i f i c a n t loss i n s o c i a l communication i n r a t i n g the communieability of the concepts. The resu l t s are in ter* preted'as demonstrating a loss of s o c i a l communication i n schizophrenia without any apparent d e f i c i t i n abstract a b i l i t y * More recent ly , Weckowiez and Mewett (1959) attempted to r e p l i c a t e the above study and obtained the opposite e f fec t j they demonstrated a d e f i c i t i n conceptual l e v e l but not i n s o c i a l communieability* In view of the apparently contradictory f indings , i t s t i l l remains to be: proven that the disturbance i n s o c i a l communication, by i t s e l f , can account for the d e f i c i t i n schizophrenic con-ceptual performance; 15 Studies by Richman (1957) and, Wexler (In Lorthrop, 1 9 6 1 ) i n v e s t i g a t i n g the influence of a f f ec t ive components on schizophrenic.performance on vocabulary d e f i n i t i o n s , revealed that a reduction i n abstract th inking was associated with the presentation of emotionally-toned words as compared wi th ,neutra l words. Evidence i s a lso presented showing that the schizophrenic's conceptual d e f i c i t i s increased by personal ly threatening topics i n the tasks• Threat, of f a i l u r e was shown to have s i m i l a r resu l t s on conceptual task performance. S t i l l i n the prel iminary stage of inves t igat ion i s a l i n e of research invo lv ing the re la t ionsh ip between the l e v e l of abstract th inking i n schizophrenics and the, phys io log i ca l responsiveness to drugs. Meadow, G-reenblatt, Funkenstein and Solomon (1953a» 1953b) suggest that cer ta in barbiturates and amphetamine drugs f a c i l i t a t e conceptual th inking i n schizophrenia . They maintain that drugs seem to improve conceptual reasoning a b i l i t y by decreasing the influence of personal reference. Funkenstein, (Jreehblatt and Solomon (1951) found, that a f t e r i n j e c t i n g epinephrine and mecholyl, there was a s i g n i f i c a n t c o r r e l a t i o n for both a group of acute and chronic schizophrenics , between the capacity for abstract th inking and the f a l l i n s y s t o l i c blood pressure below the basal l e v e l . Results of these and other studies i n th i s area are , of necess i ty , h ighly t en ta t ive . Schizophrenic Language 16 Language has so often been given such designations as the "instrument for the expression of thought" and the "external manifestations of thought" that one can scarcely speak of language without invoking the concept of thought* Language would "be meaningless without the implicit assump-tion that i t s meaning is derived from thought. It is obvious that the two processes, although not identical, are closely intertwined, "The cl a r i t y with which schizo-phrenic language exposes the process of thinking has probably been influential in the automatic translation of schizophrenic language into schizophrenic thought." (Lorenz, 1961, p.607)Thus, schizophrenic language w i l l be presumed to be a direct reflection of the underlying thought processes throughout the discussions in this study. Much of the research on schizophrenic thinking that has already been presented has indirect or even direct relevance for schizophrenic language. By the same token, investigations of language reported here inescapably deal with thought to some degree.. In many cases, i t would almost appear that the investigator has made an arbitrary decision as to whether he is studying language or thought. In the fi n a l analysis, however, most research in this area is principally concerned with the thought processes in schizo-phrenia. 17 Although the number of studies of language i s r e l a t i v e l y small, i n comparison with the number Of studies of thought processes, a variety of orientations and methodologies have been employed. Several notable studies of the q u a l i t a t i v e aspects of schizophrenic language should be mentioned:. The e a r l i e s t purely descriptive studies of schizophrenic language were conducted by White (1928), Whitehorn and Zipf (1943) and Woods (lp8>* White (1949) m attempting to understand the Banner i n which schizophrenics use and interpret ordinary language, found that the language of schizophrenics was more r i g i d , impersonal, involved and complex than the language of a matched group of normals* She also found more incomplete sentences and more r e p e t i - • tions i n words used i n sentence construction. M i r i n (1953) employed a structured interpersonal setting to evaluate schizophrenic verbal communication. He found that the schizophrenics manifested a marked degree of l a b i l i t y with regard to task orientation, and that t h e i r verbal expressions of thinking showed a degree of diffuseness and articulateness. Research into the q u a l i t a t i v e nature of schizophrenic language has diminished s i g n i f i c a n t l y i n the l a s t decade. In turning away from t h i s e a r l i e r descriptive work, investigators, have become interested i n either the nature of the disturbance i n s p e c i f i c thought processes or the relationship between language cha r a c t e r i s t i c s and personality variables. 1 8 The analysis of language or communication patterns as a method of understanding psychological processes dates back to the early 1920's. ..The. f i r s t study in this area was undertaken by Buseman (In Pasamanick, 1958) who utilised what he termed:the verb-adjective ratio to evaluate , emotional stability., . Balken and Masserman ( 1 9 4 0 ) also utilized, the verb-adjective quotient* as an index of anxiety, a high ratio indicating more anxiety* Zipf (in, Pasamanick, 1958)who spent a great deal of time studying language, introduced the type/token ratio* This sort of analysis came to, be used, quite frequently. It consists of a particular item,, for example, the adjective (type.) divided by the total number of, language units that occur in a; particular passage (token). This ratio thus utili z e d many language elements, ; ., Johnson's (1944) program of research into language behaviour was undertaken to investigate the use of this ratio in studying the relationships between personality patterns and language patterns* Fairbanks (1944) who under-took the f i r s t study in this program;was primarily concerned with methodological problems involved in language- study, However, in the course of her work she applied the type-token ratio to the analysis of schizophrenic verbal language and found that this ratio was able to make quantitative, . differentiation betxfeen language samples elicited from schizophrenics and normals.. Mann (1944), u t i l i z i n g the same type-token ratio, found that i t provided a quantitative 19 d i f f e r e n t i a t i o n of written language as well« Cobb and'Lorenz (1954) have reported the results of several studies which dealt with grammatical structure and word frequencies as an index of the relationship of language habits to personality disorders, Their methodology involved t a l l y i n g word frequencies and counting various parts of speech and assigning them to categories.. The r e s u l t i n g patterns were analyzed and related to psychological t r a i t s , of various pathological groups studied. They conclude,that "on the basis of language patterns, each group shows some d i s t i n c t feature which i s open to psychological interpreta-t i o n * . . To t h i s extent we can look upon the analysis of a sample of language as a diagnostic a i d , . . No inference i s intended that any language t r a i t i s absolutely s p e c i f i c f o r a reaction type. .Instead, the suggestion i s offered that language habits may be a broad common denominator, as i n any other psychologically determined t r a i t , " (p*672) Calvert (In Pasamanick, 1958) was s i m i l a r l y interested i n the quantification of the speech of th© schizophrenic i n terms of developing a diagnostic as well as a prognostic t o o l . His basic measuring items were called thought units, word units and l e t t e r u n i t s . These units were used i n constructing several indices; communicative .efficiency, i ntensity of communication and conceptual richness. These were variously arrived at by d i v i d i n g thought units by the number of l e t t e r units i n the speech sample and so On. These indices were compared with cer t a i n 20 categories of Rorschaek items.an4 found to be "of d e f i n i t e c l i n i c a l value" i n demonstrating certain personality t r a i t s . . I%bry (1955) took into consideration the study of both the quantitative and q u a l i t a t i v e aspects of schiso-phrenic language* . Besides counting the number of words and parts of speech i n the response of subjects to a sentence completion test, judges c l a s s i f i e d these sentences as being "normal^ and ."schizophrenic" by ra t i n g them on a number-lof. more general language cha r a c t e r i s t i c s * . Although there were no differences between the groups i n terms of frequency of words.and parts of speech, the global judgments on the; • sentences discriminated at a s i g n i f i c a n t l e v e l between the schizophrenic and the normal. Qualitative differences were also obtained on such c h a r a c t e r i s t i c s as approximation, fragmentation, overihc L a f f a l (1961) outlines a technique Of language anal-y s i s , c a l l e d the analysis of contextual associates, which he applied to an examination of the changes i n the speech patterns of a schizophrenic patient over: the course of a year of psychotherapy. The analysis involves the tabulation of frequencies of associated, categories, each category :.. comprising words grouped on the basis of s i m i l a r i t y of reference.. He found that the technique "discriminated succes f u l l y between the styles of separate, individuals and also the productions of one indi v i d u a l i n manifestly d i f f e r e n t psychological states" (p.427)? Oare should be taken i n accepting these results since, i n the experiment reported, 21 the procedure was tested on only one subject. We shall now turn to a discussion of a number of ; , research studies carried put by Gottschalk and: his . associates (1955, 1957, 1958a,' 1958b, I960, 1961) who had developed a procedure for quantifying various psychological processes through the analysis of verbal behaviour* Their specific awareness of the importance of this area arose f i r s t from their own practical needs to specify "the psycho-lo g i c a l activity of subjects: during the interview process" (Gottschalk and Hambidge, 1955, p<*387). And out of this grew an awareness of the importance of this problem in psychological investigation as a whole* In their f i r s t investigation, Gottschalk and, Hambidge (1955) set out to determine whether any relationship existed between certain patterns of xirords occurring in spoken language and various psychological states* They proposed the systematic analysis of the "form" aspects of speech that were measured include such variable as rate of speech, frequency and duration of pauses, frequency of incomplete words, repeated words, f i l l words and the l i k e . In "content" analysis which dealt with meanings, relationships, concepts, and such, with the mail-emphasis placed on the scoring of words in the context in which they are used* ,,: Speech samples were elicited from subjects under-going psychotherapy by means of asking them to relate stories to four pictures from the Thematic Apperception Test (TAT) (Murray, 1943) and by asking them to t e l l about any 22 interesting and dramatic l i f e experiences, The responses were taperecorded. Four judges were asked to rank, on a scale ranging from 1 to 10, the verbal samples in terms of a classification of items reflecting certain psychological states. These ratings were compared to a numerical index representing the rate of use of certain "form" and "content" categories of words* This procedure was repeated weekly for a period of several months. The investigators found that variations in psychological states were associated with variations in the same direction in the values of their numerical index. Thus, they were able to demonstrate a relationship between language patterns and psychodynamic patterns as they emerged in patients over a period of time. In a second study, Gottschalk, Gleser and Hambidge (1957) set out to investigate differences in the speech patterns among groups of persons differing in psychiatric diagnosis. Speech samples were elicited in the manner used in their earlier study, from a non-specified group of hos-pitalized psychiatric patients and a matched control group of non-hospitalized subjects who had no apparent personality disorders. Form and content analysis of the speech samples was carried out, It failed to demonstrate any significant differences between these two groups of subjects. Although not significant, the results seemed to indicate certain patterns which reflected the severity of personality dis-organization. This study was hampered by a number of methodological deficiencies,, Gottschalk, Gleser, Daniels, and Block (1958) next 23 directed t h e i r attention to the speaking habits of patients with schizophrenic diagnoses since they showed.readily apparent deviant c h a r a c t e r i s t i c s i n t h e i r verbal communica^ t i o h . The.:hypothesis was entertained that, the relative-degree of schizophrenic disorganization was quantitatively related to the occurrence of certain themes i n schizophrenic speech... Using f i v e male chronic schizophrenics, an index of the r e l a t i v e degree of "personal disorganization" and " s o c i a l a l i e n a t i o n " of a patient at d i f f e r e n t times was obtained from weekly three minute verbal samples. B r i e f l y , the scores on the speech samples were obtained by (a) counting the frequency of occurrence of various content and form categories; (b) weighting each categorical reference with , respeet to i t s indication.of "more or less schizophrenic disorder;" and (c) adding the weighted scores and d i v i d i n g the sum by the t o t a l number of spoken words per verbal sample to give a numerical index of the degree of schizo-* phrenic disorder. The c r i t e r i o n to which these indices were compared was the average of the ratings by two i n -dependent ps y c h i a t r i s t s who rated the patients on three separate scales: interpersonal r e l a t i o n s , mental status and personal habit patterns* The raters showed good agreement oh judgments within an i n d i v i d u a l (that i s , consistency across the ratings f o r each patient on the three scales,) The average correlation was 0*80. But they showed p r a c t i c a l l y ho agreement as to the r e l a t i v e degree of schizophrenic disorganization of d i f f e r e n t i n d i v i d u a l s . 24 The .average co r r e l a t i o n here was 0.22. The rank order correlations between the estimates obtained from the verbal analysis method and the average c r i t e r i o n r a ting f o r each patient ranged' from 0*67 to 0.90 and a l l were s i g n i f i c a n t beyond.the 3$ l e v e l * . The o v e r a l l r e s u l t s suggest that counting, and : ; scoring c e r t a i n speech patterns of schizophrenics provides a quantitative estimate of the r e l a t i v e severity of schizo-phrenics disorder within an i n d i v i d u a l . However, the , procedure did not give s a t i s f a c t o r y results towards comparing the severity of disorder between d i f f e r e n t schizophrenic patients. The problem next considered by Qottschalk and : Kaplan (1958) was that of.estimating the r e l a t i v e i n t e n s i t y of psychological c o n f l i c t , s t a t e s i n a patient at d i f f e r e n t times by means of sampling the proportion of streptococcal bacteria present i n r e l a t i o n to other cultures found i n the throat. This procedure was validated against the verbal analysis procedure outlined i n the preceding report*. The ov e r a l l c o r r e l a t i o n between the rankings of the intensity of psychological c o n f l i c t on verbal behaviour and streptococcal bacteria scales.was not s i g n i f i c a n t * The-question i s often asked whether we are interested i n a straight research approach to the thought and language patterns i n schizophrenia or whether we should aim towards immediate c l i n i c a l applications of our research* Gottschalk and his research associates tend towards the l a t t e r . < 25 Evidence of such i s displayed i n a-study whose purpose i s that of testing the effectiveness of a method of assessing the action of psychoactive;drugs on the v a r i a t i o n i n schizo-phrenic symptomatic manifestations occurring i n the;.language hehaviour of patients (Gottschalk, Gleser, Springer, Kaplan, Shannon and Ross, I960)* The- method of e l i c i t i n g verbal samples and analyzing^ them i s also i d e n t i c a l to that previously outlined,, The investigators conclude; "The -method appears to have the s e n s i t i v i t y and v e r s a t i l i t y to evaluate the possible subtle effects of psychoactive drugs on mental processes." (p.639) •. The most recent study by these investigators (Gott-schalk, Gleser, Magliocco, and D'Zmura 1961) i s a precisely controlled experiment whose purpose was primarily that of r e f i n i n g and c l a r i f y i n g e a r l i e r findings. A number of apparent d e f i c i e n c i e s of e a r l i e r studies are corrected here: t h e i r experimental groups are larger and more c l e a r l y specified, t h e i r r a t i n g scales have been revised and t h e i r v a l i d i t y increased; the categories of verbal analysis have been revised to allow for i n t e r as well as i n t r a - i n d i v l d u a l comparisons; and t h e i r analysis considers the p o s s i b i l i t y that certain speech habits previously considered unique to schizophrenics.might be found i n other psychotic disturb-ances as w e l l . The findings of t h i s study indicate that "the verbal behaviour measure employed ... i s a v a l i d measure: of the r e l a t i v e severity of the schizophrenic syndrome from patient to patient as well as of the fluctuations 26 within, a p a t i e n t % ; (p*,112) , There are a number;of points a r i s i n g out of the immediately, proceeding studies which should "be considered further. F i r s t , the procedure employed i n constructing the c r i t e r i o n rating;scales i s questionable. .One of the cardinal points .stressed in. the "building of rating scales i s that they have equ a l l y d i s c r i m i n a b l e points which can only be arrived: at a f t e r rigorous item analysis. The present scales do:not appear to have undergone such .structuring.* Items used, i n these scales, were, i n t u i t i v e l y selected from des-criptive, psychiatry and the units of the scales were ; a r b i t r a r i l y selected. .^ Further,-., the.;,ra-t^ er'..©on?*'. yentional*: Why,, then was i t necessary at a l l to construct and use a scale whose v a l i d i t y as well i s questionable? Under these circumstances, v a l i d and r e l i a b l e c l i n i c a l ratings of the severity of schizophrenia, as demonstrated by Hunt, Walker and Jones (I960) might p r o f i t a b l y take the place of the questionable c r i t e r i o n scales used. , ; Second, the analysis of the verbal samples for some of the subjects was done by only one investigator and no check of t h e . r e l i a b i l i t y of scoring f o r these passages was made* I t i s essential that the inter-scorer r e l i a b i l i t y be properly ascertained before we can f u l l y accept the findings of, t h i s study*-Further, there i s perhaps a danger involved i n making generalizations, as these investigators do, from the small samples which they used. When dealing with a language 27 study, large samples, should be/employed to obtain data which may be generalized to larger populations, Pasamanick (1958) suggests that the scope of these studies•could be enhanced by applying the concepts'of information theory to the study of schizophrenic language.. He suggests that redundancy, amount of information and hoise i n the system, might well be applied*. One f i n a l point might be using the norms of speech and language behaviour of a group of normals as a base point upon which to make a comparison of the deviation of schizo-phrenic speech* The present investigation has taken some of these factors into account i n i t s design, f i r s t , p sychiatric ratings whose v a l i d i t y i s implied and whose r e l i a b i l i t y i s demonstrated are employed here. No attempt has been made to construct a c r i t e r i o n r a ting scale. Second, Glose Procedure (Taylor, 1953) the verbal behaviour measure u t i l i z e d , i s one whose theory i s p a r t i a l l y derived from modern information theory. Further, the problem of i n t e i v score r e l i a b i l i t y f o r the analysis of, verbal samples i s e n t i r e l y avoided. F i n a l l y , Cloze also makes use of the norms of ordinary speech because i t r e f l e c t s the degree of correspondence between the language habits of the normal group f i l l i n g i n the Cloze passages and those of the schizo-phrenic group from whom the verbal samples were e l i c i t e d * An elaboration of these points i s contained i n the sections which follow. 28 Cloze .Procedure; . • ./ , ( „^,:- v s. ;;.' The Cloze Procedure (Taylor, 1953) i s a psyeholin-g u i s t i c technique for measuring the effectiveness of communication. It was o r i g i n a l l y designed f o r use i n assessing the effectiveness of other communication media as well. And at present, studies of i t s application to the understanding of the verbal behaviour of mental patients are i n progress, "Cloze" i s derived from the term "closure", the term often used to ref e r to the.notion that humans tend to perceive a f a m i l i a r pattern as a whole, even when parts of i t are missing or obscured; f o r example, seeing a broken c i r c l e as a whole by mentally closing up the gaps. One can complete the broken c i r c l e because i t s shape i s so fa m i l i a r that, although part of i t i s actually missing, i t can be recognized nonetheless; The same p r i n c i p l e applies to language.' However, the sentences of normal language are complex and are made up of many sub-patterns. One must know not only the meanings of separate words, but also the meanings of given combina-tions of them. That i s , one must know what the sentence pattern means as a whole i n order to be able to complete any missing parts of i t ; Cloze Procedure involves mechanically d e l e t i n g ' every nth word of a passage and having subjects t r y to f i l l i n the'blanks by guessing from the context of the remaining words, what the missing words should he, The number of times original words are correctly replaced constitutes a cloze score* By contrasting the cloze scores of various passages one i s able to ascertain which passages are more readily understandable relative to the others* The functional unit of measurement is the "cloze unit," which is defined by Taylor (1953) as: Any single occurrence of a successful attempt to reproduce accurately a part deleted from a message (any language product) by deciding, from the context that remains, what the missing part should be (p.416). Cloze Procedure i s defined as: A method of intercepting a message from a "transmitter" (writer or speaker), mutilating i t s language patterns by deleting parts, and so administering i t to "receiver" (readers or listeners) that their attempts to make the patterns whole again potentially yields a considerable number of cloze units (p.416), The main theoretical contributions to the notion of Cloze Procedure have come from the concepts of "total language context", "dispositional mechanisms", and s t a t i s t i c a l random sampling, language behaviour depends upon- "total context". This includes everything that tends to motivate, guide, assist or hinder behaviour* It involves verbal factors - grammatical s k i l l s and multitudes of symbols - and non-verbal ones such as fears, desires, past experience, intelligence and the like* The a b i l i t y to identify, learn or produce any language symbol depends heavily on the variable degree to which i t i s associated with everything 3 0 else by larger and.meaningful or f a m i l i a r combinations. Taylor (1953) relates the "redundancies" and "tran-s i t i o n a l p r o b a b i l i t i e s " of language to the development of " d i s p o s i t i o n a l mechanisms" that play an -important-role- i n both transmitting and receiving messages. "Redundancy" refers to the actual or implied r e p e t i t i o n of meanings and the internal' t i e between words. "Transitional probability" suggests that since some-words are more l i k e l y than others '* to appear i n certain patterns or*sequences, some transitions from one word-to the'next are therefore more probable than others. In learning to "think i n a language," an- individual develops many complex s k i l l patterns which stand for innumerable kinds of meanings and tend to become so ' automatic that they simply "present themselves" i n pertinent' sit u a t i o n s . These habits r e f l e c t the redundancies and t r a n s i t i o n a l p r o b a b i l i t i e s of the language patterns these s k i l l s involve. ' ' Habits of expression take over most of the work of translating an individual's meaning into an Organized series of language symbols f o r transmission to others. Similarly, h i s habits of reading or l i s t e n i n g cause him to anticipate words almost automatically. The Cloze method seems to deal with p a r a l l e l meaning - pattern relationships. It takes a measure of the likeness between the meanings and patterns a writer has used and the meanings and patterns the reader i s an t i c i p a t i n g while he i s reading. When words come i n se-quences that best match the existing receiving habits of the reader, he understands with l i t t l e effort, When the symbols appear .in less familiar sequence, comprehension i s slower and less s»re.*v; Sufficiently, improbable patterns seem: like nonsense; they,do not stand for anything in his experience. :, Random deletion method, or the equivalent > every nth deletion,: takes1 3^to account proportionally air kinds of words to the extent that they occur in any particular language sample* Thus words that are hard to replace are just as likely,to occur as, words that are easier to replace, and^vice^versa.,. • -: • Taylor (1953) reports the findings of several intro-dUctory studies on Cloze Procedure*. Only those results pertinent to t h e present study are discussed h e r e * - Various kinds of deletion systems were tried and an every f i f t h word deletion system was found to be more efficient than others tested w, The minimum number of every nth blanks necessary to yield a dependable equivalent to a random system was found to be fifty.', The presentation order of the passages had virtually no effect on their scores; Evaluating and scoring • synonyms did not improve t h e efficiency of:, the :'. procedure; Thus,, i t was decided to score only exact replacements as correct; In this way the problem of coder r e l i a b i l i t y • entirely avoided ;. "Onlya' relativel-y,-:small- •'; • number of cloze raters are required;. It was fbund,that the cloze scores for six subjects ranked the cloze passages in almost exactly the same way as did the performanbe of : t eighteen subjects (r = 0.98). Taylor (1956) discusses a d d i t i o n a l uses to which Close Procedure has "been put since i t s introduct ion several years e a r l i e r » The method can be appl ied to auditory as wel l as v i s u a l communication and to languages other than English* It appears to be ah e f fec t ive gauge of the r e l a t i v e comprehension of readers, of general i n t e l l i g e n c e , of s p e c i f i c t echn ica l knowledge and of success i n l earn ing . In a prel iminary report of progress on a current ser ies of exploratory studies deal ing with the app l i ca t ion ., of Close to the study of the verbal behaviour Of mental pat ients , Tay lor (1963) found that the responses to a set of Cloze items were quite sens i t ive to i n d i v i d u a l differences among these pat i ents . Inasmuch as the report was "more of a demonstration than ah experiment" (p*13) the resu l t s are ' h ighly t en ta t ive . No attempt was made to d iscr iminate between d i f f e r e n t l y defined groups on the basis of the f indings obtained. The problem under consideration i n th i s inves t iga -t i o n i s that of attempting to develop an e f fec t ive psycho-l o g i c a l measuring device for assessing the extent of disturbance i n schizophrenic t h i n k i n g . Th© Cloze Procedure was o r i g i n a l l y selected f o r use i n t h i s study because of i t s p o t e n t i a l a p p l i c a b i l i t y to the problem. It has been found to be useful i n measuring the effectiveness of communica-t i o n i n general; and more p a r t i c u l a r l y i t has proven to be an e f fec t ive t o o l for assessing and rank ordering the verba l 33 productions of normals i n terms of i t s comprehensability* It would therefore seem probable that this procedure would also be capable of rank ordering the comprehensability of language samples e l i c i t e d from a group of schizophrenic patients. Shis rank ordering of comprehensability of schizo-phrenic language i s presumably also a rank ordering of the severity of thought disturbance* Cloze i s simple and straightforward; i t does not require professionally trained experts for administration; and i t s data i s e a s i l y quantifiable. The fact that presenta-t i o n orders have no important effect on cloze scores leads to s i m p l i f i c a t i o n of experimental design. The problem of coder r e l i a b i l i t y that plague so many other procedures of language analysis i s e n t i r e l y avoided. The procedure i s r e l i a b l e . The p r a c t i c a l advantages of such a procedure are obvious. S i g n i f i c a n t also i s the fact that Cloze appears to be a measure of the aggregate influences of a l l factors which interact to a f f e c t the degree of correspondence between the communication patterns of transmitter and receiver. This i s es p e c i a l l y important since i t would be almost impossible to specify a l l of the variables which are interacting i n t h i s p a r t i c u l a r s i t u a t i o n . It i s f o r these reasons that Cloze was chosen as a desirable and reasonable approach to the study of thought disorder i n schizophrenia. Reliability of Psychiatric Ratings 34 The unreliability of c l i n i c a l judgments was parties ularly widespread u n t i l very recently. Prior to a decade ago, l i t t l e had been done towards a systematic analysis of the factors responsible for this unreliability in order that i t may be understood,, controlled and corrected. Utilizing the findings of the research in this area, one i s presently able to overcome shortcomings of earlier attempts at rating and arrive at highly reliable psychiatric evaluations. •In the f i r s t of a series of studies in this area, Amhoff (1954) assumed that both experience of the rater and anchoring of the rating continuum would have some i n f l u -ence on judgments of psychiatric disturbances. He found no anchoring effects and the findings with regard to experience of raters were a complete reversal of the predicted direc-tion. Professional clinicians were least reliable and undergraduate students were most reliable* He explained these paradoxical findings by suggesting that increased training and professional experience provides the clinician with multiple frames of reference against which to evaluate behaviour and that these may have provided a diversity on which the judgments were based. However, in subsequent studies, Hunt and Arnhoff (1956) and W.A. Hunt, Jones and E . B . Hunt (1957) found higher interr-rates r e l i a b i l i t y among clinicians than among trainees and undergraduate students. 35 Increased rater r e l i a b i l i t y resulted when more specific rating instructions were used. The ratings of experienced clinicians were also found to be more consistent over time than both the ratings of the trainees and the undergraduates. Hunt and Arnhoff (1955) asked a group of experienced clinicians to judge "how schizophrenic" vocabulary items from the Wechsler-Bellevue scale were, on a seven point scale of severity of pathology. They found inter-rater r e l i a b i l i t y ranged from 0.73 to 0.92 for vocabulary items and from 0.64 to 0*88 for comprehension items. Their attempt at constructing a scale for rating the severity of schizophrenic disturbance was ''admittedly rough" since l i t t l e attention x*as paid to theoretical problems involved in measurement scales. In view of the high r e l i a b i l i t y demonstrated when experienced clinicians were asked to rate "how schizophrenic" a series of responses to vocabulary test items were, Hunt and Jones (1958) investigated the effect of asking judges to rate specific dimensions of the schizophrenic disorder instead of the general dimension. The dimensions used were potential intelligence, commUnicability and the concrete-abstract continuum. Reliability tended to diminish slightly on the more specific dimensions but "remained satisfactorily high to indicate the value of such scaling methods in c l i n i c a l practise" (p.,239),. Inter-rater r e l i a b i l i t y on the general dimension was 0.94 while the average r e l i a b i l i t y on the specific c r i t e r i a was 0.83. 36 The above investigations studied the r e l i a b i l i t y of c l i n i c a l judgments when judges were asked to rate schizo-phrenic test responses on a seven point scale. The addition of an external criterion in a study by Hunt, Walker and Jones (I960) makes i t possible to assess the validity of such ratings? A criterion group of schizophrenic patients differing in the severity of their disorder was given the vocabulary and comprehension subtests of the Wechsler,-Bellevue scale. .These test responses were then independently rated by another group of clinicians in terms. Of the severity of schizophrenia. Using these raw scores and establishing a rank order from them made possible a comparison with the original criterion ranking for severity of disorder,;. For vocabulary, the correlation was 0.57 and for comprehension, 0.69. The f i r s t just misses significance at the 5% level and the second easily attains significance at that level. The investigators cautiously conclude that "the judges' ratings of the severity of pathology are valid within the limitations of our criterion*," (p»392) Jensen and Morris (1957,: I960) investigated factors which contributed to the unreliability of ancillary psychiatric evaluations, that i s , psychiatric evaluations by other than psychiatrists and psychologists. They have enumerated a number of factors leading to increased r e l i a b i l i t y : well-defined c r i t e r i a ; specific instructions to insure, main-tenance of intended frame of reference; training of raters; sufficient time aside from regular duties to make careful 3 7 observations on which to base evaluations; s u f f i c i e n t motivation f o r the task; and, f a m i l i a r i t y with the objectives of the research. Implications from the findings of research i n t h i s area suggest that i f due care i s exercised i n the rating procedure, r e l i a b l e psychiatric evaluations can be obtained. The recommendations and findings of the studies cited have been incorporated as f a r as possible into the rating procedures used i n the present investigation. CHAPTER I I I PROCEDURE In the present investigation an attempt was made to determine whether or not the Cloze Procedure could v a l i d l y he used to assess the r e l a t i v e severity of thought disturb-ance' i n schizophrenic patients. The -procedure employed here involved three somewhat separate steps. F i r s t , language samples were e l i c i t e d from twelve male schizo-phrenic patients i n response to TAT cards presented by the Interviewer. Second» these patients were then independently rated by two p s y c h i a t r i s t s as to the degree of schizophrenic thought disorder. F i n a l l y , the schizophrenic language samples were prepared f o r use i n the Cloze Procedure. That i s , equal length samples of speech from each patient were transcribed from tape recordings and mimeographed, with every f i f t h word being omitted and replaced by a standard length blank. These mutilated passages were then presented to a group of undergraduate psychology students, who were instructed to f i l l i n the blanks by t r y i n g to guess the correct words from the context of the remaining words. The students were, i n fact, r a t i n g the severity of schizo-phrenic thought disturbance of these patients by means of the Cloze Procedure, i n terms of the number of corr e c t l y f i l l e d - i n words i n the language samples e l i c i t e d from each patient. The c r i t e r i o n measure against which the Cloze 29 ranking of severity of thought disorganization was validated was the average rating of t h e t w o p s y c h i a t r i s t s . E l i c i t i n g Verbal Samples ( i ) Subjects Twelve male schizophrenic patients, with no compli-cating secondary diagnosis and who displayed c l i n i c a l l y manifest thinking disturbances were used as the subjects from whom the samples of schizophrenic language were e l i c i t e d . The method of rank ordering the patients as to the degree of thought disorder required that a l l subjects included i n the study be r e l a t i v e l y equally well known by the two ps y c h i a t r i s t s who served as r a t e r s . Patients whose personal physician was serving as one of the raters would be better known to him than to the other r a t e r and would therefore not meet t h i s requirement. Thus, patients were selected from psychiatric services other than those d i r e c t l y served by either of the raters. The patients were chosen from the male schizophrenic population of two wards of Crease C l i n i c by two p s y c h i a t r i s t s who were not p a r t i c i p a t i n g i n the study i n any way other than i n the o r i g i n a l s e l ection of patients. None of the patients was currently undergoing a pro* gram of Electroconvulsive Therapy (E.C.T.) treatments, but a l l were receiving some medication. However, a moderate 40 dosage of t r a n q u i l i z i n g drugs was r e l a t i v e l y consistent across a l l patients. Due to p r a c t i c a l considerations, more exact control could not be exercised over t h i s f actor. Pour patients o r i g i n a l l y selected proved untestable due to reslstiveness, muteness, bizarre behaviour and the l i k e . The mean, range and standard deviation of the age of schizophrenic patients was 29 .1 years, 19 - 41 years, and 7.6 years, respectively.- The mean education was 9.6 years, with a range of 7 to 12 years and a standard deviation of 1.6 years. ( i i ) Method Bach patient was accompanied to the interview room (the doctor's o f f i c e ) on the ward and introduced to the experimenter by the charge nurse. The experimenter began the interview by explaining to the patient that he was interested i n finding out something about the people who were i n the h o s p i t a l . This explanation of the reason f o r conducting the interview proved to be e n t i r e l y s a t i s f a c t o r y . Hone required a further explanation of the purpose of the interview. The experimenter then engaged the patient i n casual conversation about the patient's employment, l e i s u r e a c t i v i t i e s and the l i k e . Every e f f o r t was made to r e f r a i n from discussing the patient's i l l n e s s d i r e c t l y , As f a r as possible, the pre-test time with the experimenter was kept as consistent as possible between subjects, 41 A small tape recorder was in, p l a i n view on the interviewer's desk.- Permission to record the interview was requested frOm each patient. This was granted i n every case. (This procedure was followed to comply with hospital regulations), , The experimenter then handed the patient a 4" x 6" white index card on which the instructions f o r t e l l i n g s t o r i e s to TAT cards were printed. The experimenter read ~ them aloud and asked the patient to follow along on his own card. . These instructions were as follows: I am going to show you some interesting p i c -tures, one at a time; and I want you to make up as dramatic and interesting a story as you can f o r each. You can make up any kind of story you please. F i r s t , t e l l me what has led up to the event shown i n the picture; next, describe what i s happening at the moment, what the people i n the story are thinking and fe e l i n g ; and then give the out-come of the story. You may take as much time as you want, so you don't need to hurry. Each succeeding TAT card was introduced as follows: How see how well you can make up a story about t h i s picture. Cards 1, 3BM, 8BM, 11, 18GF, 18BM were presented i n that order. TAT cards were used f i r s t because they provide a constant stimulus f o r the e l i c i t a t i o n of verbal responses from the schizophrenic patients, and second, because i t was f e l t they would e l i c i t s u f f i c i e n t l y long verbal samples to be used In the Cloze Procedure, i . e . , a minimum of 250 words (Taylor, 1953). 42 When.the interview was. concluded, the experimenter thanked the'patient <andv. accompanied him to the dayroom on the ward* This procedure was followed for a l l subjects interviewed.* . Establishment of the Criterion Before beginning to select any;subjects: for our study, it.was necessary to provide a detailed, definition of schizophrenic thought disorder so that the. psychiatrists involved,would a l l be using the same cri t e r i a , both in the original selection of the patients and in ranking the relative severity of thought disorder. This definition was variously derived,from factor analytic, studies of psychotic, syndromes by Loir, Jenkins and O'Connor (1955» 1957, 1962) and from c l i n i c a l descriptions as outlined in several standard psychiatric texts (Ewalt, Strecker and Bbough, 1957? Gregory, 1961; Henderson and Gillespie, 1956). The definition originally presented as subject to consider-able revision by the psychiatrists, who in conference, . arrived at one "check l i s t " (see Appendix A) that a l l had agreed would serve as a guide in the selection and assess-ment of thought disordered schizophrenic patients. The two raters carried out several familiarization t r i a l ratings on small groups of patients, none of which were included in the f i n a l schizophrenic sample. After each rating t r i a l individual differences between the judges were discussed* Due to the small number of t r i a l runs no 43 systematic check oh t h e i r r e l i a b i l i t y was made but a work-able consistency between the raters was evident. In the actual : study, the raters interviewed each patient i n d i v i d u a l l y on the day the patient gave the experimental verbal sample* On the basis of t h i s interview plus additional information adjudged from d i r e c t observation on the ward, the raters independently rank ordered the patients f o r severity of thought disturbance according to • the instructionsS The names of the patients have been arranged i n alphabetical order. Your task i s to rank order these patients according to the severity of thought disorganization, with the low end of the scale representing^minimal disturbance and the. high end of the scale representing maximal disturbance*" In making these ratings we,-are asking you to concentrate upon the severity of : the pathology exhibited i n the patient ,;s Verbal statements themselves. We are not interested In such things as prognosis, chronieity, therapeutic indications, etc. Please attempt to use the previously agreed upon d e f i n i t i o n of thought disorder as a guide i n making your judgments; ' The r e l i a b i l i t y c o e f f i c i e n t s of the ratings by the two p s y c h i a t r i s t s was computed by Spearman's Rank Correla-t i o n Goefficient method (rho).- The two ratings were then averaged and these were arranged for each patient i n the order from the lowest spore (least sick) to the highest score ( s i c k e s t ) . A f i n a l psychiatric rank order was compiled frOm these averaged ratings. 4 4 Gloze Procedure ; ( i ) Cloze Raters Fifteen-undergraduate summer session psychology students at the University of B.C. volunteered to f i l l i n the mutilated schizophrenic language passages, that i s , to rate schizophrenic thought disorder on the Cloze Procedure., The mean age, range and standard deviation of t h i s group was 26.7 years, 21 to 4 3 years, and 3 * 9 years respectively. Their mean education was 16,5 years with a range of 16 to 18 years and a standard deviation of 0.81 years* ( i i ) Method In preparing the passages of schizophrenic language for administration to the Cloze raters the following procedures were carried outJ ' (a) The tape ;recorded speech samples were transcribed by the experimenter. -(b) Standard length passages of 3 0 0 words were taken from the language samples of each patient. (e) Sixty words were mechanically deleted from each passage by deleting every 5th word, without any regard f o r the function or-meaning of s p e c i f i c words. (d) Bach mutilated passage was reproduced (type-written and mimeographed) with a standard ten space blank (so the length would not influence the guessing) i n place of every missing word. 45 (e) ' Since order effects are not important, these passages were simply assigned to an order by random place-ment (drawing l o t s ) . (f) Copies of a l l reproduced mutilated passages (Cloze forms) and answer sheets (Appendices B and C) :were given to a l l subjects with, the following instructions: This i s part of a study dealing with the nature of the thought processes of schizophrenic ' patients; one of the ways i n which the pathology *: of schizophrenia may express i t s e l f i s through disordered thinking which results i n a t y p i c a l , ; , deviant or otherwise "abnormal" speech patterns. I have here several mimeographed sheets with a number of passages on each. Each of these passages i s a sample of schizophrenic speech from a d i f f e r e n t patient. The extent of the pathology exhibited by these patients i s not uniform, however. Your task i s to try to f i l l i n a l l the blanks i n these passages by guessing, from the context of the remaining words, what the missing word should be, Replace each blank with a single word. Contractions (for example "don't" f o r "do not") are permissable. Please work as conscientiously as you can. Please do hot confer with anyone else x*ith regard to f i l l i n g i n these passages, The passages were scored and one point was given each time an o r i g i n a l word was c o r r e c t l y replaced. The t o t a l Cloze score (the sum of the correct responses f o r a l l Cloze raters) f o r the various passages were arranged i n order from the highest score (least thought disturbance) to the lowest score (most thought disturbance). These scores were then converted to a rank ordering on a twelve point scale. This Cloze rank ordering of thought d i s * organization was then compared with the c r i t e r i o n rank 46 ordering, by Spearman's rho.. In analyzing the data, several methodological features of the Gloze Procedure were also examined. As a check on the:internal consistency of the Gloze Procedure i t s e l f and as a check on the adequacy, of language sampling, Cloze rankings on each of three equal length subsections, of the t o t a l passage e l i c i t e d from each patient were -correlated with each other by the Coefficient of Concord-ance method.. As a further check on the i n t e r n a l con-sistency within each subsection, the degree of correspondence among a l l the Cloze raters oh each subsection was computed by the method of Interclass Correlation* The r e l i a b i l i t y of Cloze raters was assessed as well by correlating the rank orderiags of a l l the raters by means Of the Interclass Correlation method and by corr e l a t i n g the rank orderings obtained with f i v e , ten and f i f t e e n cloze raters by means of the Coefficient of Concordance method.. CHAPTER IV RESULTS AND DISCUSSION With a c l e a r l y set down c r i t e r i o n f o r assessing the severity of thought disturbance and a general consistency i n rating.during several t r i a l runs, i t was expected that a reasonably high l e v e l of agreement would be reached by the raters* The i n t e r ^ r a t e r r e l i a b i l i t y c o e f f i c i e n t was computed by Spearman's rho and found to be 0.728 which i s s i g n i f i c a n t at the 0*01 l e v e l of confidence. The psychiatric rank ordering of patients i s set out i n Table 1* The high l e y e l of agreement between the raters warranted averaging the ratings i n order to a r r i v e at a single f i n a l psychiatric rank ordering which would serve as the v a l i d a t i o n c r i t e r i o n measure. The extent of agreement found i s c e r t a i n l y respect-able. The fact, however, should not be overlooked that the task of making a judgment of a s p e c i f i c dimension of schizophrenia, such as thought disorder, i s a more d i f f i c u l t one than ranking on the general dimension of schizophrenic pathology as Hunt and Jones (1958) point out. Several p r a c t i c a l factors may also have contributed to lowering the agreement between the raters. F i r s t , the administra* t i o n of medication could not be precisely controlled. Second* the ps y c h i a t r i s t s were not always able to interview 48 TABLE 1 PSYCHIATRIC RAM ORDERING OF PATIENTS IN TERMS OF DEGREE OF DISTURBED THINKING Patient Rater 1 Rater 2 Averaged Ranking F i n a l Rank -Order A A 2 . 3 B 2 • 3 2.5 .• , \ l ' • . C • • 6 8 7 • ' 7 D : . 12 10 11 ;. 11 r... . E ••: , .8 . 7 .. 7.5 ..'8- - : F 1 5 3 • 2 * 5 • • G 5 ,, • 6 5.5 H, 4 3.5 4 J 11 11 • 11 ,' 7 1 4 5 , - • i . 9 12 10,5 M 10 9 9.5 9''•;;/;V:;^V . 4 9 the patients Immeditately a f t e r one another. Thus, because of s l i g h t fluctuations i n a patient's behaviour oyer the course of several hours and because of intervening medications, moderately d i f f e r i n g degrees of severity of disturbance would be exhibited. And perhaps equally important i s the fact that both psychiatrists had only limited t r a i n i n g as r a t e r s . More extensive t r a i n i n g of the raters was not possible because of t h e i r commitment to regular hospital duties. Nonetheless, having taken those precautions which were possible, as outlined i n Chapter II under " R e l i a b i l i t y of Psychiatric Ratings", the r e l i a b i l i t y remained s a t i s f a c t o r i l y high, % t i n g s of the severity of thought disturbance on the Cloze Procedure were ascertained by t o t a l l i n g the number of correct responses f o r a l l f i f t e e n Cloze raters on each patient's language samples and arranging them i n order from the highest score, indicating l e a s t thought disturbance, to the lowest score, representing most thought disturbance. These scores were then converted to a rank ordering on a twelve point scale. These are shown i n 'Table 2, '"' Correlations were run between the rank order of the twelve patients based on the Cloze Procedure and the rank order obtained i n the external c r i t e r i o n . The correlation c o e f f i c i e n t " r " was 0,812 which i s s i g n i f i c a n t at the 0.01 l e v e l * Keeping i n mind the small number of patients and the fact that only two judges rated them on the c r i t e r i o n . 50 TABLE 2 TOTAL CLOZE SCORES ANB THEIR 'CORRESPONDING RANK ORDERING Patient Cloze Scores in Cloze units * Cloze Ranks A 480 3 B 440 C 393 7 D : 355'' 9 E 359 " 8 F 529 2 G 466 4 H 411 6 J 272 11 K 546 L 331 10 M 267 12 * word. A Cloze.Unit represents one correctly replaced 51 i t may be cautiously concluded that the Gloze ratings of schizophrenic thought pathology are v a l i d within the lim i t a t i o n s of our c r i t e r i o n . This also suggests, the p o s s i b i l i t y : t h a t Gloze ratings may eventually be u t i l i z e d as scores f o r quantitatively estimating the severity of thought disorder present i n schizophrenic patients.;. The Gloze Procedure thus appears to be sensitive to the same factors that contribute to the disordered thought processes of schizophrenics as are the psychiatric evaluations. Since schizophrenic thought processes are externally manifest i n the patients' language patterns, psychiatric ratings are of necessity then primarily based on the patients' verbal behaviour. It i s thi s same verbal behaviour upon which the Cloze assessment of severity of disturbance was made. The advantages of the present experimental method i n estimating the degree of schizophrenic disorder are: f i r s t , Cloze i s a procedure that i s simple and st r a i g h t -forward and easy to administer; professional t r a i n i n g i s not required f o r i t s use; the problem of inter-scorer r e l i a b i l i t y i s avoided because .only exact replacements are scored as being correct; f i n a l l y , i t involves only one modality of human functioning, i . e . , speech, instead of a complex c l u s t e r of many functions such as are involved i n other assessment procedures, and yet, within one indi v i d u a l , verbal behaviour appears to provide a v a l i d measure of a 52 conglomerate of functions . Taylor (1953) suggests ,that Gloze appears to. "be "a measure of the aggregate influences of a l l . f a c t o r s which interact to a f f e c t the degree of correspondence between the language patterns of transmitter and. receiver*" (p*432)* The lim i t a t i o n s of our experimental method include the following: the procedure does not enable us to assess the severity of disturbance i n patients who are either unable o r unwilling to t a l k ; next, i t has yet to be proven whether or not the procedure can s a t i s f a c t o r i l y assess . v a r i a t i o n s w i t h i n a person over a period of time* I f i t can shown that the Cloze Procedure i s sensitive to subsequent patient progress,, might be possible that such ratings may a l s o have some predictive potential* Beyond the analysis of the data for determining the v a l i d i t y of the Cloze Procedure f o r the assessment of the r e l a t i v e severity of schizophrenic thought disorder, a number of other t h e o r e t i c a l features of the technique i n i t s present application are considered. In examining the i n t e r n a l consistency or r e l i a b i l i t y of the Cloze Procedure, the o v e r a l l rank co r r e l a t i o n between the ratings on three equal subsections of twenty blanks (see Appendix F) each was computed by the Coefficient of.Concordance method; It yielded a "W" of 0*632 which i s s i g n i f i c a n t at the 0*05-l e v e l of confidence* This r e l i a b i l i t y c o e f f i c i e n t i s : moderately high. The r e l i a b i l i t y within each t h i r d was also examined, by means of the Interclass Correlation method. 53 (These results are set out i n Table 3* Variations i n the i n i t i a l set with which the Cloze raters approached the task may have contributed to the only moderate consistency which existed i n the f i r s t t h i r d . R e l i a b i l i t y increased as the raters apparently attained some degree of f a c i l i t y ifith the task i n the middle t h i r d . It i s suggested that the s i g n i f i c a n t l y decreased r e l i a b i l i t y i h the f i n a l t h i r d , which just f a i l e d to reach significance at the .05 l e v e l , l i k e l y resulted from boredom and f r u s t r a -t i o n effects which probably were operating because of the nature of the task. The r e l i a b i l i t y of Cloze raters was assessed by correl a t i n g the rank orderings by a l l the Cloze raters by means of the Interclass Correlation method. This yielded, a c o e f f i c i e n t "R" of 0.792 which was si g n i f i c a n t at the .01 l e v e l , When the rank order data are represented by the average Cloze rank order of three groups of f i v e raters each the o v e r a l l rank correlation "W" for the three groups of raters was 0.98. This .is s i g n i f i c a n t at beyond the 0.01 l e v e l of confidence. For p r a c t i c a l purposes, the l a t t e r method of analysis i s perhaps a more l o g i c a l one to use because Taylor (1953) suggests that Cloze data are not meaningful when ranked by less than f i v e or six raters. Table 4 shows that the average Cloze rank orderings f o r each of the three groups of f i v e raters i s almost i d e n t i c a l . The exceedingly high r e l i a b i l i t y of Cloze 54 TABLE 3 CORRELATION COEFFICIENTS FOR INTERNAL CONSISTENCY Portion of F i r s t Middle F i n a l Total Passage Third Third Third Correlation •'6.69* 0.81** 0.45 0.632-Coefficient * S i g n i f i c a n t at .05 l e v e l ** Significant at .01 l e v e l 55 TABLE 4 AVERAGE CL02E RANK ORDBRINGS WITH DIFFERENT GROUPS OF RATERS Patients Avg.Rank Avg.Rank Avg.Rank Order of Order of Order of t F i r s t Five Second f i v e Third Five Raters Raters Raters A 3 4 3 B 5 5 5 G 6 7 ? D 8 ' 9 9 E 9 8 ' 8 F 2 1 # 5 2 G 4 3 4 H 7 6 6 J 1 0 1 1 1 2 K 1 1 . 5 1 L 1 1 . 5 " 1 0 1 0 M H . 5 ' 1 2 1 1 56 raters would suggest that i t would "be safe to use a smaller group of raters and s t i l l expect a r e l i a b l e ranking of the patients by the Gloze Procedure. Taylor (1953) also found that a small number of Cloze raters produced highly r e l i a b l e r e s u l t s . He found that "scores from six subjects predicted almost perfectly how l a r g e r , groupsof subjects would rank the same passages" (p.427)> ' The method of assessing the degree of mental disturbance described i n this study could perhaps be adapted f o r use i n research attempting to evaluate the effect Of psychological and pharmacological therapeutic measures on various classes of personality disorders which are i h part characterized by some d i s t i n c t i v e or otherwise deviant form of verbal behaviour. I f this suggestion should prove to be of some value, then s i g n i f i c a n t progress w i l l have been made beyond the presently r e l i e d Upon subjective c l i n i c a l , observations, towards a more precise method of quantifying s p e c i f i c psychological processes. The need to move away from the sole reliance upon the mere i n t u i t i o n of a c l i n i c i a n to a more objective standard has been a pressing one since the beginnings of modern c l i n i c a l psychiatry. CHAPTER V SUMMARY AID CONCLUSIONS The purpose of the present investigation was to.: assess the a p p l i c a b i l i t y of a recently developed psycho-l i n g u i s t i c tool.known as the Gloze Procedure f o r estimat-ing the r e l a t i v e severity of thought.disorder i n schizo-phrenic patients* The hypothesis of this study arose put of a widely held b e l i e f that the analysis of various speech patterns i s a f r u i t f u l approach to the assessment of personality variables* This investigation was s p e c i f i c a l l y designed to determine whether or not the Cloze Procedure was v a l i d i n assessing differences between schizophrenic patients i n the;relative severity of thought ' disorganization. In the i n i t i a l phase of the investigation, verbal, samples were obtained from twelve male schizophrenic.. patients i n response to TAT cards. These patients were selected from the male schizophrenic population at Crease C l i n i c by two psy c h i a t r i s t s who were not d i r e c t l y involved i n the experiment. Two other psychiatrists acted as judges and independently rank ordered the twelve patients as to the r e l a t i v e degree of thought disorder. The schizophrenic language samples were prepared f o r use i n the Cloze Procedure i n the following manner. Equal length language samples from each patient were 58 transcribed from the tape recordings made during the interview session. These passages with every f i f t h word replaced by a standard length blank, were mimeographed and then presented to a group of undergraduate psychology students who -were instructed to f i l l , i n each;blank by guessing the missing word from the context.of the remain-ing words., These.mutilated language samples constitute; the Gloze-forms, ;The totalrnumber of correc t l y replaced words i n . . each language sample across a l l subjects f i l l i n g i n these passages (Gloze raters) was t a l l i e d . From these t o t a l Cloze scores a rank ordering of the severity of thought ; disorganization was obtained i n terms of .the, number of cor r e c t l y replaced words* Those passages i n which the . largest number of words were replaced correctly ; were -deemed mo3t comprehensible and thus least severe i n terms of thought.disturbance* Conversely, those passages whose r e l a t i v e incomprehensibility was reflected i n a small . number of correc t l y replaced words were rated as represent-ing, a high degree of thought, disturbance, Correlations were run between th© rank, ;order of the twelve pati.ente on the Cloze Procedure and the rank order obtained i n the external c r i t e r i o n (r = *82)* The results produced substantial evidence i n support of a.satisfactory Cloze v a l i d i t y as i t applies to the assessment of the thought processes of schizophrenic patients as they are ref l e c t e d i n t h e i r verbal behaviour. It would thus, appear, 59 that in terms of the criterion used, the Cloze procedure constitutes a valid method of providing a quantitative estimate of inter-individual variations in the degree of schizophrenic thought disturbance;, T h e present study has not undertaken to provide information on intra-rindividual differences among schizophrenic patients over a period of time*., This point warrants further investigation in light of the possible predictive value which might be attached to such findings.;. A discussion of certain theoretical aspects of the Cloze Procedure is undertaken;.. A moderately high r e l i a b i l -i t y of the Cloze Procedure was found by computing the correlation of ranks on three equal portions of each patient's langueage samples ("W" = 0*632);. The internal consitency within each of these thirds was also examined ("R" = *69; "R" * ;B1; "R" = *45); The r e l i a b i l i t y of Cloze raters was found by correlating the overall ranks of a l l fifteen Cloze raters ("R" = .792); The average Cloze rank ordering for each of three groups of five raters each are almost identical with one another ("W" = *98)* It is thus suggested that a smaller number of raters would have ranked these patients in almost exactly the same way as did the entire group of fifteen; This finding has considerable practical importance; The results appear to be reasonably conclusive. However, the reader i s cautioned against making generaliza-tions from these findings which are valid s t r i c t l y only 6 0 within the l i m i t s of the c r i t e r i o n measure and the small sample employed i n the study.- This work has been primarily exploratory i n nature.. The Gloze Procedure had previously never been, used to study the verbal behaviour of mental patients^,! However, the r e s u l t s are s u f f i c i e n t l y d e f i n i t e to warrant further investigation along s i m i l a r l i n e s . APPENDIX A C r i t e r i o n Checklist for the Assessment of Thought Disorder CRITERION CHECKLIST FOR THE ASSESSMENT OF THOUGHT DISORDER • 62 involved incougruous incomplete fragmented and/or disconnected i l l o g i c a l and u n i n t e l l i g i b l e loose confused and disorganized odd and e r r a t i c sequence of ideas f l i g h t of ideas bizarre and highly personal verbal associations vague generalizations . . uses abstract words with a lack of precision i n meaning blocking of thought processes overinclus ive stereotyped r e p e t i t i o n of phrases unusual symbolism condensation of words neologisms impaired a b i l i t y to abstract - i . e . , concrete or l i t e r a l thinking words are jumbled and incomprehensible - "word salads" sentences are disconnected use of irrelevancies i n response to questions. APPENDIX B Close Forms A V the event that 1^ would be more than 2 the story entangled i n 5 things with i t ... i t ' s 4 with other things than 5 the reading. Certain things 6 the younger ones today, 7 the younger groups today... 8 stories or anything l i k e 9 ... that interest them. As 10 go along, you know. A2) Big person, i t was... 1 there's something l i k e that. 2 seems to be somebody 3 has something to do 4 a ... has form of 5 or something or... along 6 the idea or due 7 an extreme state of 8 Well, i t doesn't show 9 well because the people, 10 person seems to have on f a i r l y good shoes. A3) A man contemplating going 1 the army, into the 2 or some such thing 3 that. Is i t a 4 ? Would this be a 5 of his or a 6  parent or a something 7 that? He expected that 8 was... be undergoing medical 9 I imagine i t s some 10 of an operation. A4) That place i s t he 1 Canyon. The Grand Canyon? 2 of a forest. Chances 3 i t ' s i n some remote 4 and there's, there's no 5 how to get there 6 a person would have 7 just assume i t was 8 some place that was 9 accessible and maybe not. 10 seem to be broken i n a triangular s,6r$ of shape. A5) It's a friend or 1 . Could be a ... she 2 be a nurse, or 3 : i n care of her 4 younger woman i n the 5 ... a younger child, younger 6  or something l i k e that. 7 i t shows them so 8 to one another, they're 9 mother and daughter or 10 such l i k e that. A6) The marks on him 1 , somebody has got a 2 on him too or 3 With those big fingers... 4 seems to be about 5 height I imagine, pretty 6 to about five feet 7 or so i n height 8 a f a i r l y good weight 9 a f a i r complexion and 10 seems to be, have some proposition next with him anyway. Bl) Looks li k e a young 1 to me. He's having 2 studying his work because 3 knows i t . Other that 4 I don't see very 5 to make a story, 6 By the picture, i t 7 l i k e he might have 8 i n a fight, so 9 fought. See, i f I 10 i t up this way, i t ' s a v i o l i n . B2)ls this a revolver 1 down here or something? 2 this lying down? What's 3 supposed to be? I'd 4 that she probably had 5 shock or had some 6 i n the family and 7 ti r e d and she's not 8 but had a great 9 of stress l a i d upon 10 and now she's broken up. B3) It's a l i t t l e hard 1 understand. Is t his something 2 supposed to have i n 3 mind, an operation or 4 ? It's a young fellow 5 . . . i t kind of beats 6 . It's l i k e i t ' s a 7 fellow and he's thinking 8 an operation his father 9 or somebody close to 10 B4) A ruin, something after 1 explosion? You won't give 2 any indication at a l l ? 3 , somebody went off the 4 end or something. I 5 believe i n war aces 6 restricted as much as 7 violent.,..resorts to violence. _8 could have been of 9 accidental nature too... i t 10 have been a gas leakage from somewhere. B5) There's a mot her and 1 daughter. The younger daughter 2 maybe been injured and 3 down the stairs and 4 picking her up,, making 5 she's s t i l l alright. And 6 happening at the moment 7 i s the mother i s 8  to get her confidence _9_ that she can guide 10 maybe up to the bedroom. B6) I t would look to I l i k e some dark night. 2 a ... looks as though 3 ... this man1 s been attacked 4 his money or something 5 has on his P e r s o n 6 somebody right behind him. 7 must be a l i t t l e 8 drunk as well or 9 because the grip that 10 his hat on i s not i n such a location C1J The child took the 1 of s i t t i n g beside the 2 . If that's a fiddle 3 what .is i t ? what _4 of instrument would that 5 ? I know what i t — 6 b u t 1 can't mention 7 . I just forget what 8 i s . I know what __ 2 of instrument ... what music 10 plays and a l l that but I can't just name i t . C2) Why i s he s i t t i n g 1 this? Well, he must 2 done something- but didn't 1 himself or so that 4 s i t t i n g l i k e that, thinking 5 i t . That's the o n l y . 6 • Maybe He's frightened of. 7 , someone, somebody, another or 8 „ somebody else or whom. 9 don't know. He 10 be frightened of someone. C3)^ This boy standing here 1 what's going on? This 2 and whatever i s going 3 here. Operation or what. 4 operation I ' l l say or 5 or trying t o got 6 . out from him or 7 him up or maybe 8 hurt badly. They're bandaging 9 up. Maybe he got 10 C4) I don't know what 1 i s . Oh, this i s 2 old, very,very, years, 3 where the crocodiles and 4 kinds of reptiles and 5 that lived and i t s 6 climate. That's what the 7 i s a l l about. The 8 here along the c l i f f 9 through and there's now 10 the c l i f f here one. peaking out from"it. C5) Somebody i s trying apparently 1 choke someone. Down below 2 stairway... somebody trying 3 hit somebody or choke 4 or something. Destroying for 5 reason or another. The 6 must have done damage 7 the person or something. 8 don't know. Similar to 9 • . That's what the picture" .10 06) This fat person s i t t i n g 1 and. standing and there's 2 around him, around his 3 here and around his 4 hand and i t s ... he's 5 on a t i e and 6 shirt and belt and 7 looking away to some 8 reason or another he's ... he 9 of himself too big 10 something. Dl) Decided, as i f lava. 1 . Do you know who 2 is? Let me get 3  lead. A l l the strings 4 ? A youngster, i n view 5 the apparent indication of 6 to play as i f 7 whereas a more mature 8 would have the i n -strument 9 hand and not necessarily 10 with which you and I might be acquainted. D2) Is i t a bed I from which he has 2 ? A blow of some 3 has been received i n 4 form of news. That 5 be one interpretation for ... 6 get down i n the 7 like that would- require 8 e x p l i c i t . There must be 9 reason for ... don't give 10 darn attitude. It might be quite early i n the morning too. D3) An accident happen? A 1 of some sort near 2 hand. A guess might 3 taken that an accident 4 occurred. A lantern, a 5 , window, clothes, for a 6 . If an accident has 7 some trace i s to 8  found of the, a ... 9 . He used ... this person 10 had an accident. D4) Is this, right side 1 ? A pause and or 2 dammed up create interest 3 •. • as well as ... any 4 set out to corral 5 for a specific flow. 6 to cross there? You 7 the answers to these? 8 head of livestock of 9 kind. A bird, rock, 10 animal has gone so far and has had to turn back. D5) I expect i t has 1 to perhaps ..ore than 2 ' • number of the family. 3 mother holding one of 4 children and i f some 5 has occurred. Why the 6 on her face seems 7 be some proof 8 worse s t i l l someone may 9 been lost i n the 10 through accident or carelessness. D6) The boy's out one 1_ of his nights out 2 the rest of the 3_ . And besides, perhaps a 4 table bridge party there 5 have been some beverages 6 an intoxicating nature. But 7 i s an urge for 6 to be made where-as 9 guest i f such be 10 case seems somewhat reluctant about breaking up at a premature hour. E l ) , s s e l l , I see a 1 thinking about the v i o l i n 2 how i t works and 3 thinking of the d i f f e r e n t 4 and hi s wondering how 5 can learn to play 6 • I don't know, i t s 7 pretty hard picture to 8 what happened or W H A T 9 up to i t . He 10 thought the v i o l i n was pretty hard instrument to make. S2) To me, i t kind 1 looks l i k e he's crying. 2 must have happened on 3 way. Maybe he's going 4 school and something happened 5 got to crying about 6 . He's kneeling down on 7 bench there, crying about 5 because he can't do 9 something at school. Maybe, 10 got a problem. E3) "Well, t h i s i s , i t 1 l i k e a medical student. 2 thinking of the future. 3 l a s t when h e ' l l be 4 to get down to 5 l i k e an ordinary doctor. 6 be able to operate 7 same as a doctor 8 he's thinking about t h i s 9 the mind. He's probably 10 an operation. E4) What's i t supposed to 1 ? This looks l i k e a 2 pass. Aspass going 3 a mountain and t h i s 4 l i k e a creature poking 5 head down through the... 6 mountain crack. The people 7 walking across here 8 go through a mountain 9 and t h i s creature i s 10 to attack them. E5) This picture...some woman 1 died or has f a i n t e d 2 something and the ' ' o t h e r 3 i s t r y i n g to help 4 and she's i n pretty 5 condition. This other woman 6 f a i n t e d or something happened 7 her and she's...this 8 woman i s t r y i n g to 9 her and the outcome 10 that a f t e r a while t h i s woman i s not dead. E6) This one looks l i k e 1 man i s t r y i n g to 2 on hi s coat or 3 us gelping him with 4 coat and he's maybe 5 a t a i l o r shop l e t ' s 6 g e t t i n g a f i t t i n g f o r 7 t a i l o r ... f o r a coat 8 t h i s one other f e l l o w 9 helping him on with IQ coat. F l ) Evidently, t h i s fellow i s 1 i n musical instruments, according 2 the way he's looking 3 i t . Well, perhaps h i s 4 or some member of 5 family i s w e l l established 6 musical instruments and he 7 i t . And at the 8 h i s pondering over the 9 . I don't know i f 10 wish or disgust. F2) Very emotional, t h i s one, 1 i t should be quite 2 to make up a 3 f o r t h i s . I'd say 4 poor g i r l has had 5 tragedy j u s t happen to 6 . There seems to be 7 on the flooa?, but 8 don't know i f i t 9 . Some emotional f a i l u r e of 10 nature. F3) Here i s a young 1 influenced by surgery. To 2 he looks l i k e he 3 be the 4 ...a professional doctor, so 5 sure a noted doc-t o r . 6 i s h i s boyhood influence 7 older people s e t t i n g i n 8 pro-f e s s i o n . He looks very 9 . For the moment he's... 10 i s more or less h i s past. F4) Very i n t e r e s t i n g picture. I t 1 great suspense. I t ' s very 2 .down almost i n t o the 3 . In the heart of 4 , something, something supernatural to 5 . There are two beasts 6 some nature and here 7 one spots the o t h e r - 8 i s prey f o r the 9 . I would say a 10 of explorers went touring down in t o the earth. F5) There's an argument, some 1 has been.going on. 2 t e r r i f i c family argu-ment has 3 going on and the 4 i n the house, the 5 perhaps, i s extre-mely angry 6 her and i n f a c t , 7 , f r a n t i c with her. The 8 she's got him pinned 9 the stairca s e t r y i n g to 10 something i n t o h i s head. F6) This man has just 1 some error, has robbed 2 done some cri m i n a l offence. 3 caught f o r the time 4 . He's caught r i g h t now 5 he i s ashamed of 6 caught, sorry more or 7 that he had done 8 at the instant that 9 caught and h e ' l l be 10 o f f to "orison. Gl) The f i r s t thing that 1 l e d up to the 2 would be the v i o l i n 3_ -front of the boy. 4 interested i n that p a r t i c u l a r 5 . There's no one else ' 6 t h i s p a r t i c u l a r picture to 7 aware of. Tt's j u s t 8 and h i s i n s t r u -ment and 9 . I t looks l i k e a 10 that he's s i t t i n g on. G2) A woman. . . l i k e a 1 i s s i t t i n g down, s i t t i n g 2 beside a bunk as 3  she was i n j a i l . 4 a shooting or something. 5 l a y i n g there. Maybe 6 ' could have murdered somebody 7 t h i s bed, i n t h i s 8 because I'm sure that 9 she was i n j a i l , 10 wouldn't have the gun with her. G3) W e l l , t h i s picture here 1 a«--of a pict u r e 2 another picture and that 3 back i s . ..I would 4 t h i s i s on a 3 > but then you've got 6 coming i n here l i k e 7 i s a window you 8 . I b e l i e v e . . . I believe 9 another p i c t u r e and two 10 are operating 1. G4) This p i c t u r e i s one 1 the roadside, on a . 2 .And t h i s rock i s 3 i n on the road 4 before: you get to 5 bridge. This object here 6_ don't know what i t 7 . I can't see i t . 8 could b e . . . i t looks 9 me l i k e i t could 10 a b u f f a l o . G5) This i s a mother 1 and carressing her daughter 2 before she's going u p s t a i r s 3 bod i n the evening 4 i t looks l i k e the 5 i s i n white more 6 l e s s nightgown. The mother _._7 has on her house 8 or evening dress. That 9 could be that she 10 have come i n l a t e at night. G6) This man i s being 1 up by somebody i n 2 background. He's got awful 3 hands. Looks l i k e he 4. be a ...some kind 5 a giant and he's 6 to persuade t h i s man 7 the p i c t u r e to go 8 or do something f o r 9 . I t seems that the 10 i s p u t t i n g up a c e r t a i n .amount of resistance. Hi) The boy has been 1 a piece of music 2 and he's playing i t 3  h i s v i o l i n . He's going 4 play i t on...right 3 the moment he's, he's 6 i t over about the 7 and the v i o l i n . I 8 he's t h i n k i n g of how 9 w i l l sound on the 10 . H2) Maybe something sad happened 1 the, i n the l i f e 2 the, t h i s person shown 3 t h i s picture and i t , 4 j u s t , at the moment 5 t h i n k i n g about the...maybe 6 mother died or something 7 * he's j u s t t h i n k i n g about 8 . He's crying on the 9 of hss bed. Get 10 head on h i s arm and he's c r y i n g . H3) This i s probably a 1 and he's probably got 2 b u l l e t wound i n h i s ... 3 h i s , h i s stomach, and 4 doctors at the moment, 5 doctors are t r y i n g to 6 the b u l l e t wound out 7 h i s stomach so he 8 die and the people 9 probably thinking that i f 10 won't get i t out i n time, h e ' l l probab-l y .. • H4) This, I don't a c t u a l l y 1 what l e d up to 2 story but i t ' s a 3 p i c t u r e of a...a 4 where there's a road 5 through the mountains and 6 a b e a u t i f u l bridge that 7 over the water where 8 water flows underneath the 9 hard rocks. Well, the 10 thing I thought... H5) W e l l , t h i s g i r l , she 1 crossed the one that's 2 her and... at the 3 t h i s other woman i s 4 t h i s woman i n the 3 here. I think they're 6 at each other, so 7 she chokes her, she 8 over dead a f t e r a 9 . I imagine the g i r l 10 die i f she's being choked. H6) F i r s t of a l l , the 1 people here shown i n 2 p i c t u r e are friends and 3 i s behind the other 4 and you can't see 5 head but he's got 6 hands showing over the 7 of the arms so 8 , he's j u s t , just a . . . 9 there and he's got 10 , got h i s arms over the shoulder... 68 J l ) A l i t t l e boy, he's 1 a broken v i o l i n and 2 pr e t t y sad about i t . 3  smashed h i s v i o l i n and 4 was a l i t t l e boy 5 home from h i s music 6 rind some ki d s smashed 7 v i o l i n on him and 8 pretty sad about i t 9 he takes i t home 10 he's looking at i t . J2) Oh, the event shown 1 the p i c t u r e . . . The g i r l 2 boy or whatever i t 3 , I can't r e a l l y t o l l 4 looking at i t . He's 5 unhappy about some-thing and 6 hanging over the edge 7 a sofa. I t looks 1 8 a chester-f i e l d , whatever you 9 to c a l l i t and 10 quite a state of anguish. J3) These men here, there's 1 here performing an emergency 2 . By the looks of 3 to me, some guy 4 been hurt i n a 5 • There's some kind of 6 old r i f l e s i t t i n g here 7 the foreground. This young 8 seems to be looking 9 f o r i t , overseeing i t maybe. 10 ju s t witnessing i t . J4) What i s t h i s at 1 ? Of course, some of 2 these paintings are l i k e 3 . These are by the 4 , h i s t o r i c a l . These are no 5 h i s t o r i c a l , h i s t o r i c a l paintings. To 6 i t looks l i k e some 7 , pack animals being driven 8 a stone bridge by 9 man, a shepherd of 10 kind. J5) To me i t looks I another h i s t o r i c a l p a i n t i n g . Looks 2 • me l i k e there was 3 s o r t of sadness i n 4 family, some sort of ... 5 that caused a great 6 of g r i e f and the 7 i n the foreground here, 8 woman I guess i t 9 , she's being comforted by 10 other woman. Jb) Somebody has a hold 1 h i s arms here. Now 2 has le d up to 3 i s probably he's some 4 of an organizer of 5 kind, some kind of 6 who, l i k e y o u r s e l f , works, 7 out by himself, he 8 out by himself and 9 to get things done 10 himself. K l ) Looks downhearted about something 1 he' s..probably i s because 2 doesn't want to play 3 instrument to begin with. 4 f e e l i n g down because maybe 5 couldn't learn to play 6. . Seems to be wondering 7 he should keep t r y i n g 8 play i t or not. 9 someone didn't l i k e the 10 he played. K2) I t looks l i k e l e t ' s 1 quite proper and i s 2 a revolver l a y i n g down 3 ? Looks l i k e i t to 4 . I t could be. Looks 5 he can't make up 6 mind whether he should 7 that revolver on something 8 not. He seems to 9 a b a t t l e with himself 10 to make up h i s mind. K3) He's having an operation. 1 a b i g r i f l e s i t t i n g 2 By the looks of 3 lyoung chap's face i f 4 operation doesn't turn out 5 be a success, he 6 take a r i f l e and 7 asking questions about why 8 didn't turn out to 9 a success. I t ' s hard 10 say what that could be. K4) .This could be the 1 of a mountain with 2 l o t of rock l y i n g 3 lo o s e l y ready to r o l l 4 I t ' l l rest on the 5 of the canyon maybe. 6 looks l i k e i t could 7 a part of a 8 of a side of 9 h i l l maybe. This looks a path through the canyon. Looks l i k e someone 1 came down e i t h e r those 2 and f e l l down and hurt or t r i e d to _4 up the s t a i r s and 5 down and got hurt. someone wanted to hold 7 other person up or 8 out what happened and I imagine i f 10 other person was holding t h i s other one up... 10 K5) This : 3 6 to 9 K6) vde 11, 3 6 looks l i k e someone 1 t h i s person from behind 2 was going to rob or commit a crime 4 r e a l l y couldn't be two 5 unless i t ' s j u s t h i s sleeve hanging there... Unless 7 j u s t one person behind 8 there. This man i s 9 to p u l l him down 10 t r y i n g to hold him up. L l j ' w e l l , t h i s i s a 1 that's concentrating on a 2 , i s n ' t i t ? And he's 3 on the v i o l i n . He 4 be, the, the, he 5 be discouraged with i t , 6 might not be. Somebody 7 want him, might have 8 him that. And k e 9 not have wanted to 10 the v i o l i n . L2) This f e l l o w has a 1 posture there. I don't 2 i f he's c r i p p l e d or 3 . But he might be 4 . This hand might be 5_ up and somebody, must 6 j u s t a k i d , he 7 outside and somebody ... man, 8 might be a grown 9 as f a r as I 10 see. L3) As f a r as I 1 see that k i d hurt 2 with a r i f l e and 3 that t h i s guy die s . . . 4 . That might be. This ... 5 k i d here might have 6 the r i f l e there, t h i s 7 there, t h i s young f e l l o w 8 . He, he might have a man. The r i f l e 10 might have shot a man. that comes under 1 that I don't know 2 about. And t h i s i s , t h i s i s a place 4 and a s w e l l , l i k e 5 with rock there and with rock down, l i k e 7 and faced with rocks 8 down. Rocks here few trees here and 10 . L4) Well, 3 6 and 9 L5) That 1s 3 woman i s n ' t i t ? This 2 a, t h i s i s a This seems l i k e " a 4 down there too or 5 . What she's t e l l i n g her.' 6 don't know. She might 7 f a i n t , the other woman 8 one of them there, 9 her f i n g e r up her, 10 her ear or something. L6) He might just have 1 hurt and he t r i e d 2 put h i s coat on some thine ~ I don't know 4 ... That's what I think 5 i s doing. He got 5 6 and he's Sving to 7 h i s coat on or _ _ 8 . Somebody just grabbed him. 9 i don't know . He 10 have just got drunk. Ml) The l i g h t i s i n 1 of him. That's a 2 instrument. Is i t a 2 _ _ or a t i t l e to 4 . He was a lad _ 5 i n ... i n making music _ 6 the r i g h t time, vou 7 . He became ... He was 8 to the note, music, 9 you see. Couldn't hear 10 • M2) Person praying, i s n ' t i t ? 1 * couch or something. 2 story about an encapsulated 3 or something and how _ 4 were brought out of j, to ... The/ could hardly 6 , could hardly think you 7 S o t *rougnt out encapsulated features of the 9 • The mind i s subjected 10 when a person can hardly think by prayer. M3) An operation, i s n ' t i t ? 1 The man i n the 2 and he was ... they 3 had t h e i r orders to 4 f o r t h and i t was 5 with i t , that he 6 always take a small 7 i n a pocket, j u s t 8 h i s heart and he'd 9 l i k e anything to make 10 through untouched. M4) A s t o r y . . . back i n 1 days. That's a wet 2 there i s i t ? Make 3 sto r y about the center 4 the earth here or 5 l i k e a teacher l i k e 6 S h a l l I make i t 7 i f I could? d e l l 8 small l a d lay sleeping 9 day and the road 10 seems to be going on i n c i r c l e s . M5) A son, mother and 1 ? Is i t a son? 2 not f o r looks. 1811 3 and s t a r t with t h i s 4 ou t l i n e . I had to 5 her daughter one day ... 6 was not too w e l l 7 and about growth and 8 that youngsters would be 9 bewildered about. So she 10 down the s t a i r s . He was going to 1 off a c l i f f or 2 . There' s a man coining behind him p u l l i n g 4 Just about to jump 3 a c l i f f and a 6 walked up and got 7 him and held on, 8 a hold of h i s 9 and shoulders and held, 10 back. APPENDIX C Cloze Answer Blank A l ) 1. 2. 3. 4. 5. ' 6. - : 7-. 8. 9. 10. A4) 1. 2. 3. 4.' 5. 6. 7. 8. 9.. 10. B l ) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. B4) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Cl) 1. 2. 3. 4. 5. 6. '7. 8. 9. 10. A2) 1. 2. 3. 4. 5. 6. . 7. 8. 9. 10. A5) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. B2) 1. 2. 3. 4» 5. 6. 7. 8. 9. 10. B5) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. C2) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. A3) 3. 4. 5. 6. 7. 8. 9. 10. A6) 1 . 2. 3. 4." 5. 6. 7. 8. 9. 10. B3) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. B6) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. C3) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. G4) 1. 2. 3. 4. 5 . 6. 7 . 8. 9. 10. Dl) 1. 2. 3 . 4. 5. 6. 7 . 8. 9. 10. D4) 1. 2. 3 . 4. 5 . 6. 7 . 8. 9. 10. E l ) 1. 2. 3. 4. 5.' 6. 7 . 8. 9. 10. E4) 1. 2. 3. 4. 5 . 6. 7 . 8. 9. 10. C5) 1. 2. 3 . 4. 5 . 6. 7 . 8. 9 . 10. D2) 1. 2. 3 . 4. 5. 6. 7 . 8. 9* 10. D5) 1.. 2. 3 . 4. 5 . 6. 7 . 8. 9. 10. E2) 1. 2. 3 . 4. 5 . 6. 7 . 8. 9. 10. E5) 1* 2. 3 . 4. 5. 6. 7 . 8'. 9. 10. G6) 1. 2. 3 . 4. 5. 6. 7 . 8. 9. 10. B3) 1. 2. 3 . 4. 5. 6. 7 . 8. 9. 10. D6) 1. 2. 3 . 4. 5 . 6. 7 . 8. 9. 10. E3) 1. 2. 3 , 4. 5 . 6. 7 . 8. 9. 10. E6) 1. 2. 3 . 4. 5 . 6. 7 . 8. 9. 10. Pi) 1 . 2. 3. 4. 5. 6. 7. 8. 9. 10. P4) 1. 2. 3. 4. 5. • 6. 7. 8. 9. 10. Gl) .1. 2. 3. 4. 5. 6. 7. 8. 9..-10. G4) 1. -2. -3. 4. 5. 6. 7. 8. 9. 10. HI) 1. 2, 3. 4. 5. 6. 7. 8. 9. 10. F2) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. F5) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. G2) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. G5.) 2. 3. 4. 5. 6. 7. 8. 9. 10. H2) 1. 2. 3.. 4, 5.. 6. 7 . 8. 9. 10. F3) 1. 2. 3. 4. 5. 6. 7. 8.. 9. 10, F6) 1. 2. 3. 4. 5.' 6. 7. 8. 9. 10. G3) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. G6) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. H3) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. H4) 1. 2 . 3. 4 . 5 . 6. 7. S . 9. 10. JI) 1. 2 . 3". 4 . 5 . 6 . 7. 3. 9'. 10. J4) 1. 2. 3. 4 . 5. 6 . 7. S . 9. 10. H5) 1. 2 . 3. 4. 5 -6. 7. 8. 9. 10. J 2 ) 1. 2 . 3. 4 . 5 . 6. 7. 8. 9. 10. J 5 ) 1, 2 . 3. 4 . 5. 6. 7. 8. 9. 10. H6) 1. 2 . 3. 4 . 5 . 6. 7. 8. 9. 10. J3 ) 1. 2.. 3. 4 . 5 . 6. 7. 8 . 9. 10. J6 ) 1. 2 . 3. 4. 5 . 6 . 7. 8. 9. 10. Kl) 1. 2 . 3. 4 . 5 . 6 . 7. 8. 9. 10. K 4 ) 1. 2 . 3. 4 . 5 . ' 6. 7. •8. 9.-' 10. K 2 ) 1. 2 . 3. 4 . 5.. 6. 7. 8. 9. 10. K5) 1. 2 . 3. 4 . 5,. 6. 7. 8. 9. 10. K3) 1. 2 . 3. 4 . 5 . 6.. 7. 8. 9. 10. K 6 ) 1. 2 . 3. 4 . 5 . 6. 7. 8. 9. .10. LI) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. L4) 1. 2. 3 . 4. 5. 6. 7. 8. 9. 10. Ml) 1. 2. 3. 4. 5. 6. . 7 . 8 . 9. 10. M4) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. L2) 1. • 2. -3. 4. 5. 6. 7. 8. 9. 10. 15) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. M2) 1. 2. 3. 4. 5. 6.-7. 8. 9. 10. M5) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. L3) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. L6) 1. 2. 3 . 4. 5. 6. 7, 8. 9. 10. M3) 1. 2. 3. 4-5. 6.. 7. 8. 9. 10. M6) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. APPENDIX D . C l o s e Raw Scores 77 CL02E RAW SCORES Raters Pa-tie n t s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Tots A 33 34 35 35 30 32 33 32 30 28 36 32 25 34 31 480 B 35 33 27 28 26 32 28 27 29 35 34 32 25 27 22 440 C 27 28 29 27 26 29 27 25 23 23 26 29 24 25 25 393 D . ., ; 28 28 24 25 18 29 23 23 25 21 28 23 20 18 22 355 E • 22 25 27 24 20 27 21 25 23 30 26 22 21 25. 21 359 P '.' 36 38 36 38 29 39 41 37 35 33 42 37 27 27 34 529 G 31 37 30 36 29 32 33 33 29 33 35 26 23 31 28 466 H 30 22 28 33 21 26 27 31 26 32 31 25 23 29 27 411 J- 18 24 15 21 13 16 21 17 20 19 19 20 16 19 i4 272 K 37 39 38 37 31 33 43 33 34 42 40 35 38 31 35 546 L • 22 28 21 20 18 22 26 23 20 25 14 28 22 19 23 331 M 16 17 16 20 s 22 17 18 18 10 22 20 17 17 16 21 267 APPENDIX E Gloze Ratings on Three Subsections 79 CLOZE RATINGS ON THREE SUBSECTIONS Average Cloze Ranking Patients on E a c h Subsection A B O D E F G H J S L M 2 9 3 8 5 4 6 7 11 1 10 12 6 4 9 8 12 2 5 3 7 1 10 11 9 2 9 6*5 8 1 4 11 12 5 6.5 10 F i r s t Third Middle Third Last Third APPENDIX F Cloze Bankings on the F i r s t Third SI GLOZE RANKINGS ON THE FIRST THIRD Rater's P a - 1 " t i e n t s 2. • . 3 4 5 6 7 8 9 10 1 1 12 13 14 15 A ' 1' 3 . 5 2 1*5 l i 5 U5 4 4 1.5 5 3*5 4 '•' 6 ; 5 2*5 6 B 6 . 5 11 7*5 7 9 7 . 5 9 ; 5 8 9 7 ; 5 8 11 10 11 11 C 6 , 5 3 . 5 4 3 . 5 1 .5 1*5 4 1*5 1.5 7 . 5 5 2 . 5 3 4 3 D 9 8 7 . 5 12 10 9v5 9 8 11 7 9 . 5 IQ 9*5 9 6 S 2 . 5 3 . 5 4 5 . 5 5 . 5 5 6 . 5 4 5 1 6 7 3 2 . 5 6 F 6 . 5 8 6 4 3 5 1 ,5 4 7 9 . 5 2 2 . 5 3 5 . 5 1.-5 G 4 3*5 4 3 5 . 5 4 .5 4 6 . 5 5 4 1 9 . 5 5 5 . 5 9 H 6 . 5 8 9 8 5*5 5 6 . 5 6 . 5 5 3 9 5 . 5 6 . 5 7 . 5 6 J 12 10 11 1 0 , 5 > 12 12 12 11 11 12 12 12 10 12 12 K 2 . 5 1 1 .5 1*5 5 . 5 3 1*5 1 .5 3 2 3 . 5 1 1 1 1 . 5 L 10 6 10 9 11 10 8 12 10 6 11 5 . 5 10 7 . 5 6 'M 11 12 12 1 0 . 5 8 11 11 10 12 9 . 5 10 8 10 9 . 5 10 APPENDIX 0 Cloze bankings on the Middle Third 33 CLOZE RANKINGS ON THE MIDDLE THIRD Raters Pa-tients 1 i 2 3 4 5 6 7 8 0 10 11 12 13 14 15 A 6,5 6,5 3 6.5 6 6.5 4 6 7 8.5 5 5 4.5 5.5 3 B 3.5 3,5 6 4 3.5 3.5 6 4.5 4.5 3.5 4 1 3 1.5 5.5 C 6.5 8,5 6 9.5 105 10 8 11 9 7 8,5 10 10 7.5 12 D .6,5 10 8 6,5 8 8.5 9 7 9 10 8,5 8.5 10 11 10.5 E 11.5 3>2 10 12 105 11,! 5 12 12 115 115 11 11 12 7.5 10,5 F 1 1.5 1 2 6 1 1 2.5 2 2 1.5 2.5 6 9 5,5 a 9 5 6 6.5 1 3,5 6 4.5 4.5 .5.5 6,5 7 7.5 3,5 4 H 3*5 .3.5 4 1 3.5 5 3 1 3 3.5 3 6. 2 1.5 1.5 j 6.5 6,5 9 6,5 6 6.5 6 10 6 5.5 6,5 4 4.5 5.5 7.5 K 2 1.5 2 3 2 2 2 2.5 1 1 1,5 2.5 1 3.5 .1.5 L 10 8.5 115 9.5 12 8.5 105 8 9 8.5 11 8.5 10 12 9 M 11.5 11 115 9 1L5 105 10 115 115 115 12 7.5 10 7.5 7.5 APPENDIX H Cloze Rankings on the Last' Third 85 CLOZE RANKINGS ON THE LAST THIRD ^ t e r s Pa-tients 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A 7.5 4 2 4.5 5 5.5 4 3.5 4 5 4 6.5 4 1. 1 B 1 2 8 1.5 5 3.5 4 7.5 9.5 1 1.5 2 2 4' 6.5 C 10 9.5 105 7 5 7.5 9.5 105 4 11 9.5 3.5 7 7 6.5 D 4 105 4.5 10 1.5 7.5 6 4 7*5 4 9 4 5 9 6.5 E 10 6 5 9.5 8.-5 5.5 1L5 5 6 5 9.5 6.5 8.5 7" 9.5 P 3.5 4 2 3 5 1,5 6 1-.5 1 3 1.5 1' 6 2.5 3.5 G 3.5 1 8 1.5 8*5 3.5 2 3.5 7.5 5 6 6.5 8.5 2.5 2 H 7.5 12 5 7 12 12 115 ia5 12 10 7 1L5 105 5 115 j 12 11 12 11 12 11 7.5 105 7.5 12 7.5 115 12 11 115 K 2 7.5 2 7 1.5 7,5 1 7.5 9*5 2 4 6.5 4.5 10 9.5 L 6 7.5 6 11 1.5 9.5 4 - 1.5 2 9 12 3.5 1 7 6.5 M 10 9.5 8 9. ,5 5 9.5 9.5 105 11 7.5 8,5 10 105 12 3.5 APPENDIX I Cloze Rankings 87 CLOZE RANKINGS Raters Pa- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 F i n a l t i e n t s Gloze "Rank' A 4 4 3 4 2 4 3.5 4 3 7 3 35 35 1, .3 .. 3 B 3 5 75 55 4 ,5' 6 45 2 5 35 35 55 85 5 C 8 7 9 7 .5.5 65 65 75 as 9. ' a5 5 5 75 6 7 D 7 7 9 8 105 65 9 7 i i 7 9 10 11 85 9 E 95 9 *5 9 9 8 105 75 85 6 85 10 9 75 105 8 F 2 2 2 1 35 2 1 1 35 1 1 2 55 2 2 2 G 5 3 4 3 35 4 3.5 25 45 35 4 7 65 25 4 4 H • 6 . 11 6 8 9 65 5 6 5 6 8 65 4, 5 .-. 6 J 11 10 12 10 12 12 105 12 105 12 11 11 12 95 12 11 •K 1 1 1 2 1 2 1 25 2 1 2 2 1 25 1 1 L 95 7 10 115 10510 8 95 105 8 12 6 8 95 7 10 M 12 12 11 115 7 11 12 11 12 10 10 12 11 12 105 12 APPENDIX J S t a t i s t i c a l Formulae 89 STATISTICAL FORMULAE (i ) Spearman's Coefficient of Rank Correlation " r " ( i i ) Kendall's Coefficient of Concordance "W" tf = 123': m2(N5-N) S' = m2(i!^-N) i e j sum of squares of deviations 12 about t h e i r mean where: "IF i s the number of ranks made "m" i s the number of raters ( i i i ) Interclass Correlation Coefficient "R" R = BCMS - WM3 ' BCMS + K-l(WMS) where: BCMS = Between Class Mean of Squares = Between Class Sum of Squares (BCSS) directed by the degrees of freedom WIS = Within Mean Squares = Within Sum of Squares (WSS) divided by degrees of freedom k ss t o t a l number of scores where: "d" i s differences between the paired ranks " I " i s the number of individuals being ranked, 90 REFERENCES Ar i e t i , S. Some aspects pf language in schizophrenia. In H. Werner, On expressive language. Worcester: Clark University Press, 1955/ 53-67. Arnhoff, F.N. Some factors influencing the unreliability of c l i n i c a l judgments. J. c l i n , Psychol.. 1954, 10, 272-275. Balken, E.R, and Masserman, J.Hi The language of phantasy. J. Psychol*. 1940, 10, 75-86* Bellak, L. (Ed.) Schizophrenia: a review of the syndrome. New York? Logos Press, .1958, 237. . Bleuler, E* Dementia Praecox or the groups of s c h i z o p h r e n i a s . New Yorks International University Press, 1950. Buseman, A. Die sprache der jugend als ausdruck der entwieklungs - rhythmik. Munich: Fisher, 1925» Cited by B, Pasamanick, Discussion of paper presented by Louis A. Gottschalk. Psych,_ res, rep*, 1958, 10, 159-171. Calvert, J.J* Verbal behaviour as a predictor of personality syndromes. Unptiblished doctoral dissertation. Ohio State University, 1950. Cited by B. Pasamanick. 1958, • Ibid. Cameron, N, Deterioration and regression in schizophrenic thinking. J. abn. soc, Psychol., 1939, 34, 265-270, Cameron, N, Schizophrenic thinking in a problem-solving situation. J. ment. Sci,, 1939. 85. 1012-1035. Cameron, N, Experimental Analysis of schizophrenic thinking* In J*S* Kasanin (Ed.), Language and thought In  schizophrenia * Berkeley: University of California Press, 1946, 50-64. Chapman, L.J, D i s t r a c t i b i l i t y in the conceptual performance of schizophrenics. J, aon.soc* Psychol., 1956, 53, 286-291* Chapman, L.J. Breadth of deviate concepts used by schizo-phrenics. J^j^a t ^ ^ 1957 si 54, 118-123, Chordokoff, B. & Mussen, P* Qualitative aspects of the 122iffiSl§BMIN?E59^,n£oTaiS-^ schizophrenics. 91 Cobb, S i & Lorenz, M, Language patterns i n psychotic and psychoneurotic subjects. A.M.A. Arch, neurol. ps.vchiat.r 1954. 72, 665-673. ! 1 * Epstein, 3. Qverinelusive thinking i n a schizophrenic and a control group. J, cons. Psychol.„ 1953, 17, 384-388. Ewalt, J-.R. Strecker, E.A. & Ebough, F.G. P r a c t i c a l c l i n i c a l  psychiatry. New York: McGraw-Hill, 1957.; ~~ Fairbank, H,; Studies i n language behaviour: I I . The quantitative d i f f e r e n t i a t i o n of samples of spoken language.: Psvchol. 1 Monogr.-. 1944» 56, 19-38. F e i f e l , H. Qualitative differences i n the vocabulary responses of normals and abnormalsv J, genet.' Psychol., 1949, 39, 151-204.• Feldman, M.J.- & Drasgow, J . A visual-verbal test for schizo-phrenia.- Pjsvcl^ 1951, 25, 55-64. Fey, E.T. The performance of young schizophrenics and young normals on the Wisconsin card sorting t e s t . J, cons * Psychol.. 1951, 15, 311-319 Fish, F.J. Schizophrenia. Baltimore: The Vlilliams and Wilkins Co., 1962. F l a v e l l , J.H. Abstract thinking and s o c i a l behaviour i n schizophrenia . J . abn. soc. Psychol.. 1956, 52, 208-211* Frankenstein, D.H. Qreenblatt, M. & Solomon^ H.'G. Autonomic changes p a r a l l e l i n g psychological changes i n mentally i l l patients. J. nerv. ment. dis.,1951. 114, 1-18. Gregory, I. Psychiatry: B i o l o g i c a l and So c i a l . Philadelphia: W.B. Saunders Co., 1961. !~ Goldstein, X. Methodological approach to the study of schizophrenic thought disorder. In J.S. -Kasanln (Ed.), 1946, op.cit.. 17-40. Goodstein, L.D. The language of schizophrenia. J . genet. Psychol.. 1951, 45, 95-104. Gottschalk, L.A. & Hambidge, G. Verbal behaviour analysis: a systematic approach to the problem of quantifying psychological processes. J . proj . techn., 1955, 19, 387-409. 92 Gottschalk, L.A., Gleser, G.C.•& Hambidge, G. Verbal. behaviour analysis: some content and form variables i n speech relevant to personality adjustment. A.M.A. Arch, neurol. psychiat., 1957*. 77, 300-311. Gottschalk, L.A., Gleser, G.C., Daniels, R.S. & Block, S. The 'speech patterns of schizophrenic patients: a method of assessing r e l a t i v e degree of personal disorganization and s o c i a l a l i e n a t i o n . J . nerv.. ment. d i s . , 1958, 127, 153-166. Gottschalk, L,A. & Kaplan, S. Towards a quantitative method of estimating variations i n the intensity of a psychological c o n f l i c t or state. A.M.A.-Arch, neurol. psychiat,; 1958, 79t 685-696. Gottschalk, L,A,» Gleser, G.C., Springer, K.J*, Kaplan, S.M, •Shanon, J . & Ross, W.D, The effects of perphenazine 0» verbal behaviour patterns: a contribution to the problem of measuring the psychological effects of psychoactive drugs. A.M.A. Arch, gen, psychiat,. •I960;:2^ 632-639* -Gottschalk,L.A., Gleser, G.C,, Magliocco, E.B. & D'Zmura, •:'» T;L* Further studies on the speech patterns of schizophrenic patiehts. J« nerv.ment.dis., 1961. 132, 101-113. > Harrington, R. & Ehrmann* J.G. Complexity of responses as a factor i n the vocabulary performance of schizophrenic J. Abn. soc. Psychol.. 1954, 49, 362-364. Henderson, D.K. & G i l l e s p i e , E.G. Textbook^of Psychiatry. London:•Oxford University Press, 1962. . Hunt, W.A. & Arnhoff, F.N. Some•standardized scales f o r disorganization i n schizophrenic thinking. J , Cons. Psychol.. 1955, 19, 3-11.1 •'• • . • . ' Hunt, W.A, & Arnhoff, F.N, The repeat r e l i a b i l i t y of c l i n i c a l judgments of test responses. J . c i i n . Psychol.. 1956, 12, 289-290. Hunt* W;A,, Jones, N;F. & Hunt, E.B. R e l i a b i l i t y o f • c l i n i c a l judgments as a function of c l i n i c a l experience. J . c l i n * Psychol.. 1957, 13, 377-378, Hunt, W,A!. &• Jones, N.F. C l i n i c a l judgments of some aspects of schizophrenic thinking.' J . c l i n . Psychol., 1958, 14, 235-239. Hunt, W.A., Walker, R;E; & Jones, N.F. The v a l i d i t y of c l i n i c a l ratings f o r estimating severity; of schizo-prenia., J . c l i n ; Psychol.. I960, 16, 391-393. 93 Jacobs,/E.A, An .investigation,of some relationships between personality disorganization and perceptual reorganiza-- tion. Unpublished doctoral dissertation* Univ. of Buffalo j 1954, Cited by iL* Bellak (Id.), 1958. op.cit., • 232.* " ' -V ••; • '•' " '" " ' ' Jensen, M*B., Morris, W.E,& Eandilakis, J* inter-and interrelationships of nursing assistants? ratings of MP hospital patients* Mimeographed, 1957,, ViAr,Hospital,.Salisbury, North Carolina* Jensen, M*B., &.Morris, W*E* Reliability - Unreliability of ancillary psychiatric evaluations * J. * clin.. Psychol * 9 I960, 16, 248-252. Johnson, WV Studies in language behaviour. I. A program of research*,Psychol.ffionogr*. 1944, 56,,lrl8. Kasahin,J*S. & Hanfmann* E* An of concept formation in- schizophrenia* Amer. J.:. Ps.vchiat** 1938. 95i 35-48* Rraepelin, E. Dementia Praecox and Paraphrenia* Edinburgh*' Livingstone, 1919. . ,, " , , . Laffal, J. Changes in language.of a schizophrenic patient , during psychotherapy. J * abn. soc. Psychol.. 1961, 63, 422-427. Lorenz, M* Problems posed by schizophrenic language* Arch. : , gen. Ps.vchiat.. .1961,, 4, 603-610. , ' • Lorr, M*,.Jenkins, R»L. & Of Connor, J.P. Factors descriptive of psychopathology.and behaviour of hospitalized psyehotics. J. abn. soc. Psychol.. 1955, 50, 78-86. Lothrop, W.W. A c r i t i c a l review of research on the. con-ceptual; thinking of schizophrenics* J. nerv. ment. dis,., 1961, 132, 118-126. ~~ " Mabry, M. A study comparing the language characteristics• .: of scattered and non-scattered schizophrenics with normals. Dissert. Abstr.. 1955, 15, 457-458. Mann, Mil* Studies in language behaviour. I l l * The , quantitative differentiation of,samples of written language. Psychol* Monogr.. 1944. 56, 41-74* Meadow, A., /Greenblatt, M., Funkenstein, D*Hr & Solomon, , H*C. Relationship between capacity for abstraction in schizophrenia and physiological response to autonomic drgs. ,J. nerv*. ment. dis., 1953, 118, 332-338i • • ' 94 Meadow, A., Greenblatt, M. & Solomon, H.C. Looseness of association and impairment of abstraction in ' schizophrenia. J. nerv* ment* dis.. 1953, 118, • 27-35.- : > •- . • / . •••»•'• Milgram, N.A* Preference for abstract versus concrete ..word meanings-in schizophrenic and brain-damaged patients. J* c l i n . Psychol., 1959, 15, 207-211* Mirin, B* The formal aspects of schizophrenic verbal communication. Genet* Psychol. Mono,s?r«9 1955, 52, .151-190. ' Moranj L.J., Moran, F*A. -& Blake, R.R..An investigation of the vocabulary performance of schizophrenics. J* genet* Psychol.. 1952, 80, 107-132* < Moran, L.J. Vocabulary knowledge and usage among normal ^ and-schizophrenic • subjects. Psychol. Honogr, 1953, 67j whole # 350* ^ Murray, H*Ai Thematic Apperception Test. CambridgeJ Harvard University Press* 1943* . Mac Gaughran* L.S* & Moran L.J * "Conceptual level" vs. ("Conceptual area" analysis of object-sorting behaviour Of schizophrenic and. non-psychiatric groups.. J* abn* soc. Psychol., 1956, 52, 43-50. Pasamanick, B* Discussion of paper presented Louis A, -Gottschalk. Psychiat * res« rep *, 1958, 10, 159-171, Payne, 1*W., Mattusek, P. George, B*I* An experimental study of schizophrenic thought disorder* J. ment. Sei.;, 1959» 105, 627-652. ' r • Rabin A * 1«, King, G. F * & Ihrmann, J. C, Vocabulary perfor-mance of short-term and long-term schizophrenics..> J * abn. soc. Psychol.. 1955, 50, 255-258, Rashkisi H*A» Three types of thinking disorder. J* nerv. ment* dis..1947. 106. 650-670. Riehman, J. The effect of the emotional tone of words upon the vocabulary responses of schizophrenics. J, gen, Psyehol.. 1957, 56; 95-119*. .-\ Sullivan, H.S* The language of schizophrenia. In J*S* Kasanin'(Bd;), 1946, op,c it,» 4-16,; - -Taylor, W.Li Cloze Procedures a new tool for measuring readability* J; Quart*. 1953; 30, 415-433.' 95 Taylor, W.L. Recent developments in the use of the Cloze Procedure. J. Quart.. 1956, 33, 42-48, 99. Taylor, W.;L, Cloze readability scores as indices of individual differences in comprehension and attitude. J. appl. Psychol.. 1957, 41, 19-26. Taylor, W.L, Application of Cloze Procedure to the verbal behaviour of mental patients. Paper read at the American Neuropsychopharmacological Association. • Hew York, 1963. Vigotsky, L.S* Thought in schizophrenia. A.M*A* Arch, neurol. psychiat.. 1934, 31, 1063-1977. r~~ Von Domarus, E» The specific laws of logic i n schizophrenia. In J,SU Kasanin (Ed.), 1946, op.cit.. 104-114. Weckowicz,-T.S, & Bleiirett, D*B, Size constancy and abstract thinking in schizophrenic patients, J. ment, s c i , ; 1959, 105, 909-934. ' Wexler, P. The influence of emotional and non-emotional content upon conceptual thinking in schizophrenia* Disser. Abstr.. 1955, 15, 1260. White, M,A. A study of schizophrenic language. J . abn. soc, Psychol., 1949. 44, 61-74* White, W.A, The language of schizophrenia. J . nerv. ment. dis.. 1928, 5, 323-343* Whitehorn, J.C. and Zipf, C*K. Schizophrenic language. A.M.A. Arch, neurol. psychiat.. 1943, 49, 831-851. Whiteman, M. Qualitative features of schizophrenic thought in the formation of social concepts. J. n e r v , ment» dis.. 1956, 124, 199-204. Woods, N; Language study in" schizophrenia. J . nerv. ment. dis., 1938, 87, 290-316.. ~" ^ '. Zipf, C.K. The psychobiology of language. Boston: Houghton-Mifflin, 1935. 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items