Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Comparative validity of a prototype scale construction strategy Broughton, Ross Harold 1981

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1981_A8 B76.pdf [ 4MB ]
Metadata
JSON: 831-1.0095026.json
JSON-LD: 831-1.0095026-ld.json
RDF/XML (Pretty): 831-1.0095026-rdf.xml
RDF/JSON: 831-1.0095026-rdf.json
Turtle: 831-1.0095026-turtle.txt
N-Triples: 831-1.0095026-rdf-ntriples.txt
Original Record: 831-1.0095026-source.json
Full Text
831-1.0095026-fulltext.txt
Citation
831-1.0095026.ris

Full Text

COMPARATIVE VALIDITY OF A PROTOTYPE SCALE CONSTRUCTION STRATEGY by ROSS HAROLD BROUGHTON B.A.(Hons), Simon Fr a s e r U n i v e r s i t y , 1978 THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS in THE FACULTY OF GRADUATE STUDIES (Department of Psychology) We accept t h i s t h e s i s as conforming to the r e q u i r e d standard THE UNIVERSITY OF BRITISH COLUMBIA June 1981 © Ross Harold Broughton, 1981 In p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t o f the requirements f o r an advanced degree a t the U n i v e r s i t y o f B r i t i s h Columbia, I agree t h a t the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r r e f e r e n c e and study. I f u r t h e r agree t h a t p e r m i s s i o n f o r e x t e n s i v e copying of t h i s t h e s i s f o r s c h o l a r l y purposes may be granted by the head o f my department or by h i s or her r e p r e s e n t a t i v e s . I t i s understood t h a t c o p y i n g or p u b l i c a t i o n o f t h i s t h e s i s f o r f i n a n c i a l g a i n s h a l l not be allowed without my w r i t t e n p e r m i s s i o n . Department of P s y c h o l o g y  The U n i v e r s i t y of B r i t i s h Columbia 2075 Wesbrook P l a c e Vancouver, Canada V6T 1W5 Date J u l y 10, 1981 DF-fi 12/19) A b s t r a c t T h i s t h e s i s examined the p r e d i c t i v e v a l i d i t y of d i f f e r e n t s t r a t e g i e s for c o n s t r u c t i n g p e r s o n a l i t y i n v e n t o r y s c a l e s . The ACL was used as a common item pool to c o n s t r u c t s e t s of 8 s c a l e s by each of 6 s t r a t e g i e s . A "prototype s t r a t e g y " was introduced i n t h i s study and i t s v a l i d i t y was compared with 4 t r a d i t i o n a l s t r a t e g i e s and 1 random s t r a t e g y in p r e d i c t i n g 21 c r i t e r i o n measures. The 4 t r a d i t i o n a l s t r a t e g i e s compared were: e m p i r i c a l , f a c t o r a n a l y t i c , r a t i o n a l , and i n t e r n a l c o n s i s t e n c y . The s u b j e c t s were 234 p a i d U n i v e r s i t y of B r i t i s h Columbia f r a t e r n i t y males ranging in age from 19-28 who s u p p l i e d ACL responses and the 21 c r i t e r i o n measures. Included i n the 21 c r i t e r i a were peer r a t i n g s on the t r a i t s of achievement, dominance, nurturance, a f f i l i a t i o n , e x h i b i t i o n , autonomy, a g g r e s s i o n , and deference; u n i v e r s i t y GPA; high and low peer nominations on the dimensions of l e a d e r s h i p , l i k e a b i l i t y , academic p o t e n t i a l , a t h l e t i c p o t e n t i a l , h e t e r o s e x u a l i t y , and success. M u l t i p l e r e g r e s s i o n procedures were used i n a double c r o s s - v a l i d a t i o n d e s i g n . R e s u l t s i n d i c a t e d that the prototype s t r a t e g y d i d b e t t e r than a l l of the other s t r a t e g i e s and s i g n i f i c a n t l y out-performed the r a t i o n a l s t r a t e g y which i t was designed to improve upon. i i i TABLE OF CONTENTS A b s t r a c t i i L i s t of Tables . v L i s t of F i g u r e s v i i Acknowledgments • • v i i i Chapter 1: I n t r o d u c t i o n 1 C a t e g o r i e s , T r a i t s , and Prototypes 3 The Prototype S t r a t e g y 11 Chapter 2: Method 13 Subjects 13 I tern Pool . 13 R a t i o n a l S c a l e s 15 I n t e r n a l Consistency S c a l e s 16 Prototype S c a l e s 16 Fa c t o r S c a l e s 20 E m p i r i c a l S c a l e s 21 Peer Ratings 25 Peer Nominations 25 Academic Achievement 26 Chapter 3: A n a l y s i s and R e s u l t s 27 Chapter 4: Summary and D i s c u s s i o n 36 Peer Ratings 38 Peer Nominations and GPA 41 A Prototype S t r a t e g y and P r o t o t y p i c Acts 43 i v References .. . .45 APPENDIX A Comparative V a l i d i t y : The model 51 The S t r a t e g i e s 52 Past Comparative V a l i d i t y S t u d i e s 56 APPENDIX B T e s t i n g S i g n i f i c a n c e of Average M u l t i p l e Rs Using Dependent Samples 76 APPENDIX C P e e r - r a t i n g and Peer-nomination Forms 80 LIST OF TABLES Table 1. Some psychometric p r o p e r t i e s of the e i g h t s c a l e s c o n s t r u c t e d by each of s i x s t r a t e g i e s . . . . 23 Table 2. Average m u l t i p l e c o r r e l a t i o n c o e f f i c i e n t s f o r s c a l e s c o n s t r u c t e d by each of s i x s t r a t e g i e s on e i g h t peer rated, t r a i t dimensions... .28 Table 3. Average m u l t i p l e c o r r e l a t i o n c o e f f i c i e n t s f o r s c a l e s c o n s t r u c t e d by each of s i x s t r a t e g i e s on twelve peer nomination dimensions and s e l f - r e p o r t e d GPA 30 APPENDIX A Table I. Nine comparative v a l i d i t y s t u d i e s LIST OF FIGURES v i i F i g u r e 1. Maximum and b a s e l i n e c o r r e l a t i o n c o e f f i c i e n t s f o r f o r t y dominance s c a l e s 33 v i i i Acknowledgments Th i s t h e s i s r epresents much more than the e f f o r t of one i n d i v i d u a l and I would l i k e to express my a p p r e c i a t i o n to each of the many persons who c o n t r i b u t e d to t h i s study. F i r s t , f o r h i s encouragement, guidance, and i n v a l u a b l e knowledge, I would l i k e to express my g r a t i t u d e to Dr. J e r r y S. Wiggins, s u p e r v i s o r of t h i s r e s e a r c h , whose u n f a i l i n g support came with the r i g h t amount of understanding, p a t i e n c e , and warmth. Dr. Wiggins not only p r o v i d e d the conceptual impetus f o r t h i s study, but a l s o the resea r c h funds which made p o s s i b l e the use of extensive t e s t i n g and a n a l y s i s procedures. My thanks go to the other committee members: Dr. Jim R u s s e l l f o r h i s i n s i g h t f u l suggestions and f i n a n c i a l support, and to Dr. Merry B u l l o c k , for her needed advice and h e l p f u l s u p e r v i s i o n . Many graduate students p a r t i c i p a t e d and helped through a l l phases of t h i s p r o j e c t . In p a r t i c u l a r , I am indebted to Stephen H o l l i d a y , who was always a v a i l a b l e as a knowledgeable c o n s u l t a n t and c o l l e a g u e , and Michael Boyes, W i l f r e d Zerbe, and Norman P h i l l i p s , who a l l gave s e l f l e s s l y of t h e i r much needed a b i l i t i e s and f r i e n d s h i p , which made t h i s p r o j e c t work. And to the a d d i t i o n a l 24 graduate students who provided the expert p r o t o t y p i c a l i t y r a t i n g s , I am a l s o g r a t e f u l . I would l i k e to thank the 234 f r a t e r n i t y men who c h e e r f u l l y cooperated i n t h i s study and p r o v i d e d the necessary peer r a t i n g s which allowed me to conduct a ix comparative v a l i d i t y a n a l y s i s . F i n a l l y , I am g r a t e f u l to Linda J . Skibo f o r p r o v i d i n g her expert t y p i n g s k i l l s and some moral support that enabled the t i m e l y completion of t h i s p r o j e c t . 1 I n t r o d u c t ion The issue of p e r s o n a l i t y inventory c o n s t r u c t i o n -a c c o r d i n g to what r a t i o n a l e or s t r a t e g y should p e r s o n a l i t y i n v e n t o r i e s be b u i l t ? - and the i s s u e of v a l i d i t y of v a r i o u s c o n s t r u c t i o n s t r a t e g i e s - how should inventory c o n s t r u c t i o n s t r a t e g i e s be compared - although l o g i c a l l y r e l a t e d , have t y p i c a l l y represented two d i s t i n c t areas of r e s e a r c h i n the t r a d i t i o n of North American p e r s o n a l i t y assessment. On the one hand, t e s t c o n s t r u c t o r s have t y p i c a l l y chosen an a l r e a d y e x i s t i n g c o n s t r u c t i o n s t r a t e g y ( e.g., e m p i r i c a l * ) , or have c o n t r i b u t e d to the development of a new s t r a t e g y , as, f o r i n s t a n c e C a t t e l l ' s work d i d f o r the f a c t o r a n a l y t i c approach. Once a choice of s t r a t e g i e s i s made, the res e a r c h e r o f t e n stays with h i s or her c h o i c e . On the other hand, a f t e r f i v e decades of s t r a t e g y development in p e r s o n a l i t y assessment there were approximately three primary p e r s o n a l i t y s c a l e c o n s t r u c t i o n s t r a t e g i e s s o l i d l y entrenched in the t e s t -b u i l d e r s marketplace, each with i t s own body of supporting evidence, but whose r e l a t i v e worth or s u p e r i o r i t y remained untested and unknown. A method that would f a i r l y allow these t e s t c o n s t r u c t i o n s t r a t e g i e s to "do t h e i r best" at p r e d i c t i n g s e t s of important s o c i a l c r i t e r i a was needed. I t was not *See Appendix A for a d i s c u s s i o n of s c a l e c o n s t r u c t i o n s t r a t e g i e s . 2 u n t i l 1967, however, that Hase and Goldberg f o r m a l i z e d what has come to be c a l l e d the comparative v a l i d i t y paradigm i n p e r s o n a l i t y assessment r e s e a r c h . As Appendix A shows in g r e a t e r d e t a i l than i s r e q u i r e d here, the r e s u l t s of the best ( f a i r e s t and most comprehensive) v a l i d i t y s t u d i e s have not r e s o l v e d the i s s u e of s t r a t e g y s u p e r i o r i t y . The f i n d i n g s to date suggest that the primary s c a l e c o n s t r u c t i o n s t r a t e g i e s ( r a t i o n a l , e m p i r i c a l , f a c t o r a n a l y t i c ) are almost equal in p r e d i c t i v e e f f e c t i v e n e s s when given a common item pool and common set of c r i t e r i a to p r e d i c t . While there has been a n o t i c e a b l e movement away from e m p i r i c a l approaches toward r a t i o n a l approaches i n the p e r s o n a l i t y assessment l i t e r a t u r e , evidence fo r the s u p e r i o r i t y of the l a t t e r i s l e s s than c o n c l u s i v e . I n t e r e s t i n g l y , what the comparative v a l i d i t y s t u d i e s have served to convey more than anything e l s e i s that a l l primary s t r a t e g i e s have performed at a very low r a t e of v a l i d i t y . These rather unimpressive f i n d i n g s have l e d some comparative v a l i d i t y r e s e a r c h e r s to suggest that f u t u r e e f f o r t s be d i r e c t e d to i n d i v i d u a l item improvement and the o p t i m i z a t i o n of a r a t i o n a l or c o n s t r u c t methodology which seemed to be performing somewhat b e t t e r than the others ( c f . , Hase & Goldberg, 1967; Goldberg, 1972; B u r i s c h , 1978). T h i s study re p r e s e n t s a merging of s t r a t e g y development with s t r a t e g y comparison. The major aim of t h i s study i s to attempt a refinement of the c l a s s i c a l r a t i o n a l approach to p e r s o n a l i t y i n v e n t o r y c o n s t r u c t i o n from w i t h i n a Hase and Goldberg (1967) comparative v a l i d i t y design that employs peer 3 r a t i n g s on the same p e r s o n a l i t y dimensions as c r i t e r i a . S p e c i f i c a l l y , t h i s study o f f e r s f o r v a l i d a t i o n a new v a r i a n t on the r a t i o n a l theme of inventory design that w i l l be termed a prototype s c a l e c o n s t r u c t i o n s t r a t e g y . The name of t h i s s t r a t e g y was borrowed from the Roschian notion of prototype and a p p l i e d here as an o r g a n i z a t i o n a l p r i n c i p l e f o r d i s p o s i t i o n a l c o n s t r u c t s . The a p p l i c a t i o n of Rosch's i n n o v a t i v e idea of prototype to the realm of p e r s o n a l i t y t e s t c o n s t r u c t i o n i s best c o n s i d e r e d in the broader context from which i t arose. C a t e g o r i e s , T r a i t s , and Prototypes The process of a s s i g n i n g people scores on p e r s o n a l i t y t r a i t dimensions (e.g. dominance) i s a process of c a t e g o r i z a t i o n . P e r s o n a l i t y i n v e n t o r i e s are s a i d to provide measurements along an assortment of these t r a i t dimensions s e l e c t e d to represent the way people vary or d i f f e r i n the r e a l world. Some have c o n s i d e r e d the f i e l d of p e r s o n a l i t y "as the general psychology of i n d i v i d u a l d i f f e r e n c e s " (Wiggins, Renner, C l o r e , and Rose, 1971). How do people d i f f e r ? For the p e r s o n o l o g i s t they d i f f e r p h e n o t y p i c a l l y ( b e h a v i o r a l l y ) and g e n o t y p i c a l l y ( c o n s t i t u t i o n a l l y ) . The term " t r a i t " has come to stand f o r both of these meanings. T r a i t s (or d i s p o s i t i o n s , or p r o c l i v i t i e s , or p r o p e n s i t i e s ) are what p e r s o n a l i t y i n v e n t o r i e s purport to measure. P h e n p t y p i c a l l y , t r a i t s can be d e f i n e d as tendencies 4 , that i s , t e n d e n c i e s . . . ...to act or not to act i n c e r t a i n ways on c e r t a i n o c c a s i o n s . These tendencies are not " p o s t u l a t e d " , they are accepted from common sense as expressed i n o r d i n a r y language usage. I f persons are not more or l e s s prone to behave i n c e r t a i n ways on c e r t a i n o c c a s i o n s , then the psychometric approach i s out of business at the o u t s e t ; as are a l l approaches to p e r s o n a l i t y study. (Wiggins, 1974, p. 6) Thus i t can be s a i d that i n d i v i d u a l s tend to behave i n c e r t a i n ways on c e r t a i n o c c a s i o n s . T r a i t s i n t h i s context r e f e r to a t t r i b u t e s of behavior. Some i n d i v i d u a l s may tend to be seen as t y p i c a l l y s e l f - s e r v i n g and others as f o r c e f u l , or dominant, or a l o o f , and so on. T h i s usage of the term " t r a i t " i s d e s c r i p t i v e and d e s c r i p t i o n of i n d i v i d u a l s i s the primary f u n c t i o n of p e r s o n a l i t y i n v e n t o r i e s . G e n o t y p i c a l l y the concept of t r a i t has come to r e f e r to "generative mechanisms or determining tendencies that are r e s p o n s i b l e f o r behavior c o n s i s t e n c i e s " , (Wiggins, 1978, p. 6). T h i s usage of the term t r a i t as c a u s a l d i s p o s i t i o n i s explanatory r a t h e r than d e s c r i p t i v e and has proved i n d i s p e n s i b l e as a h e u r i s t i c t o o l i n the work of mainstream p e r s o n a l i t y t h e o r i s t s such as A l l p o r t , Murray, and C a t t e l l . Buss and Cr a i k (1980, in press) have argued f o r a 5 s y n t h e s i s of the two d i s t i n c t usages of t r a i t s (phenotypic and genotypic) by adopting what A l s t o n (1975) c a l l s a frequency concept of d i s p o s i t i o n s . According to A l s t o n , when we say a person i s dominant (and i s possessed of a dominant d i s p o s i t i o n ) a l l t h i s means i s that (s)he w i l l emit a l a r g e number of responses that are c o n t a i n e d w i t h i n a p r e d e f i n e d category of a c t s subsumed by the l a b e l "dominance". The d i s p o s i t i o n a l or g e n o t y p i c a l component of t r a i t s e x i s t s i n the form of c o n s t r u c t s (what A l s t o n c a l l s P-C concepts) c o n c e p t u a l l y embedded w i t h i n a nomological network that i s fa r beyond the reach of a p s y c h o l o g i s t and h i s d i a g n o s t i c instruments. Yet d i s p o s i t i o n a l c o n s t r u c t s , says A l s t o n , have consequences which s p i l l over i n t o b e h a v i o r a l a c t s . P s y c h o l o g i s t s i n t e r p r e t a person's t e s t scores ( b e h a v i o r a l a c t s ) to be a r e p r e s e n t a t i o n or i n d i c a n t of u n d e r l y i n g d i s p o s i t i o n a l s t r u c t u r e s that are t h e o r e t i c a l l y r e l a t e d i n . a system of i n t e r l o c k i n g laws (see Cronbach and Meehl, 1955, Torgerson, 1958, Wiggins, 1973). T r a i t s need not be s t r u c t u r a l l y isomorphic with behavior s i n c e the behaviors that d e f i n e t r a i t s are c u l t u r a l l y d e f i n e d . That i s , an ag g r e s s i v e set of a c t s f o r one c u l t u r e may not be agg r e s s i v e for another. T h i s has l e d Wiggins (1974, p. 34) to suggest that "the o r g a n i z a t i o n or ' s t r u c t u r e ' of t r a i t s may w e l l r e s i d e w i t h i n a p a t t e r n of i n t e r r e l a t e d i n s t i t u t i o n a l r u l e s , r a t h e r than w i t h i n i n d i v i d u a l s " . Although the p h i l o s o p h i c a l i s s u e of the o n t o l o g i c a l s t a t u s of t r a i t s in the r o l e of g e n e r a t i v e mechanisms i s not 6 addressed by A l s t o n ' s frequency concept of d i s p o s i t i o n , the importance of behavior as a s o c i a l l y d e f i n e d i n d i c a n t of c o g n i t i v e s t r u c t u r e remains unchanged. B e h a v i o r a l a c t s that are given t r a i t d e s i g n a t i o n s may be t o p o g r a p h i c a l l y d i s s i m i l a r but are s a i d to comprise c a t e g o r i e s of a c t s that represent an u n d e r l y i n g s t r u c t u r e or u n i t y of some form or another. J a c c a r d (1974) has found that P e r s o n a l i t y Research Form (PRF) and C a l i f o r n i a P s y c h o l o g i c a l Inventory (CPI) dominance s c a l e s c o r r e l a t e d more h i g h l y (.60) with F i s h b e i n ' s (1972) m u l t i p l e act c r i t e r i o n (MAC), a b e h a v i o r a l c r i t e r i o n i n v o l v i n g m u l t i p l e behaviors ( t o p o g r a p h i c a l l y d i s s i m i l a r ) a c r o s s m u l t i p l e s i t u a t i o n s , than with a s i n g l e a c t , s i n g l e o b s e r v a t i o n c r i t e r i o n (.20). The MAC stems from one view of d i s p o s i t i o n s and the c o n s i s t e n c y they impart to behavior. T r a d i t i o n a l d e f i n i t i o n s of p e r s o n a l i t y (e.g. A l l p o r t , 1937) emphasize d i s p o s i t i o n s which i n f l u e n c e behavior a c r o s s s i t u a t i o n s and which l e a d to c o n s i s t e n c y in behavior. F i s h b e i n and Ajzen (1974) have i d e n t i f i e d three types of c o n s i s t e n c y i n behavior: (a) In a given s i t u a t i o n an i n d i v i d u a l may make one and only one response every time (s)he i s i n that s i t u a t i o n , ( i d e n t i c a l response c o n s i s t e n c y ) (b) S i t u a t i o n s e l i c i t t o p o g r a p h i c a l l y d i f f e r e n t responses, but a l l these responses can be c a t e g o r i z e d on one s i d e of the t r a i t dimension ( a l l dominant or a l l non-dominant) 7 ac r o s s s i t u a t i o n s or i n one s i t u a t i o n . (c) Across or in the same s i t u a t i o n , d i f f e r e n t responses f a l l on one side of the t r a i t dimension to the same o v e r a l l degree. These d e f i n i t i o n s of c o n s i s t e n c y l e a d to d i f f e r e n t types of assumed d i s p o s i t i o n s : (a) S p e c i f i c d i s p o s i t i o n s : one s i t u a t i o n - one response (b) General d i s p o s i t i o n to perform a c l a s s of beh a v i o r s , a l l of which are f u n c t i o n a l l y e q u i v a l e n t . (c) General d i s p o s i t i o n to perform behaviors to a c e r t a i n degree. If a t r a i t d i s p o s i t i o n r e f e r s to the o v e r a l l degree o f , e.g., dominance, e x h i b i t e d i n an i n d i v i d u a l ' s behavior, but not any s p e c i f i c behavior per se , then a t r a i t measure should be h i g h l y r e l a t e d to MAC, but not n e c e s s a r i l y to any s i n g l e a c t . Jac c a r d ' s study (1974) supports t h i s c o n c l u s i o n . Buss and Craik (1980, in press) subjected the Jaccard study to r e p l i c a t i o n in t h e i r a n a l y s i s of a frequency conception of dominance and p r o t o t y p i c a l l y dominant a c t s . On the b a s i s of research on n a t u r a l c a t e g o r i e s c a r r i e d out by Rosch (1975) and her c o l l e a g u e s (Rosch and Mervis 1975), Buss 8 and C r a i k proposed that a category of dominant a c t s c o u l d be s t r u c t u r e d i n t o two c l a s s e s : prototype and non-prototype. The term prototype i n v o l v e s a d i f f e r e n t view of c o g n i t i v e c a t e g o r i e s than has been h e l d t r a d i t i o n a l l y i n the past. While the t r a d i t i o n a l view holds that members of a given category are e q u i v a l e n t in terms of t h e i r d e f i n i n g f e a t u r e s , a Roschian view holds that category members are organized around prototypes or best examples. N o n - p r o t y p i c a l cases d i f f e r i n the degree to which they resemble p r o t o t y p i c a l cases. In the t r a d i t i o n a l model, c l a s s boundaries are d i s c r e t e and c l e a r l y d e f i n e d . In c o n t r a s t Roschian c a t e g o r i e s have l o o s e l y - d e f i n e d boundaries and are l i k e n e d to "fuzzy s e t s " (Rosch and Mervis, 1975) whose membership i s continuous rather than d i s c r e t e . When we imagine i n s t a n c e s of a n a t u r a l category such as "flower" we can e a s i l y d i s t i n g u i s h between good ( p r o t o t y p i c a l ) and bad ( n o n p r o t o t y p i c a l ) members. A rose and a t u l i p f o r example tend to be more f l o w e r - l i k e than a p o i n s e t t i a or water l i l l y . Thus a rose would be more p r o t o t y p i c a l (an instance par e x c e l l e n c e ) than a p o i n s e t t i a that i s nonetheless a v a l i d member of the flower category. I t i s i n t h i s sense that prototypes provide the semantic " p i v o t " around which category members are organized. As one moves away from the prototype, items i n c r e a s i n g l y become f u z z i e r u n t i l they merge with items from other r e l a t e d c a t e g o r i e s . Thus a continuum can be s a i d to e x i s t from c e n t r a l or p r o t o t y p i c a l members to p e r i p h e r a l or n o n - p r o t o t y p i c a l ones. 9 Rosch (1978) r e f e r s to the concept of prototype as a "convenient grammatical f i c t i o n " to r e f e r to the f a c t that prototypes do not e x i s t in nature. Rather, prototypes r e f e r to judgments about c l a s s members i n terms of t h e i r goodness of f i t (ranging from c l e a r to fuzzy) i n t o n a t u r a l c a t e g o r i e s . Rosch (1973) has shown that judges can r e l i a b l y r a t e the p r o t o t y p i c a l i t y of members of n a t u r a l c a t e g o r i e s . Cantor and M i s c h e l (1979 a, 1979 b) have conducted s t u d i e s that suggest that Rosch's view of the way we c a t e g o r i z e n a t u r a l o b j e c t s such as f u r n i t u r e , f r u i t , and animals, can a l s o apply to types of people, such as c u l t u r e d person and e m o t i o n a l l y unstable person. Further g e n e r a l i z a t i o n of the prototype approach has been c a r r i e d out i n the area of c a t e g o r i e s of s o c i a l s i t u a t i o n s (Cantor, M i s c h e l , & Schwartz, 1980), i n the domain of p s y c h i a t r i c diagnoses, (Cantor, et a l . , 1980; Horowitz, French, & Anderson, 1980) and with s o c i a l o c c u p a t i o n a l c a t e g o r i e s (Cohen, 1977). The frequency concept of dominance and p r o t o t y p i c a l l y dominant a c t s and the m u l t i p l e a ct c r i t e r i o n i t e n t a i l s were i n v e s t i g a t e d i n a s e r i e s of three s t u d i e s by Buss and C r a i k (1980). Study 1 i n v o l v e d a dominant act generation task f o r 75 male and female undergraduate s u b j e c t s . Subjects were asked to think of three of the most dominant people they knew (once each f o r males and females) and write down f i v e dominant a c t s or behaviors they might t y p i c a l l y perform. Study 2 was a p r o t o t y p i c a l i t y r a t i n g task f o r 57 male and female undergraduate v o l u n t e e r s and 22 expert judges. Each 10 p a r t i c i p a n t was asked to rat e the 200 a c t s (100 a c t s with a female as the a c t o r and 100 a c t s with a male as the a c t o r ) d e r i v e d from Study 1 on the extent to which they were p r o t o t y p i c a l l y dominant. Acts were a l s o i n c l u d e d from a p e r u s a l of dominance s c a l e s c u r r e n t l y i n use, e.g., Jackson, 1967, e t c . . I n s t r u c t i o n s f o r making judgments of p r o t o t y p i c a l i t y were adapted from Rosch and Mervis (1975). Approximately h a l f of the undergraduate r a t e r s provided s o c i a l d e s i r a b i l i t y r a t i n g s f o r the female a c t o r statements and the remaining h a l f d i d the same for male act o r statements. In Study 3 the MAC c r i t e r i o n was t e s t e d . I t was hypothesized that p r o t o t y p i c a l l y dominant a c t s would be p r e d i c t e d with greater accuracy than more p e r i p h e r a l members of the c a t egory. E i g h t y - t h r e e male and female v o l u n t e e r s completed dominance s c a l e s from the CPI, the PRF, and the Act Report (100 a c t s d e r i v e d from Study 1). Subjects made "yes" or "no" responses depending on whether or not they had ever performed the a c t s i n q u e s t i o n . I f a "yes" answer was checked, s u b j e c t s were asked to i n d i c a t e the frequency with which they performed the act on a t h r e e - p o i n t s c a l e . A l l p a r t i c i p a n t s were r e t e s t e d f o l l o w i n g a one week i n t e r v a l . The r e s u l t s of Study 1 and Study 2 suggest that r a t e r s are i n c o n s i d e r a b l e agreement about which a c t s are p r o t o t y p i c a l l y dominant and s o c i a l l y d e s i r a b l e ; the average alpha r e l i a b i l i t y c o e f f i c i e n t was +.95. C o r r e l a t i o n s among the mean panel r a t i n g s were s u b s t a n t i a l and s i g n i f i c a n t beyond the .001 l e v e l . In Study 3, the h y p othesis that 11 performance of p r o t o t y p i c a l l y dominant a c t s (using a m u l t i p l e - a c t c r i t e r i o n ) would be p r e d i c t e d with g r e a t e r accuracy than a c t s r a t e d low on p r o t o t y p i c a l i t y was confirmed. C o r r e l a t i o n s of the CPI, PRF, and S e l f - R a t i n g with m u l t i p l e act c r i t e r i a v a r y i n g i n degree of p r o t o t y p i c a l i t y showed a decrease i n magnitude as the p r o t o t y p i c a l i t y of the c r i t e r i a decreased. The Prototype Str a t e g y On the b a s i s of the p i o n e e r i n g r e s e a r c h on the concept of prototypes by Rosch (1975, 1978) and her c o l l e a g u e s (Rosch and Mervis, 1975) and Buss and Cr a i k (1980), the present comparative v a l i d i t y research was designed to compare a prototype s c a l e c o n s t r u c t i o n s t r a t e g y with t r a d i t i o n a l s t r a t e g i e s . What s p e c i f i c a l l y i s a prototype s c a l e c o n s t r u c t i o n s t r a t e g y ? I t i s a s t r a t e g y that views p e r s o n a l i t y inventory s c a l e s as c a t e g o r i e s of items r e p r e s e n t i n g fuzzy s e t s . Not u n l i k e the Buss and Cr a i k a n a l y s i s of se t s of dominant a c t s , t h i s study proposes that p e r s o n a l i t y t e s t items can be r a t e d a c c o r d i n g to t h e i r judged p r o t o t y p i c a l i t y ( c e n t r a l versus p e r i p h e r a l category membership) of the t r a i t dimensions to which they belong (or are judged to belong). I t i s hypothesized that s c a l e s c o n s t r u c t e d ' from p r o t o t y p i c a l items w i l l demonstrate g r e a t e r p r e d i c t i v e n e s s than s t r a t e g i e s developed by t r a d i t i o n a l means, e.g., r a t i o n a l , f a c t o r a n a l y t i c , and e m p i r i c a l . 12 Since s c a l e s h e r e t o f o r e developed from r a t i o n a l and c o n s t r u c t s t r a t e g i e s have been g a i n i n g p r e f e r r e d s t a t u s i n the l i t e r a t u r e (e.g., B u r i s c h , 1978, Goldberg, 1971, Jackson, 1980, Knudson and Golding, 1974, Wiggins, 1973) a prototype s t r a t e g y , to prove e f f e c t i v e , must c l e a r l y improve upon these s t r a t e g i e s as w e l l as the o t h e r s . The A d j e c t i v e Check L i s t , developed f i r s t by Gough (1957) then l a t e r m o d i f i e d by H e i l b r u n to i n c l u d e t h e o r e t i c a l s c a l e s based on Murray's n e e d - t r a i t system (Gough and H e i l b r u n , 1965, 1980), was chosen to form the item pool i n t h i s study. Comparison of item o v e r l a p between t h e o r e t i c a l s c a l e s and prototype s c a l e s are i n c l u d e d i n t h i s comparative v a l i d i t y a n a l y s i s . 13 CHAPTER TWO: METHOD Su b j e c t s . Two hundred and t h i r t y - f o u r undergraduate f r a t e r n i t y males from the U n i v e r s i t y of B r i t i s h Columbia were p a i d to p a r t i c i p a t e i n approximately two hours of p s y c h o l o g i c a l t e s t i n g that was c a r r i e d out over a p e r i o d of s i x weeks i n the s p r i n g of 1980. Among the tasks completed by a l l s u b j e c t s were (a) a package of s e l f - r e p o r t m a t e r i a l s i n c l u d i n g demographic and s o c i o m e t r i c data and the ACL, (b) a package of peer r a t i n g s of f i v e f e l l o w f r a t e r n i t y members per s u b j e c t on e i g h t t r a i t c a t e g o r i e s , and (c) peer nominations on s i x s o c i a l c r i t e r i a and a s e l f - r e p o r t e d index of academic achievement (GPA). Item P o o l . A common item p o o l , that of the A d j e c t i v e Check L i s t (Gough and H e i l b r u n , 1980), was used f o r a l l s c a l e s . Before any of the s c a l e s c o u l d be c o n s t r u c t e d , however, a s e r i o u s problem inherent i n the standard format of the ACL had to be d e a l t with. The problem has to do with the f a c t that ACL respondents d i f f e r widely in the t o t a l number of a d j e c t i v e s they check. For example, as c i t e d i n the 1980 ACL manual (p.5), the range of t o t a l number of a d j e c t i v e s checked f o r a sample of 1,364 men was from 13 to 298, with a mean of 91.05 and a standard d e v i a t i o n of 38.35. Since the ACL s c a l e scores are a c u t e l y s e n s i t i v e to the number of a d j e c t i v e s checked, v a r i o u s procedures have been o f f e r e d to c o r r e c t f o r t h i s d i f f i c u l t y . The t e s t authors have proposed a 1 4 fou r - c a t e g o r y system based on the number of a d j e c t i v e s checked to c o r r e c t f o r these checking tendencies. But, as Rorer (1972) r i g h t f u l l y p o i n t s out, under t h i s system two i n d i v i d u a l s who happen to check the same number of a d j e c t i v e s on a s c a l e can r e c e i v e very d i f f e r e n t standard s c o r e s . An i n d i v i d u a l can r e c e i v e a higher score than someone e l s e on a s c a l e even though s/he has checked only one item more than the o t h e r . Such problems have l e d some c r i t i c s (e.g., Rorer, 1972) to c h a r a c t e r i z e the ACL as a psychometric nightmare. However, another method e x i s t s f o r a d j u s t i n g the raw ACL scores so as to c o r r e c t f o r checking tendency (Goldberg, p e r s o n a l communication*). T h i s method i n v o l v e s i p s a t i z i n g each s u b j e c t ' s responses. This i s accomplished by s t a n d a r d i z i n g ( Z - s c o r i n g ) each s u b j e c t ' s d i s t r i b u t i o n of 300 bina r y responses (using each respondents mean and standard d e v i a t i o n f o r t o t a l number checked), thus c o n v e r t i n g i t to a new d i s t r i b u t i o n that i s quasi-normal. Using t h i s method, as was done here, checking tendencies do not confound s c a l e s c o r e s . S c a l e s . Six s e t s of e i g h t s c a l e s were compared, each set c o n s t r u c t e d by a d i f f e r e n t s t r a t e g y . The s t r a t e g i e s of t e s t c o n s t r u c t i o n compared were: (a) pro t o t y p e , (b) e m p i r i c a l , (c) f a c t o r a n a l y t i c , (d) r a t i o n a l , (e) i n t e r n a l c o n s i s t e n c y , and (f) random. The random set of s c a l e s was i n c l u d e d as a b a s e l i n e of p r e d i c t i v e u t i l i t y f o r a l l s c a l e s generated from 15 the ACL item p o o l . Each s c a l e was c o n s t r u c t e d from among the 300 a d j e c t i v e items in the ACL item p o o l . The c o n s t r u c t i o n of each of the e i g h t s c a l e s e t s w i l l be b r i e f l y d e s c r i b e d . R a t i o n a l S c a l e s . A set of e i g h t r a t i o n a l s c a l e s was developed to represent the c u r r e n t s t a t e of the a r t in a c o n s t r u c t approach to s c a l e c o n s t r u c t i o n . The e i g h t r a t i o n a l s c a l e s used in t h i s study are c o n t a i n e d i n the standard v e r s i o n of the ACL: Achievement (Ach), Dominance (Dom), Nurturance (Nur), A f f i l i a t i o n ( A f f ) , E x h i b i t i o n (Exh), Autonomy (Aut), Aggression (Agg), and Deference ( D e f ) . The number of items in these s c a l e s ranged from 34 to 46. Each of these v a r i a b l e s , as w e l l as seven o t h e r s , was s e l e c t e d by H e i l b r u n as r e p r e s e n t i n g a d i s p o s i t i o n w i t h i n Murray's ( 1 9 3 8 ) need-press system. Three c o n s i d e r a t i o n s determined the s e l e c t i o n of need ( t r a i t ) c a t e g o r i e s made by H e i l b r u n : (1) each c o u l d be d e f i n e d i n terms of observable behavior, (2) each seemed r e l e v a n t to p e r s o n a l i t y f u n c t i o n i n g w i t h i n a normal p o p u l a t i o n , and (3) there were a v a i l a b l e (Edwards, 1954) c o n v e n i e n t l y s i m p l i f i e d d e s c r i p t i o n s of the Murray v a r i a b l e s to a i d i n s e l e c t i o n of the items. The e i g h t s c a l e s s e l e c t e d f o r use i n t h i s study were judged to be the most r e l e v a n t and h i g h l y used p e r s o n a l i t y c o n s t r u c t s from the a v a i l a b l e 15 and were assumed to be e a s i l y understood by the r a t e r s . In d e r i v i n g h i s Need S c a l e s , H e i l b r u n gave Edwards' (1954) d e s c r i p t i o n s of the v a r i a b l e s to 19 graduate students 16 i n psychology and asked them to judge which a d j e c t i v e s , i f endorsed, would i n d i c a t e the presence of each need i n the endorsers. H e i l b r u n r e q u i r e d that there be at l e a s t 9 out of 19 agreements fo r the i n c l u s i o n of an a d j e c t i v e i n a s c a l e . T h i s same procedure was used to s e l e c t items which, i f endorsed, would c o n t r a - i n d i c a t e the presence of a given need. The raw scores f o r these s c a l e s were computed from the a l g e b r a i c sum (with the c o n t r a - i n d i c a t i v e items keyed n e g a t i v e l y ) of the two s e t s of a d j e c t i v e s f o r each s c a l e . The number of items i n the t h e o r e t i c a l s c a l e s ranged from 34 to 46. I n t e r n a l c o n s i s t e n c y s c a l e s . These s c a l e s were developed from subsets of items of the standard r a t i o n a l s c a l e s that were h i g h e s t i n i n t e r n a l c o n s i s t e n c y (Kuder-Richardson 20). I n t e r n a l c o n s i s t e n c y s c a l e s f o r the ei g h t t r a i t dimensions were a b b r e v i a t e d as: cAc, cDo, cNu, cAf, cEx, cAu, cAg, and cDe. The number of items in the c o n s i s t e n c y s c a l e s ranged from 19 to 26. Prototype S c a l e s . A set of e i g h t prototype s c a l e s was developed from r a t i n g s of item p r o t o t y p i c a l i t y made by 24 graduate students in psychology (13 Ph.D.'s, 11 M.A.'s). Judges were asked to rate the 300 ACL items a c c o r d i n g to how w e l l they f i t i n t o , or a p p l i e d to, a given t r a i t category, as d e f i n e d by Murray (1938). Each judge completed t h i s task 1 7 using a 7-place u n i p o l a r L i k e r t s c a l e ranging from "very p o o r l y a p p l i c a b l e " to "extremely a p p l i c a b l e " f o r two t r a i t c a t e g o r i e s . The i n s t r u c t i o n s , adapted from Rosch and Mervis (1975) were as f o l l o w s : T h i s study has to do with what we have i n mind when we use words which r e f e r to c a t e g o r i e s . L e t ' s take the word "red" as an example. Close your eyes and imagine a true red. Now imagine an orangish red... imagine a purple red. Although you might s t i l l name the orange-red or the p u r p l e -red with the term "red", they are not as good examples of red (as c l e a r cases of what "red" r e f e r s to) as the c l e a r " t r u e " red. In sh o r t , some reds are redder than o t h e r s . The same i s t r u e f o r other kinds of c a t e g o r i e s . Think of dogs. We a l l have some notion of what a " r e a l " dog, a "doggy dog" i s . To me a r e t r i e v e r or a German shepherd i s a very doggy dog. Not i c e that t h i s kind of judgment has nothing to do with how you l i k e the t h i n g ; you can l i k e a p u r p l e red b e t t e r than a true red but s t i l l r ecognize that the c o l o r you l i k e i s not a true red. You may p r e f e r to own a Pekinese without t h i n k i n g that i t i s the breed that best represents what people mean by dogginess. In t h i s s p e c i f i c study you are asked to judge how good an example of a t r a i t category (e.g., the category of DOMINANCE), v a r i o u s a d j e c t i v e s are ( e . g . , a s s e r t i v e ) . E n c l o s e d you w i l l f i n d 2 c o p i e s of a booklet c o n t a i n i n g 300 a d j e c t i v e s . The c a t e g o r i e s are _. ._ and _. For each category you are to rate how w e l l each a d j e c t i v e a p p l i e s to (or f i t s i n t o ) that category on a 7-point s c a l e . A "7" means that you f e e l the a d j e c t i v e a p p l i e s extremely well to what ._ or _• i s a c c o r d i n g to the d e f i n i t i o n s p r o v i d e d ; a "1" means you f e e l the a d j e c t i v e a p p l i e s very p o o r l y to your idea of what the category i s (or i s not a member at a l l ) . A "4" means you f e e l the a d j e c t i v e f i t s moderately w e l l . Use the other numbers of the 7-point s c a l e to i n d i c a t e i n t e r mediate judgments. On the next page you w i l l f i n d a name and a d e f i n i t i o n of the f i r s t category of 19 person you are to rate and the r a t i n g , s c a l e to use. Use t h i s page f o r re f e r e n c e while r a t i n g the a p p l i c a b i l i t y of the f i r s t set of a d j e c t i v e s to t h i s category. Simply p l a c e the a p p r o p r i a t e number from the s c a l e i n the box beside each a d j e c t i v e . The f o l l o w i n g page c o n t a i n s the same info r m a t i o n f o r the second c a t e g o r y . Use i t f o r re f e r e n c e while r a t i n g the second set of a d j e c t i v e s . I n t e r - r a t e r agreement among the judges of a d j e c t i v e p r o t o t y p i c a l i t y was assessed through the c a l c u l a t i o n of ANOVA t a b l e s and i n t r a c l a s s c o r r e l a t i o n s (ICCs) a c c o r d i n g to the procedures o u t l i n e d by Shrout and F l e i s s (1979). T h i s design avoids the problem of t i e d rankings for the a d j e c t i v e s (a problem which negates the use of K e n d a l l ' s c o e f f i c i e n t of concordance) by a n a l y z i n g the r a t i n g data through a Target X Judges two-way ANOVA. The formula f o r the c a l c u l a t i o n of the mean r a t i n g i n t r a c l a s s c o r r e l a t i o n u t i l i z e s the between-judges mean squares and the r e s i d u a l mean squares terms from the ANOVA summary t a b l e to y i e l d an unbiased estimate f o r the r e l i a b i l i t y of the s i x judges' (per category) average r a t i n g . The r e s u l t s i n d i c a t e d high i n t e r - r a t e r agreement with ICCs ranging from .76 to .88 with a mean of .82. In an attempt to make the prototype s c a l e s as comparable as p o s s i b l e to the r a t i o n a l s c a l e s , " c o n t r a - i n d i c a t i v e " items 20 were i d e n t i f i e d from the l i s t s of p r o t o t y p i c a l i t y r a t i n g s that were rated the lowest f o r each category. The number of a p r o t o t y p i c a l items used i n each of the s c a l e s was determined by the number used i n the r a t i o n a l s c a l e s . A f f i l i a t i o n s c a l e s were the only ones without negative items. Raw scores f o r the s c a l e s were computed from the a l g e b r a i c d i f f e r e n c e between the number of p o s i t i v e and negative items. The e i g h t prototype s c a l e s were l a b e l l e d : pAc, pDo, pNu, pAf, pEx, pAu, pAg, and pDe. The number of items i n each s c a l e ranged from 14 to 34. Fa c t o r S c a l e s . In c o n s t r u c t i n g f a c t o r s c a l e s from the ACL, problems are presented i f an attempt i s made to f a c t o r a l l the 300 items at once. F i r s t , attempting to f a c t o r 300 v a r i a b l e s poses too high a demand on the memory c a p a b i l i t i e s of today's computers. Another problem r e s u l t s from d i f f e r e n c e s i n s o c i a l d e s i r a b i l i t y among t e s t items. C o r r e l a t i o n s between items are at t e n u a t e d by d i f f e r e n c e s i n s o c i a l d e s i r a b i l i t y , s p e c i f i c a l l y those which are r a t e d e i t h e r very high or very low on t h i s dimension (Goldberg, 1967). Thus extreme items are best candidates f o r e l i m i n a t i o n because they confound the r e s u l t i n g f a c t o r s . To deal with these problems, the 300-item pool was d i v i d e d i n t o three subpools of 100 items each based on high, medium, and low s o c i a l d e s i r a b i l i t y r a t i n g s (the o r i g i n a l r a t i n g s were provided by Gough & Heibrun, 1980). The next stage i n v o l v e d 21 f a c t o r i n g items in the medium range of d e s i r a b i l i t y by a p r i n c i p a l components a n a l y s i s followed by an o b l i q u e f a c t o r r o t a t i o n to simple s t r u c t u r e v i a the computer program PROMAX from the MIDAS s t a t i s t i c a l package, (Fox & Guire, 1976). The t h i r d stage of a n a l y s i s i n v o l v e d c o r r e l a t i n g the remaining 200 ACL items with the e i g h t p r e l i m i n a r y s c a l e s . New items were r e t a i n e d f o r the s c a l e s whose c o r r e l a t i o n f o r any i n d i v i d u a l f a c t o r was >.20 , with the p r o v i s i o n that they d i d not c o r r e l a t e higher with any of the remaining f a c t o r s . These "second round" s c a l e s were then f a c t o r e d and r o t a t e d once aga i n . Those items l o a d i n g highest on the e i g h t p r i n c i p a l f a c t o r s were i n c l u d e d i n the f i n a l set of f a c t o r s c a l e s . The number of items in each s c a l e was matched with the r a t i o n a l s c a l e s . The f a c t o r s c a l e s were l a b e l l e d F1 to F8. E m p i r i c a l S c a l e s . E i g h t s c a l e s were developed by the e m p i r i c a l c o n t r a s t i n g groups s t r a t e g y a c c o r d i n g to the procedures o u t l i n e d by Edwards (1970). C r i t e r i o n and c o n t r o l groups were e s t a b l i s h e d on the b a s i s of the peer r a t i n g s gathered from the f r a t e r n i t y s u b j e c t s on the e i g h t t r a i t dimensions used in t h i s study (achievement, dominance, e t c . ) . F i r s t a d e r i v a t i o n sample was c r e a t e d by randomly t a k i n g h a l f of the group of 234 s u b j e c t s . Within the d e r i v a t i o n sample, c r i t e r i o n and c o n t r o l groups were determined by rank o r d e r i n g the peer scores on each of the e i g h t t r a i t s from highest to lowest. C r i t e r i o n and c o n t r o l groups were formed from the top 22 and bottom o n e - t h i r d s u b j e c t s i n these rankings, r e s p e c t i v e l y , f o r the t r a i t c a t e g o r i e s . Under the assumptions that both c r i t e r i o n and c o n t r o l groups have been drawn from a common bino m i a l d i s t r i b u t i o n and that ACL responses are normally d i s t r i b u t e d , standard normal d e v i a t e s can be obtained f o r each item on each t r a i t dimension. Those items with Z > 1 . 5 were r e t a i n e d f o r the f i n a l set of s c a l e s . Again, the number of items per s c a l e was matched with the r a t i o n a l s c a l e s . Although these e m p i r i c a l s c a l e s were contaminated with v a l i d i t y i n f o r m a t i o n from the s t a r t , and would be expected to do b e t t e r than they c o u l d otherwise, t h i s 'edge' over the other s t r a t e g i e s w i l l be taken i n t o c o n s i d e r a t i o n when the f i n d i n g s are i n t e r p r e t e d . The e m p i r i c a l s c a l e s were a b b r e v i a t e d as: eAc, eDo, eNu, eAf, eEx, eAu, eAg, and eDe. Random Sc a l e s . Items were s e l e c t e d randomly from the ACL i n order to c o n s t r u c t a set of e i g h t s c a l e s to provide a p r e d i c t i v e base l i n e a g a i n s t which- every other s t r a t e g y c o u l d be compared. These e i g h t random s c a l e s were keyed randomly. I n s e r t Table 1 about here Table 1 l i s t s the s c a l e s and the a b b r e v i a t i o n s used to represent each of the e i g h t s c a l e c o n s t r u c t i o n s t r a t e g i e s . It a l s o summarizes some psychometric p r o p e r t i e s of the eig h t TABLE 1 Some PsychofnetrIc Properties of the Eight Scales Consructed by Each of Six Strategies Scale name Number of Items Rel1abi11ty Number of 1 terns overlapping with Rational scales KR20 SB-R Prototype scales Achievement (pAch) 38 .87 . 15 27 Dominance (pDom) 40 .87 . 14 20 Nurturance (pNur) 4 e .88 . 14 24 A f f 1 1 tat Ion (pAf f) 34 .85 . 14 13 E x h i b i t i o n (pExh) 46 .81 .08 22 Autonomy (pAut) 44 .82 .09 14 Aggression (pAgg) 44 .86 . 12 23 Deference (pOef) 44 .86 . 12 19 Empirical scales Achievement (pAch) 38 .80 . 10 20 Dominance (pDom) 40 .76 .07 20 Nurturance (pNur) 46 .82 .09 26 A f f 1 1 l a t 1 o n (pAf f ) 34 .86 . 15 18 E x h i b i t i o n (pExh) 46 .81 .08 25 Autonomy (pAut) 44 .84 . 1 1 15 Aggression (pAgg) 44 .79 .08 18 Deference (pDef) 44 .83 . 10 30 Factor scales F 1 38 .89 . 18 17 F2 40 .85 12 16 F3 • 46 .90 . 16 20 F4 34 .89 . 19 10 F5 46 .83 . 10 14 F6 44 .85 . 1 1 8 F7 44 .86 12 19 F8 44 .86 . 12 . 18 Rational scales Achievement (rAch) 38 .82 . 1 1 38 Dominance (rDom) 40 .82 . 10 40 Nurturance (rNur) 46 .86 . 12 46 A f f 1 1 1 a t ion (rAf f ) 34 .87 . 16 34 E x h i b i t i o n (rExh) 46 . 8 1 .08 46 Autonomy (rAut) 44 .80 .08 44 Aggression (rAgg) 44 .86 . 12 44 Deference (rDef) 44 . 8 1 .09 44 Consistency scales Achievement (cAch) 26 .86 . 19 26 Dominance (cDom) 19 .86 .24 19 Nurturance (cNur) 28 .86 . 18 28 A f f 1 1 l a t I o n ( c A f f ) 24 .89 .25 24 E x h i b i t i o n (cExh) 26 .86 . 19 26 Autonomy (cAut) 24 .84 . 18 24 Aggression (cAgg) 25 .88 .23 25 Deference (cDef) 25 .82 . 15 25 Random scales R 1 38 .86 . 14 0 R2 40 .88 . 15 3 R3 46 .90 . 16 2 R4 34 .81 . 1 1 4 R5 46 .69 . 15 7 RE 44 .88 . 14 2 R7 44 .80 .08 10 RS 44 .7B .07 13 24 s c a l e s r e p r e s e n t i n g each s t r a t e g y . Included are the number of items in each s c a l e and an estimate of i n t e r n a l c o n s i s t e n c y (KR-20). The KR-20 values were computed from the responses of the e n t i r e sample. A l s o presented i s an i n t e r n a l c o n s i s t e n c y estimate f o r the average item i n the s c a l e , i n column "SB-R". T h i s estimate i s a c t u a l l y a r e v e r s a l of the Spearman-Brown formula i n which the s c a l e l e n g t h i s set to one item. The average o v e r l a p between the r a t i o n a l and the other s t r a t e g i e s i s p r ovided as an i n d i c a t i o n of s c a l e content s i m i l a r i t y . As Table 1 r e v e a l s , a l l s c a l e sets were remarkably r e l i a b l e , even, s u r p r i s i n g l y , the random s c a l e s . In f a c t , the most r e l i a b l e s c a l e (.90) came from random s c a l e number three (R3) which was t i e d with F3. However, as Green et a l (1977) has shown, alpha w i l l be high f o r a l a r g e number of items from a pool of g e n e r a l l y i n t e r c o r r e l a t e d items. As expected, the c o n s i s t e n c y s c a l e s y i e l d e d higher r e l i a b i l i t i e s than the r a t i o n a l s c a l e s from which they were formed. The SB-R column i n d i c a t e s that the f a c t o r and c o n s i s t e n c y items were the most c o h e s i v e . The e m p i r i c a l s c a l e s were s l i g h t l y lower i n r e l i a b i l i t y than the other s c a l e s e t s while the random s c a l e s behaved e x t r a o r d i n a r i l y l i k e the best of the primary s t r a t e g i e s on these measures. Regarding s c a l e o v e r l a p with the r a t i o n a l s c a l e s , the prototype and e m p i r i c a l s c a l e s averaged s l i g h t l y l e s s than 50% item o v e r l a p , the f a c t o r s c a l e s had 40% o v e r l a p , and the random s c a l e s 12% . 25 C r i t e r i o n Measures Twenty-one c r i t e r i a were chosen f o r assessment to represent a broad range of v a r i a b l e s which p s y c h o l o g i s t s t y p i c a l l y seek to p r e d i c t . These c r i t e r i a can be grouped i n t o two major areas: peer r a t i n g s and peer nominations. Grade p o i n t average was used as an a d d i t i o n a l c r i t e r i o n . Peer R a t i n g s . A l l s u b j e c t s were rated by three to f i v e peers on e i g h t t r a i t dimensions using an e i g h t - p l a c e b i p o l a r L i k e r t r a t i n g s c a l e . Each s u b j e c t was rated by at l e a s t three peers s e l e c t e d randomly from h i s f r a t e r n i t y . The t r a i t s r a t e d were the same dimensions assessed throughout t h i s study: Achievement, Dominance, Nurturance, A f f i l i a t i o n , E x h i b i t i o n , Autonomy, Aggression, and Deference. Each subject r e c e i v e d the same t r a i t d e s c r i p t i o n s (adapted from Murray, 1938) that were given the p r o t o t y p i c a l i t y judges and he was i n s t r u c t e d to rate f i v e of h i s f e l l o w f r a t e r n i t y members a c c o r d i n g to how s i m i l a r or d i s s i m i l a r each was to the d e s c r i p t i o n given (choices ranged from "extremely d i s s i m i l a r " to "extremely s i m i l a r " ) . Appendix C c o n t a i n s an example of the forms used. Each s u b j e c t ' s mean r a t i n g on each of the e i g h t t r a i t s was used as the c r i t e r i o n index f o r that t r a i t . Peer Nominations. A l l s u b j e c t s provided peer nominations 26 on an assortment of twelve c r i t e r i a . Each subject was asked to nominate f i v e peers who best f i t the category i n q u e s t i o n (e.g., l e a d e r s h i p ) and to nominate f i v e names that l e a s t f i t . For each category , both high and low nominations were c a l i b r a t e d on a 5-point s c a l e , i . e . , f i r s t nomination scored 5 p o i n t s , second nomination scored 4 p o i n t s , e t c . Subjects were nominated as e i t h e r "high" or "low" on the f o l l o w i n g s i x dimensions: Leadership, L i k e a b i l i t y , Academic P o t e n t i a l , I n t e r a c t i o n s with Women, and Success. Each s u b j e c t ' s raw score was s t a n d a r d i z e d . The standard scores were used as the c r i t e r i o n i n d i c e s . Academic Achievement. As an index of academic achievement each su b j e c t reported h i s cumulative Grade Point Average (GPA), represented by an average percent. 27 CHAPTER THREE: ANALYSIS and RESULTS As i n the Hase and Goldberg (1967) study, t h i s study employed a double c r o s s - v a l i d a t i o n d e s i g n . Stepwise descending m u l t i p l e r e g r e s s i o n equations were a p p l i e d to each of the 21 c r i t e r i a f o r each set of e i g h t s c a l e s . The group of 234 s u b j e c t s was s p l i t randomly in h a l f . Regression equations were developed on the f i r s t h a l f of the sample and a p p l i e d to the second h a l f . L i k e w i s e , r e g r e s s i o n equations were developed on the second h a l f of the sample and a p p l i e d to the f i r s t . T h i s design produces two d e r i v a t i o n m u l t i p l e c o r r e l a t i o n c o e f f i c i e n t s and two c r o s s - v a l i d a t e d c o e f f i c i e n t s fo r each of the s i x sets of s c a l e s on each of the 21 c r i t e r i a . The c r o s s - v a l i d a t i o n m u l t i p l e Rs, i n t h i s study as in o t h e r s , provide the primary i n d i c e s of v a l i d i t y and thus form the b a s i s f o r comparision between s t r a t e g i e s . I n s e r t Table 2 about here Tables 2 & 3 r e v e a l the b a s i c f i n d i n g s of t h i s study. Table 2 presents the average c r o s s - v a l i d a t i o n and d e r i v a t i o n Rs obtained by each s t r a t e g y of s c a l e c o n s t r u c t i o n on the e i g h t t r a i t dimensions they were b u i l t to p r e d i c t . The means of the two c r o s s - v a l i d a t i o n Rs are l i s t e d above the means of the d e r i v a t i o n Rs which are l i s t e d in parentheses. The mean Rs f o r rows and columns are a l s o p rovided. The averaging of 00 CN TABLE 2 Average Multiple Correlation C o e f f i c i e n t s for Scales Constructed by Each Six Strategies on Eight Peer Rated Trait Dimensions* C r 1 "t ©P 1 on Strategy Mean Maximum r Prototype Empirical Factor Ratlonal Cons 1stency Random ACH ( 23 41) ( 15 45) ( 28 43) ( 12 38) ( 09 42) ( 07 31) ( 16 40) .48 DOM ( 32 44) ( 32 43) ( 30 49) ( 30 43) ( 34 44) ( 08 37) ( 28 40) .55 NUR ( 26 43) ( 24 35) ( 24 32) ( 22 38) ( 30 41) ( 08 3D ( 20 37) .53 AFF ( 20 35) ( 19 35) ( 20 30) ( 13 33) ( 20 33) ( 01 28) ( 16 32) .45 EXH ( .42 .49) ( .32 46) ( .32 .40) ( 40 48) ( .24 .41) ( . 10 .21) ( .30 .39) .51 AUT ( . 19 .27) ( . 18 .30) ( .03 .27) ( . 15 .28) ( .07 .30) ( .02" .23) ( . 1 1 .28) .46 AGG ( .30 .41) ( .30 .38) ( .24 .34) ( .27 .42) ( .30 .38) ( .03 .29) ( .24 .37) .45 DEF ( .30 .39) ( .28 .44) ( .29 .47) ( .27 .42) ( .29 .42) ( . 12 .35) ( .26 .42) .50 Mean ( .28 .40) ( .25 .40) ( .24 .38) ( .23 .39) ( .23 .39) ( .04 .29) ( .22 .38) .49 The means of the two cr o s s - v a l i d a t i o n Rs are l i s t e d above the means of the der i v a t i o n Rs (In parentheses). 29 the c o e f f i c i e n t s was based on Z-transformations of the i n d i v i d u a l m u l t i p l e Rs. The column means at the bottom of Table 2 i n d i c a t e the a b i l i t y of the s i x s t r a t e g i e s to p r e d i c t the e i g h t t r a i t dimension c r i t e r i a . Although the column means for the pro t o t y p e , e m p i r i c a l , and f a c t o r a n a l y t i c s t r a t e g i e s are not s i g n i f i c a n t l y d i f f e r e n t from one another, the mean v a l i d i t y c o e f f i c i e n t f o r the prototype s t r a t e g y s i g n i f i c a n t l y exceeds those of both the r a t i o n a l and i n t e r n a l c o n s i s t e n c y s t r a t e g i e s (Z = 3.22, p < .001*). Not s u r p r i s i n g l y , the f i v e primary s t r a t e g i e s were a l l s u p e r i o r i n p r e d i c t i v e n e s s to the c o n t r o l (random) s t r a t e g y . What was s u r p r i s i n g , however, was that the f i v e p r i n c i p a l s c a l e types were not a l l e q u i v a l e n t  in p r e d i c t i v e n e s s as past s t u d i e s have found. I n s e r t Table 3 about here While Table 2 r e v e a l s that the eig h t p e e r - r a t e d t r a i t c r i t e r i a are a l l moderately p r e d i c t a b l e by the f i v e primary s t r a t e g i e s , Table 3 r e v e a l s that only s e l f - r e p o r t e d GPA was p r e d i c t a b l e to a comparable degree f o r the remaining c r i t e r i a . The twelve peer nomination dimensions were v i r t u a l l y u n p r e d i c t a b l e . The column means i n Table 3 show that three of the s c a l e types ( f a c t o r , r a t i o n a l , c o n s i s t e n c y ) *See Appendix B f o r a d i s c u s s i o n of the s t a t i s t i c used. TABLE 3 Average M u l t i p l e C o r r e l a t i o n C o e f f i c i e n t s for Scales Constructed by Each of Six Strategies on Twelve Peer Nomination Dimensions and Self-Reported GPA rn Cr1ter1on Strategy Mean Maximum r Prototype Empirical Factor Ratlonal Consistency Random MLE .05 ( .23) .06 ( .29) . 15 ( .23) .01 (.25) .08 ( 2 9 ) . 14 ( .30) .08 ( .27) .36 LLE .08 ( 3 6 ) . 16 ( 34) .09 ( 30) .04 ( 32) .03 (.33) .00 (.17) .07 (.30) .48 ML I .03 ( 21) . 1 1 ( 23) .01 (.21) . 13 (.26) . 1 1 ( .22) .02 ( .23) .07 ( .23) .44 LLI . 1 1 ( .27) .07 ( 29) .03 ( . 18) .07 ( .24) .08 (.31) .04 (.20) .07 ( 25) .48 MAC .04 ( 25) .09 ( 36) .05 (.32) .03 ( 2 5 ) .02 ( .31) . 14 ( .31) .06 (.30) . 16 LAC .05 ( 31) -.01 ( .39) -.04 ( .22) -.04 ( .31) -.02 (.31) -.05 (.21) -.02 ( 29) .41 MAT .07 ( .25) .02 ( 22) . 1 1 ( 2 9 ) -.03 ( .23) .03 ( 24) .20 (.30) .07 ( .26) .43 LAT .04 ( 39) .03 ( 37) .02 ( 3 1 ) .09 ( 3 2 ) . 12 (.35) .01 ( .24) .05 (.33) .48 MWO .07 (.24) . 12 ( .29) . 11 ( .30) -.11 ( 24) .01 ( .24) .06 (.23) .04 ( 26) .37 LWO .03 ( .35) .08 ( .35) -.04 ( 3 2 ) .06 ( 29) .06 ( .32) . 14 (.24) .06 ( 3D .50 MSU .04 ( .22) .06 ( 31) . 17 ( .24) -. 10 ( .23) . 11 (.28) . 19 ( 2 9 ) .08 ( .26) .38 LSU . 14 ( 3 7 ) . 17 ( .40) .01 (.25) .05 ( 33) . 10 ( .35) -.06 (.18) .07 ( .31) .46 GPA .30 ( 37) .22 ( 34) .09 ( .34) .23 ( .35) .25 ( .34) . 1 1 ( .31) .20 ( .34) .56 Mean .08 (.29) .09 ( 3 2 ) .06 ( 27) .03 (.28) .06 (.27) .07 (.25) .08 (.29) .42 The means of the two c r o s s - v a l i d a t i o n Rs are l i s t e d above the means of the d e r i v a t i o n Rs (In parentheses). 31 a c t u a l l y f a r e d worse in t h i s domain than d i d the random s c a l e s . The row means in both of these t a b l e s i n d i c a t e the extent to which the c r i t e r i a were p r e d i c t a b l e given these p a r t i c u l a r s c a l e c o n s t r u c t i o n s t r a t e g i e s . The "maximum r " column i n Tables 2 & 3 p r o v i d e s an index of "top-end" p r e d i c t a b i l i t y f o r a l l c r i t e r i a given t h i s p a r t i c u l a r sample and item pool responses. T h i s i s an index of what the maximum p o s s i b l e c o r r e l a t i o n between s c a l e and c r i t e r i o n would be with a f i x e d number of items. I n s e r t F i g u r e 1 about here For i n s t a n c e , given the t r a i t of dominance, the highest p o s s i b l e c o r r e l a t i o n f o r a dominance s c a l e c o n t a i n i n g f o r t y items i s .55. F i g u r e 1 p r o v i d e s a graphic r e p r e s e n t a t i o n of the h i g h e s t p o s s i b l e c o r r e l a t i o n s obtained f o r f o r t y dominance s c a l e s ranging i n item l e n g t h from one to f o r t y . "Maximum" s c a l e s were c o n s t r u c t e d by f i r s t c o r r e l a t i n g each of the 300 ACL items with each c r i t e r i o n . Items were then ranked f o r each c r i t e r i o n i n descending order by v i r t u e of t h e i r c o r r e l a t i o n with the c r i t e r i o n . S c a l e s of item l e n g t h p r o g r e s s i v e l y i n c r e a s i n g by one item were then c o n s t u c t e d . The maximum s c a l e l e n g t h was set at the l e n g t h of the r e s p e c t i v e r a t i o n a l s c a l e as were a l l s t r a t e g i e s except the i n t e r n a l c o n s i s t e n c y s t r a t e g y whose s c a l e s were set to the F i g u r e 1: Maximum and b a s e l i n e c o r r e l a t i o n c o e f f i c i e n t s f o r f o r t y dominance s c a l e s . Top l i n e r e presents c o e f f i c i e n t s ( v a l i d i t y ) o b t ained from "maximal" dominance s c a l e s ranging in number of items per s c a l e from 1 to 40. Bottom l i n e r e p r e s e n t s c o e f f i c i e n t s o b t a i n e d from 40 s c a l e s c o n s t r u c t e d randomly. Space between l i n e s s i g n i f i e s room to vary in p r e d i c t i n g peer r a t e d dominance f o r the primary s c a l e c o n s t r u c t i o n s t r a t e g i e s used in t h i s study. DOMINANCE 34 l e n g t h which maximized the i n t e r n a l c o n s i s t e n c y (KR20) of the r a t i o n a l s c a l e s , as d i s c u s s e d e a r l i e r . F i g u r e 1 r e v e a l s some i n t e r e s t i n g f i n d i n g s that were t y p i c a l of a l l the c r i t e r i a . F i r s t , the s c a l e s appear to d e r i v e t h e i r v a l i d i t y from very few items. The curve r i s e s s h a r p l y to a p l a t e a u that i s remarkably s t r a i g h t , with only minor f l u c t u a t i o n . In the case of dominance, the curve begins to reach asymptote at about s i x items. C o n s i d e r i n g a l l of the c r i t e r i a , the average number of items r e q u i r e d to reach asymptote was only e i g h t . Secondly, the v a l i d i t y of the random s c a l e s i s p r o g r e s s i v e l y worsened with the a d d i t i o n of items. The random s c a l e s d e r i v e whatever v a l i d i t y they possess from an average of 2.5 items. The gap between the random and c o r r e l a t e d s c a l e s r e p r e s e n t s the "room" w i t h i n which the s c a l e c o n s t r u c t i o n s t r a t e g i e s have to move in terms of v a l i d i t y . Whereas the random s c a l e s columns i n Tables 2 and 3 provide a p r e d i c t i v e b a s e l i n e to which the primary s t r a t e g i e s are compared, the maximum r columns provide an index of the p r e d i c t i v e c e i l i n g that would c o n s t r a i n any s t r a t e g y r e g a r d l e s s of o r i e n t a t i o n . Since the maximum r column i s sa t u r a t e d with the i d i o s y n c r a t i c response "noise" of t h i s sample, i t i s i n no way intended to imply that i t rep r e s e n t s a v a l i d s t r a t e g y on i t s own. Perhaps the most v a l u a b l e information t h i s column conveys i s that the top end of p r e d i c t a b i l i t y f o r any given c r i t e r i o n does not exceed the rat h e r moderate v a l i d i t y l e v e l of .55. The m u l t i p l e c o r r e l a t i o n c o e f f i c i e n t s presented in 35 Tables 2 and 3 were based on r e g r e s s i o n equations computed by a step-wise procedure. T h i s procedure allows f o r the i n c l u s i o n of p r e d i c t o r v a r i a b l e s i n the r e g r e s s i o n equation only i f they s i g n i f i c a n t l y i n c r e a s e the p r e d i c t a b l e c r i t e r i o n v a r i a n c e in the d e r i v a t i o n sample. As i n the Hase and Goldberg (1967) comparative v a l i d i t y study, s i g n i f i c a n c e was d e f i n e d as accounting f o r at l e a s t 1% of the c r i t e r i o n v a r i a n c e . For the r e g r e s s i o n equations developed f o r the e i g h t t r a i t dimensions, seven to e i g h t s c a l e s f o r each s t r a t e g y were t y p i c a l l y i n c l u d e d . For the peer nomination dimensions the number of s c a l e s i n c l u d e d i n the r e g r e s s i o n equations ranged from three to e i g h t with an average of f i v e s c a l e s per s t r a t e g y meeting at l e a s t the 1% v a r i a n c e c r i t e r i o n . A step-wise m u l t i p l e r e g r e s s i o n procedure was chosen i n s t e a d of a normal m u l t i p l e r e g r e s s i o n technique s i n c e m u l t i p l e Rs u s u a l l y reach a c e i l i n g a f t e r the i n c l u s i o n of only a few v a r i a b l e s . By adopting a normal m u l t i p l e -r e g r e s s i o n procedure, e r r o r v a r i a n c e would have been f o r c e d i n t o some of the equations. 36 CHAPTER FOUR: SUMMARY AND DISCUSSION The f i n d i n g s of t h i s study set i t apart from other comparative v a l i d i t y s t u d i e s i n two important ways. F i r s t , given the Hase and Goldberg (1967) c h a l l e n g e that more novel s c a l e c o n s t r u c t i o n s t r a t e g i e s may e x i s t , but that t h e i r r e l a t i v e worth must be t e s t e d comparatively, the p r i n c i p a l f i n d i n g from t h i s study i s that a prototype s c a l e c o n s t r u c t i o n s t r a t e g y can s i g n i f i c a n t l y improve upon a r a t i o n a l approach. That i s , u n l i k e past comparative v a l i d i t y s t u d i e s , the primary s t r a t e g i e s were found not to be e q u i v a l e n t . In l i g h t of the f a c t that the e m p i r i c a l s c a l e s (ranking second in t r a i t p r e d i c t i v e n e s s ) were admittedly contaminated with v a l i d i t y i n f o r m a t i o n at the outset and would be expected to l o s e v a l i d i t y under standard sampling c o n d i t i o n s , the prototype s c a l e s , a l l - i n - a l l , d i d q u i t e w e l l . Though not s t a t i s t i c a l l y s u p e r i o r to the e m p i r i c a l and f a c t o r s t r a t e g i e s , the prototype s t r a t e g y g i v e s the ACL user more of what s/he seeks, namely v a l i d i t y . Moreover, when one c o n s i d e r s the c e i l i n g and b a s e l i n e v a l i d i t y c o n s t r a i n t s imposed upon a l l s c a l e c o n s t r u c t i o n s t r a t e g i e s u t i l i z i n g the ACL item p o o l , even a piecemeal increment in v a l i d i t y would be welcomed. However, the f a i l u r e to s u b s t a n t i a l l y i n c r e a s e v a l i d i t y c o e f f i c i e n t s by f o c u s i n g on the improvement of s c a l e c o n s t r u c t i o n s t r a t e g i e s (an u n r e a l i s t i c goal given the h i s t o r y of comparative v a l i d i t y s t u d i e s ) lends f u r t h e r support to Goldberg's (1963) s u s p i c i o n that the g u i l t y 37 c u l p r i t s might w e l l be the inventory items themselves. The f a c t that the peer nomination dimensions were completely u n p r e d i c t a b l e upon c r o s s - v a l i d a t i o n may i n d i c a t e worthless peer nominations or p o i n t to a f a i l u r e on the part of the ACL items to map or cover these dimensions. Yet a comparative v a l i d i t y design only t e s t s s c a l e c o n s t r u c t i o n s t r a t e g i e s - i t i n t e n t i o n a l l y sets the item pool c o n s t a n t . The idea i s that a good s t r a t e g y should do a r e l a t i v e l y good job r e g a r d l e s s of the q u a l i t y of t e s t items ( w i t h i n reason). Given more freedom to vary, a prototype s t r a t e g y would be expected to improve p r o p o r t i o n a t e l y . Peer Ratings An i n s p e c t i o n of the i n d i v i d u a l c e l l e n t r i e s of the t r a i t dimension summary t a b l e (Table 2) allows f o r some i n f e r e n c e s regarding the p r e d i c t i o n of the c r i t e r i a used i n the present study. While the peer r a t i n g c r i t e r i a of achievement, dominance, nurturance, a f f i l i a t i o n , e x h i b i t i o n , autonomy , agg r e s s i o n , and deference were r e l a t i v e l y w e l l p r e d i c t e d by many of the s c a l e s used i n t h i s study, i n every case the prototype s c a l e s y i e l d e d higher v a l i d i t y c o e f f i c i e n t s than d i d Gough and H e i l b r u n ' s ACL r a t i o n a l s c a l e s . I t i s i n t e r e s t i n g to note that the r a t i o n a l s c a l e s used h e r e i n were s c a l e s that were r e c e n t l y r e v i s e d by the ACL t e s t authors (1980), and yet there was no gain i n v a l i d i t y as a r e s u l t of t h i s change. The r e v i s e d s c a l e s performed 38 v i r t u a l l y the same as the o r i g i n a l s . T h i s i s a c t u a l l y not too s u r p r i s i n g s i n c e only three of the e i g h t ACL s c a l e s used in the present study were a l t e r e d under the recent r e v i s i o n . Moreover, the s c a l e s that were r e v i s e d (Dom, Def, Agg) were only reduced ( s l i g h t l y ) i n s c a l e l e n g t h ; no new items were added. In r e v i s i n g the ACL s c a l e s Gough and H e i l b r u n made no s u b s t a n t i v e a l t e r a t i o n s to item content. Items were dropped from s c a l e s on the b a s i s of i t e m / s c a l e i n t e r c o r r e l a t i o n s and r e l i a b i l i t y measures. It appears from the 1980 ACL r e v i s i o n , however, that the r a t i o n a l s t r a t e g y employed in the o r i g i n a l v e r s i o n was judged to be not i n need of improvement. Or, perhaps, no means of s u b s t a n t i v e l y improving t h i s s t r a t e g y was r e a d i l y a v a i l a b l e . However, the present study, while p r o v i d i n g the f i r s t f u l l - s c a l e experimental comparison of a prototype s t r a t e g y with r a t i o n a l and other s t r a t e g i e s , i s c e r t a i n l y i n need of r e p l i c a t i o n and extension, e s p e c i a l l y with d i f f e r e n t item p o o l s , e.g., the CPI. The p r e d i c t a b i l i t y of the e i g h t peer r a t i n g c r i t e r i a now w i l l be b r i e f l y d i s c u s s e d i n t u r n . For achievement r a t i n g s i n the present study, the best p r e d i c t o r was the f a c t o r s t r a t e g y (R = .28). The f a c t o r s t r a t e g y d i d c o n s i d e r a b l y b e t t e r than the second-best prototype s t r a t e g y (+ .05) and s u b s t a n t i a l l y out-performed the remaining major s t r a t e g i e s , which b a r e l y exceeded a chance l e v e l of v a l i d i t y . The o v e r a l l p r e d i c t a b i l i t y of Achievement ranked second to l a s t (mean R = .16). Dominance r a t i n g s were p r e d i c t e d best by the c o n s i s t e n c y 39 s t r a t e g y . The step-wise m u l t i p l e r e g r e s s i o n equations f o r the ei g h t c o n s i s t e n c y s c a l e s y i e l d e d a c r o s s - v a l i d a t e d R of .34. T h i s was e s s e n t i a l l y the same l e v e l of v a l i d i t y achieved by the prototype and e m p i r i c a l s t r a t e g i e s (R = .32) which were c l o s e l y f o l l o w e d by the f a c t o r and r a t i o n a l s t r a t e g i e s (R = .30). The average c r o s s - v a l i d a t e d R f o r Dominance was .28 which p l a c e d i t at the upper end of p r e d i c t a b i l i t y i n the present study. Nurturance peer r a t i n g s a l s o were best p r e d i c t e d by the m u l t i p l e r e g r e s s i o n equations produced from the c o n s i s t e n c y s c a l e s (R = .30). Second i n p r e d i c t i v e n e s s f o r t h i s category was the prototype s t r a t e g y (.26), fo l l o w e d by the e m p i r i c a l (.24), and f a c t o r (.24) s t r a t e g i e s . The r a t i o n a l s c a l e s c l o s e l y f o l l o w e d (R = .22). Nurturance proved to be a moderately p r e d i c t a b l e t r a i t category with a mean m u l t i p l e R of .20. For the next t r a i t category, A f f i l i a t i o n , the primary s t r a t e g i e s were v i r t u a l l y e q u i v a l e n t i n t h e i r a b i l i t y to p r e d i c t i t , with the r a t i o n a l s t r a t e g y being the one noteable e x c e p t i o n . The prototype, f a c t o r , and c o n s i s t e n c y s t r a t e g i e s a l l a t t a i n e d an R of .20; the e m p i r i c a l .19; and the r a t i o n a l .13. A f f i l i a t i o n was t i e d with Achievement at the lower end of average p r e d i c t a b i l i t y (mean R = .16). E x h i b i t i o n r a t i n g s were the most p r e d i c t a b l e c r i t e r i o n used in t h i s study. The hig h e s t m u l t i p l e r e g r e s s i o n equations came from the e i g h t prototype s c a l e s (R = .42). Performing at approximately the same l e v e l were the r a t i o n a l s c a l e s (R = 40 .40); c o n s i d e r a b l y lower were the e m p i r i c a l and f a c t o r s c a l e s (R = .32); a l l of which s u b s t a n t i a l l y out-performed the c o n s i s t e n c y s c a l e s (R = .24). The average p r e d i c t a b i l i t y f o r E x h i b i t i o n , i n c l u d i n g the random s c a l e s , was .30. Autonomy r a t i n g s were not w e l l p r e d i c t e d by any set of s c a l e s i n t h i s study, p o s s i b l y because t h i s t r a i t i s l e s s c l e a r l y d e f i n e d (or evaluated) i n our c u l t u r e and consequently more d i f f i c u l t f o r peers to as s e s s . The prototype s t r a t e g y managed an average m u l t i p l e R of .19, c l o s e l y f o l l o w e d by the e m p i r i c a l s c a l e s (R = .18), then r a t i o n a l (R = .15). The c o n s i s t e n c y and f a c t o r s t r a t e g i e s performed c l o s e to chance (.07, .03, .02, r e s p e c t i v e l y ) . T h i s lack of p r e d i c t a b i l i t y was r e f l e c t e d a l s o in the lowest mean column entry in Table 2 (mean R = .11). With Aggression r a t i n g s , the m u l t i p l e r e g r e s s i o n equations f o r the e i g h t prototype s c a l e s , e m p i r i c a l s c a l e s , and c o n s i s t e n c y s c a l e s a l l performed at the same l e v e l (R = .30). The r a t i o n a l s t r a t e g y (R = .27) and f a c t o r s t r a t e g y (R = .24) fo l l o w e d . On the average, Aggression was moderately p r e d i c t e d by the f i v e primary s t r a t e g i e s i n t h i s study (mean R = .24) . The best p r e d i c t i o n of Deference r a t i n g s was obtained from the prototype s t r a t e g y (R = .30) with the f a c t o r (R = .29), c o n s i s t e n c y (R = .29), e m p i r i c a l (R = .28), and r a t i o n a l (R = .27) s t r a t e g i e s performing at the same r e l a t i v e l y high l e v e l . Deference ranked t h i r d in p r e d i c t a b i l i t y i n t h i s study (mean R = .26). 41 Peer Nominations and GPA For the remaining c r i t e r i a , only s e l f r e p o r t e d GPA c o u l d be p r e d i c t e d above chance l e v e l by the f i v e primary s c a l e c o n s t r u c t i o n s t r a t e g i e s . The mean c r o s s - v a l i d a t e d average m u l t i p l e R f o r the random s c a l e s , f o r i n s t a n c e , was .07; the most any other s t r a t e g y c o u l d improve upon t h i s f i n d i n g was an increment of .02 i n the contaminated e m p i r i c a l s c a l e s (mean R = .09). In f a c t , the h i g h e s t i n d i v i d u a l c e l l entry i n Table 3 (R = .20) came from the random s t r a t e g y in the Most A t h l e t i c c a t e gory. The next h i g h e s t c e l l entry (R = .19) a l s o came from the random s t r a t e g y i n the Most Success nomination category. While Table 3 c o n v i n c i n g l y p o r t r a y s the lack of p r e d i c t a b i l i t y of the peer nomination dimensions in t h i s study, i t a l s o underscores the need for double c r o s s -v a l i d a t i o n i n any study p u r p o r t i n g to p r e d i c t s o c i a l judgments (peer nominations or r a t i n g s ) from paper and p e n c i l p e r s o n a l i t y t e s t s . T h i s p o i n t i s most v i v i d l y i l l u s t r a t e d in the case of the e m p i r i c a l s t r a t e g y . The e m p i r i c a l s t r a t e g y produced the h i g h e s t average d e r i v a t i o n Rs, not s u r p r i s i n g l y s i n c e v a l i d i t y i n f o r m a t i o n was used in t h e i r c o n s t r u c t i o n . Yet, upon double c r o s s - v a l i d a t i o n the c o e f f i c i e n t s c o l l a p s e d . A known major f a i l i n g of beta weights in r e g r e s s i o n equations i s t h e i r n o t o r i o u s i n s t a b i l i t y when moved to d i f f e r i n g p o p u l a t i o n s . Even in a r e l a t i v e l y homogeneous sample such as the one in the present study, t h i s i n s t a b i l i t y n e v e r t h e l e s s 42 worked to erode the p r e d i c t i v e e f f e c t i v e n e s s of a l l the s t r a t e g i e s i n t h e i r performance with the twelve peer nomination dimensions. T h i s i s not to say that other f a c t o r s may not have c o n t r i b u t e d to the poor showing of t h i s group of c r i t e r i a . For example, the p a r t i c u l a r c h o i ce of dimensions used in t h i s study, or p o s s i b l e i d i o s y n c r a t i c response s t y l e s a s s o c i a t e d with the nomination task i t s e l f may have acted to d i m i n i s h v a l i d i t y . A s u r p r i s i n g f i n d i n g was that the e m p i r i c a l s t r a t e g y used in t h i s study ( c o n s t r u c t e d with the s p e c i f i c R-R r e l a t i o n s h i p s c o n t a i n e d i n t h i s sample) f a i l e d to h o l d any v a l i d i t y a f t e r double c r o s s - v a l i d a t i o n . On the other hand, s e l f - r e p o r t e d GPA was r e l a t i v e l y w e l l p r e d i c t e d i n t h i s study. The r e g r e s s i o n equations developed from the prototype s t r a t e g y were the best p r e d i c t o r s of grade p o i n t average, y i e l d i n g an average R of .30 . The c o n s i s t e n c y s t r a t e g y came next (R = .25) f o l l o w e d by the r a t i o n a l (R = .23), e m p i r i c a l (R = .22), and f a c t o r (R = .09) s t r a t e g i e s . The maximum r_ obtained from the GPA c r i t e r i o n (r_ = .56) was c o n s i d e r a b l y higher than those r e c e i v e d from the peer nomination c r i t e r i a . In f a c t , i t was the highest obtained in t h i s study although i t was c l o s e to those obtained from the peer r a t i n g s . The d i f f e r e n c e between the average maximum r_ found i n Table 3 (.42) and Table 2 (.49) suggests that the ACL item pool was not as w e l l s u i t e d f o r p r e d i c t i n g peer nominations as i t was peer r a t i n g s and s e l f - r e p o r t e d GPA. 43 A Prototype Strategy and P r o t o t y p i c Acts The second of the two core f i n d i n g s of t h i s study i s that the concept of category p r o t o t y p i c a l i t y g e n e r a l i z e s to the realm of p e r s o n a l i t y t e s t items. L i t t l e i n the way of s u b s t a n t i v e improvements have been made to a r a t i o n a l s c a l e c o n s t r u c t i o n s t r a t e g y s i n c e Murray (1938) f i r s t d e f i n e d and a l p h a b e t i c a l l y ordered h i s numerous t r a i t c a t e g o r i e s , and then Edwards (1954) a p p l i e d them to the task of s c a l e c o n s t r u c t i o n . However, with the advent of Rosch's (1975) imagi n a t i v e c o n c e p t u a l i z a t i o n of n a t u r a l c a t e g o r i e s organized around prototypes or best examples came the idea f o r an a p p l i c a t i o n of her model in the development of m u l t i p l e act c r i t e r i a by Buss and C r a i k (1980). With Buss and C r a i k ' s demonstration that m u l t i p l e - a c t c r i t e r i a are more p r e d i c t a b l e when ordered i n terms of p r o t o t y p i c a l i t y , the way was paved to t e s t these i m p l i c a t i o n s f o r p r e d i c t o r v a r i a b l e s as w e l l . S p e c i f i c a l l y , the Roschian model has been shown to have d i r e c t a p p l i c a t i o n to the refinement of a r a t i o n a l s c a l e c o n s t r u c t i o n s t r a t e g y . T h i s study has shown that a r a t i o n a l s t r a t e g y that i n c o r p o r a t e s the n o t i o n of p r o t o t y p i c a l i t y produces s c a l e s of g r e a t e r p r e d i c t i v e n e s s than a r a t i o n a l s t r a t e g y that does not i n c o r p o r a t e t h i s n o t i o n . The next step i n t h i s l i n e of i n q u i r y would be to bridge the gap between prototype s c a l e s and p r o t o t y p i c a l a c t s (or b e h a v i o r s ) . P r o t o t y p i c a l a c t s are b e t t e r p r e d i c t e d by t r a d i t i o n a l p e r s o n a l i t y s c a l e s than are other c r i t e r i a . 44 S c a l e s composed of p r o t o t y p i c a l items possess g r e a t e r p r e d i c t i v e n e s s than those which do not. Given these two statements of f a c t , i t would be reasonable to hypothesize that a s u b s t a n t i a l increment i n v a l i d i t y would be r e a l i z e d i f prototype s c a l e s were matched with p r o t o t y p i c a l c r i t e r i a . Indeed,, an improvement in the present study would have been to have employed p r e - r a t e d behaviors ( p r o t o t y p i c a l a c t s ) f o r each of the e i g h t t r a i t dimensions s e r v i n g as c r i t e r i a l i n d i c e s . By adopting the method used i n the Buss and C r a i k s t u d i e s , s u b j e c t s r a t i n g t h e i r peers would i n d i c a t e the s p e c i f i c behaviors (and t h e i r frequency) that each have performed. The p r e d i c t i v e v a l i d i t y of s c a l e c o n s t r u c t i o n s t r a t e g i e s would be assessed by c o r r e l a t i n g s c a l e scores with each of the act r e p o r t s c o l l e c t e d f o r each t r a i t dimension. While a l l comparative v a l i d i t y s t u d i e s to date have focused on the p r e d i c t o r s i d e of p r e d i c t o r / c r i t e r i o n v a l i d i t y , an extension i n t o the refinement of c r i t e r i a would expand the scope of the model and c o u l d p r o v i d e an important advance in p e r s o n a l i t y assessment technology. Thus, with the r e p l i c a t i o n and extension of the present study to d i f f e r e n t item p o o l s , d i f f e r e n t types of c r i t e r i a , and d i f f e r e n t types of s u b j e c t s (e.g., females), the d i r e c t i o n f o r r e s e a r c h i n t h i s area of p e r s o n a l i t y assessment seems c l e a r . 4 5 R E F E R E N C E S A l l p o r t , G . W . P e r s o n a l i t y : A p s y c h o l o g i c a l i n t e r p r e t a t i o n , New Y o r k : H o l t , 1 9 3 7 . A l s t o n , W . P . T r a i t s , c o n s i s t e n c y a n d c o n c e p t u a l a l t e r n a t i v e s f o r p e r s o n a l i t y t h e o r y . J o u r n a l f o r t h e T h e o r y o f S o c i a l B e h a v i o r , 1 9 7 5 , 1 7 - 4 8 . A l u m b a u g h , R . V . , H a r r y , G . D . , a n d S w e n e y , A . B . A c o m p a r i s o n o f m e t h o d s f o r c o n s t r u c t i n g p r e d i c t i v e i n s t r u m e n t s . E d u c a t i o n a l a n d P s y c h o l o g i c a l M e a s u r e m e n t , 1 9 6 9 , 2 9 , 6 3 9 - 6 5 1 . A s h t o n , S . G . , a n d G o l d b e r g , L . R . I n r e s p o n s e t o J a c k s o n ' s c h a l l e n g e : T h e c o m p a r a t i v e v a l i d i t y o f p e r s o n a l i t y s c a l e s c o n s t r u c t e d b y t h e e x t e r n a l ( e m p i r i c a l ) s t r a t e g y a n d s c a l e s d e v e l o p e d i n t u i t i v e l y b y e x p e r t s , n o v i c e s , a n d l a y m e n . J o u r n a l o f R e s e a r c h i n P e r s o n a l i t y , 1 9 7 3 , 7 , 1 - 2 0 . B e r g , I . A . M e a s u r i n g d e v i a n t b e h a v i o r b y m e a n s o f d e v i a n t r e s p o n s e s e t s . I n I . A . B e r g a n d B . M . B a s s ( E d s . ) , C o n f o r m i t y a n d d e v i a t i o n . New Y o r k : H a r p e r , 1 9 6 1 , 3 2 8 -3 7 9 . B u r i s c h , M . C o n s t r u c t i o n s t r a t e g i e s f o r m u l t i s c a l e p e r s o n a l i t y i n v e n t o r i e s . A p p l i e d P s y c h o l o g i c a l M e a s u r e m e n t , 1 9 7 8 , 2 , 9 7 - 1 1 . B u t t , D . S . , a n d F i s k e , D . W . A c o m p a r i s o n o f s t r a t e g i e s i n d e v e l o p i n g s c a l e s f o r d o m i n a n c e . P s y c h o l o g i c a l B u l l e t i n , 1 9 6 8 , - 7 0 , 5 0 5 - 5 1 9 . 46 Buss, D.M. and C r a i k , K.H. The frequency concept of d i s p o s i t i o n : Dominance and p r o t o t y p i c a l l y dominant a c t s . J o u r n a l of P e r s o n a l i t , 1980, 43, 379-392. Cantor, N. and M i s c h e l , W. Prototypes i n person p e r c e p t i o n . In L. Berkowitz (Ed.), Advances in experimental s o c i a l psychology. V o l . 12. New York: Academic Press, 1979. (a) Cantor, N. and M i s c h e l , W. P r o t o t y p i c a l i t y and p e r s o n a l i t y : E f f e c t s on f r e e r e c a l l and p e r s o n a l i t y impressions. J o u r n a l of Research in P e r s o n a l i t y , 1979, i n p r e s s , (b) Cantor, N. Smith, E.E., French,R., Mezzich, J . P s y c h i a t r i c d i a g n o s i s as prototype c a t e g o r i z a t i o n . J o u r n a l of Abnormal Psychology, 1980,89, 181-193. Cantor, N., M i s c h e l , W., Schwartz, J . A prototype a n a l y s i s of a naive psychology of s i t u a t i o n s . Unpublished manuscript, P r i n c e t o n U n i v e r s i t y , 1980. C a t t e l l , R.B., and Eber, H.W. Handbook f o r the Sixteen P e r s o n a l i t y Factor Q u e s t i o n n a i r e . Champaign, 111.: I n s t i t u t e f o r P e r s o n a l i t y and A b i l i t y T e s t i n g , 1957. Cohen, C. C o g n i t i v e b a s i s of s t e r e o t y p i n g . Paper presented at the meetings of the American P s y c h o l o g i c a l A s s o c i a t i o n , San F r a n c i s c o , August, 1977. Cronbach, L . J . and Meehl, P.E. Construct v a i l i d i t y i n p s y c h o l o g i c a l t e s t s . P s y c h o l o g i c a l B u l l e t i n , 1955, 52, 281-302. Dicken, C. Good impression, s o c i a l d e s i r a b i l i t y , and acquiescence as suppressor v a r i a b l e s . E d u c a t i o n a l and  P s y c h o l o g i c a l Measurement, 1963, 23, 699-720. 47 Edwards, A.L. Manual f o r the Edwards Personal Preference Schedule. New York: P s y c h o l o g i c a l C o r p o r a t i o n , 1954. Edwards, A.L. The measurement of p e r s o n a l i t y t r a i t s by s c a l e s and i n v e n t o r i e s . New York: H o l t , 1970. Edwards, L., French, R. and Anderson, C. The prototype of a l o n e l y person. Unpublished manuscript, S t a n f o r d U n i v e r s i t y , 1980. Eysenck, H.J., and Eysenck, S.B.G. The Eysenck P e r s o n a l i t y Inventory Manual. San Diego, C a l i f . : E d u c a t i o n a l and I n d u s t r i a l T e s t i n g S e r v i c e , 1963. F i s h b e i n , M. The p r e d i c t i o n of behaviors from a t t i t u d i n a l v a r i a b l e s . In K.K. Sereno and C D . Mortensen (Eds.), Advances in communication r e s e a r c h . New York: Harper, 1 972. F i s h b e i n , M. and Ajzen, I. A t t i t u d e s toward o b j e c t s as p r e d i t o r s of s i n g l e and m u l t i p l e b e h a v i o r a l c r i t e r i a . P s y c h o l o g i c a l Review, 1974, 81, 59-74. Fox, D.J. and Gu i r e , K.E. Documentation f o r MIDAS. U n i v e r s i t y of Michigan Press, 1976. Goldberg, L.R. A model of item ambiguity i n p e r s o n a l i t y assessment. E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 1963, 23, 467-492. Goldberg, L.R. Parameters of p e r s o n a l i t y inventory c o n s t r u c t i o n and u t i l i z a t i o n : A comparison of p r e d i c t i o n s t r a t e g i e s and t a c t i c s . M u l t i v a r i a t e Behavior Research Monograph, 1972, 2. Goldberg, L.R. and S l o v i c , P. Importance of t e s t item 48 content: An a n a l y s i s of a c o r o l l a r y of the d e v i a t i o n h y p o t h e s i s . J o u r n a l of C o u n s e l l i n g Psychology, 1967, 14, 462-472. Gough, H.G. C a l i f o r n i a P s y c h o l o g i c a l Inventory Manual. Palo A l t o , C a l i f . : C o n s u l t i n g P s y c h o l o g i s t s Press, 1957. Gough, H.G. and H e i l b r u n , A.B. The A d j e c t i v e Check L i s t Manual. Palo A l t o , C a l i f . : C o n s u l t i n g P s y c h o l o g i s t s Press, 1965, r e v i s e d 1980. Green, S.B., L i s s i t z , R.W. and Mulaik, S.K. L i m i t a t i o n s of c o e f f i c i e n t alpha as an index of t e s t u n i d i m e s i o n a l i t y . E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 1977, 37, 827-838. G u i l f o r d , J.P. I n t r o v e r s i o n - e x t r o v e r s i o n . P s y c h o l o g i c a l  B u l l e t i , 1934, 31, 331-354. G u i l f o r d , J.P. and Zimmerman, W.S. The Guilford-Zimmerman Temperament Survey, Manual of I n s t r u c t i o n s and I n t e r p r e t a t i o n s . Beverly H i l l s , C a l i f . : Sheridan, 1949. Hase, H.D. and Goldberg, L.R. Comparative v a l i d i t y of d i f f e r e n t s t r a t e g i e s of c o n s t r u c t i n g p e r s o n a l i t y inventory s c a l e s . P s y c h o l o g i c a l B u l l e t i n , 1967, 67, 4, 231-248. Hathaway, S.R., and Mckinley, J.C. The Minnesota M u l t i p h a s i c P e r s o n a l i t y Inventory, ( r e v i s e d ) . New York: P s y c h o l o g i c a l C o r p o r a t i o n , 1951. Hermans, H.J.M. The v a l i d i t y of d i f f e r e n t s t r a t e g i e s of s c a l e c o n s t r u c t i o n i n p r e d i c t i n g academic achievement. E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 1969, 29, 49 877-883, Jaccard, J . J . P r e d i c t i n g s o c i a l behavior from p e r s o n a l i t y t r a i t s . J o u r n a l of Research i n P e r s o n a l i t y , 1974, 7, 358-367. Jackson, D.N. P e r s o n a l i t y Research Form Manual. Goshen, New York: 1967. (b) Jackson, D.N. The dynamics of s t r u c t u r e d p e r s o n a l i t y t e s t s . P s y c h o l o g i c a l Review, 1971, 78, 229-248. Jackson, D.N. The r e l a t i v e v a l i d i t y of s c a l e s prepared by naive item w r i t e r s and those abased on e m p i r i c a l methods of p e r s o n a l i t y s c a l e c o n s t r u c t i o n . E d u c a t i o n a l and  P s y c h o l o g i c a l Measurement, 1975, 35, 361-370. Jackson, D.N. and Messick, S. Content and s t y l e in p e r s o n a l i t y assessment. P s y c h o l o g i c a l B u l l e t i n , 1958, 55, 243-252. Jackson, D.N. and Panounen, S.V. P e r s o n a l i t y s t r u c t u r e and assessment. Annual Review of Psychology, 1980, 31, 503-551 . Knudson, R.M., and Golding, S.L. Comparative v a l i d i t y of t r a d i t i o n a l versus S-R format i n v e n t o r i e s of i n t e r p e r s o n a l behavior. J o u r n a l of Research i n P e r s o n a l i t y , 1974, 8, 111-127. L o v e l l , V.R. Components of v a r i a n c e i n two p e r s o n a l i t y i n v e n t o r i e s . Unpublished d o c t o r a l d i s s e r t a t i o n , S t a n f o r d U n i v e r s i t y , 1964. M i s c h e l , W. P e r s o n a l i t y and assessment. New York: Wiley, 1968. 50 Murray, H.A.. E x p l o r a t i o n s i n p e r s o n a l i t y . New York: Oxford U n i v e r s i t y Press, 1938. Norman, W.T. R e l a t i v e importance of t e s t item content. J o u r n a l of C o n s u l t i n g Psychology, 1963, 27, 166-174. Rosch, E.H. On the i n t e r n a l s t r u c t u r e of p e r c e p t u a l and semantic c a t e g o r i e s . In T.E. Moore (Ed), C o g n i t i v e development and the a c q u i s i t i o n of language. New York: Academic Press, 1973. Rosch, E. P r i n c i p l e s of c a t e g o r i z a t i o n . In E. Rosch and D.B. Ll o y d (Eds.) C o g n i t i o n and c a t e g o r i z a t i o n . H i l l s d a l e , N.J.: Erlbaum, 1978. Rosch, E. C o g n i t i v e r e f e r e n c e p o i n t s . C o g n i t i v e Psychology, 1975, 7, 532-547. (a) Rosch, E. and Mervis, C B . Family resemblences: Stud i e s i n the i n t e r n a l s t r u c t u r e of c a t e g o r i e s . C o g n i t i v e Psychology, 1975, 7, 573- 605. (b) Shrout, P.E. and F l e i s s , J.L. I n t r a c l a s s c o r r e l a t i o n s : Uses in a s s e s s i n g r a t e r r e l i a b i l i t y . P s y c h o l o g i c a l B u l l e t i n , 1979, 86, 420-428. Strong, E.K., J r . Strong V o c a t i o n a l I n t e r e s t Blank. Palo A l t o , C a l i f . : C o n s u l t i n g P s y c h o l o g i s t s Press, 1959. Torgerson, W.S. Theory and methods of s c a l i n g . New York: Wiley, 1958. Wiggins, J.S. P e r s o n a l i t y and p r e d i c t i o n : P r i n c i p l e s of  p e r s o n a l i t y assessment. Reading, Mass.: Addison, 1973. Wiggins, J.S. In defense of t r a i t s . I n v i t e d address to the Ninth Annual Symposium on Recent Developments in the use 51 of the MMPL, held i n Los Angeles, February 28, 1974. Wiggins, J.S., Renner, K.E., C l o r e , G.L. and Rose, R.J. The  psychology of p e r s o n a l i t y . Reading, Mass.: Addison, 1973. Woodworth, R.S. Personal Data Sheet. Chicago, 111.: S t o e l t i n g , 1 9 1 7 . 52 APPENDIX A Comparative V a l i d i t y : The model Given a l a r g e pool of p e r s o n a l i t y items, by what s t r a t e g y might item subsets be assembled to form a m u l t i - s c a l e inventory that would be maximally e f f i c i e n t i n p r e d i c t i n g a d i v e r s e a r r a y of important s o c i a l c r i t e r i a ? T h i s q u e s t i o n , posed by Harold Hase and Lewis Goldberg in 1967, marked the formal beginnings of an important methodological paradigm in p e r s o n a l i t y inventory r e s e a r c h termed the comparative v a l i d i t y paradigm. The paradigm provi d e s f o r s t u d i e s that i n v o l v e an a n a l y s i s of s c a l e c o n t r u c t i o n s t r a t e g i e s , s p e c i f i c a l l y t h e i r d i f f e r e n t i a l p r e d i c t i v e n e s s i n f o r e c a s t i n g sets of common c r i t e r i a . Such an a n a l y s i s i s important f o r any would-be p e r s o n a l i t y t e s t b u i l d e r i n the s e l e c t i o n of a s c a l e c o n s t r u c t i o n s t r a t e g y best s u i t e d to the p r e d i c t i v e task at hand. On the b a s i s of what evidence should a given s t r a t e g y be s e l e c t e d ? Not only does there t y p i c a l l y e x i s t a l a r g e pool of p e r s o n a l i t y t e s t items from which to choose but the growing l i s t of s t r a t e g i e s f o r t h e i r combination o f f e r s c o n t r a d i c t o r y assembly p l a n s . Yet each one promises optimal r e s u l t s . 53 A t y p i c a l comparative v a l i d i t y study s e t s out to t e s t the claims of a r e p r e s e n t a t i v e assortment of v a r i o u s t e s t c o n s t r u c t i o n s t r a t e g i e s i n popular use while h o l d i n g the item pool constant. The chosen item pool i s u s u a l l y one that i s i n high c u r r e n t usage; the s u b j e c t s are u s u a l l y those fo r whom c r i t e r i o n i n f o r m a t i o n i s a v a i l a b l e (often i n the form of peer r a t i n g s ) ; and the c r i t e r i a by which the s t r a t e g i e s are to be judged r e f l e c t g o als commonly a s p i r e d to by a l l of them ( i . e . , a l l dominance s c a l e s , r e g a r d l e s s of the s t r a t e g y by which they were c o n s t r u c t e d , a s p i r e to p r e d i c t a c c u r a t e l y people who are judged by others to be dominant). The S t r a t e g i e s Goldberg (1972, p. 12) d e f i n e d two kinds of i n t i m a t e l y r e l a t e d s t r a t e g i e s - s t r a t e g i e s f o r c o n s t r u c t i n g s c a l e s and s t r a t e g i e s f o r c o n s t r u c t i n g i n v e n t o r i e s - i n the f o l l o w i n g way: A s t r a t e g y of s c a l e c o n s t r u c t i o n may be -d e f i n e d as a systematic procedure f o r grouping and keying item responses so as to form a composite scor e . Correspondingly, a s t r a t e g y of inventory c o n s t r u c t i o n may be d e f i n e d as a systematic procedure f o r grouping and keying items to form a set of s c a l e s from the same item p o o l . The same author o u t l i n e d three assumptions which u n d e r l i e the 54 use of any s t r a t e g y : (a) a t e s t - c o n s t r u c t o r t y p i c a l l y assembles t e s t items that appear, on a p r i o r i grounds, to cover the domain of i n t e r e s t , (b) the items assembled are s a i d to be redundant, that i s , t r a i t s are covered by more than one item, and (c) although any s i n g l e item may have low r e l i a b i l i t y , the r e l i a b i l i t y of items combined i n t o s c a l e s i s s u b s t a n t i a l l y h i g h e r . Although comparative v a l i d i t y s t u d i e s stem from a r e c e n t l y developed assessment model in p e r s o n a l i t y inventory r e s e a r c h , the inventory c o n s t r u c t i o n s t r a t e g i e s they embrace date back to the o r i g i n s of s t r u c t u r e d p s y c h o l o g i c a l t e s t i n g i n North America. Hase and Goldberg (1967) and Goldberg (1972) have i d e n t i f i e d three general c l a s s e s of s t r a t e g i e s f o r developing p e r s o n a l i t y i n v e n t o r i e s which are termed " i n t e r n a l " , " e x t e r n a l " , and " i n t u i t i v e " . An example of an i n t e r n a l s t r a t e g y i s one that employs the technique of f a c t o r a n a l y s i s to d e r i v e s c a l e items. An e x t e r n a l s t r a t e g y adheres to an e m p i r i c a l or c r i t e r i o n group procedure, and the i n t u i t i v e s t r a t e g y i s determined, f o r the most p a r t , by r a t i o n a l or t h e o r e t i c a l c o n s i d e r a t i o n s . Wiggins (1973) d i s t i n g u i s h e s between r a t i o n a l and c o n s t r u c t approaches to the i n t e r p r e t a t i o n of responses i n p e r s o n a l i t y , adjustment, or i n t e r e s t i n v e n t o r i e s . Whereas a r a t i o n a l approach assumes a correspondence between v e r b a l and w r i t t e n r e p o r t s and h y p o t h e t i c a l i n t e r n a l d i s p o s i t i o n s without e x p l i c a t i o n of the p r e c i s e nature of such a l i n k a g e , a c o n s t r u c t approach views a t e s t response as the 55 m a n i f e s t a t i o n of an u n d e r l y i n g p e r s o n a l i t y c o n s t r u c t (p. 398). To p o s t u l a t e a s e r i e s of p e r s o n a l i t y c o n s t r u c t s which are s a i d to u n d e r l i e behavior presupposes the e x i s t e n c e of a theory or i n t e r l o c k i n g system of laws that s t a t e s the i n t e r r e l a t i o n s among c o n s t r u c t s and t h e i r b e h a v i o r a l consequences. In c o n t r a s t , an e m p i r i c a l p o i n t of view postpones a t t a c h i n g r a t i o n a l or c o n s t r u c t s i g n i f i c a n c e to t e s t item responses u n t i l s p e c i f i c R-R r e l a t i o n s h i p s between items and c r i t e r i o n behaviors have been e s t a b l i s h e d . The p r o l i f e r a t i o n of p e r s o n a l i t y , adjustment, and i n t e r e s t i n v e n t o r i e s i n North America was h e r a l d e d by Woodworth's Personal Data Sheet in 1917. Woodworth c o n s t r u c t e d h i s s c a l e s a c c o r d i n g to a r a t i o n a l p o i n t of view; he s e l e c t e d items f o r h i s s c a l e s a c c o r d i n g to t h e i r correspondence or presumed relevance to n e u r o t i c symptomatology. Although c u r r e n t use of t h i s s t r a t e g y has undergone a marked degree of s t a t i s t i c a l s o p h i s t i c a t i o n i n " p u r i f y i n g " r a t i o n a l l y d e r i v e d items, the c o n s t r u c t approach i s becoming more prominent f o r many good p h i l o s o p h i c a l reasons (see Wiggins 1973, pp. 383-384). A recent example of a t e s t c o n s t r u c t e d a c c o r d i n g to a c o n s t r u c t p o i n t of view i s Jackson's (1967) P e r s o n a l i t y Research Form. H i s t o r i c a l l y , the next general s t r a t e g y to appear was the e m p i r i c a l s t r a t e g y . E. K. Strong's V o c a t i o n a l I n t e r e s t Blank (1927) was the major forerunner that arose, along with Hathaway, McKinley, and Meehl's Minnesota M u l t i p h a s i c P e r s o n a l i t y Inventory (1943), as a r e a c t i o n and an 56 a l t e r n a t i v e to the r a t i o n a l approach, not u n l i k e the b e h a v i o r i s t ' s r e a c t i o n to s t r u c t u r a l i s m (Wiggins 1973). As mentioned above, an e m p i r i c a l s t r a t e g y attempts to assemble items that demonstrate p o s i t i v e c o r r e l a t i o n s with designated c r i t e r i a e x t e r n a l to the t e s t i t s e l f . The e m p i r i c a l or group d i s c r i m i n a t i v e s t r a t e g y i s so named because items are d e r i v e d a c c o r d i n g to t h e i r a b i l i t y to d i s c r i m i n a t e s i g n i f i c a n t l y between groups of s u b j e c t s who r e s i d e i n d i f f e r e n t e x t e r n a l l y d e f i n e d c a t e g o r i e s , as do, f o r example, p s y c h o t i c s and normals. Items gain i n c l u s i o n on i n v e n t o r i e s such as the MMPI by v i r t u e of t h e i r e m p i r i c a l r e l a t i o n s h i p with important c r i t e r i a , not t h e i r t h e o r e t i c a l ( c o n s t r u c t ) relevance to them. The C a l i f o r n i a P s y c h o l o g i c a l Inventory (Gough, 1957) i s a more recent example of an e m p i r i c a l l y c o n s t r u c t e d t e s t . I t i s one that has appeared f r e q u e n t l y i n comparative v a l i d i t y s t u d i e s . F a c t o r a n a l y t i c approaches to s c a l e c o n s t r u c t i o n are termed i n t e r n a l s t r a t e g i e s a c c o r d i n g to Hase and Goldberg (1967) s i n c e i t i s the i n t e r n a l s t r u c t u r e of the t e s t items and the t e s t b u i l d e r ' s i n t e r p r e t a t i o n of the f a c t o r p a t t e r n s that determine the number of s c a l e s , t h e i r l a b e l l i n g and item makeup. Items that l o a d h i g h l y on, say, a " s o c i a b i l i t y " f a c t o r gain i n c l u s i o n on the s o c i b i l i t y s c a l e while lower l o a d i n g items are excluded but may load higher on other f a c t o r s . Optimal s c a l e s are s a i d to " a r i s e " out of the i n t e r n a l s t r u c t u r e of the item pool when exposed v i a the technique of f a c t o r a n a l y s i s . G u i l f o r d ' s i n a u g u r a l Nebraska 57 P e r s o n a l i t y Inventory (1934) helped pave the way f o r the Guilford-Zimmerman Temperament Survey (1949). Other notable f a c t o r a n a l y t i c a l l y d e r i v e d t e s t s are C a t t e l l ' s 16 P e r s o n a l i t y F a c t o r Q u e s t i o n n a i r e (1949) and the Eysenck P e r s o n a l i t y Inventory (1964). Past Comparative V a l i d i t y S t u d i e s There have been approximately fourteen s t u d i e s from 1953 to 1978 i n which at l e a s t two s c a l e c o n s t r u c t i o n s t r a t e g i e s have been compared on at l e a s t one c r i t e r i o n . In t h i s s e c t i o n we s h a l l c o n s i d e r more than h a l f of these s t u d i e s i n v a r y i n g amounts of d e t a i l so as to h i g h l i g h t the advancements while o v e r l o o k i n g r e p e t i t i v e m a t e r i a l . The h i s t o r i c a l order i n which they appeared s h a l l be preserved i n the order viewed here: Norman (1963), Hase and Goldberg (1967), Butt and F i s k e (1968), Alumbaugh, Davis, and Sweeney (1969), Goldberg (1972), Ashton and Goldberg (1973), Knudson and Golding (1974), Jackson (1975), and B u r i s c h (1978). I n s e r t Table A about here Table A p r o v i d e s the reader with an overview of the s i m i l a r i t i e s and d i f f e r e n c e s that c h a r a c t e r i z e the nine TABLE I Nine Comparative V a l i d i t y Studies Author(s), Year Subjects Item Pool Cr iterion Procedure Strategies ValId 1ty (rpc) Norman, 1963 215 f r a t e r n l t y mal es D e s c r i p t i v e Adjective Inventory. Welsh Figure Preference Test. Occupational Preference Inventory. peer rat tngs (a) emp1rIca1 (b) r a t i o n a l (a) .00 (b) .30 Hase ft Goldberg. 1967 200 soror1ty women C a l i f o r n i a Psychological Inventory peer rat Ings (a) factor a n a l y t i c (b) emplrtcal (c) theoretleal (d) rat tonal (e) s t y l I s t l c ( f ) random (a) .26 (b) .26 (c) .26 (d) .27 (e) .15 (f ). 10 Butt ft Mske. 1968 200 students 248 a 1rmen Dominance scales from Gough (1951), Edwards (1959). A l l p o r t (1939), Jackson (1966), C a t t e l l (1957). Thurstone (1953) c o r r e l a t i o n s with other psychological test var tables (a) rntlonal facet (b) f a c t o r l a l facet (c) rattonal tra1t (d) f a c t o r l a l t r a i t N/A A1umbaugh et a l . 1964 73 males 82 females 16 Personality Factors neurotles vs. alcoholtes (a) factor analysts (b) stepwlse regression (c) ranked c o r r e l a t i o n s N/A Goldberg, 1972 200 soror1ty women C a l i f o r n i a Psychological Inventory. Minnesota Multiphasic Personality Inventory peer ra 11ngs (a )factor analytIc (b) emp1rlea 1 (c) theoret teal (d) rat tona1 (e) s t y l t s t 1 c (f Jrandom (a) .28 (b) .28 (c) .29 (d) .32 (e) .13 (f ) . 10 Ashton 6 Goldberg. 1973 168 females C a l I f o r n l a Psychological inventory. P e r s o n a l i t y Research Form. Intu111ve Sea 1es peer rat1ngs (a tempirlea 1 (b) theoretleal (c) ratlona1 (d) r a t l o n a l (a) .27 (b) .35 (c) .26 (d) .29 Knudson ft Goldtng, 1974 64 high school seniors Schedule of Interpersonal Response, Rational S-R Inventory, P e r s o n a l i t y Research Form, Interpersonal Check L i s t s e l f & peer ratIngs (a) factor analytIc (b) theoret teal (c) theoretleal (d) theoretleal (a) .09 (b) 26 (c) .16 (d) 37 Jackson. 1975 116 fema1es C a l I f o r n l a Psychological Inventory. I n t u i t i v e Scales, Jackson Psychological Inventory s e l f ft peer ratIngs (a) emplrleal (b) theoretleal (c) r a t l o n a l < a).09 (b) .29 (c) .25 Bur I sen. 1978 78 males 60 females Frelburger Person!Iqhkeltstnventar peer (a)emplrlcal (a).43 ratings (b)factor a n a l y t i c (b).39 (c ) t h e o r e t l e a l ( c).39 59 comparative v a l i d i t y s t u d i e s covered in t h i s s e c t i o n . Included i n t h i s t a b l e are the s u b j e c t s , item pools, c r i t e r i o n procedures, and s c a l e c o n s t r u c t i o n s t r a t e g i e s used in each of theses s t u d i e s . In a d d i t i o n , the " v a l i d i t y " column l o c a t e d at the f a r r i g h t of the t a b l e r e v e a l s the p r i n c i p a l f i n d i n g s of each of these s t u d i e s . While there are some d i s c r e p a n c i e s between f i n d i n g s , a general p a t t e r n of equivalence among s t r a t e g i e s (as i n d i c a t e d by the p r e d i c t o r / c r i t e r i o n r i n the v a l i d i t y column) emerges. Though not c o n c l u s i v e , there does, however, appear to be a t r e n d i n which the r a t i o n a l or t h e o r e t i c a l s t r a t e g i e s perform s l i g h t l y b e t t e r than the o t h e r s . The i m p l i c a t i o n s of these f i n d i n g s are d i s c u s s e d i n what f o l l o w s . Norman's study p r o v i d e s a good i n t r o d u c t i o n to two of the most popular s c a l e c o n s t r u c t i o n s t r a t e g i e s , the e m p i r i c a l and r a t i o n a l approaches. In response to Berg's (1959, 1961) c l a i m that the s p e c i f i c item content of p e r s o n a l i t y t e s t s i s unimportant i n p r e d i c t i n g important s o c i a l c r i t e r i a such as deviance, Norman (1963) c a r r i e d out what can now be c a l l e d a comparative v a l i d i t y study to t e s t t h i s c l a i m . Berg's s t r a t e g y f o r c o n s t r u c t i n g h i s P e r c e p t u a l Reaction Test (PRT) was q u i t e r a d i c a l l y an e m p i r i c a l or c r i t e r i o n group method. He found s i g n i f i c a n t l y d i f f e r e n t response p a t t e r n s on the PRT f o r groups c l a s s e d d i f f e r e n t l y i n p s y c h i a t r i c d i a g n o s i s , i n age, i n s e v e r i t y of p h y s i c a l i l l n e s s , and i n mental r e t a r d a t i o n . On the b a s i s of these and other f i n d i n g s Berg formulated h i s c o n t r o v e r s i a l d e v i a t i o n hypothesis (1957). 60 T h i s hypothesis s t a t e s that a t y p i c a l (deviant) p o p u l a t i o n s , c l a s s i f i e d on the b a s i s of c r i t i c a l behavior p a t t e r n s (e.g., symptoms), s h a l l a l s o respond a t y p i c a l l y i n n o n c r i t i c a l areas of behavior (e.g., i n pr e f e r e n c e s f o r geometric shapes). "Deviance", as used by Berg, denotes a s t a t i s t i c a l l y s i g n i f i c a n t d i f f e r e n c e between responses of two d i s t i n c t p o p u l a t i o n s of people. Likewise, a "deviant response" can r e f e r to any response category f o r which d i f f e r e n c e s in r e l a t i v e frequency of occurrence are observed: "Not merely the behavior d e v i a t i o n s such as s c h i z o p h r e n i a are i n c l u d e d but any abnormal behavior that i s ab-normal i n the l i t e r a l sense of being away from normal", (Berg, 1961, p. 334). Berg p o s i t s much explanatory weight in h i s d e v i a t i o n hypothesis as a u n i f y i n g p r i n c i p l e capable of accounting f o r a wide v a r i e t y of f i n d i n g s . H is P e r c e p t u a l Reaction Test (PRT), composed of 60 a b s t r a c t designs and f i g u r e s to which respondents i n d i c a t e t h e i r l e v e l of p r e f e r e n c e , represents a good (but extreme) example of a p u r e l y e m p i r i c a l t e s t s t r a t e g y . With the many broad i m p l i c a t i o n s of Berg's d e v i a t i o n hypothesis i n mind, Norman (1963) set up a d i f f e r e n t i a l c l a s s i f i c a t i o n problem to t e s t the v a l i d i t y of i r r e l e v a n t versus r e l e v a n t t e s t items. The approach Norman was advo c a t i n g , i n stark c o n t r a s t to Berg's, can be termed a r a t i o n a l s t r a t e g y . It i s a r a t i o n a l s t r a t e g y , however, based on f i v e of C a t t e l l ' s f a c t o r a n a l y t i c a l l y d e r i v e d source t r a i t s or p e r s o n a l i t y f a c t o r s . Although C a t t e l l added 61 t h e o r e t i c a l meaning to h i s source t r a i t s a f t e r he had d e r i v e d them f a c t o r a n a l y t i c a l l y , Norman r a t i o n a l l y c o n s t r u c t e d s c a l e s from a d j e c t i v e s judged to be s i m i l a r to them. Norman developed three i n v e n t o r i e s f o r h i s 1963 study s p e c i f i c a l l y to represent three l e v e l s of item relevance to C a t t e l l ' s f i v e f a c t o r s — h i g h , medium, and low. The h i g h l y r e l e v a n t t e s t was c a l l e d the D e s c r i p t i v e A d j e c t i v e Inventory (DAI) and was composed of 200 " c h e c k - l i s t " a d j e c t i v e s d e r i v e d i n the manner d e s c r i b e d above. The i n t e r m e d i a t e l y r e l e v a n t t e s t was composed of o c c u p a t i o n a l t i t l e s and c a l l e d the Oc c u p a t i o n a l Preference Inventory (OPI), and 400 f i g u r a l designs from the Welsh F i g u r e Preference Test (WFPT) pro v i d e d the set of s t i m u l i f o r the low_ relevance category. The f i n d i n g s were c l e a r - c u t . The e m p i r i c a l s c a l e s from the WFPT had zero v a l i d i t y c o e f f i c i e n t s . In c o n t r a s t , the DAI s c a l e s were a l l p o s i t i v e and s i g n i f i c a n t , averaging .30. Needless to say, Norman's major premise - that i n c e r t a i n kinds of p e r s o n a l i t y assessment, r e l e v a n t t e s t item content i s of c o n s i d e r a b l e importance - was u n e q u i v o c a l l y borne out i n h i s comparative v a l i d i t y study. There had been other s t u d i e s p r i o r to Norman's work that had a l s o sought to compare s c a l e c o n s t r u c t i o n s t r a t e g i e s (e.g., Berkeley, 1953; Gee 1955; H e i l b r u n , 1962; Schumacher, 1959). Though not on the s c a l e of Norman's study and u s u a l l y with only one c r i t e r i o n f o r comparison, these s t u d i e s were s i m i l a r to Norman's in that they sought to compare two s t r a t e g i e s . I t was not u n t i l 1967 when Hase and Goldberg 62 f o r m a l l y i n t r o d u c e d t h e i r comparative v a l i d i t y model that the major s t r a t e g i e s of s c a l e c o n s t r u c t i o n then a v a i l a b l e r e c e i v e d a systematic experimental comparison on a d i v e r s e a r r a y of important e x t e r n a l c r i t e r i a . The C a l i f o r n i a P s y c h o l o g i c a l Inventory (CPI), developed by Gough (1957), was the common item pool f o r the Hase and Goldberg study. Six s t r a t e g i e s were used to c o n s t r u c t eleven s c a l e s that were a l l compared on t h i r t e e n c r i t e r i o n measures. The s u b j e c t s were 200 U n i v e r s i t y of Oregon freshman women. They s u p p l i e d CPI responses, peer r a t i n g s on the chosen t r a i t s of dominance, s o c i a b i l i t y , r e s p o n s i b i l i t y , psychological-mindedness, and f e m i n i n i t y and an assortment of e i g h t s o c i o m e t r i c and demographic c r i t e r i a , i . e . , a peer r a t i n g of how w e l l S was known, S's average number of dates per month; c o l l e g e GPA; achievement r e l a t i v e to a b i l i t y ; c o l l e g e major; and c o l l e g e dropout. The s i x s t r a t e g i e s compared were: f a c t o r a n a l y t i c , e m p i r i c a l group d i s c r i m i n a t i v e , i n t u i t i v e - t h e o r e t i c a l , i n t u i t i v e - r a t i o n a l , s t y l i s t i c - p s y c h o m e t r i c , and random. For the f a c t o r s c a l e s the authors chose not to analyse the whole item p o o l . In the f i r s t p l a c e an a n a l y s i s of a 468 item t e s t such as the CPI posed too high a demand on the computer c a p a c i t y of the day. Secondly, to c o n t r o l f o r attenuated c o r r e l a t i o n s stemming from d i f f e r e n c e s i n the way c e r t a i n items are t y p i c a l y endorsed by respondents (e.g., some items in the CPI are answered in the same d i r e c t i o n by everyone), the CPI was broken down i n t o four subpools of 117 6 3 items each. The c a t e g o r i e s ranged from extremely high item endorsement to r e l a t i v e l y balanced endorsement f r e q u e n c i e s . The item-extremeness l e v e l s were e s t a b l i s h e d from the reponses of 179 U n i v e r s i t y of Oregon freshman women who a l s o p r ovided the sample f o r the eleven f a c t o r s c a l e s developed. Hase and Goldberg began t h e i r f a c t o r a n a l y s i s by f a c t o r i n g the group of items i n the middle range of endorsement. The items i d e n t i f i e d by a Varimax r o t a t i o n of a p r i n c i p a l a x i s a n a l y s i s were augmented by c l u s t e r s t r u c t u r e a n a l y s i s . T h i s program i d e n t i f i e d a d d i t i o n a l items which loaded on the f a c t o r s as i n i t i a l l y d e f i n e d and added to the i n t e r n a l c o n s i s t e n c y of the f a c t o r . The items thus s e l e c t e d f o r each of 14 i n i t i a l f a c t o r s were then r e t a i n e d as d e f i n e r s or "markers" while unused items from L e v e l A (middle range of endorsement) were excluded. (Hase & Goldberg, 1967, p 234) F o l l o w i n g t h i s i n i t i a l a n a l y s i s , s l i g h t l y l e s s balanced items were added to the item p o o l . Items that loaded on the i n i t i a l f a c t o r s were r e t a i n e d as were those which d e f i n e d new dimensions. T h i s same procedure was f o l l o w e d f o r the remaining groups. The e m p i r i c a l s c a l e s used i n t h i s study were the eleven e m p i r i c a l s c a l e s which Gough o r i g i n a l l y developed f o r the 64 CPI. The s c a l e s were d e r i v e d from c o l l e g e and high school students whose peer nominations were used to e s t a b l i s h the c o n t r a s t e d (high-low) groups f o r each of the eleven dimensions. Other a l r e a d y e x i s t i n g groups such as males vs. females and d e l i n q u e n t s vs. nondelinquents were used along with a u t h o r i t a r i a n s vs. n o n a u t h o r i t a r i a n s s e l e c t e d from scores on another q u e s t i o n n a i r e . Hase and Goldberg developed t h e i r t h e o r e t i c a l ( c o n s t r u c t ) s c a l e s on.the b a s i s of Murray's (1938) system of manifest needs. Murray's t h e o r e t i c a l model was chosen because i t has r e c e i v e d high r e c o g n i t i o n i n the area of p e r s o n a l i t y r e s e a r c h , moreover, Murray's system, u n l i k e many ot h e r s , c o n t a i n s w e l l e x p l i c a t e d , c l e a r l y d e f i n e d concepts. These q u a l i t i e s g r e a t l y a i d the task of matching t e s t items with t h e o r e t i c a l concepts, hence the use of Murray's theory in t h i s study. Hase and Goldberg used three advanced graduate students i n c l i n i c a l psychology to s e l e c t items on the b a s i s of Murray's system. The judges chose CPI items that appeared to tap each of the d e s c r i p t i o n s of eleven needs they were g i v e n . Items that were r e t a i n e d f o r the s c a l e s had been agreed upon by at l e a s t two of the three judges. As mentioned e a r l i e r , when b u i l d i n g i n t u i t i v e - r a t i o n a l s c a l e s the i n v e s t i g a t o r s e l e c t s items on the b a s i s of h i s i n t u i t i v e understandidng of the dimension or t r a i t he wishes to a s s e s s . The i n v e s t i g a t o r does not r e l y on any formal p s y c h o l o g i c a l theory to guide h i s s e l e c t i o n p rocess. The CPI c o n t a i n s four s c a l e s c o n s t r u c t e d by t h i s s t r a t e g y : S o c i a l 65 Presence (Sp), Self-Acceptance (Sa), S e l f - C o n t r o l ( Sc), and F l e x i b i l i t y ( F x ) . Gough s e l e c t e d those items which appeared to tap these dimensions and submitted them to a sample of su b j e c t s , then i d e n t i f i e d extreme s c o r e r s on the s c a l e s and r e t a i n e d those items that d i s c r i m i n a t e d between them. In a procedure analogous to Gough's, Hase developed seven i n t u i t i v e - r a t i o n a l s c a l e s . Of the two c o n t r o l s t r a t e g i e s used i n t h i s study very l i t t l e need be s a i d . The " s t y l i s t i c s c a l e s " were c o n s t r u c t e d to t e s t the Jackson and Messick (1958) c l a i m that the major common f a c t o r s i n dichotomous answer p e r s o n a l i t y i n v e n t o r i e s (e.g., t r u e - f a l s e , agree-disagree) are "acquiescence" and " s o c i a l d e s i r a b i l i t y " . Hase and Goldberg used L o v e l l ' s (1964) nine CPI s c a l e s keyed f o r endorsement percentage and d e s i r a b i l i t y r a t i n g s along with Gough's 28-item Communality (Cm) s c a l e and the CPI s o c i a l d e s i r a b i l i t y s c a l e developed by Dicken (1963). F i n a l l y , the random s c a l e s were c o n s t r u c t e d from sets of 25 items, each s e l e c t e d randomly from the 468-item CPI. The procedures f o r keying were a l s o random. "The random s c a l e s p rovided a p r e d i c t i v e base l i n e a g a i n s t which a l l other s t r a t e g i e s c o u l d be compared", (p. 235). T e s t - r e t e s t r e l i a b i l i t y estimates f o r each of the s i x sets of eleven s c a l e s developed by Hase and Goldberg were obtained from double a d m i n i s t r a t i o n s of the CPI to the sample of 179 freshman women, and two i n t e r n a l c o n s i s t e n c y estimates (KR 20 and r i i , an i n t e r n a l c o n s i s t e n c y estimate f o r the average item i n a s c a l e based on a r e v e r s a l of the Spearman-66 Brown c o r r e c t i o n ) d e r i v e d from the same sample of women. These three r e l i a b i l i t y e stimates were found to be h i g h l y i n t e r r e l a t e d . The s c a l e s developed by a f a c t o r - a n a l y t i c s t r a t e g y proved to have higher r e l i a b i l i t i e s than the r e s t . While the s t y l i s t i c s c a l e s possessed a moderate l e v e l of r e l i a b i l i t y they were c o n s i d e r a b l y lower than a l l except the random s c a l e s which, not s u r p r i s i n g l y , were the lowest. The authors a l s o examined two a d d i t i o n a l psychometric p r o p e r t i e s — a v e r a g e i n t e r s c a l e c o r r e l a t i o n s and average number of o v e r l a p p i n g items. The e m p i r i c a l s c a l e s had the h i g h e s t l e v e l of average i n t e r s c a l e c o r r e l a t i o n (+.33) and the r a t i o n a l s c a l e s were only s l i g h t l y lower (+.29). These two s t r a t e g i e s were the only ones for which any s u b s t a n t i a l item o v e r l a p occurred (an average, of about 3 o v e r l a p p i n g items among s c a l e p a i r s ) . The method of a n a l y s i s i n the Hase and Goldberg study was a double c r o s s - v a l i d a t i o n design which produced d e r i v a t i o n and c r o s s - v a l i d a t e d v a l i d i t y c o e f f i c i e n t s f o r a l l of the c r i t e r i a . The b a s i c f i n d i n g of t h e i r study was that the four p r i n c i p a l s t r a t e g i e s were v i r t u a l l y equal in t h e i r p r e d i c t i v e e f f e c t i v e n e s s . The c o n t r o l s c a l e s ( s t y l i s t i c and random) were s u b s t a n t i a l l y lower in p r e d i c t i v e n e s s than the r e s t . The authors i n t e r p r e t e d t h e i r p r i n c i p a l f i n d i n g of e q u i v a l e n c e in p r e d i c t i v e n e s s among the major s t r a t e g i e s of s c a l e c o n s t r u c t i o n as suggesting that s t r a t e g i e s , per se, cannot be h e l d r e s p o n s i b l e f o r the t y p i c a l l y low " p e r s o n a l i t y 67 c o e f f i c i e n t s " r e p o r t e d i n the l i t e r a t u r e . Instead the " g u i l t y c u l p r i t s " may very w e l l be the items themselves. Moreover, Hase and Goldberg p o i n t e d out that "dogmatic a s s e r t i o n s of the s u p e r i o r i t y of one s t r a t e g y over another are premature" (pp. 242-243). In regard to the i s s u e of item relevance and p r e d i c t i v e v a l i d i t y d i s c u s s e d e a r l i e r i n Norman's study, Hase and Goldberg add f u r t h e r support to h i s f i n d i n g s . Of the f i v e peer r a t e d c r i t e r i a (dominance, s o c i a b i l i t y , r e s p o n s i b i l i t y , psychological-mindedness, and f e m i n i n i t y ) the r a t i o n a l s c a l e s s i g n i f i c a n t l y , though not d r a m a t i c a l l y , outperformed a l l o t h e r s . The mean v a l i d i t y c o e f f i c i e n t f o r the r a t i o n a l s c a l e s on these f i v e c r i t e r i a was +.33, and the average c o e f f i c i e n t f o r the e m p i r i c a l s c a l e s was +.24. Since i t s p u b l i c a t i o n the Hase and Goldberg comparative v a l i d i t y study has been c r i t i c i z e d on methodological grounds p r i m a r i l y with respect to the m u l t i p l e c o r r e l a t i o n aproach adopted by the authors and t h e i r use of "suboptimal" techniques in s c a l e c o n s t r u c t i o n and item a n a l y s i s . A l s o , the c h o i c e of c r i t e r i o n measures has been questioned (see B u r i s c h , 1978, and Jackson, 1980). L a t e r on i n t h i s and other s e c t i o n s we s h a l l look c l o s e r at these c r i t i c i s m s and c o n s i d e r procedures suggested to improve the comparative v a l i d i t y model. Butt and F i s k e (1968) compared four d i f f e r e n t s c a l e c o n s t r u c t i o n s t r a t e g i e s f o r developing dominance s c a l e s . The authors f i r s t d i s t i n g u i s h e d between t r a i t and f a c e t s t r a t e g i e s , on the one hand, and f a c t o r i a l and r a t i o n a l 68 s t r a t e g i e s on the o t her. "The d i s t i n c t i o n between t r a i t and f a c e t i n v o l v e s the degree to which the p e r s o n a l i t y v a r i a b l e i s c o n c e p t u a l l y d e l i n e a t e d and s u b d i v i d e d before .scales are developed to measure i t " , (p. 505). In a t r a i t s t r a t e g y f o r developing dominance s c a l e s the t r a i t (dominance) i s assumed to be a r e l a t i v e l y u n i t a r y p e r s o n a l i t y c o n s t r u c t and i s measured at a molar l e v e l . In a f a c e t s t r a t e g y , however, a t r a i t such as dominance i s s a i d to be composed of s e v e r a l elements or f a c e t s (e.g., b e h a v i o r a l and m o t i v a t i o n a l ) and i s measured at a molecular l e v e l . That i s , t r a i t s may be broken down i n t o t h e i r c o n s t i t u e n t p a r t s or f a c e t s and i n d i v i d u a l , homogeneous s c a l e s can be c o n s t r u c t e d to measure each one. While the f a c e t and t r a i t s t r a t e g i e s i n v o l v e a d i s t i n c t i o n between molar and molecular conception p r i o r to the assembly of s c a l e items, f a c t o r i a l versus r a t i o n a l s t r a t e g i e s d i c t a t e how items are to be s e l e c t e d a f t e r a f a c e t or t r a i t d e s i g n a t i o n i s made. According to Butt and F i s k e ' s approach to s t r a t e g y c l a s s i f i c a t i o n , t r a i t versus f a c e t s t r a t e g i e s can be f u r t h e r d i v i d e d i n t o f a c t o r a n a l y t i c or r a t i o n a l item s e l e c t i o n s t r a t e g i e s . The r e s u l t i s four b a s i c s t r a t e g i e s : r a t i o n a l f a c e t , f a c t o r i a l f a c e t , r a t i o n a l t r a i t , and f a c t o r i a l t r a i t . Each s t r a t e g y was represented by more than one item pool, some in common usage (e.g., C a t t e l l ' s 16 PF) and others were c o n s t r u c t e d by the authors s p e c i f i c a l l y f o r t h i s study. U n l i k e the Hase and Goldberg study, where peer r a t i n g s formed the b a s i s of a c r i t e r i o n a n a l y s i s of the s t r a t e g i e s , t h i s 69 study focused on s c a l e s d i f f e r i n g i n scope and s p e c i f i c a t i o n of content that were evaluated i n terms of i n t e r n a l s t a t i s t i c s ( r e l i a b i l i t i e s ) , t e s t i n t e r c o r r e l a t i o n s , c o r r e l a t i o n s with presumably r e l a t e d v a r i a b l e s , and promise of t h e o r e t i c a l c o n t r i b u t i o n . The authors concluded that the f a c e t s t r a t e g y was s u p e r i o r to the t r a i t s t r a t e g y on these measures and that a r a t i o n a l f a c e t s t r a t e g y showed the most promise i n " c o n t r i b u t i n g to the understanding of dominance in terms of theory and i n terms of c l a r i f y i n g the content of p u b l i s h e d dominance s c a l e s " , (p. 518). Alumbaugh, Davis, and Sweney (1969) compared three s t r a t e g i e s f o r r e f i n i n g C a t t e l l ' s 16 PF items: f a c t o r a n a l y s i s , stepwise r e g r e s s i o n , and rankings of item c o r r e l a t i o n s . These methods were used to c o n s t r u c t optimal s c a l e s of 20 items each that were used to d i s c r i m i n a t e a l c o h o l i c s from n e u r o t i c s . The authors chose to study t h i s problem e m p i r i c a l l y in order to provide c o n t r o l f o r t e s t i n g s i g n i f i c a n c e . " I t was hoped that the r e s u l t s from such a venture would give evidence concerning the best s t r a t e g y f o r s e l e c t i n g items in t e s t c o n s t r u c t i o n " , (p.641). A t o t a l of 12 s c a l e s were c o n s t r u c t e d to represent these three methods. D i s c r i m i n a n t f u n c t i o n s were used to t e s t the p r e d i c t i v e n e s s of the 12 s c a l e s on d e r i v a t i o n and c r o s s - v a l i d a t i o n samples. The authors concluded, on the b a s i s of number of m i s c l a s s i f i c a t i o n s , that f a c t o r a n a l y s i s and stepwise r e g r e s s i o n techniques o f f e r e d g r e a t e r d i s c r i m i n a b i l i t y over c o r r e l a t i o n a l rankings. Factor s c a l e s showed grea t e r 70 g e n e r a l i t y when c r o s s - v a l i d a t e d than d i d the step-wise r e g r e s s i o n s t r a t e g y , which suggests a tendency for the l a t t e r method to c a p i t a l i z e on s a m p l e - s p e c i f i c v a r i a n c e . Goldberg (1972) submitted the data o r i g i n a l l y r e p o r t e d in the Hase and Goldberg (1967) study to an e x t e n s i v e r e a n a l y s i s . Three major sources of v a r i a n c e inherent i n p e r s o n a l i t y i n v e n t o r i e s were i n v e s t i g a t e d : (a) the s c a l e c o n s t r u c t i o n s t r a t e g y , (b) the number of p r e d i c t o r s u t i l i z e d , and (c) the type of p r e d i c t i o n f u n c t i o n s used. The r e s u l t s confirmed the e a r l i e r f i n d i n g that the three major in v e n t o r y c o n s t r u c t i o n s t r a t e g i e s ( e x t e r n a l , i n t e r n a l , and i n t u i t i v e ) produced e q u i v a l e n t c r o s s - v a l i d i t i e s but a " s i z e a b l e c r i t e r i a - b y - s t r a t e g i e s i n t e r a c t i o n e f f e c t " was a l s o i n evidence. Goldberg d i s t i n g u i s h e d between i n v e n t o r i e s v a r y i n g i n "bandwidth" and " f i d e l i t y " . I n v e n t o r i e s with broad, bandwidths are s a i d to tap a broad range of b e h a v i o r a l outcomes while i n v e n t o r i e s high in f i d e l i t y u s u a l l y have narrower bandwiths but g r e a t e r v a l i d i t y or accuracy in the p r e d i c t i o n s they make. Goldberg found that i n v e n t o r i e s c o n s t r u c t e d v i a an e x t e r n a l s t r a t e g y c o u l d produce broader band-widths but lower f i d e l i t i e s than the i n t u i t i v e or i n t e r n a l s t r a t e g i e s . Neither the p a r t i c u l a r number of s c a l e s used in the p r e d i c t i o n f u n c t i o n s , nor the type of p r e d i c t i o n f u n c t i o n s used (e.g., p r e d i c t o r f u n c t i o n s based on o r i g i n a l versus s t a n d a r d i z e d scores) a l t e r e d these f i n d i n g s . Yet the h i g h e s t average v a l i d i t y value across a l l c r i t e r i a (+.36) was produced from a 71 f i v e - s c a l e R a t i o n a l subset of the 11 o r i g i n a l s c a l e s . Goldberg s t a t e d that "the f i v e - s c a l e set emerged as both a r e l a t i v e l y broad bandwidth and a r e l a t i v e l y high f i d e l i t y instrument", (p. 520). In 1971 Jackson i s s u e d a c h a l l e n g e to p e r s o n a l i t y r e s e a r c h e r s to compare even the most inexperienced item w r i t e r s with the most e l a b o r a t e e m p i r i c a l i t e m - s e l e c t i o n procedures in use. Jackson's e x p e c t a t i o n s i n t h i s regard were c l e a r l y s t a t e d -The author would f u l l y expect under c r o s s - v a l i d a t i o n that even an inexperienced item w r i t e r would be s u p e r i o r to e m p i r i c a l item s e l e c t i o n with a t y p i c a l heterogeneous item p o o l , (p. 238). In 1973 Ashton and Goldberg p u b l i s h e d a response to t h i s c h a l l e n g e . They compared the v a l i d i t y of an e m p i r i c a l s t r a t e g y a g a i n s t s c a l e s c o n s t r u c t e d i n t u i t i v e l y by experts, n o v i c e s , and laymen. F i f t e e n graduate students (novices) in psychology and 15 students with no formal t r a i n i n g i n psychology (laymen) were each p a i d to c o n s t r u c t one 20-item s c a l e to measure s o c i a b i l i t y , achievement, or dominance w i t h i n a two hour time l i m i t . Combined with the P e r s o n a l i t y Research Form and the C a l i f o r n i a P s y c h o l o g i c a l Inventory these s c a l e s were admi n i s t e r e d to 168 women f o r which peer r a t i n g s were a v a i l a b l e . 72 From Ashton and Goldberg's f i n d i n g s i t appears that Jackson's e x p e c t a t i o n s were well-founded. Although s c a l e s developed by inexperienced item w r i t e r s (laymen) had lower average v a l i d i t i e s than e m p i r i c a l l y developed s c a l e s (+.18 and +.27, r e s p e c t i v e l y ) , the average v a l i d i t y of the s c a l e s developed by psychology students (no v i c e s , +.29) and the most r e l i a b l e s c a l e s c o n s t r u c t e d by laymen (+.26) were e s s e n t i a l l y the same as the e m p i r i c a l s c a l e s . Furthermore, the most r e l i a b l e novice s c a l e s produced an average v a l i d i t y (+.34) p r a c t i c a l l y i n d i s t i n g u i s h a b l e from that produced by the t a r g e t e d PRF s c a l e s (+.35), and both were c o n s i d e r a b l y more v a l i d than any of the sets of CPI s c a l e s (+.27). T h i s study can best be summed up by the authors (Ashton and Goldberg, p. 17): C e r t a i n l y the most s t r i k i n g f i n d i n g of the present study was that the average graduate student i n pyschology - i n 2 hours or l e s s - was capable of producing p e r s o n a l i t y s c a l e s of equal r e l i a b l i l i t y and v a l i d i t y to those developed by the f a r more expensive and time-consuming E x t e r n a l s t r a t e g y . Knudson and Golding (1974) compared the v a l i d i t i e s of two "S-R format" i n v e n t o r i e s of i n t e r p e r s o n a l behavior with two t r a d i t i o n a l i n v e n t o r i e s , the P e r s o n a l i t y Research Form and the I n t e r p e r s o n a l Check L i s t ( ICL). An "S-R format" 73 approach or s t r a t e g y f o r inventory c o n s t r u c t i o n i n v o l v e d the s e p a r a t i o n of stimulus s i t u a t i o n s and response c a t e g o r i e s to take i n t o account s i t u a t i o n a l sources of v a r i a n c e i n behavior. Stimulus items and response items are p a i r e d i n a l l p o s s i b l e combinations. I n v e n t o r i e s c o n s t r u c t e d i n t h i s way are s a i d to "permit d e s c r i p t i o n of an i n d i v i d u a l i n terms of the kinds of responses he i s l i k e l y to make in each of the v a r i o u s s i t u a t i o n s " , (p. 112). The two i n v e n t o r i e s c o n s t r u c t e d by the S-R format s t r a t e g y were Kinnane et a l ' s , (1969) schedule of I n t e r p e r s o n a l Responses (SIR) and the R a t i o n a l S-R Inventory ( R a t i o n a l S-R) developed by Knudson and Golding s p e c i f i c a l l y f o r t h e i r study. The v a l i d i t i e s of the four i n v e n t o r i e s were compared on 14 s e l f - r e p o r t and peer r a t i n g c r i t e r i a in groups of 64 male and female high school s e n i o r s . Despite the added s i t u a t i o n a l s p e c i f i c i t y i n the S-R format the t r a d i t i o n a l i n v e n t o r i e s proved to be f a r s u p e r i o r in terms of average v a l i d i t i e s (PRF, .37; ICL, .26; SIR, .09; and R a t i o n a l S-R, .16). The r e s u l t s of t h i s study l e n t f u r t h e r support to a c o n s t r u c t s t r a t e g y of i n v e n t o r y c o n s t r u c t i o n as represented i n Jackson's PRF and the R a t i o n a l S-R which was developed from the SIR a c c o r d i n g to s u b s t a n t i v e or c o n s t r u c t c o n s i d e r a t i o n s . Jackson (1975) c a r r i e d out a r e p l i c a t i o n and extension of the Ashton and Goldberg (1973) study which pr o v i d e d evidence to suggest that novice item w r i t e r s can outperform an e m p i r i c a l s t r a t e g y in terms of t h e i r comparative v a l i d i t y 74 over a wide a r r a y of important c r i t e r i a . Jackson employed a d i f f e r e n t set of p e r s o n a l i t y c o n s t r u c t s , d i f f e r e n t methods fo r o b t a i n i n g peer r a t i n g s , and a d i f f e r e n t p o p u l a t i o n of item w r i t e r s . The item w r i t e r s were 22 undergraduates majoring i n psychology a l l of whom were e n r o l l e d in the author's course i n p s y c h o l o g i c a l t e s t s and measurement. They r e c e i v e d i n s t r u c t i o n i n t e s t c o n s t r u c t i o n , the hypothesis and f i n d i n g s of Ashton and Goldberg's (1973) study, and a 45-minute l e c t u r e on the b a s i c s of item w r i t i n g ( i n c l u d i n g content s a t u r a t i o n and s o c i a l d e s i r a b i l i t y ) . The item w r i t e r s were then given two hours to c o n s t r u c t one 16-item s c a l e based on d e f i n i t i o n s of one of three c o n s t r u c t s : s o c i a l p a r t i c i p a t i o n , t o l e r a n c e , and s e l f - e s t e e m . The f i n d i n g s p r o v i d e d a strong r e p l i c a t i o n of the e a r l i e r study. The v a l i d i t i e s produced by undergraduate majors i n psychology "were almost as high i n v a l i d i t y as those prepared by the Ashton-Goldberg graduate students", (p. 369). According to Jackson, f i n d i n g s such as these can do nothing but "hasten the demise of the unquestioned ascendance of the E x t e r n a l s t r a t e g y of p e r s o n a l i t y s c a l e c o n s t r u c t i o n " (p. 369). In the most recent comparative v a l i d i t y study to date, B u r i s c h (1978) submitted the Goldberg (1972) r e a n a l y s i s of the Hase and Goldbeg (1967) study to s t i l l f u r t h e r a n a l y s i s v i a r e p l i c a t i o n . B u r i s c h attempted to c o r r e c t some pr o c e d u r a l d i f f i c u l t i e s inherent i n the Goldberg design (mentioned e a r l i e r ) stemming from the use of a m u l t i p l e c o r r e l a t i o n approach i n s t e a d of attempting to e s t a b l i s h convergent and 75 d i s c r i m i n a n t v a l i d i t y . ' " T r a i t - r e l e v a n t " c r i t e r i a were a l s o used i n s t e a d of the p u r p o r t e d l y "ad hoc" c r i t e r i a used by Goldberg (Jackson, 1980). As regards the f i r s t d i f f i c u l t y , B u r i s c h makes the f o l l o w i n g p o i n t e d o b s e r v a t i o n : If the s o c i a b i l i t y s c a l e i n an inventory does not c o r r e l a t e with a peer r a t i n g of s o c i a b i l i t y , while s c a l e s f o r dominance and " i n t e l l e c t u a l e f f i c i e n c y " do, the m u l t i p l e c o r r e l a t i o n between the whole set and the r a t i n g can be q u i t e h i g h . Should i t then be claimed that " v a l i d i t y " has been demonstrated? In f a c t , as Tables 6 and 7 i n Hase and Goldberg (1967) show, some c r i t e r i a were most h i g h l y c o r r e l a t e d with the "wrong" p r e d i c t o r s , (p. 99) There were other problems with the Goldberg study. The techniques f o r c o n s t r u c t i n g the s c a l e s and the item analyses were found to be l a c k i n g , as were the c r i t e r i o n measures. In the e a r l i e r s t u d i e s the item a n a l y s i s f o r the r a t i o n a l s c a l e s only e l i m i n a t e d items that c o r r e l a t e d below +.19 with t h e i r own s c a l e , while c o r r e l a t i o n s with other s c a l e s were not in s p e c t e d . There was item o v e r l a p among s c a l e s , the r e l a t i o n s of c r i t e r i a to p r e d i c t o r s was u n c l e a r , and no e m p i r i c a l data were used to p u r i f y the t h e o r e t i c a l ( c o n s t r u c t ) s c a l e s . The number of items per s c a l e was not held c o n s t a n t . F i n a l l y there were sample e f f e c t s , e.g., the f a c t o r s c a l e s were 76 c o n s t r u c t e d from " p r a c t i c a l l y the same set of data as that used f o r v a l i d a t i o n purposes", ( B u r i s c h , 1978, p. 101). In an attempt to reduce the handicap of methodological impurity p l a g u i n g the e a r l i e r Goldberg s t u d i e s , B u r i s c h c o r r e c t e d f o r these and other problems i n h i s r e p l i c a t i o n study, e.g., by p a i n s t a k i n g l y anchoring 100-point r a t i n g s c a l e s , and so on. Yet, in the f i n a l a n a l y s i s , the r e s u l t s of the two s t u d i e s were the same. B u r i s c h found that very l i t t l e i n the way of v a r i a t i o n in v a l i d i t y c o u l d be a t t r i b u t e d to the major c o n s t r u c t i o n s t r a t e g i e s . He suggested that the next step should i n v o l v e "more s e r i o u s c r i t e r i a and o p t i m i z a t i o n of the deductive ( c o n s t r u c t ) methodology", (p. 110). 77 APPENDIX B Testing S i g n i f i c a n c e of Average M u l t i p l e Rs Using Dependent Samples Glass and Stanley (1970, pp.313-314) have developed a s t a t i s t i c a l t e s t f or comparing v a l i d i t y c o e f f i c i e n t s obtained from p r e d i c t o r s operating from the same sample, a s t a t i s t i c d i r e c t l y a p p l i c a b l e to the present study. The type of s i t u a t i o n the authors envisioned for t h i s s t a t i s t i c was one in which more than one p r e d i c t o r was a v a i l a b l e from a sample yet only one p r e d i c t o r could be used. The s t a t i s t i c t e s t s the hypothesis that the observed d i f f e r e n c e between two sample Rs (say, Rxy and Rxz) represents a true d i f f e r e n c e between two population Rs (Pxy and Pxz). S p e c i f i c a l l y , the n u l l hypothesis to be t e s t e d i s that c r i t e r i o n X has the same c o r r e l a t i o n with two p r e d i c t o r s , Y and Z, against the a l t e r n a t i v e hypothesis that Pxy and Pxz are not equal. Ho: Pxy = Pyz Ha: Pxy * Pyz The t e s t s t a t i s t i c f o r t e s t i n g Ho against Ha i s : Vn( Rxy - Rxz ) Z = (1-Rxy) + (1-Rxz) - 2Ryz - (2 Ryz-RxyRyz)(1-Rxy-Rxz-Ryz) where n i s the sample s i z e , Rxy i s the sample average m u l t i p l e c o r r e l a t i o n of Y Rxz i s the sample average m u l t i p l e c o r r e l a t i o n of Z and Ryz i s the sample c o r r e l a t i o n of Y and Z. The assumptions underlying t h i s technique are (1) that there e x i s t three b i v a r i a t e - n o r m a l p o p u l a t i o n s , one f o r each of the three Rs used i n the equation, (2) that the sample of sub j e c t s i s random, and (3) that the c o e f f i c i e n t s compared are not independent. Plugging i n to the above formula with the r e s u l t s from Table 2 we get : prototype R = Rxy = .28 , r a t i o n a l R = Rxz = .23 , 7 9 w i t h Ryz = .97 and f i n d : \ [234 ( . 2 8 - . 2 3 ) Z = ( . 8 4 9 3 + . 8 9 7 0 - 1 . 8 2 5 3 - ( 1 . 8 7 5 6 ) ( - . 0 7 2 2 ) . 7 6 4 9 . 2 3 7 5 = 3 . 2 2 0 6 * * p < . 0 0 0 1 80 APPENDIX C Pe e r - r a t i n g and Peer-nomination Forms (1) P e e r - r a t i n g i n s t r u c t i o n s and r a t i n g s c a l e s . For each person, p l a c e a check (V) on the a p p r o p r i a t e l i n e a c c o r d i n g to how d i s s i m i l a r or s i m i l a r he i s t o the d e s c r i p t i o n g i v e n . ACHIEVEMENT D e f i n i t i o n : s t r i v e s to be outstanding or e x c e l l e n t i n p u r s u i t s of s o c i a l l y r e cognized s i g n i f i c a n c e , as i n h i s career or school work. f i r s t person to be r a t e d Extremely Very Quite S l i g h t l y S l i g h t l y Quite Very Extremely d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r second person to be rated Extremely Very Quite S l i g h t l y S l i g h t l y Quite Very Extremely d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r t h i r d person t o be r a t e d Extremely Very Quite S l i g h t l y S l i g h t l y Quite Very Extremely d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f o u r t h person to be r a t e d Extremely Very Quite S l i g h t l y S l i g h t l y Quite Very Extremely d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f i f t h person t o be r a t e d Extremely Very Quite S l i g h t l y S l i g h t l y Quite Very Extremely d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r 81 DOMINANCE D e f i n i t i o n : s e e k s and s u s t a i n s l e a d e r s h i p r o l e s i n g r o u p s and c o n t r o l l i n g i n i n d i v i d u a l r e l a t i o n s h i p s . o r i s i n f l u e n t i a l f i r s t p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r s e c o n d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r t h i r d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f o u r t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f i f t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r 82 NURTURANCE D e f i n i t i o n : e ngages i n b e h a v i o r s w h i c h p r o v i d e m a t e r i a l o r e m o t i o n a l b e n e f i t s t o o t h e r s . f i r s t p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r s e c o n d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y " Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r t h i r d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s - i m i l a r f o u r t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f i f t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r 83 AFFILIATION D e f i n i t i o n : s e e k s and s u s t a i n s numerous p e r s o n a l f r i e n d s h i p s . f i r s t p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r s e c o n d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r t h i r d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f o u r t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f i f t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r Q u i t e V e r y E x t r e m e l y „ s i m i l a r s i m i l a r s i m i l a r EXHIBITION D e f i n i t i o n : b e h a v e s i n s u c h a way a s t o r e c e i v e t h e i m m e d i a t e a t t e n t i o n of o t h e r s . f i r s t p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r s e c o n d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r t h i r d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f o u r t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f i f t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r AUTONOMY D e f i n i t i o n : a c t s i n d e p e n d e n t l y of o t h e r s o r of s o c i a l e x p e c t a t i o n s . f i r s t p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r s e c o n d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r t h i r d p e r s o n t o be r a t e d E x t r e m e l y V e r y Qui t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f o u r t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f i f t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r AGGRESSION D e f i n i t i o n : e n g a g e s i n p u b l i c c r i t i c i s m o f o t h e r s , makes f u n of o t h e r s , and o f t e n e m o t i o n a l l y o r p h y s i c a l l y h u r t s o t h e r s . f i r s t p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r s e c o n d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r t h i r d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f o u r t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f i f t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r 87 SUBORDINATION D e f i n i t i o n : s e e k s and s u s t a i n s s u b o r d i n a t e o r i n f e r i o r r o l e s i n r e l a t i o n s h i p s w i t h o t h e r s . f i r s t p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r s e c o n d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r t h i r d p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f o u r t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r f i f t h p e r s o n t o be r a t e d E x t r e m e l y V e r y Q u i t e S l i g h t l y S l i g h t l y Q u i t e V e r y E x t r e m e l y d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r d i s s i m i l a r s i m i l a r s i m i l a r s i m i l a r s i m i l a r 88 (2) P e e r - n o m i n a t i o n f o r m s F o r t h e f o l l o w i n g s e c t i o n , s e l e c t 5 names f r o m y o u r f r a t e r n i t y who most f i t t h e c a t e g o r y i n q u e s t i o n a n d 5 names t h a t l e a s t f i t . Once y o u h a v e c h o s e n f i v e names f o r e a c h l i s t , r a n k them s o t h a t t h e f i r s t name l i s t e d i s t h e p e r s o n t h a t b e s t f i t s t h e c a t e g o r y , t h e 2nd name l i s t e d i s t h e p e r s o n t h a t r a n k s 2 n d , and so o n . Rank t h o s e 5 men who a r e most l i k e l y t o be l e a d e r s : l . 2. 3. 4 . ; 5._ Rank t h o s e 5 men who a r e l e a s t l i k e l y t o be l e a d e r s : 1 . 2. 3. 4 . 5. Rank t h o s e 5 men who a r e most h i g h l y l i k e d i n y o u r f r a t e r n i t y : 1 . 2. 3. 4. 5. Rank t h o s e 5 men who a r e l e a s t l i k e d i n y o u r f r a t e r n i t y : 1 . 2. 3. 4. 5. Rank t h o s e 5 men w i t h t h e most a c a d e m i c p o t e n t i a l i n y o u r f r a t e r n i t y : 1 . 2. _ 3. 4. 5 . Rank t h o s e 5 men w i t h t h e l e a s t a c a d e m i c p o t e n t i a l : 1 . 2. 3. 4. • 5 . 8 9 R a n k t h o s e 5 men w i t h t h e m o s t a t h l e t i c p o t e n t i a l i n y o u r f r a t e r n i t y : 1 . 2. 3. 4. 5 . R a n k t h o s e 5 m e n w i t h t h e l e a s t a t h l e t i c p o t e n t i a l : 1 . 2. 3. 4. 5 . R a n k t h o s e 5 men w h o i n t e r a c t w i t h w o m e n t h e m o s t : 1 . 2 . 3. 4. 5 . R a n k t h o s e 5 m e n w h o i n t e r a c t w i t h w o m e n t h e l e a s t : 1 . 2. 3. 4. 5 . R a n k t h o s e 5 men w h o a r e m o s t l i k e l y t o s u c c e e d i n t h e i r l i f e : 1 . 2. 3. 4. 5 . R a n k t h o s e 5 men w h o a r e l e a s t l i k e l y t o s u c c e e d i n t h e i r l i f e : 1 . 2. 3. 4. 5 . 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0095026/manifest

Comment

Related Items