UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

The development of a motivational distortion scale for use with the counterproductivity scale of the… Ng, Ee-Ling 2002

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2002-0511.pdf [ 3.17MB ]
Metadata
JSON: 831-1.0090735.json
JSON-LD: 831-1.0090735-ld.json
RDF/XML (Pretty): 831-1.0090735-rdf.xml
RDF/JSON: 831-1.0090735-rdf.json
Turtle: 831-1.0090735-turtle.txt
N-Triples: 831-1.0090735-rdf-ntriples.txt
Original Record: 831-1.0090735-source.json
Full Text
831-1.0090735-fulltext.txt
Citation
831-1.0090735.ris

Full Text

THE D E V E L O P M E N T OF A MOTIVATIONAL DISTORTION SCALE FOR USE WITH THE COUNTERPRODUCTIVITY SCALE OF THE CALIFORNIA PSYCHOLOGICAL INVENTORY by EE-LING N G B. A. (Hons.), The University of Nottingham, 1997 A THESIS SUBMITTED IN PARTIAL F U L F I L M E N T OF THE REQUIREMENTS FOR THE DEGREE OF M A S T E R OF ARTS in THE F A C U L T Y OF G R A D U A T E STUDIES (Department of Psychology, Psychometrics Programme) We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH C O L U M B I A August 2002 © Ee-Ling Ng, 2002 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of Psychology The University of British Columbia Vancouver, Canada Date XI / ^ ^ t C ^ -2-002 ii Abstract A sample of 250 undergraduates was administered the California Psychological Inventory (CPI), Form 434, on two separate occasions, under two separate conditions—(a) in which they were to respond honestly, and (b) where they were to respond as if being considered for a job they valued. Four different indices were used to assess a subject's change in responses, between the two conditions, on a CPI-based counterproductivity (CPI-Cp) scale. Using a composite of these four indices as a criterion measure of motivational distortion, a 56-item CPI-based motivational distortion scale (CPI-MD) was developed on the basis of item-total correlations with this criterion. Internal consistency and test-retest reliability estimates of the CPI-MD scale ranged between .72 and .86. Correlations with NEO PI-R domain and facet scale provided evidence of construct validity. The CPI-MD scale was used to develop an adjustment equation to correct CPI-Cp scale scores for motivational distortion. Adjusted CPI-Cp scores manifested somewhat different correlations with other personality variables than did scores on the original CPI-Cp scale, and an assessment of criterion-related validity for various job-related criteria in two separate workplace samples showed significant increases, for the adjusted scale, in validity in one sample. iii Table of Contents Abstract i i Table of Contents i i i List of Tables v Study 1 : Development of a Motivational Distortion Criterion Method . 7 Results 12 Study 2: Development of the Motivational Distortion Predictor Scale Samples 13 Method 14 Results 17 Summary of Results 22 Study 3: Development of the Motivational Distortion Predictor Scale Samples 22 Method 25 Results 28 Summary of Results 33 Overall Conclusions and Discussion 34 References 36 Appendix A Analyses to Determine the Effectiveness of the Experimental Procedure in Study 1 53 Appendix B Complete Administration Instructions for the Two Experimental Conditions in Study 1 57 Appendix C Additional Information about the Phi Correlation and Variance of the Differences Measures of Change (Based on Lautenschlager, 1986) 58 Appendix D Estimation of Reliability for the Four Change Measures and the Composite Motivational Distortion Criterion 60 Appendix E Correlations between the Motivational Distortion Criterion and the 20 Standard CPI Scales 62 Appendix F Two Alternative Methods of Deriving Adjustment Equations to Correct for Motivational Distortion in CPI-Cp Scores 63 Appendix G Correlations of Adjusted and Unadjusted CPI-Cp Scale Scores with Scores on the NEO PI-R Domain and Facet Scales 66 iv Appendix H Criterion-Related Validity Analyses for Adjusted CPI-Cp Scores with Adjustments Made on the Basis of the CPI Fake-Good Scale 68 List of Tables V Table 1 Means, Standard Deviations, and Intercorrelation Matrix of the Four Measures of Change 42 Table 2 Factor Pattern and Factor Score Coefficient Vectors Associated with the First Principal Component 43 Table 3 The 56 CPI Items Selected for the CPI-MD Scale 44 Table 4 Normative Data for the CPI-MD Scale 46 Table 5 Internal-Consistency Reliability Estimates and Standard Errors of Measurement for the CPI-MD Scale in Three Separate-Gender Samples 47 Table 6 Correlations of Scores on the CPI-MD Scale with Scores on the Five NEO PI-R Domain Scales and the NEO PI-R Neuroticism, Extraversion, and Conscientiousness Facet Scales 48 Table 7 Descriptive Statistics and Correlation of the CPI-MD Scale and M D Criterion Scores 1 49 Table 8 Comparison of Means, Standard Deviations, and Correlations between Honest-Condition/Unadjusted and Adjusted CPI-Cp Scores in Three Separate-Gender Samples 1 50 Table 9 Correlations of Adjusted and Unadjusted CPI-Cp Scale Scores with NEO PI-R Domain and Conscientiousness Facet Scale Scores 51 Table 10 Correlations between the CPI-Cp Scale Scores (With and Without Adjustments for Motivational-Distortion) and Counterproductive-Behavior Criteria in Two Non-Management Level Samples 52 Table A l Means and Standard Deviations of the CPI-Cp Scores in the Employment and Honest Conditions of the Experimental Sample 55 Table A2 Descriptive Statistics of the CPI-Cp Scores in the Experimental Sample and a Normative Undergraduate Student Sample 56 Table F l Means and Standard Deviations of the Employment-Condition CPI-Cp and CPI-M D Scale Scores 65 Table HI Correlations between the CPI-Cp Scale Scores (With and Without Two Different Adjustments for Motivational-Distortion) and Counterproductive Behavior Criteria in Two Non-Management Level Samples 70 1 Introduction With growing evidence that certain relevant personality constructs have substantial predictive validity for various job-related criteria, there has been an increasing interest in the use of personality inventories for personnel selection purposes (Barrick & Mount, 1991; Hough, Eaton, Dunnette, Kamp, & McCloy, 1990; Hough & Schneider, 1996; Ones, Viswesvaran & Schmidt, 1993). However, this increased use has also been met by concerns about the susceptibility of personality measures to motivational distortion. Motivational distortion (MD) can generally be defined as an altering of one's responses on a test because of the circumstances under which he/she is taking it. In a job application context, motivational distortion can be considered faking-good or presenting an artificially favorable impression of oneself in order to increase one's chances of getting the job. The purpose of this study was to develop a measure of motivational distortion for the California Psychological Inventory (CPI), which could be used as a basis for adjusting personality scale scores in order to obtain more honest score estimates for hiring purposes. The relevance of this research is predicated on the negative impact of M D on personality testing for personnel selection. Hence, a summary of the research evidence pertaining to the occurrence and impact of M D in personnel selection will be presented. This is followed by a brief review on how the validity scales included within existing personality inventories can be used to adjust content scale scores for M D . Finally, as this study focuses on making M D corrections for a CPI-based measure of counterproductivity (CPI-Cp scale; Hakstian, Farrell, & Tweed, 2002), a brief description of the CPI-Cp scale will be provided. Does Motivational Distortion Exist? Research findings have clearly established that subjects can distort their responses on non-cognitive measures in the desired direction, when instructed to do so (e.g., Alliger & Dwight, 2000; Douglas, McDaniel, & Snell, 1996; Hough et al., 1990; McFarland, 2000; 2 McFarland & Ryan, 2000; Ones & Viswesvaran, 1998a; Rosse, Stecher, Miller, & Levin, 1998). Viswesvaran and Ones (1999) performed a meta-analysis to quantify the magnitude of distortion and found that on both personality and integrity tests, subjects who had been instructed to fake-good increased their scores by about half a standard deviation on average. In general, the main criticism that has been leveled against research studies on motivational distortion in personnel testing is that while such studies have demonstrated that applicants can "fake" if they choose to do so, they do not provide evidence that applicants do "fake" in reality. However, a survey of the research literature yielded several recent studies that provide evidence of distortion in real applicant subjects (e.g., Barrick & Mount, 1996; Becker & Colquitt, 1992; Ellingson, Sackett, & Hough, 1999; Rosse et al., 1998). The applicant-based motivational distortion studies cited here involved either personality test measures or biodata inventories. It appears that more research using applicant subjects is needed on integrity tests, as existing research studies have tended to be laboratory-based studies in which employment conditions were experimentally simulated (e.g., Cunningham, Wong, & Barbee, 1994; Lobello & Sims, 1990; Ryan & Sackett, 1987). The Impact of Motivational Distortion on Validity in Personnel Selection Research studies on the impact of motivational distortion in a personnel selection context have led to conflicting conclusions. In general, studies support the fact that distortion results in an over-representation of "fakers" in the top end of the score distribution, who are therefore more likely to be hired in a top-down selection process (Christiansen, Goffin, Johnston, & Rothstein, 1994; Hough, 1998; Rosse et al., 1998). However, while M D can affect selection decisions, the preponderance of research evidence seems to indicate that it does not degrade criterion-related validity (Christiansen et al., 1994; Hough et al., 1990; Ones & Viswesvaran, 1998a, 1998b; Ones, Viswesvaran, & Schmidt, 1993; Ones, Viswesvaran, & Reiss, 1996; Zickar, Ross, & Levin, 1996). In contrast, only a few individual studies have provided evidence which 3 empirically supports the argument that faking degrades criterion-related validities, and these were laboratory studies where faking was computer-simulated or experimentally induced (e.g., Douglas, McDaniel, & Snell, 1996; McDaniel, Douglas, & Snell, 1997; Zickar, 1997). There have been attempts to provide some explanation for the discrepant findings, within the research literature, regarding the impact of M D on criterion-related validity. These explanations include (a) differences in the extent to which faking can be assumed to occur across the different samples utilized in various studies—e.g., student samples, job incumbents, and job applicants—(Hough, 1998); (b) different methods of operationalizing and conceptualizing faking (e.g., Kluger & Collela, 1993; McFarland, 2000; McFarland & Ryan, 2000; Paulhus, 1991); and (c) methodological limitations of using correlational analyses to represent the effects of faking (e.g., Alliger & Dwight, 2000; Rosse et al., 1998). Research findings on the effects of motivational distortion on the construct validity of non-cognitive test measures have also been equivocal. On the one hand, studies have found significant correlations between these test measures and social desirability scores (Cunningham, Wong, & Barbee, 1994; Guastello & Rieke, 1991; Ones, Viswesvaran, & Reiss, 1996; Rosse et al., 1998), and factor analytic studies have provided evidence that motivational distortion damages the structural integrity of personality measures in applicant samples (Barrick & Mount, 1996; Cellar, Miller, Doverspike, & Klawsky,1996; Douglas, McDaniel, & Snell, 1996; Livneh & Livneh, 1989; Schmidt & Ryan, 1993; Zickar & Robie, 1999). On the other hand, other studies have demonstrated factorial invariance in faking samples (Collins & Glese, 1998; Ellingson, Smith, & Sackett, 2001; Smith, Hanges, & Dickson, 2001; Ones & Viswesvaran, 1998a). Overall, it would appear that Ones, Viswesvaran, and Schmidt's (1993) dismissal of social desirability as a "red herring" is premature. Despite existing evidence of the contrary, there is also sufficient evidence to suggest that faking can have a negative impact on selection 4 decisions, and criterion-related and construct validity, and therefore remains a legitimate concern that needs to be addressed with further research. Correcting Personality Scale Scores for Motivational Distortion Because o f concerns about motivational distortion as a contaminant, many established personality inventories such as the M M P I (Dahlstrom, Welsh, & Dahlstrom, 1972, 1975), Sixteen Personality Factor Questionnaire (Winder, O'Dell, & Karson, 1975), Personality Research Form (Jackson, 1967), Hogan's Personality Inventory (Hogan, 1986; Hogan & Hogan, 1972), and California Psychological Inventory (CPI; Gough, 1957, 1987; Gough & Bradley, 1996), include validity scales to detect various aspects o f motivational distortion. These validity or M D scales are generally either special-purpose scales or are linear combinations o f personality content scales. One way in which these M D scales have been used to address motivational distortion in personality testing is to use these scale scores as a basis for correcting personality content scale scores for the effects o f distortion. This correction is effected by regressing the personality scale scores on the M D scale score, with the residual representing the personality scale score adjusted for distortion plus error. As is typical with much o f the research on faking, there is also disagreement regarding the use o f score corrections to address the influence o f M D . The main argument against the use o f score corrections stem from meta-analytic evidence demonstrating that M D (broadly conceptualized as social desirability) does not degrade criterion-related validity. Others have suggested that social desirability can be job-related and therefore score corrections could inadvertently partial out substantive or content-related variance (McCrae & Costa, 1983; Nicholson & Hogan, 1990; Zerbe & Paulhus, 1987). Moreover, studies have shown that corrected scores do not approximate honest scores (e.g., Ellingson, Sackett, & Hough, 1999; Weiner & Gibson, 2000). 5 Despite the work cited above, some studies have demonstrated that using corrected personality scale scores resulted in changes to the rank ordering of candidates which would consequently affect top-down hiring decisions (Christiansen et al., 1994; Hough, 1998; Rosse et al., 1998). Other researchers who support the use of corrected scores have put forward the reasoning that M D is a confounding variable, and, in the interest of construct validity, controlling the variance of M D in personality scale scores would lead to a more precise and valid measurement of the intended construct of interest (e.g., Helmes, 2000). The California Psychological Inventory (CPI) and its Validity Scales The California Psychological has been widely used and researched in applied contexts, including that of personnel selection, and has an established ability to predict externally-measured job-related or organizational criteria. The current version of the CPI consists of 434 items measuring 20 standard scales and 13 special-purposes scales. Of the 20 standard scales, three scales were originally constructed to measure motivational distortion and detect invalid protocols. The Wb (Well-being) scale assessed tendencies to fake-bad; the Gi (Good Impression) scale assessed tendencies to fake-good, and the Cm (Communality) scale assessed random responding. It was found that by optimally combining these three validity scales, along with other standard CPI scales, prediction equations could be constructed to provide a more precise classification of invalid protocols (Lanning, 1989). In the current edition of the CPI manual (Gough & Bradley, 1996), the fake-good protocol validity equation for CPI Form 434 is a linear combination of five separate CPI standard scales: Dominance, Empathy, Good Impression, Well-being, and Flexibility. The CPI-Counterproductivity (CPI-Cp) Scale Hakstian et al. (2002) developed a CPI-Counterproductivity scale (CPI-Cp) designed to measure tendencies to engage in a broad range of counterproductive behaviors, that is, behaviors 6 that were detrimental to the objectives of the organization such as property theft, disruptiveness, absenteeism, tardiness, and withholding effort. On the basis of four student samples and three workplace samples, the CPI-Cp scale was shown to possess high reliability and validity. The decision to focus on developing motivational-distortion-adjustments for the CPI-Cp scale in this study was based on the fact that this scale was likely to be the most susceptible to motivational distortion in a personnel selection context. Purposes of the Present Research The twofold purpose of the present research was (a) to develop a measure of motivational distortion tendencies, by means of items from the CPI, (referred to, in what follows, as the CPI-MD scale), and (b) to use this scale as a basis for developing a means of correcting CPI-Cp scale scores for motivational distortion. Unlike the existing CPI fake-good scale described in the CPI manual (which will be referred to as CPI-FG in what follows) that measures motivational distortion tendencies on the basis of five standard CPI scales, the proposed CPI-MD scale avoids introducing extraneous variance by measuring M D on the basis of only those CPI items that are empirically related to motivational distortion. In this manner, such a scale is expected to be an improvement upon the existing fake-good scale, as it would offer a more effective and precise measurement of a respondent's tendency to distort his/her responses on the CPI when the inventory is taken under personnel selection conditions. The research was conducted in three separate but related studies: (a) development of the motivational distortion criterion (Study 1); (b) development of the motivational distortion predictor scale (Study 2); and (c) development of the adjustment equation for CPI-Cp scores and an evaluation of the psychometric impact of using adjusted scores on the CPI-Cp scale (Study 3). 7 Study 1: Development of a Motivational Distortion Criterion Research Aims The aim of this study was to develop a reliable measure of motivational distortion from four separate measures of change that had previously been used to index motivational distortion (Gordon & Gross, 1987; Lautenschlager, 1986). The purpose of this measure was for use as a criterion for the development of the motivational distortion predictor scale in the next study. Method Sample A total of 250 subjects, consisting of 55 male and 195 female undergraduate students at the University of British Columbia (UBC), took the CPI Form 434 on two separate occasions (one in an employment condition and one in an honest condition), with a time interval of about 5-7 days between administrations. Subjects were administered the CPI in small-group settings of about 3-5, in a testing room or classroom at the university. These student subjects participated in the experiment for course credit, and were also provided with feedback on their personality profile. Administration Procedures A repeated measures experimental design was used in which all subjects were administered the CPI under the employment condition first, followed by the honest condition, a week later. Although previous research had indicated an order effect with repeated measures designs (e.g., Ellingson, Sackett, & Hough, 1999; Hough et al., 1990), standardizing the experimental order for all subjects was preferred because it allowed the employment condition to more closely simulate the usual applicant setting where applicants would not have had prior exposure to the personality inventory. The concerns with regard to the contamination of honest-condition data as a result of a practice effect were addressed in some separate additional 8 analyses. The results suggested that this was not the case, as the honest condition scores were representative of normative data for undergraduate students (see Appendix A). The significant part of the honest- and employment-condition instructions that were read out to subjects is as follows (the complete set of instructions can be found in Appendix B): Honest: Please respond to the statements as you honestly see yourself to be. Employment: Imagine that you are applying for a job that you would like to have. You have been requested to complete this personality inventory as part of the selection process. Your responses on the inventory will provide valuable information about yourself that will be used as part of the decision-making process. Please respond to the items as you would i f you were applying for this job. To prevent subjects from lapsing into honest responding during the employment condition, subjects were reminded of the instructions (i.e., to complete the inventory as though they were applying for a job), and visual reminders of the instructions (in an abridged form) were also placed on every page of the inventory. Honest responding in the normal condition was encouraged both with guarantees of confidentiality and with an incentive that subjects would be provided feedback on their personality profile based on their responses in this condition. Previous studies on motivational distortion have tended to instruct participants to deliberately create a favorable impression of themselves when completing the test measures. While such instructions successfully induce faking, they poorly simulate real-applicant settings because the natural deterrents to motivational distortion (e.g., fear of detection) that occur in such settings are eliminated and real individual differences in motivational distortion are suppressed. As a result, observable motivational-distortion variance is reduced and the ability of such empirical studies to detect the effects of distortion is likewise minimized (see Douglas, McDaniel, & Snell, 1999; McFarland, 2000; Zickar, 1997). In contrast, the instructions 9 suggested here for the employment condition were believed to be a closer simulation of a personnel selection context and would consequently be more conducive to capturing the distribution of motivational distortion scores that can be assumed to exist in an employment setting. However, concerns with regard to the potency of the present employment-condition instructions in inducing motivational distortion were addressed in an adjunct analysis (see Appendix A). Operationalization of the Motivational Distortion Construct Motivational distortion was operationalized as a within-subject measure of change between a condition that provided motivation for distortion (the employment condition) and one that did not (the honest condition). As a within-subject measure, each subject acted as his/her own control and therefore any systematic changes in responses between one condition and the next, could be construed as intentional distortion. Four separate measures of change were used to assess the degree to which subjects' responses to the CPI -Cp scale varied from one condition to the other. Taken together, these measures were able to assess both overall and item-level shifts in responses to the CPI -Cp scale (Gordon & Gross, 1978; Lautenschlager, 1986). For measures 1, 2 and 4 below, larger values represent more distortion, whereas for measure 3, a larger value indicated less distortion. As measures 3 and 4 are conceptually more complicated, additional information is provided in Appendix C for further explanations on the computation of these two measures. A brief description of the four measures is provided here: (1) Raw-score difference: The difference in the CPI -Cp scale scores obtained in the employment condition (CPI-CpemPioyment) and the honest condition (CPI-Cphonest)- The difference was obtained by: CPI-Cphonest scores - CPI-CpemPioyment scores. (2) Residual change scores: The residual about the regression line from regressing the C P I -Cphonest on CPI-Cpempioyment scale. That is, the residual change score was obtained by: C P I -10 Cphonest - CPI-Cphonest, where CPI-Cphonest is the predicted CPI-Cphonest score from the C P I -Cpemployment SCOre. (3) Phi correlation: The within-subject correlation coefficient which represented, for each subject, the consistency of true-false responses on the 80 CPI-Cp items between the employment and honest conditions. (4) Variance of the differences : The within-subject variance of the differences in the responses to each of the 80 CPI -Cp items between the two experimental conditions. (The data were coded '1 ' for true and '2' for false.) As noted, more detail on measures 3 and 4 appear in Appendix C. Measurement and Analysis of the Motivational Distortion Criterion Gender differences were tested for all four measures of change and significant gender mean differences were found on the Raw-score Difference measure with females obtaining higher mean scores than males, <248) = 233, p < .025, A = .352, .95 C.I. for A = (.054, .651). For the other three measures, gender differences were not significant (p > .25). Therefore, scores on the Raw-score Difference measure were mean-deviated within gender before being pooled for the correlational analyses to follow. The separate-gender descriptive statistics and the pooled-gender intercorrelation of these four measures appear in Table 1. Insert Table 1 about here The four measures were highly and uniformly correlated and this suggested a common underlying construct. To maximize reliability, a composite of the four measures was used as the final motivational distortion criterion (MD criterion) and the weighting of variables for this composite was determined on the basis of first principal component scores. The weight vector for obtaining the first principal component was derived from the principal component analysis of the 4 x 4 intercorrelation matrix found in Table 1. As noted 11 earlier, this intercorrelation matrix was based on scores that had been mean-deviated, within gender, on the Raw-score Difference measure. However, to retain existing gender differences in the resulting criterion scores, the weight vector obtained from the principal component analysis was applied to the non-mean-deviated z-scores of all four measures and then standardized to the metric of the Raw-score Difference measure, to yield the appropriate scores for adjustment later. Reliability Estimates of the Four Measures of Change and the MD Criterion Reliability estimates were obtained for each of the four change measures. The reliability estimates for both the Raw-score Difference and Residual Change Scores were computed using equations developed by Williams, Zimmerman and Magazzati (1987). Deriving reliability estimates for the Phi Correlation and Variance of the Differences measures presented some complications because these were single-value, computationally-derived measures. It was decided that by splitting the 80-item CPI-Cp scale into two equal subscales comprising the first 40 items and the last 40 items, two separate estimates (one for each subscale) of both the Phi Correlation and Variance of the Differences measures could be obtained for each subject. The correlation between these two estimates across all subjects would therefore represent a form of split-half reliability. Finally, using standard equations for the reliability of a differentially-weighted composite (see, e.g., Horst, 1966, pp.280-281), a reliability estimate for the M D criterion (the principal component scores) was also obtained. Further elaboration on the computation of these reliability estimates appear in Appendix D. Additional Analysis of the Motivational Distortion Criterion Measure The motivational distortion criterion measure was correlated with the 20 standard CPI scales. This analysis was undertaken to determine, as far as possible, the conceptual meaning of the M D criterion measure. Because it is not central to the purposes of the present study, results of this analysis appear in Appendix E. 12 Results As mentioned earlier, the high level of intercorrelation amongst the four measures of change indicated one single underlying factor (see Table 1). Principal component analysis of the 4 x 4 R matrix revealed one eigenvalue greater than 1.0 and this factor accounted for 89% of the variance. The factor pattern vector and factor score coefficient vector (the weight vector for obtaining the standardized first principal component) appear in Table 2. Insert Table 2 about here From the reliability analyses, it was found that the average reliability estimate of the four separate measures of change was .76. (The specific' reliability estimates for each of the four measures appear along the principal diagonal of the intercorrelation matrix in Table 1.) The overall internal consistency reliability estimate of the M D criterion was .93. Overall, the results show that the four measures of change were individually reliable and highly correlated, and that a composite consisting of the first principal component of the four measures would provide a more reliable measure of motivational distortion. This then constituted the final M D criterion. Study 2: Development of the Motivational Distortion Predictor Scale Research Aims In the previous study, a motivational distortion measure was developed under experimental conditions in which both distorted and honest responses were obtained from the same subject. Consequently, this measure of motivational distortion cannot be used in applied settings because it is dependent on the availability of both distorted and honest data from the same individual. In a hiring context, only one set of responses would be available and these responses would likely have been contaminated by motivational distortion. Therefore, the aim of 13 this study was to develop a CPI-based motivational distortion predictor scale (CPI-MD scale) that could be used to estimate the amount of distortion within a set of CPI scores obtained under employment conditions. Samples A total of six distinct samples were employed in this study. Besides the UBC Sample from Study 1 (named U B C Sample 4 in what follows), five additional samples from the Hakstian et al. (2002) study were used. The total number of subjects for each of these additional samples is presented below, along with the analyses for which the sample was used in this study. The exact ns for each of the analyses to follow are provided later with the results. 1. UBC Sample 1—A total of 1,083 subjects consisting of 350 male and 733 female U B C undergraduate students who were administered the CPI during the years 1988-1995. This sample was used to develop norms, obtain internal-consistency reliability estimates, and assess the construct validity of the CPI-MD scale. 2. UBC Sample 2—A total of 229 subjects, consisting of 71 male and 158 female U B C undergraduate students who were administered the CPI during the period 1998-1999. These subjects completed the CPI at two separate occasions with a time interval of 2-3 weeks. This sample was used to derive test-retest reliability estimates of the predictor scale (CPI-MD scale) and the CPI-FG scale. 3. UBC Sample 3—A total of 320 subjects consisting of 98 male and 222 female U B C undergraduate students. Subjects in this sample were administered both the CPI and NEO PI-R (Costa & McCrae, 1992) during the period 1998-2000. The construct validity of the CPI-MD scale was assessed with this sample, using the Five Factor Model framework represented by the NEO PI-R. 4. UBC Sample 4—A total of 250 subjects consisting of 55 male and 195 female UBC undergraduate students. This sample was first used in Study 1 for the development of the M D 14 criterion measure, and in this study, only the employment condition data were used for the development of the CPI-MD scale. Various other correlational analyses, conducted to compare the CPI-MD scale with the CPI-FG scale, were also based on this sample. (The sample characteristics can be found in the methods section of Study 1.) In the above four U B C samples, the age range was narrow, with a median age of around 20. Subjects were of mixed race, with slightly more than half Caucasian and the rest mainly Asian and Indo-Canadian. These student subjects were all administered the CPI in small group settings of about 3 or 4 to 45-50, in a classroom or testing room at UBC. 5. Management Employee Sample—A total of 457 subjects consisting of 245 male and 212 female management employees of two large Canadian telecommunications companies. Subjects ranged in ages of between 22 and 61 with a median of around 40 for males and 37 for females. They were mainly first- and second-level management personnel (from all company departments) and were administered the CPI, between the mid- to late-1990's, as part of more comprehensive, on-going assessment-center research studies. This sample was used to provide norms and to estimate internal consistency reliability for the CPI-MD scale. 6. Non-Management Job-Applicant Sample—A total of 528 subjects consisting of 193 male and 335 female applicants for a variety of non-management positions in a call center in Vancouver, British Columbia. These subjects were administered the CPI, between 1995-1999, as part of a longer test battery that included cognitive-ability tests and other assessment methods. This sample was used to provide norms and to estimate internal consistency reliability for the CPI-MD scale. Method Selection of Items Using what can be considered a purely empirical methodology, the CPI-MD scale items were selected from the CPI (Form 434) on the basis of their correlations with the M D criterion 15 measure developed in Study 1. For this item-selection process, the employment-condition CPI data were correlated with the M D criterion using UBC Sample 4. As there were no significant gender differences on the M D criterion scores (p > . 10), the correlational analyses were performed with a pooled-gender data sample. Given the potential for inflation due to capitalization on chance in these analyses, only those items that correlated with the M D criterion scores at or beyond the .01 level of significance (nondirectional) were considered. This standard guaranteed that, under ideal conditions, not more than 4 or 5 items would be selected because of Type I error. By this process, 56 items were selected for the CPI-MD scale; these are presented in Table 3. The scoring procedure is to assign one point for each item under the "True" heading responded to with a "true" response and one point for each "False" item responded to with a "false". Thus, CPI-MD scores can potentially range from 0 to 56 with higher scores reflecting greater tendencies for motivational distortion. Insert Table 3 about here The distribution of CPI-MD scale scores was examined under the two conditions— employment and honest—for the experimental sample (UBC Sample 4). In the honest condition, the distribution was unimodal and very close to symmetric (summary statistics for the CPI-MD scale under the honest condition appear in Appendix F, Table F l ) . The distribution of the CPI-M D scale scores under the employment condition, however, was unimodal but slightly negatively skewed (sample estimate of the index of skewness was -.20). Establishment of Norms for the CPI-MD Scale and Analysis of Gender Differences Three different kinds of normative data for the CPI-MD scale were derived in the present study: U B C Sample 1 provided the normative distribution for an undergraduate student population, whilst the management employee and non-management job-applicant sample 16 (Samples 5 and 6 above, respectively) provided two separate sets of industrial norms. In all three samples, separate-gender norms were provided and analyses of gender mean differences on the CPI-MD scale were performed. These were statistically powerful analyses due to the large sample sizes of the gender-groups. Estimation of Internal-Consistency and Test-Retest Reliability The three separate-gender normative data samples above were also used to estimate the internal-consistency reliability (Cronbach's alpha) of the CPI-MD scale. In addition, using subjects from U B C Sample 2, separate-gender estimates of the short-term stability of the CPI-M D scale were also obtained using test-retest correlations between scale scores obtained at two time points (with a 2-3 week interval between administrations). Estimation of Construct Validity The Five Factor Model was used as a framework for examining the psychological underpinnings of the motivational distortion construct measured by the CPI-MD scale. Using U B C Sample 3, separate-gender correlations were obtained between CPI-MD scores and the five NEO PI-R domain scales (reflecting the Big Five personality factors), that is, Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness. In addition, where these correlations were found to be statistically significant, correlations between CPI-MD scores and the facet scales associated with those particular domain scales were also obtained. Correlation between the CPI-MD and CPI-Cp scales To provide an estimate of the extent to which the CPI-Cp scale scores were associated with CPI-MD scores, these were correlated, using the employment-condition data from U B C Sample 4. Because of significant gender differences on these two scales (p < .01 for both), scale scores were mean-deviated by gender before being pooled for the correlational analysis. 17 Estimation of Criterion-Related Validity As an estimate of the criterion-related validity of the CPI-MD scale, scores on this 56-item scale were correlated with the motivational distortion criterion measure (MD criterion) developed in Study 1, using data from U B C Sample 4. As noted earlier, because of significant gender differences on the CPI-MD scale, scores on this scale were mean-deviated by gender before correlation with the M D criterion. (No gender differences were found on the M D criterion as reported earlier.) Given the purely empirical item-selection process, the obtained criterion-related validity estimate would be inflated to some extent because of capitalization on chance. Comparisons between the CPI-MD Scale and the CPI Manual's Fake-Good Scale The current motivational distortion predictor scale was intended to be an improvement on the existing CPI Fake-good validity scale (CPI-FG) from the CPI manual (Gough and Bradley, 1996) by offering a more precise measurement of the motivational distortion construct. To evaluate the relative effectiveness of the CPI-MD and CPI-FG scale as measures of motivational distortion, a few additional analyses were conducted to compare the psychometric properties of these two scales. Using the employment-condition data from U B C Sample 4, correlations of the CPI-FG scale with the M D criterion and the CPI-MD scale were obtained. Because of significant gender differences on both the CPI-MD and CPI-FG scale (p < .01 for both), the correlations were obtained using scores that had first been mean-deviated within the separate-gender samples (to scale out gender mean differences). Using U B C Sample 2, stability estimates of the CPI-FG scale were also obtained to compare its test-retest reliability with that of the CPI-MD scale. Results Norms for the CPI-MD Scale and Analyses of Gender Differences Table 4 presents the means and standard deviations of the CPI-MD scale for three different groups of North American subjects, separately by gender. These norms provide a useful 18 source of reference for future use, although the development of local norms is strongly advised where possible. Insert Table 4 about here From Table 4, it can be seen that in samples where the test scores had the greatest consequences, the CPI-MD mean scores were correspondingly larger: the undergraduate student norm group showed the lowest mean scores, and the non-management job-applicants norm group showed the highest mean scores. The analyses of gender differences for the three norm groups revealed no significant differences in all three samples (p > .25). Evidence of Reliability for the CPI-MD Scale The internal consistency reliability estimates (Cronbach's alpha) for the separate-gender samples of undergraduate students, management employees, and non-management job-applicants, appear in Table 5. Insert Table 5 about here It is encouraging to note that among the three samples, the non-management job-applicants sample yielded the highest internal consistency reliability estimates, as this was the population for which the CPI-MD scale was targeted for applied use. The comparatively higher internal consistency reliability estimates obtained for this sample could be a result of the fact that the CPI-MD scale was specifically developed to capture the construct of motivational distortion as it occurred within a hiring context. Also, of course, as can be seen from Table 4, this sample demonstrated the greatest variability on the CPI-MD scale. The short-term stability estimates (test-retest correlations with 2-3 week interval) of the CPI-MD scale, computed separately by gender, were: males, .803 (n = 71); females, .856 (n = 19 158). These separate-gender stability coefficients were not significantly different (p> .10), and were, therefore, pooled to yield a single estimate of stability of .841 (n = 229). Evidence of Construct Validity of the CPI-MD Scale Scores on the CPI-MD scale were correlated with the five domain scales of the NEO PI-R, separately by gender. In both gender groups, scores on three domain scales were significantly correlated with CPI-MD scale scores (p < .001, non-directional)—Neuroticism, Extraversion, and Conscientiousness—and the CPI-MD scale was correlated with the facet scales of these three domains. The correlations appear in Table 6. Insert Table 6 about here The pattern of correlations of the CPI-MD scale with the five NEO PI-R domain scales was similar across both genders: Extraversion showed the highest correlation followed by Conscientiousness and Neuroticism (ordered by magnitude of correlation coefficient obtained). The correlations between the CPI-MD scale with the facet scales of Neuroticism, Extraversion, and Conscientiousness also revealed a relatively similar profile between the genders. For the Extraversion domain, Assertiveness emerged as the dominant component in both genders (disattenuated r = .84). For the Conscientiousness domain, the key facets were Achievement Striving and Self-Discipline for males, and Achievement Striving and Competence for females. Lastly, for the Neuroticism domain, the two highest correlations (both negatively correlated with CPI-MD) were Self-Consciousness and Vulnerability in both genders, although the level of correlations was higher for females. Overall, the pattern of correlations suggest that for both males and females, motivational distortion in an employment context is associated with higher levels of Assertiveness and Achievement-Striving, and lower levels of Self-consciousness and Vulnerability. 20 There are two possible explanations for the observed pattern of correlations. One interpretation is that motivational distortion within an employment context is related to real individual differences on these personality dimensions, or alternatively, that measurement of these personality dimensions is particularly susceptible to distortion in an employment context. The present experimental design does not allow for a more definitive interpretation to be made. The reader is referred to other recent empirical studies (e.g., McFarland & Ryan, 2000; Ones, Viswesvaran, & Reiss, 1996; Rosse et al., 1998) that may cast some light on this issue, even though their findings are somewhat disparate. The above results differ from the meta-analytic research findings in Ones, Viswesvaran and Reiss's (1996) study, which showed that, among the Big Five personality factors (using self-report measures), social desirability scales correlated the highest with Emotional Stability (the inverse construct to Neuroticism), followed by Conscientiousness and Agreeableness. In contrast, as reported above, the results from this study suggest that motivational distortion is associated most highly with Extraversion, followed by Conscientiousness, and Emotional Stability, in that order. One explanation for the differences in results obtained is that whereas the Ones, Viswesvaran, and Reiss (1996) study used a broad conceptualization of the social desirability construct, the present study specifically focused on the motivational distortion that occurs within an employment context. If this is true, the difference in the correlational results obtained between the two studies might be an indication of a need to refine the concept of social desirability in future research on motivational distortion within employment settings. Indeed, some researchers have pointed out that, within employment settings, job desirability might be a more relevant construct than social desirability (e.g., Kluger & Collela, 1993). Correlation between the CPI-MD and CPI-Cp Scale The pooled-gender correlation between CPI-MD and CPI-Cp scale scores (both from employment-condition data) was -.587 (n = 250, p < .0001). Although the obtained correlation 2 1 was inflated as a result of overlapping item content between the two scales (14 common items in total), it provides an indication that CPI-Cp scores (employment-condition) contain variance related to motivational distortion. (Note: The correlation between the CPI-Cp scores and the M D criterion was found to be -.204, p < .001.) Estimation of Criterion-Related Validity The mean correlation (over the two gender samples) between the CPI-MD scale and the M D criterion was .543 (n = 250). This correlation is clearly inflated because of capitalization on chance (the CPI-MD scale items were selected on the basis of item-criterion correlations with the M D criterion). Because of sample size (n = 250), it was not practical to attempt to establish empirical cross-validation in this study. Comparisons between the CPI-MD Scale and the CPI Manual's Fake-Good Scale The pooled-gender correlation between the CPI-MD scale and the CPI-FG scale was .843 (n = 250). The high correlation is to be expected, considering the fact that both scales are tapping into the same global construct of motivational distortion. The stability estimates of the CPI-FG scale were: for men, r=638 (n = 71); for women, r=.818 = 158). These estimates were both significantly lower than those of the CPI-MD scale (p < .05). The criterion-related validity of the CPI-FG scale (for the M D criterion) was .365 (over both gender samples), and although the value is statistically significant (p < .0005, directional), it is significantly lower (p < .001) than the corresponding correlation for the CPI-MD scale (.543). Thus, even though some reduction in the criterion-related validity of the CPI-MD scale (for the M D criterion) can be expected on cross-validation, the present results seem to suggest that the CPI-MD scale offers a relatively more reliable and valid measure of motivational distortion than does the CPI-FG scale. 22 Summary of Results A 56-item CPI-based measure of motivational distortion (the CPI-MD scale) was developed on the basis of item-criterion correlations with the motivational distortion criterion measure developed in Study 1. Separate-gender norms for the CPI-MD scale were provided for three different populations—undergraduate students, management employees, and non-management job applicants. Internal consistency and stability reliability estimates of the scale ranged between .72 and .86 across these same three populations. Employing the Five Factor Model, as measured by the NEO PI-R, as a framework for construct validation, employment-centered motivational distortion was found to be associated with Extraversion, Conscientiousness, and Emotional Stability. A more fine-grained approach using the NEO PI-R facet scales further revealed that the dominant components associated with motivational distortion were Assertiveness, Achievement-Striving, low Self-consciousness, and Invulnerability. Finally, when compared with the CPI Manual's Fake-good scale, the present CPI-MD scale demonstrated relatively better reliability and criterion-related validity. Study 3: Development of an Adjustment Equation to Correct CPI-Cp Scale Scores for Motivational Distortion Research Aims Using the previously developed CPI-MD scale as a predictor measure of motivational distortion, this study aimed to develop a correction equation for adjusting CPI-Cp scale scores for the effects of motivational distortion so as to derive more honest score estimates for use in personnel decision-making. Samples A total of six samples were used in this study. With the exception of one sample (Sample 6 below), all of the other five samples in this study were previously used and described 23 in Study 2. The six data samples employed in this study are listed below, along with the sample sizes and analyses for which the sample was used. Additional information is provided here for Samples 5 and 6 as this information was not previously given. 1. UBC Sample 4—250 subjects (55 males, 195 females)—This sample was used in the development of the adjustment equations, and for some additional correlational analyses that were conducted to assess the effectiveness of the developed score adjustments in approximating honest scores. 2. UBC Sample 2—229 subjects (71 males, 158 females)—This sample was used to obtain stability estimates of the adjusted CPI-Cp scale scores. 3. UBC Sample 3— 320 subjects (98 males, 222 females)—This sample was used to assess the construct validity of adjusted CPI-Cp scale scores. 4. Management Employee Sample—457 subjects (245 males, 212 females)—This sample was used to provide norms of the adjusted CPI-Cp scale scores. 5. Non-Management Job Applicant Sample—528 subjects (193 males, 335 females)—The full sample was used to provide norms of the adjusted CPI-Cp scale scores, and a subset of this sample was used for assessing the criterion-related validity of adjusted CPI-Cp scores. This subsample consisted of 79 applicants for a telephone service representative's job in a large Canadian telecommunications company, with a ratio of about two females to every male job applicant. These subjects were assessed with an experimental assessment battery (including the CPI) after a positive hiring decision had been made (on the basis of an earlier selection procedure), but before commencement on the job. A performance-appraisal inventory was completed on them by their supervisors about 3-4 months into their jobs, and performance ratings on this inventory was used as the job-criterion measure in the criterion-related validity analyses. 24 6. Non-Management Employee Sample—This sample comprised a total of 91 subjects consisting of 80 male and 11 female employees of an armored-car company in Vancouver, British Columbia. The vast majority of the subjects were Caucasians, with an age range of early 20's to mid-50's, and a median age of 32. During the early-to mid-1990's, these subjects were administered the CPI (Form 462) prior to being hired although these data were not available for decision-making at the time of hiring. In 1996, their personnel files were accessed as a source of criterion data for the Hakstian et al. (2002) study. At that time, the subjects had been employed for anywhere from a few months to nearly five years, with a mean tenure of about two years. This sample, though small, was valuable as it provided an estimate of long-term predictive validity coefficients of the adjusted CPI-Cp scores. Job-Performance Criterion Measures Used in Criterion-Related Validity Analyses The two employment samples (Sample 5 and 6 above) used for the assessment of criterion-related validity of the adjusted CPI-Cp scale scores were originally used in the Hakstian et al. (2002) study. A description of the criterion variables used in these two employment samples is provided here: 1. Non-management sample of telephone service representatives (Subset of Sample 5). For this sample, the criterion variables were measured by a performance-appraisal instrument comprising 44 behavioral observation scale items assessing various work-related dimensions. Of these, three work dimensions were conceptually related to the Conscientiousness construct—(a) Availability for Work, (b) Desire to Improve, and (c) Use of Time. In addition, a sum of the 44 BOS items provided an Overall Performance measure. 2. Sample of armored-car company employees (Sample 6). The counterproductivity job behavior criterion in this sample was assessed from anecdotal file commentary, reprimands, disciplinary notations, etc., contained within the personnel file data of the employees. Hakstian et al. (2002) devised a coding system whereby file data were coded for frequency and seriousness 25 of various on-the-job counterproductive behaviors. The resulting criterion scores fell on a 6-point scale ranging from 0 (no counterproductive behaviors noted) to 5 (several and serious incidents—including discharge). They reported that these scores were extremely positively skewed: 42% of the sample received a zero score, the overall mean was 1.22 (on the 0-5 scale) and standard deviation was 1.44. Method Development of the Adjustment Equation Conceptually, the strategy for developing this adjustment equation involved predicting the amount of motivational distortion contained within a set of CPI data obtained under employment conditions, and then increasing the CPI-Cp scale scores by this amount of predicted distortion. (Note: Counterproductivity is a negative construct and subjects would aim to depress their CPI-Cp scores under employment conditions.) To predict the amount of motivational distortion, a regression equation was constructed to predict M D criterion scores from CPI-MD scale scores using employment-condition data from U B C Sample 4 (Sample 1 in this study). The dependent variable used in the regression was the M D criterion developed in Study 1 as these criterion scores indicated the extent to which subjects had distorted their CPI-Cp scores in the employment condition. To obtain the regression equation, the appropriate means, standard deviations, and correlation coefficient were substituted into the following raw-score regression equation formula: Y = r x y ^ X + ( Y - r x y ^ X ) , (1) where Y represents the M D criterion, X represents the CPI-MD scale, and r x y represents the correlation between the CPI-MD scale and the M D criterion. The means, standard deviations, and correlation coefficient were based on pooled-gender data so as to obtain a more stable and generic equation that could be applied for both genders. Even though significant gender 26 differences were reported for the CPI-MD scale (in Study 2), separate-gender equations were not preferable because of the small sample size for males. Two other alternative methods of developing the adjustment equation were considered, but as the three sets of adjusted CPI-Cp scores were highly correlated (average intercorrelation = .99), the procedure described above was favored as it offered the most conceptually parsimonious method of adjustment. Detailed descriptions of the alternative methods of adjustment, along with the equations for obtaining adjusted CPI-Cp scores by these methods, can be found in Appendix F. Correlations between Honest/Unadjusted CPI-Cp Scores and Adjusted CPI-Cp Scores To provide an estimate of the extent to which the developed motivational-distortion score adjustments to the CPI-Cp scale approximated honest scores, adjusted CPI-Cp scores (based on CPI scores from the employment condition) were correlated with CPI-Cp scores from the honest condition, using data from U B C Sample 4. In addition, correlations between unadjusted and adjusted CPI-Cp scores were obtained for the two employment samples—Samples 4 and 5. The relationship between these two sets of scores would provide an indication of the extent to which motivational distortion contaminated CPI-Cp scores obtained under employment conditions. Norms for the CPI-Cp Scale Using Adjusted Scores Separate-gender norms for the adjusted CPI-Cp scale were developed using the two workplace samples: (a) the management-level sample (Sample 4 above), and (b) the non-management-level sample (Sample 5 above). Analyses of gender differences on the adjusted CPI-Cp scale scores were conducted in both samples. In addition, inferential tests of mean differences were also conducted to compare unadjusted and adjusted CPI-Cp scores, separately by gender. 27 Reliability of the Adjusted CPI-Cp Scale Using U B C Sample 2 (Sample 2 in this study), the stability estimates of the adjusted CPI-Cp scores was assessed, separately by gender, using test-retest correlations between adjusted scores from two time points, with a 2-3 week time interval between administrations. Construct Validity of the Adjusted CPI-Cp Scale Using the Five Factor Model as a framework to assess the impact on the construct validity of the CPI-Cp scale when scores were corrected for motivational distortion, adjusted CPI-Cp scale scores were correlated with scores on the five NEO PI-R (Costa & McCrae, 1992) domain scales and the six NEO PI-R Conscientiousness facet scales, separately by gender. Using U B C Sample 3, this set of correlations was compared with that obtained from unadjusted scale scores. (The Conscientiousness facet scales were emphasized in this study because of their specific relevance to the counterproductivity construct. The comprehensive set of correlational analyses, with all NEO PI-R facet scales included, appear in Appendix G.) Assessment of the Criterion-Related Validity of Adjusted CPI-Cp Scores CPI-Cp data from the two employment samples (Subsample 5 and Sample 6) were corrected for the effects of motivational distortion and the resulting adjusted CPI-Cp scores were correlated with criterion scores. Due to sample size constraints, these correlations were based on pooled-gender samples. As pointed out in the Hakstian et al. (2002) study, given that there were no significant gender mean differences on the criterion measures in both samples, the resulting criterion correlations should be relatively free of between-groups correlational distortion. The criterion correlations were also partially disattenuated for criterion unreliability for the purposes of estimating true validity. In addition, for the armored-car company employees sample, job tenure was an obvious contaminating variable in the criterion scores (correlating .39 with the criterion), as the number of counterproductivity work incidents recorded in the employee personnel files would be a function of tenure. Therefore semi-partial correlations were 28 computed for this sample, with the length of tenure partialed out of the criterion scores in the CPI-Cp-criterion correlation. Results Adjustment Equation for Correcting CPI-Cp Scale Scores for Motivational Distortion As outlined above, the first step in obtaining the adjustment equation involved constructing a regression equation for predicting M D criterion scores (Y) from CPI-MD scale scores (X). The means, standard deviations, and correlation coefficients of these two variables in the regression equation appear in Table 7. Insert Table 7 about here Substituting these values into the raw-score regression equation formula in Eqn. (1), we obtain the predicted raw score on motivational distortion as: Y = .637X - 10.869. Therefore, adding the predicted amount of motivational distortion to the original CPI-Cp scale scores, the adjustment equation for correcting CPI-Cp scale scores for the effects of motivational distortion is: Adjusted CPI-Cp = Original CPI-Cp + .637(CPI-MD) - 10.869. Correlations between Adjusted/Honest CPI-Cp Scores and Unadjusted CPI-Cp Scores The correlations between employment-condition adjusted CPI-Cp scores and honest-condition CPI-Cp scores in the undergraduate student experimental sample (UBC Sample 4), and the correlations between unadjusted and adjusted CPI-Cp scores in two workplace samples, together with the associated descriptive statistics, appear in Table 8. Insert Table 8 about here Inferential tests revealed that the three pairs of separate-gender correlations were not significantly different at the .05 level and therefore they were pooled across gender, within each sample, to yield a weighted combined-gender correlation. The resulting pooled-gender 29 correlations are: (1) .654 (n = 250) for the adjusted-honest correlation in the undergraduate student sample; (2) .843 (n = 457) for the adjusted-unadjusted correlation in the management-level employees; and (3) .861 (n = 528) for the adjusted-unadjusted correlation in the non-management job-applicant sample. Although the correlation between the honest-condition and adjusted employment-condition CPI-Cp scores (.654, pooled-gender) from the experimental undergraduate student sample which the CPI-MD scale was developed, can be considered smaller than expected, this is significantly greater (p < .01) than the correlation obtained between honest-condition and unadjusted employment-condition scores (r=.517). Therefore, the results provide evidence to suggest that the proposed MD-adjustments to the CPI-Cp scale scores brought the employment-condition scores in closer alignment with honest-condition scores. Norms for the CPI-Cp Scale Using Adjusted Scores Separate-gender workplace-sample norms of the adjusted CPI-Cp scale scores are provided for management-level employees and non-management job-applicants. The normative data appear in Table 8 as descriptive statistics for the second and third sample-groups. The results from the analysis of gender mean differences (males - females) for the adjusted CPI-Cp scores was not significant for the management sample (p > .50) but significant for the non-management job applicants sample, ^(526) = 3.93, p < .001, A = .351, .95 C.I. for A = (.175, .526). Comparing the means of adjusted and unadjusted CPI-Cp scores within each gender, results showed a significant increase (at the/? < .001 level) in CPI-Cp mean scores when adjustments were made, in both the management and non-management level data samples, for both genders. The increase in mean scores ranged from about .85 to 1.20 standard deviation units between the two samples. These effect sizes are larger than those obtained by Alliger and Dwight (2000) in their meta-analytic study on the susceptibility of integrity tests to distortion. They found effect sizes of about Vi to Vi standard deviation for personality-based tests. 30 It is interesting to note as well that when adjustments for motivational distortion were made to the CPI-Cp scale scores from the employment samples, the resulting adjusted scale means ended up within .10 standard deviation units of the separate-gender means in the CPI Basic Norm Sample given in Hakstian et al. (2002). Reliability of the Adjusted CPI-Cp Scale The test-retest correlations (with a 2-3 week interval between administrations), computed separately by gender, were .883 (n = 71) for males and .869 (n = 158) for females. As these two stability coefficients for adjusted CPI-CP scale scores were not significantly different, they were pooled to yield a weighted estimate of .873 (n = 229) across the genders. The stability estimate of the adjusted CPI-Cp scale scores was comparable to that of the original (unadjusted) CPI-Cp scale reported by Hakstian et al. (2002) i.e. r = .893. This indicates that the adjustments for motivational distortion did not substantially reduce the stability of this scale. Construct Validity of the Adjusted CPI-Cp Scale The correlations of CPI-Cp scale scores (both unadjusted and adjusted) with the five NEO PI-R domain scales and the six NEO PI-R Conscientiousness facet scales appear in Table 9, separately by gender. Insert Table 9 about here In both the male and female samples, the obtained pattern of correlations with NEO PI-R domain scales differed between adjusted and unadjusted CPI-Cp scores. For males, whereas unadjusted CPI-Cp scores suggested that self-reported counterproductivity was associated with Extraversion, Agreeableness and Conscientiousness, adjusted scores suggested an association only with Extraversion and Agreeableness. In addition, when compared with unadjusted scores, the level of correlations obtained between adjusted CPI-Cp scores and NEO PI-R domain scores was significantly higher for Extraversion, and significantly lower for Neuroticism and 31 Conscientiousness (all at p < .001, nondirectional). A similar pattern was observed amongst the females: when compared with unadjusted CPI-Cp scores, adjusted scores showed a significantly stronger relationship between self-reported counterproductivity and Extraversion, and a significantly weaker relationship with Neuroticism and Conscientiousness (p < .001, nondirectional). At a global level, the pattern of correlations between CPI-Cp scores and NEO PI-R Conscientiousness facet scales was similar for both adjusted and unadjusted scores, in both gender samples. Among the six facet scales, Deliberation, Dutifulness and Self-discipline showed the strongest association with self-reported counterproductivity for both adjusted and unadjusted scores, although the level of correlation was reduced with adjusted scores. Looking at gender differences for the five NEO PI-R domains and six Conscientiousness facet scales, we see identical results for both unadjusted and adjusted CPI-Cp scores. Among the five domain scales, significant gender differences were found for Neuroticism (p < .05, nondirectional), with self-reported counterproductivity showing a Neuroticism component among women but not among men. Among the six Conscientiousness facet scales, significantly stronger correlations were obtained for women on both the Competence (p < .01, nondirectional) and Self-discipline (p < .05, nondirectional) scales. These results further emphasize the underlying differences in the psychological underpinnings of the counterproductivity construct between the two genders, that were first identified by Hakstian et al. (2002). At this point, it is important to remind the reader that these correlations were obtained in a sample (Sample 3 in this study) in which no motivational distortion was expected, as this was an undergraduate student sample to which both the CPI and NEO PI-R were administered under normal conditions. Thus, in these analyses, we cannot assert that the adjusted CPI-Cp scores are, in fact, more honest than the unadjusted scores. In fact, the CPI-MD mean scores obtained by 32 this sample suggested a low incidence of motivational distortion overall (for males, 21.47, n = 98; for females, 23.01, n = 222), even though inferential tests revealed significant mean differences (at p < .001) between the unadjusted and adjusted CPI-Cp scale scores, for both genders. Therefore, it is necessary to view the above results with this caveat in mind, and additional research is required to verify these findings. Assessment of the Criterion-Related Validity of Adjusted CPI-Cp Scores The correlations between the adjusted CPI-Cp scale scores and related job-performance criteria in two workplace samples appear in Table 10, along with the criterion correlations obtained using unadjusted CPI-CP scores. Both bivariate correlations and true validity coefficients, in which criterion correlations have been partially disattenuated for criterion unreliability, are presented. For the armored-car employee sample, semi-partial correlations, with the length of tenure partialed out of the criterion in the predictor-criterion correlations, are given as well. For partial disattenuation, inter-rater reliability estimates were needed, but not available in the present data sets. Thus, the meta-analytically-derived values of Viswesvaran, Ones, and Schmidt (1996) were used (as Hakstian et al., 2002, had done). Insert Table 10 about here For the sample of telephone service representatives, the pattern of criterion correlations across the four job-performance criteria revealed that criterion-related validity tended to increase when adjusted CPI-Cp scale scores were used. Inferential test results demonstrated a significant increase in validity coefficients for the Desire to Improve criterion [/(76) = 1.73,/? < .05] and the Overall Performance criterion [7(76) = 2.27,/? < .025]. For the armored-car employee sample, the bivariate correlations showed a non-significant increase in the bivariate criterion-related validity coefficient (from .25 to .28) with the use of adjusted CPI-CP scores. However, the semi-partial correlations showed a bridging of the 3 3 difference in criterion-related validity coefficients between using adjusted or unadjusted CPI-Cp scores. Because of a higher correlation between the adjusted CPI-Cp scores and length of tenure ( r = .13, p > .10), than was obtained for the unadjusted CPI-Cp scores and length of tenure, partialling out the effects of tenure from the criterion correlations led to a decrease in the validity coefficients for the adjusted scores. (In contrast, the correlation between unadjusted CPI-Cp scores and tenure was .03.) For comparative purposes, similar criterion-related analyses were conducted with CPI-Cp scores that were adjusted for motivational distortion on the basis of the CPI Manual's Fake-good scale (CPI-FG scale). Using an identical developmental procedure to that described above, a similar adjustment equation was obtained by building a regression equation to predict M D criterion scores from the CPI-FG scale scores and then adding this predicted amount of distortion to original CPI-Cp scale scores. The results obtained by using this new adjustment equation indicated similar improvements in criterion-related validity although the magnitude of increase was relatively less than that obtained by adjustments made using the CPI-MD scale developed in this study (See Appendix H). Summary of Results On the basis of a pooled-gender sample of 250 undergraduate student subjects, an adjustment equation was developed to correct CPI-Cp scores for the effects of motivational distortion. The effect of applying this equation was to increase CPI-Cp scores by the amount of distortion predicted on the basis of CPI-MD scale scores. Adjustments to the CPI-Cp scale scores brought the scores in closer alignment with honest scores as evidenced by a higher correlation obtained between adjusted and honest-condition scores as compared to unadjusted and honest-condition scores. Using adjusted scores, a new set of CPI-Cp scale norms were provided for two workplace samples. 34 Adjusted CPI-Cp scores maintained the high stability estimates of the original CPI-Cp scale but showed some changes in construct validity. When scores were adjusted for motivational distortion, correlations between the CPI-Cp scale and the Big Five factors of personality suggested that self-reported counterproductivity was related to Extraversion and Agreeableness for men, and predominantly to Extraversion, Agreeableness, and Conscientiousness for women. Finally, the study also demonstrated that when CPI-Cp scores were adjusted for motivational distortion, criterion-related validity for certain job-performance criteria in one employee sample increased. Overall Conclusions and Discussion The purpose of the present research was to develop a measure of motivational distortion that could be used as a basis for making adjustments to CPI-Cp scale scores for motivational distortion as it occurs under employment conditions. The CPI-MD scale provided such a measure, and as a self-report personality-based measure, demonstrated fully adequate reliability. The correlation of CPI-MD scale scores with the Big Five personality factors (using the NEO P i -ll.) revealed that motivational distortion was positively associated with Extraversion, Conscientiousness, and Emotional Stability. This pattern of correlations differed from other research involving various social desirability measures and is an indication that different psychological constructs might be needed to explain the concept of motivational distortion within organizational settings. There are several advantages in the developed CPI-MD scale. As noted earlier, when compared to the Fake-good scale from the CPI manual, the CPI-MD scale offers a more precise measurement of motivational distortion, as it is based only on those CPI items that are construct-relevant. Preliminary results also suggest that motivational distortion adjustments made using the CPI-MD scale were relatively more effective in increasing criterion-related validity of the CPI-35 Cp scale, as compared to adjustments made using the CPI-FG scale. In addition, when compared with other available social desirability or response distortion/validity scales, the present CPI-MD scale is especially adapted for use in employment settings. Not only was it designed to measure motivational distortion as it specifically occurs under employment conditions (as a result of its developmental methodology), but as a personality-based measure, it offers a covert assessment of distortion which reduces somewhat the likelihood of contamination by motivational distortion on the very scale itself. At this juncture, it is important to point out that the 56 items of the CPI-MD scale were identified using a student sample with simulated employment conditions and as such, the item-composition might reflect to some extent this experimental approach. Further research in applied settings is required to externally validate this scale. It should be noted as well that the present CPI-MD scale was developed with the specific intention of using this scale as a basis for adjusting scores on the CPI-Cp scale for motivational distortion. The use of the CPI-MD scale for alternative purposes, including the derivation of adjustment equations for other personality scales has not been tested. However, the results from this study indicate the viability of the scale construction methodology used, and with this methodology, similar motivational distortion measures and adjustment equations could be obtained for other personality inventories/scales. Finally, this study provides evidence that adjusting personality scale scores for motivational distortion can enhance the psychometric properties of the scale being measured. In this case, the results showed that score-adjustments preserved test-retest reliability, altered the construct validity of the scale being measured, and led to an increase in criterion-related validity in one workplace sample. Although this is only a single study in the face of many other studies that have found evidence of the contrary, these findings contribute to ongoing research that supports the argument for controlling the effects of motivational distortion in personnel testing. 36 References Alliger, G. M . , & Dwight, S. A. (2000). A meta-analytic investigation of the susceptibility of integrity tests to faking and coaching. Educational and Psychological Measurement, 60(1), 59-72. Barrick, M . R., & Mount, M . K. (1991). The big-five personality dimensions and job performance. Personnel Psychology, 44, 1-26. Barrick, M . R., & Mount, M . K. (1996). Effects of impression management and self-deception on the predictive validity of personality constructs. Journal of Applied Psychology, 81, 262-272. Becker, T. E., & Colquitt, A. L. (1992). Potential versus actual faking on a biodata form: An analysis along several dimensions of item type. Personnel Psychology, 45, 389-406. Cellar, D. F., Miller, M . L., Doverspike, D. D., & Klawsky, J. D. (1996). Comparison of factor structures and criterion-related validity coefficients for two measures of personality based on the five factor model. Journal of Applied Psychology, 81(6), 694-704. Christiansen, N . D., Goffin, R. D., Johnston, N . G , 8c Rothstein, M G . (1994). Correcting the 16PF for faking: Effects on criterion-related validity and individual hiring decisions. Personnel Psychology, 47, 847-860. Collins, J. M . , & Glese, D. H. (1998). Race, job applicants and the Five Factor Model, Journal of Applied Psychology, 83, 531-544. Costa, P. T., Jr., & McCrae, R. R. (1992). The Revised NEO Personality Inventory (NEO PI-R) and NEO Five Factor Inventory (NEO-FFI) professional manual. Odessa, FL: Psychological Assessment Resources. Cunningham, M . R., Wong, D. T., & Barbee, A. P. (1994). Self-presentation dynamics on overt integrity tests: Experimental studies of the Reid Report. Journal of Applied Psychology, 79(5), 643-658. 37 Dahlstrom, W. G., Welsh, G. S., & Dahlstrom, L. E. (Eds.) (1972). An MMPI Handbook (Vol. 1). Minneapolis: University of Minnesota Press. Dahlstrom, W. G., Welsh, G. S., & Dahlstrom, L. E. (Eds.) (1975). An MMP I Handbook (Vol. 2). Minneapolis: University of Minnesota Press. Douglas, E. F., McDaniel, M . A., & Snell, A. F. (1996, August). The validity of non-cognitive measures decays when applicants fake. Paper presented at the annual meeting of the Academy of Management, Cincinnati, OH. Ellingson, J. E., Sackett, P. R., & Hough, L. M . (1999). Social desirability corrections in personality measurement: Issues of applicant comparison and construct validity. Journal of Applied Psychology, 84(2), 155-166. Ellingson, J. E., Smith, D. B., & Sackett, P. R. (2001). Investigating the influence of social desirability on personality factor structure. Journal of Applied Psychology, 86(1), 122-133. Gordon, M . E., & Gross, R. H. (1978). A critique of methods for operationalizing the concept of fakeability. Educational and Psychological Measurement, 38, 771-782. Gough, H . G. (1957). Manual for the California Psychological Inventory. Palo Alto, CA: Consulting Psychologists Press. Gough, H. G. (1987). The California Psychological Inventory administrator's guide. Palo Alto, CA: Consulting Psychologists Press. Gough, H . G., & Bradley, P. (1996). The California Psychological Inventory manual (3rd ed.). Palo Alto, CA: Consulting Psychologists Press. Guastello, S. J., & Rieke, M . L. (1991). A review and critique of honest test research. Behavioral Sciences and the Law, 9, 501-523. Hakstian, A. R., Farrell, S., & Tweed, R. T. (2002). The assessment of counterproductive tendencies by means of the California Psychological Inventory. International Journal of Selection and Assessment, 10(1/2), 58-86. 38 Helmes, E. (2000). The role of social desirability in the assessment of personality constructs. In R. D. Goffin & E. Helmes (Eds.), Problems and solutions in human assessment: Honoring Douglas N. Jackson at seventy, (pp. 21-40). Norwell, M A : Kluwer Academic Publishers. Hogan, R. (1986). Hogan Personality Inventory. Minneapolis, M N : National Computer Systems. Hogan, J., & Hogan, R. (1972). Hogan Personality Inventory manual. Tulsa, OK: Hogan Assessment Systems. Horst, P. (1966). Psychological Measurement and Prediction. Belmont, CA: Wadsworth Publishing. Hough, L . M . (1998). Effects of intentional distortion in personality measurement and evaluation of suggested palliatives. Human Performance, 5, 139-155. Hough, L . M . Eaton, N . L. , Dunnette, M . D., Kamp, J. D., & McCloy, R. A. (1990). Criterion-related validities of personality constructs and the effect of response distortion on those validities [Monograph]. Journal of Applied Psychology, 75, 581-595. Hough, L. M . , & Schneider, R. J. (1996). The frontiers of I/O personality research. In K.R. Murphy (Ed.,), Individual differences and behavior in organizations (pp. 31-88). San Francisco: Jossey-Bass. Jackson, D. N . (1967). Personality Research Form manual (3 r d ed.). Port Huron, MI: Research Psychologists Press. Kluger, A. N . , & Colella, A. (1993). Beyond the mean bias: The effect of warning against faking on biodata item variances. Personnel Psychology, 46, 763-780. Lanning, K. (1989). Detection of invalid response patterns on the California Psychological Inventory. Applied Psychological Measurement, 13, 45-56. Lautenschlager, G. J. (1986). Within-subject measures for the assessment of individual differences in faking. Educational and Psychological Measurement, 46, 309-316. 39 Livneh, H. , & Livneh, C. (1989). The five-factor model of personality: Is evidence of its cross-measure validity premature? Personality and Individual Differences, 10(1), 75-80. LoBello, S. G., & Sims, B. N . (1990). Fakability of a commercially produced pre-employment test. Journal of Business and Psychology, 8, 265-273. McCrae, R. R., & Costa, P. T., Jr. (1983). Social desirability scales: More substance than style. Journal of Consulting and Clinical Psychology, 51, 882-888. McDaniel, M . A. , Douglas, E. F., & Snell, A. F. (1997). A survey of deception among job seekers. Paper presented at the 12 th annual meeting of the Society for Industrial and Organizational Psychology. McFarland, L . A. (2000). Toward an integrated model of applicant faking. Unpublished manuscript, Michigan State University. McFarland, L. A., & Ryan, A. M . (2000). Variance in faking across noncognitive measures. Journal of Applied Psychology, 85(5), 812-821. Murphy, K. R., & Davidshofer, C. O. (1998). Psychological testing: Principles and applications. Upper Saddle River, NJ: Prentice-Hall. Nicholson, R. A., & Hogan, R. (1990). The construct validity of social desirability. American Psychologist, 45, 290-292. Ones, D. S., & Viswesvaran, C. (1998a). The effects of social desirability and faking on personality and integrity assessment for personnel selection. Human Performance, 11(2/3), 245-269. Ones, D. S., & Viswesvaran, C. (1998b). Integrity testing in organizations. In R. W. Griffin, A. O'Leary-Kelly & J. M . Collins (Eds.), Dysfunctional behavior in organizations: Vol 2. Non-violent behaviors in organizations. Greenwich, CT: JAI. 40 Ones, D. S., Viswesvaran, C , & Reiss, A. D. (1996). Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81(6), 660-679. Ones, D. S., Viswesvaran, C , & Schmidt, F. L. (1993). Comprehensive meta-analysis of integrity test validities: Findings and implications for personnel selection and theories of job performance [Monograph]. Journal of Applied Psychology, 78, 619-103. Paulhus, D. L . (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17-59). San Diego, CA: Academic Press. Rosse, J. G., Stecher, M . D., Miller, J. L. , & Levin, R. A. (1998). The impact of response distortion on preemployment personality testing and hiring decisions. Journal of Applied Psychology, 83(4), 634-644. Ryan, A. M . , & Sackett, P. R. (1987). Pre-employment honesty testing: Fakability, reactions of test takers, and company image. Journal of Business and Psychology, 1(3), 248-256. Schmidt, M . J., & Ryan, A. M . (1993). The Big Five in personnel selection: Factor structure in applicant and nonapplicant populations. Journal of Applied Psychology, 78, 966-974. Smith, D. B. , Hanges, P. J., & Dickson, M . W. (2001). Personnel selection and the Five-Factor Model: Reexamining the effects of applicants' frame of reference. Journal of Applied Psychology, 86(2), 304-315. Viswesvaran, C , & Ones, D. S. (1999). Meta analyses of fakability estimates: Implications for personality measurement. Educational and Psychological Measurement, 59(2), 197-210. Viswesvaran, C , Ones, D. S., & Schmidt, F. L . (1996). Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology, 81, 557-574. 41 Weiner, J. A. , & Gibson, W. M . (April 2000). Practical effects offaking on job applicant attitude test scores. Paper presented at the 15 t h annual meeting of the Society of Industrial and Organizational Psychology, New Orleans. Williams, R. H. , Zimmerman, D. W., & Mazzagatti, R. D. (1987). Large sample estimates of the reliability of the simple, residualized and base-free measures gain scores. Journal of Experimental Education, 55, 116-118. Winder, P., O'Dell, J. W., & Karson, S. (1975). New motivational distortion scales for the 16PF. Journal of Personality Assessment, 39, 532-537. Zerbe, W. J., & Paulhus, D. L. (1987). Socially desirable responding in organizational behavior: A reconception. Academy of Management, 12(2), 250-264. Zickar, M . , Ross, J., & Levin, R. (1996). Modeling the effects of faking on personality scales. Paper presented at the Eleventh Annual Conference of the Society for Industrial and Organizational Psychology. San Diego, CA. Zickar, M . J. (1997). Computer simulation of faking on a personality test. Paper presented at the annual meeting of the Society for Industrial and Organizational Psychology, St. Louis, MO. Zickar, M . J., & Robie, C. (1999). Modeling faking good on personality items: An item-level analysis. Journal of Applied Psychology, 84(4), 551-563. 42 Table 1 Means, Standard Deviations, and Intercorrelation Matrix of the Four Measures of Change Measure of Change3 Males Females Intercorrelation Matrix (n = 55) (n = 195) (combined gender, n = 250) Mean SD Mean SD 1 2 3 4 1. Raw score differenceb 8.96 11.72 12.79 10.50 (.78) 2. Residual change (standardized) -.13 1.12 .04 .96 .91 (.82) 3. Phi correlation .51 .26 .49 .22 -.81 -.79 (.70) 4. Variance of the differences .23 .13 .25 .11 .81 .80 -.99 (.74) Note: The values in parentheses along the principal diagonal o f the intercorrelation matrix represent reliability estimates. aSee the text for a description of the four measures of change. bScores on the raw-score difference variable were mean-deviated by gender (to scale out gender differences) before pooling for the correlational analysis. 43 Table 2 Factor Pattern and Factor Score Coefficient Vectors Associated with the First Principal Component. Pattern Vector Factor Score Coefficient Vector Variables3 (Weight Vector) 1. Raw score difference .936 .263 2. Standardized residual change .927 .260 3. Phi correlation -.954 -.268 4. Variance of the differences .956 .269 Variance accounted for by first principal component: 89%. aSee the text for a description of the four measures of change. 44 Table 3 The 56 CPI Items Selected for the CPI-MD Scale (Item numbers given are for CPI Form 434) Item No. Item Statement True: 5 Our thinking would be a lot better off if we would just forget about words like "probably," approximately," and "perhaps." 14 I always follow the rule: business before pleasure 52 I usually take an active part in the entertainment at parties. 65 I think I would like the work of a clerk in a large department store. 69 I would disapprove of anyone's drinking to the point of intoxication at a party. 167 I would like to belong to several clubs or lodges. 179 When I work on a committee I like to take charge of things. 195 The most important things to me are my duties to my job and to other people. 202 If given the chance, I would make a good leader of people. 218 I love to go to dances. 222 I would like to belong to a discussion and study club. 223 I keep out of trouble at all costs. 226 People seem naturally to turn to me when decisions have to be made. 239 I like to talk before groups of people. 254 I have never deliberately told a lie. 255 Only a fool would try to change our American way of life. 278 If I get too much change in a store, I always give it back. 292 I used to like it very much when one of my papers was read to the class in school. 305 I often wish people would be more definite about things. 319 In a group, I usually take the responsibility for getting people introduced. 345 I go out of my way to meet trouble rather than try to escape it. 354 A strong person will be able to make up his or her mind even on the most difficult questions. 355 I have strong political opinions. 359 I think I am usually a leader in my group. 361 I like to have a place for everything and everything in its place. 370 Disobedience to any government is never justified. 371 I can't really enjoy a rest or vacation unless I have earned it by some hard work. 376 I enjoy planning things and deciding what each person should do. 382 Success is a matter of will power. 392 I daydream very little. 403 I have a natural talent for influencing people. 408 I always see to it that my work is carefully planned and organized. 424 The one to whom I was most attached and whom I most admired as a child was a woman (mother, sister, aunt, or other woman). (table continues) 45 Table 3 (Continued) The 56 CPI Items Selected for the CPI-MD Scale (Item numbers given are for CPI Form 434) Item No. Item Statement False: 30 I gossip a little at times. 38 It is hard for me to start a conversation with strangers. 55 Some of my family have quick tempers. 66 Sometimes I feel like swearing. 91 Sometimes I think of things too bad to talk about. 124 I am likely not to speak to people until they speak to me. 139 It is all right to get around the law if you don't actually break it. 151 I'm not the type to be a political leader. 156 I hardly ever get excited or thrilled. 170 I often act on the spur of the moment without stopping to think. 191 I can remember "playing sick" to get out of something. 203 When things go wrong, I sometimes blame the other person. 238 Sometimes I just can't seem to get going. 258 In school I found it very hard to talk before the class. 262 There have been a few times when I have been very mean to another person. 268 At times I have been very anxious to get away from my family. 275 Sometimes I rather enjoy going against the rules and doing things I'm not supposed to do. 313 I dislike to have to talk in front of a group of people. 334 I get nervous when I have to ask someone for a job. 379 I would rather not have very much responsibility for other people. 414 I do not read every editorial in the newspaper every day. 416 I don't think I'm quite as happy as others seem to be. 418 I am embarrassed with people I do not know well. 426 I get tired more easily than other people seem to. Note. Modified and reproduced by special permission of the Publisher, Consulting Psychologists Press, Inc., Palo Alto, CA 94303 from California Psychological Inventory™, Form 434 by Harrison G. Gough. Copyright 1987 by Consulting Psychologists Press, Inc. All rights reserved. Further reproduction is prohibited without the Publisher's written consent. 46 Table 4 Normative Data for the CPI-MD Scale Norm Group3 n M s Undergraduate University Students Males 350 23.47 6.83 Females 739 23.07 6.20 Management-Level Employees Males 245 29.00 7.62 Females 212 28.32 7.05 Non-Management-Level Job Applicants Males 193 35.11 8.05 Females 335 34.19 7.58 Note: The tabled means and standard deviations are for the 56-item CPI-MD scale described in the text, composed of the items listed in Table 3. aThese norm samples are obtained from the Hakstian et al. (2002) study. See the text for demographic details about each norm group listed above. 47 Table 5 Internal-Consistency (Alpha) Reliability Estimates, rtt, and Standard Errors of Measurement, se, for the CPI-MD Scale in Three Separate-Gender Samples Sample n a sa r t t se Undergraduate Students Males Females Management-Level Employees Males Females Non-Management-Level Job Applicants Males 165 8.11 .86 3.03 Females 286 7.52 .83 3.10 aSample sizes and standard deviations differ slightly in some cases from those in Table 4 because only those subjects who had responded to every one of the 56 items of the CPI-MD scale were included in the analyses of internal-consistency reliability. 339 6.87 .77 <> 3.29 691 6.21 .72 3.29 184 7.72 .83 3.18 180 7.19 .81 3.13 48 Table 6 Correlations of Scores on the CPI-MD Scale with Scores on the Five NEO PI-R Domain Scales and the NEO PI-R Neuroticism, Extraversion and Conscientiousness Facet Scales. Males (n = 98) Females (n = 222) Scale Correlated with CPI-MD Bivariate r Disattenuated" r Bivariate r Disattenuated" r NEO PI-R Domain Scales Neuroticism -.32* -.41* -.40* -.49* Extraversion .51* .66* .49* .61* Openness .10 .13 .07 .09 Agreeableness -.06 -.08 -.02 -.03 Conscientiousness .36* .46* .45* .54* Neuroticism Facet Scales Anxiety -.17 -.24 -.24* -.32* Angry Hostility -.16 -.24 -.17 -.22 Depression -.20 -.34 -.35* -.44* Self-Consciousness -.34* -.49* -.43* -.62* Impulsiveness -.16 -.24 -.25* -.38* Vulnerability -.34* -.48* -.41* -.55* Extraversion Facet Scales Warmth .35* .47* .36* .48* Gregariousness .37* .51* .27* .37* Assertiveness .61* .84* .65* .84* Activity .34* .50* .36* .66* Excitement-Seeking .12 .20 .02 .04 Positive Emotions .34* .47* .35* .46* Conscientiousness Facet Scales Competence .24 .38 .39* .54* Order .15 .22 .36* .49* Dutifulness .20 .34 .31* .47* Achievement Striving .38* .53* .41* .55* Self-Discipline .37* .52* .35* .45* Deliberation .09 .14 .22* .29* "These correlations are fully disattenuated on the basis of the specific internal-consistency reliability estimates obtained in these samples. *p < .01 (non-directional). 49 Table 7 Descriptive Statistics and Correlation of the CPI-MD Scale andMD Criterion Scores. Mean SD Measure (Variable in Regression) (n = 250) (w = 250) Motivational distortion criterion (Y) 11.95 10.87 CPI-MD scale (X) 35.82 9.26 Correlation, r^, = .543 50 Table 8 Comparison of Means, Standard Deviations, and Correlations between Honest-Condition/Unadjusted and Adjusted CPI-Cp scores in Three Separate-Gender Samples 1. Undergraduate Student Sample3 Males Females (n = 55) (n = 195) Type of CPI-Cp Score Mean SD Mean SD 1. Honest Condition 35.53 13.67 34.08 10.98 2. Adjusted 36.53 10.08 33.80 7.67 Correlation (r^) .70 .64 2. Management Level Employees Sample Males Females (n = 245) (n = 212) Type of CPI-Cp Score Mean SD Mean SD 1. Unadjusted 28.39 9.19 28.45 8.07 2. Adjusted 35.99 8.67 35.62 7.57 Correlation (ri 2) .85 .83 3. Non-Management Level Job-Applicants Sample Males Females (n = 193) (n = 335) Type of CPI-Cp Score Mean SD Mean SD 1. Unadjusted 23.94 9.98 21.62 9.63 2. Adjusted 35.43 7.64 32.53 8.34 Correlation (ri 2) .86 .86 'This is the sample in which the CPI-MD scale was developed. 51 Table 9 Correlations of Adjusted and Unadjusted CPI-Cp Conscientiousness Facet Scale Scores Scale Scores with NEO PI-R Domain and Males (n = 98) Females (n = 222) NEO PI-R Scale Correlated with Unadjusted Adjusted Unadjusted Adjusted CPI-Cp CPI-Cp CPI-Cp CPI-Cp CPI-Cp Domain Scales Neuroticism .17 .05 .45* .30* Extraversion .39* .58* .20* .39* Openness .19 .23 .16 * .19* Agreeableness -.33* -.35* -.44* -.45* Conscientiousness -.38* -.24 -.60* -.44* Conscientiousness Facet Scales Competence -.02 .07 -.43* -.28* Order -.18 -.12 -.31* -.17* Dutifulness -.37* -.29* -.52* -.41* Achievement Striving -.15 -.01 -.37* -.22* Self-Discipline -.29* -.15 -.53* -.41* Deliberation -.51* -.47* -.59* -.52* *p < . 01 (nondirectional). 52 Table 10 Correlations between the CPI-Cp Scale Scores (with and without adjustments for motivational-distortion) and Counterproductive-Behavior Criteria in Two Non-Management Level Samples 1. Telephone service representatives (n = 79, combined genders) Counterproductive-Behavior Criterion Form of Criterion Correlation Availability Desire to Use of Time Overall for Work Improve Performance Unadjusted Bivariate r -.23* _ 2 7 * * -.26* -.23* Unadjusted True Validity 3 -.31* -.36** -.35* -.32* Adjusted Bivariate r -.26* _ 2 7 * * * _ 29** -.36*** Adjusted True Validity 3 -.35* _ 4 9 * * * _ 3 9 * * -.50*** 2. Armored-car company employees (n = 91, combined genders) Counterproductive-Behavior Criterion Form of Criterion Correlation Bivariate correlation Semi-partial correlationb Unadjusted Bivariate/Semi-partial rh .25** .26* Unadjusted True Validity 3 .33** .35* Adjusted Bivariate/Semi-partial r° .28** .25** Adjusted True Validity 3 .37** .33** Note: *p < .05; **p < .01; ***p < .001 (all directional). Criterion correlations for the unadjusted CPI-Cp scores are taken from Hakstian et al. (2002). 3These true validities are the bivariate (or semi-partial) correlations corrected for attenuation in the criterion variable only by means of the inter-rater reliability estimates discussed in the text. bLength of tenure was partialed out of the criterion variable in the CPI-Cp (adjusted and unadjusted) vs. criterion correlation since scores on this criterion were a function of the number of counterproductive work incidents recorded in employees' files over the tenure of their employment (see text for more detail). 53 Appendix A - Analyses to Determine the Effectiveness of the Experimental Procedure in Study 1 As identified within the body of the thesis, there were two main concerns with regard to the effectiveness of the experimental procedure in Study 1. First, there were concerns that the instructions given to simulate an employment context might not have been sufficiently potent to have elicited motivational distortion for a sample consisting of undergraduate students for whom the personality test results had effectively no consequence. The second concern was with regard to the lack of randomization in the administration of experimental conditions to the subjects. Instead, all subjects took the CPI under the employment condition followed by the honest condition. As indicated in the body of the thesis, the intention was to closely simulate employment conditions in which prior viewing of the personality test would not have taken place. However, without controlling for the effects of experimental order, there was concern that the honest condition might not truly elicit honest responding, but was in some way eliciting contaminated responses, by having been preceded by the employment condition. The above two concerns can be summarized in the following two research questions: 1. Were the CPI-Counterproductivity scale scores obtained in the employment condition significantly lower than those obtained in the honest condition? (The employment condition represented the condition for motivational distortion and it would be expected that subjects would tend to lower their counterproductivity scores.) 2. Were the CPI-Counterproductivity scale scores obtained in the honest condition comparable to a normative score distribution for undergraduate students? If the research provided affirmative support for the above two questions, there would be sufficient evidence to suggest that (1) the employment-condition instructions were able to elicit 54 motivational distortion, and (2) responses to the CPI in the honest condition were not significantly contaminated by prior exposure to the inventory under a different test-taking condition. Method and Results For research question 1, using the U B C sample from Study 1, separate-gender paired samples 7-tests were conducted to determine whether CPI-Cpempioyment mean scores were lower than CPI-Cphonest mean scores. The separate-gender means and standard deviations for the CPI-Cp scores in the employment and honest conditions appear in Table A l . Inferential tests showed significant differences for both genders: for males, 7(54) = -5.67, p < .0001, A = -.771, .95 C.I. for A = (-1.044, -.499); for females, 7(194) = -17.02, p< . 0001, A = -1.311, .95 C.I. for A = (-1.463, -1.159). Therefore, there is evidence to suggest that the experimental manipulation used in Study 1 successfully elicited motivational distortion. Insert Table A l about here For research question 2, CPI-Cp mean scores obtained from the honest condition in Study 1 were compared with a large undergraduate student sample (UBC Sample 1 from Study 2—350 males, 739 females), separately by gender. The incidence of motivational distortion in this normative sample can be assumed to be minimal as the student subjects had been asked to complete the CPI honestly, for research purposes. The descriptive statistics appear in Table A2. Insert Table A2 about here Inferential test results showed non-significant mean differences for both males (p > .10) and females (p > .25). Therefore, there is insufficient evidence to suggest that preceding the honest condition with the employment condition affected the integrity of the CPI data obtained in the honest condition. 55 Table A l Means and Standard Deviations of the CPI-Cp Scores in the Employment and Honest conditions of the Experimental Sample Males Females ( « = 55) (n = 195) Experimental Condition Mean SD Mean SD Employment 26.56 11.63 21.29 9.76 Honest 35.53 13.67 34.08 10.98 56 Table A2 Descriptive Statistics of the CPI-Cp scores in the Experimental Sample and a Normative Undergraduate Student Sample. Males Females Sample N Mean SD N Mean SD Experimental 55 35.53 13.67 195 34.08 10.98 Normative 350 38.03 11.32 739 34.96 10.29 57 Appendix B - Administration Instructions for the Two Experimental Conditions in Study 1 The full instructions for the employment condition are as follows: I will be describing to you a scenario and would like you to put yourself into that scenario when you are responding to the personality test. Imagine that you are applying for a job that you would like to have. This could be a career that you are pursuing or a job that you are currently applying for. After sending in your application form, you are informed by the company that part of their hiring procedure requires that you complete some paper-and-pencil tests. During the test session at their office, you are requested to complete this personality inventory (hand the CPI to the subject at this point). The HR representative informs you that your responses on the inventory will provide valuable information about yourself that will be used as part of the decision-making process. The personality inventory consists of 434 true/false statements. It will take you about lhr to complete this and you will need to respond to every item. If you have no questions, please begin. Remember to respond to the items as you would if you were applying for this job. The full instructions for the honest condition are as follows: You will notice that this is the same personality inventory that you completed last week. In that session, you were asked to complete the inventory as you would if you were applying for a job that you really want. It is normal for people who are applying for a job to respond to personality questionnaires in a different way from how they would normally. This might or might not have been true for you. Regardless, this time round, I would like you to respond to the statements as you truly see yourself to be. You are assured once again that the information we gather from you in this experiment will be kept confidential and only used for research purposes. In addition, the feedback that you will receive on your personality profile will be based on your responses to the inventory in today's session. Therefore, in order for this feedback to be as accurate as possible, you are encouraged to be respond to the statements in the inventory as honestly as you can. 58 Appendix C - Additional Information about the Phi Correlation and Variance of the Differences Measures of Change (based on Lautenschlager, 1986) Phi Correlation Measure As the CPI comprises true-false items, a 2 x 2 contingency table for each subject can be set up, with the response choices for the employment condition represented by the rows and the honest condition by the columns. As an example, one cell, say the "True-True" cell would contain the number of items on which this subject answered True on both administrations. A phi correlation coefficient can therefore be obtained for each subject where large positive values indicate greater consistency of responses or more specifically, less distortion of responses in the employment condition. As the phi coefficient tends towards zero or becomes negative, this would indicate greater amounts of response shifts or motivational distortion. Therefore, this within-subject Phi Correlation Measure provides an indication of the consistency of a subject's responses to the CPI-Cp scale items between the employment and honest conditions. Variance of the Differences Measure The Variance of the Differences measure represents the within-subject variance of the differences in the responses to each of the 80 CPI-Cp items taken in the two response conditions—employment and honest. In the present study, CPI items were coded 1 for true and 2 for false. A difference in "scores" (honest - employment) was computed for each of the 80 CPI-Cp items. A difference of "0" is obtained if responses on that item remained consistently true or false between the two conditions, a " - 1 " i f the responses changed from true to false, and a "+1" if the responses changed from false to true. Each subject would obtain a total of 80 separate item-level difference scores (corresponding to the 80 CPI-Cp items) and the variance of these differences was termed the within-subject Variance of the Differences measure. Large values 59 would indicate more shifts in responses across the 80 items and hence a greater amount of motivational distortion having taken place in the employment condition. Smaller values would indicate consistency in responding or less distortion. For a more detailed review of the use of these two measures as within-subject measures of individual differences in motivational distortion, the reader is referred to the original source, Lautenschlager (1986). 6 0 Appendix D: Estimation of Reliability for the Four Change Measures and the Composite Motivational Distortion Criterion From Williams, Zimmerman, and Mazzagatti (1987), the reliability of simple difference scores (rdd) and residualized difference scores (r r e s) can be calculated using the following equations: Si 2 ~t~ S2 2 — 2ri2SiS2 ' _ T22 + Ti2 2 T n — 2f i2 2 ^es — 1 2 > I — In where si and s2 represent the standard deviation of the CPI-Cp scale scores obtained in the honest and employment condition, respectively, rn and r 2 2 represent the internal consistency reliability estimates of the CPI-Cp scale scores obtained in the honest and employment condition, respectively, and r i 2 represents the correlation between honest-condition and employment-condition CPI-Cp scores. For the Phi Correlation Measure, split-half reliability estimates were used as indicated in the body of the thesis. The 80 CPI-Cp items were separated into two groups of 40 items. Taking the first 40 items, a phi coefficient was computed for each subject that indicated his/her response-consistency to these items between the two test conditions—employment and honest. A second phi coefficient was similarly computed using the last 40 items. Therefore, each subject would receive two separate phi coefficient values and the correlation between these two sets of coefficients (across all subjects) represented a measure of split-half reliability. For the Variance of the Differences Measure, a similar procedure of obtaining split-half reliability estimates was used. For each subject, two separate variances were obtained from the two different groups of 40 CPI-Cp items. The correlation of these two variances, across all subjects, constituted the split-half reliability estimate. 61 Finally, the reliability estimate, r?c, of the aggregated M D criterion measure was calculated using the standard equation for estimating the reliability of a differentially-weighted linear combination of measures, each of whose reliabilities are known (see, e.g., Horst, 1966, pp.280-281). As the M D criterion scores were the first principal component scores of the four change measures, the weight vector was simply the weight vector for obtaining the first principal component from standard scores on the constituent variables. The formula for r P C is as follows: a'D,„2Dua rPC = 1 - - ; , a DaxRxxD„xa where Du = I - D r i i , D r i i is a diagonal matrix of the reliabilities of the measures, a is the first principal component weight vector, D 0 x is the diagonal matrix of the standard deviations of the measures, and R x x is the matrix of intercorrelations of the four measures. In the present case, D C x was simply the identity matrix as the first principal component was a (0, 1) variable prior to rescaling into the metric of the raw-change scores. 62 Appendix E: Correlations between the Motivational Distortion Criterion and the 20 Standard CPI Scales Correlation with M D Criterion U B C Undergraduates Males Females CPI (Form 434) Standard Scale (w = 55) (w = 195) Dominance .34* .30* Capacity for Status .15 .19* Sociability .26 .25* Social Presence .03 -.01 Self-Acceptance .07 .16* Independence .05 .09 Empathy .10 .05 Responsibility .07 .19* Socialization -.01 .11 Self-Control -.00 .17* Good Impression .18 .31* Communality -.06 -.04 Well-Being .00 .16* Tolerance -.01 -.00 Achievement via Conformance .16 .24* Achievement via Independence -.09 -.09 Intellectual Efficiency -.00 .06 Psychological Mindedness .01 -.04 Flexibility -.23 -.24* Femininity/Masculinity .09 .03 *p < .05 (nondirectional). 63 Appendix F: Two Alternative Methods of Deriving Adjustment Equations to Correct for Motivational Distortion in CPI-Cp Scores Two alternative methods of developing an adjustment equation to correct CPI-Cp scores for motivational distortion, were considered. As mentioned in the thesis body, scores obtained from these two methods correlated almost perfectly (average r - .99) with scores obtained with the method of adjustment presented in the thesis body. The high correlation amongst the three methods is not surprising as the three separate methods of adjustments are, in effect, simply different linear combinations of the CPI-MD and CPI-Cp scale scores. A l l three methods involved a prediction of the amount of motivational distortion contained within a set of CPI data, and adding this predicted amount of distortion to original CPI-Cp scores. The difference lay in the means with which the prediction of motivational distortion was derived. The concept behind these two alternative methods are presented here along with the resulting adjustment equations. In both cases, the equations were based on data from the experimental sample of Study 1. Alternative Method 1 First, a regression equation for predicting employment-condition CPI-Cp scale scores (CPI-Cpempioyment) from CPI-MD scale scores was obtained, separately by gender (due to gender differences on the latter scale). Substituting the appropriate means, standard deviations and correlation coefficient for the CPI-MD scale and CPI-Cpempioyment scores from Table F l into Equation 1 in Study 3, the separate-gender raw-score regression equations are: For males: Y = -.739X + 50.732; For females : Y = -.626X + 44.252, where Y represents the predicted CPI-CPempi0yment score and X , the CPI-MD scale score. 64 Insert Table F l about here The residuals about this regression line (Y - Y) represent the portion of CPI-Cpempioyment scores that cannot be predicted by CPI-MD scale scores and therefore, can be construed as a more honest approximation of CPI-Cp scores. Correspondingly, the separate-gender adjustment equations are simply the equations for obtaining these residual scores: For males: Adjusted CPI-Cp = Original CPI-Cp + .739 (CPI-MD) - 50.732; For females: Adjusted CPI-Cp = Original CPI-Cp + .626 (CPI-MD) - 44.252. It should be noted that this method of adjustment results in an overall mean of zero for the adjusted CPI-Cp scale scores. To obtain mean scale scores that are more reflective of honest scores, the results from the adjustment equations can be rescaled to a different metric. Alternative Method 2 In this method, a generic adjustment equation that could be applied to both genders, was developed. As before, a regression equation was constructed for predicting employment-condition CPI-Cp scale scores (CPI-Cpempioyment) from CPI-MD scale scores. However, in this method, the means and standard deviation for the Y variable in the regression equation were based on that of the raw-score difference metric (Y = 11.95 and Sy = 10.87). (Note: the raw-score difference is one of the four change measures used in Study 1). Substituting these Y -values, together with the total-group means, and standard deviations for the CPI-MD scale, and the correlation of-.587 between the CPI-CPempioyment and CPI-MD scores from Table E l , the raw score regression equation is: Y = -.690X + 12.77, where Y and X represent the predicted CPI-CPempioyment score and CPI-MD scale score, respectively. The generic adjustment equation would therefore be: Adjusted CPI-Cp = Original CPI-Cp + .690 (CPI-MD scale score) - 12.77. 65 Table F l Means and Standard Deviations of the Employment-Condition CPI-Cp and CPI-MD Scale Scores Males Females Total (n = 55) (« = 195) in = 250) CPI-scale Mean SD Mean SD Mean SD CPI-Cp (Y) 26.56 11.63 21.29 9.76 22.45 10.41 CPI-MD (X) 32.71 9.24 36.70 9.16 35.82 9.26 Correlation, r^ (combined genders)8 = -.587 CPI-MD (Honest condition) 25.20 7.92 25.41 7.00 25.36 7.20 (for comparative purposes) a The pooled-gender correlation was obtained between the CPI-Cp and CPI-MD scale scores with scores on both scales mean-deviated by gender, to scale out mean differences. A pooled-gender correlation provided a more stable estimate and was preferred over separate-gender correlations. 66 Appendix G: Correlations of Adjusted and Unadjusted CPI-Cp Scale Scores with Scores on the NEO PI-R Domain and Facet Scales Males (n = 98) Females (n = 222) NEO PI-R Scale Correlated with Unadjusted Adjusted Unadjusted Adjusted CPI-Cp CPI-Cp CPI-Cp CPI-Cp CPI-Cp Domain Scales Neuroticism .17 .05 .45* .30* Extraversion .39* .58* .20* .39* Openness .19 .23 .16 .19* Agreeableness -.33* -.35* -.44* -.45* Conscientiousness -.38* -.24 -.60* -.44* Neuroticism Facet Scales Anxiety .11 .05 .21* .12 Angry Hostility .28* .22 .27* .42* Depression .05 -.03 .34* .21* Self-Consciousness -.06 -.18 .15 -.01 Impulsiveness .37* .30* .43* .34* Vulnerability .01 -.11 .41* .26* Extraversion Facet Scales Warmth .18 .30* -.04 .10 Gregariousness .37* .49* .20* .31* Assertiveness .27* .48* .09 .34* Activity .24 .36* .05 .20* Excitement-Seeking .44* .48* .43* .45* Positive Emotions .19 .31* .08 .22* Openness Facet Scales Fantasy .32* .30* .32* .28* Aesthetics .11 .15 .09 .14 Feelings .26* .32* .18* .23* Actions .13 .13 .11 .14 Ideas .04 .09 -.09 -.05 Values -.01 -.03 .02 -.01 Agreeableness Facet Scales Trust -.04 -.05 -.24* -.23* Modesty -.51* -.53* -.45* -.45* Compliance -.00 .06 -.31* -.23* Altruism -.33* -.36* -.40* -.43* Straightforwardness -.25 -.32* -.14 -.24* Tender-mindedness -.14 -.18 -.15* .-16 (table continues) 67 Appendix G (Continued) Males (n = 98) Females (n = 222) NEO PI-R Scale Correlated with Unadjusted Adjusted Unadjusted Adjusted CPI-Cp CPI-Cp CPI-Cp CPI-Cp CPI-Cp Conscientiousness Facet Scales Competence -.02 .07 -.43* -.28* Order -.18 -.12 -.31* -.17* Dutifiilness -.37* -.29* -.52* -.41* Achievement Striving -.15 -.01 -.37* -.22* Self-Discipline -.29* -.15 -.53* -.41* Deliberation -.51* -.47* -.59* -.52* *p < .01 (non-directional). 68 Appendix H - Criterion-Related Validity Analyses for Adjusted CPI-Cp Scores with Adjustments Made on the Basis of the CPI Fake-Good Scale A motivational-distortion adjustment equation for CPI-Cp scale scores was developed on the basis of the CPI-FG scale scores. Using this set of adjusted CPI-Cp scale scores, the criterion-related validity of the CPI-Cp scale was re-assessed and compared with the results presented in the body of the thesis in which adjustments were based on the CPI-MD scale. This additional analysis provides a comparison of the two motivational distortion scales in terms of their relative effectiveness in improving criterion-related validity. Development of the Adjustment Equation Using an identical developmental procedure to that described in Study 3, a regression equation was constructed for predicting M D criterion scores from the CPI-FG scale scores. On the basis of pooled-gender data from U B C Sample 4 (« = 250), the mean and standard deviation for the CPI-FG scale were 56.88 and 4.18 respectively; the mean and standard deviation for the M D criterion were 11.95 and 10.87; the correlation between the M D criterion and CPI-FG scale scores was .365. With these values, the raw score regression equation for predicting the M D criterion from CPI-FG scale scores is: Y = .949X - 42.024, where Y and X represent the predicted M D criterion score and the CPI-FG scale score, respectively. As Y is an indication of the amount of motivational distortion within a set of CPI data obtained under employment-conditions, adding these predicted scores to the original CPI-Cp scale scores would render a more honest estimate of the latter. Therefore, the adjustment equation is: Adjusted CPI-Cp = Original CPI-Cp + .949 (CPI-FG scale score) - 42.024. Criterion-Related Validity Analyses Using the adjustment equation above, CPI-Cp scores were corrected for motivational distortion and using these adjusted scores (FG-adjusted CPI-Cp), the criterion-related validity of 69 the CPI-Cp scale was reassessed in the two employee samples identified in the main paper. The correlations between the FG-adjusted CPI-Cp scores and related job-performance criteria appear in Table HI , along with the bivariate criterion correlations obtained using unadjusted CPI-CP scores and the previous set of adjusted CPI-Cp scores (CPI-Cp scale scores adjusted for motivational distortion using the CPI-MD scale; MD-adjusted CPI-Cp). Insert Table HI about here For the sample of telephone service representatives, the FG-adjusted CPI-Cp scores showed a significant increase in criterion correlations for the Desire to Improve [7(76) = 1.804, p < .05] and Overall Performance criteria [t(16) = 2.275, p < .025], when compared to unadjusted CPI-Cp scores. These results are similar to that obtained using MD-adjusted CPI-Cp scores. For the armored-car employees sample, FG-adjusted CPI-Cp scores did not show any change in the bivariate criterion correlations when compared with unadjusted scores. This contrasts with M D -adjusted scores which showed a slight, albeit non-significant, increase in criterion correlations. When semi-partial correlations are compared, no differences between unadjusted, MD-adjusted and FG-adjusted CPI-Cp scores were observed. Overall, the pattern of criterion correlations suggest that criterion-related validity was increased when CPI-Cp scores were adjusted for motivational distortion and that the magnitude of increase tended to be relatively larger for the MD-adjusted scores when compared to the FG-adjusted scores, although the differential improvements between the two methods of adjustments were not statistically significant at the .05 level. More research is needed, using larger data samples, to provide more conclusive results. 70 Table H I Correlations between the CPI-Cp Scale Scores (With and Without Two Different Adjustments for Motivational-Distortion) and Counterproductive-Behavior Criteria in Two Non-Management Level Samples 1. Telephone service representatives (« = 79, combined genders) Counterproductive-Behavior Criterion Predictor Scale Score Availability Desire to Use of Time Overall for Work Improve Performance Original CPI-Cp -.23* -.27** -.26* -.23* MD-Adjusted CPI-Cp a _ 26* — 37*** _ 29** — 36*** FG-Adjusted CPI-Cp b -.25* -.36*** -.26* -.34** 2. Armored-car company employees (n = 91, combined genders) Counterproductive-Behavior Criterion Predictor Scale Score Bivariate correlation Semi-partial correlation0 Original CPI-Cp .25* .26* MD-Adjusted CPI-Cp a .28** .25** FG-Adjusted CPI-Cp b .25* .25* Note: *p < .05; **p < .01; ***p < .001 (all directional). Criterion correlations for the unadjusted CPI-Cp scores are taken from Hakstian et al. (2002). aThe MD-adjusted CPI-Cp scale refers to CPI-Cp scale scores that have been adjusted for motivational distortion using the adjustment equation obtained on the basis of the CPI-MD scale. bThe FG-adjusted CPI-Cp scale refers to CPI-Cp scale scores that have been adjusted for motivational distortion using the adjustment equation obtained on the basis of the CPI manual's fake-good scale. °Length of tenure was partialed out of the criterion scores in the CPI-Cp (adjusted and unadjusted) vs. criterion correlation since scores on this criterion were a function of the number of counterproductive work incidents recorded in employee's files over the tenure of their employment (see text for more detail). 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0090735/manifest

Comment

Related Items