Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A Study of factors affecting clinical performance grades of nursing students Kruger, Mary Boghos 1980

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
831-UBC_1980_A5_7 K78.pdf [ 4.78MB ]
Metadata
JSON: 831-1.0100219.json
JSON-LD: 831-1.0100219-ld.json
RDF/XML (Pretty): 831-1.0100219-rdf.xml
RDF/JSON: 831-1.0100219-rdf.json
Turtle: 831-1.0100219-turtle.txt
N-Triples: 831-1.0100219-rdf-ntriples.txt
Original Record: 831-1.0100219-source.json
Full Text
831-1.0100219-fulltext.txt
Citation
831-1.0100219.ris

Full Text

C- I A STUDY OF FACTORS AFFECTING CL IN ICAL PERFORMANCE GRADES OF N U R S I N G STUDENTS MARY BOGHOS KRUGER B . S . N . , University of British Columbia, 1973 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN NURS ING in THE FACULTY OF GRADUATE STUDIES (School of Nursing) We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA February 1980 © Mary Boghos Kruger, 1980 \ In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of Br it ish Columbia, I agree that the Library shall make i t freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the Head of my Department or by his representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. The University of Brit ish Columbia 2075 Wesbrook Place Vancouver, Canada V6T 1W5 Department of D E - 6 B P 7 5 - 5 1 1 E ABSTRACT A study of factors a f f ec t i ng c l i n i c a l per formance grades of nursing students This study i nvo l ved an inves t iga t ion of factors a f f ec t i ng c l i n i c a l per formance grades of f irst and second year nursing students in a two -yea r D ip l oma program. The purpose of the study was to increase know ledge of factors wh i c h may cor re la te w i t h c l i n i c a l performance of nursing students; re la ted goals were to assist nurse educators in p red i c t i ng student c l i n i c a l performance and p rov id ing gu idance a c c o r d i n g l y . The study was p lanned to test the f o l l ow ing hypotheses: 1 . G rades in Nu r s i n g , B i o l ogy , and Psycho logy courses dur ing the f irst semester of the nursing program accoun t for a s i gn i f i c an t proport ion of the va r i ance in c l i n i c a l performance scores of students; 2. Comp l e x i t y of the nursing s i tua t ion accounts for an add i t i ona l s i gn i f i c an t proport ion of the va r i ance in c l i n i c a l performance scores of students. The study popu la t i on consisted of 59 f irst and second year nursing students. The dependent v a r i ab l e was the average of the c l i n i c a l per formance scores of the students g i v en by the c l i n i c a l instructors (who wrote and scored the anecdo ta l records) and two nursing judges (who scored the wr i t ten anecdo ta l records) . The independent va r iab les were the grades in first semester Nu r s i n g , B i o l ogy , and Psycho logy; and the comp l ex i t y of the nursing s i t ua t i on . i i Data for the study were collected over a three-week period. This phase included: (a) writing of anecdotal records of students' clinical performance by their instructors, and subsequently scoring of these records by the clinical instructor and two nursing judges using the scoring instrument; (b) assessing the level of complexity of the students' cl inical assignments by their cl inical instructors, using the complexity of the nursing situation instrument; and (c) obtaining the students' first semester Nursing theory, Psychology, and Biology grades from their permanent records. The clinical instructors and the nursing judges were trained in the writing and scoring of anecdotal records, and in determining the complexity of the students' assignments by utilizing a 10 minute videotape prepared by the investigator depicting the performance of a nursing student in a simulated cl inical situation. Analyses of the data were carried out in two phases. Phase one included product moment correlational analysis and general izabi I ity analysis to determine the reliability of the clinical scores. Phase two included stepwise multiple regression analysis to determine the predictors of cl inical scores. The findings of the study showed that the inter-rater correlations among the cl inical instructors and two nursing judges were moderately strong (.58 to .84). Likewise generalizability analysis showed that the clinical scores were reliable. The data on complexity of the nursing situation showed very little variability in each of the four semesters and were therefore deleted from the regression analysis. The regression analysis showed that in Semesters II and III 51 and 76 percent of the variance in clinical performance scores could be accounted for by the grades iii in first semester Nursing, Psychology, and Biology. However, in Semesters IV and V the amount of variance accounted for by the independent variables was not significant. Thus, hypothesis 1 is accepted in the case of Semester II and III students, and rejected in the case of Semester IV and V students. iv TABLE O F C O N T E N T S Page ABSTRACT i i LIST O F TABLES x LIST O F F IGURES x i i A C K N O W L E D G E M E N T S x i i i C H A P T E R I. I N T R O D U C T I O N 1 Purpose of the Study 5 Statement of the Problem 6 Hypotheses 6 Va r i ab l e s 6 S i g n i f i c a n c e o f the Study 7 Ope r a t i o n a l De f in i t i ons 9 Assumptions 11 II . REVIEW O F RELATED LITERATURE 12 Introduct ion 12 Educa t iona l Eva lua t i on 12 C l i n i c a l Performance Eva lua t i on Versus Classroom Eva lua t i on 15 Obse r va t i ona l Techn iques in Eva lua t i on 21 Anecdo t a l Record Techn ique of Eva lua t i on 22 v C H A P T E R P a 9 e Pred i c t i on of A c a d e m i c Performance Versus C l i n i c a l Performance 25 The C r i t e r i on Problem 25 P red i c t i ve Studies 26 Summary 29 III. RESEARCH M E T H O D O L O G Y 31 Introduct ion 31 Sampl ing 31 Inst i tut ion 31 Semesters 32 Weeks 32 Instructors 33 Students . 34 Semester II sample 34 Semester III sample 35 Semester IV sample 35 Semester V sample 35 Con f i d en t i a l i t y 36 Instrumentat ion 36 Comp l e x i t y Sca l e 36 Ane cdo t a l Record 37 v i CHAPTER Page Scoring system 37 Procedure. . . 38 Development of Videotape 38 The Training of C l i n i c a l Instructors and Judges 39 Data Col lect ion 40 Method of Analysis 41 Phase One 41 Phase Two 42 IV. RESULTS OF ANALYS IS 44 Complexity of the Nursing Situation 44 Inter-Rater Reliabil it ies of C l i n i c a l Scores 45 Correlational Analyses 45 Semester II 45 Semester III 45 Semester IV 45 Semester V 45 Genera l i zab i l i ty Analyses 50 Semester II 50 Semester III 50 Semester V 54 Correlation Coefficients Versus Genera l i zab i l i ty Coefficients 54 Regression Analyses 57 v i i CHAPTER Page P re l im inary Ana lyses: Raw C l i n i c a l Scores Versus S tandard i zed C l i n i c a l Scores 57 Semester II 57 Semester III 57 F i na l Ana lyses : Raw C l i n i c a l Scores 60 Semester II 60 Semester III i . . . . 62 Semester IV 64 Semester V 64 V . S U M M A R Y , F I N D I N G S A N D C O N C L U S I O N S , L I M I T A T I O N S , A N D R E C O M M E N D A T I O N S 68 F ind ings and Conc lus ions 71 L imi tat ions of the Study 76 Recommendat ions 77 REFERENCES 79 A P P E N D I C E S 83 A P P E N D I X A : Letter Requesting C o l l e g e Permission 84 A P P E N D I X B: Memo G r an t i n g C o l l e g e Permission 86 A P P E N D I X C: Exp lanat ions to Instructors 88 A P P E N D I X D: Consent Form (Instructors) 91 A P P E N D I X E; Exp lanat ions to Students 93 A P P E N D I X F: Consent Form (Students) 95 v i i i CHAPTER Page APPENDIX G: Complexity of the Nursing Situation 97 APPENDIX H: Anecdotal Record 102 APPENDIX I: Scoring System 104 APPENDIX J: Generalizability Analysis 112 APPENDIX K: Standardization Procedure 116 APPENDIX L: Stepwise Multiple Regression Analysis 119 LIST O F TABLES TABLE Page 1 Mean s , Standard Dev i a t i on s , and Intercorre lat ions Among Nurs ing Judges (J) in Semester II 46 2 Mean s , Standard Dev i a t i on s , and Intercorre lat ions Among Nurs ing Judges (J) in Semester III 47 3 Mean s , Standard Dev i a t i on s , and Intercorre lat ions Among Nurs ing Judges (J) in Semester IV 48 4 Mean s , Standard Dev i a t i on s , and Intercorre lat ions Among Nurs ing Judges (J) in Semester V 49 5 Ana lys i s of V a r i a n c e of C l i n i c a l Scores of Semester II Students: Effects of Times on Judges 51 6 V a r i a n c e Components Con t r i bu t i ng to Un iverse Score V a r i a n c e and Expected Obse rved Score V a r i a n c e of Judges and Week s , and G e n e r a l i z a b i l i t y Coe f f i c i en t s of Semester II 52 7. Ana l y s i s of V a r i a n c e of C l i n i c a l Scores of Semester III Students: Effects of Times on Judges 53 8 Ana l y s i s of V a r i a n c e o f C l i n i c a l Scores o f Semester V Students: Effects of Times on Judges 55 9 A Compar i son of G e n e r a l i z a b i l i t y Coe f f i c i en t s and M e a n Co r r e l a t i on Coe f f i c i en t s Across Judges in Semesters II , III, and V 56 10 Summary of Stepwise M u l t i p l e Regression Ana l y s i s o f Week 3 of Semester II: A Compar ison of Raw C l i n i c a l Scores to S tandard i zed C l i n i c a l Scores 58 11 Summary of Stepwise M u l t i p l e Regression Ana lys i s of Week 3 o f Semester III: A Compar i son of Raw C l i n i c a l Scores to S tandard i zed C l i n i c a l Scores 59 12 Summary of Stepwise M u l t i p l e Regression Ana lyses : Semester II . . 61 x TABLE 13 Summary of Stepwise Multiple 14 Summary of Stepwise Multiple 15 Summary of Stepwise Multiple Page Regression Analyses: Semester III . . 63 Regression Analyses: Semester IV . . 65 Regression Analyses: Semester V . . 66 xi LIST OF FIGURES FIGURE Page 1 Standardization Procedure 118 xii ACKNOWLEDGEMENTS I would like to express my gratitude to the administrators, nursing department chairperson, members of the audiovisual department, participating nursing instructors, and students of Semesters II, III, IV, and V of Vancouver Community College, Langara, for making this study possible. To the members of my thesis committee—Mrs. Sheila Stanton, the chairperson; Dr. Margaret Campbell; Dr. Todd Rogers; and Dr. Robert Conry— I wish to express my sincerest gratitude for the support, guidance, and expertise that they provided to me throughout the conception and execution of the study. Most particularly, I wish to thank my husband for his patience, understanding and encouragement. xiii Chapte r I I N T R O D U C T I O N Eva lua t i on of students in nursing educa t i on programs and attempts to i den t i f y factors wh i c h a f fec t the i r c l i n i c a l per formance con t inue to be comp lex and cha l l e ng i ng tasks. There are a number of reported nursing studies in wh i c h the ob j e c t i v e was to ident i f y factors assoc iated w i th successfu l comp le t i on of a program of s tudy . Fewer studies have focussed on i den t i f y i ng predictors o f c l i n i c a l pe r fo rmance . The la t ter is the focus for the present s tudy . It w i l l attempt to ident i f y a cadem i c and s i tua t iona l factors assoc iated w i t h successfu l c l i n i c a l pe r fo rmance . Tay l o r , N a h m , Q u i n n , Harms, M u l a i k , and M u l a i k (1963), in a comprehens ive r ev i ew of research in the area of s e l e c t i on and recru i tment of nursing students, made this statement; A l t hough i t is useful to know how w e l l a test works for groups in g ene r a l , the more important quest ion is how w e l l i t works for a pa r t i cu l a r group in the s i tua t ion under cons i de ra t i on . Howeve r , the most important quest ion may be: How w e l l . w i l l the graduate nurse perform on the j o b ? The present test batter ies were not des igned or v a l i d a t ed to answer this que s t i on . The few studies that have attempted to use ce r t a i n test scores to p red i c t success on the job have shown that there is no co r re l a t i on between these test performances and per formance on the j ob . He re , t h en , is a w i d e - open area that u rgent l y needs to be inves t iga ted (p. 17) . 1 2 They further expounded that no one test or test battery can be specified as the best predictor for all nursing schools, and that any test will vary in usefulness from school to school and from year to year. Consequently, they suggested that each school should evaluate its own selection program. In summarizing selection and recruitment measures, Taylor et al. (1963) stated that grade point average and other tests of cognitive abilities appeared to be the best predictors of academic success in nursing theory courses. However, grade point average, personality, and attitude tests have not been useful in predicting success in the practical aspects of nursing education or on-the-job performance. These authors have concluded that there is a great need for research utilizing multiple variables of achievement, aptitude, personality characteristics, motivation, interests, and other types of predictors. In 1965, Taylor et al. reported research on the related problem of attrition, including this statement: Further research in this area is sorely needed because of the high cost, both to the individuals and institutions of attrition in nursing schools and turnover among nursing personnel. Estimates of the cost of recruiting, selection, inducting, and training one individual for different occupations in the world of work range from hundreds to thousands of dollars to the organization, not counting the loss of a person in the training spot or the headaches of repeating the entire cycle to get a replacement into the status of a full-fledged worker. There is also an unestimated psychological and financial cost to the individual in terms of loss of time and effort and the personal dis-appointment and frustrations when one has an unsuccessful experience and perhaps even finds himself in the wrong field (pp. 52-53). The findings of two more recent studies (Goza, 1970; Gore, 1973) have supported the statements quoted above and, more importantly, have provided new findings comparing dropouts and graduates with favorable performance evaluation. 3 G o z a deve l oped prof i les of dropouts , g raduates , graduates who passed the State Board Test Pool Examinat ion (SBTPE) in nursing on the i n i t i a l a t tempt , graduates who d id not pass SBTPE, graduates who r e ce i ved a f avo rab le work performance e v a l u a t i o n , and a sample of assoc iate degree nursing students. He found that graduates had s i gn i f i c an t l y h igher means than the dropouts on the f i na l grade po int average and on the Respons ib i l i ty s ca l e of the Go rdon Personal P r o f i l e . L i k ew i s e , the graduates who passed the SBTPE had means s i gn i f i c an t l y h igher than the SBTPE fa i lures on Ame r i c an C o l l e g e Test ing Program, C a l i f o r n i a Test of Men t a l Ma t u r i t y , f i na l G P A , a l l 14 of the N a t i o n a l League for Nurs ing A ch i e vemen t tests, the S o c i a b i l i t y s ca l e o f the G o r d o n Personal P r o f i l e , and the O r i g i n a l Th ink i ng sca l e of Go r don Personal Inventory . He a lso found an interest ing d i f f e rence between means of graduates who r e ce i ved a favorab le work performance eva l ua t i on and of those who d id not on the A s cendency sca le of the Go r don Personal P ro f i l e and the V i g o r s ca l e of Go r don Personal Inventory . G o z a conc l uded that the reason for minimum a cadem i c success of dropouts was due to the l ack o f responsib le a t t i t ude , rather than a d e f i c i e n c y in a b i l i t y ; and that unfavorab le work performance eva l ua t i on r e ce i ved by some of the graduates appeared to be the result of persona l i t y d i f fe rences rather than d i f fe rences in a b i l i t y . Research done by G o r e (1973), who deve l oped p red i c t i on equat ions for three assoc ia te degree nursing programs, showed that p red i c t i on equat ions cou l d not be gene r a l i z ed from one ins t i tu t ion to ano ther . He therefore recommended that each nursing schoo l should deve l op the i r own p r ed i c t i v e c r i t e r i a , rather than 4 using common ones. Goza's findings also revealed that students who were predicted to fail in one nursing program may well be predicted to pass in another nursing program. He concluded that this should be a crucial factor in the student's choice of nursing program. The findings of Goza and Gore support Taylor et al.'s advice that each nursing school determine its own selection and recruitment criteria, so that they can adequately counsel their prospective applicants and provide guidance through-out their training program. Another factor which may affect clinical performance is complexity of the student's assignment. Traditionally, patient assignments of graduate nurses were determined by the number of nursing activities/tasks to be performed by the nurse in a given period of time, and the nature or complexity of these activities. O f the many variables that affect quality patient care, the designers of quality assurance programs in nursing have identified complexity of nursing activities as a major factor contributing to the provision of quality nursing care to individual patients. The proponents of quality patient care programs believe that adequate staffing of nursing units—staffing based on the number of patients in a unit, number of nursing activities and the complexity of these activities—directly influences the type of care given to the patients in these units. If the clinical performance of the graduate nurse is influenced by the number of patients she is assigned to, which in turn is based upon the number of activities she has to perform and their level of complexity, then the performance of the student nurse will also be influenced by the type of patient assignment she is given. 5 O f the studies done on predictors of c l i n i c a l pe r fo rmance , there was no ment ion of c omp l e x i t y of the nursing ass ignment/s i tuat ion or how it may i n f l uence per fo rmance . Howeve r , on numerous occas i ons , the invest igator has noted that staf f , instructors, and students suggest that comp lex assignments are the cause of "poor " pe r fo rmance . This hypothesis merits i n ve s t i ga t i on , to ascer ta in whether , and to what ex t en t , s i tua t iona l c omp l ex i t y af fects c l i n i c a l pe r fo rmance . M a n y factors i n f l uence a nursing student 's successfu l comp le t i on of a nursing p rogram. In order to i den t i f y some o f the factors w h i c h may a f f ec t a nursing student 's c l i n i c a l pe r fo rmance , this study w i l l i den t i f y the extent to wh i c h a cadem i c grades in f irst semester N u r s i n g , B i o l ogy , and Psycho logy can be used to p red i c t the c l i n i c a l per formance of nursing students in a two - yea r communi ty c o l l e g e nursing program. A l s o , the c omp l e x i t y of the student 's assignment w i l l be stud ied to determine the extent to wh i c h this v a r i ab l e af fects c l i n i c a l pe r fo rm-ance g rades . The comp l ex i t y of the nursing a c t i v i t i e s was added because recent studies have i den t i f i ed i t as a major factor con t r i bu t ing to the nurse's per formance and hence to qua l i t y pat ient c a r e . Purpose o f the Study The purpose of this study is to increase know ledge of the factors wh i c h may cor re la te w i t h c l i n i c a l per formance of nursing students; re la ted goals are to assist nurse educators in p red i c t i ng student performance and p rov id ing g u i d a n c e . 6 Statement of the Problem S p e c i f i c a l l y , this study w i l l de termine the degree to w h i c h (a) a c adem i c grades and (b) c omp l e x i t y o f the nursing s i tua t ion e xp l a i n scores in c l i n i c a l per formance for f irst and second yea r nursing students. Hypotheses 1 . G rades in N u r s i n g , B i o l ogy , and Psycho logy courses dur ing the f irst semester of the nursing program accoun t for a s i gn i f i c an t proport ion of the v a r i a n ce in c l i n i c a l per formance scores of students. 2 . Comp l e x i t y o f the nursing s i tua t ion accounts for an add i t i ona l s i gn i f i c an t proport ion of the va r i an ce in c l i n i c a l per formance scores of students. Va r i ab l e s The dependent v a r i ab l e emp loyed in this research was the c l i n i c a l per formance score: the average of marks g i v en by the c l i n i c a l instructor (who wrote and scored the anecdo ta l records of the student 's c l i n i c a l per formance dur ing the weeks o f January 30 , and February 6 and 1 3 , 1978) , and the scores g i v en by two nursing judges (instructors w i t h a work ing know ledge of c l i n i c a l i n s t ruc t i on , who eva lua ted the student's per formance using the anecdo ta l records wr i t ten by the c l i n i c a l instructors) from that semester. The independent va r i ab les used in the p red i c t i on were: 1 . the grades in the f irst semester Nu r s i n g , B i o l ogy , and Psycho logy courses; 2 . the comp lex i t y of the nursing s i tua t ion as assessed by the c l i n i c a l ins t ructor . S i gn i f i c an ce of the Study M a n y of the p red i c t i on studies r ev i ewed have focussed on success in nursing schoo l as the va r i ab l e of c o n c e r n . Howeve r , on l y a few studies have inves t iga ted the re la t ionsh ip between theory scores and c l i n i c a l pe r fo rmance , wh i c h is more re levant to the u l t imate goa l of prepar ing the nurse to perform success fu l l y in a va r i e t y of c l i n i c a l set t ings . Furthermore, a l though the few studies wh i ch have studied a cadem i c a b i l i t y as a p red i c to r of grades in c l i n i c a l per formance have found nega t i ve or weak pos i t i ve re la t i onsh ips , they d id not assess (1) the r e l i a b i l i t y of the c l i n i c a l performance grades or (2) the r e l a t i v e cont r ibut ions of the compulsory cou r se s—Nurs i ng theory , B i o l ogy , and Psycho logy-w h i c h are deemed to be theo re t i ca l know ledge prerequ is i te to the prov i s ion of the type o f care that i n d i v i dua l pat ients requi re or request . A r ev i ew of the l i te rature on too ls/methods used to eva lua te c l i n i c a l per formance revea led one study by Rines (1963) that reported "77 percent of the nursing instructors used anecdo ta l records w i th va ry i ng degrees of success" (p. 26 ) . 8 In summary, the need to identify predictors of clinical performance arose from the enduring problem of how best to prepare nurses. This study was initiated as a result of dissatisfaction, expressed by both students and instructors, with clinical evaluation in general and specifically with procedures for assigning clinical grades. Furthermore, an unpublished survey of nursing students' attitudes toward various methods of clinical evaluation (conducted by the investigator in 1976) showed that students generally prefer a combination of several evaluation methods: anecdotal records, simulation, clinical examination, and self evaluation. It was beyond the scope of the present study to examine several evaluation techniques in evaluating clinical performance, and then to use the method(s) which had the best reliability, validity, objectivity, and practicality. Therefore, the anecdotal record method of evaluation was chosen, since it was currently in use to some extent, and based on the belief that it would provide instructors with data to use in the course of clinical teaching and evaluation. Finally, results of this study could help to identify those students in this school who may need considerable counselling and assistance during the course of their programs, and thus enable instructors to take some preventative action and possibly to decrease attrition and the cost of nursing education. 9 Operational Definitions The following terms are used: nursing student - a student who is enrolled in a two-year college nursing program preparatory to nurse registration, nursing program - a six-semester program in a community college leading to a diploma in nursing, clinical area - a hospital ward. clinical objectives - learning objectives to be achieved by the students in the cl inical area. performance - the behaviors of the nursing student in the clinical area, as related to relevant clinical objectives, cl inical instructor - a nursing instructor who teaches, observes, and evaluates the student's performance in the clinical area, performance evaluation - an evaluation of the student's performance to determine whether clinical objectives have been met. nursing judge - a nursing Instructor, with a working knowledge of instruction in any one of Semesters II, III, IV and V of the nursing program, who evaluates the performance of the nursing student using the anecdotal notes written by the clinical instructor, cl inical performance score - the average of the scores given by the clinical instructor and two nursing judges, each of whom evaluated anecdotal notes on the performance of the student during the weeks of January 30, February 6, and February 13, 1978. 10 SCI - scores given by the c l in i ca l instructor. S N J - scores given by the nursing judge. Grade Point Equivalent - the letter grades of students in first semester Nursing, Biology, and Psychology were transformed to numerical grades using the scale below. Letter Grade* Grade Point Equivalent A 4 B 3 C 2 P 1 F 0 complexity of the nursing situation (CNS) - is determined by four variables: (1) the activit ies required of the student in the c l i n i ca l area, including (a) administering treatments prescribed by the patient's physician; (b) administering treatments prescribed by the nursing team; (2) the condition of the patient; (3) the ava i lab i l i ty to the student of appropriate patient experiences; (4) the revision of the student's assignments based on the instructor's assessment of the student's abi l i t ies. *Vancouver Community Co l lege, Langara 1977-1978 Calendar. 11 Assumptions The design and execution of this study rests upon these assumptions: 1 . the clinical objectives of a given semester are decided and agreed upon by the instructors responsible for that semester, and hence represent their expectati of the students enrolled in that semester. 2. the clinical instructor can validly and reliably judge the complexity of the nursing situation. Chapter II REVIEW OF RELATED LITERATURE Introduction Adequate evaluation and adequate prediction of the students' clinical performance are problems that nursing education shares with other cognate disciplines such as medical education and dental education. The process of clinical evaluation requires observational techniques for collecting data in various clinical settings. This very process, although used extensively for decades, has created problems for the educators and the students alike. Educational Evaluation Gronlund (1976) defined evaluation as a "systematic process of determining the extent to which educational objectives are achieved by pupils" (p. 6). He distinguished between evaluation and measurement, stating, "Evaluation is a much more comprehensive and inclusive term than measurement" (p. 6), which includes both quantitative and qulitative descriptions of pupil behavior, plus value judgments concerning the desirability of that behavior. Similarly Gordon (1960) described evaluation as". . . the measure of the extent to which the performance achieves certain desired goals" (p. 364). 12 13 In describing evaluation of students, MacKay (1974) stated that ". . .we must view evaluation as a means and not an end, as an ongoing process with the purpose being promotion of fuller development of the learner" (p. 4). Likewise, Gordon (1960) asserted that "The most important use of an evaluation is to assist the person to improve his performance" (p. 366). In her effort to establish productive evaluation, MacKay (1974) has outlined the following guidelines: 1 . the purpose of evaluation as a means of facilitating learning should be known and accepted by both student and teacher; 2. the standards of evaluation should be known and shared, or goals for behavior based on assessed learning needs should be established; 3. the standards or goals should be realistic; the student should be potentially capable of meeting the objectives for performance normally expected for that level of preparation and his unique idiosyncrasies taken into consideration; and 4. behaviors within a specified time interval only should be evaluated, (p. 5) Using a different perspective, Schweer and Gebbie (1976) stated that the trend in evaluation of clinical nursing is toward an ongoing evaluation program aimed at determining student growth in becoming a skillful practitioner. They also identified purposes that evaluation processes could serve. These are as follows: 1 . to determine the background ability each student brings to the learning situation; 2. to determine the ability of students to use nursing theory as a whole while progressing in the development of clinical skills; 14 3. to determine the progress by each student at frequent intervals during the assignment; 4. to discover learning difficulties of individual students and the group in order to adjust the teaching, including remedial or advanced assignments, as needed; 5. to provide reinforcement of learning for students; 6. to foster development of self-evaluation; 7. to determine the readiness of students to become self-directive; 8. to determine the effectiveness of teaching techniques; and 9. to obtain data for conducting research studies relating to student achievement, (p. 166) Evaluation as described by Gordon (1960), MacKay (1974), and Schweer and Gebbie (1976) suggests that they are using this term to mean both formative and summative evaluation of performance. Formative evaluation helps the student to learn by pointing out his strengths and weaknesses, and summative evaluation measures to what extent the student has met the course objectives. Most students and teachers fail to make the distinction between evaluation and grading. However, the distinction was made by MacKay (1974), who stated that "the assignment of grades requires a further step of measuring the achieved levels of competency in respect to specified behavior characteristics of students in general in differing levels of expertise" (p. 5). To establish educationally sound evaluation programs, Gronlund (1976) and Rines (1963) have proposed the following: 1 . identify and define the objective of the program in terms of desired changes in the behavior of students; 2. determine evaluation tools; 15 3. construct tools which are valid and reliable; 4. decide on utilization of these tools; 5. decide what points in the course will be appropriate to evaluate the student's performance; 6. outline a program of evaluation; 7. plan and select situations to show desired change of behavior; 8. collect the responses; and 9. interpret the results. The above program of evaluation indicates that the specific activities of the educator related to evaluation are also a part of the teaching process, since they help in defining the objectives, and in the planning and selecting of situations to show the desired behavior of the student. Although the interdependence of teaching and evaluation is recognized and accepted by nurse educators, Rines (1963) makes a strong point that If evaluation is to be conducted with maximum efficiency and results, regular times should be designated for the purpose, separate from the teaching process . . . [the instructor] should plan her observations and evaluations of student behaviors just as she plans for paper-and-pencil tests and examinations. These should be planned frequently enough . . . so that she [the instructor] cangetan estimate of behavior trends, (pp. 54-55) Clinical Performance Evaluation Versus Classroom Evaluation Most nurse educators agree that the process of evaluation in the clinical area is difficult and complex (cf. NLN Report, 1972; Litwack, Sakata, and Wykle, 16 1972). In the clinical area, the student is learning to apply knowledge acquired in the classroom and laboratory. Moreover, as Wood (1972) has suggested, "The student is learning behavioral more than intellectual skills" (p. 336). Most nurse educators will also agree that the achievement of clinical objectives is poorly measured by conventional classroom tests, whether teacher-made or standardized. Furthermore, clinical experiences are not uniform—not all students will have the opportunity to care for the same patient with a given problem. Therefore, to the extent that each individual and his reactions are unique, the clinical experiences of the students will vary greatly, and thus make standard teaching and evaluation almost impossible. Another difference between clinical and classroom evaluation is that various individuals in the clinical area add input and thus affect the student's behavior. It is therefore crucial that the input of these individuals—patients, nursing staff, and clinical instructors—be utilized for evaluation of the student's performance. Taylor et al. (1965) have pointed out that "the lack of relationship between performance in theory courses and clinical practice courses may point up a deficiency in traditional methods of evaluating students" (p. 38). To explain the above, they have quoted the following shortcoming in nursing education initially described by McManus (1949): many of those who teach nurses . . . have had no formal training as teachers and therefore tend to follow the patterns and teaching they themselves have experienced. They are not aware of the need to develop the higher mental abilities in this field as in other professions. Only exceptional teachers and students manage to free themselves from traditional patterns of teaching and learning. 17 Among the nursing educators who have had some preparation for teaching are many who still erroneously use instructional methods which demand the exercise of little more than the lower mental processes of memorization and recall. Frequently facts are taught and facts are learned in catechismal fashion. Testing and grading systems tend to put a premium upon fact-getting and rote-learning of subject matter. Skills are frequently taught exclusively in the classroom, with drill on specific procedures. Although the students spend a considerable portion of their time in the clinical nursing practice, the theory is likely to remain sterile and compartmentalized and will rarely, and only accidentally transfer unaided to give the practice meaning. . . . (pp. 38-39) Lancaster (1972) in her Opinion Survey of nurse teachers in Scotland has addressed the question of whether the present tutors' courses give the kind of preparation which will enable future nurse teachers to cope effectively with the professional demands made upon them. The kind of work done in the past . . . in the traditional type of nursing school hardly seems to be a realistic basis for decisions regarding the preparation and qualifications appropriate to nurse tutors in the future, (p. 5) Lancaster stated that "the answer to this question depends on the definition of the tutor's role" (p. 5). She has suggested that the role of the teacher depends on many things, such as the type of students, the type of subjects to be taught, the attitudes and values of the educational system, and the teaching-learning environment. Furthermore, she has concluded that the preparation of teachers is crucial in the development and maintenance of effective nursing practice. But, the role of the teacher must first be explicitly defined so that criteria may be developed to assess the teacher training programs. A quarter century later, Infante (1975) and Miller (1976) expressed the very same concerns about the shortcomings in nursing education as McManus (1949). Commenting on the preparation of nurse educators, Sister Miller (1976) 18 stated that "most graduates of master's programs become teachers immediately after graduation, but has their education prepared them for this role?" (p. 752). She found out that only 21 of the 64 graduate nursing programs offer preparation in teaching. Infante (1975) stated emphatically that nurse educators should be educated in strategies of clinical teaching: Teachers in graduate schools who are preparing teachers for under-graduate programs in nursing might place more emphasis on strategies of teaching in the clinical laboratory. Strategies for teaching in the clinical laboratory, the heart of professional education, have been noticeably absent from course outlines. Perhaps increased opportunity for future teachers to apply theory to practice under guidance is also needed. Just as changes in the service of a professional practicum must start with the educational program of which he is a product, so changes in the type of teaching done by teachers . . . must start with changes in the educational programs that prepare these teachers, (p. 60) Similarly, Miller (1976) stated: The best method to help [graduate] students to internalize the principles of learning, planning, and evaluation, along with the philosophy of education and the act of teaching, so that they may function effectively as teachers of nursing . . . is for those teaching graduate students . . . to be role models who facilitate the learning process for the student, (p. 752) This lack of educational programs in clinical teaching and evaluation has made nurse educators vulnerable, and has encouraged the adoption of evaluation tools and practices to which they themselves were subjected as students. These evaluation practices, although not educationally sound, could prove to be very efficient and effective. However, unless clinical evaluation research endorses such practices, nurse educators must not assume that clinical grades reflect the students' true performance. This viewpoint has been asserted 19 by Champney, Chatfield, and Sims (1975) who stated that it is dangerous to adopt scales and objectives unless the team of instructors who have to use them are in complete agreement. Tate (1962) stated, "the more complex the behavior to be evaluated, the more difficult the task of building an evaluation instrument" (p. 36). Another area which has influenced clinical performance grades of nursing students is team teaching. Various approaches to this technique were described by Schweer and Gebbie (1976) as having (1) all clinical teachers work together as a team in supervising the student, (2) certain portions of the program taught by a master clinical teacher, with other clinical teachers super-vising groups of students, and (3) specialists in the field of nursing contribute to the clinical teaching and evaluation. These various approaches seem to have created inconsistencies among evaluators of clinical performance, because of (1) lack of communication among clinical instructors, (2) lack of training of these instructors relative to evaluation and methodology, and (3) lack of adequate time for instructors in the team to discuss clinical objectives, and the behaviors that the student is expected to demonstrate to successfully meet the objectives. Hayter (1973) demonstrated that the effects of the difficulties created by team teaching, although they cannot be avoided entirely, could be diminished if each instructor in the team understood the objectives and the specific behaviors expected of a given level of student. She found that the agreement among 31 evaluators as to the grades of three students was 76% after discussing the objectives, as opposed to 44% before the objectives were discussed. 20 In her extensive review of the literature, Hart (1974) found that nursing schools have been concerned with the evaluation of clinical performance of students, and in some instances, have employed ameliorative measures. These measures include involving the faculty in the development of a devised rating scale or check list, the employment of student evaluations, and the identification of behaviors in the clinical setting which are regarded as satisfactory for a given level of student in the nursing curriculum. Finally, the issue of assigning either a letter or a numerical value for clinical performance has perturbed many nurse educators. Rines (1963) wrote on this issue: Human behavior of any description is much too complex to permit fine discriminations . . . we do not know enough about the way in which students learn or what influences learning to say more that [sic] a behavior is acceptable or unacceptable, (p. 17) Hart (1974), in a survey of 105 nurse educators to determine their perception of the significance of criteria and tools employed in the evaluation of clinical performance of baccalaureate students, found that some of the least significant criteria and tools were the use of a devised rating scale and the percentage/letter grade for the course. Historically, says Hart, the letter/ percentage grade was widely used in the evaluation of the students' performance. However, she concluded that educators in baccalaureate programs are cognizant of the many variables in the evaluation of the students' clinical performance which cannot be reduced to a single variable representing a percentage or a letter grade. This finding supports the statement by Rines (1963), that clinical performance is too complex to permit fine discrimination. 21 It is interesting to note the discrepancy between the ameliorative measure—involving the faculty in the development of a devised rating scale or check list as stated by Hart—and her finding of a devised rating scale as the least significant criterion and tool. Observational Techniques in Evaluation Some aspects of learning and development can be evaluated only by direct observation (Ahmann & Glock, 1971; Gronlund, 1976); however, evaluations based on observations have serious limitations. Thorndike and Hagen (1961) have stated that "Evaluation based on naturally occurring situations of life is likely to vary from person to person" (p. 17). Clinical nurse educators continue to receive numerous complaints from the students about discrepancies between instructors or between clinical settings in clinical performance evaluations and grades assigned. To minimize these discrepancies Schweer and Gebbie (1976) concluded that "observation continues to be regarded as an effective means of evaluating nursing performance, but teachers must have a thorough knowledge of the exact behaviors to be accomplished during a given experience, along with an analysis of the level of each performance to be rated according to a standardized scale" (p. 172). Other problems in performance evaluation have been discussed by Champney, Chatfield, and Sims (1975). These authors have suggested that "raters do better if carefully trained with respect to the distribution of abilities and the nature of the scale, and cautioned against common errors" (p. 168). 22 Four rater idiosyncrasies identified by them were: (1) leniency; (2) halo effect; (3) logical error; and (4) central tendency (p. 168). The limitations of direct observation as identified by Ahmann and Glock (1971), Fuerst and Wolff (1969), Gronlund (1976), Rines (1963), and Thorndike and Hagen (1961) were as follows: 1 . the difficulty in determining the significance of an isolated item of behavior; 2. the complexity involved in identifying the meaningful and productive set of behavior categories to observe; 3. the problems arising from the biases of the observer, who may bring his own needs and feelings to the evaluation; 4. the decrease of cogency in observations which arises as the student numbers increase; and 5. the high cost of direct observations. Anecdotal Record Technique of Evaluation The anecdotal record technique is an informal, direct method of recording behavior observed in a natural setting (Thorndike & Hagen, 1961), which provides a large amount of information about the learning and development of the student. Gronlund (1976) stated that "Anecdotal records are factual descriptions of the meaningful incidents and events which the teacher has observed in the lives of his pupils" (p. 429). Similarly, Fivars and Gosnell (1966), and Fuerst and Wolff (1969) have described anecdotal records as informal descriptions of the actual observed behavior and activities of students. 23 The advantages of such anecdotal records, described by Ahmann and Glock (1971), Gronlund (1976), Rines (1963), and Thorndik and Hagen (1961), are as follows: 1. lend themselves well to frequent observations of short duration; 2. make it possible to see trends in behavior over a period of time; 3. provide a description of actual behavior in natural situation; 4. provide a check on other evaluation methods; 5. make it possible to gather evidence on events that are exceptional but significant; 6. increase the observer's awareness; and 7. make instructors more diligent in observations. Like any evaluation tool, the anecdotal record technique has its disadvantages, which are similar to the limitations of direct observational tech-niques used to evaluate behavior in naturally occurring situations of life (Ahmann & Glock, 1971; Gronlund, 1976; Rines, 1963; and Thorndike & Hagen, 1961). The disadvantages include: 1. the time required to maintain the records; 2. the difficulty in being objective when observing and reporting student behavior; 3. the difficulty in obtaining an adequate sample of behavior; 4. the tendency to accumulate unfavorable anecdotes; 5. the difficulty of periodically summarizing the anecdotes into some kind of framework; 6. the tendency to report behavior without describing the setting; 24 7. the tendency to evaluate rather than report behavior; and 8. the tendency to describe behavior in general terms. The suggestions made by Gronlund (1976) and Thorndike and Hagen (1961) for writing proper anecdotal records are: 1 . determine in advance what you want to observe, but be alert for unusual behavior; 2. observe and record enough of the situation to make the behavior meaningful; 3. make a record of the incident as soon after the observation as possible; 4. limit each anecdote to a brief description of a single specific incident; 5. keep the factual description of the incident and the interpretation of it separate; 6. record both positive and negative behaviors; 7. collect a number of anecdotes on a student before drawing inferences concerning typical behavior; 8. obtain practice in writing anecdotal records; 9. teach the observers what to look for; and 10. train the observers in standards of judgment. In summary, anecdotal records provide a systematic procedure for recording observation, but Blood and Budd (1972) state that it does not guarantee that these observations will be systematically made nor directed toward specific relevant behaviors. Two authors, Blood and Budd (1972) and Rines (1963) have suggested that perhaps time sampling systems might provide an opportunity to systematically observe student behavior. 25 Prediction of Academic Performance Versus Clinical Performance The Criterion Problem In a review of research studies and practices dealing with selection, recruitment and prediction, Taylor et al. (1963) found that when academic grades in nursing were used as the criterion, the best predictors—in terms of single highest correlations—were pre-nursing College Grade Point Average (.63), the Otis (.48), and High School Grade Point Average (.43) (p. 33). Another review of research studies by the same authors in 1965, revealed that the best predictors were Otis (.65), American College Test (ACT) total score (.64), and High School Grade Point Average (.51). In 1963, Taylor et al., when using clinical grades as the criterion, found that The three best predictors with highest single correlations again are pre-nursing College Grade Point Average (.59), the Otis (.59), and the High School Grade Point Average (.37). In general, the correlations drop from academic or theory grades to clinical or practice grades indicating that the best predictors of the former criterion are not necessarily the best predictors of the latter criterion and that we have not yet learned how to predict the latter criterion, (p. 33) In their discussion of the tremendous variability in the results with various predictors and criteria for success in nursing schools, Taylor et al. (1963) suggested that variability in results can in part be attributed to the many different ways of measuring the criteria from one study or location to the next. Furthermore, they indicated that the prediction problem may be confounded by the many complex 26 behaviors that are neither recognized in training programs nor in job evaluations as crucial for success at this point in time. These authors state that much more work is needed on the measurement problems involved in developing meaningful, reliable and valid clinical criterion measures for success in nursing schools, and that these criteria once developed should then be carefully related to "on-the-job" criteria for success. Predictive Studies There are very few reported predictive studies on clinical performance in nursing. Ryden (1977) studied the predictive value of a clinical examination of interpersonal relationship skills and subsequent interpersonal competence in clinical situations as measured by ratings of clinical instructors, and obtained a coefficient of correlation .45 (significant at p <.05). Although this is a positive relationship, it is not high, and therefore suggests that there perhaps are other factors which influence performance in the clinical area. In their study of "Measurement and Prediction of Nursing Performance", Taylor et al. (1965) found that the graded performance of students in nursing theory and general education courses was not closely related to their performance in clinical nursing courses. These authors have suggested that perhaps the clinical grades represent measures of additional abilities and behaviors from those measured by academic courses. The shortcomings in nursing education described earlier by Taylor et al. (1965, pp. 38-39) may indicate one reason for nursing theory and clinical practice grades not correlating highly. They have suggested that the lack of relationship 27 between performance in theory and clinical practice courses may point out the undesirable utilization of the deficient traditional methods of evaluating students, especially because of the lack of relationship between classroom and clinical grades. These authors have postulated that this lack of correlation between theory and practice has occurred because: 1 . theory courses have not stressed basic nursing processes which are relevant across patient problems, service settings, and functions; 2. instead of being expected to assume responsibility for acquiring the information needed to deal with a particular problem, students were given the answers; 3. until very recently these courses have been built around clinical areas and/or disease entities rather than patient problems; 4. lack of specificity in criteria by which students have been evaluated as they practice nursing in the clinical units; 5. objectives have not been stated in terms of behaviors that could be easily evaluated; and 6. learning how to practice as a professional is a much more complex learning process than learning of a theory per se or the learning of a patient care procedure, (p. 39) In a study of prediction of success of community college nursing students, Powers (1974) studied the interpersonal competencies of 80 students in their second semester using selected items from the Minnesota Multiphasic Personality Inventory (MMPI), the California Psychological Inventory (CPI), the Strong Vocation Interest Blank (SVIB), and the American College Test (ACT). The results indicated that only two subtests from the four predictor variables produced F values significant at .05 level when correlated with interpersonal competencies of nursing students. 28 These items were "Social Introversion" from the MMPI, which had a multiple correlation coefficient of .1021; and "Good Impression" from the CPI which increased the multiple correlation coefficient to .2182 (p. 56). A review of medical literature revealed results similar to those found in nursing—that is, low correlations between clinical performance grades and academic grades. Korman and Stubblefield (1971) administered a rating scale assessing internship performance and then correlated the results with medical school grades. They found that GPA bore no significant relationship to any of the evaluative components of internship performance. Similarly, Wiener, Koran, Mitchell, Schattner, Fierstein, and Hotchkiss (1976) correlated the scores of medical students for interviewing, physical examinations, and total clinical skills with the scores on the final multiple-choice exam given at the end of clerkship. The respective product moment correlations were .36, .58, and .58. Using a broader sampling of physician performance, Gonnella, Goran, Williamson, and Cotsonas, N., Jr. (1970) and Leaper, G i l l , Staniland, Horrocks, and De Dombal (1973) found little relationship between written examination scores and clinical performance scores. These findings suggest that there is a weak relationship between students' knowledge and the application of that knowledge. However, these investigators did not take into account all the other factors which affect evaluation of clinical performance. 29 The review of the literature did not reveal any information relevant to the alleged relationship of complexity of the nursing situation and poor clinical performance. The strong feelings of students, staff, and instructors on this issue need attention. Summary Research has been sporadic and meager on what constitutes competent clinical performance, on approaches to grading it, on tools to measure it, and on corresponding issues of validity and objectivity. In the few studies which have been completed, findings have been discouraging—correlations between clinical performance grades and academic or psychological tests have ranged from 0 to .65. Furthermore, the correlations have varied from one institution to another and from one sample to another, making it difficult to generalize to the population of nursing students at large. Some of the numerous reasons given by various researchers for these low correlations are: more subjectivity is involved in assigning clinical grades than academic grades; components of satisfactory clinical performance have not yet been identified; training of nurse educators is inadequate in the case of clinical teaching-learning-evaluation processes; there are substantial differences in the grading methods used; there is real variability in the quality of students; there are substantial differences in the curricula; many problems are inherent in the use of observational techniques in the evaluation and the grading of 30 clinical performance; there are no standardized evaluation procedures; problems of communication and consistency among instructors are inherent in team teaching; and traditional methods of clinical teaching and evaluation are outmoded. Only a few of the studies indicated clearly whether the clinical performance grade did indeed comprise the clinical grade or was just one component, along with theory, of the clinical grade. Likewise, very few studies attempted to ascertain the reliability of the so-called clinical grades before studying variables which would predict it. The limited and controversial research on clinical performance evaluation —and specifically the weak relationship between knowledge and its application in real life situations—requires vigorous efforts to identify influential factors and to analyze how grading processes could be improved so as to benefit the individual nurse, her patient, and the nursing profession. Chapter III RESEARCH METHODOLOGY Introduction In order to assess the extent to which first semester grades in Nursing theory, Psychology, and Biology courses, and complexity of the nursing situation were valid predictors of clinical performance of nursing students, a sample of associate degree nursing students was selected. Likewise, a sample of clinical instructors was selected. Sampling Institution Requests to conduct the study at Vancouver Community College, Langara, British Columbia were made to the Dean of Administrative and Student Services (Appendix A), and to the chairperson of the Nursing Department. Each was approached personally and given the research proposal. Written permission to proceed was subsequently received from the Dean of Administrative and Student Services (Appendix B), followed by oral permission from the Nursing Department chairperson. 31 32 Semesters The two-year Nursing Program at Vancouver Community College, Langara is divided into six semesters, each 13 to 14 weeks long. Because students in Semesters I and VI had, respectively, very short and very long clinical experiences, they were deleted from the study; thus, students in the remaining semesters were included. These groups had clinical experience throughout the semester ranging from 8 to 16 hours per week. Weeks Each semester was comprised of two rotations, each six to seven weeks in length. The grading system used by the nursing faculty was such that at the mid-term (end of first rotation), the students received a formative evaluation, and at the end of the semester they received a final clinical grade. Only the anecdotal records written during the first rotation were used in order to eliminate the possible influence of this research on students' grades at the end of the semester. The clinical instructors were asked to write anecdotes about the perform-ance of their students for the fourth, fifth, and sixth weeks of the first rotation, which extended from January 30 to February 13, 1978. These weeks were chosen because (1) the first week of the rotation was orientation, (2) the second week would give the students a chance to establish themselves and allow them to begin to feel comfortable in the clinical area, and (3) the third week would give the instructors a chance to practise writing anecdotal records. Furthermore, the anecdotal records of only three weeks were chosen to keep the task of writing and scoring them reasonable and manageable for the nursing instructors. 33 Instructors Approximately two months prior to the initiation of the study, permission was requested from the Nursing Department chairperson to present the highlights of the study to faculty members in a general meeting. The purposes of this presentation were to ascertain whether interest in the study was sufficient, and to answer questions. The investigator (one of the instructors who was not included in the study) met with the instructors. The proposed study was explained, questions were answered, and feedback was obtained indicating that the majority of faculty were in favor of the study and interested in its results. During the week of January 9, 1978, each instructor from Semesters II, III, IV, and V, was given a sheet explaining the study (Appendix C) and a consent form (Appendix D); they were asked to return the consent form to the investigator by January 13, 1978. Some of the instructors asked for additional information or clarification, which was provided as requested. There were four instructors in Semester II, three of whom consented to participate in the study. Similarly, there were four instructors in Semester III, three of whom consented to participate in the study. One of these instructors consented only to write anecdotes and judge the performance of her own students. Another instructor who was not teaching in Semester III but had taught in it several times, consented to be a judge. Semester IV had four instructors, all of whom consented to participate fully in the study. 34 Finally, Semester V had five instructors, all of whom consented to participate. Also, another instructor who had taught in that semester and was at that time teaching in Semester I, wanted to participate, and consented to be a judge in Semester V. In summary, of 17 instructors in Semesters II, III, IV, and V, 15 consented to write anecdotal records and be judges. Two other instructors who were not teaching in those semesters but had taught in them before, consented to be judges. Students After obtaining the signed consents from the instructors indicating their willingness to participate in the study, the investigator asked the leaders of Semesters II, III, IV, and V respectively for 15 to 20 minutes of their class time during the week of January 16, 1978. In these sessions, the investigator explained the study to the students, and distributed both the explanation (Appendix E) and the consent forms (Appendix F). Students were asked to complete the consent form and return it to the investigator by the end of that week. Furthermore, they were told that if they had any questions regarding any part of the study to contact the investigator. Semester II sample. There were 35 students registered in the clinical nursing course, 30 of whom agreed to participate. The group of students who consented to participate, but whose instructor wished not to participate, were eliminated from the study. Also, to equalize the workload of the three 35 participating instructors, five students were randomly chosen from each group. And of these 15 students, one withdrew from the semester, leaving a final sample of 14 students. Semester III sample. There were 25 students registered in Semester III clinical course, 21 of whom consented to participate. Those students who consented to participate, but whose instructor wished not to participate, were eliminated from the study. Of the remaining 14 students, two were in a specialized clinical area and, therefore, were eliminated from the study. The final sample of Semester III was comprised of 12 students. Semester IV sample. There were 32 students registered in the clinical nursing course of Semester IV, 24 of whom agreed to participate. To equalize the workload of the four instructors, four students were randomly chosen for each instructor, yielding a final sample of 16 students. Semester V sample. Finally, Semester V had 28 students registered in the clinical nursing course, 22 of whom agreed to participate. Two of these students had advanced credits in the independent variables from countries other than Canada and United States. However, the assessed transcripts did not indicate the equivalent college grades of these students. Consequently, it was decided to eliminate them from the study. Also, three students withdrew from the semester. The final sample was comprised of 17 students. In summary, the total sample of students at the end of data collection period was 59: 14 from Semester II, 12 from Semester III, 16 from Semester IV, and 17 from Semester V. 36 Confidentiality To ensure confidentiality, each instructor and student was assigned a number by the investigator. Each anecdotal note was typed before being given to the judges to eliminate effects of handwriting and the identities of instructor and student. Also, the investigator proofread all the anecdotal notes and, where the need arose, names of students were deleted and substituted with appropriate terms such as, "the student", "she", "he", et cetera. Instrumentation Complexity Scale The decision as to who could best determine the complexity of the nursing situation was complicated by the many constraints on time and resources. It was felt that complexity could be most reliably and objectively assessed by a group of observers trained to follow up each student and record minute-to-minute changes in their patients' health status and to observe the students' abilities. A panel of these observers could then judge the complexity of the nursing situation as recorded. However, considering the constraints mentioned above, the investi-gator believed that the best persons to determine the complexity of the students' assignments were their clinical instructors, because they were in a position to be most knowledgeable about the students' assignments and the factors affecting them. A form was then developed (Appendix G) to be used by the clinical instructors for rating the complexity of the nursing situation. This form delineated 37 three levels of complexity, and the instructor was asked to circle one level at the end of each clinical experience. Each of the three levels of complexity was respectively described by four criteria. These were (1) activities of the student; (2) condition of the patient; (3) patient experiences; and (4) revisions of student's assignments based on the instructor's assessments of the student's abilities/performance. Furthermore, the form was accompanied by guidelines for its utilization—these were specific examples which the instructor had to follow to decide on the level of complexity. Definitions of the three levels of complexity were based on clinical experience and on numerous discussions with both instructors and students about the difficulty of clinical assignments. Resources were not sufficient to allow a complete pilot test of the instrument. However, four instructors were consulted; each read it independently and reacted, and the form was refined accordingly. Anecdotal Record The anecdotal record form (Appendix H) was adapted from Rines (1963) following a review of the literature and the assessment of the needs of this study. The following elements were noted: identification data (student and instructor numbers, date and semester); description of the situation; description of the student's behavior; and the performance score given. Scoring system. The purpose of the scoring system was to provide the judges with a uniform measuring device, thereby minimizing scorer subjectivity and increasing reliability of the scores given. 38 The scoring system (Appendix I) developed by the investigator is a Likert-type scale with a range of 1 to 9, representing unsatisfactory performance to superior performance. This scale was further divided into five major categories which were more familiar to the instructors so that there would be an association between the scoring system and the grading system used at the college. This assisted in utilizing the scoring procedure. The five categories were A, B, C, P, and F. Each of these grading categories was further described by a set of six criteria which were thought to be critical in the evaluation of the students' attainment of each of the clinical objectives. Nebulous terminology was defined, so as to provide the judges with a common understanding of the crucial terms and thereby increase objectivity and reliability of the scores. Procedure Development of the Videotape In order to assist clinical instructors in the writing of anecdotal records and the use of these records to judge the clinical performance of the students, training procedures were developed to sharpen observational skills, and to allow practice in the above activities in a simulated setting. The investigator wrote a script for a nine-minute videotape, titled "Evaluation of a Student's Clinical Performance", which has five sections designed to depict the various activities of a nursing student in the clinical laboratory. 39 The five vignettes and their respective lengths were: Vignette Length 1 . The student doing a brief assessment of the 2-j min. patient before breakfast. 2. The student giving subcutaneous injection 2^min. to patient. 3. The student reporting to the team leader. imin. 4. The student and the instructor discussing 2 min. the student's nursing care plan on the patient. 5. The instructor checking the student's l^min. charting on one of the patients. The Training of Clinical Instructors and Judges Separate workshops were scheduled for the instructors of each of the Semesters II, III, IV, and V. The instructors who had consented to participate in the study were notified in writing to attend these workshops on the evaluation of clinical performance. Other instructors who had indicated an interest in the topic were also invited to attend. The purpose of the workshops was to teach instructors how to (1) write anecdotal records according to the sample description of anecdotal records adapted from Rines (1963); (2) score these, employing the scoring system (Appendix I); and (3) determine the complexity of the nursing situation. Each workshop was conducted over a period of three to four hours, during which time the following activities were carried out: (1) reviewing the advantages, disadvantages, and characteristics of effective anecdotal records; 40 (2) viewing each of the five sections of the videotape, followed by writing and discussing the performance of that student based on the clinical objectives and the assignment; (3) scoring this student's performance using the scoring system; and (4) determining the complexify level of the student's assignment. Data Collection The instructors wrote anecdotal records for the last three weeks of the first rotation of their semesters. They were asked to write three anecdotes on each student, one anecdote for each week of clinical experience. The written anecdotal records and the complexity of the nursing situation were collected weekly from the clinical instructors. These anecdotal records were then typed, duplicated, and distributed to three judges—the clinical instructor and two other instructors—who scored them independently using the scoring system (Appendix I), and the clinical objectives of the student. These scores were then analyzed to determine their reliability using conventional correlational and generalizability analyses. During the week of January 16, 1978, the investigator obtained from the registrar's office the participating students' grades in their first semester Nursing theory, Biology, and Psychology courses. High school grade point averages (GPA's) were not used because the college did not require that information from applicants. These students were admitted into the nursing program on the condition of high school graduation and/or their ability to successfully complete appropriate "make up" courses. 41 Method of Analysis The analyses of the data were executed in two phases. First, it was necessary to assess the reliability of the clinical scores before proceeding to the stepwise multiple regression analyses. The following is a description of the two phases of the data analyses. Phase One During phase one, preliminary analyses of the data were conducted to determine (1) inter-rater reliability by correlating the scores given by the clinical instructors and the two nursing judges; and (2) generalizability coefficients among judges. The scores for these analyses which ranged from 1 to 9 were derived as shown in Appendix I. The latter was believed to be a more sensitive and informative measure of the reliability of the clinical scores. Thus, both methods of analyses were carried out and the results compared. A program from the Statistical Package for the Social Sciences (SPSS.V701) was used to compute the Pearson Product Moment Correlation Coefficients for the data of each of the four semesters separately and within each of the four semesters, each week separately. Each correlation coefficient (r) in turn was converted to Fisher's Z employing an r-to-Z transformation table. Next the Z values obtained for each of the three weeks of each of the fours semesters were added up separately and then the mean Z values were calculated. The mean Z values obtained for each of the weeks were afterwards converted to Pearson Product Moment Correlation Coefficients utilizing Fisher's procedure. 4 2 Then, the mean Z values for the three weeks of each of the semesters were added up and the total semester mean Z value was calculated and subsequently converted to Pearson Product Moment Correlation Coefficients employing the r-to-Z transformation procedure. Finally, the significance of the r values of each of the Semesters II to V, and each of the three weeks of those semesters were assessed against an a level of .05. The UCLA Biomedical Program BMD08V (Appendix J) was used to compute the generalizability coefficients across the nursing judges and weeks. These analyses were carried out with the data from Semesters II, III, and part of V. The fact that, within Semesters IV and V, the same three judges did not score the anecdotal records of all corresponding students—all Semester IV students and some Semester V students—resulted in a not fully crossed design. Therefore, for these two semesters, it was not possible to complete the generalizability analyses. Phase Two Before conducting the regression analyses called for in phase two, it was first necessary to examine the potential influence of judges. This was completed by comparing the regressionxesuIts using observed ratings with the regression results obtained when scores were standardized within judges. Regression analyses on a sample of the data were compared, utilizing standardized scores in one analysis and raw clinical scores in another. The clinical scores of Week 3 of both Semesters If and III were then standardized across the three judges following the procedure elaborated in Appendix K. The significance level for the regression analyses was set at a = .10 at each step (Green & Tull, 1965, p. 73). Given the exploratory nature of the study and the available small sample sizes, a more relaxed level of significance was adopted so as to ensure a reasonable power. The independent variables—first semester grades in Nursing theory, Biology, and Psychology—were quantified by equating the letter grades to numerical grade point equivalents according to the college calendar (refer to operational definitions in Chapter I). Upon inspection of the scores on complexity of the nursing situation, it was noted that they were mainly level 2 across all the students throughout the three weeks. It was subsequently decided to delete this variable because it would not yield enough variance to warrant its addition to the multiple regression equation. Finally, using the stepwise multiple regression program (Appendix L), the criterion scores—mean pooled raw clinical scores and mean standardized clinical scores—were analyzed and their prediction equations were compared to determine whether there was a significant difference between standardized scores and raw scores. Results of this analysis revealed that there were no significant differences between the prediction equations using standardized or raw clinical scores. Consequently, the rest of the data were analyzed using the mean pooled raw clinical scores and employing the linear model: YSCI +SNJ's (2) K 1 . n. , n . , = 3g + 3] Nursing + 32 Biology + 3g Psychology + e o The stepwise multiple regression program was used, and the clinical scores from each of the three weeks, and from the three weeks combined, were analyzed for each of Semesters II, III, IV, and V.^ ^ If a student was absent for a week, her score for the three weeks combined was determined by averaging her scores of the other two weeks. Chapter IV RESULTS OF ANALYSIS In this chapter are presented the results of the analyses on the complexity of the nursing situation; the correlations of the inter-rater reliabilities and generalizability coefficients; and the correlations among the clinical performance and Nursing, Biology, and Psychology grades. Complexity of the Nursing Situation The independent variable, complexity of the nursing situation, was deleted from the regression analyses of Semesters II, III, IV, and V because complexity scores within each of the three weeks, and between the three weeks, revealed that there was essentially no difference in the complexity of the nursing situation in which these students were asked to work. Possible reasons for this are: (a) the tendency of clinical instructors to perceive that the students' assignments were almost always within their abilities; (b) the phenomenon of central tendency. Because the instrument had only three levels of complexity, it may have encouraged instructors to avoid extreme ratings in assessing the complexity level of their students' situations; 44 45 (c) although the levels of complexity were not labeled as such, it appeared that the clinical instructors unconsciously thought of level 2 representing the desired and ideal complexity of students' assignments. The consequence of having to delete the variable, complexity of the nursing situation, limits the study to only academic predictors. Furthermore, it prevents an examination of possible relationships or interactions that may have existed between the academic predictors and complexity of nursing situation. Inter-Rater Reliabilities of Clinical Scores Correlational Analyses Semester II. Table 1 shows the mean and the standard deviation for each of the three judges, and the correlations between the judges for each of the three weeks. Also, it shows the mean correlations of each week and the three weeks combined. The mean correlation for Semester II was .75. Semester III. Table 2 gives the correlational results for Semester III. The overall semester correlation was .58. Semester IV. Table 3 gives the correlational results for Semester IV. In Weeks 2 and 3, the r's between Judges 2 and 4 could not be computed because there was no variance in the scores given by Judge 2. The overall semester correlation was .84. Semester V. Table 4 gives the correlational results for Semester V. The overall semester correlation was .73. TABLE 1 MEANS, STANDARD DEVIATIONS, AND INTERCORRELATIONS AMONG NURSING JUDGES (J) IN SEMESTER II Correlations between Week Means SD Judges0 Mean J 1 J 2 J 3 JT J 2 J3 J1J2 J 1 J 3 J 2 J 3 1 5.00 5.29 4.50 0.88 0.91 0.92 . 58 . 72 . 82 . 72 2 4.75 4.93 4.32 0.83 1.39 0.85 . 62 . 48 . 88 . 70 3 4.61 4.79 4.29 1.13 1.31 0.83 . 80 . 79 . 84 . 81 Combined .75 Correlations are based on sample size of 14. Mean correlations were obtained by converting r's between judges to Fisher Z's, averaging these, and reconverting them to correlation coefficients. TABLE 2 MEANS, STANDARD DEVIATIONS, AND INTERCORRELATIONS AMONG NURSING JUDGES (J) IN SEMESTER III Correlations between Week Means SD Judges0 J l J2 J3 J l J2 J3 J1 J2/ J1 J3/ J2 J3/ M e a n / K 1 N X N X N N 1 5.96 5.82 6.41 1.35 0.75 1.14 .53/ .49/ .57/ .53/ M l M l M l M l 2 5.75 5.63 5.54 1.06 0.93 1.03 .66/ .55/ .28/ .51/ '12 M2 /12 /12 3 5.75 5.50 5.38 1.36 0.71 0.88 .88, .62/ .55 , .72 , M2 M2 M2 x12 Combined .58/ M2 aThe numbers indicate sample size. Mean correlations were obtained by converting r's between judges to Fisher Z's, averaging these, and reconverting them to correlation coefficients. 4^ TABLE 3 MEANS, STANDARD DEVIATIONS, AND INTERCORRELATIONS AMONG NURSING JUDGES (J) IN SEMESTER IV Week Means SD Cor re ilations between Judges0 Jl h h J4 Jl h h J4 J1 J2/ 'N J1 J3/ X N J1 J4/ J2 J3/ X N X N X N J3 J4/ Mean/ N 1 4.55 5.25 6.09 5.36 1.51 0.62 1.38 2.46 .63/ '8 .81 / 77 • 9 1/ 7 .04/ X8 .76/ X8 .88/ 77 .75/ '15 2 4.46 5.00 5.50 6.10 1.06 0.45 1.08 1.60 .92/ .93/ f7 • 9 % .81 / •7 .59/ .89/ X14 3 4.68 5.00 5.40 6.30 1.10 0.47 1.58 1.77 .81 / '7 .84/ '8 .89/ • 9 2/ 6 */ 6 • 7 6 4 .85/ X14 Combined .84/ X16 'The numbers indicate sample size. Mean r's were obtained by converting r's between judges to Fisher Z's, averaging these, and reconverting them to correlation coefficients. *Coefficient could not be computed because there was no variance in the scores given by Judge 2 for those students co-rated with Judge 4. C O 49 TABLE 4 M E A N S , S T A N D A R D D E V I A T I O N S , A N D I N T E R C O R R E L A T I O N S A M O N G N U R S I N G J U D G E S (J) I N SEMESTER V Week 1 2 J l J2 Means -<3 J4 J 5 J 6 J l J2 SD J3 J4 Co r re l a t i ons between J udge s 0 J 5 J6 J l J 3 , J l J 4 / J l J 5 / J l J 6 / J 2 J 3 / J2J4/ J 2 J 5 / J 2 J 6 N N N N N N N 5 . 3 8 5 . 6 7 5 . 8 5 5 . 00 4 . 9 3 5 . 0 5 0 . 7 9 1.80 1 . 1 1 1 . 1 6 1 .57 1.38 . 9 5 / . 9 7 / . 8 2 / . 8 7 / . 6 9 / 1 . 0 0 / . 5 8 / . 7 0 / • 9 7 / 4 •8 2 / 4 . 7 9 / J 3 / 4 • 0 7 / 4 •4 % • 5 8 / 4 • 9 4 / 4 • 8 8 / 4 • 5 8 / 4 ' 3 ' 3 v 5 • 5 0 / 3 . 5 0 / . 8 3 / Comb ined °The numbers i nd i c a t e sample s i z e . M e a n co r re la t i ons were ob ta ined by . conve r t i ng r's between judges to F isher Z ' s , ave rag ing these , and reconve r t i ng them to co r r e l a t i on coe f f i c i en t s . 50 In summary, the co r re l a t i on coe f f i c i en t s ob ta ined in Semesters II , III , I V , and V show that the in te r - ra te r r e l i a b i l i t i e s o f rat ings of c l i n i c a l per formance o f nursing students were moderate to modera te ly s t rong , va r y i ng between . 58 and . 8 4 . G e n e r a l i z a b i l i t y Ana lyses Semester II . Tab le 5 reports a summary o f the g e n e r a l i z a b i l i t y ana lys is for Semester II . Estimates of the va r i an ce components for instructors (I), weeks (W), and weeks -by - j udges (WJ) were n ega t i v e . The nega t i ve est imates were subst i tuted w i t h ze ro va lues as recommended by C ronbach et a l . (1972). Consequen t l y , the expec ted v a r i a n c e s — E o 2 ( x ) — Q f f h e above sources were set to z e r o . In exam in i ng the rema in ing e igh t expec ted va r i an ce s , it c an be seen that the two sources of v a r i an ce—judges and students (S) were r e l a t i v e l y l a r ge , and therefore cont r ibu te most to the un iverse score v a r i a n c e , and hence to the g e n e r a l i z a b i l i t y c oe f f i c i en t s . Tab le 6 prov ides an examp le of how the un iverse score v a r i a n c e , the observed score v a r i a n c e , and the g e n e r a l i z a b i l i t y coe f f i c i en t s of both judges and weeks were compu ted . The g e n e r a l i z a b i l i t y coe f f i c i en t s (Ep z ) for the judges and weeks were . 75 and . 7 9 , r e spe c t i v e l y , i nd i c a t i ng that the c l i n i c a l scores were stab le both.across judges and over t ime . Semester III. Tab le 7 reports a summary of the A N O V A wh i c h supports the g e n e r a l i z a b i l i t y ana lys is for Semester III. Estimates o f the va r i an ce components for instructors, weeks , judges , and ins t ruc to rs -by-weeks (IW) were nega t i v e , and therefore g i v en a v a l ue of z e r o . The largest v a r i an ce source was the one by students, f o l l owed by the source s tudents -by -weeks nested w i t h i n instructor (SW:I). The g e n e r a l i z a b i l i t y 51 TABLE 5 A N A L Y S I S O F V A R I A N C E O F C L I N I C A L S C O R E S O F SEMESTER II STUDENTS : EFFECTS O F TIMES O N J U D G E S Source o f V a r i a n c e Sum of Squares Degrees of Freedom M e a n Square Est imate of V a r i a n c e Component F requency W i t h i n E a 2 ( x ) Instructors 0) 4 . 3 9 2 2 . 2 0 - 0 . 0 8 7 3 *0 .0000 Weeks (W) 1.03 2 0 . 52 - 0 . 0 1 4 3 *0 .0000 Judges (J) 9 . 2 3 2 4 . 61 0 . 124 3 0 . 0 413 Students S:l 4 7 . 8 5 9 5 . 32 0 .591 12 0 . 0492 IW 4 . 7 9 4 1.20 0 . 0 1 5 9 0 . 0 0 1 6 IJ 11 . 93 4 2 . 9 8 0 . 2 3 6 9 0 . 0262 W J 0 . 1 6 4 0 . 0 4 - 0 . 0 11 9 *0 .0000 SW:I 18 . 40 18 1.02 0 .341 36 0 . 0 095 SJ:I 2 . 7 4 18 0 . 1 5 0 .051 36 0 . 0014 IWJ 1.64 8 0 .21 0 . 0 0 9 27 0 . 0 0 0 3 SWJ:I 6 . 14 36 0 . 1 7 0 .171 108 0 . 0 0 1 6 * Expected var iances o f nega t i ve est imates o f va r i ance components are set to z e r o . Ea (x) means expec ted va r i an ce of each sou r ce . S:l means students nested w i t h i n ins t ructor . 52 TABLE 6 V A R I A N C E C O M P O N E N T S C O N T R I B U T I N G T O U N I V E R S E S C O R E V A R I A N C E A N D EXPECTED OBSERVED S C O R E V A R I A N C E O F J U D G E S A N D W E E K S , A N D G E N E R A L I Z A B I L I T Y C O E F F I C I E N T S O F SEMESTER II Source Un ive rse Score Expected Obse rved Score G e n e r a l i z a b i l i t y o f V a r i a n c e V a r i a n c e Coe f f i c i e n t V a r i a n c e J Judges 0.0413 0.0413 Students: 1 0.0492 0.0492 u IJ 0.0262 W J 0.0000 D SJ:I 0.0014 IWJ 0.0003 G SWJ:I 0.0016 E Tota l 0.0905 0.1200 S 0.0905 0.1200 = 0.75 W Weeks 0.0000 0.0000 Students: I 0.0492 0.0492 E IW 0.0016 Wl 0.0000 E SW:I 0.0095 IWJ 0.0003 K SWJ:I 0.0016 S Tota l 0.0492 0.0622 0 ^ = 0 79 0.0622 u ' / y Key : Week = W Student = S Judge = J Student nested w i t h i n Instructor = S;l Instructor = I No t e : Above te rm ino logy is e labora ted in Append i x J . 53 TABLE 7 A N A L Y S I S O F V A R I A N C E O F C L I N I C A L S C O R E S O F SEMESTER III STUDENTS : EFFECTS O F TIMES O N J U D G E S Source V a r i a n c e Sum of Squares Degrees of Freedom M e a n Square Estimate of V a r i a n c e Component F requency W i t h i n Ea 2 ( x ) Instructors 0) 0 . 6 7 1 0 . 6 7 - 0 . 1 2 7 2 *0 .0000 Weeks (W) 0 . 2 0 2 0 . 1 0 - 0 . 1 2 8 3 *0 .0000 Judges (J) 0 . 5 2 2 0 . 2 6 - 0 . 0 1 4 3 *0 .0000 Students S : l 7 5 . 2 5 10 7 . 5 3 0 . 8 3 6 12 0 . 0 6 9 7 IW 2 . 84 2 1.42 - 0 . 1 8 3 6 *0 .0000 IJ 5.41 2 2.71 0 . 1 0 9 6 0 .0181 W J 2 . 5 5 4 0 . 6 4 0 . 0 3 0 9 0 . 0033 SW:I 9 4 . 1 3 20 4 .71 1.569 36 0 . 0 4 3 6 SJ:I 14.91 20 0 . 7 5 0 . 2 4 9 36 0 . 0 069 IWJ 1 .36 4 0 . 3 4 0 . 0 10 18 0 . 0 0 0 5 SWJ:I 1 1 . 2 6 40 0 . 2 8 0 . 282 108 0 . 0 0 2 6 *Expected var iances o f nega t i ve est imates of v a r i an ce components are set to z e r o . Ea (x) means expec ted va r i ance of e a ch sou rce . S:l means students nested w i t h i n ins t ructor . 54 coe f f i c i en t s for the judges and weeks were . 69 and . 5 8 , r e spe c t i v e l y , i nd i c a t i ng that the c l i n i c a l scores show more s t ab i l i t y over judges than across t ime in Semester III. A poss ib le reason for the lat ter cou ld be e xp l a i ned by the p red i c ted growth in students ove r t ime . Semester V . Tab le 8 reports a summary of the A N O V A wh i ch supports the g e n e r a l i z a b i l i t y ana lys is for Semester V . Estimates of the v a r i an ce components for instructors, i n s t ruc to r s -by -weeks -by - judges ( IWJ) , ins t ructors -by- judges ( IJ) , and weeks -by - j udges (WJ) were nega t i v e , and therefore g i v en a v a l ue of z e r o . G e n e r a l i z a b i l i t y coe f f i c i en t s for judges and weeks were . 90 and .62 r e spe c t i v e l y , i nd i c a t i ng that the c l i n i c a l scores show more s t ab i l i t y over judges than across t ime in Semester V . A g a i n , a possib le reason for the lat ter cou l d be e xp l a i ned by the p red i c ted growth in the per formance of these students over t ime . Co r r e l a t i on Coe f f i c i en t s Versus G e n e r a l i z a b i l i t y Coe f f i c i en t s Tab le 9 provides the g e n e r a l i z a b i l i t y (G) for judges and the co r re l a t i on coe f f i c i en t s of Semesters II , III, and V . Both the G c oe f f i c i e n t and the co r re l a t i on c oe f f i c i e n t for Semester II were i d e n t i c a l , . 7 5 . The G coe f f i c i e n t of Semester III, howeve r , was h igher than the co r re l a t i on c o e f f i c i e n t . This d i s c repancy of . 11 is perhaps due to the fact that the G coe f f i c i en t is a more sens i t i ve measure of r e l i a b i l i t y than the conven t i ona l co r re l a t i ona l c o e f f i c i e n t . F i n a l l y , the G coe f f i c i e n t of Semester V was . 9 0 , larger than the co r r e l a t i on c oe f f i c i e n t wh i c h was . 7 5 . The d i sc repancy of . 15 between the two coe f f i c i en t s of Semester V was not on l y due to 55 TABLE 8 A N A L Y S I S O F V A R I A N C E O F C L I N I C A L S C O R E S O F SEMESTER V STUDENTS : EFFECTS O F TIMES O N J U D G E S Source of V a r i a n c e Sum of Squares Degrees of Freedom M e a n Square Estimate o f V a r i a n c e Component F requency W i t h i n Ea 2 ( x ) Instructors (1) 8 . 9 6 1 8 . 9 6 - 0 . 0 3 2 2 *0 .0000 Weeks (W) 3 0 . 7 9 2 15 . 39 0 . 4 6 5 3 0 . 1 5 4 8 Judges (J) 6 . 04 2 3 . 02 0 .131 3 0 . 0 4 3 7 Students S:l 3 9 . 3 5 4 9 . 8 4 1.093 6 . 0 . 1822 IW 2 0 . 9 0 2 10 . 45 0 . 3 8 0 6 0 . 0 633 IJ 1.15 2 0 . 5 7 - 0 . 0 0 9 6 *0 .0000 W J 0 . 3 5 4 0 . 0 9 - 0 . 101 9 *0 .0000 SW:I 5 6 . 2 6 8 7 . 0 3 2 . 344 18 0 . 1302 SJ:I 5 . 2 6 8 0 . 6 6 0 . 2 1 9 18 0 . 0122 IWJ 0 .91 4 0 . 2 3 - 0 . 1 5 6 18 *0 .0000 SWJ:I 1 1 . 1 3 16 0 . 7 0 0 . 6 9 6 54 0 . 0 1 2 9 *Expected va r i ances of nega t i ve estimates of v a r i an ce components are set to z e r o . E a 2 ( x ) means expec ted va r i an ce of each sou r ce . S:l means students nested w i t h i n ins t ructor . 56 TABLE 9 A C O M P A R I S O N O F G E N E R A L I Z A B I L I T Y C O E F F I C I E N T S A N D M E A N C O R R E L A T I O N C O E F F I C I E N T S A C R O S S J U D G E S I N SEMESTERS II , III , A N D V a M e a n Semester G e n e r a l i z a b i l i t y Co r r e l a t i on Coe f f i c i en t s . 75 . 5 8 * . 7 5 *This mean co r r e l a t i on c oe f f i c i e n t was computed on the scores o f the same sample s i ze of s ix students that was used to c a l c u l a t e the G coe f f i c i e n t in Semester V . a Semes te r IV da ta were not su i tab le for g e n e r a l i z a b i l i t y ana lys is because the same three judges d i d not score the anecdo ta l records of a l l corresponding students. II . 75 II . 69 V .90 57 the f a c t that G c oe f f i c i e n t is a more sens i t i ve measure of r e l i a b i l i t y , but a l so poss ib ly that on l y the da ta generated by a smal l sample of s ix students was u t i l i z e d to compute the g e n e r a l i z a b i l i t y c o e f f i c i e n t . In summary, the g e n e r a l i z a b i l i t y coe f f i c i en t s and the co r re l a t i on coe f f i c i en t s across the three judges were comparab le in Semesters II , III, and V . Phase two ana l y se s—mu l t i p l e regress ions—were thus i n i t i a t e d . Regression Ana lyses Pre l im inary Ana lyses : Raw C l i n i c a l Scores Versus S tandard i zed C l i n i c a l Scores Semester II . Tab le 10 shows the means, standard d e v i a t i o n , co r re l a t i on c oe f f i c i e n t s , and c oe f f i c i e n t of de te rm ina t ion (R 2 ) for both the raw and the s tandard i zed c l i n i c a l scores of Semester I I , Week 3 . Compar ing the stat is t ics of the raw c l i n i c a l scores to the s tandard i zed c l i n i c a l scores , we no t i ce that the i r standard dev i a t i on s , co r r e l a t i on c oe f f i c i e n t s , and the R z are very c l o s e . For e x amp l e , the R 2 of the raw and s tandard i zed c l i n i c a l scores were . 419 and . 4 3 2 , r e spe c t i v e l y . Fur thermore, the order of entry of the independent va r i ab l e s in the regression equat ions was the s ame—Psycho l ogy f i r s t , then B io l ogy , and f i n a l l y N u r s i n g . In c on c l u s i o n , there were no s i gn i f i c an t d i f fe rences between the regression results of the raw and s tandard i zed c l i n i c a l scores of Semester II , Week 3 . Semester III. The m e a n , standard d e v i a t i o n , co r r e l a t i on ma t r i x , and R 2 for both the raw and the s tandard i zed c l i n i c a l scores o f Semester III, Week 3 are shown in Tab le 1 1 . L ike the Semester II regression ana lyses , the results of TABLE 10 S U M M A R Y O F STEPWISE MULT IP LE R E G R E S S I O N A N A L Y S I S O F WEEK 3 O F SEMESTER II: A C O M P A R I S O N O F RAW C L I N I C A L SCORES T O S T A N D A R D I Z E D C L I N I C A L SCORES Raw vs S tandard i zed V a r i a b l e Stat ist ics X SD C l i n i c a l Scores r Nurs ing Psychology B io logy R2 Raw C l i n i c a l Scores C l i n i c a l score Nurs ing Psycho logy B io logy 4 . 7 0 8 2 . 4 17 2 . 8 33 2 . 0 8 3 0 .899 0 . 5 15 0 . 937 0 . 289 .384 .620 .534 - . 0 1 5 - . 2 5 5 - . 2 8 0 .419 .385 .412 C l i n i c a l score - 0 . 0 0 2 0 . 852 .394 .636 - . 0 3 9 S tandard i zed Nurs ing 2 . 417 0 . 515 .534 - . 2 5 5 .432 C l i n i c a l Psycho logy 2 . 8 3 3 0 . 9 37 - . 2 8 0 .404 Scores B io logy 2 . 0 83 0 . 2 89 .425 N = 12 (the same sample of 12 students used in g ene r a l i z a b i l i t y analys is) TABLE 11 S U M M A R Y O F STEPWISE MULT IPLE R E G R E S S I O N A N A L Y S I S O F WEEK 3 O F SEMESTER III: A C O M P A R I S O N O F RAW C L I N I C A L SCORES T O S T A N D A R D I Z E D C L I N I C A L SCORES Raw Stat ist ics vs V a r i a b l e r S tandard i zed X SD C l i n i c a l Nurs ing Psychology B io logy Scores C l i n i c a l score 5.541 0 . 8 79 .761 .761 .465 Raw Nu r s i ng 2 . 8 33 0 . 577 1.000 .369 . 579 C l i n i c a l Psycho logy 2 . 833 0 . 5 7 7 .369 Scores B io logy 3 . 000 0 . 8 5 3 .618 C l i n i c a l score - 0 . 021 0 . 8 5 7 .742 .742 .448 S tandard i zed Nurs ing 2 . 8 33 0 . 577 1.000 . 369 .551 C l i n i c a l Psycho logy 2 . 8 33 0 . 5 77 .369 Scores B io logy 3 . 000 0 . 853 . 586 N = 12 60 Semester III revea led that the standard dev i a t i on s , co r r e l a t i on c oe f f i c i e n t s , and the R^'s of both the raw and the s tandard i zed c l i n i c a l scores were app rox imate l y the same. For e x amp l e , the R^'s of the raw and the s tandard i zed analyses were . 618 and . 5 8 6 , r e spe c t i v e l y . Fur thermore, the order of the entry of the independent va r i ab l es in the regression equat ions was the s ame—Nu r s i n g and B i o l ogy . It should be noted that because Nurs ing and Psycho logy were pe r f e c t l y co r re la ted (r = 1 .00) , the computer a lgor ism se lec ted N u r s i n g . In c on c l u s i o n , there were no important d i f fe rences between the results of the regression analyses o f the raw and the s tandard i zed c l i n i c a l scores of Week 3 of Semesters II and III. H e n c e , it was de c i ded to use the mean raw c l i n i c a l scores to do the regression ana l y ses—the c l i n i c a l scores g i v en by the three nursing j udges—wi thou t s tandard i z ing them. F i na l Ana lyses: Raw C l i n i c a l Scores Semester II . Tab le 12 shows the means, standard dev i a t i on s , co r r e l a t i on mat r i ces , coe f f i c i en t s o f de te rmina t ion (R ) and the pa r t i a l r's for the regression ana lys is o f the da ta of Semester II . The results of Week 1 show that on l y Psycho logy entered the mu l t i p l e regression equa t i o n , p roduc ing an R z o f . 2 5 9 , wh i ch was s i gn i f i c an t at the . 1 0 l e ve l or bet ter . A n inspec t ion of the pa r t i a l r's shows that B io l ogy wou ld have entered next in the regression equa t i o n . Howeve r , its con t r i bu t i on to the p red i c t i on of the c l i n i c a l scores was not s i gn i f i c an t at the . 10 l e v e l . Consequen t l y , i t d id not enter the regression equa t i o n . TABLE 12 S U M M A R Y O F STEPWISE MULT IPLE R E G R E S S I O N A N A L Y S E S : SEMESTER II Stat ist ics Week V a r i a b l e X SD r R* Part ia l r C l i n i c a l Nurs ing Psycho logy B io logy C l i n i c a l score 4 . 9 3 0 .81 .104 .509 .085 1 Nurs ing 2.21 0 . 8 0 .499 - . 0 7 7 - . 201 Psycho logy B io logy 2 . 7 9 2 . 0 7 0 . 8 9 0 . 2 7 - . 2 5 3 .259 . 257 C l i n i c a l score 4 . 6 7 0 .91 .580 .411 - . 0 5 3 2 Nurs ing 2.21 0 . 8 0 .499 - . 0 7 7 .337 Psycho logy B io logy 2 . 7 9 2 . 0 7 0 . 8 9 0 . 2 7 - . 2 5 3 .356 .034 C l i n i c a l score 4 . 5 6 1.02 .643 . 636 .031 3 Nurs ing 2.21 0 . 8 0 .499 - . 0 7 7 .413 Psycho logy B io logy 2 . 7 9 2 . 0 7 0 . 8 9 0 . 2 7 - . 2 5 3 .545 .577 C l i n i c a l score 4 . 7 2 0 . 7 5 .560 .632 .023 Comb ined Nurs ing 2.21 0 . 8 0 .499 - . 0 7 7 .480 Psycho logy B io logy 2 . 79 2 . 0 7 0 . 8 9 0 . 2 7 - . 2 5 3 .400 .509 N = 14 62 The results of Week 2 show that both Nurs ing and Psycho logy entered in the regression equa t i on , p roduc ing an R of .356, wh i c h was s i gn i f i c an t at a = .10. The pa r t i a l r of B i o l ogy , when both Nurs ing and Psycho logy had entered in the regression equa t i o n , was .034, con f i rm ing that it wou ld not have added much to the regression c o e f f i c i e n t . The results of Week 3, on the other hand , show that a l l three independent va r i ab les entered the regression equa t i on , p roduc ing an R 2 = .577, wh i ch was s i gn i f i c an t at a = .10. The order o f the entry was: N u r s i n g , Psycho logy , B i o l ogy . F i n a l l y , the regression results of Weeks 1, 2, and 3 combined show that o a l l three va r i ab les entered the regression equa t i o n , p roduc ing an R of .509, wh i c h was s i gn i f i c an t at the .10 l e v e l . In contrast to Week 3, the order of entry of the independent va r i ab les was: Psycho logy , N u r s i n g , B i o l ogy . In summary, 51 percent of the va r i an ce of the comb ined c l i n i c a l scores in Semester II can be at t r ibuted to var ia t ions in grades in Psycho logy , N u r s i n g , and B i o l ogy . Semester III. Tab le 13 shows the means, standard dev i a t i on s , co r re l a t i on mat r i ces , coe f f i c i en t s of de te rm ina t ion (R 2), and the pa r t i a l r's for the regression analyses of the da ta of Semester III. An exam ina t i on of the co r r e l a t i on matr ices shows that Nurs ing and Psycho logy had a co r re l a t i on c oe f f i c i e n t o f 1.00. Consequen t l y , on l y one of these two va r i ab les wou ld have had the oppor tun i ty to enter in the regression equa t ion ( a = .10). Howeve r , e i ther of these v a r i a b l e s — Psycho logy or Nu r s i n g—when used a lone in the regression ana l y s i s , wou ld have y i e l d ed subs tant ia l l y the same resul ts . TABLE 13 S U M M A R Y O F STEPWISE MULT IPLE R E G R E S S I O N A N A L Y S E S : SEMESTER III Week V a r i a b l e N X SD C l i n i c a l Score Co r re l a t i on Ma t r i x Nurs ing Psychology B io logy R 2 Par t ia l r C l i n i c a l score 11 6 . 0 6 0 . 8 9 .45 . 45 .46 1 Nu r s i ng 11 2 . 82 0 . 6 0 1.00 .43 Psycho logy 11 2 . 82 0 . 6 0 .43 B i o l ogy 11 3 . 0 9 0 . 8 3 C l i n i c a l score 12 5 . 64 0 . 8 2 . 79 . 79 .65 2 Nurs ing 12 2 . 8 3 0 . 5 8 1.00 . 37 .621 Psycho logy 12 2 . 8 3 0 . 5 8 .37 .000 B io logy 12 3 . 0 0 0 . 8 5 .768 C l i n i c a l score 12 5 . 54 0 . 8 8 . 76 . 7 6 .47 3 Nurs ing 12 2 . 8 3 0 . 5 8 1.00 .37 .579 Psycho logy 12 2 . 8 3 0 . 5 8 .37 . 000 B io logy 12 3 . 0 0 0 . 8 5 .618 C l i n i c a l score 12 5 . 74 0 .71 .80 . 80 .62 Comb ined Nurs ing 12 2 . 8 3 0 . 5 8 1.00 . 37 .637 Psycho logy 12 2 . 8 3 0 . 5 8 . 37 .000 B io logy 12 3 . 0 0 0 . 8 5 .760 Data for one student was missing for one week; in this case c l i n i c a l scores for Weeks 1, 2 , and 3 comb ined were computed by ave rag ing her scores on the remain ing two weeks . 64 In Week 1, none of the variables entered the regression equation. Unl ike Week 1, in Week 2 both Nursing and Biology entered the regression equation, producing an R 2 of .768 which was significant at the .10 level or better. The partial r of Psychology was .000, an expected va lue, given the fact that Psychology and Nursing had a perfect r of 1 .00 . In Week 3, again Nursing and Biology entered in the regression equation, producing an R of .620 which was significant at a = .10 . F ina l ly , combining Weeks 1, 2, and 3 yielded R .76, significant at a = .10 . In summary, 76 percent of the variance of the c l in i ca l scores of Semester III students could be attributed to their first semester Nursing theory and Biology grades. Semester IV. Table 14 shows the means, standard deviations and the correlation matrices for Semester IV. An inspection of the correlation matrices shows that the correlation coefficients between the dependent var iab le—c l in i ca l scores—and the independent var iables—Nurs ing, Psychology, and Biology—were smaller than the correlation coefficients between the independent variables. The above, together with the problem of small sample s ize, created a situation where none of the variables entered the regression equation at the set a level of . 10 . Semester V . Table 15 shows the means, standard deviations and the correlation matrices for Semester V . Like Semester IV, the correlation coefficients between the dependent and the independent variables were in general smaller than the r's between the independent variables. The latter, together with the problem TABLE 14 S U M M A R Y O F STEPWISE MULT IPLE R E G R E S S I O N A N A L Y S E S : SEMESTER IV Week V a r i a b l e N X SD C l i n i c a l Score Co r re l a t i on Nurs ing Ma t r i x Psychology B io logy C l i n i c a l score 15 5.31 1.42 .331 .282 .015 1 Nurs ing 15 2 . 9 3 0 . 7 0 .620 .514 Psycho logy 15 3 . 00 2 . 65 .452 B io logy 15 2 . 6 7 0 . 72 C l i n i c a l score 14 5 . 2 4 0 . 90 . 397 . 359 . 250 2 Nurs ing 14 3 . 0 0 0 . 6 8 .552 . 486 Psycho logy 14 3 . 07 0 . 62 . 396 B io l ogy 14 2 . 7 9 0 . 7 0 C l i n i c a l score 14 5 . 3 5 1.20 .289 . 213 . 020 3 Nurs ing 14 2 . 8 5 0 . 6 9 .684 .591 Psycho logy 14 3 . 00 0 .71 . 488 -B io l ogy 14 2 . 7 7 0 . 7 3 C l i n i c a l score 16 5 . 3 3 0 . 9 3 .389 . 298 . 037 Comb ined Nurs ing 16 2 . 9 4 0 . 6 8 . 620 . 513 Psycho logy 16 3 . 00 0 . 6 3 . 449 B io logy 16 2 . 6 9 0 . 7 0 Data for f i v e students was missing for one week; in these cases c l i n i c a l scores for Weeks 1, 2, and 3 combined were computed by ave rag ing the i r scores on the remain ing two weeks . TABLE 15 S U M M A R Y O F STEPWISE MULT IPLE R E G R E S S I O N A N A L Y S E S : SEMESTER V Week V a r i a b l e N X SD C l i n i c a l Score Co r re l a t i on Nurs ing Ma t r i x Psycho logy B io logy C l i n i c a l score 17 5 . 3 0 1.16 - . 311 . 303 . 035 1 Nurs ing 17 3 . 12 0 . 8 6 - . 1 1 5 . 499 Psycho logy 17 3 . 12 0 . 7 8 - . 3 3 3 B io logy 17 2 . 82 0 . 64 C l i n i c a l score 16 4 . 7 3 0.91 .056 . 109 . 094 2 Nurs ing 16 3 . 1 9 0 . 8 3 - . 1 3 6 . 436 Psycho logy 16 3 . 1 3 0 .81 - . 3 6 7 B io logy 16 2 . 8 8 0 . 62 C l i n i c a l score 17 5 . 2 9 0 . 9 8 .095 . 349 .002 3 Nurs ing 17 3 . 12 0 . 8 6 - . 1 1 5 . 499 Psycho logy 17 3 . 12 0 . 7 8 - . 3 3 3 B io logy 17 2 . 82 0 . 64 C l i n i c a l score 17 5 . 14 0 . 8 3 - . 1 3 7 .311 .002 Comb ined Nurs ing 17 3 . 12 0 . 8 6 - . 1 1 5 . 499 Psycho logy 17 3 . 12 0 . 7 8 - . 3 3 3 B io logy 17 2 . 82 0 . 6 4 Da ta for one student was missing for one week; in this case c l i n i c a l scores for Weeks 1, 2 , and 3 combined were computed by ave rag ing her scores on the remain ing two weeks . 67 o f smal l sample s i z e , c rea ted a s i tua t ion where the con t r i bu t i on of the var iab les was not s i gn i f i c an t at the .10 l e v e l , and therefore d id not enter in the regression equa t i on . In summary, 51 and 76 percent of the va r i an ce in the c l i n i c a l per formance scores of Semesters II and III nursing students cou ld be accoun ted for by the i r f irst semester Nurs ing theo ry , B i o l ogy , and Psycho logy grades . Howeve r , in both Semesters IV and V , the coe f f i c i en t s of de te rminat ion were smal l and not s i gn i f i c an t . Further discussions of these f ind ings are presented in Chap te r V . Chapte r V S U M M A R Y , F I N D I N G S A N D C O N C L U S I O N S , L I M I T A T I O N S , A N D R E C O M M E N D A T I O N S The purpose of this study was to increase knowledge of factors wh i c h co r re l a te w i th the c l i n i c a l per formance of nursing students, thereby assist ing nurse educators to better p red i c t students' pe r fo rmance , and a c c o r d i n g l y g i v e counse l . A search of the l i te ra ture revea led that ve ry few studies have been done in the area of p red i c t i on of c l i n i c a l performance of nursing students. O f the few studies done , howeve r , the reported re lat ionsh ips between performance in theory courses and c l i n i c a l p rac t i ce courses were weak . Tay lor et a l . (1965) suggested that perhaps the c l i n i c a l grades represent measures of behaviors and ab i l i t i e s add i t i ona l to those measured by a cadem i c courses. Furthermore, none of the studies addressed the quest ion of r e l i a b i l i t y of the c l i n i c a l grades that were be ing p r ed i c t ed . The lat ter has d isturbed many nurse educators , who have expressed d issat i s fac t ion w i th the t y p i c a l process of c l i n i c a l e v a l u a t i o n . The problem of the present study was to determine the r e l a t i v e degree to wh i c h (a) a cadem i c grades and (b) comp l ex i t y of the nursing s i tua t ion pred i c t scores in c l i n i c a l per formance for first and second year nursing students. 68 69 The student sample consisted of 14 Semester II students; 12 Semester III students; 16 Semester IV students; and 17 Semester V students. O f the 16 instructors who took part in the s tudy , two ac ted on ly as nursing judges, and 14 ac ted as both c l i n i c a l instructors and judges. Rines ' (1963) anecdo ta l record form was adapted to meet the needs of the s tudy . The scor ing instrument, des igned by the inves t iga to r , con ta ined c r i t e r i a w i t h wh i c h the c l i n i c a l instructors and judges were to use a 9 po int L i ke r t - t ype sca le to rate the degree of student ach i evement of stated c l i n i c a l course ob j e c t i v e s . The sca le was made more e x p l i c i t by emp loy ing the conven t i ona l c o l l e g e grad ing system to e xp l a i n the numer i ca l scores on the s c a l e . For e x amp l e , any score from 8 to 9 was equ i va l en t to super ior pe r fo rmance , or a letter grade of " A " s t and ing . The instrument for measur ing comp l e x i t y of the nursing s i tua t ion was a lso des igned for this s tudy . A three po int s ca l e r e f l e c t i ng d i f fe rent leve ls of c omp l e x i t y was used . Each l eve l of c omp l ex i t y was descr ibed by four major c r i t e r i a wh i c h were to be used by the c l i n i c a l instructors to make the i r dec i s ions (see Append i x G ) . The v i d eo t ape - e v a l u a t i o n of a student 's c l i n i c a l per formance was des igned by the inves t iga to r . Its purpose was to t ra in instructors in the wr i t i ng of anecdo ta l records about students ' c l i n i c a l pe r fo rmance . The t ape , cons is t ing of f i v e v igne t tes , ran for a tota l of n ine minutes and dep i c t ed the per formance of a nursing student ca r i ng for two pa t ien ts . 70 Separate workshops were conduc ted by the invest igator to t ra in the instructors of Semesters II, III, I V , and V , in wr i t i ng and scor ing anecdo ta l records , and de te rmin ing the l e ve l of c omp l ex i t y of the nursing s i t u a t i on . The workshops were conduc ted in two parts . F i rs t , the advantages and d isadvantages of the anecdo ta l record techn ique of eva l ua t i on were d i scussed, f o l l owed by a d iscuss ion of each of the f i v e v igne t tes . This was f o l l owed by p r a c t i c e in record ing and scor ing the anecdo ta l record of the student 's per formance observed on the v i d eo t ape , and de te rmin ing the l eve l of c omp l ex i t y o f this student 's c l i n i c a l ass ignment. Data for the study were c o l l e c t e d over a t h ree -week pe r i o d . These data i n c l uded : (a) c l i n i c a l per formance scores ob ta ined from anecdo ta l records o f students ' c l i n i c a l per formance prepared by the i r instructors a n d , subsequent ly , scored by the c l i n i c a l instructor and two nursing judges using the student 's c l i n i c a l ob jec t i ves and the scor ing instrument; (b) ra t ing of the l e ve l of c omp l ex i t y of the students ' c l i n i c a l assignments by the i r c l i n i c a l inst ructors , us ing the c omp l e x i t y of the nursing s i tua t ion instrument; and (c) f irst semester Nurs ing t heo r y , Psycho logy , and B io logy grades ob ta ined from the students ' permanent records in the Registrar 's o f f i c e . Da ta analyses were ca r r i ed out in two phases. Phase one i nvo l ved the u t i l i z a t i o n of two s ta t i s t i ca l methods to a na l y z e the r e l i a b i l i t y o f the c l i n i c a l per formance scores: (1) conven t i ona l co r re l a t i on ana lyses; and (2) g e n e r a l i z a b i l i t y ana l yses . The dependent v a r i a b l e was the average of the c l i n i c a l per formance scores of the students g i v en by the c l i n i c a l instructors (who wrote and scored 71 the anecdo ta l records) and two nursing judges (who scored the wr i t t en anecdo ta l records) . The independent va r iab les were the grades in f irst semester N u r s i n g , B i o l ogy , and Psycho logy; and the comp l e x i t y of the nursing s i t u a t i o n . In phase t w o , mu l t i p l e regression analyses were conduc ted to determine the proport ion o f va r i ance of the c l i n i c a l per formance scores in Semesters I I , III , I V , and V a c coun ted for by the a cadem i c grades ob ta ined in Semester I N u r s i n g , Psycho logy , and B i o l ogy . F ind ings and Conc lus ions Both the g e n e r a l i z a b i l i t y and co r r e l a t i ona l analyses showed that the in te r - ra te r r e l i a b i l i t i e s among the three judges in a l l o f the four semesters were moderate to moderate ly strong (from . 58 to . 8 4 ) . The g e n e r a l i z a b i l i t y (G) coe f f i c i en t s were usua l l y h igher than the co r r e l a t i on c oe f f i c i e n t s . This d i s c repancy c an be e xp l a i n ed by the fac t that the G coe f f i c i e n t is a more sens i t i ve measure of r e l i a b i l i t y when compared w i t h the conven t i ona l co r r e l a t i on c o e f f i c i e n t . G i v e n that the c l i n i c a l scores were found to be su f f i c i en t l y r e l i a b l e , the da ta ana lys i s p roceeded to phase t w o . Inspect ion of the rat ings of the leve ls of c omp l e x i t y o f the nursing s i tua t i on in each semester r evea led m in ima l to no va r i a t i on from l eve l two c o m p l e x i t y . Consequen t l y , the independent v a r i ab l e c omp l e x i t y of the nursing s i tua t ion cou l d not be used in the regression ana l y s i s . 72 The regression ana lys is of Semester II da ta r evea l ed that 51 percent of the v a r i an ce of the c l i n i c a l scores c ou l d be e xp l a i ned by the three rema in ing independent v a r i a b l e s — a c a d e m i c grades in f irst semester N u r s i n g , P sycho logy , and B i o l ogy . The squared mu l t i p l e co r re l a t i on c oe f f i c i e n t was s i gn i f i c an t at . 10 l e ve l of s i g n i f i c a n c e . For Semester III, 76 percent of the va r i an ce o f the c l i n i c a l score was accoun ted for by the Nurs ing or Psycho logy grades , and the B io logy grades (p. < . 1 0 ) . In this ana l y s i s , Nu r s ing and Psycho logy grades were pe r f e c t l y c o r r e l a t ed . Consequen t l y , e i t he r o f these v a r i a b l e s , when used a l o n e , e xp l a i n ed the same amount of v a r i an ce in the c l i n i c a l scores . In contrast , the regression analyses for Semesters IV and V r evea l ed that on l y 20 and 16 percent of the v a r i an ce of the c l i n i c a l scores r e spec t i v e l y , was accoun ted for by the three independent v a r i a b l e s . These results were not s i g n i f i c a n t . In summary, i t appears that the a cadem i c grades in N u r s i n g , P sycho logy , and B io logy courses ob ta ined in f irst semester of the nursing program are better pred ictors of c l i n i c a l per formance in ea r l i e r semesters, Semesters II and III, than in later semesters, Semesters IV and V . Based on the above f i nd ings , the f o l l ow i ng conc lus ions were made: 1. The anecdo ta l record method o f c l i n i c a l per formance e va l ua t i on y i e l d e d a c c ep t ab l y r e l i a b l e c l i n i c a l scores where a l l the c l i n i c a l instructors (those instructors who wrote the anecdo ta l records) and a l l the judges (those who eva lua ted the wr i t ten anecdo ta l records) were t ra ined us ing a p lanned ins t ruc t iona l p rogram. 2 . The grades in Psycho logy , N u r s i n g , and B io logy courses dur ing the f i rst semester of the nursing program accoun ted for a s i gn i f i c an t proport ion of the 73 va r i ance in the c l i n i c a l per formance scores of nursing students in Semester II . L i k ew i s e , Nurs ing or P sycho logy , and B io logy grades ob ta ined in the f irst semester of the nursing program accoun ted for a s i gn i f i c an t proport ion of the va r i an ce in the c l i n i c a l per formance scores of Semester III nursing students. In Semesters IV and V , howeve r , these grades f a i l e d to accoun t for a s i gn i f i c an t proport ion o f the va r i an ce in c l i n i c a l per formance scores of these students. There fo re , hypothesis 1—grades in N u r s i n g , B i o l ogy , and Psycho logy courses dur ing the f irst semester o f the nursing program accoun t for a s i gn i f i c an t proport ion o f the va r i an ce in c l i n i c a l per formance scores of s tudents—is a c cep ted in the case of Semester II and III students, and re jec ted in the case of Semester IV and V students. 3. The c omp l e x i t y of the nursing s i tua t ion cou l d not be used in the regression ana lys is because it showed no v a r i a b i l i t y across the three weeks and the four semesters. There fo re , hypothesis 2—comp lex i t y of the nursing s i tua t ion accounts for an add i t i ona l s i gn i f i c an t proport ion of the v a r i a n ce in c l i n i c a l per formance scores o f s tuden t s—cou ld not be tes ted . There are severa l in te r re la ted factors wh i c h may e x p l a i n the d i f fe rences noted between Semesters II and III and Semesters IV and V c l i n i c a l per formance and hence the mu l t i p l e cor re la t ions w i th the independent va r iab les s e l e c t e d . As the nursing students progress from one semester to ano the r , the i r respons ib i l i t i es i n c r ease . They are expec ted to (1) be more independent; (2) have acqu i red a reper to i re of sk i l l s and competenc ies to make more independent dec is ions that requ i re h igher leve ls of c ogn i t i v e ab i l i t i e s ; (3) p rov ide nursing care to i nd i v i dua l s 74 with complex health problems; (4) provide nursing care for a larger number of individuals; (5) assume a greater role as a member of the nursing team; and (6) require less direct instructor supervision. A l l the above factors may have influenced the nature of the students' c l i n i ca l assignments, the amount of required and/or desired instructor supervision, and the accessibil ity of the c l i n i ca l instructor to the student. Based on this information, it can be safely assumed that as the students progress from Semester II to Semester V , their c l i n i ca l assignments w i l l increase in number and complexity, they w i l l require less direct instructor supervision, and w i l l have less access to their c l i n i ca l instructors. It can be conjectured that the aforementioned may affect the nature of the content of the anecdotal records written by the c l i n i ca l instructors, and hence the c l i n i ca l scores of the students. Consequently, it can be expected that these factors may influence the performance scores of Semester IV and V students more than Semester II or III students. Therefore, to the extent that the nature of the students' assignments change as a result of their natural progression in the nursing program, their c l i n i ca l performance may be affected, and thus may not correlate as highly with previously identified variables. Furthermore, there were temporal factors which may have contributed to the differences noted in the multiple correlation coefficients of Semesters IV and V . These senior students had additional c l i n i ca l experiences, and higher level Nursing, Psychology, and Biology courses which may have affected their attitudes, motivation, competencies, perceptions of the role of the nurse, perceptions of the reality of c l i n i c a l nursing, hence their c l i n i ca l performance and c l i n i ca l scores. 75 F i n a l l y , a poss ib le fac tor wh i c h may have i n f l uenced the performance o f Semester II and III students was the increase in the amount and type of c l i n i c a l exper iences of these students in the i r f irst semester of nurs ing . These jun ior students had more c l i n i c a l exper iences in the i r Semester I c l i n i c a l nursing course compared to the senior students of Semesters IV and V . It is important to no te , howeve r , that this fac tor has s im i l a r i t i e s to the tempora l factors descr ibed a bo v e , wh i c h were con jec tu red to have cont r ibu ted to the d i f fe rences noted in the mu l t i p l e co r re l a t i on coe f f i c i en t s between Semesters II and III and Semesters IV and V . There were severa l factors wh i c h may have i n f l uenced the ratings of the independent v a r i a b l e , the comp lex i t y of the nursing s i t u a t i on , y i e l d i n g comp l e x i t y leve ls wh i c h were essent ia l l y the s a m e — l e v e l two . The c r i t e r i a for l eve l two comp l ex i t y were: 1. a c t i v i t i e s of the s tudent—treatments prescr ibed by the doc to r and/o r the nursing team are moderate in number; the student has had numerous opportun i t ies to pract i se the sk i l l s and a c t i v i t i e s of the prev ious semester, and has m in ima l /modera te opportun i t ies to pract i se the sk i l l s and a c t i v i t i e s learned in the present semester; 2 . c ond i t i on of the pa t i e n t—the pat ient ' s c ond i t i on is chang ing and requires moderate mod i f i c a t i on of the student 's p l an of care; 3 . pat ient e x p e r i e n c e s — a v a i l a b l e pat ient exper i ences meet the learn ing needs of students in a g i v en semester; su i tab le s tudent /pat ien t r a t i o , w i t h pat ients who prov ide des i red exper iences; and 4 . rev is ions of student's assignment based on the instructor 's assessment of the student 's a b i l i t i e s / p e r -f o rmance—the assignment requires: in tegra t ion of p rev ious ly acqu i red and some 76 new l y learned knowledge; coo rd ina t i on of nursing a c t i v i t i e s ; f l e x i b i l i t y in the student 's o rgan i za t i on and p l an o f care ; p rob l em-so l v i ng a c t i v i t i e s (assessment, p l a nn i ng , imp lemen ta t i on , and eva l ua t i on ) ; and the student can cope w i th assignments for his semester l e ve l (for further e xp l ana t i on see Append i x G ) . This va r i ab l e had three leve ls of c omp l ex i t y wh i c h may have encouraged the c l i n i c a l instructors to pe r ce i v e and choose l e ve l two comp l ex i t y as the des i red and i dea l c omp l ex i t y of the nursing s i t u a t i o n . It is a l so poss ib le that in r ea l i t y comp l ex i t y of the nursing s i tua t ion does not v a r y . L imi tat ions of the Study 1. Conc lus ions can on l y be gene r a l i z ed to the pa r t i cu l a r ins t i tu t ion that was samp led . Howeve r , there are no known reasons for b e l i e v i ng that c o l l e g e nursing instructors are not t y p i c a l of a much larger popu la t i on of nursing instructors in other inst i tut ions o f fer ing a d i p l oma in nurs ing . 2. The fac t that the comp l ex i t y sca le had on l y three leve ls of c o m p l e x i t y , a p s y cho l og i c a l phenomenon o f ra t ing towards the cente r may have occur red y i e l d i n g comp l e x i t y scores wh i c h were essen t i a l l y the same. A l s o , this s ca l e was not p i l o t tes ted . 3. The dependent v a r i a b l e — t h e c l i n i c a l pe r fo rmance—was on l y assessed for the f irst ha l f of the semester, and therefore may not r e f l e c t the students ' per formance for the who le semester. 77 Recommendations This study was exploratory in nature and served to identify the degree to which academic grades in Nursing, Psychology, and Biology courses in first semester of the nursing program correlate with the cl inical performance scores of nursing students. The recommendations based on the results of the study will be presented in two parts, recommendations for future research and recommen-dations for nurse educators. The recommendations for future research are as follows: 1. The complexity of the nursing situation be studied further to establish whether the tool used to assess complexity was indeed insensitive in differentiating levels of complexity, or whether the nursing situation itself lacked complexity. Furthermore, the tool to measure the complexity of the nursing situation be refined using four levels rather than three, so that differences in the complexity of the nursing situation are more finely differentiated. 2. A study be conducted with a second group of Semester IV and V students, substituting the theory grades obtained in their first semester Nursing, Psychology, and Biology courses with their third and fourth semester Nursing, Psychology, and Biology courses respectively to control the possible influence of the temporal factor of additional learning experiences. 3 . A study be conducted to determine whether the multiple correlation coefficients obtained utilizing the cl inical scores over a three-week period during the first half of the semester are comparable to the correlation coefficients obtained utilizing the cl inical grades at the end of the semester. The recommendat ions for nurse educators are as fo l l ows: 1. Results of this study be used w i th cau t i on when p rov id ing gu idance to nursing students en ro l l ed in two - yea r nursing programs. 2. Results of this study be used w i th cau t i on when p red i c t i ng c l i n i c a l per formance scores of students en ro l l ed in two - yea r nursing programs from the i r f irst semester grades in N u r s i n g , Psycho logy , and B io l ogy . REFERENCES 79 80 Ahmann , S. J . , & G l o c k , M . D . Eva lua t ing Pupi l G r ow th (2nd e d . ) . Boston; A l l y n and Bacon , I n c . , 1971 . B lood , D. F . , & Budd, W . C . Educa t iona l Measurement and E va l u a t i o n . N e w York: Harper and Row, Publ ishers, I n c . , 1972 . Champney , B. , C h a t f i e l d , K. M . , & S ims, A . Methods of Assessment. A G u i d e for Teachers of Nurses , ed i ted by E l i z abe th Raybou ld . O x f o r d : B l a c kwe l l S c i e n t i f i c P u b l i c a t i o n , 1975 , 155 - 173 . C r o n b a c h , L. J . , et a l . The Dependab i l i t y of Behav iora l Measurements: Theory of G e n e r a l i z a b i l i t y for Scores and Pro f i l es . N e w York : John W i l e y and Sons, I n c . , 1972. F i va r s , G . , & G o s n e l l , D. Nurs ing Eva lua t i on : The Problem and the Process. N e w York: The M a c m i l l a n C o . , 1966. Fuerst, E. V . , & W o l f f , L. Teach ing Fundamentals o f Nurs ing (4th e d . ) . Ph i l a de l ph i a : J . B. L ipp inco t t C o . , 1969. G o n n e l l a , J . S . , G o r a n , M . J . , W i l l i a m s o n , J . W . , & Cotsonas , N . , J r . Eva lua t i on o f Pat ient Ca r e : A n A p p r o a c h . Jou rna l of Ame r i c an M e d i c a l A s so c i a t i o n , 1970, 214 , 2040 -2043" G o r d o n , P. Eva lua t i on : A Tool in Nurs ing S e r v i c e . The Ame r i c an Journa l of N u r s i n g , 1960 , 3 , 3 6 4 - 3 6 7 . G o r e , B. W . The M u l t i v a r i a t e P red i c t i on of C o l l e g e G r a d e Point Ave rage and State Board Examinat ion Scores in Se l ec ted Miss iss ipp i Assoc ia te ree Nurs ing Programs (Doctora l D i sser ta t ion) . M iss i ss ipp i : Un i ve r s i t y of Southern M i s s i s s i pp i , 1973. G o z a , J . T . A n Invest igat ion of the A c adem i c Po t en t i a l , A c a d e m i c A ch i e v emen t , and Persona l i ty of Par t i c ipants in an Assoc ia te Degree Nurs ing Program (Doctora l D i sser ta t ion) . Texas: East Texas State Un i v e r s i t y , 1970. G r e e n , P. E . , & T u l l , D. S. Research for Ma r ke t i ng Dec is ions (2nd e d . ) . N e w Jersey: P r e n t i c e - H a l l , I n c . , 1970. G r o n l u n d , N . E. Measurement and Eva lua t ion in Teach ing (3rd e d . ) . N e w Yo rk : M a c m i l l a n Pub l i sh ing C o . , I n c . , 1976. Ha r t , W . M . The Eva lua t i on of the Performance of Bacca laurea te Students in C l i n i c a l Nu r s i ng (Doctora l D i sser ta t ion) . Ind iana Un i v e r s i t y , M a y 1974. Hay te r , J . An App roach to Laboratory E va l u a t i o n . The Journa l of Nurs ing Educa t i on , 1973 , 112, 1 7 - 2 2 . H e s l i n , P. Eva lua t ing C l i n i c a l Per formance. Nurs ing O u t l o o k , 163, 11 , 3 4 4 - 3 4 5 . — Infante, M . S. The C l i n i c a l Laboratory in Nurs ing Educa t i on . N e w York: John W i l e y and Sons, I n c . , 1 975 . K e r l i n ge r , F. N . , & Pedhazur , E. J . M u l t i p l e Regression in Behav iora l Research. N e w York : H o l t , Rinehart and W in s t on , I n c . , 1973 . Ko rman , M . , & S t ubb l e f i e l d , L. R. M e d i c a l Schoo l Eva lua t i on and Intern-ship Per fo rmance. Journa l of M e d i c a l Educa t i on , 1971 , 4 6 , 6 7 0 - 6 7 3 . K ruge r , M . Eva lua t i on of a Student 's C l i n i c a l Per formance, a V i d e o t a p e . Vancouve r ! Vancouve r Communi ty C o l l e g e , Langa ra , 1978 . Lancaster , A . Nurse Teachers: the Report of an O p i n i o n Su rvey . Belfast: W . & G . Baird L t d . , 1972 . Leaper , D. J . , G i l l , P. W . , S t an i l a nd , J . R., Hor rocks , J . C , & De Domba l , F. T. C l i n i c a l D iagnos t i c Progress: An Ana l y s i s . Brit ish M e d i c a l J o u r n a l , 1973 , 3 , 5 6 9 - 5 7 4 . L i twack , L . , S a ka t a , R., & W y k l e , M . C o u n s e l l i n g , Eva lua t i on and Student Deve lopment in Nurs ing Educa t i on . Ph i l ade l ph i a : W . B. Saunders, 1972 . M a c K a y , R. C . Eva lua t ion of Facu l t y and Students. . . . A Means Towards Fu l l e r Commun i ca t i on and G r ea t e r P roduc t i v i t y . The Journa l o f Nurs ing Educa t i on , 1974, 13, 3 - 7 . M c M a n u s , R. L. The Ef fect of Exper ience on Nurs ing A c h i e v e m e n t . N e w York : Teachers C o l l e g e , Co l umb i a Un i ve r s i t y Bureau of Pub l i c a t i on s , 1949. N a t i o n a l League for N u r s i n g . Eva lua t i on - An O b j e c t i v e A p p r o a c h . N e w York: The League , 1972 . Powers, B. P red i c t i on of Success of Communi ty C o l l e g e Nu r s i ng Students (Doctora l D i sser ta t ion) . A r i z o n a State Un i v e r s i t y , December 1974. Pub l i c a t i on Manua l of the Amer i c an Psycho log i ca l A s s o c i a t i o n . Ma r y l a nd ; Ga ramond/P r i demark Press, I n c . , 1975. 82 Rines, A . R. Eva lua t ing Student Progress in Learn ing the P rac t i ce of N u r s i n g . N e w York : Bureau o f Pub l i c a t i ons , Teachers C o l l e g e , Co l umb i a Un i v e r s i t y , 1963 . Ryden, M . B. The P red i c t i ve V a l u e of a C l i n i c a l Examinat ion of Inter-personal Re la t ionsh ip S k i l l s . Journa l of Nurs ing Educa t i on , 1977 , 1 6 , 5 , 2 7 - 3 1 . S chwee r , J . E . , & G e b b i e , K . M . C r e a t i v e Teach ing in C l i n i c a l Nu r s ing (3rd e d . ) . Sa in t Louis: The C . V . Mosby Company , 1976. Sister M i l l e r , P. A Teach ing P r a c t i c um . Nurs ing O u t l o o k , 1976, 2 4 , 5 , 7 5 2 - 7 5 3 . _ Ta te , B. L. Eva lua t ing the Nurse 's C l i n i c a l Per formance. Nurs ing O u t l o o k , 1962 , 10 , 3 5 - 3 7 . Tay l o r , C . W . , N a h m , H . , Q u i n n , M . , Harms, M . , M u l a i k , J . , & M u l a i k , S . A . Se l e c t i on and Recrui tment of Nurses and Nurs ing Students: A Rev iew of Research Studies and P rac t i ce s . U tah : Un i ve r s i t y of U tah Press, 1963 . Tay l o r , C . W . , N a h m , H . , Q u i n n , M . , Harms, M . , M u l a i k , J . , & M u l a i k , S. A . Report of Measurement and P red i c t i on of Nurs ing Per formance, Part I. Sa l t Lake C i t y : Un i ve r s i t y of Utah Press, J965~. Tho rnd i ke , R. L. , & H a g e n , E. Measurement and Eva lua t ion in Psycho logy and Educa t ion (2nd e d . ) . N e w York : John W i l e y and Sons, I n c . , 1961 . W i e ne r , S. L . , K o r an , L . , M i t c h e l l , P . , Schat tner , G . , F i e r s t e i n , J . , & Ho tchk i s s , E. C l i n i c a l Sk i l l s : Q u a n t i t a t i v e Measurement . N e w Yo rk State Journa l of M e d i c i n e , 1976 , 76 , 6 1 0 - 6 1 2 . W o o d , V . A . A Problem That Won ' t G o A w a y . Internat iona l Nurs ing Rev i ew , 1972, 19 , 3 3 6 - 3 4 3 . A P P E N D I C E S 83 A P P E N D I X A 84 A P P E N D I X B 86 L A N G A R A - 1 Vancouver Conrnunity College, Langara. 87 M E M O to M r s . M . K ruge r , Nurs ing Department from A . L . D a r t n e l l , A c t i n g P r i n c i pa l date January 9 , 1978 subject Research I have very c a r e f u l l y read your proposal and a lso have discussed the matter w i th M r s . Kno r . This memo w i l l g i v e you permiss ion to proceed as ou t l i ned in your p roposa l . If there is any way in wh i c h I can he lp wou ld you p lease let me know . A L D / S B Enc: c . c . M r s . E. Knor A P P E N D I X C 88 89 E X P L A N A T I O N S T O I N S T R U C T O R S Dear Co l l e ague ; I discussed my proposal w i th you at f a cu l t y meet ings in both A p r i l and O c t o b e r , 1977 and wou ld now l i k e to request your p a r t i c i p a t i o n . As you w i l l remember, the study is to determine what proport ion of the va r i an ce in the c l i n i c a l per formance score of the nursing student is cont r ibuted by the grades in the first semester nurs ing , b i o l ogy and psycho logy courses, and the comp l e x i t y of the nursing s i t u a t i o n . The in format ion gathered w i l l he lp nurse educators be cogn i zan t of the e f fec t of such factors on the c l i n i c a l performance grade of the nursing students and a c c o r d i n g l y eva l ua t e the students ' pe r fo rmance . The anecdo ta l records w i l l be typed and coded to ensure c on f i d e n t i a l i t y . I wou ld l i ke to make it ve ry c l e a r that this is not an eva l ua t i on of your a b i l i t y to wr i te anecdo ta l notes, make c l i n i c a l ass ignments, or grade students ' pe r fo rmance . A copy of the comp le ted study w i l l be p rov ided to the nursing department and made a v a i l a b l e to a l l pa r t i c i pan t s . Your pa r t i c i pa t i on w i l l e n t a i l : (1) wr i t i ng anecdo ta l notes about the performance of 4 - 8 pa r t i c i pa t i ng students on the des ignated forms dur ing the weeks of January 2 3 , 30 and February 6 and 13 , 1978 . (2) g i v i n g a score to each anecdo ta l record a c co rd i ng to the c l i n i c a l ob jec t i ves of the student and the scor ing system prov ided to you by the inves t iga to r . (3) de te rmin ing the l e ve l of c omp l e x i t y of the nursing s i tua t ion a c co rd i ng to the c r i t e r i a (see Append i x G ) . (4) vo lun tee r i ng about 30 to 40 minutes for ea ch pa r t i c i pa t i ng student to complete a c t i v i t i e s 1 -3 . (5) a c t i ng as a nursing j udge , to eva lua te the per formance of 10 -14 students on the basis of anecdo ta l records wr i t ten by other instructors from your semester. A P P E N D I X D 91 92 C O N S E N T F O R M Dear Co l l e ague : Please i nd i ca te your w i l l i ngness to pa r t i c i pa te in this study by s ign ing your name in the appropr ia te space p rov ided b e l ow , and as soon as possib le return the consent form to M a r y K ruge r , o f f i ce number A l 6 7 c . You may keep the a t tached in format ion i f you w i s h . I, , agree (OR I, do not wish) to pa r t i c i pa te in the study by Ma r y Kruger t i t l ed : What proport ion of the va r i an ce in the c l i n i c a l per formance score of the nursing student is con t r ibu ted by the grades in the first semester nurs ing , b i o l o g y , and psycho logy courses and by the comp l ex i t y of the nursing s i t u a t i on . I understand that the study w i l l i n vo l ve wr i t i ng anecdotes about the per formance o f pa r t i c i pa t i ng students in my c l i n i c a l a rea for the weeks of January 2 3 , 3 0 , and February 6 and 13 , 1978; de te rmin ing the comp l ex i t y of the nursing s i tua t ion a c co rd i ng to descr ibed c r i t e r i a ; and scor ing the anecdo ta l notes of pa r t i c i pa t i ng students from the semester emp loy ing the course ob jec t i ves and the scor ing system. I have been assured that the f ind ings w i l l not i n f l uence my pos i t ion in the school and that my name w i l l not appear in any wr i t t en repor t . I am aware that I am free to w i thdraw from the study at any t ime . S igned A P P E N D I X E 93 A P P E N D I X F 95 96 C O N S E N T F O R M Dear Student: Please i nd i ca te your w i l l i ngness to pa r t i c i pa te in this study by s ign ing your name in the appropr ia te space be l ow , and return the consent form to Ma r y K ruge r , o f f i c e number A167c as soon as poss ib le . I, , agree (OR I, do not wish) to pa r t i c i pa te in the study by Ma r y Kruger t i t l e d : What proport ion of the v a r i an ce in the c l i n i c a l per formance score of the nursing student is cont r ibu ted by the grades in the f irst semester nurs ing , b i o l o g y , and psycho logy courses, and by the c omp l e x i t y of the nursing s i t u a t i on . I understand that the study w i l l i n vo l v e ob ta in ing my first semester nurs ing , b i o l o g y , and psycho logy grades from the registrar 's o f f i c e and scor ing of the anecdo ta l records about my per fo rmance , anonymous ly , for the weeks of January 23 to February 2 0 , 1978 . I have been assured that the f ind ings w i l l not i n f l uence my standing in the schoo l and that my name w i l l not appear in any wr i t ten repor t . I am aware that I am free to w i thd raw from the study at any t ime . S igned A P P E N D I X G 97' C O M P L E X I T Y O F THE N U R S I N G S I T U A T I O N l eve l 1 leve l 2 l eve l 3 O n a comp l e x i t y sca le ranging from leve l 1 to leve l 3 sketched above , c i r c l e on the r ight the l e ve l that best represents the student's assignment acco rd ing to the c r i t e r i a desc r ibed . C i r c l e on l y O N E leve l l eve l 1 l eve l 2 l eve l 3 Desc r ip t i on of levels 1, 2 and 3 LEVEL 1 LEVEL 2 LEVEL 3 1. a c t i v i t i e s of the student a c t i v i t i e s of the student a c t i v i t i e s of the student Treatments prescr ibed by the D r . a n d / or the nursing team are min ima l in number; the student has had numerous prev ious opportun i t ies to pract ise these sk i l l s and a c t i v i t i e s , and has m in ima l or no oppor tun i ty to pract i se the a c t i v i t i e s and sk i l l s learned in the present semester. Treatments prescr ibed by the D r . a n d / or the nursing team are moderate in number; the student has had numerous oppor tun i t ies to pract ise the sk i l l s and a c t i v i t i e s of the previous semester, and has min ima l /modera te oppo r t un i -t ies to pract ise the sk i l l s and a c t i v i t i e s learned in the present semester. Treatments prescr ibed by the Dr . a n d / or the nursing team are numerous; the student has had m in ima l /modera te opportuni t ies to pract ise the sk i l l s and a c t i v i t i e s learned in the prev ious semester, and has m in ima l /modera te opportuni t ies to pract ise the sk i l l s and a c t i v i t i e s learned in the present semester. LEVEL 1 LEVEL 2 LEVEL 3 2 . cond i t i on of the pa t ien t cond i t i on of the pat ient cond i t i on of the pat ient The pat ient ' s cond i t i on is s tab le and requires min ima l or no mod i f i c a t i o n of the student 's p lan o f c a r e . The pat ient ' s cond i t i on is chang ing and requires moderate mod i f i c a t i on of the student 's p lan of c a r e . The pat ient 's cond i t i on is chang ing rap id l y and requires maximum m o d i -f i c a t i on of the student's p lan of c a r e . 3 . pa t ient exper iences pat ient exper iences pat ient exper iences (a) not enough pat ients on the ward p rov id ing the des i red expe r i ences . (b) too many students on the ward; there fore , l im i ted su i tab le pat ient exper iences for the students of the g i ven semester. (c) i n i t i a l appropr ia te pa t ien t a s s i gn -ment; however , the c l i n i c a l instructor changes the assignment because (1) pat ient transferred to another ward (2) pat ient ' s c ond i t i on has worsened (3) pat ient is d i scharged (4) pat ient has e x p i r e d . a v a i l a b l e pat ient exper iences meet the learn ing needs of students in a g i v en semester; su i tab le s tudent / pa t ien t ra t i o , w i th pat ients who prov ide des ired expe r i ences . the student's i n i t i a l assignment is a cco rd ing to the learn ing needs of the students in a g i v en semester; however , the pat ient 's(s ' ) c ond i t i on changes and the c l i n i c a l instructor dec ides that the student should cont inue to care for the pat ient(s) in co l l abo ra t i on w i th other member(s) o f the nursing team. LEVEL 1 LEVEL 2 LEVEL 3 rev is ions of student's assignment based rev is ions of student's assignment based revis ions of student's assignment based on the instructor 's assessment of the on the instructor 's assessment of the on the instructor's assessment of the student 's ab i l i t i e s / pe r f o rmance student 's ab i l i t i e s /pe r fo rmance student's ab i l i t i e s /pe r fo rmance The assignment requires: (a) in tegra t ion of p rev ious l y acqu i red know ledge ; (b) coo rd ina t i on of nurs ing a c t i v i t i e s ; (c) p rob l em-so l v i ng a c t i v i t i e s : assessment, p l a n n i n g , imp l emen -t a t i o n , and e v a l u a t i o n . Student can not cope w i th the type of assignment for his semester l e v e l . There fore , (a) the c l i n i c a l instructor changes the assignment (b) the c l i n i c a l instructor chooses assignments that she perce ives that the student is ab l e to manage . The assignment requires: (a) in tegra t ion of prev ious ly acqu i red and some newly learned k now -ledge; (b) coo rd ina t i on of nursing ac t i v i t i e s ; (c) f l e x i b i l i t y in the student's o rgan i za t i on and p lan of care; (d) p rob lem-so lv ing ac t i v i t i e s : assessment, p l ann i ng , imp l emen -t a t i o n , and e v a l u a t i o n . Student can cope w i th assignments for his semester l e v e l . The assignment requires: (a) in tegrat ion of p rev ious ly and new ly learned knowledge; (b) coord ina t ion of nursing a c t i v i t i e s ; (c) maximum f l e x i b i l i t y in the student's o rgan i za t i on and p lan of care; (d) constant p rob lem-so lv ing ac t i v i t i e s : assessment, p l a nn i ng , imp lementa t ion , and e v a l u a t i o n . Student can not cope w i th the type of assignments for his semester l e v e l . Therefore c l i n i c a l instructor s p e c i -f i c a l l y chooses the assignment to ident i f y the student's strengths and weaknesses. Student can cope w i th the type of assignments for his semester l eve l and c l i n i c a l instructor s p e c i f i c a l l y chooses the assignment to mot iva te the student. 101 To determine the comp l e x i t y l eve l of the student 's ass ignment, you must f o l l ow the sequent ia l procedure b e l o w . (1) Read the va r i ab les descr ibed under leve ls 1, 2 , and 3 . (2) Determine how many of the va r i ab les in each l eve l app l y to the student 's ass ignment. (3) Dec i de on the l eve l that best represents the student 's ass ignment. Below are three poss ib le s i tuat ions and the appropr ia te dec i s i on in e a c h . (a) i f a l l the var iab les from one l eve l app l y to the student 's ass ignment, then that is the appropr ia te l e v e l . (b) i f two of the var iab les from one l e ve l and one va r i ab l e from each of the other leve ls app l y to the student 's ass ignment, then the dec i s i on depends on the pa t t e rn . There are three s i tuat ions under this ca tegory: (1) i f two va r i ab les from leve l 1, and one va r i ab l e from each o f the leve l s 2 and 3 a p p l y , then the dec i s i on is towards the cent ra l t endency , thus l e ve l 2 . (2) i f one va r i ab l e from each of the leve ls 1 and 3 , and two var iab les from leve l 2 a p p l y , then the dec i s i on is l eve l 2 . (3) i f one va r i ab l e from each of the leve ls 1 and 2 , and two va r i ab les from leve l 3 app l y to the student 's ass ignment, then the dec i s i on is l eve l 3 . (c) i f two of the va r i ab les from one leve l and two var iab les from another l eve l app l y to the student 's ass ignment, then the dec i s i on is the h igher l e v e l . For e xamp l e , j f there are two var iab les from l eve l 2 , and two va r i ab les from l eve l 3 , then the dec i s i on is l eve l 3 . (4) C i r c l e the l eve l d e c i d ed in step 3 . A P P E N D I X H 102 A N E C D O T A L R E C O R D Semester Instructor Number Date Student Number Desc r ip t i on of the S i tua t ion Desc r ip t i on of the Student 's Behav ior Performance Score A P P E N D I X I 104 S C O R I N G SYSTEM O n a sca le of 1 to 9 , 1 represent ing unsat is factory to 9 representing superior per formance, c i r c l e on the r ight the number that best descr ibes the student 's ach ievement of the 6 major ob j e c t i v e s . F P C B A 1 2 3 4 5 6 7 8 9 Unsat i s fac tory Ma rg i n a l Ave rage Above Superior Ave rage SEMESTER II Ob j e c t i v e s C i r c l e O N E 2 3 4 5 6 7 8 9 1. U t i l i z e the prob lem so lv ing approach to prov ide systematic c a re . 2 3 4 5 6 7 8 9 2 . Commun ica te w i t h others in an e f f e c t i v e and systematic manner. 2 3 4 5 6 7 8 9 3 . V a l u e i nd i v i dua l s as un ique and hav ing wo r t h . 2 3 4 5 6 7 8 9 4 . Demonstrate o r gan i z a t i ona l s k i l l s . 2 3 4 5 6 7 8 9 5 . Assume his ro le as a member of the hea l th t eam. 2 3 4 5 6 7 8 9 6 . Demonstrate respons ib i l i t y and a c c oun t ab i l i t y for his own behav io r . 2 3 4 5 6 7 8 9 No t e : The above are the te rmina l ob jec t i ves under wh i ch you have spec i f i c ob jec t i ves for your c l i n i c a l course . Us ing the c r i t e r i a under the scor ing system descr ibed in Append i x I ranging from "unsat is factory" to "super io r " a l ong w i t h the corresponding letter grades used at the c o l l e ge w i l l enab le you to se lec t the appropr ia te f igure rang ing from 1 to 9 . S C O R I N G SYSTEM O n a s ca l e of 1 to 9 , 1 represent ing unsat i s factory to 9 represent ing super ior per formance, c i r c l e on the r ight the number that best descr ibes the student 's ach ievement of the 6 major ob j e c t i v e s . F P C B A 1 8 Unsat i s fac tory SEMESTER III Ma r g i n a l O b j e c t i v e s Ave rage Above Ave rage Superior C i r c l e O N E 1 . U t i l i z e the nursing process to p rov ide systemat ic nursing c a r e . 2 . Commun ica te w i th others in an e f f e c t i v e and systemat ic manner. 3 . V a l u e i nd i v i dua l s as un ique and hav ing wo r t h . 4 . Demonstrate o rgan i za t i ona l s k i l l s . 5 . Demonstrate a c c oun t ab i l i t y for the care of his assigned pat ients . 6 . Demonstrate respons ib i l i t y for his behav i o r . No t e : The above are the termina l ob jec t i ves under wh i ch you have spec i f i c ob jec t i ves for your c l i n i c a l course, Us ing the c r i t e r i a under the scor ing system descr ibed in Append i x I ranging from "unsat i s fac tory" to "super io r " a long w i th the corresponding le t ter grades used at the co l l e ge w i l l enab le you to se lec t the appropr ia te f igure ranging from 1 to 9 . 2 2 2 2 2 2 2 3 3 3 3 3 3 3 4 4 4 4 4 4 4 5 5 5 5 5 5 5 6 6 6 6 6 6 6 7 7 7 7 7 7 7 8 8 8 8 8 8 8 9 9 9 9 9 9 9 S C O R I N G S Y S T E M O n a sca le o f 1 to 9 , 1 represent ing unsat is factory to 9 represent ing superior per formance, c i r c l e on the r ight the number that best descr ibes the student 's ach ievement of the 6 major ob j e c t i v e s . F P C B A 1 8 Unsat i s fac tory SEMESTER IV Ma r g i n a l Ob j e c t i v e s Ave rage Above Average C i r c l e O N E 1 . U t i l i z e the problem so l v i ng approach to prov ide systematic nursing c a r e . 2 . Commun ica te w i t h others in an e f f e c t i v e and systematic manner. 3 . V a l u e ind iv idua l s as un ique and hav ing wor th . 4 . Demonstrate o rgan i z a t i ona l s k i l l s . 5 . Assume his ro le as a member of the hea l th team. 6 . Demonstrate respons ib i l i t y through his behav io r . 2 2 2 2 2 2 2 3 3 3 3 3 3 3 Superior 4 4 4 4 4 4 4 5 5 5 5 5 5 5 6 6 6 6 6 6 6 7 7 7 7 7 7 7 8 8 8 8 8 8 8 9 9 9 9 9 9 9 N o t e : The above are the te rmina l ob jec t i ves under wh i ch you have spec i f i c ob jec t i ves for your c l i n i c a l course. Us ing the c r i t e r i a under the scor ing system descr ibed in Append i x I ranging from "unsat i s fac tory" to "super io r " a long w i th the corresponding letter grades used at the co l l e ge w i l l enab le you to se lec t the appropr ia te f igure rang ing from 1 to 9 . S C O R I N G S Y S T E M O n a sca le of 1 to 9 , 1 represent ing unsat is factory to 9 represent ing superior per formance, c i r c l e on the r ight the number that best descr ibes the student 's ach ievement of the 6 major ob j e c t i v e s . F P C B A 1 8 Unsat i s fac tory SEMESTER V Ma r g i n a l O b j e c t i v e s Ave rage Above Average C i r c l e O N E 1. U t i l i z e the problem so l v ing approach to prov ide ca re . 2 . Commun ica te w i th others in an e f f e c t i v e and systematic manner. 3 . V a l u e i nd i v i dua l s as un ique and hav ing wo r th . 4 . Demonstrate o rgan i z a t i ona l s k i l l s . 5 . Assume his ro le as a member of the hea l th t eam. 2 2 2 2 2 2 3 3 3 3 3 3 Superior 4 5 6 4 5 6 4 5 6 4 5 6 4 5 6 4 5 6 7 7 7 7 7 8 9 8 9 8 9 8 9 8 9 7 8 9 3 4 5 6 7 8 9 6 . Demonstrate respons ib i l i t y for his own behav io r . No t e : The above are the te rm ina l ob jec t i ves under wh i ch you have spec i f i c ob jec t i ves for your c l i n i c a l course, Us ing the c r i t e r i a under the scor ing system descr ibed in Append i x I ranging from "unsat i s fac tory" to "super io r " a long w i th the corresponding letter grades used at the co l l e ge w i l l enab le you to se lec t the appropr ia te f igure rang ing from 1 to 9 . Categor ies Letter G r a d e Nume r i c a l G r a d e C r i t e r i a Unsa t i s -fac tory 1 2 The student fa i l s to meet the course ob jec t i ves ; (1) requires consistent de t a i l ed exp l ana t i on of p r i n c i p l e s , concepts , and procedures; (2) requires consistent teacher or staff superv is ion; (3) fa i l s to app l y the p rob lem-so lv ing process; (4) is unab le to demonstrate i n i t i a t i v e i n meet ing ob jec t i ves ; (5) is incons istent in app l y ing prev ious ly learned knowledge; (6 ) is f requent l y inconsistent and unsafe in his per formance of communica t ion and psychomotor s k i l l s . Ma r g i n a l The student is inconsistent in meet ing the course ob jec t i ves : (1) requires frequent de ta i l ed exp lana t i on of p r i n c i p l e s , concepts , and procedures; (2) requires frequent teacher or staff superv is ion; (3) o c c a s i o na l l y app l ies the p rob lem-so lv ing process; (4) o c c a s i ona l l y demonstrates appropr iate i n i t i a t i v e in meet ing ob jec t i ves ; (5) o c c a s i o na l l y app l ies prev ious ly learned knowledge; (6 ) is usua l l y inconsistent and unsafe in his per formance of communica t ion and psychomotor s k i l l s . Def in i t ions: Usua l l y - cus tomar i l y , hab i tua l l y Frequent ly - happen ing at short in te rva l s , o f ten occu r r i ng O c c a s i o n a l l y - happen ing i r r egu l a r l y , coming now and then Rarely - se ldom, not often Appropr ia te - s p e c i a l l y s u i t ab l e , proper , f i t Consistent - marked by r egu l a r i t y , or steady con t i nu i t y throughout; showing no s i gn i f i c an t c hange , unevenness, or con t rad i c t i on In i t i a t i ve - a b i l i t y to o r ig ina te ac t i ons , to i n i t i a t e des i rab le steps Ma rg i na l - c lose to the lower l im i t of q u a l i f i c a t i o n or a c c ep t ab i l i t y o NO Categor ies Letter G r a d e N u m e r i c a l G r a d e C r i t e r i a Ave rage 4 5 The student meets a l l of the ove ra l l course ob jec t i ves : (1) requires occas iona l exp l ana t i on of p r i n c i p l e s , concepts , and procedures; (2) requires occas iona l teacher or staff superv is ion; (3) usua l l y app l ies the p rob lem-so lv ing process; (4) usua l l y demonstrates appropr iate i n i t i a t i v e in meet ing ob jec t i ves ; (5) usua l l y app l ies prev ious ly learned knowledge; (6) is usua l l y consistent and safe in his performance of communicat ion and psychomotor sk i I Is. Above Ave rage 6 7 The student meets a l l of the ove ra l l and most of the spe c i f i c ob jec t i ves : (1) requires occas iona l exp l ana t i on of p r i n c i p l e s , concepts , and procedures; (2) requires appropr iate teacher or staff superv is ion; (3) f requent l y app l ies the p rob lem-so lv ing process; (4) f requent ly demonstrates appropr iate i n i t i a t i v e in meet ing ob jec t ives; (5) f requent ly app l ies prev ious ly learned knowledge; (6) is consistent and safe in his performance of commun ica t ion and psychomotor s k i l l s . Def in i t ions: Usua l l y - cus tomar i l y , hab i t ua l l y Frequent ly - happen ing at short in te rva l s , often occur r ing O c c a s i o n a l l y - happen ing i r r egu la r l y , coming now and then Rarely - se ldom, not often Appropr ia te - s pe c i a l l y su i t ab l e , proper , f i t Consistent - marked by r egu l a r i t y , or steady con t inu i t y throughout; showing no s i gn i f i can t c hange , unevenness, or con t rad i c t i on In i t i a t i ve - a b i l i t y to o r i g ina te ac t i ons , to i n i t i a t e des i rab le steps Ma rg i na l - c lose to the lower l im i t of q ua l i f i c a t i o n or a c c ep t ab i l i t y ~ Categor ies Letter G r a d e N u m e r i c a l G r a d e C r i t e r i a The student meets a l l o f the ove ra l l and Def in i t ions: s pe c i f i c ob jec t i ves : (1) rare ly requires exp l ana t i on of p r i n c i p l e s , Usua l l y - cus tomar i l y , hab i t ua l l y concep t s , and procedures; (2) requires appropr ia te teacher or staff Frequent ly - happen ing at short 8 9 superv is ion; in te rva l s , of ten occu r r i ng Super ior A (3) cons is tent ly app l ies the p rob lem-so lv ing process; O c c a s i o n a l l y - happen ing (4) cons is tent ly demonstrates appropr iate i r r egu la r l y , coming now and then i n i t i a t i v e in meet ing ob jec t ives; (5) cons is tent ly app l ies prev ious ly learned Rarely - se ldom, not often knowledge; (6) is consistent and safe in his performance Appropr ia te - s p e c i a l l y s u i t ab l e , of commun ica t i on and psychomotor s k i l l s . proper , f i t Consistent - marked by r egu l a r i t y , or steady con t i nu i t y throughout; showing no s i gn i f i can t c hange , unevenness, or con t r ad i c t i on In i t i a t i ve - a b i l i t y to o r ig ina te ac t i ons , to i n i t i a t e des i rab le steps Ma rg i na l - c lose to the lower l im i t of q ua l i f i c a t i o n or a c c ep t ab i l i t y A P P E N D I X J 112 113 G E N E R A L I Z A B I L I T Y A N A L Y S I S The theory of g e n e r a l i z a b i l i t y proposed by L. J . C ronbach et a l . (1972) employs the analys is o f va r i ance to determine coe f f i c i en t s of r e l i a b i l i t y . Mo re s p e c i f i c a l l y , this theory is mot iva ted by the f o l l ow i ng quest ions: (1) how can a person's score best be es t ima ted? and (2) how large is the error that arises from incomp le te observa t ions? The term " g e n e r a l i z a b i l i t y " refers to the gene r a l i z a t i o n from observed (sample) scores, to universe scores (persons' mean scores over a l l a c cep t ab l e observa t ion) . C ronbach et a l . state that the un iversa l score "emphasizes that the invest igator is mak ing an in fe rence from a sample o f observed d a t a , and a lso that there is more than one universe to wh i c h he might g e n e r a l i z e " (p. 18) . In a g e n e r a l i z a b i l i t y (G) s tudy, the researcher co l l e c t s data from wh i c h estimates can be made of the components of va r i ance for measurements made for a ce r ta in p rocedure . Furthermore, the researcher obtains two or more scores for the i nd i v i dua l by observ ing h im under d i f fe rent cond i t i ons , and then examines the cons is tency of these scores. It is necessary here to de f i ne some of the essent ia l te rm ino logy of the theory of g e n e r a l i z a b i l i t y , to f a c i l i t a t e the desc r ip t i on of its a pp l i c a t i o n in the present s tudy. A n y observat ions made are descr ibed in terms of cond i t i on s , de f i ned as "the task or st imulus presented, the day and hour , the sett ing in wh i c h the observat ion is made, the observer , and possib le add i t i ona l features of the operat ions pe r fo rmed" (Cronbach et a l . , p . 17). The term f a c e t , howeve r , is used to refer to cond i t ions of a ce r ta in k i n d . The face ts , a l one or in c omb i na t i on , de f ine the universe in ques t i on . There fore , a un iverse of observat ions cou ld be cha ra c t e r i z ed w i th respect to one , two , or more facets . The theory of g e n e r a l i z a b i l i t y makes the f o l l ow i ng assumptions: (1) cond i t ions of observat ions are not necessar i l y p a r a l l e l ; (2) cond i t ions (pa r t i cu la r l y test items) are randomly supp l ied from a un i ve r se , or a c co rd i ng to a s t ra t i f i ed des ign; and (3) two or more facets may be ana l y z ed s imu l taneous l y . 114 G e n e r a l i z a b i l i t y ana lys i s prov ides est imate components o f those facets represented in the exper imenta l de s i gn . These est imates of v a r i an ce components are then used to determine the c oe f f i c i e n t of g e n e r a l i z a b i l i t y . F i n a l l y , the c oe f f i c i e n t of g e n e r a l i z a b i l i t y (Ep 2 ) is determined by the " ra t i o of the un i ve r se -score va r i ance to the expec ted observed-score v a r i a n c e — t h e va r i an ce l i k e l y to be obta ined under a ce r t a i n exper imenta l p l a n " (Gronbach et a l . , p. 17). In the nota t ion E p 2 , E stands for expec ted v a l u e , and p 2 is the squared co r re l a t i on of observed score w i th un iverse s co re . The va l ue of the G c oe f f i c i e n t ranges from ze ro to o n e , and it expresses how w e l l the observat ions are l i k e l y to loca te the i n d i v i d ua l r e l a t i v e to other members of the popu l a t i o n . In this s tudy , the B M D 0 8 V computer program was used to a na l y z e the d a t a . This program generated est imates of va r i ance components (var iances o f hypothes i zed components of an observed sco re) , wh i c h were u t i l i z e d in c a l c u l a t i n g g e n e r a l i z a b i l i t y c oe f f i c i e n t s . This study was ma in l y conce rned w i th the a c cu r a c y of g ene r a l i z i n g over: (1) rat ings of three nursing judges, and (2) rat ings gathered dur ing three d i f fe rent weeks . The study emp loyed a nested des ign w i th three f ace t s . These were: (1) Students , nested w i t h i n instructors (S; l ) , (2) Judges ( J ) , and (3) Weeks (W) The des ign of the study was (S:l) x J x W . The d iagram be low shows s chema t i c a l l y the forms o f ove r l app ing va r i an ce s . \ Instructors 0) Students \ (S Judges uag (J) 115 A n exam ina t i on o f the des ign of the study i den t i f i ed 11 sources of v a r i an ce ( i . e . , components) . These were: instructors (I); weeks (W); students, nested w i t h i n instructors (S:l); judges (J); i n s t ruc to r -by -week i n te rac t i on (IxW); i n s t ruc to r -by - judge i n te rac t i on (IxJ); w e e k - b y - j u d g e i n te rac t i on (WxJ); s t uden t - by -week i n te rac t i on (SxW); s tuden t -by - j udge i n te rac t i on (SxJ); i n s t r u c t o r - b y -week -by - j udge i n te rac t i on ( IxWxJ) ; a n d , f i n a l l y , s t uden t -by -w e e k - b y - j u d g e i n te rac t i on ( S ; l xWx J ) . F i n a l l y , the g e n e r a l i z a b i l i t y coe f f i c i en t s were c a l c u l a t e d w i th the f o l l ow i ng formula: a 2 (p) + ^ ' i ) 0 2 - ( M * ) A?) + (Vn'i) ° 2 (P i/I*) + ° 2 & va r i ance component for persons. (In the s tudy , e i ther judges or weeks) number of cond i t ions for face t i (items) v a r i an ce component for the persons-by- i tems in te rac t ions for a f i x ed set of i tems. (In this s tudy , students nested w i t h i n instructor) the error , wh i c h inc ludes p i , e components of va r i ance The numerator and the denominator represent the un iverse-score va r i ance and the expec ted observed-score v a r i a n c e , r e spe c t i v e l y . whe r e , cr zp = a 2 (P i/I*) = "a2S A P P E N D I X K 116 117 S T A N D A R D I Z A T I O N PROCEDURE The f o l l ow i ng steps e xp l a i n the procedure used to s tandard i ze the raw c l i n i c a l scores g i v en by the c l i n i c a l instructor and two other nursing judges. For steps 1 to 8 refer to F igure 1. (1) The means o f each judge was compu ted . (2) The va r i an ce of each judge was computed . (3) The standard dev i a t i o n of each judge was compu ted . (4) N e x t , from the score of each student in Co l umn was subtracted the mean of , and subsequent ly , that number was d i v i d ed by the standard dev i a t i o n (SD) of J ] . For e x amp l e , the s tandard i zed score of student 303 g i v en by Judge 1 was c a l c u l a t e d by this f o rmu l a . 3 . 5 - 5 . 75 S D ( J j ) (5) Us ing the above method , the other two scores g i v en to these students were adjusted for Judges 2 and 3 . For e x amp l e , 4 . 0 - 5 . 5 and 5 . 0 - 5 . 3 8 were the s tandard i zed S D ( J 2 ) S D ( J 3 ) scores o f student 303 for Judges 2 and 3 . (6) The poo led mean s tandard i zed score of each student was then c a l c u l a t e d by add ing together the three s tandard i zed scores g i ven to her by Judges 1, 2 , and 3 , and d i v i d i n g this sum by th ree . For e x amp l e , the s tandard i zed score of student 303 across the three judges was c a l c u l a t e d by this formula: , 3 . 5 - 5 . 7 5 , , 4 . 0 - 5 . 5 N , 5 . 0 - 5 . 3 8 N  ( SD ( J ] ) ' + ( S D ( J 2 ) } + ( S D ( J 3 ) } 3 (7) The above computat ions were repeated to s tandard i ze the c l i n i c a l scores o f Week 3 of both Semesters II and III students. SCORES G I V E N BY THE J U D G E S F O R THE S T U D E N T S I N SEMESTER III J U D G E S (J) S C O R E S T U D E N T N O . J l J2 J 3 303 3 . 5 4 . 0 5 . 0 304 4 . 5 4 . 5 5 . 0 305 6 . 5 6 . 0 6 . 0 306 7 . 0 6 . 0 5 . 0 307 7 . 0 6 . 0 6 . 0 308 8 . 0 6 . 0 6 . 0 309 6 . 0 6 . 0 5 . 5 310 4 . 0 5 . 0 4 . 0 311 5 . 0 5 . 0 5 . 0 312 6 . 0 6 . 0 7 . 0 313 6 . 5 6 . 0 6 . 0 314 5 . 0 5 . 5 4 . 0 X . 5 . 7 5 5 . 5 5 . 3 8 F igure 1 A P P E N D I X L 119 120 STEPWISE MULT IPLE R E G R E S S I O N A N A L Y S I S The BMD:02R computer program was emp loyed to do the stepwise mu l t i p l e regression ana l y s i s . This program pr inted out the f o l l ow i ng stat is t ics: the mean and standard dev i a t i o n of each va r i ab l e ; the c ova r i an ce and c o r r e -l a t i on matr ices; the ana lys is of v a r i an ce tab le as each independent v a r i ab l e entered the regression equat ion; the mu l t i p l e co r r e l a t i on c oe f f i c i e n t (R) and its standard error of est imate for ea ch step; the in te rcept constant , the pa r t i a l regression weights (beta we igh t s ) , the i r standard error and l e ve l of s i gn i f i c ance ; and f i n a l l y , the coe f f i c i en t s o f de te rmina t ion ( R 2 ) , the semipar t i a l cor re la t ions and the i r l eve l of s i g n i f i c a n c e . The c oe f f i c i e n t of mu l t i p l e co r re l a t i on (R) is de f i ned by Ke r l i nge r and Pedhazur (1973) as the product moment co r r e l a t i on of the p red i c ted va r i ab l e ( Y 1 ) — w h i c h is a l inear comb ina t i on of the independent va r iab les (X ' s )—and the observed Y . The va lues of R range from ze ro to one , and when squared ( R 2 ) , and mu l t i p l i e d by 100 y i e l d the percent o f v a r i an ce of the dependent v a r i ab l e (Y) wh i c h is e xp l a i ned by the l i nea r comb ina t i on of the independent v a r i a b l e s . The omnibus F s ta t i s t i c is c a l c u l a t e d as each independent va r i ab l e enters the regression equa t i on . This s ta t i s t i c is c a l c u l a t e d by u t i l i z i n g the summary da ta from the A N O V A t a b l e . The F s tat i s t i c was c a l c u l a t e d w i th the f o l l ow i ng formula: ^ r eg r e s s i on /d f j SS res idua l /d f2 degrees of freedom assoc iated w i th sums of squares of regression w h i c h is equa l to K , the number of independent va r i ab les in the regression equa t i o n . N - K - 1, where N is equa l to the sample s i ze This F s ta t i s t i c was used to test the s i gn i f i c an ce of the mu l t i p l e co r re l a t i on c oe f f i c i e n t (R), as ea ch new va r i ab l e entered the regression equa t i on . In a d d i t i o n , at each step the standard error o f est imate ( S E e s t ) of R was c a l c u l a t e d , wh i c h is the standard dev i a t i o n of the res iduals in Y at a pa r t i cu l a r step in the ana l y s i s . where d f ] = d f 2 = 121 The formula for the S E _ s t was as fo l lows: / ^ r e s i d u a l / N - K - 1 This s ta t i s t i c is an index of the d ispers ion of the p red i c ted Y measure (Y 1 ) about the regression l i n e , and is used to e va l ua t e the est imate o f the dependent v a r i a b l e (Y) based on the independent v a r i ab l e s . For e x amp l e , i f the S E _ s t of R is larger when compared to the standard dev i a t i on of the dependent v a r i a b l e ( Y ) , then the est imate of Y on the basis of the independent va r i ab les is poor . The beta weights (b's) are the sample pa r t i a l regression coe f f i c i en t s wh i c h i nd i ca t e the r e l a t i ve importance of the independent va r i ab l es (X's) in mak ing the p red i c t ions to the dependent va r i ab l e ( Y ) . The standard error of regression weights is the s tat i s t i c wh i c h ind i ca tes the v a r i a b i l i t y of the errors and prov ides a measure w i t h wh i c h to compare the stat is t ics whose s i g n i f i c a n ce are be ing tes ted . The formula used to c a l c u l a t e the latter is: S E b l ' (SEes t ) 2 S S X i ( l - R 2 1 . 2 3 ) where . SE^ = Standard error of beta we igh t of independent v a r i ab l e ^ number 1 S S ^ _ Sums of squares of independent v a r i a b l e number 1 o R^l . 2 3 = Square mu l t i p l e co r r e l a t i on c oe f f i c i e n t between the independent v a r i ab l e number 1 used as a dependent v a r i a b l e , and the remain ing independent va r i ab l es 2 and 3 The pa r t i a l co r r e l a t i on is used when s ta t i s t i ca l cont ro l of v a r i an ce is de s i r ed . A t each step of the mu l t i p l e regression ana l y s i s , the pa r t i a l r's are c a l c u l a t e d wh i c h show the co r r e l a t i on between two va r i ab les w i th the th i rd v a r i a b l e he ld constant . F i n a l l y , the semipar t i a l co r r e l a t i on is c a l c u l a t e d . U n l i k e the pa r t i a l r 's, the semipar t i a l r's represent the co r r e l a t i on between two var iab les w i t h the i n f l uence of another va r i ab l e ( s ) , removed from on l y one of the va r i ab les be ing c o r r e l a t ed . Mo re s p e c i f i c a l l y , the square of this s ta t i s t i c te l l s the un ique con t r i bu t i on to the v a r i an ce o f the dependent v a r i ab l e that each independent v a r i a b l e adds after the va r i ance con t r i bu t i on of the p reced ing var iab le(s) is accoun ted for; howeve r , this app l i e s on l y to the pa r t i cu l a r order in wh i c h the independent va r i ab les enter the regression equa t i o n . 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            data-media="{[{embed.selectedMedia}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0100219/manifest

Comment

Related Items