Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A Study of factors affecting clinical performance grades of nursing students Kruger, Mary Boghos 1980

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1980_A5_7 K78.pdf [ 4.78MB ]
Metadata
JSON: 831-1.0100219.json
JSON-LD: 831-1.0100219-ld.json
RDF/XML (Pretty): 831-1.0100219-rdf.xml
RDF/JSON: 831-1.0100219-rdf.json
Turtle: 831-1.0100219-turtle.txt
N-Triples: 831-1.0100219-rdf-ntriples.txt
Original Record: 831-1.0100219-source.json
Full Text
831-1.0100219-fulltext.txt
Citation
831-1.0100219.ris

Full Text

C- I  A S T U D Y O F FACTORS A F F E C T I N G C L I N I C A L P E R F O R M A N C E GRADES OF N U R S I N G STUDENTS  M A R Y B O G H O S KRUGER B . S . N . , University of British C o l u m b i a , 1973  A THESIS SUBMITTED I N PARTIAL F U L F I L M E N T O F THE REQUIREMENTS FOR THE DEGREE O F MASTER O F S C I E N C E I N N U R S I N G in THE F A C U L T Y O F G R A D U A T E STUDIES (School of Nursing)  We accept this thesis as conforming to the required standard  THE U N I V E R S I T Y O F BRITISH C O L U M B I A February 1980 ©  Mary Boghos Kruger, 1980  \  In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of B r i t i s h Columbia, I agree that the Library shall make i t freely available for reference and study. I further agree that permission for extensive copying of this  thesis  for scholarly purposes may be granted by the Head of my Department or by his representatives.  It is understood that copying or publication  of this thesis for financial gain shall not be allowed without my written permission.  Department of The University of B r i t i s h Columbia 2075 Wesbrook Place Vancouver, Canada V6T 1W5  DE-6  BP 75-51 1 E  ABSTRACT A study o f factors a f f e c t i n g c l i n i c a l p e r f o r m a n c e grades of nursing students  This study i n v o l v e d a n i n v e s t i g a t i o n of factors a f f e c t i n g c l i n i c a l p e r f o r m a n c e grades of first a n d s e c o n d y e a r nursing students in a t w o - y e a r D i p l o m a p r o g r a m . The purpose o f the study was t o i n c r e a s e k n o w l e d g e o f factors w h i c h may c o r r e l a t e w i t h c l i n i c a l p e r f o r m a n c e of nursing students; r e l a t e d g o a l s w e r e to assist nurse e d u c a t o r s in p r e d i c t i n g student c l i n i c a l p e r f o r m a n c e and p r o v i d i n g g u i d a n c e accordingly. The study was p l a n n e d to test the f o l l o w i n g hypotheses: 1.  G r a d e s in N u r s i n g , B i o l o g y , and P s y c h o l o g y courses d u r i n g the first  semester of the nursing program a c c o u n t for a s i g n i f i c a n t p r o p o r t i o n of the v a r i a n c e i n c l i n i c a l p e r f o r m a n c e scores of students; 2.  C o m p l e x i t y o f the nursing s i t u a t i o n a c c o u n t s for a n a d d i t i o n a l s i g n i f i c a n t  p r o p o r t i o n of the v a r i a n c e in c l i n i c a l p e r f o r m a n c e scores o f s t u d e n t s . The study p o p u l a t i o n c o n s i s t e d o f 5 9 first and s e c o n d y e a r nursing s t u d e n t s .  The  d e p e n d e n t v a r i a b l e was the a v e r a g e of the c l i n i c a l p e r f o r m a n c e scores o f the students g i v e n by the c l i n i c a l instructors (who w r o t e a n d s c o r e d the a n e c d o t a l records) a n d t w o nursing judges (who s c o r e d the w r i t t e n a n e c d o t a l r e c o r d s ) .  The  i n d e p e n d e n t v a r i a b l e s w e r e the grades in first semester N u r s i n g , B i o l o g y , a n d P s y c h o l o g y ; and the c o m p l e x i t y o f the nursing s i t u a t i o n . ii  Data for the study were collected over a three-week period.  This phase included:  (a) writing of anecdotal records of students' c l i n i c a l performance by their instructors, and subsequently scoring of these records by the c l i n i c a l instructor and two nursing judges using the scoring instrument; (b) assessing the level of complexity of the students' c l i n i c a l assignments by their c l i n i c a l instructors, using the complexity of the nursing situation instrument; and (c) obtaining the students' first semester Nursing theory, Psychology, and Biology grades from their permanent records. The c l i n i c a l instructors and the nursing judges were trained in the writing and scoring of anecdotal records, and in determining the complexity of the students' assignments by utilizing a 10 minute videotape prepared by the investigator depicting the performance of a nursing student in a simulated c l i n i c a l situation. Analyses of the data were carried out in two phases.  Phase one included product  moment correlational analysis and general izabi I ity analysis to determine the reliability of the c l i n i c a l scores.  Phase two included stepwise multiple regression  analysis to determine the predictors of c l i n i c a l scores. The findings of the study showed that the inter-rater correlations among the c l i n i c a l instructors and two nursing judges were moderately strong (.58 to .84). Likewise generalizability analysis showed that the c l i n i c a l scores were reliable. The data on complexity of the nursing situation showed very little variability in each of the four semesters and were therefore deleted from the regression analysis. The regression analysis showed that in Semesters II and III 51 and 76 percent of the variance in c l i n i c a l performance scores could be accounted for by the grades  iii  in first semester Nursing, Psychology, and Biology.  However, in Semesters IV  and V the amount of variance accounted for by the independent variables was not significant.  Thus, hypothesis 1 is accepted in the case of Semester II and  students, and rejected in the case of Semester IV and V students.  iv  III  TABLE O F  CONTENTS Page  ABSTRACT  ii  LIST O F T A B L E S  x  LIST O F F I G U R E S  xii  ACKNOWLEDGEMENTS  xiii  CHAPTER I.  INTRODUCTION  1  Purpose o f the S t u d y  5  S t a t e m e n t o f the P r o b l e m  6  Hypotheses  6  Variables  6  S i g n i f i c a n c e o f the S t u d y  7  Operational Definitions  9  Assumptions II.  11  REVIEW O F RELATED LITERATURE  12  Introduction  12  Educational Evaluation  12  C l i n i c a l P e r f o r m a n c e E v a l u a t i o n Versus C l a s s r o o m Evaluation  15  O b s e r v a t i o n a l Techniques in E v a l u a t i o n A n e c d o t a l Record T e c h n i q u e o f E v a l u a t i o n v  21 22  CHAPTER  P  a  9  e  P r e d i c t i o n o f A c a d e m i c P e r f o r m a n c e Versus C l i n i c a l Performance  25  The C r i t e r i o n Problem  25  P r e d i c t i v e Studies  26  Summary III.  29  RESEARCH M E T H O D O L O G Y  31  Introduction  31  Sampling  31  Institution  31  Semesters  32  Weeks  32  Instructors  33  Students .  34  Semester II s a m p l e  34  Semester III s a m p l e  35  Semester IV s a m p l e  35  Semester V s a m p l e  35  Confidentiality  36  Instrumentation  36  Complexity Scale  36  A n e c d o t a l Record  37 vi  CHAPTER  Page Scoring system  37  Procedure. . .  38  Development of Videotape  38  The Training of C l i n i c a l Instructors and Judges  39  Data C o l l e c t i o n  40  Method of Analysis  IV.  41  Phase O n e  41  Phase Two  42  RESULTS O F A N A L Y S I S  44  Complexity of the Nursing Situation  44  Inter-Rater Reliabilities of C l i n i c a l Scores  45  Correlational Analyses  45  Semester II  45  Semester III  45  Semester IV  45  Semester V  45  G e n e r a l i z a b i l i t y Analyses  50  Semester II  50  Semester III  50  Semester V  54  Correlation Coefficients Versus G e n e r a l i z a b i l i t y Coefficients  54  Regression Analyses  57 vii  CHAPTER  Page Preliminary Analyses:  Raw C l i n i c a l S c o r e s Versus  S t a n d a r d i z e d C l i n i c a l Scores  57  Semester II  57  Semester III  57  Final Analyses:  Raw C l i n i c a l Scores  60  Semester II  60  i . . . .  Semester III  V.  62  Semester IV  64  Semester V  64  SUMMARY, FINDINGS A N D  CONCLUSIONS,  LIMITATIONS, A N D RECOMMENDATIONS  68  Findings and Conclusions  71  L i m i t a t i o n s o f the Study  76  Recommendations  77  REFERENCES  79  APPENDICES  83  APPENDIX A  :  Letter Requesting C o l l e g e Permission  84  A P P E N D I X B:  M e m o G r a n t i n g C o l l e g e Permission  86  A P P E N D I X C:  E x p l a n a t i o n s t o Instructors  88  A P P E N D I X D:  C o n s e n t Form (Instructors)  91  A P P E N D I X E;  E x p l a n a t i o n s t o Students  93  A P P E N D I X F:  C o n s e n t Form (Students)  95  viii  CHAPTER APPENDIX G: Complexity of the Nursing Situation  Page 97  APPENDIX H: Anecdotal Record  102  APPENDIX I: Scoring System  104  APPENDIX J: Generalizability Analysis  112  APPENDIX K: Standardization Procedure  116  APPENDIX L: Stepwise Multiple Regression Analysis  119  LIST O F T A B L E S TABLE 1  Page M e a n s , Standard D e v i a t i o n s , and Intercorrelations Among N u r s i n g Judges (J) i n Semester II  2  3  4  5  6  7.  8  M e a n s , Standard D e v i a t i o n s , and Intercorrelations Among N u r s i n g Judges (J) in Semester III  47  M e a n s , Standard D e v i a t i o n s , and Intercorrelations Among N u r s i n g Judges (J) in Semester IV  48  M e a n s , Standard D e v i a t i o n s , and Intercorrelations Among N u r s i n g Judges (J) i n Semester V  49  A n a l y s i s o f V a r i a n c e o f C l i n i c a l Scores o f Semester II Students: Effects of Times o n Judges  51  V a r i a n c e Components Contributing to Universe Score V a r i a n c e and E x p e c t e d O b s e r v e d S c o r e V a r i a n c e of J u d g e s a n d W e e k s , a n d G e n e r a l i z a b i l i t y C o e f f i c i e n t s of Semester II  52  A n a l y s i s o f V a r i a n c e o f C l i n i c a l S c o r e s of Semester III Students: Effects o f Times on Judges  53  A n a l y s i s o f V a r i a n c e o f C l i n i c a l Scores o f Semester V Students: Effects o f Times on J u d g e s  9  46  55  A Comparison of G e n e r a l i z a b i l i t y Coefficients and M e a n C o r r e l a t i o n C o e f f i c i e n t s Across J u d g e s i n Semesters I I , III, a n d V  10  56  Summary o f S t e p w i s e M u l t i p l e Regression A n a l y s i s o f W e e k 3 o f Semester II: A C o m p a r i s o n of Raw C l i n i c a l Scores to S t a n d a r d i z e d C l i n i c a l Scores  11  58  Summary of S t e p w i s e M u l t i p l e Regression A n a l y s i s of W e e k 3 o f Semester III: A C o m p a r i s o n of Raw C l i n i c a l Scores t o S t a n d a r d i z e d C l i n i c a l Scores  12  Summary o f S t e p w i s e M u l t i p l e Regression A n a l y s e s : Semester II . .  x  59 61  Page  TABLE 13  Summary of Stepwise Multiple Regression Analyses: Semester III . .  63  14  Summary of Stepwise Multiple Regression Analyses: Semester IV . .  65  15  Summary of Stepwise Multiple Regression Analyses: Semester V . .  66  xi  LIST OF FIGURES FIGURE 1  Page Standardization Procedure  118  xii  ACKNOWLEDGEMENTS  I would like to express my gratitude to the administrators, nursing department chairperson, members of the audiovisual department, participating nursing instructors, and students of Semesters II, III,  IV, and V of Vancouver  Community College, Langara, for making this study possible. To the members of my thesis committee—Mrs. Sheila Stanton, the chairperson; Dr. Margaret Campbell; Dr. Todd Rogers; and Dr. Robert Conry— I wish to express my sincerest gratitude for the support, guidance, and expertise that they provided to me throughout the conception and execution of the study. Most particularly, I wish to thank my husband for his patience, understanding and encouragement.  xiii  Chapter I  INTRODUCTION  E v a l u a t i o n o f students i n nursing e d u c a t i o n programs and attempts to i d e n t i f y factors w h i c h a f f e c t t h e i r c l i n i c a l p e r f o r m a n c e c o n t i n u e t o be c o m p l e x and c h a l l e n g i n g t a s k s .  There are a number o f r e p o r t e d nursing studies in w h i c h  the o b j e c t i v e was to i d e n t i f y f a c t o r s a s s o c i a t e d w i t h s u c c e s s f u l c o m p l e t i o n of a program of s t u d y .  F e w e r studies h a v e focussed on i d e n t i f y i n g p r e d i c t o r s o f  c l i n i c a l performance.  The l a t t e r is the focus for the present s t u d y .  It w i l l  a t t e m p t to i d e n t i f y a c a d e m i c and s i t u a t i o n a l factors a s s o c i a t e d w i t h s u c c e s s f u l c l i n i c a l performance. T a y l o r , N a h m , Q u i n n , H a r m s , M u l a i k , a n d M u l a i k ( 1 9 6 3 ) , in a c o m p r e h e n s i v e r e v i e w o f r e s e a r c h i n the a r e a of s e l e c t i o n and r e c r u i t m e n t of nursing s t u d e n t s , made this statement; A l t h o u g h i t is useful t o k n o w h o w w e l l a test works for groups i n g e n e r a l , the more i m p o r t a n t q u e s t i o n is h o w w e l l it works for a p a r t i c u l a r group i n the s i t u a t i o n under c o n s i d e r a t i o n . q u e s t i o n may be:  H o w e v e r , the most important  H o w w e l l . w i l l the g r a d u a t e nurse perform o n the j o b ?  The present test b a t t e r i e s w e r e not d e s i g n e d or v a l i d a t e d to answer this question.  The f e w studies t h a t h a v e a t t e m p t e d to use c e r t a i n test scores  to p r e d i c t success on the j o b h a v e shown that t h e r e is no c o r r e l a t i o n b e t w e e n these test p e r f o r m a n c e s a n d p e r f o r m a n c e o n the j o b .  Here,  t h e n , is a w i d e - o p e n a r e a that u r g e n t l y needs t o be i n v e s t i g a t e d (p. 1 7 ) .  1  2 They further expounded that no one test or test battery can be specified as the best predictor for all nursing schools, and that any test will vary in usefulness from school to school and from year to year. Consequently, they suggested that each school should evaluate its own selection program. In summarizing selection and recruitment measures, Taylor et al. (1963) stated that grade point average and other tests of cognitive abilities appeared to be the best predictors of academic success in nursing theory courses.  However,  grade point average, personality, and attitude tests have not been useful in predicting success in the practical aspects of nursing education or on-the-job performance. These authors have concluded that there is a great need for research utilizing multiple variables of achievement, aptitude, personality characteristics, motivation, interests, and other types of predictors. In 1965, Taylor et a l . reported research on the related problem of attrition, including this statement: Further research in this area is sorely needed because of the high cost, both to the individuals and institutions of attrition in nursing schools and turnover among nursing personnel. Estimates of the cost of recruiting, selection, inducting, and training one individual for different occupations in the world of work range from hundreds to thousands of dollars to the organization, not counting the loss of a person in the training spot or the headaches of repeating the entire cycle to get a replacement into the status of a full-fledged worker. There is also an unestimated psychological and financial cost to the individual in terms of loss of time and effort and the personal disappointment and frustrations when one has an unsuccessful experience and perhaps even finds himself in the wrong field (pp. 52-53). The findings of two more recent studies (Goza, 1970; Gore, 1973) have supported the statements quoted above and, more importantly, have provided new findings comparing dropouts and graduates with favorable performance evaluation.  3 G o z a d e v e l o p e d p r o f i l e s of d r o p o u t s , g r a d u a t e s , g r a d u a t e s w h o passed the State Board Test P o o l E x a m i n a t i o n (SBTPE) in nursing o n the i n i t i a l a t t e m p t , g r a d u a t e s w h o d i d not pass S B T P E , g r a d u a t e s w h o r e c e i v e d a f a v o r a b l e w o r k p e r f o r m a n c e e v a l u a t i o n , and a s a m p l e o f a s s o c i a t e d e g r e e nursing s t u d e n t s .  He found t h a t  g r a d u a t e s had s i g n i f i c a n t l y h i g h e r means t h a n the dropouts o n the f i n a l g r a d e p o i n t a v e r a g e and on the R e s p o n s i b i l i t y s c a l e of the G o r d o n Personal P r o f i l e .  Likewise,  t h e g r a d u a t e s w h o passed the SBTPE had means s i g n i f i c a n t l y h i g h e r t h a n the SBTPE f a i l u r e s o n A m e r i c a n C o l l e g e T e s t i n g P r o g r a m , C a l i f o r n i a Test of M e n t a l  Maturity,  f i n a l G P A , a l l 14 of the N a t i o n a l L e a g u e for N u r s i n g A c h i e v e m e n t tests, the S o c i a b i l i t y s c a l e o f the G o r d o n P e r s o n a l P r o f i l e , a n d the O r i g i n a l T h i n k i n g s c a l e of G o r d o n Personal I n v e n t o r y .  H e a l s o found a n i n t e r e s t i n g d i f f e r e n c e b e t w e e n  means o f graduates w h o r e c e i v e d a f a v o r a b l e w o r k p e r f o r m a n c e e v a l u a t i o n a n d o f those w h o d i d not on the A s c e n d e n c y s c a l e of the G o r d o n Personal P r o f i l e and the V i g o r s c a l e of G o r d o n Personal I n v e n t o r y . G o z a c o n c l u d e d t h a t the reason for m i n i m u m a c a d e m i c success o f dropouts was due t o the l a c k o f r e s p o n s i b l e a t t i t u d e , r a t h e r t h a n a d e f i c i e n c y i n a b i l i t y ; and that u n f a v o r a b l e w o r k p e r f o r m a n c e e v a l u a t i o n r e c e i v e d by some o f t h e g r a d u a t e s a p p e a r e d to be the result o f p e r s o n a l i t y d i f f e r e n c e s r a t h e r than d i f f e r e n c e s i n ability. Research d o n e by G o r e ( 1 9 7 3 ) , w h o d e v e l o p e d p r e d i c t i o n e q u a t i o n s for t h r e e a s s o c i a t e d e g r e e nursing programs, s h o w e d t h a t p r e d i c t i o n e q u a t i o n s c o u l d not be g e n e r a l i z e d from one i n s t i t u t i o n to a n o t h e r .  He therefore recommended  that e a c h nursing s c h o o l s h o u l d d e v e l o p t h e i r o w n p r e d i c t i v e c r i t e r i a , r a t h e r t h a n  4 using common ones.  G o z a ' s findings also revealed that students who were predicted  to fail in one nursing program may well be predicted to pass in another nursing program.  He concluded that this should be a crucial factor in the student's choice  of nursing program. The findings of G o z a and Gore support Taylor et al.'s advice that each nursing school determine its own selection and recruitment criteria, so that they can adequately counsel their prospective applicants and provide guidance throughout their training program. Another factor which may affect c l i n i c a l performance is complexity of the student's assignment.  Traditionally, patient assignments of graduate nurses  were determined by the number of nursing activities/tasks to be performed by the nurse in a given period of time, and the nature or complexity of these activities. O f the many variables that affect quality patient care, the designers of quality assurance programs in nursing have identified complexity of nursing activities as a major factor contributing to the provision of quality nursing care to individual patients.  The proponents of quality patient care programs believe that adequate  staffing of nursing units—staffing based on the number of patients in a unit, number of nursing activities and the complexity of these a c t i v i t i e s — d i r e c t l y influences the type of care given to the patients in these units. If the c l i n i c a l performance of the graduate nurse is influenced by the number of patients she is assigned to, which in turn is based upon the number of activities she has to perform and their level of complexity, then the performance of the student nurse will also be influenced by the type of patient assignment she is given.  5 O f the studies d o n e on p r e d i c t o r s of c l i n i c a l p e r f o r m a n c e , there was n o m e n t i o n of c o m p l e x i t y o f the nursing a s s i g n m e n t / s i t u a t i o n or h o w it may i n f l u e n c e performance.  H o w e v e r , o n numerous o c c a s i o n s , the i n v e s t i g a t o r has n o t e d t h a t  s t a f f , i n s t r u c t o r s , and students suggest that c o m p l e x assignments are the c a u s e o f "poor" performance.  This hypothesis merits i n v e s t i g a t i o n , to a s c e r t a i n w h e t h e r ,  a n d to w h a t e x t e n t , s i t u a t i o n a l c o m p l e x i t y a f f e c t s c l i n i c a l p e r f o r m a n c e . M a n y factors i n f l u e n c e a nursing student's s u c c e s s f u l c o m p l e t i o n o f a nursing p r o g r a m .  In o r d e r t o i d e n t i f y some o f t h e factors w h i c h m a y a f f e c t a  nursing student's c l i n i c a l p e r f o r m a n c e , this study w i l l i d e n t i f y t h e e x t e n t t o w h i c h a c a d e m i c grades i n first semester N u r s i n g , B i o l o g y , a n d P s y c h o l o g y c a n be used t o p r e d i c t the c l i n i c a l p e r f o r m a n c e o f nursing students i n a t w o - y e a r c o m m u n i t y c o l l e g e nursing p r o g r a m .  A l s o , the c o m p l e x i t y of the student's assignment w i l l  be s t u d i e d to d e t e r m i n e the e x t e n t t o w h i c h this v a r i a b l e a f f e c t s c l i n i c a l p e r f o r m ance grades.  The c o m p l e x i t y of the nursing a c t i v i t i e s was a d d e d b e c a u s e r e c e n t  studies h a v e i d e n t i f i e d i t as a major f a c t o r c o n t r i b u t i n g to the nurse's p e r f o r m a n c e and h e n c e t o q u a l i t y p a t i e n t c a r e .  Purpose o f t h e Study  The purpose of this study is to i n c r e a s e k n o w l e d g e o f the factors w h i c h may c o r r e l a t e w i t h c l i n i c a l p e r f o r m a n c e of nursing students; r e l a t e d g o a l s are t o assist nurse e d u c a t o r s i n p r e d i c t i n g student p e r f o r m a n c e a n d p r o v i d i n g g u i d a n c e .  6 S t a t e m e n t of the P r o b l e m  S p e c i f i c a l l y , this study w i l l d e t e r m i n e the d e g r e e t o w h i c h (a) a c a d e m i c grades a n d (b) c o m p l e x i t y o f the nursing s i t u a t i o n e x p l a i n scores i n c l i n i c a l p e r f o r m a n c e for first a n d s e c o n d y e a r nursing s t u d e n t s .  Hypotheses  1.  G r a d e s in N u r s i n g , B i o l o g y , and P s y c h o l o g y courses d u r i n g the first  semester o f the nursing program a c c o u n t for a s i g n i f i c a n t p r o p o r t i o n of the v a r i a n c e i n c l i n i c a l p e r f o r m a n c e scores of s t u d e n t s . 2.  C o m p l e x i t y o f the nursing s i t u a t i o n a c c o u n t s for a n a d d i t i o n a l s i g n i f i c a n t  p r o p o r t i o n o f the v a r i a n c e in c l i n i c a l p e r f o r m a n c e scores o f s t u d e n t s .  Variables  The d e p e n d e n t v a r i a b l e e m p l o y e d i n this r e s e a r c h was the c l i n i c a l performance score:  the a v e r a g e of marks g i v e n by the c l i n i c a l i n s t r u c t o r (who  w r o t e and s c o r e d the a n e c d o t a l records of the student's c l i n i c a l p e r f o r m a n c e d u r i n g the w e e k s o f J a n u a r y 3 0 , a n d F e b r u a r y 6 a n d 1 3 , 1 9 7 8 ) , a n d the scores g i v e n b y t w o nursing judges (instructors w i t h a w o r k i n g k n o w l e d g e o f c l i n i c a l i n s t r u c t i o n , w h o e v a l u a t e d the student's p e r f o r m a n c e using the a n e c d o t a l records w r i t t e n by the c l i n i c a l instructors) from t h a t s e m e s t e r .  The i n d e p e n d e n t v a r i a b l e s used i n the p r e d i c t i o n w e r e : 1.  the grades i n the first semester N u r s i n g , B i o l o g y , a n d P s y c h o l o g y  courses; 2.  the c o m p l e x i t y o f t h e nursing s i t u a t i o n as assessed by the c l i n i c a l  instructor.  S i g n i f i c a n c e o f the S t u d y  M a n y o f the p r e d i c t i o n studies r e v i e w e d h a v e focussed on success i n nursing s c h o o l as the v a r i a b l e o f c o n c e r n .  H o w e v e r , o n l y a few studies h a v e  i n v e s t i g a t e d the r e l a t i o n s h i p b e t w e e n t h e o r y scores a n d c l i n i c a l p e r f o r m a n c e , w h i c h is more r e l e v a n t t o the u l t i m a t e g o a l of p r e p a r i n g the nurse t o perform s u c c e s s f u l l y i n a v a r i e t y of c l i n i c a l s e t t i n g s .  F u r t h e r m o r e , a l t h o u g h the f e w  studies w h i c h h a v e s t u d i e d a c a d e m i c a b i l i t y as a p r e d i c t o r of grades in c l i n i c a l p e r f o r m a n c e h a v e f o u n d n e g a t i v e or w e a k p o s i t i v e r e l a t i o n s h i p s , t h e y d i d not assess (1) the r e l i a b i l i t y o f t h e c l i n i c a l p e r f o r m a n c e grades or (2) the r e l a t i v e c o n t r i b u t i o n s of the c o m p u l s o r y c o u r s e s — N u r s i n g t h e o r y , B i o l o g y , and P s y c h o l o g y w h i c h are d e e m e d to be t h e o r e t i c a l k n o w l e d g e p r e r e q u i s i t e to the p r o v i s i o n o f the t y p e o f c a r e t h a t i n d i v i d u a l p a t i e n t s r e q u i r e or r e q u e s t . A r e v i e w of the l i t e r a t u r e o n t o o l s / m e t h o d s used t o e v a l u a t e c l i n i c a l p e r f o r m a n c e r e v e a l e d one study by Rines (1963) that r e p o r t e d "77 p e r c e n t of the nursing (p. 2 6 ) .  instructors used a n e c d o t a l records w i t h v a r y i n g degrees of s u c c e s s "  8 In summary, the need to identify predictors of clinical performance arose from the enduring problem of how best to prepare nurses. This study was initiated as a result of dissatisfaction, expressed by both students and instructors, with clinical evaluation in general and specifically with procedures for assigning clinical grades. Furthermore, an unpublished survey of nursing students' attitudes toward various methods of clinical evaluation (conducted by the investigator in 1976) showed that students generally prefer a combination of several evaluation methods: anecdotal records, simulation, clinical examination, and self evaluation. It was beyond the scope of the present study to examine several evaluation techniques in evaluating clinical performance, and then to use the method(s) which had the best reliability, validity, objectivity, and practicality. Therefore, the anecdotal record method of evaluation was chosen, since it was currently in use to some extent, and based on the belief that it would provide instructors with data to use in the course of clinical teaching and evaluation. Finally, results of this study could help to identify those students in this school who may need considerable counselling and assistance during the course of their programs, and thus enable instructors to take some preventative action and possibly to decrease attrition and the cost of nursing education.  9 Operational Definitions  The following terms are used: nursing student - a student who is enrolled in a two-year college nursing program preparatory to nurse registration, nursing program - a six-semester program in a community college leading to a diploma in nursing, c l i n i c a l area - a hospital ward. c l i n i c a l objectives - learning objectives to be achieved by the students in the c l i n i c a l area. performance - the behaviors of the nursing student in the c l i n i c a l area, as related to relevant c l i n i c a l objectives, c l i n i c a l instructor - a nursing instructor who teaches, observes, and evaluates the student's performance in the c l i n i c a l area, performance evaluation - an evaluation of the student's performance to determine whether c l i n i c a l objectives have been met. nursing judge - a nursing Instructor, with a working knowledge of instruction in any one of Semesters II,  III,  IV and V of the nursing program, who  evaluates the performance of the nursing student using the anecdotal notes written by the c l i n i c a l instructor, c l i n i c a l performance score - the average of the scores given by the c l i n i c a l instructor and two nursing judges, each of whom evaluated anecdotal notes on the performance of the student during the weeks of January 30, February 6, and February 13, 1978.  10 SCI - scores given by the c l i n i c a l instructor. S N J - scores given by the nursing judge. Grade Point Equivalent - the letter grades of students in first semester Nursing, Biology, and Psychology were transformed to numerical grades using the scale below. Letter G r a d e *  Grade Point Equivalent  A  4  B  3  C  2  P  1  F  0  complexity of the nursing situation (CNS) - is determined by four variables: (1)  the activities required of the student in the c l i n i c a l area, including (a) administering treatments prescribed by the patient's physician; (b) administering treatments prescribed by the nursing team;  (2)  the condition of the patient;  (3)  the a v a i l a b i l i t y to the student of appropriate patient experiences;  (4)  the revision of the student's assignments based on the instructor's assessment of the student's a b i l i t i e s .  *Vancouver Community C o l l e g e , Langara 1977-1978 Calendar.  11 Assumptions  The design and execution of this study rests upon these assumptions: 1.  the clinical objectives of a given semester are decided and agreed upon  by the instructors responsible for that semester, and hence represent their expectati of the students enrolled in that semester. 2.  the clinical instructor can validly and reliably judge the complexity of  the nursing situation.  Chapter II  REVIEW OF RELATED LITERATURE  Introduction  Adequate evaluation and adequate prediction of the students' clinical performance are problems that nursing education shares with other cognate disciplines such as medical education and dental education. The process of clinical evaluation requires observational techniques for collecting data in various clinical settings. This very process, although used extensively for decades, has created problems for the educators and the students alike.  Educational Evaluation  Gronlund (1976) defined evaluation as a "systematic process of determining the extent to which educational objectives are achieved by pupils" (p. 6).  He  distinguished between evaluation and measurement, stating, "Evaluation is a much more comprehensive and inclusive term than measurement" (p. 6), which includes both quantitative and qulitative descriptions of pupil behavior, plus value judgments concerning the desirability of that behavior. Similarly Gordon (1960) described evaluation as". . . the measure of the extent to which the performance achieves certain desired goals" (p. 364). 12  13 In describing evaluation of students, MacKay (1974) stated that ". . .we  must view evaluation as a means and not an end, as an ongoing process  with the purpose being promotion of fuller development of the learner" (p. 4). Likewise, Gordon (1960) asserted that "The most important use of an evaluation is to assist the person to improve his performance" (p. 366). In her effort to establish productive evaluation, MacKay (1974) has outlined the following guidelines: 1.  the purpose of evaluation as a means of facilitating learning should be known and accepted by both student and teacher;  2.  the standards of evaluation should be known and shared, or goals for behavior based on assessed learning needs should be established;  3.  the standards or goals should be realistic; the student should be potentially capable of meeting the objectives for performance normally expected for that level of preparation and his unique idiosyncrasies taken into consideration; and  4.  behaviors within a specified time interval only should be evaluated, (p. 5)  Using a different perspective, Schweer and Gebbie (1976) stated that the trend in evaluation of clinical nursing is toward an ongoing evaluation program aimed at determining student growth in becoming a skillful practitioner. They also identified purposes that evaluation processes could serve. These are as follows: 1.  to determine the background ability each student brings to the learning situation;  2.  to determine the ability of students to use nursing theory as a whole while progressing in the development of clinical skills;  14 3.  to determine the progress by each student at frequent intervals during the assignment;  4.  to discover learning difficulties of individual students and the group in order to adjust the teaching, including remedial or advanced assignments, as needed;  5.  to provide reinforcement of learning for students;  6.  to foster development of self-evaluation;  7.  to determine the readiness of students to become self-directive;  8.  to determine the effectiveness of teaching techniques; and  9.  to obtain data for conducting research studies relating to student achievement, (p. 166)  Evaluation as described by Gordon (1960), MacKay (1974), and Schweer and Gebbie (1976) suggests that they are using this term to mean both formative and summative evaluation of performance. Formative evaluation helps the student to learn by pointing out his strengths and weaknesses, and summative evaluation measures to what extent the student has met the course objectives. Most students and teachers fail to make the distinction between evaluation and grading. However, the distinction was made by MacKay (1974), who stated that "the assignment of grades requires a further step of measuring the achieved levels of competency in respect to specified behavior characteristics of students in general in differing levels of expertise" (p. 5). To establish educationally sound evaluation programs, Gronlund (1976) and Rines (1963) have proposed the following: 1.  identify and define the objective of the program in terms of desired changes in the behavior of students;  2.  determine evaluation tools;  15 3.  construct tools which are valid and reliable;  4.  decide on utilization of these tools;  5.  decide what points in the course will be appropriate to evaluate the student's performance;  6.  outline a program of evaluation;  7.  plan and select situations to show desired change of behavior;  8.  collect the responses; and  9.  interpret the results. The above program of evaluation indicates that the specific activities of  the educator related to evaluation are also a part of the teaching process, since they help in defining the objectives, and in the planning and selecting of situations to show the desired behavior of the student. Although the interdependence of teaching and evaluation is recognized and accepted by nurse educators, Rines (1963) makes a strong point that If evaluation is to be conducted with maximum efficiency and results, regular times should be designated for the purpose, separate from the teaching process . . . [the instructor] should plan her observations and evaluations of student behaviors just as she plans for paper-andpencil tests and examinations. These should be planned frequently enough . . . so that she [the instructor] cangetan estimate of behavior trends, (pp. 54-55)  Clinical Performance Evaluation Versus Classroom Evaluation  Most nurse educators agree that the process of evaluation in the clinical area is difficult and complex (cf. N L N  Report, 1972; Litwack, Sakata, and Wykle,  16 1972). In the clinical area, the student is learning to apply knowledge acquired in the classroom and laboratory. Moreover, as Wood (1972) has suggested, "The student is learning behavioral more than intellectual skills" (p. 336). Most nurse educators will also agree that the achievement of clinical objectives is poorly measured by conventional classroom tests, whether teachermade or standardized. Furthermore, clinical experiences are not uniform—not all students will have the opportunity to care for the same patient with a given problem. Therefore, to the extent that each individual and his reactions are unique, the clinical experiences of the students will vary greatly, and thus make standard teaching and evaluation almost impossible. Another difference between clinical and classroom evaluation is that various individuals in the clinical area add input and thus affect the student's behavior. It is therefore crucial that the input of these individuals—patients, nursing staff, and clinical instructors—be utilized for evaluation of the student's performance. Taylor et al. (1965) have pointed out that "the lack of relationship between performance in theory courses and clinical practice courses may point up a deficiency in traditional methods of evaluating students" (p. 38). To explain the above, they have quoted the following shortcoming in nursing education initially described by McManus (1949): many of those who teach nurses . . . have had no formal training as teachers and therefore tend to follow the patterns and teaching they themselves have experienced. They are not aware of the need to develop the higher mental abilities in this field as in other professions. Only exceptional teachers and students manage to free themselves from traditional patterns of teaching and learning.  17 Among the nursing educators who have had some preparation for teaching are many who still erroneously use instructional methods which demand the exercise of little more than the lower mental processes of memorization and recall. Frequently facts are taught and facts are learned in catechismal fashion. Testing and grading systems tend to put a premium upon fact-getting and rote-learning of subject matter. Skills are frequently taught exclusively in the classroom, with drill on specific procedures. Although the students spend a considerable portion of their time in the clinical nursing practice, the theory is likely to remain sterile and compartmentalized and will rarely, and only accidentally transfer unaided to give the practice meaning. . . . (pp. 38-39) Lancaster (1972) in her Opinion Survey of nurse teachers in Scotland has addressed the question of whether the present tutors' courses give the kind of preparation which will enable future nurse teachers to cope effectively with the professional demands made upon them. The kind of work done in the past . . . in the traditional type of nursing school hardly seems to be a realistic basis for decisions regarding the preparation and qualifications appropriate to nurse tutors in the future, (p. 5) Lancaster stated that "the answer to this question depends on the definition of the tutor's role" (p. 5). She has suggested that the role of the teacher depends on many things, such as the type of students, the type of subjects to be taught, the attitudes and values of the educational system, and the teaching-learning environment. Furthermore, she has concluded that the preparation of teachers is crucial in the development and maintenance of effective nursing practice.  But, the role of the teacher must first be explicitly  defined so that criteria may be developed to assess the teacher training programs. A quarter century later, Infante (1975) and Miller (1976) expressed the very same concerns about the shortcomings in nursing education as McManus (1949). Commenting on the preparation of nurse educators, Sister Miller (1976)  18 stated that "most graduates of master's programs become teachers immediately after graduation, but has their education prepared them for this role?" (p. 752). She found out that only 21 of the 64 graduate nursing programs offer preparation in teaching. Infante (1975) stated emphatically that nurse educators should be educated in strategies of clinical teaching: Teachers in graduate schools who are preparing teachers for undergraduate programs in nursing might place more emphasis on strategies of teaching in the clinical laboratory. Strategies for teaching in the clinical laboratory, the heart of professional education, have been noticeably absent from course outlines. Perhaps increased opportunity for future teachers to apply theory to practice under guidance is also needed. Just as changes in the service of a professional practicum must start with the educational program of which he is a product, so changes in the type of teaching done by teachers . . . must start with changes in the educational programs that prepare these teachers, (p. 60) Similarly, Miller (1976) stated: The best method to help [graduate] students to internalize the principles of learning, planning, and evaluation, along with the philosophy of education and the act of teaching, so that they may function effectively as teachers of nursing . . . is for those teaching graduate students . . . to be role models who facilitate the learning process for the student, (p. 752) This lack of educational programs in clinical teaching and evaluation has made nurse educators vulnerable, and has encouraged the adoption of evaluation tools and practices to which they themselves were subjected as students. These evaluation practices, although not educationally sound, could prove to be very efficient and effective. However, unless clinical evaluation research endorses such practices, nurse educators must not assume that clinical grades reflect the students' true performance. This viewpoint has been asserted  19 by Champney, Chatfield, and Sims (1975) who stated that it is dangerous to adopt scales and objectives unless the team of instructors who have to use them are in complete agreement. Tate (1962) stated, "the more complex the behavior to be evaluated, the more difficult the task of building an evaluation instrument" (p. 36). Another area which has influenced clinical performance grades of nursing students is team teaching. Various approaches to this technique were described by Schweer and Gebbie (1976) as having (1) all clinical teachers work together as a team in supervising the student, (2) certain portions of the program taught by a master clinical teacher, with other clinical teachers supervising groups of students, and (3) specialists in the field of nursing contribute to the clinical teaching and evaluation. These various approaches seem to have created inconsistencies among evaluators of clinical performance, because of (1) lack of communication among clinical instructors, (2) lack of training of these instructors relative to evaluation and methodology, and (3) lack of adequate time for instructors in the team to discuss clinical objectives, and the behaviors that the student is expected to demonstrate to successfully meet the objectives. Hayter (1973) demonstrated that the effects of the difficulties created by team teaching, although they cannot be avoided entirely, could be diminished if each instructor in the team understood the objectives and the specific behaviors expected of a given level of student.  She found that the agreement among 31  evaluators as to the grades of three students was 7 6 % after discussing the objectives, as opposed to 4 4 % before the objectives were discussed.  20 In her extensive review of the literature, Hart (1974) found that nursing schools have been concerned with the evaluation of clinical performance of students, and in some instances, have employed ameliorative measures. These measures include involving the faculty in the development of a devised rating scale or check list, the employment of student evaluations, and the identification of behaviors in the clinical setting which are regarded as satisfactory for a given level of student in the nursing curriculum. Finally, the issue of assigning either a letter or a numerical value for clinical performance has perturbed many nurse educators. Rines (1963) wrote on this issue: Human behavior of any description is much too complex to permit fine discriminations . . . we do not know enough about the way in which students learn or what influences learning to say more that [sic] a behavior is acceptable or unacceptable, (p. 17) Hart (1974), in a survey of 105 nurse educators to determine their perception of the significance of criteria and tools employed in the evaluation of clinical performance of baccalaureate students, found that some of the least significant criteria and tools were the use of a devised rating scale and the percentage/letter grade for the course. Historically, says Hart, the letter/ percentage grade was widely used in the evaluation of the students' performance. However, she concluded that educators in baccalaureate programs are cognizant of the many variables in the evaluation of the students' clinical performance which cannot be reduced to a single variable representing a percentage or a letter grade.  This finding supports the statement by Rines (1963), that clinical  performance is too complex to permit fine discrimination.  21 It is interesting to note the discrepancy between the ameliorative measure—involving the faculty in the development of a devised rating scale or check list as stated by Hart—and her finding of a devised rating scale as the least significant criterion and tool.  Observational Techniques in Evaluation  Some aspects of learning and development can be evaluated only by direct observation (Ahmann & Glock, 1971; Gronlund, 1976); however, evaluations based on observations have serious limitations. Thorndike and Hagen (1961) have stated that "Evaluation based on naturally occurring situations of life is likely to vary from person to person" (p. 17). Clinical nurse educators continue to receive numerous complaints from the students about discrepancies between instructors or between clinical settings in clinical performance evaluations and grades assigned. To minimize these discrepancies Schweer and Gebbie (1976) concluded that "observation continues to be regarded as an effective means of evaluating nursing performance, but teachers must have a thorough knowledge of the exact behaviors to be accomplished during a given experience, along with an analysis of the level of each performance to be rated according to a standardized scale" (p. 172). Other problems in performance evaluation have been discussed by Champney, Chatfield, and Sims (1975). These authors have suggested that "raters do better if carefully trained with respect to the distribution of abilities and the nature of the scale, and cautioned against common errors" (p. 168).  22 Four rater idiosyncrasies identified by them were: (1) leniency; (2) halo effect; (3) logical error; and (4) central tendency (p. 168). The limitations of direct observation as identified by Ahmann and Glock (1971), Fuerst and Wolff (1969), Gronlund (1976), Rines (1963), and Thorndike and Hagen (1961) were as follows: 1.  the difficulty in determining the significance of an isolated item of behavior;  2.  the complexity involved in identifying the meaningful and productive set of behavior categories to observe;  3.  the problems arising from the biases of the observer, who may bring his own needs and feelings to the evaluation;  4.  the decrease of cogency in observations which arises as the student numbers increase; and  5.  the high cost of direct observations.  Anecdotal Record Technique of Evaluation The anecdotal record technique is an informal, direct method of recording behavior observed in a natural setting (Thorndike & Hagen, 1961), which provides a large amount of information about the learning and development of the student. Gronlund (1976) stated that "Anecdotal records are factual descriptions of the meaningful incidents and events which the teacher has observed in the lives of his pupils" (p. 429). Similarly, Fivars and Gosnell (1966), and Fuerst and Wolff (1969) have described anecdotal records as informal descriptions of the actual observed behavior and activities of students.  23 The advantages of such anecdotal records, described by Ahmann and Glock (1971), Gronlund (1976), Rines (1963), and Thorndik and Hagen (1961), are as follows: 1.  lend themselves well to frequent observations of short duration;  2.  make it possible to see trends in behavior over a period of time;  3.  provide a description of actual behavior in natural situation;  4.  provide a check on other evaluation methods;  5.  make it possible to gather evidence on events that are exceptional but significant;  6.  increase the observer's awareness; and  7.  make instructors more diligent in observations. Like any evaluation tool, the anecdotal record technique has its  disadvantages, which are similar to the limitations of direct observational techniques used to evaluate behavior in naturally occurring situations of life (Ahmann & Glock, 1971; Gronlund, 1976; Rines, 1963; and Thorndike & Hagen, 1961). The disadvantages include: 1.  the time required to maintain the records;  2.  the difficulty in being objective when observing and reporting student behavior;  3.  the difficulty in obtaining an adequate sample of behavior;  4.  the tendency to accumulate unfavorable anecdotes;  5.  the difficulty of periodically summarizing the anecdotes into some kind of framework;  6.  the tendency to report behavior without describing the setting;  24 7.  the tendency to evaluate rather than report behavior; and  8.  the tendency to describe behavior in general terms.  The suggestions made by Gronlund (1976) and Thorndike and Hagen (1961) for writing proper anecdotal records are: 1.  determine in advance what you want to observe, but be alert for unusual behavior;  2.  observe and record enough of the situation to make the behavior meaningful;  3.  make a record of the incident as soon after the observation as possible;  4.  limit each anecdote to a brief description of a single specific incident;  5.  keep the factual description of the incident and the interpretation of it separate;  6.  record both positive and negative behaviors;  7.  collect a number of anecdotes on a student before drawing inferences concerning typical behavior;  8.  obtain practice in writing anecdotal records;  9.  teach the observers what to look for; and  10.  train the observers in standards of judgment. In summary, anecdotal records provide a systematic procedure for  recording observation, but Blood and Budd (1972) state that it does not guarantee that these observations will be systematically made nor directed toward specific relevant behaviors. Two authors, Blood and Budd (1972) and Rines (1963) have suggested that perhaps time sampling systems might provide an opportunity to systematically observe student behavior.  25 Prediction of Academic Performance Versus Clinical Performance  The Criterion Problem In a review of research studies and practices dealing with selection, recruitment and prediction, Taylor et a l . (1963) found that when academic grades in nursing were used as the criterion, the best predictors—in terms of single highest correlations—were pre-nursing College Grade Point Average (.63), the Otis (.48), and High School Grade Point Average (.43) (p. 33). Another review of research studies by the same authors in 1965, revealed that the best predictors were Otis (.65), American College Test (ACT) total score (.64), and High School Grade Point Average (.51). In 1963, Taylor et a l . , when using clinical grades as the criterion, found that The three best predictors with highest single correlations again are pre-nursing College Grade Point Average (.59), the Otis (.59), and the High School Grade Point Average (.37). In general, the correlations drop from academic or theory grades to clinical or practice grades indicating that the best predictors of the former criterion are not necessarily the best predictors of the latter criterion and that we have not yet learned how to predict the latter criterion, (p. 33) In their discussion of the tremendous variability in the results with various predictors and criteria for success in nursing schools, Taylor et al. (1963) suggested that variability in results can in part be attributed to the many different ways of measuring the criteria from one study or location to the next. Furthermore, they indicated that the prediction problem may be confounded by the many complex  26 behaviors that are neither recognized in training programs nor in job evaluations as crucial for success at this point in time. These authors state that much more work is needed on the measurement problems involved in developing meaningful, reliable and valid clinical criterion measures for success in nursing schools, and that these criteria once developed should then be carefully related to "on-the-job" criteria for success.  Predictive Studies There are very few reported predictive studies on clinical performance in nursing. Ryden (1977) studied the predictive value of a clinical examination of interpersonal relationship skills and subsequent interpersonal competence in clinical situations as measured by ratings of clinical instructors, and obtained a coefficient of correlation .45 (significant at p <.05). Although this is a positive relationship, it is not high, and therefore suggests that there perhaps are other factors which influence performance in the clinical area. In their study of "Measurement and Prediction of Nursing Performance", Taylor et al. (1965) found that the graded performance of students in nursing theory and general education courses was not closely related to their performance in clinical nursing courses. These authors have suggested that perhaps the clinical grades represent measures of additional abilities and behaviors from those measured by academic courses. The shortcomings in nursing education described earlier by Taylor et al. (1965, pp. 38-39) may indicate one reason for nursing theory and clinical practice grades not correlating highly. They have suggested that the lack of relationship  27 between performance in theory and clinical practice courses may point out the undesirable utilization of the deficient traditional methods of evaluating students, especially because of the lack of relationship between classroom and clinical grades. These authors have postulated that this lack of correlation between theory and practice has occurred because: 1.  theory courses have not stressed basic nursing processes which are relevant across patient problems, service settings, and functions;  2.  instead of being expected to assume responsibility for acquiring the information needed to deal with a particular problem, students were given the answers;  3.  until very recently these courses have been built around clinical areas and/or disease entities rather than patient problems;  4.  lack of specificity in criteria by which students have been evaluated as they practice nursing in the clinical units;  5.  objectives have not been stated in terms of behaviors that could be easily evaluated; and  6.  learning how to practice as a professional is a much more complex learning process than learning of a theory per se or the learning of a patient care procedure, (p. 39) In a study of prediction of success of community college nursing students,  Powers (1974) studied the interpersonal competencies of 80 students in their second semester using selected items from the Minnesota Multiphasic Personality Inventory (MMPI), the California Psychological Inventory (CPI), the Strong Vocation Interest Blank (SVIB), and the American College Test (ACT). The results indicated that only two subtests from the four predictor variables produced F values significant at .05 level when correlated with interpersonal competencies of nursing students.  28 These items were "Social Introversion" from the MMPI, which had a multiple correlation coefficient of .1021; and "Good Impression" from the CPI which increased the multiple correlation coefficient to .2182 (p. 56). A review of medical literature revealed results similar to those found in nursing—that is, low correlations between clinical performance grades and academic grades. Korman and Stubblefield (1971) administered a rating scale assessing internship performance and then correlated the results with medical school grades. They found that GPA bore no significant relationship to any of the evaluative components of internship performance. Similarly, Wiener, Koran, Mitchell, Schattner, Fierstein, and Hotchkiss (1976) correlated the scores of medical students for interviewing, physical examinations, and total clinical skills with the scores on the final multiple-choice exam given at the end of clerkship. The respective product moment correlations were .36, .58, and .58.  Using a broader sampling of  physician performance, Gonnella, Goran, Williamson, and Cotsonas, N., Jr. (1970) and Leaper, G i l l , Staniland, Horrocks, and De Dombal (1973) found little relationship between written examination scores and clinical performance scores. These findings suggest that there is a weak relationship between students' knowledge and the application of that knowledge. However, these investigators did not take into account all the other factors which affect evaluation of clinical performance.  29 The review of the literature did not reveal any information relevant to the alleged relationship of complexity of the nursing situation and poor clinical performance. The strong feelings of students, staff, and instructors on this issue need attention.  Summary  Research has been sporadic and meager on what constitutes competent clinical performance, on approaches to grading it, on tools to measure it, and on corresponding issues of validity and objectivity. In the few studies which have been completed, findings have been discouraging—correlations between clinical performance grades and academic or psychological tests have ranged from 0 to .65.  Furthermore, the correlations have varied from one institution  to another and from one sample to another, making it difficult to generalize to the population of nursing students at large. Some of the numerous reasons given by various researchers for these low correlations are: more subjectivity is involved in assigning clinical grades than academic grades; components of satisfactory clinical performance have not yet been identified; training of nurse educators is inadequate in the case of clinical teaching-learning-evaluation processes; there are substantial differences in the grading methods used; there is real variability in the quality of students; there are substantial differences in the curricula; many problems are inherent in the use of observational techniques in the evaluation and the grading of  30 clinical performance; there are no standardized evaluation procedures; problems of communication and consistency among instructors are inherent in team teaching; and traditional methods of clinical teaching and evaluation are outmoded. Only a few of the studies indicated clearly whether the clinical performance grade did indeed comprise the clinical grade or was just one component, along with theory, of the clinical grade. Likewise, very few studies attempted to ascertain the reliability of the so-called clinical grades before studying variables which would predict it. The limited and controversial research on clinical performance evaluation — a n d specifically the weak relationship between knowledge and its application in real life situations—requires vigorous efforts to identify influential factors and to analyze how grading processes could be improved so as to benefit the individual nurse, her patient, and the nursing profession.  Chapter III  RESEARCH M E T H O D O L O G Y  Introduction  In order to assess the extent to which first semester grades in Nursing theory, Psychology, and Biology courses, and complexity of the nursing situation were valid predictors of clinical performance of nursing students, a sample of associate degree nursing students was selected. Likewise, a sample of clinical instructors was selected.  Sampling  Institution Requests to conduct the study at Vancouver Community College, Langara, British Columbia were made to the Dean of Administrative and Student Services (Appendix A), and to the chairperson of the Nursing Department. Each was approached personally and given the research proposal. Written permission to proceed was subsequently received from the Dean of Administrative and Student Services (Appendix B), followed by oral permission from the Nursing Department chairperson.  31  32 Semesters The two-year Nursing Program at Vancouver Community College, Langara is divided into six semesters, each 13 to 14 weeks long. Because students in Semesters I and VI had, respectively, very short and very long clinical experiences, they were deleted from the study; thus, students in the remaining semesters were included. These groups had clinical experience throughout the semester ranging from 8 to 16 hours per week.  Weeks Each semester was comprised of two rotations, each six to seven weeks in length. The grading system used by the nursing faculty was such that at the mid-term (end of first rotation), the students received a formative evaluation, and at the end of the semester they received a final clinical grade. Only the anecdotal records written during the first rotation were used in order to eliminate the possible influence of this research on students' grades at the end of the semester. The clinical instructors were asked to write anecdotes about the performance of their students for the fourth, fifth, and sixth weeks of the first rotation, which extended from January 30 to February 13, 1978. These weeks were chosen because (1) the first week of the rotation was orientation, (2) the second week would give the students a chance to establish themselves and allow them to begin to feel comfortable in the clinical area, and (3) the third week would give the instructors a chance to practise writing anecdotal records. Furthermore, the anecdotal records of only three weeks were chosen to keep the task of writing and scoring them reasonable and manageable for the nursing instructors.  33 Instructors Approximately two months prior to the initiation of the study, permission was requested from the Nursing Department chairperson to present the highlights of the study to faculty members in a general meeting.  The purposes of this  presentation were to ascertain whether interest in the study was sufficient, and to answer questions. The investigator (one of the instructors who was not included in the study) met with the instructors. The proposed study was explained, questions were answered, and feedback was obtained indicating that the majority of faculty were in favor of the study and interested in its results. During the week of January 9, 1978, each instructor from Semesters II, III, IV, and V, was given a sheet explaining the study (Appendix C) and a consent form (Appendix D); they were asked to return the consent form to the investigator by January 13, 1978. Some of the instructors asked for additional information or clarification, which was provided as requested. There were four instructors in Semester II, three of whom consented to participate in the study. Similarly, there were four instructors in Semester III, three of whom consented to participate in the study. One of these instructors consented only to write anecdotes and judge the performance of her own students. Another instructor who was not teaching in Semester III but had taught in it several times, consented to be a judge. Semester IV had four instructors, all of whom consented to participate fully in the study.  34 Finally, Semester V had five instructors, all of whom consented to participate. Also, another instructor who had taught in that semester and was at that time teaching in Semester I, wanted to participate, and consented to be a judge in Semester V. In summary, of 17 instructors in Semesters II, III, IV, and V, 15 consented to write anecdotal records and be judges. Two other instructors who were not teaching in those semesters but had taught in them before, consented to be judges.  Students After obtaining the signed consents from the instructors indicating their willingness to participate in the study, the investigator asked the leaders of Semesters II, III, IV, and V respectively for 15 to 20 minutes of their class time during the week of January 16, 1978. In these sessions, the investigator explained the study to the students, and distributed both the explanation (Appendix E) and the consent forms (Appendix F). Students were asked to complete the consent form and return it to the investigator by the end of that week. Furthermore, they were told that if they had any questions regarding any part of the study to contact the investigator. Semester II sample. There were 35 students registered in the clinical nursing course, 30 of whom agreed to participate. The group of students who consented to participate, but whose instructor wished not to participate, were eliminated from the study. Also, to equalize the workload of the three  35 participating instructors, five students were randomly chosen from each group. And of these 15 students, one withdrew from the semester, leaving a final sample of 14 students. Semester III sample. There were 25 students registered in Semester III clinical course, 21 of whom consented to participate. Those students who consented to participate, but whose instructor wished not to participate, were eliminated from the study. O f the remaining 14 students, two were in a specialized clinical area and, therefore, were eliminated from the study. The final sample of Semester III was comprised of 12 students. Semester IV sample. There were 32 students registered in the clinical nursing course of Semester IV, 24 of whom agreed to participate. To equalize the workload of the four instructors, four students were randomly chosen for each instructor, yielding a final sample of 16 students. Semester V sample. Finally, Semester V had 28 students registered in the clinical nursing course, 22 of whom agreed to participate. Two of these students had advanced credits in the independent variables from countries other than Canada and United States. However, the assessed transcripts did not indicate the equivalent college grades of these students. Consequently, it was decided to eliminate them from the study. Also, three students withdrew from the semester. The final sample was comprised of 17 students. In summary, the total sample of students at the end of data collection period was 59: 14 from Semester II, 12 from Semester III, 16 from Semester IV, and 17 from Semester V.  36 Confidentiality  To ensure confidentiality, each instructor and student was assigned a number by the investigator. Each anecdotal note was typed before being given to the judges to eliminate effects of handwriting and the identities of instructor and student. Also, the investigator proofread all the anecdotal notes and, where the need arose, names of students were deleted and substituted with appropriate terms such as, "the student", "she", "he", et cetera.  Instrumentation  Complexity Scale The decision as to who could best determine the complexity of the nursing situation was complicated by the many constraints on time and resources. It was felt that complexity could be most reliably and objectively assessed by a group of observers trained to follow up each student and record minute-to-minute changes in their patients' health status and to observe the students' abilities. A panel of these observers could then judge the complexity of the nursing situation as recorded. However, considering the constraints mentioned above, the investigator believed that the best persons to determine the complexity of the students' assignments were their clinical instructors, because they were in a position to be most knowledgeable about the students' assignments and the factors affecting them. A form was then developed (Appendix G) to be used by the clinical instructors for rating the complexity of the nursing situation. This form delineated  37 three levels of complexity, and the instructor was asked to circle one level at the end of each clinical experience. Each of the three levels of complexity was respectively described by four criteria. These were (1) activities of the student; (2) condition of the patient; (3) patient experiences; and (4) revisions of student's assignments based on the instructor's assessments of the student's abilities/performance. Furthermore, the form was accompanied by guidelines for its utilization—these were specific examples which the instructor had to follow to decide on the level of complexity. Definitions of the three levels of complexity were based on clinical experience and on numerous discussions with both instructors and students about the difficulty of clinical assignments.  Resources were not sufficient to allow a  complete pilot test of the instrument. However, four instructors were consulted; each read it independently and reacted, and the form was refined accordingly.  Anecdotal Record The anecdotal record form (Appendix H) was adapted from Rines (1963) following a review of the literature and the assessment of the needs of this study. The following elements were noted: identification data (student and instructor numbers, date and semester); description of the situation; description of the student's behavior; and the performance score given. Scoring system. The purpose of the scoring system was to provide the judges with a uniform measuring device, thereby minimizing scorer subjectivity and increasing reliability of the scores given.  38 The scoring system (Appendix I) developed by the investigator is a Likert-type scale with a range of 1 to 9, representing unsatisfactory performance to superior performance. This scale was further divided into five major categories which were more familiar to the instructors so that there would be an association between the scoring system and the grading system used at the college. This assisted in utilizing the scoring procedure. The five categories were A, B, C, P, and F. Each of these grading categories was further described by a set of six criteria which were thought to be critical in the evaluation of the students' attainment of each of the clinical objectives. Nebulous terminology was defined, so as to provide the judges with a common understanding of the crucial terms and thereby increase objectivity and reliability of the scores.  Procedure  Development of the Videotape In order to assist clinical instructors in the writing of anecdotal records and the use of these records to judge the clinical performance of the students, training procedures were developed to sharpen observational skills, and to allow practice in the above activities in a simulated setting. The investigator wrote a script for a nine-minute videotape, titled "Evaluation of a Student's Clinical Performance", which has five sections designed to depict the various activities of a nursing student in the clinical laboratory.  39 The five vignettes and their respective lengths were: Vignette  Length  1.  The student doing a brief assessment of the patient before breakfast.  2-j min.  2.  The student giving subcutaneous injection to patient.  2^min.  3.  The student reporting to the team leader.  imin.  4.  The student and the instructor discussing the student's nursing care plan on the patient.  2 min.  5.  The instructor checking the student's charting on one of the patients.  l^min.  The Training of Clinical Instructors and Judges Separate workshops were scheduled for the instructors of each of the Semesters II, III, IV, and V.  The instructors who had consented to participate in  the study were notified in writing to attend these workshops on the evaluation of clinical performance. Other instructors who had indicated an interest in the topic were also invited to attend. The purpose of the workshops was to teach instructors how to (1) write anecdotal records according to the sample description of anecdotal records adapted from Rines (1963); (2) score these, employing the scoring system (Appendix I); and (3) determine the complexity of the nursing situation. Each workshop was conducted over a period of three to four hours, during which time the following activities were carried out: (1) reviewing the advantages, disadvantages, and characteristics of effective anecdotal records;  40 (2) viewing each of the five sections of the videotape, followed by writing and discussing the performance of that student based on the clinical objectives and the assignment; (3) scoring this student's performance using the scoring system; and (4) determining the complexify level of the student's assignment.  Data Collection The instructors wrote anecdotal records for the last three weeks of the first rotation of their semesters.  They were asked to write three anecdotes on  each student, one anecdote for each week of clinical experience. The written anecdotal records and the complexity of the nursing situation were collected weekly from the clinical instructors. These anecdotal records were then typed, duplicated, and distributed to three judges—the clinical instructor and two other instructors—who scored them independently using the scoring system (Appendix I), and the clinical objectives of the student. These scores were then analyzed to determine their reliability using conventional correlational and generalizability analyses. During the week of January 16, 1978, the investigator obtained from the registrar's office the participating students' grades in their first semester Nursing theory, Biology, and Psychology courses. High school grade point averages (GPA's) were not used because the college did not require that information from applicants. These students were admitted into the nursing program on the condition of high school graduation and/or their ability to successfully complete appropriate "make up" courses.  41 Method of Analysis  The analyses of the data were executed in two phases. First, it was necessary to assess the reliability of the clinical scores before proceeding to the stepwise multiple regression analyses. The following is a description of the two phases of the data analyses.  Phase One During phase one, preliminary analyses of the data were conducted to determine (1) inter-rater reliability by correlating the scores given by the clinical instructors and the two nursing judges; and (2) generalizability coefficients among judges. The scores for these analyses which ranged from 1 to 9 were derived as shown in Appendix I. The latter was believed to be a more sensitive and informative measure of the reliability of the clinical scores. Thus, both methods of analyses were carried out and the results compared. A program from the Statistical Package for the Social Sciences (SPSS.V701) was used to compute the Pearson Product Moment Correlation Coefficients for the data of each of the four semesters separately and within each of the four semesters, each week separately. Each correlation coefficient (r) in turn was converted to Fisher's Z employing an r-to-Z transformation table. Next the Z values obtained for each of the three weeks of each of the fours semesters were added up separately and then the mean Z values were calculated. The mean Z values obtained for each of the weeks were afterwards converted to Pearson Product Moment Correlation Coefficients utilizing Fisher's procedure.  42  Then, the mean Z values for the three weeks of each of the semesters were added up and the total semester mean Z value was calculated and subsequently converted to Pearson Product Moment Correlation Coefficients employing the r-to-Z transformation procedure.  Finally, the significance of the r values of  each of the Semesters II to V, and each of the three weeks of those semesters were assessed against an a level of .05. The UCLA Biomedical Program BMD08V (Appendix J) was used to compute the generalizability coefficients across the nursing judges and weeks. These analyses were carried out with the data from Semesters II, III, and part of V. The fact that, within Semesters IV and V, the same three judges did not score the anecdotal records of all corresponding students—all Semester IV students and some Semester V students—resulted in a not fully crossed design. Therefore, for these two semesters, it was not possible to complete the generalizability analyses.  Phase Two Before conducting the regression analyses called for in phase two, it was first necessary to examine the potential influence of judges. This was completed by comparing the regressionxesuIts using observed ratings with the regression results obtained when scores were standardized within judges.  Regression analyses on a  sample of the data were compared, utilizing standardized scores in one analysis and raw clinical scores in another.  The clinical scores of Week 3 of both Semesters  If and III were then standardized across the three judges following the procedure elaborated in Appendix K. The significance level for the regression analyses was set at a = .10 at each step (Green & Tull, 1965, p. 73). Given the exploratory nature of the study  and the available small sample sizes, a more relaxed level of significance was adopted so as to ensure a reasonable power. The independent variables—first semester grades in Nursing theory, Biology, and Psychology—were quantified by equating the letter grades to numerical grade point equivalents according to the college calendar (refer to operational definitions in Chapter I). Upon inspection of the scores on complexity of the nursing situation, it was noted that they were mainly level 2 across all the students throughout the three weeks. It was subsequently decided to delete this variable because it would not yield enough variance to warrant its addition to the multiple regression equation. Finally, using the stepwise multiple regression program (Appendix L), the criterion scores—mean pooled raw clinical scores and mean standardized clinical scores—were analyzed and their prediction equations were compared to determine whether there was a significant difference between standardized scores and raw scores. Results of this analysis revealed that there were no significant differences between the prediction equations using standardized or raw clinical scores. Consequently, the rest of the data were analyzed using the mean pooled raw clinical scores and employing the linear model:  Y  S C I +SNJ's (2)  o  . . , . , = 3g + 3] Nursing + 32 Biology + 3g Psychology + K 1  n  n  e  The stepwise multiple regression program was used, and the clinical scores from each of the three weeks, and from the three weeks combined, were analyzed for each of Semesters II, III, IV, and V.^  ^ If a student was absent for a week, her score for the three weeks combined was determined by averaging her scores of the other two weeks.  Chapter IV  RESULTS OF ANALYSIS  In this chapter are presented the results of the analyses on the complexity of the nursing situation; the correlations of the inter-rater reliabilities and generalizability coefficients; and the correlations among the clinical performance and Nursing, Biology, and Psychology grades.  Complexity of the Nursing Situation  The independent variable, complexity of the nursing situation, was deleted from the regression analyses of Semesters II, III, IV, and V because complexity scores within each of the three weeks, and between the three weeks, revealed that there was essentially no difference in the complexity of the nursing situation in which these students were asked to work. Possible reasons for this are: (a) the tendency of clinical instructors to perceive that the students' assignments were almost always within their abilities; (b) the phenomenon of central tendency.  Because the instrument had only  three levels of complexity, it may have encouraged instructors to avoid extreme ratings in assessing the complexity level of their students' situations;  44  45 (c) although the levels of complexity were not labeled as such, it appeared that the clinical instructors unconsciously thought of level 2 representing the desired and ideal complexity of students' assignments. The consequence of having to delete the variable, complexity of the nursing situation, limits the study to only academic predictors. Furthermore, it prevents an examination of possible relationships or interactions that may have existed between the academic predictors and complexity of nursing situation.  Inter-Rater Reliabilities of Clinical Scores  Correlational Analyses Semester II. Table 1 shows the mean and the standard deviation for each of the three judges, and the correlations between the judges for each of the three weeks. Also, it shows the mean correlations of each week and the three weeks combined. The mean correlation for Semester II was .75. Semester III. Table 2 gives the correlational results for Semester III. The overall semester correlation was .58. Semester IV. Table 3 gives the correlational results for Semester IV. In Weeks 2 and 3, the r's between Judges 2 and 4 could not be computed because there was no variance in the scores given by Judge 2. The overall semester correlation was .84. Semester V. Table 4 gives the correlational results for Semester V. The overall semester correlation was .73.  TABLE 1 MEANS, STANDARD DEVIATIONS, AND INTERCORRELATIONS A M O N G NURSING JUDGES (J) IN SEMESTER II  Week  Means J  1  J  2  Correlations between Judges  SD J  3  JT  J  0  2  J3  J1J2  J  1 3 J  J  Mean  2 3 J  1  5.00  5.29  4.50  0.88  0.91  0.92  . 58  . 72  . 82  . 72  2  4.75  4.93  4.32  0.83  1.39  0.85  . 62  . 48  . 88  . 70  3  4.61  4.79  4.29  1.13  1.31  0.83  . 80  . 79  . 84  . 81  Combined  .75  Correlations are based on sample size of 14. Mean correlations were obtained by converting r's between judges to Fisher Z's, averaging these, and reconverting them to correlation coefficients.  TABLE 2 MEANS, STANDARD DEVIATIONS, AND INTERCORRELATIONS A M O N G NURSING JUDGES (J) IN SEMESTER III  Week  Means J  l  J  2  Correlations between Judges  SD J  3  J  l  J  2  0  J  3  J  1 2/ J  J  1 3/ N J  J  2 3/ N J  X  M e a n X  /  K 1  N  N  1  5.96  5.82  6.41  1.35  0.75  1.14  .53/ Ml  .49/ Ml  .57/ Ml  .53/ Ml  2  5.75  5.63  5.54  1.06  0.93  1.03  .66/ '12  .55/ M2  .28/ 12  .51/ 12  .88,  .62/ M2  .55 , M2  .72 , 12  3  5.75  5.50  5.38  1.36  0.71  0.88  M2  /  Combined  a  The numbers  /  x  .58/ M2  indicate sample size.  Mean correlations were obtained by converting r's between judges to Fisher Z's, averaging these, and reconverting them to correlation coefficients.  4^  TABLE 3 MEANS, STANDARD DEVIATIONS, AND INTERCORRELATIONS A M O N G NURSING JUDGES (J) IN SEMESTER IV  Week  Means Jl  1  4.55  2  4.46  h 5.25 5.00  h  J  4  6.09 5.36 5.50  Cor reilations between Judges  SD  6.10  h  Jl  1.51 1.06  0.62 0.45  h 1.38 1.08  J  4  2.46 1.60  J  1 2/ 'N J  .63/  '8  .92/  J  1 3/ N J  X  .81 / 7 7  .93/  7  f  3  4.68  5.00  5.40  6.30  1.10  0.47  1.58  1.77  .81 / '7  .84/  '8  J  1 4/ N J  X  • / 91  7  • % 9  .89/  J  2 3/  0  J  J  X  N  .04/ 8  X  N  .76/ 8  X  X  .81 / •7  • / 92  6  3 4/  Mean/ N  .88/ 7  .75/ '15  .59/  .89/ 14  J  7  X  */  • 4 7 6  6  Combined  .85/ 14 X  .84/ 16 X  'The numbers  indicate sample size.  Mean r's were obtained by converting r's between judges to Fisher Z's, averaging these, and reconverting them to correlation coefficients. *Coefficient could not be computed because there was no variance in the scores given by Judge 2 for those students co-rated with Judge 4. CO  49  TABLE 4 MEANS, STANDARD DEVIATIONS, A N D AMONG  Week  Means Jl  1  5.38  J2  5.67  -<3  5.85  INTERCORRELATIONS  N U R S I N G J U D G E S (J) I N S E M E S T E R V  SD J4  5.00  J5  4.93  J  6  5.05  2  Jl  0.79  J2  1.80  J3  Correlations between Judges J4  1.111.16  J  5  1.57  J6  1.38  JlJ3,  .95/  5 8  /  JlJ4/  •  J  4  J l J  5  JlJ6/  /  N  .97/  .79/  •  N  •  3  /  9 4  .82/  /  9 7  /  N  4  4  4  •  • •  8 2  0 7  8 8  .87/  /  4  /  4  /  4  J2J  3  N  •  •  4  5 8  /  N  .69/  %  J2J4/  /  0  J2J5/ N  1.00/  •50/  .70/  '3  3  .50/  v  5  .83/  4  Combined  ° T h e numbers  6  N  .58/  '3  J2J  indicate sample s i z e .  M e a n c o r r e l a t i o n s w e r e o b t a i n e d b y . c o n v e r t i n g r's b e t w e e n judges t o F i s h e r Z ' s , a v e r a g i n g t h e s e , a n d r e c o n v e r t i n g them t o c o r r e l a t i o n c o e f f i c i e n t s .  50 In s u m m a r y , the c o r r e l a t i o n c o e f f i c i e n t s o b t a i n e d i n Semesters I I , I I I , I V , a n d V show t h a t t h e i n t e r - r a t e r r e l i a b i l i t i e s o f ratings of c l i n i c a l p e r f o r m a n c e o f nursing students w e r e m o d e r a t e t o m o d e r a t e l y s t r o n g , v a r y i n g b e t w e e n . 5 8 a n d . 8 4 .  G e n e r a l i z a b i l i t y Analyses Semester I I . for Semester I I .  T a b l e 5 reports a summary o f the g e n e r a l i z a b i l i t y a n a l y s i s  Estimates of the v a r i a n c e components for instructors (I), w e e k s ( W ) ,  a n d w e e k s - b y - j u d g e s (WJ) w e r e n e g a t i v e .  The n e g a t i v e estimates w e r e s u b s t i t u t e d  w i t h z e r o v a l u e s as r e c o m m e n d e d by C r o n b a c h e t a l . ( 1 9 7 2 ) . expected v a r i a n c e s — E o 2 ( ) — f f h x  Q  e  C o n s e q u e n t l y , the  a b o v e sources w e r e set t o z e r o .  In e x a m i n i n g the r e m a i n i n g e i g h t e x p e c t e d v a r i a n c e s , it c a n be seen t h a t the t w o sources of v a r i a n c e — j u d g e s a n d students (S) w e r e r e l a t i v e l y l a r g e , a n d t h e r e f o r e c o n t r i b u t e most to the u n i v e r s e s c o r e v a r i a n c e , a n d h e n c e t o t h e generalizability coefficients.  T a b l e 6 p r o v i d e s a n e x a m p l e o f h o w the u n i v e r s e  s c o r e v a r i a n c e , the o b s e r v e d s c o r e v a r i a n c e , a n d the g e n e r a l i z a b i l i t y c o e f f i c i e n t s of both judges a n d w e e k s w e r e c o m p u t e d .  The g e n e r a l i z a b i l i t y c o e f f i c i e n t s (Ep ) z  for t h e judges a n d w e e k s w e r e . 7 5 a n d . 7 9 , r e s p e c t i v e l y , i n d i c a t i n g t h a t the c l i n i c a l scores w e r e s t a b l e both.across judges a n d o v e r t i m e . Semester III.  T a b l e 7 reports a summary o f the A N O V A w h i c h supports  the g e n e r a l i z a b i l i t y a n a l y s i s for Semester III.  Estimates o f the v a r i a n c e components  for i n s t r u c t o r s , w e e k s , j u d g e s , a n d i n s t r u c t o r s - b y - w e e k s (IW) w e r e n e g a t i v e , a n d therefore g i v e n a v a l u e of z e r o . The largest v a r i a n c e source was the one by s t u d e n t s , f o l l o w e d by the s o u r c e s t u d e n t s - b y - w e e k s nested w i t h i n i n s t r u c t o r ( S W : I ) .  The g e n e r a l i z a b i l i t y  51 TABLE 5 A N A L Y S I S O F V A R I A N C E O F C L I N I C A L S C O R E S O F S E M E S T E R II STUDENTS: EFFECTS O F TIMES O N  JUDGES  Ea (x)  Mean Square  Estimate of Variance Component  Frequency Within  2  2.20  -0.087  3  *0.0000  1.03  2  0.52  -0.014  3  *0.0000  9.23  2  4.61  0.124  3  0.0413  47.85  9  5.32  0.591  12  0.0492  IW  4.79  4  1.20  0.015  9  0.0016  IJ  11.93  4  2.98  0.236  9  0.0262  WJ  0.16  4  0.04  -0.011  9  *0.0000  SW:I  18.40  18  1.02  0.341  36  0.0095  SJ:I  2.74  18  0.15  0.051  36  0.0014  IWJ  1.64  8  0.21  0.009  27  0.0003  SWJ:I  6.14  36  0.17  0.171  108  0.0016  Source of Variance  Instructors  Sum of Squares  Degrees  4.39  of Freedom  2  0) Weeks (W) Judges (J) Students S:l  * E x p e c t e d v a r i a n c e s o f n e g a t i v e estimates o f v a r i a n c e c o m p o n e n t s are set t o z e r o . Ea (x) means e x p e c t e d v a r i a n c e of e a c h s o u r c e . S:l means students nested w i t h i n i n s t r u c t o r .  52 TABLE 6 V A R I A N C E C O M P O N E N T S CONTRIBUTING T O UNIVERSE SCORE V A R I A N C E A N D EXPECTED OBSERVED SCORE V A R I A N C E JUDGES A N D WEEKS, A N D  OF  GENERALIZABILITY  C O E F F I C I E N T S O F S E M E S T E R II  Source of  Universe Score  Expected Observed Score  Variance  Variance  0.0413 0.0492  0.0413 0.0492 0.0262  Generalizability Coefficient  Variance  J  Judges Students: 1  u  IJ WJ  D  SJ:I IWJ  G  SWJ:I  E  Total  0.0000 0.0014 0.0003 0.0016 0.0905  0.1200 0.0905  S  0.1200  W E E  Weeks  0.0000  0.0000  Students: I  0.0492  0.0492  IW  0.0016  Wl  0.0000  SW:I  0.0095  IWJ  0.0003  K  SWJ:I  0.0016  S  Total  0.0492  = 0.75  0.0622  0 ^ = 0 79 0.0622 ' u  Key:  Week = W Judge = J  Student = S S t u d e n t nested w i t h i n Instructor = S;l  Instructor = I Note:  /  A b o v e t e r m i n o l o g y is e l a b o r a t e d i n A p p e n d i x J .  y  53 TABLE 7 A N A L Y S I S O F V A R I A N C E O F C L I N I C A L S C O R E S O F S E M E S T E R III STUDENTS: EFFECTS O F TIMES O N  Source Variance  Sum of Squares  Degrees of Freedom  Mean Square  JUDGES  Estimate o f Variance Component  Frequency Within  Ea (x) 2  0.67  1  0.67  -0.127  2  *0.0000  0.20  2  0.10  -0.128  3  *0.0000  0.52  2  0.26  -0.014  3  *0.0000  75.25  10  7.53  0.836  12  0.0697  IW  2.84  2  1.42  -0.183  6  *0.0000  IJ  5.41  2  2.71  0.109  6  0.0181  WJ  2.55  4  0.64  0.030  9  0.0033  SW:I  94.13  20  4.71  1.569  36  0.0436  SJ:I  14.91  20  0.75  0.249  36  0.0069  IWJ  1.36  4  0.34  0.010  18  0.0005  11.26  40  0.28  0.282  108  0.0026  Instructors  0) Weeks (W) Judges (J) Students S l :  SWJ:I  * E x p e c t e d v a r i a n c e s o f n e g a t i v e estimates o f v a r i a n c e components are set to z e r o . Ea (x) means e x p e c t e d v a r i a n c e o f e a c h s o u r c e . S:l means students nested w i t h i n i n s t r u c t o r .  54 c o e f f i c i e n t s for the judges and w e e k s w e r e . 6 9 a n d . 5 8 , r e s p e c t i v e l y ,  indicating  t h a t the c l i n i c a l scores show more s t a b i l i t y o v e r judges t h a n across t i m e i n Semester III. A p o s s i b l e reason for the l a t t e r c o u l d be e x p l a i n e d by t h e p r e d i c t e d g r o w t h i n students over time. Semester V .  T a b l e 8 reports a summary of the A N O V A w h i c h supports the  g e n e r a l i z a b i l i t y a n a l y s i s for Semester V .  Estimates of the v a r i a n c e components for  instructors, instructors-by-weeks-by-judges (IWJ), instructors-by-judges (IJ), and w e e k s - b y - j u d g e s (WJ) w e r e n e g a t i v e , and t h e r e f o r e g i v e n a v a l u e of z e r o . G e n e r a l i z a b i l i t y c o e f f i c i e n t s for judges and w e e k s w e r e . 9 0 a n d . 6 2 r e s p e c t i v e l y , i n d i c a t i n g that the c l i n i c a l scores show more s t a b i l i t y o v e r judges t h a n across t i m e in Semester V .  A g a i n , a p o s s i b l e reason for the l a t t e r c o u l d be  e x p l a i n e d by the p r e d i c t e d g r o w t h i n the p e r f o r m a n c e o f these students o v e r t i m e .  C o r r e l a t i o n C o e f f i c i e n t s Versus G e n e r a l i z a b i l i t y  T a b l e 9 p r o v i d e s the g e n e r a l i z a b i l i t y c o e f f i c i e n t s of Semesters I I , III, and V .  Coefficients  (G) for judges a n d the c o r r e l a t i o n  Both the G c o e f f i c i e n t a n d the c o r r e l a t i o n  c o e f f i c i e n t for Semester II w e r e i d e n t i c a l , . 7 5 .  The G c o e f f i c i e n t of Semester III,  h o w e v e r , was h i g h e r t h a n the c o r r e l a t i o n c o e f f i c i e n t .  This d i s c r e p a n c y of . 11 is  perhaps d u e to the f a c t that the G c o e f f i c i e n t is a more s e n s i t i v e measure of r e l i a b i l i t y t h a n the c o n v e n t i o n a l c o r r e l a t i o n a l c o e f f i c i e n t .  F i n a l l y , the G  coefficient  of Semester V was . 9 0 , l a r g e r t h a n the c o r r e l a t i o n c o e f f i c i e n t w h i c h was . 7 5 .  The  d i s c r e p a n c y of . 1 5 b e t w e e n t h e t w o c o e f f i c i e n t s of Semester V was not o n l y d u e to  55 TABLE 8 A N A L Y S I S O F V A R I A N C E O F C L I N I C A L S C O R E S O F SEMESTER V S T U D E N T S : EFFECTS O F TIMES O N  JUDGES  Mean Square  Estimate o f Variance Component  Frequency Within  Ea (x)  1  8.96  -0.032  2  *0.0000  30.79  2  15.39  0.465  3  0.1548  6.04  2  3.02  0.131  3  0.0437  Students S:l  39.35  4  9.84  1.093  6  IW  20.90  2  10.45  0.380  6  0.0633  IJ  1.15  2  0.57  -0.009  6  *0.0000  WJ  0.35  4  0.09  -0.101  9  *0.0000  SW:I  56.26  8  7.03  2.344  18  0.1302  SJ:I  5.26  8  0.66  0.219  18  0.0122  IWJ  0.91  4  0.23  -0.156  18  *0.0000  11.13  16  0.70  0.696  54  0.0129  Source of Variance  Instructors  Sum of Squares  Degrees of  8.96  Freedom  2  (1) Weeks (W) Judges (J)  SWJ:I  .  0.1822  * E x p e c t e d v a r i a n c e s o f n e g a t i v e estimates o f v a r i a n c e c o m p o n e n t s are set to z e r o . E a ( x ) means e x p e c t e d v a r i a n c e o f e a c h s o u r c e . 2  S:l means students nested w i t h i n i n s t r u c t o r .  56  TABLE 9 A C O M P A R I S O N OF GENERALIZABILITY COEFFICIENTS A N D M E A N CORRELATION COEFFICIENTS ACROSS J U D G E S I N SEMESTERS II, III, A N D V a  Mean Semester  Generalizability  Correlation Coefficients  II  .75  .75  II  .69  .58  V  .90  *.75  *This m e a n c o r r e l a t i o n c o e f f i c i e n t was c o m p u t e d on t h e scores o f t h e same s a m p l e s i z e of s i x students t h a t was used to c a l c u l a t e the G c o e f f i c i e n t i n Semester V . a  S e m e s t e r IV d a t a w e r e not s u i t a b l e for g e n e r a l i z a b i l i t y a n a l y s i s b e c a u s e the same t h r e e judges d i d not s c o r e the a n e c d o t a l records o f a l l c o r r e s p o n d i n g s t u d e n t s .  57 the f a c t t h a t G c o e f f i c i e n t is a more s e n s i t i v e measure o f r e l i a b i l i t y , but a l s o p o s s i b l y t h a t o n l y the d a t a g e n e r a t e d by a s m a l l s a m p l e o f s i x students was u t i l i z e d to c o m p u t e the g e n e r a l i z a b i l i t y c o e f f i c i e n t . In s u m m a r y , the g e n e r a l i z a b i l i t y c o e f f i c i e n t s a n d the c o r r e l a t i o n c o e f f i c i e n t s across the t h r e e judges w e r e c o m p a r a b l e i n Semesters I I , III, a n d V . Phase t w o a n a l y s e s — m u l t i p l e r e g r e s s i o n s — w e r e thus i n i t i a t e d .  Regression A n a l y s e s  Preliminary Analyses:  Raw C l i n i c a l Scores Versus S t a n d a r d i z e d C l i n i c a l Scores  Semester I I .  T a b l e 10 shows the m e a n s , standard d e v i a t i o n , c o r r e l a t i o n  c o e f f i c i e n t s , and c o e f f i c i e n t o f d e t e r m i n a t i o n ( R ) for both the raw a n d the 2  s t a n d a r d i z e d c l i n i c a l scores of Semester I I , W e e k 3 .  Comparing the statistics  o f the raw c l i n i c a l scores t o the s t a n d a r d i z e d c l i n i c a l s c o r e s , w e n o t i c e that t h e i r s t a n d a r d d e v i a t i o n s , c o r r e l a t i o n c o e f f i c i e n t s , a n d the R For e x a m p l e , the R .432, respectively.  2  z  are very c l o s e .  of the raw and s t a n d a r d i z e d c l i n i c a l scores w e r e . 4 1 9 a n d F u r t h e r m o r e , the o r d e r o f e n t r y o f the i n d e p e n d e n t v a r i a b l e s  i n t h e regression e q u a t i o n s was the s a m e — P s y c h o l o g y f i r s t , t h e n B i o l o g y , a n d finally Nursing. In c o n c l u s i o n , t h e r e w e r e n o s i g n i f i c a n t d i f f e r e n c e s b e t w e e n the regression results o f the raw a n d s t a n d a r d i z e d c l i n i c a l scores of Semester I I , W e e k 3 . Semester III. R  2  The m e a n , s t a n d a r d d e v i a t i o n , c o r r e l a t i o n m a t r i x , a n d  f o r both the raw and the s t a n d a r d i z e d c l i n i c a l scores o f Semester III, W e e k 3  a r e shown i n T a b l e 1 1 .  L i k e the Semester II regression a n a l y s e s , the results o f  T A B L E 10 S U M M A R Y O F S T E P W I S E M U L T I P L E R E G R E S S I O N A N A L Y S I S O F W E E K 3 O F S E M E S T E R II: A C O M P A R I S O N O F RAW C L I N I C A L SCORES TO STANDARDIZED C L I N I C A L SCORES  Raw vs  Statistics Variable  X  Clinical  Standardized  R2  r  SD Nursing  Psychology  Biology  .620  -.015  .534  -.255  .419  -.280  .385  Scores  C l i n i c a l score  4.708  0.899  Nursing  2.417  0.515  Psychology  2.833  0.937  Biology  2.083  0.289  -0.002  0.852  Nursing  2.417  0.515  Psychology  2.833  0.937  Biology  2.083  0.289  .384  Raw Clinical Scores  C l i n i c a l score  .412  .394  .636  -.039  .534  -.255  .432  -.280  .404  Standardized Clinical Scores  N = 12 (the same s a m p l e o f 12 students used in g e n e r a l i z a b i l i t y a n a l y s i s )  .425  T A B L E 11 S U M M A R Y O F STEPWISE M U L T I P L E R E G R E S S I O N A N A L Y S I S O F W E E K 3 O F S E M E S T E R III: A C O M P A R I S O N O F RAW C L I N I C A L SCORES T O S T A N D A R D I Z E D C L I N I C A L SCORES  Statistics  Raw vs  Variable  Standardized  r X  SD  Clinical  Nursing  Psychology  Biology  Scores  C l i n i c a l score  5.541  0.879  Nursing  2.833  0.577  Psychology  2.833  0.577  Biology  3.000  0.853  -0.021  0.857  Nursing  2.833  0.577  Psychology  2.833  0.577  Biology  3.000  0.853  .761  .761  .465  1.000  .369  Raw .579  Clinical .369  Scores  C l i n i c a l score  .618  .742  .742  .448  1.000  .369  Standardized .551  Clinical .369  Scores  N = 12  .586  60 Semester III r e v e a l e d t h a t the standard d e v i a t i o n s , c o r r e l a t i o n c o e f f i c i e n t s , a n d the R^'s of both the raw a n d t h e s t a n d a r d i z e d c l i n i c a l scores w e r e a p p r o x i m a t e l y the s a m e .  For e x a m p l e , the R^'s of the raw a n d the s t a n d a r d i z e d a n a l y s e s w e r e  .618 and . 5 8 6 , r e s p e c t i v e l y .  F u r t h e r m o r e , the order o f the e n t r y o f the i n d e p e n d e n t  v a r i a b l e s i n the regression e q u a t i o n s was the s a m e — N u r s i n g a n d B i o l o g y .  It s h o u l d  be n o t e d t h a t b e c a u s e N u r s i n g a n d P s y c h o l o g y w e r e p e r f e c t l y c o r r e l a t e d (r = 1 . 0 0 ) , the computer algorism selected N u r s i n g . In c o n c l u s i o n , there w e r e no i m p o r t a n t d i f f e r e n c e s b e t w e e n the results of the regression a n a l y s e s o f the r a w a n d the s t a n d a r d i z e d c l i n i c a l scores of W e e k 3 o f Semesters II and III.  H e n c e , it was d e c i d e d t o use the mean r a w c l i n i c a l scores  to d o t h e regression a n a l y s e s — t h e c l i n i c a l scores g i v e n b y t h e t h r e e nursing judges—without standardizing them.  Final Analyses:  Raw C l i n i c a l Scores  Semester I I .  T a b l e 12 shows the m e a n s , s t a n d a r d d e v i a t i o n s ,  correlation  m a t r i c e s , c o e f f i c i e n t s o f d e t e r m i n a t i o n (R ) a n d the p a r t i a l r's for the regression a n a l y s i s o f the d a t a o f Semester I I .  The results o f W e e k 1 show t h a t o n l y P s y c h o l o g y  e n t e r e d the m u l t i p l e regression e q u a t i o n , p r o d u c i n g an R s i g n i f i c a n t at the . 1 0 l e v e l or b e t t e r .  z  o f . 2 5 9 , w h i c h was  A n i n s p e c t i o n o f the p a r t i a l r's shows t h a t  B i o l o g y w o u l d h a v e e n t e r e d n e x t i n the regression e q u a t i o n .  H o w e v e r , its  c o n t r i b u t i o n to the p r e d i c t i o n of the c l i n i c a l scores was not s i g n i f i c a n t at the .10 l e v e l .  C o n s e q u e n t l y , i t d i d not e n t e r the regression e q u a t i o n .  T A B L E 12 S U M M A R Y O F STEPWISE M U L T I P L E R E G R E S S I O N A N A L Y S E S : S E M E S T E R II  Statistics Week  Variable  X  Clinical  C l i n i c a l score 1  2  3  Combined  0.81  Nursing  4.93 2.21  Psychology  2.79  Biology  2.07  0.89 0.27  C l i n i c a l score  4.67  0.91  Nursing  2.21  Psychology  2.79  0.80 0.89  Biology  2.07  0.27  C l i n i c a l score Nursing  4.56 2.21  1.02 0.80  Psychology  2.79  0.89  Biology  2.07  0.27  C l i n i c a l score  4.72  0.75  Nursing  2.21  0.80  Psychology Biology  N = 14  Nursing  .104  0.89  2.07  0.27  Psychology  .509 .499  0.80  2.79  R*  r  SD  Partial r  Biology  .085 -.077 -.253  -.201 .259 .257  .580  .411 .499  -.053 -.077  .337  -.253  .356 .034  .643  .636 .499  .031 -.077 -.253  .560  .632 .499  .023 -.077 -.253  .413 .545 .577  .480 .400 .509  62 The results of W e e k 2 show that both N u r s i n g and P s y c h o l o g y e n t e r e d i n t h e regression e q u a t i o n , p r o d u c i n g a n R  of .356, w h i c h was s i g n i f i c a n t at a =  .10.  The p a r t i a l r of B i o l o g y , w h e n both N u r s i n g a n d P s y c h o l o g y had e n t e r e d in the regression e q u a t i o n , was .034,  c o n f i r m i n g that it w o u l d not h a v e a d d e d much t o  the regression c o e f f i c i e n t . The results of W e e k 3, on the o t h e r h a n d , show t h a t a l l three i n d e p e n d e n t v a r i a b l e s e n t e r e d the regression e q u a t i o n , p r o d u c i n g a n R s i g n i f i c a n t at a = .10.  The order o f the e n t r y was:  2  w h i c h was  = .577,  Nursing, Psychology,  Biology. F i n a l l y , t h e regression results of W e e k s 1, 2 , a n d 3 c o m b i n e d show that o  a l l three v a r i a b l e s e n t e r e d the regression e q u a t i o n , p r o d u c i n g an R was s i g n i f i c a n t a t the .10 l e v e l . i n d e p e n d e n t v a r i a b l e s was:  of .509,  which  In contrast to W e e k 3, the order o f e n t r y of t h e  Psychology, Nursing,  Biology.  In s u m m a r y , 51 p e r c e n t o f the v a r i a n c e o f t h e c o m b i n e d c l i n i c a l scores in Semester II c a n be a t t r i b u t e d t o v a r i a t i o n s i n grades in P s y c h o l o g y , N u r s i n g , a n d Biology. Semester III.  T a b l e 13 shows the m e a n s , s t a n d a r d d e v i a t i o n s ,  correlation  m a t r i c e s , c o e f f i c i e n t s o f d e t e r m i n a t i o n (R ), a n d the p a r t i a l r's for the regression 2  a n a l y s e s of the d a t a o f Semester III.  A n e x a m i n a t i o n of the c o r r e l a t i o n m a t r i c e s  shows that N u r s i n g and P s y c h o l o g y had a c o r r e l a t i o n c o e f f i c i e n t o f  1.00.  C o n s e q u e n t l y , o n l y one of these t w o v a r i a b l e s w o u l d h a v e had the o p p o r t u n i t y t o e n t e r in the regression e q u a t i o n ( a = .10).  H o w e v e r , e i t h e r of these v a r i a b l e s —  P s y c h o l o g y or N u r s i n g — w h e n used a l o n e i n the regression a n a l y s i s , w o u l d h a v e y i e l d e d s u b s t a n t i a l l y the same r e s u l t s .  T A B L E 13 S U M M A R Y O F STEPWISE M U L T I P L E R E G R E S S I O N A N A L Y S E S :  Week  Variable  N  X  SEMESTER III  Correlation Matrix  SD Clinical  Nursing  Psychology  Biology  R  2  Partial r  Score  1  2  3  Combined  C l i n i c a l score  11 11  6.06 2.82  0.89  Nursing  .45  0.60  Psychology  11  2.82  0.60  Biology  11  3.09  0.83  C l i n i c a l score  5.64  0.82  Nursing  12 12  2.83  0.58  Psychology  12  2.83  0.58  Biology  12  3.00  0.85  C l i n i c a l score  5.54  0.88  Nursing  12 12  2.83  0.58  Psychology  12  2.83  0.58  Biology  12  3.00  0.85  C l i n i c a l score  12  5.74  0.71  Nursing  12  2.83  0.58  Psychology  12  2.83  0.58  Biology  12  3.00  0.85  .45  .46  1.00  .43 .43  .79  .79 1.00  .65 .37  .621  .37  .000 .768  .76  .76  .47  1.00  .37  .579  .37  .000 .618  .80  .80  .62  1.00  .37 .37  .637 .000 .760  D a t a for one student was missing for one w e e k ; i n this case c l i n i c a l scores for W e e k s 1, 2 , and 3 c o m b i n e d w e r e c o m p u t e d by a v e r a g i n g her scores on the r e m a i n i n g t w o w e e k s .  64 In Week 1, none of the variables entered the regression equation.  Unlike  Week 1, in Week 2 both Nursing and Biology entered the regression equation, producing an R of .768 which was significant at the .10 level or better. 2  The  partial r of Psychology was . 0 0 0 , an expected v a l u e , given the fact that Psychology and Nursing had a perfect r of 1 . 0 0 . In Week 3, again Nursing and Biology entered in the regression equation, producing an R of .620 which was significant at a = . 1 0 . F i n a l l y , combining Weeks 1, 2, and 3 yielded R  . 7 6 , significant at  a = .10. In summary, 76 percent of the variance of the c l i n i c a l scores of Semester III students could be attributed to their first semester Nursing theory and Biology grades. Semester IV.  Table 14 shows the means, standard deviations and the  correlation matrices for Semester IV.  An inspection of the correlation matrices  shows that the correlation coefficients between the dependent v a r i a b l e — c l i n i c a l scores—and the independent v a r i a b l e s — N u r s i n g , Psychology, and Biology—were smaller than the correlation coefficients between the independent variables.  The  above, together with the problem of small sample s i z e , created a situation where none of the variables entered the regression equation at the set a level of . 1 0 . Semester V .  Table 15 shows the means, standard deviations and the  correlation matrices for Semester V .  Like Semester IV, the correlation coefficients  between the dependent and the independent variables were in general smaller than the r's between the independent variables.  The latter, together with the problem  T A B L E 14 S U M M A R Y O F STEPWISE M U L T I P L E R E G R E S S I O N A N A L Y S E S :  Week  Variable  N  X  S E M E S T E R IV  Correlation Matrix  SD Clinical  Nursing  Psychology  Biology  .331  .282  .015  .620  .514 .452  .359 .552  .250  Score  15  5.31  1.42 0.70  Psychology  15 15  2.93  Biology  15  3.00 2.67  2.65 0.72  C l i n i c a l score  14  5.24  0.90  C l i n i c a l score 1  2  3  -  Combined  Nursing  Nursing  14  Psychology  14  3.00 3.07  0.68 0.62  Biology  14  2.79  0.70  C l i n i c a l score  14  5.35  Nursing  14  2.85  1.20 0.69  Psychology  14  3.00  0.71  Biology  14  2.77  0.73  C l i n i c a l score  16  0.93  Nursing  16  5.33 2.94  Psychology  16  3.00  0.63  Biology  16  2.69  0.70  .397  .486 .396  .289  .213  .020  .684  .591 .488  .389  0.68  .298  .037  .620  .513 .449  D a t a for f i v e students was missing for one w e e k ; in these cases c l i n i c a l scores for W e e k s 1, 2, c o m b i n e d w e r e c o m p u t e d by a v e r a g i n g t h e i r scores on the r e m a i n i n g two w e e k s .  and 3  T A B L E 15 S U M M A R Y O F STEPWISE M U L T I P L E R E G R E S S I O N A N A L Y S E S :  Week  Variable  N  X  SEMESTER V  Correlation Matrix  SD Clinical  Nursing  Psychology  Biology  .303  .035  -.115  .499  Score  C l i n i c a l score 1  2  Combined  5.30 3.12  1.16 0.86 0.78 0.64  Nursing  17  Psychology  17  3.12  Biology  17  2.82  C l i n i c a l score  16 16  4.73 3.19  0.91  3.13  Biology  16 16  0.81 0.62  C l i n i c a l score  17  Nursing Psychology  3  17  2.88  Nursing  17  5.29 3.12  Psychology  17  3.12  Biology  17  2.82  C l i n i c a l score  17  Nursing Psychology  17 17  5.14 3.12 3.12  Biology  17  2.82  -.311  -.333  .056  0.83  0.98 0.78 0.64  .094  -.136  .436 -.367  .095  .349 -.115  0.86  0.83  .109  .002 .499 -.333  -.137  0.86 0.78 0.64  .311 -.115  .002 .499 -.333  D a t a for o n e student was missing for o n e w e e k ; in this case c l i n i c a l scores for W e e k s 1, 2 , a n d 3 c o m b i n e d w e r e c o m p u t e d by a v e r a g i n g her scores on the r e m a i n i n g two w e e k s .  67 o f s m a l l s a m p l e s i z e , c r e a t e d a s i t u a t i o n w h e r e the c o n t r i b u t i o n of the v a r i a b l e s was not s i g n i f i c a n t at the . 1 0 l e v e l , and t h e r e f o r e d i d not e n t e r in the regression equation. In s u m m a r y , 51 a n d 7 6 p e r c e n t of the v a r i a n c e i n the c l i n i c a l p e r f o r m a n c e scores of Semesters II a n d III nursing students c o u l d be a c c o u n t e d for by t h e i r first semester N u r s i n g t h e o r y , B i o l o g y , a n d P s y c h o l o g y g r a d e s .  H o w e v e r , i n both  Semesters IV a n d V , the c o e f f i c i e n t s of d e t e r m i n a t i o n w e r e s m a l l a n d not s i g n i f i c a n t . Further discussions of these f i n d i n g s are presented i n C h a p t e r V .  Chapter V  SUMMARY,  FINDINGS AND  LIMITATIONS, A N D  CONCLUSIONS,  RECOMMENDATIONS  The purpose of this study was to i n c r e a s e k n o w l e d g e of factors w h i c h c o r r e l a t e w i t h the c l i n i c a l p e r f o r m a n c e of nursing students, t h e r e b y assisting nurse e d u c a t o r s to b e t t e r p r e d i c t students' p e r f o r m a n c e , and a c c o r d i n g l y g i v e counsel. A s e a r c h of the l i t e r a t u r e r e v e a l e d that v e r y f e w studies h a v e b e e n d o n e i n the a r e a o f p r e d i c t i o n of c l i n i c a l p e r f o r m a n c e of nursing s t u d e n t s .  O f the f e w  studies d o n e , h o w e v e r , the reported r e l a t i o n s h i p s b e t w e e n p e r f o r m a n c e i n t h e o r y courses and c l i n i c a l p r a c t i c e courses w e r e w e a k .  T a y l o r et a l . (1965) suggested  that perhaps the c l i n i c a l grades represent measures of b e h a v i o r s a n d a b i l i t i e s a d d i t i o n a l to those measured by a c a d e m i c c o u r s e s .  F u r t h e r m o r e , none of the  studies addressed the q u e s t i o n of r e l i a b i l i t y of the c l i n i c a l grades that w e r e b e i n g predicted.  The l a t t e r has d i s t u r b e d many nurse e d u c a t o r s , w h o h a v e expressed  d i s s a t i s f a c t i o n w i t h the t y p i c a l process of c l i n i c a l e v a l u a t i o n . The p r o b l e m of the present study was to d e t e r m i n e the r e l a t i v e d e g r e e to w h i c h (a) a c a d e m i c grades and (b) c o m p l e x i t y of the nursing s i t u a t i o n p r e d i c t scores i n c l i n i c a l p e r f o r m a n c e for first and s e c o n d y e a r nursing s t u d e n t s .  68  69 The student s a m p l e c o n s i s t e d of 14 Semester II students; 12 Semester III students; 16 Semester IV students; a n d 17 Semester V s t u d e n t s .  O f the 16  instructors w h o t o o k part i n the s t u d y , t w o a c t e d o n l y as nursing j u d g e s , a n d 14 a c t e d as both c l i n i c a l instructors a n d j u d g e s . R i n e s ' (1963) a n e c d o t a l r e c o r d form was a d a p t e d t o meet the needs of the s t u d y .  The s c o r i n g i n s t r u m e n t , d e s i g n e d b y the i n v e s t i g a t o r ,  contained  c r i t e r i a w i t h w h i c h the c l i n i c a l instructors a n d judges w e r e t o use a 9 p o i n t L i k e r t - t y p e s c a l e to rate t h e d e g r e e o f student a c h i e v e m e n t o f stated c l i n i c a l course o b j e c t i v e s .  The s c a l e was made more e x p l i c i t by e m p l o y i n g the  c o n v e n t i o n a l c o l l e g e g r a d i n g system to e x p l a i n the n u m e r i c a l scores on the scale.  For e x a m p l e , a n y s c o r e from 8 t o 9 was e q u i v a l e n t t o s u p e r i o r  p e r f o r m a n c e , or a l e t t e r g r a d e o f " A " s t a n d i n g . The instrument for m e a s u r i n g c o m p l e x i t y of the nursing s i t u a t i o n was a l s o d e s i g n e d for this s t u d y . c o m p l e x i t y was u s e d .  A t h r e e p o i n t s c a l e r e f l e c t i n g d i f f e r e n t l e v e l s of  E a c h l e v e l o f c o m p l e x i t y was d e s c r i b e d b y four m a j o r  c r i t e r i a w h i c h w e r e to be used by the c l i n i c a l instructors to m a k e t h e i r d e c i s i o n s (see A p p e n d i x G ) . The v i d e o t a p e - e v a l u a t i o n o f a student's c l i n i c a l p e r f o r m a n c e was d e s i g n e d by the i n v e s t i g a t o r .  Its purpose was to t r a i n instructors i n the w r i t i n g  o f a n e c d o t a l records a b o u t s t u d e n t s ' c l i n i c a l p e r f o r m a n c e .  The t a p e , c o n s i s t i n g  of f i v e v i g n e t t e s , ran for a t o t a l of n i n e minutes a n d d e p i c t e d the p e r f o r m a n c e o f a nursing student c a r i n g for t w o p a t i e n t s .  70 S e p a r a t e workshops w e r e c o n d u c t e d by the i n v e s t i g a t o r to t r a i n the instructors o f Semesters II, III, I V , a n d V , in w r i t i n g and s c o r i n g a n e c d o t a l r e c o r d s , a n d d e t e r m i n i n g the l e v e l of c o m p l e x i t y of the nursing s i t u a t i o n . workshops w e r e c o n d u c t e d in t w o p a r t s .  The  F i r s t , the a d v a n t a g e s a n d d i s a d v a n t a g e s  of the a n e c d o t a l r e c o r d t e c h n i q u e o f e v a l u a t i o n w e r e d i s c u s s e d , f o l l o w e d by a d i s c u s s i o n of e a c h o f the f i v e v i g n e t t e s .  This was f o l l o w e d by p r a c t i c e i n  r e c o r d i n g a n d s c o r i n g the a n e c d o t a l r e c o r d o f the s t u d e n t ' s p e r f o r m a n c e o b s e r v e d on the v i d e o t a p e , and d e t e r m i n i n g the l e v e l o f c o m p l e x i t y o f this student's c l i n i c a l assignment. D a t a for t h e study w e r e c o l l e c t e d o v e r a t h r e e - w e e k p e r i o d . data included:  These  (a) c l i n i c a l p e r f o r m a n c e scores o b t a i n e d from a n e c d o t a l records  o f s t u d e n t s ' c l i n i c a l p e r f o r m a n c e p r e p a r e d b y t h e i r instructors a n d , s u b s e q u e n t l y , s c o r e d by the c l i n i c a l i n s t r u c t o r and t w o nursing judges using the student's c l i n i c a l o b j e c t i v e s a n d the s c o r i n g instrument; (b) r a t i n g o f the l e v e l o f c o m p l e x i t y o f the s t u d e n t s ' c l i n i c a l assignments by t h e i r c l i n i c a l i n s t r u c t o r s , u s i n g the c o m p l e x i t y o f the nursing s i t u a t i o n instrument; a n d (c) first semester N u r s i n g t h e o r y , P s y c h o l o g y , a n d B i o l o g y grades o b t a i n e d from the s t u d e n t s ' p e r m a n e n t records in the Registrar's o f f i c e . D a t a a n a l y s e s w e r e c a r r i e d out in t w o p h a s e s .  Phase o n e i n v o l v e d the  u t i l i z a t i o n of t w o s t a t i s t i c a l methods to a n a l y z e the r e l i a b i l i t y o f t h e c l i n i c a l p e r f o r m a n c e scores: analyses.  (1) c o n v e n t i o n a l c o r r e l a t i o n a n a l y s e s ; a n d (2) g e n e r a l i z a b i l i t y  The d e p e n d e n t v a r i a b l e was the a v e r a g e o f t h e c l i n i c a l p e r f o r m a n c e  scores o f the students g i v e n by the c l i n i c a l instructors (who w r o t e a n d s c o r e d  71 t h e a n e c d o t a l records) a n d t w o nursing judges (who s c o r e d t h e w r i t t e n a n e c d o t a l records).  The i n d e p e n d e n t v a r i a b l e s w e r e t h e grades i n first semester N u r s i n g ,  B i o l o g y , and P s y c h o l o g y ; a n d the c o m p l e x i t y o f the nursing s i t u a t i o n .  In phase  t w o , m u l t i p l e regression a n a l y s e s w e r e c o n d u c t e d t o d e t e r m i n e the p r o p o r t i o n o f v a r i a n c e of the c l i n i c a l p e r f o r m a n c e scores in Semesters I I , I I I , I V , a n d V a c c o u n t e d for by the a c a d e m i c grades o b t a i n e d i n Semester I N u r s i n g , P s y c h o l o g y , and Biology.  Findings and Conclusions  Both the g e n e r a l i z a b i l i t y a n d c o r r e l a t i o n a l a n a l y s e s showed t h a t the i n t e r - r a t e r r e l i a b i l i t i e s among the t h r e e judges i n a l l o f the four semesters w e r e m o d e r a t e t o m o d e r a t e l y strong (from . 5 8 t o . 8 4 ) .  The g e n e r a l i z a b i l i t y  c o e f f i c i e n t s w e r e u s u a l l y h i g h e r than t h e c o r r e l a t i o n c o e f f i c i e n t s .  (G)  This d i s c r e p a n c y  c a n be e x p l a i n e d by the f a c t that t h e G c o e f f i c i e n t is a more s e n s i t i v e measure of r e l i a b i l i t y w h e n c o m p a r e d w i t h t h e c o n v e n t i o n a l c o r r e l a t i o n c o e f f i c i e n t . G i v e n t h a t the c l i n i c a l scores w e r e f o u n d t o be s u f f i c i e n t l y r e l i a b l e , t h e d a t a a n a l y s i s p r o c e e d e d t o phase t w o .  I n s p e c t i o n o f the ratings of the l e v e l s  of c o m p l e x i t y o f t h e nursing s i t u a t i o n i n e a c h semester r e v e a l e d m i n i m a l t o no v a r i a t i o n from l e v e l t w o c o m p l e x i t y .  C o n s e q u e n t l y , the i n d e p e n d e n t v a r i a b l e  c o m p l e x i t y of the nursing s i t u a t i o n c o u l d not be used in t h e regression a n a l y s i s .  72 The regression a n a l y s i s o f Semester II d a t a r e v e a l e d t h a t 51 p e r c e n t of t h e v a r i a n c e o f the c l i n i c a l scores c o u l d be e x p l a i n e d b y the t h r e e r e m a i n i n g i n d e p e n d e n t v a r i a b l e s — a c a d e m i c grades i n first semester N u r s i n g , and Biology.  Psychology,  The squared m u l t i p l e c o r r e l a t i o n c o e f f i c i e n t was s i g n i f i c a n t at  .10 level of s i g n i f i c a n c e . For Semester III, 7 6 p e r c e n t o f the v a r i a n c e o f the c l i n i c a l s c o r e was a c c o u n t e d for by t h e N u r s i n g or P s y c h o l o g y g r a d e s , a n d the B i o l o g y grades (p. < . 1 0 ) .  In this a n a l y s i s , N u r s i n g a n d P s y c h o l o g y grades w e r e p e r f e c t l y  correlated.  C o n s e q u e n t l y , e i t h e r o f these v a r i a b l e s , w h e n used a l o n e , e x p l a i n e d  t h e same amount of v a r i a n c e i n the c l i n i c a l s c o r e s . In c o n t r a s t , the regression a n a l y s e s for Semesters IV a n d V r e v e a l e d t h a t o n l y 20 and 16 p e r c e n t of t h e v a r i a n c e o f the c l i n i c a l scores r e s p e c t i v e l y , was a c c o u n t e d for b y t h e t h r e e i n d e p e n d e n t v a r i a b l e s . significant.  These results w e r e not  In s u m m a r y , i t appears t h a t t h e a c a d e m i c grades i n N u r s i n g ,  P s y c h o l o g y , and B i o l o g y courses o b t a i n e d i n first semester o f the nursing program are b e t t e r p r e d i c t o r s o f c l i n i c a l p e r f o r m a n c e i n e a r l i e r semesters, Semesters II a n d III, t h a n i n l a t e r semesters, Semesters I V a n d V . Based on the a b o v e f i n d i n g s , the f o l l o w i n g c o n c l u s i o n s w e r e made: 1.  The a n e c d o t a l r e c o r d m e t h o d o f c l i n i c a l p e r f o r m a n c e e v a l u a t i o n y i e l d e d  a c c e p t a b l y r e l i a b l e c l i n i c a l scores w h e r e a l l the c l i n i c a l instructors (those instructors w h o w r o t e t h e a n e c d o t a l records) and a l l the judges (those w h o e v a l u a t e d the w r i t t e n a n e c d o t a l records) w e r e t r a i n e d u s i n g a p l a n n e d i n s t r u c t i o n a l p r o g r a m . 2.  The grades in P s y c h o l o g y , N u r s i n g , a n d B i o l o g y courses d u r i n g the first  semester of t h e nursing program a c c o u n t e d for a s i g n i f i c a n t p r o p o r t i o n o f the  73 v a r i a n c e i n the c l i n i c a l p e r f o r m a n c e scores of nursing students i n Semester I I . L i k e w i s e , N u r s i n g or P s y c h o l o g y , and B i o l o g y grades o b t a i n e d i n t h e first semester of the nursing program a c c o u n t e d for a s i g n i f i c a n t p r o p o r t i o n of the v a r i a n c e i n t h e c l i n i c a l p e r f o r m a n c e scores o f Semester III nursing s t u d e n t s . In Semesters IV a n d V , h o w e v e r , these grades f a i l e d to a c c o u n t for a s i g n i f i c a n t p r o p o r t i o n o f the v a r i a n c e in c l i n i c a l p e r f o r m a n c e scores of these students.  T h e r e f o r e , hypothesis 1—grades in N u r s i n g , B i o l o g y , a n d P s y c h o l o g y  courses d u r i n g t h e first semester o f the nursing program a c c o u n t for a s i g n i f i c a n t p r o p o r t i o n o f t h e v a r i a n c e i n c l i n i c a l p e r f o r m a n c e scores o f s t u d e n t s — i s a c c e p t e d i n the c a s e o f Semester II a n d III s t u d e n t s , a n d r e j e c t e d in t h e case o f Semester IV and V students. 3.  The c o m p l e x i t y o f t h e nursing s i t u a t i o n c o u l d not be used i n the regression  a n a l y s i s b e c a u s e it s h o w e d no v a r i a b i l i t y across the t h r e e w e e k s and the four semesters.  T h e r e f o r e , h y p o t h e s i s 2 — c o m p l e x i t y o f the nursing s i t u a t i o n a c c o u n t s  for a n a d d i t i o n a l s i g n i f i c a n t p r o p o r t i o n o f the v a r i a n c e i n c l i n i c a l p e r f o r m a n c e scores o f s t u d e n t s — c o u l d not be t e s t e d . There a r e s e v e r a l i n t e r r e l a t e d factors w h i c h m a y e x p l a i n the d i f f e r e n c e s n o t e d b e t w e e n Semesters II a n d III a n d Semesters IV a n d V c l i n i c a l p e r f o r m a n c e a n d h e n c e the m u l t i p l e c o r r e l a t i o n s w i t h the i n d e p e n d e n t v a r i a b l e s s e l e c t e d . A s t h e nursing students progress from one semester to a n o t h e r , t h e i r r e s p o n s i b i l i t i e s increase.  T h e y are e x p e c t e d t o (1) be more i n d e p e n d e n t ; (2) h a v e a c q u i r e d a  r e p e r t o i r e o f s k i l l s a n d c o m p e t e n c i e s to m a k e more i n d e p e n d e n t d e c i s i o n s t h a t r e q u i r e h i g h e r l e v e l s of c o g n i t i v e a b i l i t i e s ; (3) p r o v i d e nursing c a r e t o i n d i v i d u a l s  74 with complex health problems; (4) provide nursing care for a larger number of individuals; (5) assume a greater role as a member of the nursing team; and (6) require less direct instructor supervision. A l l the above factors may have influenced the nature of the students' c l i n i c a l assignments, the amount of required and/or desired instructor supervision, and the accessibility of the c l i n i c a l instructor to the student.  Based on this  information, it can be safely assumed that as the students progress from Semester II to Semester V , their c l i n i c a l assignments w i l l increase in number and complexity, they w i l l require less direct instructor supervision, and w i l l have less access to their c l i n i c a l instructors.  It can be conjectured that the aforementioned may  affect the nature of the content of the anecdotal records written by the c l i n i c a l instructors, and hence the c l i n i c a l scores of the students.  Consequently, it can  be expected that these factors may influence the performance scores of Semester IV and V students more than Semester II or III students.  Therefore, to the extent that  the nature of the students' assignments change as a result of their natural progression in the nursing program, their c l i n i c a l performance may be a f f e c t e d , and thus may not correlate as highly with previously identified variables. Furthermore, there were temporal factors which may have contributed to the differences noted in the multiple correlation coefficients of Semesters IV and V .  These senior students had additional c l i n i c a l experiences, and higher  level Nursing, Psychology, and Biology courses which may have affected their attitudes, motivation, competencies, perceptions of the role of the nurse, perceptions of the reality of c l i n i c a l nursing, hence their c l i n i c a l performance and c l i n i c a l scores.  75 F i n a l l y , a p o s s i b l e f a c t o r w h i c h may h a v e i n f l u e n c e d the p e r f o r m a n c e o f Semester II and III students  was the i n c r e a s e i n the amount a n d t y p e of  c l i n i c a l e x p e r i e n c e s o f these students i n t h e i r first semester o f n u r s i n g .  These  j u n i o r students had more c l i n i c a l e x p e r i e n c e s in t h e i r Semester I c l i n i c a l nursing course c o m p a r e d t o the s e n i o r students o f Semesters IV a n d V .  It is i m p o r t a n t  to n o t e , h o w e v e r , t h a t this f a c t o r has s i m i l a r i t i e s to the t e m p o r a l factors d e s c r i b e d a b o v e , w h i c h w e r e c o n j e c t u r e d to h a v e c o n t r i b u t e d to the d i f f e r e n c e s n o t e d i n the m u l t i p l e c o r r e l a t i o n c o e f f i c i e n t s b e t w e e n Semesters II and III a n d Semesters IV and V . There w e r e s e v e r a l factors w h i c h m a y h a v e i n f l u e n c e d the ratings o f the i n d e p e n d e n t v a r i a b l e , the c o m p l e x i t y of the nursing s i t u a t i o n , y i e l d i n g c o m p l e x i t y l e v e l s w h i c h w e r e e s s e n t i a l l y the s a m e — l e v e l t w o . for l e v e l t w o c o m p l e x i t y w e r e :  The c r i t e r i a  1. a c t i v i t i e s o f the s t u d e n t — t r e a t m e n t s p r e s c r i b e d  b y the d o c t o r a n d / o r the nursing team are moderate in number; the student has had numerous o p p o r t u n i t i e s to p r a c t i s e the s k i l l s a n d a c t i v i t i e s o f the p r e v i o u s s e m e s t e r , and has m i n i m a l / m o d e r a t e o p p o r t u n i t i e s to p r a c t i s e the s k i l l s a n d a c t i v i t i e s l e a r n e d i n the present semester;  2 . c o n d i t i o n of the p a t i e n t — t h e  p a t i e n t ' s c o n d i t i o n is c h a n g i n g a n d r e q u i r e s m o d e r a t e m o d i f i c a t i o n o f the student's p l a n of c a r e ;  3 . p a t i e n t e x p e r i e n c e s — a v a i l a b l e p a t i e n t e x p e r i e n c e s meet the  l e a r n i n g needs of students in a g i v e n semester; s u i t a b l e s t u d e n t / p a t i e n t r a t i o , w i t h patients who provide desired experiences; and  4 . r e v i s i o n s of student's  assignment based on the i n s t r u c t o r ' s assessment of the student's a b i l i t i e s / p e r f o r m a n c e — t h e assignment requires:  i n t e g r a t i o n of p r e v i o u s l y a c q u i r e d and some  76 n e w l y l e a r n e d k n o w l e d g e ; c o o r d i n a t i o n of nursing a c t i v i t i e s ; f l e x i b i l i t y i n the student's o r g a n i z a t i o n a n d p l a n o f c a r e ; p r o b l e m - s o l v i n g a c t i v i t i e s  (assessment,  p l a n n i n g , i m p l e m e n t a t i o n , a n d e v a l u a t i o n ) ; a n d the student c a n c o p e w i t h assignments for his semester l e v e l (for further e x p l a n a t i o n see A p p e n d i x G ) . This v a r i a b l e had three l e v e l s o f c o m p l e x i t y w h i c h m a y h a v e e n c o u r a g e d the c l i n i c a l instructors t o p e r c e i v e a n d choose l e v e l t w o c o m p l e x i t y as the d e s i r e d a n d i d e a l c o m p l e x i t y o f the nursing s i t u a t i o n .  It is a l s o p o s s i b l e that i n r e a l i t y  c o m p l e x i t y o f the nursing s i t u a t i o n does not v a r y .  L i m i t a t i o n s o f the S t u d y  1.  C o n c l u s i o n s c a n o n l y be g e n e r a l i z e d t o the p a r t i c u l a r i n s t i t u t i o n t h a t  was s a m p l e d .  H o w e v e r , there a r e n o k n o w n reasons for b e l i e v i n g t h a t c o l l e g e  nursing instructors a r e not t y p i c a l o f a m u c h l a r g e r p o p u l a t i o n o f nursing instructors i n o t h e r i n s t i t u t i o n s o f f e r i n g a d i p l o m a i n n u r s i n g . 2.  The f a c t t h a t t h e c o m p l e x i t y s c a l e had o n l y t h r e e l e v e l s o f c o m p l e x i t y ,  a p s y c h o l o g i c a l p h e n o m e n o n o f r a t i n g towards the c e n t e r may h a v e o c c u r r e d y i e l d i n g c o m p l e x i t y scores w h i c h w e r e e s s e n t i a l l y the s a m e .  A l s o , this s c a l e  was not p i l o t t e s t e d . 3.  The d e p e n d e n t v a r i a b l e — t h e c l i n i c a l p e r f o r m a n c e — w a s o n l y assessed  for the first h a l f of the semester, a n d t h e r e f o r e may not r e f l e c t the s t u d e n t s ' p e r f o r m a n c e for the w h o l e s e m e s t e r .  77 Recommendations  This study was exploratory in nature and served to identify the degree to which academic grades in Nursing, Psychology, and Biology courses in first semester of the nursing program correlate with the c l i n i c a l performance scores of nursing students.  The recommendations based on the results of the study will  be presented in two parts, recommendations for future research and recommendations for nurse educators. The recommendations for future research are as follows: 1.  The complexity of the nursing situation be studied further to establish  whether the tool used to assess complexity was indeed insensitive in differentiating levels of complexity, or whether the nursing situation itself lacked complexity. Furthermore, the tool to measure the complexity of the nursing situation be refined using four levels rather than three, so that differences in the complexity of the nursing situation are more finely differentiated. 2.  A study be conducted with a second group of Semester IV and V students,  substituting the theory grades obtained in their first semester Nursing,  Psychology,  and Biology courses with their third and fourth semester Nursing, Psychology, and Biology courses respectively to control the possible influence of the temporal factor of additional learning experiences. 3.  A study be conducted to determine whether the multiple correlation  coefficients obtained utilizing the c l i n i c a l scores over a three-week period during the first half of the semester are comparable to the correlation coefficients obtained utilizing the c l i n i c a l grades at the end of the semester.  The r e c o m m e n d a t i o n s for nurse e d u c a t o r s are as f o l l o w s : 1.  Results of this study be used w i t h c a u t i o n w h e n p r o v i d i n g g u i d a n c e to  nursing students e n r o l l e d i n t w o - y e a r nursing programs. 2.  Results of this study be used w i t h c a u t i o n w h e n p r e d i c t i n g c l i n i c a l  p e r f o r m a n c e scores o f students e n r o l l e d in t w o - y e a r nursing programs from t h e i r first semester grades in N u r s i n g , P s y c h o l o g y , a n d B i o l o g y .  REFERENCES  79  80 A h m a n n , S. J . , & G l o c k , M . D. Boston;  E v a l u a t i n g P u p i l G r o w t h (2nd e d . ) .  A l l y n and Bacon, I n c . , 1971.  Blood, D. F.,  & Budd, W . C .  N e w York:  Educational Measurement and E v a l u a t i o n .  H a r p e r and Row, P u b l i s h e r s , I n c . ,  1972.  C h a m p n e y , B . , C h a t f i e l d , K . M . , & S i m s , A . M e t h o d s of Assessment. A G u i d e for T e a c h e r s of N u r s e s , e d i t e d by E l i z a b e t h R a y b o u l d . Oxford: Blackwell Scientific Publication, 1975, 155-173. C r o n b a c h , L. J . , et a l . The D e p e n d a b i l i t y of B e h a v i o r a l M e a s u r e m e n t s : T h e o r y of G e n e r a l i z a b i l i t y for Scores and P r o f i l e s . N e w Y o r k : John W i l e y and Sons, I n c . , 1972. Fivars, G . , & G o s n e l l , D. Process.  N e w York:  Nursing Evaluation:  The M a c m i l l a n C o . ,  The P r o b l e m a n d the  1966.  Fuerst, E. V . , & W o l f f , L. T e a c h i n g F u n d a m e n t a l s o f N u r s i n g (4th e d . ) . P h i l a d e l p h i a : J . B. L i p p i n c o t t C o . , 1 9 6 9 . G o n n e l l a , J . S . , G o r a n , M . J . , W i l l i a m s o n , J . W . , & Cotsonas, N . , Evaluation of Patient Care:  An Approach.  Jr.  Journal of A m e r i c a n  M e d i c a l Association, 1970, 214, 2040-2043" G o r d o n , P.  Evaluation:  A T o o l in N u r s i n g S e r v i c e .  The A m e r i c a n J o u r n a l  of N u r s i n g , 1 9 6 0 , 3 , 3 6 4 - 3 6 7 . G o r e , B. W .  The M u l t i v a r i a t e P r e d i c t i o n of C o l l e g e G r a d e P o i n t A v e r a g e  a n d State Board E x a m i n a t i o n Scores in S e l e c t e d M i s s i s s i p p i A s s o c i a t e ree N u r s i n g Programs ( D o c t o r a l D i s s e r t a t i o n ) .  Mississippi:  U n i v e r s i t y of Southern M i s s i s s i p p i , 1 9 7 3 . G o z a , J . T.  A n I n v e s t i g a t i o n of the A c a d e m i c P o t e n t i a l , A c a d e m i c  A c h i e v e m e n t , and P e r s o n a l i t y of P a r t i c i p a n t s in a n A s s o c i a t e D e g r e e N u r s i n g Program ( D o c t o r a l D i s s e r t a t i o n ) . University, G r e e n , P. E . ,  N e w Jersey: G r o n l u n d , N . E.  Hart, W . M .  East Texas State  1970. & T u l l , D. S.  N e w York:  Texas:  Research for M a r k e t i n g D e c i s i o n s (2nd e d . ) .  Prentice-Hall, Inc.,  1970.  M e a s u r e m e n t and E v a l u a t i o n i n T e a c h i n g (3rd e d . ) . M a c m i l l a n Publishing C o . , I n c . ,  1976.  The E v a l u a t i o n of the P e r f o r m a n c e of B a c c a l a u r e a t e Students  in C l i n i c a l N u r s i n g (Doctoral Dissertation). M a y 1974.  Indiana U n i v e r s i t y ,  Hayter, J .  A n A p p r o a c h to L a b o r a t o r y E v a l u a t i o n .  The J o u r n a l of N u r s i n g  E d u c a t i o n , 1 9 7 3 , 112, 1 7 - 2 2 . H e s l i n , P.  Evaluating C l i n i c a l Performance.  Nursing O u t l o o k , 163, 11,  344-345.  —  I n f a n t e , M . S . The C l i n i c a l L a b o r a t o r y i n N u r s i n g E d u c a t i o n . J o h n W i l e y and Sons, I n c . , 1 9 7 5 . K e r l i n g e r , F. N . , & P e d h a z u r , E. J . Research.  N e w York:  M u l t i p l e Regression i n B e h a v i o r a l  H o l t , Rinehart a n d W i n s t o n , I n c . , 1 9 7 3 .  K o r m a n , M . , & S t u b b l e f i e l d , L. R. ship P e r f o r m a n c e .  N e w York:  M e d i c a l S c h o o l E v a l u a t i o n and I n t e r n -  J o u r n a l of M e d i c a l E d u c a t i o n , 1 9 7 1 , 4 6 , 6 7 0 - 6 7 3 .  K r u g e r , M . E v a l u a t i o n of a S t u d e n t ' s C l i n i c a l P e r f o r m a n c e , a V i d e o t a p e . Vancouver! Vancouver Community C o l l e g e , Langara, 1978. L a n c a s t e r , A . N u r s e Teachers: the Report o f a n O p i n i o n S u r v e y . Belfast: W . & G . Baird L t d . , 1 9 7 2 . L e a p e r , D . J . , G i l l , P. W . , S t a n i l a n d , J . R., H o r r o c k s , J . C , & D e D o m b a l , F. T . C l i n i c a l D i a g n o s t i c Progress: A n A n a l y s i s . British M e d i c a l J o u r n a l , 1 9 7 3 , 3 , 5 6 9 - 5 7 4 . Litwack, L.,  S a k a t a , R.,  & Wykle, M.  C o u n s e l l i n g , E v a l u a t i o n and  S t u d e n t D e v e l o p m e n t in N u r s i n g E d u c a t i o n . W.  Philadelphia:  B. S a u n d e r s , 1 9 7 2 .  M a c K a y , R. C .  Evaluation of Faculty and Students. . . .  A Means  Towards F u l l e r C o m m u n i c a t i o n a n d G r e a t e r P r o d u c t i v i t y .  The  Journal of Nursing Education, 1974, 13, 3 - 7 . M c M a n u s , R. L. N e w York:  The E f f e c t of E x p e r i e n c e on N u r s i n g A c h i e v e m e n t . Teachers C o l l e g e , C o l u m b i a U n i v e r s i t y Bureau of  Publications,  1949.  N a t i o n a l L e a g u e for N u r s i n g . N e w York: P o w e r s , B.  Evaluation - An Objective Approach.  The L e a g u e , 1 9 7 2 .  P r e d i c t i o n of Success o f C o m m u n i t y C o l l e g e N u r s i n g Students  (Doctoral Dissertation).  A r i z o n a State U n i v e r s i t y , December 1974.  P u b l i c a t i o n M a n u a l of the A m e r i c a n P s y c h o l o g i c a l A s s o c i a t i o n . G a r a m o n d / P r i d e m a r k Press, I n c . ,  1975.  Maryland;  82 R i n e s , A . R.  E v a l u a t i n g Student Progress in L e a r n i n g the P r a c t i c e of  Nursing.  N e w York:  Columbia University,  Bureau o f P u b l i c a t i o n s , Teachers C o l l e g e , 1963.  R y d e n , M . B. The P r e d i c t i v e V a l u e o f a C l i n i c a l E x a m i n a t i o n of I n t e r p e r s o n a l R e l a t i o n s h i p S k i l l s . J o u r n a l of N u r s i n g E d u c a t i o n , 1 9 7 7 , 16,5,27-31. Schweer, J . E., & G e b b i e , K. M . (3rd e d . ) .  S a i n t Louis:  Sister M i l l e r , P.  C r e a t i v e Teaching in C l i n i c a l  The C . V . M o s b y C o m p a n y ,  A Teaching Practicum.  Nursing O u t l o o k , 1976, 2 4 , 5 ,  752-753. T a t e , B. L.  Nursing  1976.  _  E v a l u a t i n g the N u r s e ' s C l i n i c a l P e r f o r m a n c e .  Nursing  O u t l o o k , 1962, 10, 3 5 - 3 7 . Taylor, C . W . ,  N a h m , H . , Q u i n n , M . , Harms, M . , M u l a i k , J . , &  M u l a i k , S. A . Students:  S e l e c t i o n a n d R e c r u i t m e n t of N u r s e s and N u r s i n g  A R e v i e w of Research Studies and P r a c t i c e s .  Utah:  U n i v e r s i t y of U t a h Press, 1 9 6 3 . Taylor, C . W . , N a h m , H . , Q u i n n , M . , Harms, M . , M u l a i k , J . , & M u l a i k , S . A . Report o f M e a s u r e m e n t a n d P r e d i c t i o n o f N u r s i n g P e r f o r m a n c e , Part I. S a l t L a k e C i t y : U n i v e r s i t y of U t a h Press, J965~. T h o r n d i k e , R. L . ,  & H a g e n , E.  a n d E d u c a t i o n (2nd e d . ) . W i e n e r , S. L.,  K o r a n , L.,  & H o t c h k i s s , E.  Measurement and Evaluation in Psychology N e w York:  M i t c h e l l , P.,  C l i n i c a l Skills:  J o h n W i l e y and S o n s , I n c . , 1 9 6 1 . Schattner, G . , Fierstein, J . ,  Quantitative Measurement.  New  Y o r k State J o u r n a l o f M e d i c i n e , 1 9 7 6 , 7 6 , 6 1 0 - 6 1 2 . Wood, V . A.  A Problem That W o n ' t G o A w a y .  Review, 1972, 19, 336-343.  International Nursing  APPENDICES  83  APPENDIX  84  A  APPENDIX B  86  L A N G A R A  Vancouver Conrnunity College, Langara.  M E M O to  M r s . M . Kruger, Nursing Department  from  A . L . Dartnell, Acting Principal  date  January 9 ,  subject  1978  Research  I h a v e v e r y c a r e f u l l y r e a d y o u r proposal a n d a l s o h a v e discussed the m a t t e r w i t h M r s . K n o r . This memo w i l l g i v e y o u p e r m i s s i o n to p r o c e e d as o u t l i n e d in y o u r p r o p o s a l . If there is a n y w a y in w h i c h I c a n h e l p w o u l d y o u p l e a s e let me k n o w .  ALD/SB Enc: c . c . M r s . E.  Knor  87  -  1  APPENDIX C  88  89  EXPLANATIONS TO  INSTRUCTORS  Dear Colleague; I discussed my proposal w i t h y o u at f a c u l t y m e e t i n g s i n both A p r i l a n d O c t o b e r , 1977 a n d w o u l d n o w l i k e to request y o u r p a r t i c i p a t i o n . As y o u w i l l r e m e m b e r , the study is to d e t e r m i n e w h a t p r o p o r t i o n o f the v a r i a n c e i n the c l i n i c a l p e r f o r m a n c e s c o r e of the nursing student is c o n t r i b u t e d by the grades in the first semester n u r s i n g , b i o l o g y and p s y c h o l o g y c o u r s e s , a n d the c o m p l e x i t y o f the nursing s i t u a t i o n . The i n f o r m a t i o n g a t h e r e d w i l l h e l p nurse e d u c a t o r s be c o g n i z a n t o f the e f f e c t o f s u c h factors on the c l i n i c a l p e r f o r m a n c e g r a d e of the nursing students a n d a c c o r d i n g l y e v a l u a t e the students' p e r f o r m a n c e . The a n e c d o t a l records w i l l be t y p e d a n d c o d e d to ensure c o n f i d e n t i a l i t y . I w o u l d l i k e t o m a k e it v e r y c l e a r that this is not a n e v a l u a t i o n of y o u r a b i l i t y t o w r i t e a n e c d o t a l n o t e s , m a k e c l i n i c a l a s s i g n m e n t s , or g r a d e students' performance. A c o p y of the c o m p l e t e d study w i l l be p r o v i d e d to the nursing d e p a r t m e n t a n d made a v a i l a b l e to a l l p a r t i c i p a n t s . Your participation w i l l entail: (1)  w r i t i n g a n e c d o t a l notes about the p e r f o r m a n c e of 4 - 8 p a r t i c i p a t i n g students on the d e s i g n a t e d forms d u r i n g the w e e k s of J a n u a r y 2 3 , 30 a n d F e b r u a r y 6 a n d 1 3 , 1 9 7 8 .  (2)  g i v i n g a s c o r e to e a c h a n e c d o t a l r e c o r d a c c o r d i n g t o the c l i n i c a l o b j e c t i v e s o f the student and the s c o r i n g system p r o v i d e d to y o u b y t h e i n v e s t i g a t o r .  (3)  d e t e r m i n i n g t h e l e v e l of c o m p l e x i t y of the nursing s i t u a t i o n a c c o r d i n g t o the c r i t e r i a (see A p p e n d i x  (4)  v o l u n t e e r i n g a b o u t 30 t o 4 0 minutes for e a c h p a r t i c i p a t i n g student to c o m p l e t e a c t i v i t i e s  (5)  G).  1-3.  a c t i n g as a nursing j u d g e , to e v a l u a t e the p e r f o r m a n c e of 1 0 - 1 4 students on the basis o f a n e c d o t a l records w r i t t e n by o t h e r instructors from y o u r semester.  APPENDIX D  91  92  CONSENT  FORM  Dear Colleague: Please i n d i c a t e y o u r w i l l i n g n e s s to p a r t i c i p a t e i n this study by s i g n i n g y o u r name in the a p p r o p r i a t e s p a c e p r o v i d e d b e l o w , a n d as soon as p o s s i b l e return the c o n s e n t form to M a r y K r u g e r , o f f i c e number A l 6 7 c . Y o u may k e e p the a t t a c h e d i n f o r m a t i o n i f y o u w i s h .  I, , a g r e e (OR I, d o not wish) to p a r t i c i p a t e in the study b y M a r y K r u g e r t i t l e d : W h a t p r o p o r t i o n of the v a r i a n c e i n the c l i n i c a l p e r f o r m a n c e s c o r e of the nursing student is c o n t r i b u t e d b y the grades in the first semester n u r s i n g , b i o l o g y , and p s y c h o l o g y courses and by the c o m p l e x i t y of the nursing s i t u a t i o n . I understand t h a t the study w i l l i n v o l v e w r i t i n g a n e c d o t e s about the p e r f o r m a n c e o f p a r t i c i p a t i n g students i n my c l i n i c a l a r e a for t h e w e e k s of J a n u a r y 2 3 , 3 0 , and F e b r u a r y 6 a n d 1 3 , 1978; d e t e r m i n i n g the c o m p l e x i t y o f the nursing s i t u a t i o n a c c o r d i n g t o d e s c r i b e d c r i t e r i a ; and s c o r i n g the a n e c d o t a l notes o f p a r t i c i p a t i n g students from the semester e m p l o y i n g t h e course o b j e c t i v e s a n d the s c o r i n g s y s t e m . I h a v e b e e n assured t h a t the f i n d i n g s w i l l not i n f l u e n c e my p o s i t i o n i n t h e s c h o o l a n d that my name w i l l not a p p e a r in a n y w r i t t e n r e p o r t . I am a w a r e that I am free t o w i t h d r a w from the study at a n y t i m e .  Signed  APPENDIX E  93  APPENDIX F  95  96  CONSENT  FORM  D e a r Student: Please i n d i c a t e y o u r w i l l i n g n e s s to p a r t i c i p a t e i n this study by s i g n i n g y o u r name in the a p p r o p r i a t e s p a c e b e l o w , a n d return the c o n s e n t form to M a r y K r u g e r , o f f i c e number A 1 6 7 c as soon as p o s s i b l e .  I, , a g r e e ( O R I, d o not wish) to p a r t i c i p a t e i n the study by M a r y K r u g e r t i t l e d : W h a t p r o p o r t i o n o f the v a r i a n c e i n the c l i n i c a l p e r f o r m a n c e score of the nursing student is c o n t r i b u t e d by the grades i n t h e first semester n u r s i n g , b i o l o g y , a n d p s y c h o l o g y c o u r s e s , and b y the c o m p l e x i t y o f the nursing s i t u a t i o n . I understand that the study w i l l i n v o l v e o b t a i n i n g my first semester n u r s i n g , b i o l o g y , and p s y c h o l o g y grades from the registrar's o f f i c e and s c o r i n g of the a n e c d o t a l records a b o u t my p e r f o r m a n c e , a n o n y m o u s l y , for the w e e k s of J a n u a r y 2 3 to F e b r u a r y 2 0 , 1978. I h a v e been assured that t h e f i n d i n g s w i l l not i n f l u e n c e my s t a n d i n g i n the s c h o o l a n d that my name w i l l not a p p e a r i n a n y w r i t t e n r e p o r t . I am a w a r e that I am free to w i t h d r a w from the study at a n y t i m e .  Signed  APPENDIX  97'  G  C O M P L E X I T Y O F THE N U R S I N G  level 1  SITUATION  level 2  level 3  O n a c o m p l e x i t y s c a l e r a n g i n g from l e v e l 1 to l e v e l 3 s k e t c h e d a b o v e , c i r c l e o n the right the l e v e l that best represents the student's assignment a c c o r d i n g to the c r i t e r i a d e s c r i b e d .  C i r c l e only O N E level  LEVEL 1 1.  a c t i v i t i e s of the student  level 1  D e s c r i p t i o n o f l e v e l s 1, 2 and 3 LEVEL 2 a c t i v i t i e s of the student  level 2  level 3  LEVEL 3 a c t i v i t i e s of the student  Treatments p r e s c r i b e d b y the D r . a n d /  Treatments p r e s c r i b e d by the D r . a n d /  Treatments p r e s c r i b e d by the D r . a n d /  or the nursing team a r e m i n i m a l i n  or the nursing team are moderate in  or the nursing team are numerous; the  number; the student has had numerous  number; the student has had numerous  student has had m i n i m a l / m o d e r a t e  p r e v i o u s o p p o r t u n i t i e s to p r a c t i s e these  o p p o r t u n i t i e s to p r a c t i s e the s k i l l s a n d  o p p o r t u n i t i e s to p r a c t i s e the s k i l l s  s k i l l s and a c t i v i t i e s , a n d has m i n i m a l  a c t i v i t i e s o f the p r e v i o u s semester,  and a c t i v i t i e s learned i n the p r e v i o u s  or no o p p o r t u n i t y to p r a c t i s e the  a n d has m i n i m a l / m o d e r a t e o p p o r t u n i -  semester, and has m i n i m a l / m o d e r a t e  a c t i v i t i e s a n d s k i l l s l e a r n e d in the  ties t o p r a c t i s e the s k i l l s and a c t i v i t i e s o p p o r t u n i t i e s to p r a c t i s e the s k i l l s l e a r n e d in the present semester. and a c t i v i t i e s l e a r n e d i n the present  present s e m e s t e r .  semester.  2.  LEVEL 1  LEVEL 2  LEVEL 3  c o n d i t i o n of the p a t i e n t  c o n d i t i o n of the p a t i e n t  c o n d i t i o n of the p a t i e n t  The p a t i e n t ' s c o n d i t i o n is s t a b l e a n d  The p a t i e n t ' s c o n d i t i o n is c h a n g i n g  The p a t i e n t ' s c o n d i t i o n is c h a n g i n g  requires m i n i m a l or no m o d i f i c a t i o n  a n d requires moderate m o d i f i c a t i o n  r a p i d l y and requires maximum m o d i -  of the student's p l a n o f c a r e .  o f the student's p l a n of c a r e .  f i c a t i o n of the student's p l a n o f care.  3.  patient experiences  patient experiences  (a) not e n o u g h p a t i e n t s on the w a r d  a v a i l a b l e p a t i e n t e x p e r i e n c e s meet  the student's i n i t i a l assignment is  t h e l e a r n i n g needs o f students in a  a c c o r d i n g t o the l e a r n i n g needs o f  g i v e n semester; s u i t a b l e s t u d e n t /  the students in a g i v e n semester;  p r o v i d i n g the d e s i r e d e x p e r i e n c e s . (b) t o o m a n y students on the w a r d ;  patient experiences  therefore, limited suitable patient  patient ratio, with patients who  h o w e v e r , the p a t i e n t ' s ( s ' ) c o n d i t i o n  e x p e r i e n c e s for the students o f  provide desired e x p e r i e n c e s .  c h a n g e s and the c l i n i c a l i n s t r u c t o r  the g i v e n semester. (c) i n i t i a l a p p r o p r i a t e p a t i e n t a s s i g n -  d e c i d e s that the student s h o u l d c o n t i n u e to c a r e for the patient(s)  ment; h o w e v e r , the c l i n i c a l  in c o l l a b o r a t i o n w i t h o t h e r member(s)  instructor c h a n g e s the assignment  o f the nursing t e a m .  because (1) p a t i e n t transferred to a n o t h e r ward (2) p a t i e n t ' s c o n d i t i o n has worsened (3) p a t i e n t is d i s c h a r g e d (4) p a t i e n t has e x p i r e d .  LEVEL 1  LEVEL 2  LEVEL 3  r e v i s i o n s of student's assignment based  r e v i s i o n s of student's assignment based  r e v i s i o n s of student's assignment based  on t h e i n s t r u c t o r ' s assessment o f the  on the instructor's assessment of the  on the instructor's assessment o f the  student's a b i l i t i e s / p e r f o r m a n c e  student's abilities/performance  student's a b i l i t i e s / p e r f o r m a n c e  The assignment requires:  The assignment requires:  The assignment requires:  (a) i n t e g r a t i o n of p r e v i o u s l y a c q u i r e d  (a) i n t e g r a t i o n of p r e v i o u s l y a c q u i r e d  (a) i n t e g r a t i o n of p r e v i o u s l y and  a n d some n e w l y l e a r n e d k n o w -  newly learned knowledge;  knowledge; (b) c o o r d i n a t i o n of n u r s i n g a c t i v i t i e s ; (c) p r o b l e m - s o l v i n g a c t i v i t i e s : assessment, p l a n n i n g , i m p l e m e n t a t i o n , and e v a l u a t i o n .  ledge; (b) c o o r d i n a t i o n of nursing a c t i v i t i e s ; (c) f l e x i b i l i t y in the student's o r g a n i z a t i o n and p l a n o f c a r e ; (d) p r o b l e m - s o l v i n g a c t i v i t i e s :  Student c a n not c o p e w i t h the t y p e of assignment for his semester l e v e l . Therefore, (a) the c l i n i c a l i n s t r u c t o r c h a n g e s the assignment (b) the c l i n i c a l i n s t r u c t o r chooses assignments t h a t she p e r c e i v e s that the student is a b l e to m a n a g e .  (b) c o o r d i n a t i o n of nursing a c t i v i t i e s ; (c) maximum f l e x i b i l i t y in the student's o r g a n i z a t i o n a n d p l a n of c a r e ; (d) constant p r o b l e m - s o l v i n g  assessment, p l a n n i n g , i m p l e m e n -  activities:  t a t i o n , and e v a l u a t i o n .  i m p l e m e n t a t i o n , and e v a l u a t i o n .  S t u d e n t c a n c o p e w i t h assignments for his semester l e v e l .  assessment, p l a n n i n g ,  Student c a n not c o p e w i t h the t y p e of assignments for his semester l e v e l . Therefore c l i n i c a l instructor s p e c i f i c a l l y chooses the assignment t o i d e n t i f y the student's strengths and weaknesses. Student c a n c o p e w i t h the t y p e o f assignments for his semester l e v e l a n d c l i n i c a l instructor s p e c i f i c a l l y chooses the assignment to m o t i v a t e the s t u d e n t .  101 To d e t e r m i n e the c o m p l e x i t y l e v e l of the student's a s s i g n m e n t , y o u must f o l l o w the s e q u e n t i a l p r o c e d u r e b e l o w . (1)  Read the v a r i a b l e s d e s c r i b e d under l e v e l s 1, 2 , a n d 3 .  (2)  D e t e r m i n e how many o f the v a r i a b l e s i n e a c h l e v e l a p p l y t o the student's a s s i g n m e n t .  (3)  D e c i d e on the l e v e l that best represents the student's a s s i g n m e n t . B e l o w are t h r e e p o s s i b l e s i t u a t i o n s a n d the a p p r o p r i a t e d e c i s i o n in e a c h . (a) i f a l l the v a r i a b l e s from one l e v e l a p p l y to the student's a s s i g n m e n t , t h e n that is the a p p r o p r i a t e l e v e l . (b)  (c)  (4)  i f t w o o f the v a r i a b l e s from one l e v e l a n d o n e v a r i a b l e from e a c h o f the o t h e r l e v e l s a p p l y t o the student's a s s i g n m e n t , then the d e c i s i o n depends on the p a t t e r n . There are t h r e e s i t u a t i o n s under this c a t e g o r y : (1)  if t w o v a r i a b l e s from l e v e l 1, a n d o n e v a r i a b l e from e a c h o f t h e l e v e l s 2 a n d 3 a p p l y , t h e n the d e c i s i o n is towards the c e n t r a l t e n d e n c y , thus l e v e l 2 .  (2)  i f one v a r i a b l e from e a c h of the l e v e l s 1 a n d 3 , a n d t w o v a r i a b l e s from l e v e l 2 a p p l y , t h e n the d e c i s i o n is l e v e l 2 .  (3)  if o n e v a r i a b l e from e a c h o f the l e v e l s 1 a n d 2 , and t w o v a r i a b l e s from l e v e l 3 a p p l y to the student's a s s i g n m e n t , t h e n the d e c i s i o n is level 3.  i f t w o of the v a r i a b l e s from o n e l e v e l a n d t w o v a r i a b l e s from a n o t h e r l e v e l a p p l y to the student's a s s i g n m e n t , t h e n the d e c i s i o n is the h i g h e r l e v e l . For e x a m p l e , j f there are t w o v a r i a b l e s from l e v e l 2 , a n d t w o v a r i a b l e s from l e v e l 3 , t h e n the d e c i s i o n is l e v e l 3 .  C i r c l e the l e v e l d e c i d e d i n step 3 .  APPENDIX H  102  ANECDOTAL  RECORD  Semester  Instructor N u m b e r  Date  Student N u m b e r  D e s c r i p t i o n of the S i t u a t i o n  D e s c r i p t i o n of the S t u d e n t ' s B e h a v i o r  Performance Score  APPENDIX I  104  SCORING  SYSTEM  O n a s c a l e o f 1 to 9 , 1 r e p r e s e n t i n g u n s a t i s f a c t o r y to 9 representing superior p e r f o r m a n c e , c i r c l e on the r i g h t the number that best d e s c r i b e s the student's a c h i e v e m e n t of the 6 major o b j e c t i v e s . F 1 Unsatisfactory  P 2  C 3  B 4  Marginal  5  A  6  7  Average  8  9  Above  Superior  Average S E M E S T E R II Objectives  Circle  ONE  2  3  4  5  6  7  8  9  1.  U t i l i z e the p r o b l e m s o l v i n g a p p r o a c h to p r o v i d e systematic c a r e .  2  3  4  5  6  7  8  9  2.  C o m m u n i c a t e w i t h others i n an e f f e c t i v e and systematic m a n n e r .  2  3  4  5  6  7  8  9  3.  V a l u e i n d i v i d u a l s as u n i q u e and h a v i n g w o r t h .  2  3  4  5  6  7  8  9  4.  Demonstrate o r g a n i z a t i o n a l s k i l l s .  2  3  4  5  6  7  8  9  5.  Assume his r o l e as a member of the h e a l t h t e a m .  2  3  4  5  6  7  8  9  6.  Demonstrate r e s p o n s i b i l i t y and a c c o u n t a b i l i t y for his own b e h a v i o r .  2  3  4  5  6  7  8  9  Note:  The a b o v e are the t e r m i n a l o b j e c t i v e s under w h i c h y o u h a v e s p e c i f i c o b j e c t i v e s for your c l i n i c a l c o u r s e . U s i n g the c r i t e r i a u n d e r the s c o r i n g system d e s c r i b e d in A p p e n d i x I r a n g i n g from " u n s a t i s f a c t o r y " to " s u p e r i o r " a l o n g w i t h the c o r r e s p o n d i n g l e t t e r grades used at the c o l l e g e w i l l e n a b l e you to s e l e c t the a p p r o p r i a t e f i g u r e r a n g i n g from 1 to 9 .  SCORING  SYSTEM  O n a s c a l e o f 1 t o 9 , 1 r e p r e s e n t i n g u n s a t i s f a c t o r y to 9 representing s u p e r i o r p e r f o r m a n c e , c i r c l e on the r i g h t the number t h a t best d e s c r i b e s the student's a c h i e v e m e n t of the 6 major o b j e c t i v e s . F  P  C  B  A  1 Unsatisfactory  8 Marginal  Average  Above  Superior  Average S E M E S T E R III Objectives  Circle  ONE  2  3  4  5  6  7  8  9  1.  U t i l i z e the nursing process to p r o v i d e s y s t e m a t i c nursing c a r e .  2  3  4  5  6  7  8  9  2.  C o m m u n i c a t e w i t h others in a n e f f e c t i v e and s y s t e m a t i c m a n n e r .  2  3  4  5  6  7  8  9  3.  V a l u e i n d i v i d u a l s as u n i q u e a n d h a v i n g w o r t h .  2  3  4  5  6  7  8  9  4.  Demonstrate o r g a n i z a t i o n a l s k i l l s .  2  3  4  5  6  7  8  9  5.  Demonstrate a c c o u n t a b i l i t y for the c a r e o f his assigned p a t i e n t s .  2  3  4  5  6  7  8  9  6.  Demonstrate r e s p o n s i b i l i t y for his b e h a v i o r .  2  3  4  5  6  7  8  9  Note:  The a b o v e a r e the t e r m i n a l o b j e c t i v e s under w h i c h y o u h a v e s p e c i f i c o b j e c t i v e s for your c l i n i c a l c o u r s e , U s i n g the c r i t e r i a u n d e r the s c o r i n g system d e s c r i b e d in A p p e n d i x I r a n g i n g from " u n s a t i s f a c t o r y " to " s u p e r i o r " a l o n g w i t h the c o r r e s p o n d i n g l e t t e r grades used at the c o l l e g e w i l l e n a b l e you to s e l e c t the a p p r o p r i a t e f i g u r e r a n g i n g from 1 to 9 .  SCORING  SYSTEM  O n a s c a l e o f 1 to 9 , 1 r e p r e s e n t i n g u n s a t i s f a c t o r y to 9 r e p r e s e n t i n g superior p e r f o r m a n c e , c i r c l e on the right the number that best d e s c r i b e s the s t u d e n t ' s a c h i e v e m e n t of the 6 m a j o r o b j e c t i v e s . F  P  C  B  A  1 Unsatisfactory  8 Marginal  Average  Above  Superior  Average SEMESTER IV Objectives  Circle  ONE  2  3  4  5  6  7  8  9  1.  U t i l i z e the p r o b l e m s o l v i n g a p p r o a c h to p r o v i d e s y s t e m a t i c nursing care.  2  3  4  5  6  7  8  9  2.  C o m m u n i c a t e w i t h others i n a n e f f e c t i v e and s y s t e m a t i c m a n n e r .  2  3  4  5  6  7  8  9  3.  V a l u e i n d i v i d u a l s as u n i q u e a n d h a v i n g w o r t h .  2  3  4  5  6  7  8  9  4.  Demonstrate o r g a n i z a t i o n a l s k i l l s .  2  3  4  5  6  7  8  9  5.  Assume his r o l e as a member o f the h e a l t h t e a m .  2  3  4  5  6  7  8  9  6.  D e m o n s t r a t e r e s p o n s i b i l i t y through his b e h a v i o r .  2  3  4  5  6  7  8  9  Note:  The a b o v e are the t e r m i n a l o b j e c t i v e s under w h i c h y o u h a v e s p e c i f i c o b j e c t i v e s for y o u r c l i n i c a l c o u r s e . U s i n g the c r i t e r i a u n d e r the s c o r i n g system d e s c r i b e d in A p p e n d i x I r a n g i n g from " u n s a t i s f a c t o r y " to " s u p e r i o r " a l o n g w i t h the c o r r e s p o n d i n g letter grades used a t the c o l l e g e w i l l e n a b l e y o u to s e l e c t the a p p r o p r i a t e f i g u r e r a n g i n g from 1 to 9 .  SCORING  SYSTEM  O n a s c a l e o f 1 t o 9 , 1 r e p r e s e n t i n g u n s a t i s f a c t o r y to 9 r e p r e s e n t i n g superior p e r f o r m a n c e , c i r c l e on the r i g h t the number t h a t best d e s c r i b e s the student's a c h i e v e m e n t of the 6 m a j o r o b j e c t i v e s . F  P  C  B  A  1 Unsatisfactory  8 Marginal  Average  Above  Superior  Average SEMESTER V Objectives  Circle  ONE  2  3  4  5  6  7  8  9  1.  U t i l i z e the p r o b l e m s o l v i n g a p p r o a c h to p r o v i d e c a r e .  2  3  4  5  6  7  8  9  2.  C o m m u n i c a t e w i t h others in a n e f f e c t i v e and s y s t e m a t i c m a n n e r .  2  3  4  5  6  7  8  9  3.  V a l u e i n d i v i d u a l s as u n i q u e a n d h a v i n g w o r t h .  2  3  4  5  6  7  8  9  4.  Demonstrate o r g a n i z a t i o n a l s k i l l s .  2  3  4  5  6  7  8  9  5.  Assume his r o l e as a member o f the h e a l t h t e a m .  2  3  4  5  6  7  8  9  6.  Demonstrate r e s p o n s i b i l i t y for his o w n b e h a v i o r .  3  4  5  6  7  8  9  Note:  The a b o v e are the t e r m i n a l o b j e c t i v e s under w h i c h y o u h a v e s p e c i f i c o b j e c t i v e s for your c l i n i c a l c o u r s e , U s i n g t h e c r i t e r i a u n d e r the s c o r i n g system d e s c r i b e d in A p p e n d i x I r a n g i n g from " u n s a t i s f a c t o r y " to " s u p e r i o r " a l o n g w i t h the c o r r e s p o n d i n g l e t t e r grades used at the c o l l e g e w i l l e n a b l e y o u t o s e l e c t the a p p r o p r i a t e f i g u r e r a n g i n g from 1 to 9 .  Categories  Letter Grade  Numerical  Criteria  Grade  The student f a i l s to meet the course o b j e c t i v e s ; (1)  of p r i n c i p l e s , c o n c e p t s , and procedures; (2)  Definitions:  requires c o n s i s t e n t d e t a i l e d e x p l a n a t i o n requires consistent t e a c h e r or staff  Usually - customarily, habitually  supervision; Unsatisfactory  1  2  (3)  f a i l s to a p p l y the p r o b l e m - s o l v i n g process;  (4)  is u n a b l e t o demonstrate i n i t i a t i v e i n  F r e q u e n t l y - h a p p e n i n g at short intervals, often occurring  meeting objectives; (5)  is i n c o n s i s t e n t in a p p l y i n g p r e v i o u s l y learned knowledge;  (6)  O c c a s i o n a l l y - happening i r r e g u l a r l y , c o m i n g now a n d t h e n  is f r e q u e n t l y i n c o n s i s t e n t and unsafe in his p e r f o r m a n c e o f c o m m u n i c a t i o n a n d  R a r e l y - s e l d o m , not o f t e n  psychomotor s k i l l s . Appropriate - s p e c i a l l y suitable, proper, fit The student is i n c o n s i s t e n t in m e e t i n g the course o b j e c t i v e s : (1) (2) Marginal  C o n s i s t e n t - m a r k e d by r e g u l a r i t y ,  requires frequent d e t a i l e d e x p l a n a t i o n  or steady c o n t i n u i t y t h r o u g h o u t ;  of p r i n c i p l e s , c o n c e p t s , and procedures;  s h o w i n g no s i g n i f i c a n t c h a n g e ,  requires frequent t e a c h e r or staff  u n e v e n n e s s , or c o n t r a d i c t i o n  supervision; (3) (4)  o c c a s i o n a l l y a p p l i e s the p r o b l e m - s o l v i n g  Initiative - a b i l i t y to originate  process;  a c t i o n s , to i n i t i a t e d e s i r a b l e  o c c a s i o n a l l y demonstrates a p p r o p r i a t e  steps  i n i t i a t i v e in meeting objectives; (5)  o c c a s i o n a l l y applies previously learned knowledge;  (6)  is u s u a l l y i n c o n s i s t e n t and unsafe in his p e r f o r m a n c e of c o m m u n i c a t i o n and psychomotor s k i l l s .  M a r g i n a l - c l o s e to the l o w e r l i m i t of q u a l i f i c a t i o n or acceptability  o  NO  Categories  Average  Letter  Numerical  Grade  Grade  4  5  Criteria The student meets a l l of the o v e r a l l course objectives:  Definitions:  (1)  requires o c c a s i o n a l e x p l a n a t i o n o f p r i n c i p l e s , c o n c e p t s , and procedures;  Usually - customarily, habitually  (2)  requires o c c a s i o n a l t e a c h e r or staff supervision;  (3)  u s u a l l y a p p l i e s the p r o b l e m - s o l v i n g process;  F r e q u e n t l y - h a p p e n i n g at short intervals, often occurring  (4)  u s u a l l y demonstrates a p p r o p r i a t e i n i t i a t i v e in m e e t i n g o b j e c t i v e s ;  O c c a s i o n a l l y - happening  (5)  usually applies previously learned knowledge;  (6)  is u s u a l l y consistent and safe in his p e r f o r m a n c e of c o m m u n i c a t i o n and p s y c h o m o t o r s k i I Is.  i r r e g u l a r l y , c o m i n g now and t h e n Rarely - s e l d o m , not o f t e n Appropriate - specially suitable, p r o p e r , fit  The student meets a l l of the o v e r a l l and most of  C o n s i s t e n t - marked by r e g u l a r i t y ,  the s p e c i f i c o b j e c t i v e s :  or steady c o n t i n u i t y throughout;  (1)  requires o c c a s i o n a l e x p l a n a t i o n o f  showing no s i g n i f i c a n t c h a n g e ,  p r i n c i p l e s , c o n c e p t s , and procedures;  u n e v e n n e s s , or c o n t r a d i c t i o n  (2) requires a p p r o p r i a t e t e a c h e r or staff supervision; Above  6  7  (3) f r e q u e n t l y a p p l i e s the p r o b l e m - s o l v i n g process;  Average  I n i t i a t i v e - a b i l i t y to o r i g i n a t e a c t i o n s , to i n i t i a t e d e s i r a b l e steps  (4)  f r e q u e n t l y demonstrates a p p r o p r i a t e  (5)  f r e q u e n t l y a p p l i e s p r e v i o u s l y learned  l i m i t of q u a l i f i c a t i o n or  knowledge;  acceptability  i n i t i a t i v e in meeting objectives;  (6)  is c o n s i s t e n t a n d safe in his p e r f o r m a n c e of c o m m u n i c a t i o n and psychomotor s k i l l s .  M a r g i n a l - c l o s e to the l o w e r ~  Categories  Letter Grade  Numerical Grade  Criteria The student meets a l l o f the o v e r a l l and  Definitions:  specific objectives:  Superior  A  8  9  (1)  r a r e l y requires e x p l a n a t i o n of p r i n c i p l e s , c o n c e p t s , and procedures;  Usually - customarily,  (2)  requires a p p r o p r i a t e t e a c h e r or staff supervision;  F r e q u e n t l y - h a p p e n i n g at short  (3) c o n s i s t e n t l y a p p l i e s the p r o b l e m - s o l v i n g process; (4)  c o n s i s t e n t l y demonstrates a p p r o p r i a t e i n i t i a t i v e in m e e t i n g o b j e c t i v e s ;  (5)  consistently applies previously learned  habitually  intervals, often occurring O c c a s i o n a l l y - happening i r r e g u l a r l y , c o m i n g n o w a n d then R a r e l y - s e l d o m , not o f t e n  knowledge; (6)  is consistent a n d safe in his p e r f o r m a n c e of c o m m u n i c a t i o n a n d psychomotor s k i l l s .  Appropriate - specially suitable, proper, fit C o n s i s t e n t - m a r k e d by r e g u l a r i t y , or steady c o n t i n u i t y throughout; s h o w i n g no s i g n i f i c a n t c h a n g e , u n e v e n n e s s , or c o n t r a d i c t i o n Initiative - a b i l i t y to originate a c t i o n s , to i n i t i a t e d e s i r a b l e steps M a r g i n a l - c l o s e to the l o w e r l i m i t o f q u a l i f i c a t i o n or acceptability  APPENDIX J  112  113  GENERALIZABILITY  ANALYSIS  The t h e o r y of g e n e r a l i z a b i l i t y proposed by L. J . C r o n b a c h et a l . (1972) employs the a n a l y s i s o f v a r i a n c e to d e t e r m i n e c o e f f i c i e n t s of r e l i a b i l i t y . M o r e s p e c i f i c a l l y , this t h e o r y is m o t i v a t e d by the f o l l o w i n g questions: (1) h o w c a n a person's score best be e s t i m a t e d ? and (2) how l a r g e is the error that arises from i n c o m p l e t e o b s e r v a t i o n s ? The term " g e n e r a l i z a b i l i t y " refers to the g e n e r a l i z a t i o n from observed (sample) s c o r e s , to u n i v e r s e scores (persons' m e a n scores o v e r a l l a c c e p t a b l e o b s e r v a t i o n ) . C r o n b a c h et a l . state that the u n i v e r s a l score "emphasizes that the i n v e s t i g a t o r is m a k i n g an i n f e r e n c e from a s a m p l e o f o b s e r v e d d a t a , and a l s o that there is more t h a n o n e u n i v e r s e to w h i c h he m i g h t g e n e r a l i z e " ( p . 1 8 ) . In a g e n e r a l i z a b i l i t y ( G ) s t u d y , the r e s e a r c h e r c o l l e c t s d a t a from w h i c h estimates c a n be made o f the components o f v a r i a n c e for measurements made for a c e r t a i n p r o c e d u r e . F u r t h e r m o r e , the r e s e a r c h e r o b t a i n s t w o or more scores for the i n d i v i d u a l by o b s e r v i n g h i m u n d e r d i f f e r e n t c o n d i t i o n s , a n d t h e n e x a m i n e s the c o n s i s t e n c y o f these s c o r e s . It is necessary here to d e f i n e some of the e s s e n t i a l t e r m i n o l o g y of the t h e o r y of g e n e r a l i z a b i l i t y , to f a c i l i t a t e the d e s c r i p t i o n of its a p p l i c a t i o n in the present s t u d y . A n y o b s e r v a t i o n s m a d e are d e s c r i b e d i n terms of c o n d i t i o n s , d e f i n e d as "the task or stimulus p r e s e n t e d , the d a y a n d h o u r , the s e t t i n g i n w h i c h the o b s e r v a t i o n is m a d e , the o b s e r v e r , and p o s s i b l e a d d i t i o n a l features of the o p e r a t i o n s p e r f o r m e d " ( C r o n b a c h et a l . , p . 1 7 ) .  The term f a c e t , h o w e v e r ,  is used to refer to c o n d i t i o n s of a c e r t a i n k i n d . The f a c e t s , a l o n e or in c o m b i n a t i o n , d e f i n e the u n i v e r s e i n q u e s t i o n . T h e r e f o r e , a u n i v e r s e o f o b s e r v a t i o n s c o u l d be c h a r a c t e r i z e d w i t h r e s p e c t to o n e , t w o , or more f a c e t s . The t h e o r y o f g e n e r a l i z a b i l i t y makes the f o l l o w i n g assumptions: (1)  c o n d i t i o n s of o b s e r v a t i o n s a r e not n e c e s s a r i l y p a r a l l e l ;  (2)  c o n d i t i o n s ( p a r t i c u l a r l y test items) are r a n d o m l y s u p p l i e d from a  u n i v e r s e , or a c c o r d i n g to a s t r a t i f i e d d e s i g n ; a n d (3)  t w o or more f a c e t s may be a n a l y z e d s i m u l t a n e o u s l y .  114 G e n e r a l i z a b i l i t y a n a l y s i s p r o v i d e s e s t i m a t e c o m p o n e n t s o f those f a c e t s r e p r e s e n t e d i n t h e e x p e r i m e n t a l d e s i g n . These estimates o f v a r i a n c e components are t h e n used t o d e t e r m i n e t h e c o e f f i c i e n t o f g e n e r a l i z a b i l i t y . F i n a l l y , the c o e f f i c i e n t of g e n e r a l i z a b i l i t y ( E p ) is d e t e r m i n e d by the " r a t i o o f the u n i v e r s e s c o r e v a r i a n c e to t h e e x p e c t e d o b s e r v e d - s c o r e v a r i a n c e — t h e v a r i a n c e l i k e l y t o be o b t a i n e d u n d e r a c e r t a i n e x p e r i m e n t a l p l a n " ( G r o n b a c h e t a l . , p. 1 7 ) . 2  In the n o t a t i o n E p , E stands for e x p e c t e d v a l u e , a n d p 2  c o r r e l a t i o n of observed s c o r e w i t h u n i v e r s e s c o r e .  2  is the squared  The v a l u e of the G  coefficient  ranges from z e r o t o o n e , a n d it expresses h o w w e l l the o b s e r v a t i o n s are l i k e l y t o l o c a t e the i n d i v i d u a l r e l a t i v e to o t h e r members o f the p o p u l a t i o n . In this s t u d y , the B M D 0 8 V c o m p u t e r program was used t o a n a l y z e t h e d a t a . This program g e n e r a t e d estimates of v a r i a n c e c o m p o n e n t s ( v a r i a n c e s o f h y p o t h e s i z e d components o f a n o b s e r v e d s c o r e ) , w h i c h w e r e u t i l i z e d i n calculating generalizability coefficients. This study was m a i n l y c o n c e r n e d w i t h t h e a c c u r a c y o f g e n e r a l i z i n g over: (1)  ratings o f t h r e e nursing j u d g e s , a n d  (2)  ratings g a t h e r e d d u r i n g t h r e e d i f f e r e n t w e e k s .  The study e m p l o y e d a nested d e s i g n w i t h t h r e e f a c e t s . (1)  S t u d e n t s , nested w i t h i n instructors ( S ; l ) ,  (2) (3)  J u d g e s ( J ) , and W e e k s (W)  The d e s i g n o f the study was (S:l) x J x W . the forms o f o v e r l a p p i n g v a r i a n c e s .  These w e r e :  The d i a g r a m b e l o w shows s c h e m a t i c a l l y  \  Instructors  0)  \  Students (S  Judges uag  (J)  115 A n e x a m i n a t i o n o f the d e s i g n of the study i d e n t i f i e d 11 sources o f v a r i a n c e ( i . e . , c o m p o n e n t s ) . These w e r e : instructors (I); w e e k s (W); s t u d e n t s , nested w i t h i n instructors (S:l); judges (J); i n s t r u c t o r - b y - w e e k i n t e r a c t i o n (IxW); i n s t r u c t o r - b y - j u d g e i n t e r a c t i o n (IxJ); w e e k - b y - j u d g e i n t e r a c t i o n ( W x J ) ; s t u d e n t - b y - w e e k i n t e r a c t i o n (SxW); s t u d e n t - b y - j u d g e i n t e r a c t i o n (SxJ); instructor-by-week-by-judge interaction (IxWxJ); a n d , f i n a l l y , student-byweek-by-judge interaction (S;lxWxJ). F i n a l l y , the g e n e r a l i z a b i l i t y c o e f f i c i e n t s w e r e c a l c u l a t e d w i t h the following formula:  a (p) 2  A?) where,  cr p z  +  ^ ' i ) -(M*) 0  2  + (Vn'i) °  2  (Pi/I*)  +  °  = v a r i a n c e c o m p o n e n t for persons. judges or weeks)  2 &  (In the s t u d y , e i t h e r  number of c o n d i t i o n s for f a c e t i (items) a (Pi/I*) 2  "a S 2  = v a r i a n c e c o m p o n e n t for the p e r s o n s - b y - i t e m s i n t e r a c t i o n s for a f i x e d set of i t e m s . (In this s t u d y , students nested w i t h i n instructor) the e r r o r , w h i c h i n c l u d e s p i , e components of v a r i a n c e  The numerator a n d the d e n o m i n a t o r represent the u n i v e r s e - s c o r e v a r i a n c e a n d the e x p e c t e d o b s e r v e d - s c o r e v a r i a n c e , r e s p e c t i v e l y .  APPENDIX K  116  117  STANDARDIZATION  PROCEDURE  The f o l l o w i n g steps e x p l a i n t h e p r o c e d u r e used t o s t a n d a r d i z e the r a w c l i n i c a l scores g i v e n b y the c l i n i c a l i n s t r u c t o r a n d t w o o t h e r nursing j u d g e s . For steps 1 t o 8 r e f e r t o F i g u r e 1. (1)  The means o f e a c h judge was c o m p u t e d .  (2)  The v a r i a n c e of e a c h judge was c o m p u t e d .  (3)  The s t a n d a r d d e v i a t i o n o f e a c h judge was c o m p u t e d .  (4)  N e x t , from the s c o r e o f e a c h student i n C o l u m n was s u b t r a c t e d the mean of , a n d s u b s e q u e n t l y , t h a t number was d i v i d e d by the s t a n d a r d d e v i a t i o n (SD) o f J ] . For e x a m p l e , the s t a n d a r d i z e d s c o r e o f student 3 0 3 g i v e n by J u d g e 1 was c a l c u l a t e d b y this formula. 3.5 - 5.75 SD(Jj)  (5)  U s i n g t h e a b o v e m e t h o d , the o t h e r t w o scores g i v e n to these students w e r e a d j u s t e d for Judges 2 a n d 3 . For e x a m p l e ,  4.0 - 5.5  and  5.0 - 5.38  w e r e the s t a n d a r d i z e d  SD(J ) SD(J ) scores o f student 3 0 3 for J u d g e s 2 a n d 3 . 2  (6)  3  The p o o l e d mean s t a n d a r d i z e d score o f e a c h student was t h e n c a l c u l a t e d by a d d i n g t o g e t h e r the t h r e e s t a n d a r d i z e d scores g i v e n t o her by Judges 1, 2 , and 3 , a n d d i v i d i n g this sum by t h r e e . For e x a m p l e , the s t a n d a r d i z e d score o f student 3 0 3 across the three judges was c a l c u l a t e d b y this f o r m u l a : ,3.5 - 5.75, (  SD(J])  '  ,4.0 - 5.5 +  (  SD(J ) 2  }  ,5.0 - 5.38  N +  (  SD(J )  N  }  3  3 (7)  The a b o v e c o m p u t a t i o n s w e r e r e p e a t e d to s t a n d a r d i z e the c l i n i c a l scores o f W e e k 3 of both Semesters II a n d III s t u d e n t s .  S C O R E S G I V E N B Y T H E J U D G E S F O R THE S T U D E N T S I N S E M E S T E R III  J U D G E S (J) S C O R E STUDENT  NO.  Jl  J2  J3  303  3.5  4.0  5.0  304  4.5  4.5  5.0  305  6.5  6.0  6.0  306  7.0  6.0  5.0  307  7.0  6.0  6.0  308  8.0  6.0  6.0  309  6.0  6.0  5.5  310  4.0  5.0  4.0  311  5.0  5.0  5.0  312  6.0  6.0  7.0  313  6.5  6.0  6.0  314  5.0  5.5  4.0  5.75  5.5  5.38  X.  Figure 1  APPENDIX L  119  120  STEPWISE M U L T I P L E R E G R E S S I O N  ANALYSIS  The B M D : 0 2 R c o m p u t e r program was e m p l o y e d t o d o t h e s t e p w i s e m u l t i p l e regression a n a l y s i s . This program p r i n t e d out the f o l l o w i n g s t a t i s t i c s : the mean a n d standard d e v i a t i o n of e a c h v a r i a b l e ; t h e c o v a r i a n c e and c o r r e l a t i o n m a t r i c e s ; the a n a l y s i s o f v a r i a n c e t a b l e as e a c h i n d e p e n d e n t v a r i a b l e e n t e r e d the regression e q u a t i o n ; the m u l t i p l e c o r r e l a t i o n c o e f f i c i e n t (R) a n d its standard error o f e s t i m a t e for e a c h step; the i n t e r c e p t c o n s t a n t , the p a r t i a l regression w e i g h t s (beta w e i g h t s ) , t h e i r standard error and l e v e l o f s i g n i f i c a n c e ; a n d f i n a l l y , the c o e f f i c i e n t s o f d e t e r m i n a t i o n ( R ) , the s e m i p a r t i a l c o r r e l a t i o n s and t h e i r l e v e l of s i g n i f i c a n c e . 2  The c o e f f i c i e n t o f m u l t i p l e c o r r e l a t i o n (R) is d e f i n e d by K e r l i n g e r a n d P e d h a z u r (1973) as the p r o d u c t moment c o r r e l a t i o n o f the p r e d i c t e d v a r i a b l e ( Y ) — w h i c h is a l i n e a r c o m b i n a t i o n of the i n d e p e n d e n t v a r i a b l e s ( X ' s ) — a n d the o b s e r v e d Y . The v a l u e s o f R range from z e r o to o n e , and w h e n s q u a r e d ( R ) , a n d m u l t i p l i e d b y 100 y i e l d t h e p e r c e n t o f v a r i a n c e o f the d e p e n d e n t v a r i a b l e (Y) w h i c h is e x p l a i n e d by the l i n e a r c o m b i n a t i o n o f the independent v a r i a b l e s . 1  2  The omnibus F s t a t i s t i c is c a l c u l a t e d as e a c h i n d e p e n d e n t v a r i a b l e enters the regression e q u a t i o n . This s t a t i s t i c is c a l c u l a t e d by u t i l i z i n g the summary d a t a from the A N O V A t a b l e . The F s t a t i s t i c was c a l c u l a t e d w i t h the f o l l o w i n g formula: ^regression/dfj SSresidual/df2 where  df]  = degrees o f f r e e d o m a s s o c i a t e d w i t h sums of squares o f regression w h i c h is e q u a l t o K , t h e n u m b e r of i n d e p e n d e n t v a r i a b l e s i n the regression e q u a t i o n .  df  2  = N - K - 1, w h e r e N is e q u a l t o the s a m p l e s i z e  This F s t a t i s t i c was used to test t h e s i g n i f i c a n c e o f the m u l t i p l e c o r r e l a t i o n c o e f f i c i e n t (R), as e a c h n e w v a r i a b l e e n t e r e d the regression e q u a t i o n . a d d i t i o n , at e a c h step the s t a n d a r d error o f e s t i m a t e ( S E  e s t  ) o f R was  c a l c u l a t e d , w h i c h is the standard d e v i a t i o n o f the r e s i d u a l s i n Y at a p a r t i c u l a r step i n the a n a l y s i s .  In  121 The f o r m u l a for the S E _  s t  was as f o l l o w s :  /^residual / N - K - 1  This s t a t i s t i c is a n i n d e x of t h e d i s p e r s i o n of the p r e d i c t e d Y measure ( Y ) a b o u t the regression l i n e , and is used t o e v a l u a t e the e s t i m a t e o f t h e d e p e n d e n t v a r i a b l e (Y) based o n the i n d e p e n d e n t v a r i a b l e s . For e x a m p l e , i f the S E _ o f R is l a r g e r w h e n c o m p a r e d to t h e standard d e v i a t i o n o f the d e p e n d e n t v a r i a b l e ( Y ) , then the e s t i m a t e o f Y on the basis of t h e i n d e p e n d e n t v a r i a b l e s is p o o r . The b e t a w e i g h t s (b's) a r e the s a m p l e p a r t i a l regression c o e f f i c i e n t s w h i c h i n d i c a t e the r e l a t i v e i m p o r t a n c e o f the i n d e p e n d e n t v a r i a b l e s (X's) i n m a k i n g the p r e d i c t i o n s t o the d e p e n d e n t v a r i a b l e ( Y ) . The s t a n d a r d error o f regression w e i g h t s is the s t a t i s t i c w h i c h i n d i c a t e s the v a r i a b i l i t y of the errors a n d p r o v i d e s a measure w i t h w h i c h t o c o m p a r e the s t a t i s t i c s whose s i g n i f i c a n c e a r e b e i n g t e s t e d . The f o r m u l a used to c a l c u l a t e the l a t t e r is: 1  s t  '(SEest) S  where  E  b  l  . SE^  S S  2  Xi(l-R 1.23) 2  = S t a n d a r d error of b e t a w e i g h t of i n d e p e n d e n t v a r i a b l e ^ number 1  SS^  Sums of squares of i n d e p e n d e n t v a r i a b l e number 1  _  o R^l . 2 3  = S q u a r e m u l t i p l e c o r r e l a t i o n c o e f f i c i e n t b e t w e e n the i n d e p e n d e n t v a r i a b l e number 1 used as a d e p e n d e n t v a r i a b l e , a n d the r e m a i n i n g i n d e p e n d e n t v a r i a b l e s 2 and 3  The p a r t i a l c o r r e l a t i o n is used w h e n s t a t i s t i c a l c o n t r o l o f v a r i a n c e is d e s i r e d .  A t e a c h step o f t h e m u l t i p l e regression a n a l y s i s , the p a r t i a l r's  are c a l c u l a t e d w h i c h s h o w t h e c o r r e l a t i o n b e t w e e n t w o v a r i a b l e s w i t h the third variable held constant. F i n a l l y , the s e m i p a r t i a l c o r r e l a t i o n is c a l c u l a t e d .  U n l i k e the  p a r t i a l r's, the s e m i p a r t i a l r's represent the c o r r e l a t i o n b e t w e e n t w o v a r i a b l e s w i t h the i n f l u e n c e o f a n o t h e r v a r i a b l e ( s ) , r e m o v e d from o n l y one o f the v a r i a b l e s being correlated.  M o r e s p e c i f i c a l l y , the square o f this s t a t i s t i c t e l l s the u n i q u e  c o n t r i b u t i o n t o the v a r i a n c e o f the d e p e n d e n t v a r i a b l e t h a t e a c h i n d e p e n d e n t v a r i a b l e adds a f t e r the v a r i a n c e c o n t r i b u t i o n of the p r e c e d i n g v a r i a b l e ( s ) is a c c o u n t e d for; h o w e v e r , this a p p l i e s o n l y to the p a r t i c u l a r o r d e r in w h i c h t h e i n d e p e n d e n t v a r i a b l e s e n t e r the regression e q u a t i o n .  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0100219/manifest

Comment

Related Items