UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

The development of an evaluation Q-sort : a study of nursing instructors Neylan, Margaret Sarah 1966

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1966_A8 N45.pdf [ 3.77MB ]
Metadata
JSON: 831-1.0104652.json
JSON-LD: 831-1.0104652-ld.json
RDF/XML (Pretty): 831-1.0104652-rdf.xml
RDF/JSON: 831-1.0104652-rdf.json
Turtle: 831-1.0104652-turtle.txt
N-Triples: 831-1.0104652-rdf-ntriples.txt
Original Record: 831-1.0104652-source.json
Full Text
831-1.0104652-fulltext.txt
Citation
831-1.0104652.ris

Full Text

THE DEVELOPMENT OP AN EVALUATION Q-SORT: A STUDY OP NURSING INSTRUCTORS by MARGARET S. NEYLAN B.N., McGill University, 1957 A THESIS SUBMITTED IN PARTIAL FULFILMENT OP THE REQUIREMENTS FOR THE DEGREE OP MASTER OP ARTS (ADULT EDUCATION) i n the Faculty of Education We accept t h i s thesis as conforming to the required standard THE UNIVERSITY OP BRITISH COLUMBIA A p r i l , 1966 In present ing th i s thes i s in p a r t i a l f u l f i lment of the requirements for an advanced degree at the Un ivers i ty of B r i t i s h Columbia, I agree that the L ib ra ry sha l l make i t f r e e l y ava i l ab le for reference and study. I fur ther agree that permission for extens ive copying of th i s thes i s for s cho la r l y purposes may be granted by the Head of my Department or by h i s representat ives. It is understood that copying or pub l i ca t i on of th i s thes i s for f i n a n c i a l gain sha l l not be allowed without my wr i t ten permission Department The Un iver s i t y of B r i t i s h Columbia Vancouver 8, Canada i i ABSTRACT The purpose of this study was to develop an Evaluation Q-Sort and to test i t by measuring the perceptions held by-nursing, i n s t r u c t o r s of the r e l a t i v e importance of f i v e func-tions and effects of evaluation. The functions and effects i d e n t i f i e d f o r study were: the measurement of student achieve-ment, the measurement of student progress, psychological e f f e c t s of evaluation, the influence of evaluation on teach-ing, and the influence of evaluation on administration. An Evaluation Q-Sort was developed and used to measure the per-ceptions of evaluation held by the 111 nursing, instructors i n the s i x professional nursing schools i n the Lower Mainland and Vancouver Island areas of the Province of B r i t i s h Columbia. The population was divided into ten c l a s s i f i c a t i o n s according to various c r i t e r i a r elated to r o l e , experience, preparation, and i n s t r u c t i o n a l s e t t i n g . The central hypothesis assumed that the group of instructors as a whole would not assign greater importance to anyone of the f i v e functions and effects of evaluation. The nine sub-hypotheses assumed that the percep-tions of evaluation held by nursing instructors would not be influenced by the variables selected f o r study. The .05 l e v e l of significance was used i n the study. The re s u l t s indicated that the nursing instructors did ascribe s i g n i f i c a n t l y d i f f e r e n t degrees of importance to the f i v e functions and effects of evaluation. Measurement of stu-dent achievement was ascribed l e a s t importance and measurement l i i of student progress was ascribed most importance among the functions and effects studied. In addition, differences were found with respect to the nature of the instructors* r e s p o n s i b i l i t i e s , the type of school i n whicn she taught, and her stated l e v e l of s a t i s f a c t i o n with preparation as an evaluator. No differences were found with respect to length of experience i n nursing service or education, pre-paration as an in s t r u c t o r , course i n tests and measurements, i n s t r u c t i o n a l focus, and i n s t r u c t i o n a l s e t t i n g . i v . TABLE OP CONTENTS ABSTRACT TABLE OP CONTENTS LIST OP TABLES ACKNOWLEDGEMENTS CHAPTER I. PRESENTATION OP THE PROBLEM Statement of the Problem Significance of the Study D e f i n i t i o n of Terms CHAPTER I I . REVIEW OP THE LITERATURE Evaluation i n Nursing Q-Technique CHAPTER I I I . RESEARCH PROCEDURES Development of Evaluation Q-Sort Item Selection Category Selection Test for Truth and Importance Test f o r V a l i d i t y of Category Pina l Selection of Items R e l i a b i l i t y Population Studied Characteristics Studied C o l l e c t i o n of Data Processing of Data CHAPTER IV. PRESENTATION OP FINDINGS Analysis o f Q-Sort Scores of A l l Instructors Analysis of Q-Sort Scores of Various C l a s s i f i c a t i o n s of Instructors Analysis of Q-Sort Item Scores Tabulation of Sources of Consultation CHAPTER V. CONCLUSIONS AND IMPLICATIONS BIBLIOGRAPHY APPENDIX I APPENDIX 2 LIST OP TABLES Number of Items Accepted by Category R e l i a b i l i t y Data on Evaluation Q-Sort C l a s s i f i c a t i o n of Instructors Means, Standard Deviations and Ranks' of Category and Sub-category Scores of the Total Group Category and Sub-category Mean Scores and P-values of Q-sorts: Group One: C l a s s i f i e d According to Years of Ex-perience i n Nursing Service Category and Sub-category Mean Scores and P-values of Q-sorts: Group Two: C l a s s i f i e d According to Year s of Ex-perience i n Nursing Education 'Category and Sub-category Mean Scores and P-values of Q-Sorts: Group Three: C l a s s i f i e d According to Type of Pre-paration Category and Sub-category Mean Scores and P-values of Q-sorts: Group Pour: C l a s s i f i e d According to Extent of Teaching R e s p o n s i b i l i t i e s Category and Sub-category Mean Scores and P-values of Q-sorts: Group F i v e : C l a s s i f i e d According to Extent of Administrative R e s p o n s i b i l i t i e s Category and Sub-category Mean Scores and P-values of Q-sorts: Group Six: C l a s s i f i e d According to Instruc t i o n a l Setting Category and Sub-category Mean Scores and P-values of Q-sorts: Group Seven: C l a s s i f i e d According to Teaching Focus Category and Sub-category Mean Scores and F-values of Q-sorts: Group Eight: C l a s s i f i e d According to Type of School i n Which They Teach Table Thirteen Category and Sub-category Mean Scores and P-values of Q-sorts: Group Nine: C l a s s i f i e d According to a Course i n Tests and Measurements Table One Table Two Table Three Table Pour Table Five Table Six Table Seven Table Eight Table Nine Table Ten Table Eleven Table Twelve L i s t of Tables (cont'd) Table Fourteen Table F i f t e e n Table Sixteen Table Seventeen Category and Sub-category Mean Scores and F-values of Q-sorts: Group Ten: C l a s s i f i e d According to Stated Degree of S a t i s f a c t i o n with Preparation as an evaluator The Ten Most Important Items i n Rank Order The Ten Least Important Items i n Rank Order Nursing Instructors Sources of Consultation for Evaluative Problems i n Rank Order ACKNOWLEDGMENTS The writer would l i k e to express appreciation to the judges who assisted In the development of the Evaluation Q-Sort, to the Instructors who served as subjects In this study, to Professor K.D. Craig, Department of Psychology, who assisted i n the analysis of the data, and to Professor Coolie Verner who gave guidance and encouragement. CHAPTER I PRESENTATION OP THE PROBLEM The evaluation of students has been considered a problem i n nursing education from the beginning. In nursing, evalua-t i o n i s l a r g e l y a matter of assessing the performance of learned nursing s k i l l s and t h i s has been recognized by educa-tors as the d i f f i c u l t aspect of evaluation i n which to develop r e l i a b l e and v a l i d measuring instruments. Some of the d i f f i c u l t y encountered i n nursing with the evaluation of performance can be traced to early schools of nursing where a student was regarded more as a worker than as a learner. Since then, however, learning has become increasingly important, consequently, i n t e r e s t i n evaluation has also increased. Evaluation i s an important aspect of the educational process and may be used f o r a v a r i e t y of purposes such as measuring achievement, motivating learning, and assessing i n -s t r u c t i o n a l or administrative p r a c t i c e s . Nursing, schools have tended to focus on the t r a d i t i o n a l instruments of measurement such as r a t i n g scales, check l i s t s and anecdotal notes, among others. Nursing i n s t r u c t o r s , however, receive l i t t l e t r a i n i n g i n evaluation. Statement of the Problem Since nursing instru c t o r s receive l i t t l e t r a i n i n g i n evaluation, i t i s presumed that t h e i r perception of the functions 1 2 of evaluation w i l l be influenced by t h e i r experience both i n nursing i t s e l f and i n the i n s t r u c t i o n of nursing students. In order to assess the perception of the functions and ef-fect s of evaluation i n nursing education held by nursing i n -structors an Evaluation Q-Sort was constructed and applied to a group of nursing i n s t r u c t o r s . Hypothesis The central hypothesis of t h i s study i s : Nursing i n s t r u c t o r s do not assign greater importance to any one of the functions or effects of evaluation i d e n t i f i e d i n an Evaluation Q-sort. There were a number of sub-hypotheses developed and tested In order to examine the influence of various aspects of experience and r o l e upon the Evaluation Q-sorts of nursing i n s t r u c t o r s . These are as follows: 1. The Q-sorts of nursing i n s t r u c t o r s are not influenced by length of experience i n nursing service. 2. The Q-sorts of nursing instructors are not influenced by length of experience i n nursing education. 3. The Q-sorts of nursing instructors are not influenced by type of preparation as an i n s t r u c t o r . l±. The Q-sorts of nursing i n s t r u c t o r s are not influenced by the nature of t h e i r r e s p o n s i b i l i t i e s . 5>. The Q-sorts of nursing Instructors are not influenced by the nature of the i n s t r u c t i o n a l s e t t i n g . 3 6. The Q-sorts of nursing i n s t r u c t o r s are not influenced by t h e i r i n s t r u c t i o n a l focus, 7. The Q-sorts of nursing Instructors are not influenced by the type of school i n which they teach. 8. The Q-sorts of nursing instructors are not influenced by a course i n tests and measurements. 9. The Q-sorts of nursing, i n s t r u c t o r s are not influenced by degree of s a t i s f a c t i o n with preparation as evaluators. Significance of the Study The Royal Commission on Health Services In Canada (79) recommended extended and Improved nursing services. In order to implement this recommendation i t w i l l be necessary to i n -crease the number of nurses trained and to Improve the nature and q u a l i t y of the t r a i n i n g provided. This i s being brought about by the gradual movement away from the e a r l i e r apprentice-ship concept of nursing, t r a i n i n g to an educational approach ( i n which emphasis i s placed equally upon learning and per-formance. At the heart of nursing education and of c r u c i a l concern to the nursing profession i s the problem of the evaluation of both nursing education and nursing service. In order to improve evaluation In nursing i t i s necessary to know the way In which nursing Instructors perceive of evalua-tion, therefore, this study Is directed to an assessment of the perceptions of the effeets and functions of evaluation held by nursing i n s t r u c t o r s as measured by an Evaluation Q-Sort. Furthermore, the Evaluation Q-Sort which i s developed f o r t h i s study w i l l provide a tool which can be used with other populations elsewhere i n assessing perceptions of evaluators. D e f i n i t i o n of Terms A number of terms are used here i n a s p e c i f i c sense. These terms are defined as follows: Perception, t h i s term i s used i n the same way as i t i s defined by Morgan C69:l60) to denote "....awareness of our-selves and of objects, q u a l i t i e s and relationships i n our environment. 0 Evaluation, this term i s used to denote "....the process of ascertaining or judging, the value or amount of something.by careful a p p r a i s a l " as defined by Good (ii4:209). CHAPTER II REVIEW OF THE LITERATURE There i s a rather considerable body of material r e l a t e d to evaluation i n nursing education but very l i t t l e of i t represents any substantial research. Furthermore, there i s material r e l a t e d to the use of the Q-Sort technique i n simi l a r or related studies. Both of these categories of r e -search l i t e r a t u r e are reviewed below. Literature about Evaluation i n Nursing There has been no substantial review of research related s p e c i f i c a l l y to evaluation i n nursing education or service although a recent survey of research i n nursing by Simmons and Henderson (83: 376-389) included a b r i e f summary of research r e l a t e d to evaluation. Tschudln (92) summarized much of the current thinking i n respect to evaluation i n nursing education. For the most part the l i t e r a t u r e on evaluation has centered about instruments f o r measuring performance i n nursing s i t u a t i o n s . Among the many reports of research r e l a t e d to instruments designed to measure lear n -ing and performance i n c l i n i c a l situations are those by Abhold (1), Boozer, (7), Brester (11), F i e l d (33), Flanagan (3k), Fletcher (35), Gerchberg. (lj.0), Heter (52), Hoffman (56), Meyer (67), N o l l (7l+), and Small (8I4.) • Lucas (614.) investigated reactions of student nurses to the evaluation of t h e i r performance i n c l i n i c a l p r a c t i c e . 6 She investigated attitudes at s i x month i n t e r v a l s and detected a developmental sequence of attitudes towards evaluation* I n i t i a l l y there was an unemotional acceptance of evaluation as an i n t e g r a l part of learning.but t h i s was superseded by an i n t e r v a l during which attitudes were mixed. At t h i s time the students questioned the v a l i d i t y of evalua-t i o n and the competence of the evaluator. Six months l a t e r the students were guarded i n the expression of more d i f f u s e attitudes. At the next Interval the students were vocal i n expressing, t h e i r opinions of evaluation and a small number found evaluation h e l p f u l , some found i t p a i n f u l while the large majority were c r i t i c a l . Students l i s t e d evaluation as being worthless, meaningless or Inaccurate. At the f i n a l measurement some students were sympathetically aware of the problems of evaluation, some were c r i t i c a l but the larger group expressed in d i f f e r e n c e . Lucas did not examine the variables r e l a t e d to t h i s sequential development of attitudes, however, she did note instructors'comments as to the function of evaluation. Instructors noted that they used evaluation to motivate, to punish, to reward, to measure achievement, or to indicate progress. Any possible relationships between an in s t r u c t o r ' s perception and use of evaluation, and the development of student attitudes was not examined further by Lucas. A study by Howard and Berkowitz (58) investigated the 7 reactions of non-nursing students towards those who evaluated t h e i r performance of a task i n a laboratory setting, however, the attitude of the evaluators was not studied. Rines. (77:19) study of the b e l i e f s and practices i n respect to evaluation held by junior college nursing i n s t r u c t o r s summarized the functions of evaluation as follows: 1. Determining, the progress a student i s making towards achieving the goals of the program. 2. Helping the in d i v i d u a l student maintain strengths and eliminate weaknesses. 3« Helping,the teacher improve her teaching, ij.. Determining, the worth of the undertaking i n general. f>. C l a r i f y i n g and defining, educational ob-j e c t i v e s . 6. Developing.more r e l i a b l e instruments for evaluation. 7. Motivating,the student. 8. Providing psychological seourity f o r the students, s t a f f and community. 9. Providing, c e r t i f i c a t i o n to meet l e g a l r e-quirements. Rihes noted that i n s t r u c t o r s use evaluation f o r d i f f e r e n t functions or combination of functions so that as students progress from one learning experience to another they may be evaluated by instructors who use evaluation for d i f f e r e n t functions. A lack of common understanding between the stu-dent and the ins t r u c t o r may give r i s e to some of the negative and i n d i f f e r e n t f e e l i n g s which come to be associated with evaluation as indicated above i n the Lucas study* In the area of nursing service, as i n nursing education, evaluation has been of concern both i n measuring personnel 8 performances and In assessing patients*responses to nursing care. In a study measuring s t a f f nursing performance, G-orham (ij.6) noted that nurses d i s l i k e evaluating s t a f f nurses and that s t a f f nurses resent the comments and c r i t i c i s m s . In a s i m i l a r study Rosen (78:82) noted, "There seems to be a pervasive impression among, supervisory personnel that the evaluation and counselling procedure i s a forbidding or re-pulsive task, rather than one which i s apt to promote growth and i n s i g h t on the part of s t a f f nurses and better under-standing of t h e i r needs, strengths and d e f i c i e n c i e s by supervision". A number of writers have attempted to replace these negative attitudes towards evaluation by creating the con-cept that evaluation i s a p o s i t i v e , helpful process enhancing growth and confidence i n the professional nurse and assuring patients of competent nursing.care. Tate (89:36) summarized this trend i n this way. " I t Is only i n the past ten years that the majority of nurses have come to r e a l i z e that our methods (of evaluation) are antiquated and should be r e -placed by more r e l i a b l e ones.... Today evaluation of nursing practice i s the major delemma faced by those who are experimenting with what they consider new and better methods of nursing p r a c t i c e . " Within the various c l i n i c a l areas of practice, e f f o r t s have been made to study and improve the ways i n which per-formance i s evaluated. Butler (ll|.), Chernushin (19)» and Schultz (80) are representative of such writers within psy-c h i a t r i c nursing. In these studies consideration was given 9 to the nature of evaluation and various measurement tools were developed and tested, however, the perceptions of those p a r t i c i p a t i n g i n the evaluation process were not examined. Freeman (37) has summarized the trends i n thinking about evaluation i n public health p r a c t i c e . Hansen (i|.9)> Shyen (82), and Glaser (ij.3) reported upon various extensive research studies attempting to develop s p e c i f i c , r e l i a b l e instruments to measure performance i n public health nursing. Within the content of general hospitals research has focussed upon the s t a f f nurse and has dealt either with the philosophy of evaluation or i t has discussed a s p e c i f i c evaluative instrument. Included i n this group of writers are Malaspina (65)# Church (20), G i l c h r i s t (ij.2), Woodworth (101), and Cochran (21). A study by Medaris (66) conoerned the evaluation of the nursing supervisor. Tate (90) studied the performance of the s t a f f nurse and Ellsworth (32) studied the nursing aide. Research studies i n this area of practice were summarized by Tate (73) i n a publication issued by the National League f o r Nursing. This review of l i t e r a t u r e on evaluation i n nursing education and service found a considerable volume of mater-i a l on the nature, philosophy, and goals of evaluation i n addition to a number of studies devoted to tool development. Some of these represented the r e s u l t s of researoh by students while others were extensive research endeavors directed by 10 experienced researchers, however, l i t t l e attention has been directed to the study of the perception of evaluation held by i n s t r u c t o r s . L i t e r a t u r e About the Q-Sort Technique The Q-Sort technique of psychological measurement had i t s o r i g i n i n Q-methodology as developed by Stephenson (86). He used Q-methodology to derive hypotheses from theory and Q-technique to test such hypotheses. Reactions to Stephenson 1s work have varied. Butler and Pisk (15) be-l i e v e d that t h i s approach was a major contribution to assessment, Cronbach and Glaser (26) have expressed cautions optimism, while C a t t e l l (17) has been c r i t i c a l . Cronbach (214.J 378-9) c i t e d certain advantages of the Q-sort as a data gathering device. " In the Q-sort we have a variant of the forced choice prooedure which has so many psychometric advantages. For one thing, this method of i n t e r r o -gation Is much more penetrating than the common questionnaire where the person can say "Yes" to a l l the favorable symptoms and "No" to a l l the unfavor-able ones. The method i s free from these i d i o -syncracies of response which cause some persons to respond "cannot say" twice as often as others, and so make the i r scores noncomparable. The forced choice requires every person to put himself on the measuring scale i n much the same manner. Since more statements are placed i n the middle p i l e the subject i s freed from many d i f f i c u l t and rather unimportant discriminations he would have to make i f he were forced to rank every statement. And the f a c t that discrimination near the center of the scale Is d i f f i c u l t , i s reduced by the f a c t that i n product-moment correlations the erid c e l l s received greatest weight." 11 Acceptance of the $-sort as a psychological measuring device i n the health professions was hastened by i t s use by Rogers as reported by Mowrer (71) for assessing percep-tions of changes i n persons undergoing psychotherapy. Whiting (96) used the Q-sort to assess the perceptions of the functions of nurses held by doctors and nurses. He stated that one of the central problems f o r which the Q-sort i s suitable to c o l l e c t data i s "the problem of c o r r e l a t i o n or degree of s i m i l a r i t y , between d i f f e r e n t i n -dividuals or d i f f e r e n t groups' attitudes, expectations or opinions at a given time." (97*71)• Whiting stressed the importance of careful preparation of items as "careless item w r i t i n g . w i l l confront a subject with l o g i c a l l y mean-ingless choices." (97:73). Whiting modified the actual sort from a one to four step method to reduce d i f f i c u l t i e s i n ranking a large number of items. Gorhara (l)-7) and Butler (13) used Q-sorts to study a t t i -tudes i n p s y c h i a t r i c nursing. Draper (28), Bower (9) , Oldridge (76) and Dunlap (30) used this technique to mea-sure a t t i t u d i n a l change as a r e s u l t of an educational ex-perience while Tyler (9i+) and Kerlinger (61) used Q-sorts to investigate concepts i n the f i e l d of education. CHAPTER I I I RESEARCH PROCEDURES The purpose of this study i s to measure the percep-tions of the functions and effects of evaluation i n nursing education held by nursing instructors* The population studied consisted of the nursing, instr u c t o r s located i n professional schools of nursing i n the Lower Mainland and Vancouver Island areas of the Province of B r i t i s h Columbia. The data were co l l e c t e d by interviews with 105 of the 111 i n s t r u c t o r s i d e n t i f i e d using an Evaluation Q-Sort, The data were then analyzed with respect to certain character-i s t i c s of the population. Development of the Evaluation Q-Sort The Evaluation Q-Sort consists of 56 statements about the effects and the functions of evaluation. The subject was required to rank these items i n a forced choice normal d i s t r i b u t i o n pattern according to the importance which was ascribed to each statement. Item Selection Items were obtained from books and a r t i c l e s i n profes-sional journals as well as published and unpublished studies r e l a t i n g to evaluation i n both nursing and education. Each item was a simple declarative statement beginning,with "Evaluation" as the subject of the sentence followed by a 12 13 verb denoting, a function or e f f e c t of e v a l u a t i o n 1 . This method of item preparation followed the pattern established by M i i t i n g (96:21}.). The resultant l i s t of 16I|. Items re-presented a wide range of e f f e c t s and functions of evalua-t i o n and constituted the basic components of the Evaluation Q-Sort used In this study. Category Selection Prom the review of the professional l i t e r a t u r e on evaluation the functions and effects of evaluation were i d e n t i f i e d and the I6I4. items were grouped into the following categories: 1. Achievement: Evaluation i s a process that measures the performance of a l l students i n a group, at the conclusion of a learning experience, with respect to the degree of achievement of s p e c i f i e d learning objectives. 2. Progress: Evaluation i s a process that assesses the behavior of i n d i v i d u a l students i n a learning s i t u a t i o n i n order to define her learning.needs and problems as well as her pro-gress towards achieving s p e c i f i e d learning.objectives. 3. Psychological E f f e o t s t Evaluation i s a process which influences the motiva-ti o n , attitudes, f e e l i n g s and i n t e r a c t i o n of students and i n s t r u c t o r s . T See Appendix II 14 Teaching; Evaluation i s a process that influences teaohing. £• Administration: Evaluation i s a proeess that Influences the administration of a school* Test f o r Truth and Importance Three members of the Faculty of Education at the University of B r i t i s h Columbia served as a panel of judges to determine the generally accepted truth and importance of the I62j. statements which were submitted to them. The judges inspected each statement and were asked to make two separate decisions about I t : (1) to determine i f , i n his opinion, i t expressed a true function or effe o t of evalua-t i o n and (2) to determine i f I t was of importance to the student, the ins t r u c t o r , the administrator or the communi-2 ty • The judges unaminously selected 127 items as repre-senting, true functions or ef f e c t s of evaluation, while of these, 102 were designated as important by two of the three judges. Test for V a l i d i t y of Category In addition, a panel of twelve judges were used to test the v a l i d i t y of each category. These judges were selected from among,professional nurses with master's degrees and a l l but one of these were practising.nurses. ^ See Appendix I 15 Each judge was provided with a box that had s i x s l o t s i n the top. The t i t l e of each of the categories was attached to a s l o t while the s i x t h s l o t had a "no category" l a b e l attached. The 127 items selected by the f i r s t panel of judges were typed on 3" x 5 W cards and the judges were i n -structed to read each item and place i t i n one of the f i v e categories. I f , i n the i r judgment, i t did not f i t into one of the f i v e i d e n t i f i e d categories, they were instructed to place i t i n the s i x t h s l o t . 3 The c r i t e r i o n established f o r the acceptance of an item was that nine of the twelve judges must agree on the place-ment of an item i n a p a r t i c u l a r category. The d i s t r i b u t i o n of these items among the categories i s shown i n Table One. TABLE ONE NUMBER OP ITEMS ACCEPTED BY CATEGORIES Category Number of Items % 1. Achievement 10 llj. 2. Progress 8 11 3. Psychological E f f e c t s 19 26.5 k.. Teaching 19 26.5 5 . Administration 16 22 Total 72 100$ ^See Appendix I 16 In view of the smaller number of items retained i n categories 1 and 2, i t was decided to combine these to form a single category e n t i t l e d Achievement and Progress with sub-categories 1A - Achievement and IB - Progress, and to re-number cate-gories 3> k a n c * 5 accordingly. F i n a l Selection of Items The f i n a l step was the selection of items f o r the Evaluation Q-Sort from the 72 items remaining at the conclu-sion of the test f o r v a l i d i t y of category. Items were r e j e c -ted which had been designated as unimportant by two out of three judges of the f i r s t panel. The f i n a l items were s e l e c t -ed so as to reduce the r e p e t i t i o n i n content of items and at the conclusion of this process there were fourteen items i n k each of the four categories of the Evaluation Q-Sort. Category 1 (Achievement and Progress Items 1-llj. Sub-Category 1A - (Achievement) Items 1-7 Sub-Category IB - (Progress) Items 8-lij. Category 2 (Psychological E f f e c t s ) Items 15-28 Category 3 (Teaching) Items 29-U2 Category ij. (Administration) Items Ij.3-56 R e l i a b i l i t y The f i n a l Evaluation Q-Sort consisted of £6 items divided equally into four categories. Pri o r to using the instrument for data c o l l e c t i o n i n the study, the t e s t - r e t e s t method of ^see A p p e n d II " 1? determining consistency of response was performed using the practice established by Whiting (96: l|5-6) and Butler (13: i|.l~5)» A group of homogeneous subjects which had a status similar to the population to be tested and which was r e l a t i v e l y free from educational experiences with r e -spect to evaluation was used for the test and r e - t e s t . This group consisted of f i f t e e n nursing instructors i n a non-professional nursing school. They performed the Q-Sort on two separate occasions with twelve days intervening be-tween the two s o r t s . This group resembled the actual sample i n that they taught a variety o f nursing subjects, had diverse kinds of preparation, and had a range of experience i n a v a r i e t y of.nursing practice areas. Two of the test-r e t e s t group were male whereas the population studied! were female. A small amount of exposure to education regarding evaluation was found to have occurred during the i n t e r v a l , however, i t was not considered such as to e f f e c t the outcome of the t e s t - r e t e s t procedure. The length of the i n t e r v a l between the sorts was set at twelve days to reduce the possi-b i l i t y that memory might dic t a t e the placement of items on the second test and to minimize the l i k e l i h o o d of changes i n attitudes towards evaluation. Each ins t r u c t o r completed a data sheet f o r the Evaluation Q-Sort at the f i r s t t e s t . The determination of r e l i a b i l i t y was calculated by a product-moment correlation using the scores f o r each test ^Two members of the group attended a nursing i n s t i t u t e i n which one hour was devoted to a discussion o f evaluation. 18 administration. Care was exercised to ensure that both tests were given under similar conditions so that the corr e l a t i o n c o e f f i c i e n t would more accurately indicate the amount of error attributable to the test i t s e l f . Whiting (96$i|.5) has reported r e l i a b i l i t y c o e f f i c i e n t s f o r Q-sorts ranging from .lj.0 to .80 when the t e s t - r e t e s t method was used with the c o r r e l a t i o n between scores on the f i r s t and second tests oomputed by the Pearson " r n method. Thirteen of the f i f t e e n i n s t r u c t o r s i n the group completed both t e s t s . The cor r e l a t i o n c o e f f i c i e n t s f o r a l l subjects are shown In Table Two and the mean c o e f f i c i e n t of .72 approaches the highest fig u r e reported by Whiting. TABLE TWO RELIABILITY DATA ON EVALUATION Q-SORT Instructor Correlation 1 • 66 2 .76 3 .69 k .88 5 .79 6 .69 7 .75 8 .71*. 9 .73 10 .59 11 .71 12 • 72 13 .63 Mean .72 19 Population Studied There are s i x professional schools of nursing.in the Lower Mainland and Vancouver Island areas of the Province of B r i t i s h Columbia. Five of these schools are located i n hospitals and one i s at the University of B r i t i s h Columbia* A l l f u l l time nursing instru c t o r s and Directors of Nursing with r e s p o n s i b i l i t y f o r both nursing service and education i n these schools constituted the population f o r the study and numbered 111 i n d i v i d u a l s * The entire population was female* Characteristics Studied A data sheet was used to record pertinent information regarding, c h a r a c t e r i s t i c s selected f o r analysis* This form was pretested on the instructors making up the t e s t - r e t e s t group. As a r e s u l t of this pre-test, ambiguous items were restructured to form the f i n a l data sheet. One d i f f i c u l t y that did not appear i n the pre-test was encountered l a t e r i n using the data sheet. This arose In connection with one item on the sheet which recorded the i n s t r u c t i o n a l s e t t i n g i n which the nursing Instructor operated. The form asked the respondent to designate mainly "classroom" or mainly " e l i n i c a l " , however, many instructors divided t h e i r time evenly between the two settings consequently a t h i r d a l t e r -. nate to accommodate this was added i n the tabulation of the data f o r Group Six. 20 The population studied was c l a s s i f i e d into sub-groups that were appropriate f o r each of the ch a r a c t e r i s t i c s as shown on Table Three. One c h a r a c t e r i s t i c (Group 10) was a scaled item i n which a scale score was computed from weighted responses. These ten primary groups and thei r appropriate sub-groups constituted the independent variables that were tested against the Evaluation Q-Sort to detect any s t a t i s -t i c a l l y s i g n i f i c a n t differences i n responses that may r e s u l t from the c h a r a c t e r i s t i c s . C o l l e c t i o n of Data The Evaluation Q-Sort and the data sheet were administered to the population i n the spring of 1965. Data were secured from 105 of the 111 subjects. Of the 6 subjects from whom data were not obtained, 1 had resigned, 3 were on vacation and 2 did not attend. Interest i n and cooperation with the study were high. Many subjects found the Q-Sort i n t e r e s t i n g , challenging, and provocative while a few reported the experi-ence as di s t r e s s i n g and confusing. The tests were adminis-tered by the writer and an assistant, both of whom followed the same procedure. Prior to administration of the ,<£-sort i t was emphasized that the tool was designed to test opinions, not 6 knowledge, and that the returns were anonymous. The Q-sort required each i n s t r u c t o r to rank i n r e l a t i v e importance, 56 items regarding evaluation on a nine-point normal d i s t r i b u t i o n continuum. Multiple sets of Q-sort cards See Appendix I 21 TABLE THREE CLASSIFICATION OF INSTRUCTORS Group CHARACTERISTIC Sub-Groups 1. Years of Experience i n Nursing Service 1. 2. 3. 0-2 years 3-5 years 6-10 years If.1 5 . 11-20 years over 20 years 2. Years of Experience i n Nursing Education 1. 2. 3. 0-2 years 3-5 years 6-10 years 4. 5 . 11-20 years over 20 years 3. Program Taken as Preparation to Instruct 1. 2. 3. none diploma basic bacca-laureate k. 5. post-basic baccalaureate master's 4- I n s t r u c t i o n a l R e s p o n s i b i l i t i e s 1. 2. f u l l time half time 3. less than half time 5. Administrative R e s p o n s i b i l i t i e s 1. 2. f u l l time half time 3. less than half time 6. Inst r u c t i o n a l Setting 1. 2. classroom c l i n i c a l 3-half classroom and half c l i n i c a l 7. Instruct i o n a l Focus 1. 2. Physical Sciences So c i a l Sciences 3. Administration Teaching, Supervision 8. Type of School 1. Hospital 2. University 9. Course i n Tests and Measure- 1. No 2. Yes ments 10. Degree of S a t i s f a c t i o n with 1. Low 3» High Preparation as an Instructor 2. Average 22 allowed from one to ten subjects to be tested at one time. The time required to complete the Evaluation Q-Sort and the data sheet ranged from lj.0 to 90 minutes with an average of 50 minutes. Processing of the Data Each item was assigned an item score which was determined by the pos i t i o n assigned to i t on the nine-point continuum of the forced choice d i s t r i b u t i o n made by each i n s t r u c t o r . A mean score was computed f o r each item. In addition, a category score was computed by t o t a l l i n g the values assigned each item i n that category. A mean score was established f o r each cate-gory f o r the t o t a l number of instructors doing the Q-Sort. A mean category score was computed f o r each sub-group derived by c l a s s i f y i n g the ins t r u c t o r s according to the ten c l a s s i f i -cations on the basis of the ch a r a c t e r i s t i c s I d e n t i f i e d e a r l i e r . In order to test the central hypothesis concerning the r e l a t i v e importance assigned to the functions and effects of evaluation, single factor analysis of variance and the Newman-Keuls (99: 30i+-12) technique were used to test differences among and between the mean scores of categories and sub-cate-gories, of the 105 i n s t r u c t o r s . Single factor analysis of variance was performed on the mean category and sub-category scores of each of the sub-groups of the ten cha r a c t e r i s t i c s i n order to test the sub-hypotheses concerned with various aspects of the preparation, experience and i n s t r u c t i o n a l roles of the i n s t r u c t o r s . CHAPTER IV PRESENTATION OP FINDINGS The r e s u l t s of this study are presented i n three sections. The f i r s t section contains the analysis of the Evaluation Q-Sorts of the t o t a l population to test the central hypothesis of the study. The second section contains the analysis of the Evaluation Q-Sort scores of the various c l a s s i f i c a t i o n s of instructors to test the sub-hypotheses of the study. The l a s t section presents some information concerning item analy-s i s and sources of consultation. Analysis of the Q-Sort Scores of A l l Instructors The central hypothesis of the study stated that the instructors would not assign greater importance to any one of the functions or effects of evaluation included i n the study. In order to test this hypothesis, the Q-Sort scores of the 105 instructors were examined. Mean scores, standard deviations, and ranks of category and sub-category scores of the t o t a l group are presented i n Table Pour. Scores for the sub-categories were doubled to f a c i l i t a t e comparison with categories. A single f a c t o r analysis of variance among the mean scores of the four categories yielded an P-value of 3*96 which i s s i g n i f i c a n t at the .0£ l e v e l . Since this indicated that there was a s i g n i f i c a n t difference but did not indicate 23 2k TABLE POUR MEANS, STANDARD DEVIATIONS AND RANKS OP CATEGORY AND SUB-CATEGORY Q-SORT SCORES OP THE TOTAL GROUP Category or Sub-category Mean S.D. Rank Category 1 - Achievement & Progress 68.65 6.10 3 Sub-cat 1A - Achievement 83.88 8.81 6 Sub-cat IB - Progress 53.82 8.56 1 Category 2 - Psychological E f f e c t s 71.40 8.94 5 Category 3 - Teaching 68.67 5.87 2 Category k - Administration 70.95 6.1+4 4 which i n d i v i d u a l pairs of means had a s i g n i f i c a n t difference, the Newman-Keuls procedure was used to examine the differences between a l l possible pairs of means. This test of the data indicated that there were no s i g n i f i c a n t differences between pairs of means, however, the differences between Psychological E f f e c t s and Teaching, as well as between Achievement and Progress and Psychological E f f e c t s , approached s i g n i f i c a n c e . Since Winer (99) indicates that the Newman-Keuls procedure i s more conservative than the single f a c t o r analysis of variance, the differences between the indicated pairs of means were re-examined using a single factor analysis of variance. Once again there was inconsistency i n the findings as the difference between Psychological E f f e c t s and Teaching, yielded an P-value of 4«84 which i s s i g n i f i c a n t at the .05 l e v e l whereas the difference between Achievement and Progress and 25 Psychological Effects did not y i e l d a s i g n i f i c a n t F-Value. Since s t a t i s t i c a l tests depend heavily upon p r o b a b i l i t i e s i n formulating decisions about hypotheses, i t i s not unusual for a marginal difference to y i e l d inconsistent r e s u l t s . The inconsistencies argue fo r a conservative evaluation of the data. There i s some tendency for instructors to attach more importance to the function of i n f l u e n c i n g teaching than they do to the psychological effects of evaluation. A single factor analysis of variance among categories 2, 3> k- and sub-categories 1A and IB yielded an F-value of 175«96 which i s s i g n i f i c a n t at the .01 l e v e l of confidence. The Newman-Keuls procedure again was used to examine d i f f e r -ences between each possible pairs of means of this group of scores. S i g n i f i c a n t differences were found between the two sub-categories and when each sub-category i s compared with each of the three categories. This indicates that nursing instructors consider the measurement of student progress to be more important than any of the other functions and effects of evaluation, while at the same time considering the mea-surement of student achievement to be less important. In view of t h i s , therefore, the central hypothesis of this study i s rejected. Analysis of the Q-Sort Scores of Various  C l a s s i f i c a t i o n s of the Instructors Findings r e s u l t i n g from the analysis of the Q-sorts of various c l a s s i f i c a t i o n s of the instructors w i l l be presented 26 to test each of the sub-hypotheses proposed i n this study. Sub-hypothesis 1. The Q-sorts of instructors are not influenced by length of experience i n nursing service. A single factor analysis of variance was performed on the category and sub-category scores of instructors grouped according to lengtn of experience i n nursing service. The data are presented i n Table F i v e . 'Differences among vhe Q-sorts of instructors c l a s s i f i e d according to length of ex-perience i n nursing service are not s i g n i f i c a n t at the .05 l e v e l , therefore this sub-hypothesis i s accepted. Nursing instructors do not have perceptions o f the r o l e of evaluation whicn d i f f e r according to t h e i r length of experience i n nursing, service. Sub-hypothesis 2. The Q-sorts of nursing instructors are not influenced by length of experience i n nursing educa-tio n . A single factor analysis of variance was performed on the category and sub-category scores of instructors grouped accord-ing to length of experience i n nursing education. The data are presented i n Table Six. Differences i n Q-sorts of i n s t r u c -tors c l a s s i f i e d according to years of experience i n nursing education are not s i g n i f i c a n t at the .05 l e v e l , therefore this hypothesis i s accepted. Nursing instructors do not have per-ceptions of evaluation that d i f f e r according to their length of experience i n nursing education. 27 TABLE FIVE CATEGORY AND SUB-CATEGORY MEAN SCORES AND F-VALUES OF Q-SORTS: GROUP ONE: CLASSIFIED ACCORDING TO YEARS OF EXPERIENCE IN NURSING SERVICE Years of Moono Experience No. i n % i n u e a n s  i n Nursing Group Group Ca t . l Cat.2 Cat.3 Cat.lj. S.cat.lA S.cat.IB Service 0-2 38 36 3-5 25 2k 6-10 18 17 11-20 20 19 over 20 k k Total 105 100 F-Value 67.92 72.08 68.^7 69.20 70.814. 68.72 68.56 70.144 68.33 70.25 72.35 68.65 70.00 68.25 71.75 O.53 0.29 0.30 {j.1.02 28.89 71.16 I4.2.28 26.92 72.56 k2'kk 26.11 68.60 ij-2.55 27.70 69.50 43.25 26.75 I.0I4. 0.66 0.35 28 TABLE SIX CATEGORY AND SUB-CATEGORY MEAN SCORES AND P-VALUES OF Q-SORTS: GROUP WO: CLASSIFIED ACCORDING TO YEARS OF EXPERIENCE IN NURSING EDUCATION Years of Experience No. i n % of Means  i n Nursing Group Group Cat.l Cat.2 Cat.3 Cat.t^. S.cat.lA S. cat.IB Service 0-2 25 21* 67.60 72.76 67.00 72.61* 1*1.61*. 25.96 3-5 38 36 68.39 73.15 68.13 70.26 lj.1.60 26.79 6-10 21 20 70.67 70.10; 70.90 68. llj. 1*2.86 27.81 11-20 16 15 67.81 70.56 73.00 1*1.1*1* 27.00 over 20 5 5 72.1*0 68.20 65.60 73.00 1*3.80 28.70 Total 105 100 F-Value 1.53 0.29 0.30 l.Oi). 0.66 0.35 29 Sub-hypothesis 3. The Q-sorts of nursing instructors are not influenced by the type of preparation as an in s t r u c -tor. A single factor analysis o f variance was performed on the category and sub-category scores of instructors grouped according to type of preparation. The data are presented i n Table Seven. Differences i n Q-sort scores among groups of instructors with various types of preparation are not s i g n i f i c a n t at the .05 l e v e l , therefore, this sub-hypothesis i s accepted. The perception of evaluation held by nursing instructors i s not affected by the type of preparation to i n s t r u c t . Sub-hypothesis k» The Q-sorts of instructors are not Influenced by the nature of the i r r e s p o n s i b i l i t i e s . The nature of the r e s p o n s i b i l i t i e s of the instructors tested were measured i n terms of the amount of time devoted to i n s t r u c t i o n and to administration. While a l l respondents indicated a measure of the i r i n s t r u c t i o n a l r e s p o n s i b i l i t i e s , only 65 indicated any administrative r e s p o n s i b i l i t i e s . At this point there i s some inconsistency i n the data which cannot be explained. A single f a c t o r analysis was performed on category and sub-category scores of instruct o r s grouped according to the extent of their i n s t r u c t i o n a l r e s p o n s i b i l i t i e s . These data are presented i n Table Eight. S i g n i f i c a n t differences at 30 TABLE SEVEN CATEGORY AND SUB-CATEGORY MEAN SCORES AND P-VALUES OP Q-SORTS: GROUP THREE: CLASSIFIED ACCORDING TO TYPE OP PREPARATION Type of No.in Preparation Group % of - Means Group Cat . l Cat.2 Cat.3 Cat.lj. S-cat.lA S-cat.lB None 3 3 71.67 68.33 68.67 70.00 1*1*.00 27.67 Diploma 28 27 68.32 71.71 69.61 70.21 1*1.53 26.79 Basic Bacca- 36 laureate 31* 68.61 70.80 68.75 71.80 1*2.61* 25.97 Post-Basic 26 Baccalaureate 25 69.23 72.73 67.92 70.00 1*1.69 27.51* Master's 12 11 69.33 70.1*2 67.83 72.1*1 1*0.83 28.50 Total 105 100 F-Value 0.26 0.30 0.31* 0.55 0.63 1.10 31 TABLE EIGHT CATEGORY AND SUB-CATEGORY MEAN SCORES AND P-VALUES OP Q-SORTS: GROUP POUR: CLASSIFIED ACCORDING TO EXTENT OF INSTRUCTIONAL RESPONSIBILITIES Instruc t i o n a l R e s p o n s i b i l i -ti e s No. i n % of Means Group Group Cat.l Cat.2 Cat.3 Cat.l}. S-cat.lA S r c a t . l l Fulltime 87 83 68.59 71.79 68.47 71.01 41.91 26.68 Half-time 5 5 66.00 71.60 68.20 74.20 41.00 25.00 Less than half time 13 12 71.77 68.77 70.15 69.31 42.54 29.23 Total 105 100 F-Value 2.10 0.62 0.1+8 1.05 0.22 2.84* •K- S i g n i f i c a n t at the .05 l e v e l 32 the .0£ l e v e l of confidence are found i n the Q-sorts of instructors grouped according to i n s t r u c t i o n a l responsi-b i l i t i e s . Instructors reporting less than half-time i n s t r u c t i o n a l r e s p o n s i b i l i t y attach le s s importance to the measurements of student progress than do those who have f u l l or half-time i n s t r u c t i o n a l r e s p o n s i b i l i t i e s . A s i m i l a r analysis was performed on scores grouped according to the extent of administrative r e s p o n s i b i l i t i e s . These data are presented i n Table Nine. S i g n i f i c a n t d i f f e r -ences at the .05 l e v e l of confidence are found i n the Q-sorts of instructors grouped according to administrative respon-s i b i l i t y . Those with f u l l time administrative responsi-b i l i t y attach less importance to the measurement of student progress than do those with administrative r e s p o n s i b i l i t i e s reported as half time or l e s s . In view of the f a c t that there are s i g n i f i c a n t d i f f e r -ences according to the amount of time devoted to instruc-t i o n and to administration, this sub-hypothesis i s rejected. Consequently, there are differences i n the perceptions of evaluation which are related to the r e s p o n s i b i l i t i e s of the nursing i n s t r u c t o r . Sub-hypothesis 5>. The Q-sorts of nursing instructors are not influenced by the nature of the i n s t r u c t i o n a l s e t t i n g . Since i n s t r u c t i o n i n schools of nursing tends to be either i n a classroom or a c l i n i c a l s e t t i n g this character-33 TABLE NINE CATEGORY AND SUB-CATEGORY MEAN SCORES AND P-VALUES OP Q-SORTS: GROUP FIVE: CLASSIFIED ACCORDING TO EXTENT OP ADMINISTRATIVE RESPONSIBILITIES Admini s t r a t i ve R e s p o n s i b i l i t i e s No* i n Group % of Group Means Cat.l Cat.2 Cat.3 Cat.lj. S-cat.lA S-cat f u l l time 11 17 71.73 68.91 69.54 69*82 42.47 29.45 half time 4 6 65.00 71.00 70.75 73.25 40.75 24.25 less than half time 50 77 68*34 71.86 68.56 71.15 41.92 26*42 Total 65 100 F-Values 2.23 0.48 0*30 0*54 0.15 3»48* * S i g n i f i c a n t at the .05 l e v e l 34 i s t i c was used to test i f there were any s i g n i f i c a n t differences among, nursing instructors with respect to the amount of time they spent i n (1) classroom i n s t r u c t i o n , (2) c l i n i c a l s i t u a t i o n s , or (3) equally i n both. A single factor analysis of variance among these groups showed no s i g n i f i c a n t difference at the .05 l e v e l of confidence, therefore, this sub-hypothesis i s accepted. Sub-hypothesis 6. The Q-sorts of nursing instructors are not influenced by th e i r i n s t r u c t i o n a l focus. The i n s t r u c t i o n a l focus of the respondents was measured by an i d e n t i f i c a t i o n of the types of courses which they taught. These were c l a s s i f i e d into three main types which consisted of courses i n the (1) physical sciences, (2) s o c i a l sciences, and (3) administration, teaching or supervision. A s i n g l e factor analysis of variance among instructors grouped accord-ing to these three main types of courses produced no s i g n i f i -cant differences at the .05 l e v e l of confidence; therefore, this sub-hypothesis i s accepted. Thus, the perceptions of evaluation held by nursing instructors i s not influenced by the type of course they teach. Sub-hypothesis 7» The Q-sorts of nursing instructors are not influenced by the type of school i n which they teach. A single factor analysis of variance was performed on the category and sub-category scores of instructors grouped according to the type of school i n which they teach. The data 35 TABLE TEN CATEGORY AND SUB-CATEGORY MEAN SCORES AND P-VALUES OP Q-SORTS: GROUP SIX: CLASSIFIED ACCORDING TO INSTRUC-TIONAL SETTING Instrue- No. i n % of ti o n a l Group Group C a t . l Cat.2 Cat.3 Cat.lj. Sub-oat.1A Sub-cat.IB Setting Classroom 33 32 70.15 70.76 69.09 69.82 1*2.15 28.00 C l i n i c a l 50 1+8 67.6ij. 71.98 68.70 71.72 1*1.7** 25.90 Classroom & C l i n i c a l 20 20 69.14-5 71.1*0 67.1*0 71.35 1*1.90 27.55 Total 103 100 P-Values 1.78 0.17 0.51* 0.87 0.08 2.97 36 TABLE ELEVEN CATEGORY AND SUB-CATEGORY MEAN SCORES AND P-VALUES OP Q-SORTS: GROUP SEVEN: CLASSIFIED ACCORDING TO TEACHING FOCUS Teaching No. i n % of Means Focus Group Group Ca t . l Cat.2 Cat.3 Cat.Ij. S-cat.lA S-cat.lB Social Sciences 36 35 70.1)7 71.08 67.69 70.72 I4.2.I+2 28.06 Physical Sciences 56 $k 67.79 71.57 68.86 71.62 l(.1.59 26.20 Administra-t i v e , Supervision 11 11 68.1*5 72.27 70.00 69.09 I4.I.8I* 26.61). Teaching. Total 103 100 P-Values 2.10 0.07 0.79 O.76 O.37 2.28 37 are presented i n Table Twelve. S i g n i f i c a n t differences at the .05 l e v e l of confidence are found between the Q-sorts of nursing i n s t r u c t o r s teaching i n d i f f e r e n t types of schools. These differences occur i n the categories of Achievement and Progress, Psychological E f f e c t s . When measurement of achievement i s considered separately i n sub-category 1A, the difference i s s i g n i f i c a n t at the .01 l e v e l . Sub-hypothesis 7 i s rejected. Instructors who teach i n hospital schools attach l e s s Importance to the measurement of student achievement and progress and more importance to the psychological effects of evaluation than do nursing instructors i n u n i v e r s i t y schools. Sub-hypothesis 8. The Q-sorts of nursing instructors are not influenced by having had a course i n tests and measurements• A single factor analysis of variance was performed on the category and sub-category scores of the instru c t o r s grouped as to whether or not they had a course i n tests and measurements. The data are presented i n Table Thirteen. Differences i n Q-sorts of nursing instructors who have had a course i n tests and measurements and the Q-sorts of nursing i n s t r u c t o r s who have not had such a course are not s i g n i f i c a n t at the .0£ l e v e l , therefore, this sub-hypothesis i s accepted. This indicates that naving had a course i n tests and measurements does not influence the perception of evaluation held by nursing i n s t r u c t o r s . 38 TABLE TWELVE CATEGORY AND SUB-CATEGORY MEAN SCORES AND P-VALUES OP Q-SORTS: GROUP EIGHT: CLASSIFIED ACCORDING TO TYPE OP SCHOOL IN WHICH THEY TEACH Type of No. i n School Group % of Group Means Cat. l Cat.2 Cat . 3 Cat.4 S-cat.lA S-cat.lB Hospital 89 85 69.45 70.46 68.79 70.99 42.57 26.88 University 16 15 65.56 75.69 68.00 70.75 38.44 27.12 Total 105 100 P-Values 5.60* 4 . 3 3 * 0.24 0.02 12.84** 0.04 * S i g n i f i c a n t at the .05 l e v e l ** S i g n i f i c a n t at the .01 l e v e l 39 TABLE THIRTEEN CATEGORY AND SUB-CATEGORY MEAN SCORES AND P-VALUES OP Q-SORTS: GROUP NINE: CLASSIFIED ACCORDING TO A (COURSE IN TESTS AND MEASUREMENTS Course i n Tests and Measurements No. i n Group % of Group Means Cat.l Cat.2 Cat.3 Cat.l; S« -cat.lA S-cat.lB No 74 70 69.2b 71.01+ 68.63 70.95 42.18 27.08 Yes 31 30 67.90 72.29 68.74 70.97 41.39 26.52 Total 105 100 F-Values 1.05 0 . 4 l 0.07 0.01 O.67 O.41 ho Sub-hypothesis 9. The Q-sorts of nursing instructors are not influenced by stated degree of s a t i s f a c t i o n with preparation as an evaluator. In view of the f a c t that d i f f e r e n t instructors may have d i f f e r i n g perceptions of t h e i r own a b i l i t y and prepara-tion f o r evaluation, a scale was devised to measure an individu a l s s a t i s f a c t i o n with her preparation i n evalua-t i o n . This scale consisted of eight items which were weighted and which provided a scale score. These scores were translated into three degrees of s a t i s f a c t i o n : (1) low, (2) medium, and (3) high. The Q-Sort scores for each category and sub-category were tested by a single factor analysis of variance, and s i g n i f i c a n t differences were found at the .05 l e v e l among the degrees of s a t i s f a c t i o n so that this sub-hypothesis i s rejected. The data are pre-sented i n Table Fourteen. There i s a s i g n i f i c a n t r e l a t i o n s h i p between the degree of s a t i s f a c t i o n with the preparation as an evaluator and the importance attached to the measurement of achievement and progress. Instructors with a high l e v e l of s a t i s f a c -t i o n attach greater Importance to achievement and progress than to any of the other functions and effects of evalua-tion tested. When the measurement of achievement i s con-sidered separately as sub-category 1A, the differences are not s i g n i f i c a n t , however, when progress, sub-category IB, i s tested independently the differences were s i g n i f i c a n t kl TABLE FOURTEEN CATEGORY AND SUB-CATEGORY MEAN SCORES AND F-VALUES OF Q-SORTS: GROUP TEN: CLASSIFIED ACCORDING TO STATED DEGREE OF SATISFACTION WITH PREPARATION AS AN EVALUATOR Stated Degree of S a t i s f a c t i o n and Preparation as an Evaluator No . i n Group % of Group Cat.l Cat.2 Cat.3 Cat.i|. S-cat.lA S-cat.lB low 9 9 73.22 66.56 72.33 67.67 1+2.79 30.1+1* medium 81 77 68.83 72.33 68.02 7O.7I4- 1*2.02 26.80 high 15 Ik 66.1*0 69.33 69.93 71*.07 1*1.00 25.14-0 Total 105 100 F-Values 3.60* 2.15 2.70 3.0k 0.1+9 1*.67* # S i g n i f i c a n t at the .05 l e v e l at the .05 l e v e l . Thus, instructors with higher l e v e l s of s a t i s f a c t i o n attach greater importance to the measure-ment of progress than do instruc t o r s with lower leve l s of s a t i s f a c t i o n . Summary The analysis of the data shows that an instructor's perception of the importance of various functions and effects of evaluation i s influenced by certa i n factors i n her experience and preparation. S i g n i f i c a n t differences i n the Q-sorts were found that i n d i c a t e that three of the cha r a c t e r i s t i c s studied appear to be rel a t e d to the 1 n d i v i -dual's perception of the function and effects of evalua-t i o n . Instructors whose r e s p o n s i b i l i t i e s are primarily administrative attach l e s s importance to the measurement of student progress than do those whose r e s p o n s i b i l i t i e s are mainly i n s t r u c t i o n a l . Those instructors working i n hospital schools attach less importance to the measurement of student achievement and progress and more importance to the psychological e f f e c t s of evaluation than do those working i n un i v e r s i t y schools of nursing. F i n a l l y , higher degrees of s a t i s f a c t i o n with preparation for evaluation i s associated with the attachment of greater importance to the measurement of student progress than to any of the other functions of evaluation. 43 Analyses of Q-Sort Item Scores The i n d i v i d u a l item statements which make up the Evaluation Q-Sort were analyzed and ranked i n order of the importance assigned them by nursing i n s t r u c t o r s . The mean value and the standard deviation of the ten most important items are shown i n Table F i f t e e n . A l l four categories of items are represented among the ten most important items, however, f i f t y percent of the items are from sub-category IB which relates to progress and twenty percent from Cate-gory 4 which involves administration. A l l of the remaining categories are represented by ten percent of the items. The preponderance of items related to progress reinforces the analysis presented e a r l i e r i n which the t o t a l group of i n -structors tested tended to rate the measurement of student progress as the most important o f the functions and effects of evaluation considered i n this study. The ten items ranked as l e a s t important are shown i n Table Sixteen. A l l four categories are again represented among the l e a s t important items, however, f i f t y percent are from sub-category 1A which rel a t e s to achievement and t h i r t y percent are from Category 4» which involves administration. The two remaining categories are represented by ten percent of the items. The preponderance of items re l a t e d to achieve-ment reinforces the analysis presented e a r l i e r i n which the t o t a l group of Instructors tested tended to rate the measure-ment of student achievement as the l e a s t important of the TABLE FIFTEEN THE TEN MOST IMPORTANT ITEMS IN RANK ORDER Rank of No. of Item Item Item Category Sub-Category Mean S.D. 2 3 5 6 7 8 9 10 1 Evaluation i s used to determine the extent to which students are achieving the educa-t i o n a l goals of the program. II4. Evaluation i s used to locate Individual learning needs of students. I4.9 Evaluation i s used to assess how well the school i s meeting i t s educational objec-t i v e s . 10 Evaluation i s used to determine the pro-gress students are making i n the learning s i t u a t i o n . 11 Evaluation i s used to show a student how she i s progressing. 13 Evaluation assesses potential f o r further growth i n students 9 Evaluation assesses the extent of changes taking place i n students. lj.6 Evaluation i s used to guide curriculum r e v i s i o n . 28 Evaluation affects confidence of students i n a new learning situation. 34 Evaluation assesses the eff i c i e n c y of teaching methods. 1.Achieve-ment & Progress 1. II n l+.Admini-s t r a t i o n 1.Achieve-ment & Progress 1. » » 1. " " 1. » " ij..Admini-s t r a t i o n 2. Psycholo-g i c a l E f f e c t s 3. Teaching 1.Achieve-ment 2.Progress 2.Progress 2. » 2. » 2. " 2.86 1.53 3.06 1.15 3.30 l.lj.1 3.30 1.18 3.1|5 1.21 3.77 1.29 3.91 1.38 3.914- 1.21 I4..83 I.2I4. I4..98 1.12 TABLE SIXTEEN THE TEN LEAST IMPORTANT ITEMS IN RANK ORDER Rank of No. of Item Category Sub-Category Mean S.D. Item Item 2 3 5 6 7 8 9 10 Evaluation means grading 3 Evaluation i s used to compare the per-formance of groups of students. 22 Evaluation influences peer relationships among students. Evaluation i s used to place students i n categories of achievement 5 Evaluation i s used to compare the per-formance of groups of students. 35 Evaluation i s used to provide an i n -structor with feedback on her teaching. 53 Evaluation i s used by the director i n deciding upon the retention of instruc-tor s . 54 Evaluation i s used by the director i n deciding upon promotion of instructors. 7 Evaluation i s used to determine the current status of students. 55 Evaluation helps administration under-stand the problems faced by students. 1.Achieve- 1.Achievement ment & Progress 1. " " 1. " 2.Psycholo-g i c a l E f f e c t s 1.Achieve- 1.Achievement ment & 7.99 1.22 Progress 1. " " 3.Teaching 4*Admini-s t r a t i o n 1. 4. " 1.Achieve- 1.Achievement ment & Progress i+. Admini-s t r a t i o n 7.39 6.90 6.45 6.29 6.03 5.84 5.69 1.28 1.16 6.77 1.11 1.02 1.57 1.02 1.02 1.23 5.68 1.58 1*6 functions and effects of evaluation considered i n this s tudy. Tabulation of Sources of Consultation The instructors were asked to indicate which of eight possible sources of help they consulted when a problem arose i n evaluation. They were permitted any number of the sources i f they consulted more than one source. The rank ordering of these choices i s presented i n Table Seventeen. TABLE SEVENTEEN SOURCES OF CONSULTATION USED BY NURSING-INSTRUCTORS IN RANK ORDER Rank Consultant Number of Percent Cumula-Instructions tiv e C i t i n g the Percen-Source tage 1 Fellow Instructor 73 30.1+ 30.1+ 2 Director of School 69 28.7 59.1 3 Head Nurse 52 21.7 80.8 4 Nursing Supervisor 19 7.9 88.7 5 Someone Else 17 7.1 95*8 6 Educator 6 2.5 98.3 7 Psychologist 1+ 1.7 100. o No one 0 0 100. Total 2*4.0 100 Persons most frequently cited as source of consultation f o r evaluative problems are nurses; fellow i n s t r u c t o r s , the director of the school, the head nurse or the nursing super-v i s o r . When the percentage checking these sources are t o t a l l e d I t shows that nurses consult mainly with other nurses who may or may not have had any greater knowledge regarding evaluation than they do themselves* The very low use of the psychologist and the educator, who may be presumed to be experts i n evaluation, may indicate that the nurses are unaware that these sources ex i s t , an unwillingness to con-s u l t members of another profession, or i t may indicate that these sources are r e l a t i v e l y inaccessible to nursing, i n -structors* I t i s possible that instructors oonstrue the "problem i n evaluation" as being e s s e n t i a l l y one o f i n s u f f i c i e n t or contradictory data concerning the performance of a part i c u -l a r student rather than a matter of more general concern* I f this i s their understanding, then the frequent consulta-tion with other nurses would r a i s e fewer questions r e l a t i n g to the general problem of evaluation. CHAPTER V CONCLUSIONS AND IMPLICATIONS Evaluation i s recognized as an important problem i n nursing service and education. Some writers have been con-cerned with the enunciation of a philosophy of evaluation f o r nursing while some have designed instruments to measure performance i n c l i n i c a l s i t u a t i o n s ; however, no one has examined heretofore the perceptions of evaluation held by nursing i n s t r u c t o r s . The present study sought to measure the perceptions held by nursing instructors of the r e l a t i v e importance of f i v e functions and effects of evaluation. These were i d e n t i f i e d as (1) the measurement of student achievement, (2) the measurement of student progress, (3) the psycholo-g i c a l effects of evaluation, (2+) the influence of evaluation on teaching, and (5^ the influence of evaluation on admini-s t r a t i v e behavior. In order to measure the perceptions of nursing in s t r u c t o r s , the Q-sort was selected as a suitable instrument. Data were collected from the nursing, i n s t r u c -tors i n the professional schools of nursing i n the Lower Mainland and Vancouver Island areas of B r i t i s h Columbia. The single factor analysis of variance and the Newman-Keuls Method of examining differences between pairs of means was performed on the Q-sort scores of a l l instructors to test the main hypothesis of the study, using the .05 l e v e l 48 k9 of confidence. As a r e s u l t of this analysis the central hypothesis was rejected since student achievement i s per-ceived by instructors as l e a s t important and student progress as most important among the functions and effects of evaluation considered i n this study. The single factor analysis of variance was performed on the Q-sort scores of the various groupings of i n s t r u c -tors to test the sub-hypotheses proposed i n t h i s study. Results led to the acceptance of the following sub-hypotheses: 1. The Q-sorts of nursing instructors are not i n -fluenced by length of experience i n nursing service. 2. The Q-sorts of nursing instru c t o r s are not i n -fluenced by length of experience i n nursing education. 3. The Q-sorts of nursing instru c t o r s are not i n -fluenced by type of preparation as an i n s t r u c -tor. 5. The Q-sorts of nursing, i n s t r u c t o r s are not i n -fluenced by the nature of the i n s t r u c t i o n a l s e t t i n g . 6. The ^-sorts of nursing instructors are not i n -fluenced by t h e i r i n s t r u c t i o n a l focus. 8. T h e ^ s o r t s of nursing instructors are not i n -fluenced by having had a course i n tests and measurements. Three sub-hypotheses were rejected, namely: ij.. The Q-sorts of nursing instru c t o r s are not i n -fluenced by the nature of t h e i r r e s p o n s i b i l i t i e s . 7. The Q-sorts of nursing instru c t o r s are not i n -fluenced by the type of school i n whicn they teach. 9. The 4-sorts of nursing instructors are not i n -fluenced by degree of s a t i s f a c t i o n with preparation as evaluators. 50 This study has determined that nursing i n s t r u c t o r s attach s i g n i f i c a n t l y d i f f e r e n t degrees of importance to the functions and e f f e c t s of evaluation considered i n this study. Least importance i s ascribed to the measurement of student achievement and most importance to the measure-ment of student progress. Moderate importance i s attached to the other three functions and effects studied. Varia-bles that a f f e c t the perceptions of evaluation held by nursing i n s t r u c t o r s are the nature of her r e s p o n s i b i l i t i e s , the type of school i n which she teaches, and her stated l e v e l of s a t i s f a c t i o n with preparation as an evaluator. Since the Evaluation Q-Sort has proven to be a s a t i s -factory instrument to measure the perceptions of evaluation held by nursing in s t r u c t o r s , i t could be used to study the perceptions held by other groups associated with nursing education. I t could, f o r example, be used to assess the perceptions of evaluation held by nursing students and nursing service personnel. This would provide an opportunity to compare the perceptions of evaluation held by students, by service personnel and by Instructors - the three groups most concerned with the evaluation of the c l i n i c a l perfor-mance of student nurses. Such a comparison would test whether a lack of common understanding, of evaluation gives r i s e to the negative feelings towards evaluation reported by Lucas (61+), Rosen ( 78 )» and Gorham (1+6). A s i m i l a r study 51 of the perceptions of evaluation held by students at various i n t e r v a l s i n their program might locate possible changes i n perception which could help to explain the developmental se-quence of attitudes described by Lucas (61j.). A scrutiny of the analysis of the scores of the Evaluation Q-Sort by begin-ning nursing students would i d e n t i f y differences i n percep-tions from those held by inst r u c t o r s , thus i n d i c a t i n g changes needed to reconcile any divergent views of the functions of evaluation. Various learning experiences to a f f e c t such r e c o n c i l i a t i o n of perceptions could be planned as an i n t e g r a l part of the program of the nursing student. Furthermore, this study suggests other matters f o r discussion and exploration. I t has been demonstrated that differences do e x i s t among instructors with respect to the importance they ascribe to the f i v e effects and functions of evaluation included i n this study. I t can be postulated that i t i s important f o r nursing educators to have a common under-standing of such a c r i t i c a l element i n the educational pro-cess. Means might be sought to study the functions and pur-poses of evaluation i n an e f f o r t to achieve greater agree-ment among ins t r u c t o r s than exists at present. Within a p a r t i c u l a r nursing school such differences i n the perceptions of evaluation among, the s t a f f are probable. The proportions of administration to teaching r e s p o n s i b i l i t i e s and the l e v e l of s a t i s f a c t i o n with preparation as an evaluator 52 have been demonstrated to be i n f l u e n t i a l i n producing such divergence of opinions and emphasizes the need f o r enunciat-ing a philosophy of evaluation to guide the evaluative pro-cess. Such a philosophy should be made known to students as well as to the s t a f f of the school. I f instructors perceived of and used evaluation i n a consistent manner, comparison made of the performance of students i n various courses would be more meaningful and the students would be more l i k e l y to experience evaluation as that p o s i t i v e , help-ing, growth-inducing process described so frequently and so longingly throughout the nursing l i t e r a t u r e on evaluation. The p r o b a b i l i t y that students would develop evaluative s k i l l s to appraise nursing care as well as the i r own performance would be enhanced. I t should be noted also that increasing an ins t r u c t o r ' s s k i l l and self-confidence i n evaluation w i l l probably influence her perception of the process. Nursing instru c t o r s teach In either a hospital or uni v e r s i t y school during the course of their professional careers. Recognizing that differences i n the perceptions of evaluation do exis t between hos p i t a l and un i v e r s i t y schools emphasizes the Importance of enunciating a philosophy of evaluation and using i t i n the orient a t i o n of new i n s t r u c -tor s . Since many graduates from hospital schools seek further eduoation i n uni v e r s i t y schools, they w i l l be expec-ted to accommodate themselves to a d i f f e r e n t emphasis i n evaluation whicn raises an important question concerning the 53 orientation of the nurse to the new educational setting. This study of the perceptions of evaluation held by nursing instructors has demonstrated that s i g n i f i c a n t d i f -ferences do e x i s t among instructors concerning the import-ance of the f i v e functions and effects of evaluation selec-ted f o r study. I f this i s found to be the case i n Schools of Nursing i t w i l l probably be found to e x i s t also i n other educational situations and i n s t i t u t i o n s . The Evaluation Q-Sort developed f o r this study i s applicable to similar measurements i n other situations such as professional or vocational schools and with instructors i n adult education programs* Only through such continuous research and educa-t i o n can we develop evaluation that Is functional and uniform. 5k BIBLIOGRAPHY. 1. Abhold, Avis Ann. "An Evaluation of Some A b i l i t i e s of a Selected Group of Pe d i a t r i c Nursing Students to Apply Certain S o c i o l o g i c a l and Psychological Concepts i n the P e d i a t r i c Mealtime Situation at One Hospital i n Seattle," M.A. Thesis, University of Washington, 1956. 2. Ahmann, J . Stanley and Glock, Marvin D. "Evaluating Pupil Growth". 2nd ed., Boston: A l l y n and Bacon, 1963. 3. Anastasi, Anne. Psychological Testing. New York: MacMillan Company, 1961. i j . . Bauernfeind, Robert H. Building a School Testing Program. Boston: Houghton-Mifflin, 1963. 5 . Block, J. Q-sort Method In Personality Assessment and Psy-c h i a t r i c Research. S p r i n g f i e l d , I l l i n o i s : Thomas, 1961. 6. Bloom, Benjamin S. (ed), Taxonomy of Educational Objectives. New York: Longman's Green & Co., 1956. 7. Boozer, Mary K. "A Study to Determine the Usefulness of ah Instrument to Evaluate the A b i l i t y to Apply Selected Administrative Concepts i n the Immediate Solution of Nursing Problems," M.A. Thesis, University of Washington, 1955* 8. Bourque, Ellsworth. "A Study by Q-Technique of How the Disabled Person Perceives his D i s a b i l i t y , " Doctoral Dissertation, Boston University, 1958. 9. Bower, E l i Mlchal. "The Application of Q-Methodology i n Investigating ^ Changes i n Self and Ideal-Self as a Result of a Mental Health Workshop," Ed. D., Disser-tation, Stanford University, 19f?l+.. 10. B r a d f i e l d , James M., and Moredock, Stewart. Measurement and Evaluation i n Education. New York: MacMillan, 1957. 11. Brester, Mary and others. "Evaluating Nursing. Students" American Journal of Nursing, 62t 117-119 (1962). 12. Burt, S i r C y r i l . "Correlations Between Persons," B r i t i s h  Journal of Psychology, 28: 59-96 (1937). 55 13. Butler, Herbert John. "The Role of the P s y c h i a t r i c Nurse as Perceived by Nurses, Members of the Related D i s c i p l i n e s Within the Mental Hospital, and Nursing Students," Ph. Ed. Dissertation, Boston University, 1959. 14* , and Flood, Francis R., "Evaluating Nursing Care i n a Mental Hospital," American Journal of Nursing, 62: 84-85 (1962). 15. Butler, J.M. and Fiske, D.W. "Theory and Techniques of Assessment," Annual Review of Psychology, 6: 327-356 (1955)" 16. Campbell, William G. Form and Style i n Thesis Writing. Boston: Houghton-Mifflin Company, 1954. 17. C a t t e l l , R.B. "On The Disuse and Misuse of the P, Q, Qs and 0 Technique," C l i n i c a l Psychology, 8: 203-205 (195D. 18. Chamberlain, Martin N. "The Professional Adult Educator: an Examination of his Charact e r i s t i c s and the Programs of Graduate Study Which Prepare him for Work i n the F i e l d , " Boctoral Dissertation, University of Chicago, I960. 19. Chernushin, Mary and Galvin, James. "Evaluation that Is Frankly Subjective," American Journal of Nursing, 62: 92-93 (1962H 20. Church, Jean G. "Evaluation of Nursing Service Personnel," Canadian Nurse. 58: 109-112 (1962). 21. Cochran, Thelraa C. and Hansen, Paul J. "A Nurse-Devised Evaluation System f o r Nurses," Hospitals, 36: IOO-IOI4., 161-162 (1962). 22. Cohen, J . "An a i d i n the Computation of Correlation Based on Q-sorts," Psychological B u l l e t i n , 54* 138-139 (1957). 23. Creasor, J . "An Aid i n Calculating,.Q-sort Factor Arrays," Journal of C l i n i c a l Psychology, 11: 195-196 (1955). 24» Cronbach, Lee J. "Correlations Between Persons as a Research Tool," i n Psychotherapy: Theory and Research edited by 0. Hobart Mowrer, New York: Ronald Press, 1953. 56 2£. Cronbach, Lee J . Essentials of Psychological Testing. 2nd ed., New York: Harper & Brothers, I960. 26. and Gleser, Goldine C. "Review of Stephenson, ¥. The Study of Behavior: Q-Technique and i t s Methodology," Psychometries, 19: 327-330 (1954)' 27* Downie, N o r v i l l e M. Fundamentals of Measurement: Techniques and Practices. New York: Oxford Uni-v e r s i t y Press,. 195>8. 28. Draper, James Anson. "A Study of Part i c i p a n t Objectives i n a Selected Management I n s t i t u t e , " Ph.D. Dissertation, University of WIsoonsin, 1961+.. 29. Dressel, Paul L. and Associates. Eyaluatlon i n Higher Education. Boston: Houghton-Mifflin, 1961. 30. Dunlap, Marjorie S. and Hadley, Betty Jo. "Quasi Q-sort Methodology, i n Self-Evaluation of Conference Leadership S k i l l , " Nursing Research, llj.: 119-125 (1965). 31. Edwards, A. " S o c i a l D e s i r a b i l i t y and Q-sorts," Journal of Consult. Psychology. 19: lj.62-lf.65 (1955)* 32. Ellsworth, R.B. and Others. "The A.P.E.V. Scale: a Study of Aide Performance as Perceived by Nurses, Aides and Patients," Nursing Research, 11: 15-20 (1962). * — 33. F i e l d , Lois R. "An Evaluation of Student Progress i n Understanding and Application of Mental Health P r i n c i p l e s i n Basic Nursing Education," M.A. Thesis, University of Washington, 1954. 3l|-. Flanagan, John C. and Others. "Evaluating, Student Performance," American Journal o f Nursing, 63: 96-99 (1963) 35* Fletcher, M. Josephine. "Development and Testing of a Tool for the Evaluation of Nursing Students i n Medical Surgical Practice i n Diploma Schools of Nursing," M.A. Thesis, De Paul University, Chicago, 1962-63. 36. Frank, G.H. "Notes on the R e l i a b i l i t y of Q-sort Data," Psychological Reports. 2: 182 (1956). 57 37• Freeman, Ruth E. "Measuring, the Effectiveness of Public Health Nursing Service," Nursing Outlook, 9: 605-607 (1961). 38. Garrett, Henry E. S t a t i s t i c s i n Psychology and Educa-t i o n . 5th ed., New York: Longman's Green and Company, 1958• 39. Gerberich, Joseph R. and others. Measurement and Evaluation In the Modern School. New York: D. McKay Co., 1962. ij.0. Gerchberg, L.R. An Observational Method f o r Evaluating  the Performance of Nursing Students i n C l i n i c a l  Situations. New York: National League f o r Nur-sing, 1962. 41. • "An Observational Method f o r Evaluating Student Nurse Performance i n the C l i n i c a l Experi-ence," M.A. Thesis, San Francisco State College, 1961-62. ij.2. G i l c h r i s t , Joan M. "Performance Evaluation i n Nursing Service," Canadian Nurse, 58: 113-116 (1961). 43* Glaser, William A. and others. "The Performance of Students i n Public Health," Nursing Researoh, 14s 138-143 (1965). l|4» Good, Carter V. (ed.). Dictionary of Education. 2nd ed., New York: McGraw-Hill, 1959. 45* Goodling, R.A. and Guthrie, G.M. "Some P r a c t i c a l Considerations i n Q-sort Item Selection," Journal  of Consult. Psychology, 3: 70-72 (1958). 46. Gorham, W.A. "Methods f o r Measuring Staff Nursing Performance," Nursing Researoh. 12$ 4 . - H (1963). 47» Gorham, Donald R. "An Evaluation of Attitudes To-wards Psych i a t r i c Nursing Care," Nursing Re-search, 7: 71-76 (1958). 4 8 . Guilford, J.P. Psychometric Methods. New York: McGraw-Hill, 1954. 4 9 . Hansen, Ann C. and Upshaw, Harry S. "Evaluating Within the Context of Role Analysis," Nursing Research 11: 56-58 (1962). 58 50. Hartley, Marion. "A Q-technique study of Changes i n Self Concept During Psychotherapy," Doctoral Di s s e r t a t i o n , University of Chicago, 195l« 51. Heslln, P h y l l i s . "Evaluating C l i n i c a l Experience," Nursing Outlook, 11: 344-345 ( 1963). 52. Heter, Ruth J . "Development of an Evaluative Instru-ment to Aid i n Measuring the A b i l i t y to Apply Administrative Concepts i n the Solution of Nur-sing Problems," M.A. Thesis, University of Washington, 1955* 53. Hilden, A.H. "Q-sort Correlation: S t a b i l i t y and Random Choice of Statements." Journal of Consult. Psychology. 22: 45-50 (1958). 54* . Manual f o r Q-sort and Random Sets of Personal Concepts. Webster Groves, Mo: Author, 1954» ! 55» Hoffman, Katharine J. "A Suggested Method f o r the Development of a Tool to Aid i n the Evaluation of Performance i n Nursing," Ph.D. Dissertation, University of Washington, 1956. 56. Hoffman, Banesh. The Tyranny of Testing. New York: Crowell-Collier Press, 1962. 57* Hood, Paul IDbuglas. "Q-Methodology: a Teohnique f o r Measuring, Frames of Reference, Boctoral Disserta-ti o n , Ohio State University, 1953* 58. Howard, Robert C. and Berkowitz, Leonard. "Reactions to the Evaluation-of One's Performance," Journal  of Personality. 26: 494-507 (1958). 59« Ingmire, A l i c e E. "The Attitudes of Student Nurses at the University of C a l i f o r n i a , " Doctoral Dissertation, Stanford University, 1949. 60. Jones, Austin. "The D i s t r i b u t i o n of T r a i t s i n Current Q-sort Methodology," The Journal o f Abnormal and S o c i a l Psychology. 52: 90-96 (1956). 61. Kerlinger, Fred N. "Attitude Structure of the I n d i v i -dual - a Q-sort of Educational Attitudes of Pro-fessors and Laymen," Genetic Psychology Monographs, 53: 283-329 (1956). 59 62. Kretch, David and others. Individual i n Society. New York: McGraw-Hill, 1962. 63* Latchaw, Marjorie and Brown,Camille. The Evaluation  Process i n Health Education, Physical Education  and Recreation." Englewood C l i f f s , N.J.: Prentice Ha l l , 1962. 6I4.. Lucas, Pauline. "Some Factors Which Influence Reactions of Student Nurses to Evaluation of t h e i r Perfor-mance i n C l i n i c a l Practice i n a Selected School of Nursing i n Seattle," M.A. Thesis, University of Washington, 1954• 65. Malaspina, Helen. "How to Evaluate Nurses' Work," Modern Hospital, 97: 71-74 (1961). 66. Medaris, E l i z a b e t h B. "Evaluation of the Supervisor," Nursing Outlook, 11: 660-661 (1963). 67. Meyer, Marjorie A. "Evaluation of Achievement of Stu-dents i n Medical Surgical Nursing," M.A. Thesis, Keene Teacher's College, 1961-62. 68. M i l l e r , Harry L. and McGuire, Christine H. Evaluating L i b e r a l Adult Education. Chicago: Center-for Study of L i b e r a l Education f o r Adults, 1961. 69. Morgan, C l i f f o r d T. Introduction to Psychology. New York: McGraw-Hill, 1956. 70. Mouly, George J. Sclenee of Educational Research. New York: American Book Co., 1963. 71. Mowrer, Hobart. Psychotherapy: Theory and Research. New York: Ronald Press, 1953. 72. Nahm, Helen and others. "Evaluating Students' Progress i n C l i n i c a l Experience," American Journal of  Nursing, 50: 309-311 (1950"H 73* National League for Nursing. A Method for Rating the  Proficiency of the Hospital General Staff Nurse - Manual of D i r e c t i o n s . New York: National League fo r Nursing, 1964• 60 7l|.« N o l l , Charlotte I . "An Investigation of Methods Used i n the Evaluation of C l i n i c a l Practice of the Student i n P s y c h i a t r i c Nursing," M.A. Thesis, MacMurray College, Jacksonville, 1962-63. 75. Odell, C.W. "Marks and Marking,Systems," Encyclo-pedia of Educational Research, rev. ed. New York: MacMillan Company, 1950* Pp. 711-717* 76. Oldridge, Olive Ambrose. "An Experimental Study of Two Guidance Emphases i n the Elementary School," Doctoral Dissertation, University of Southern C a l i f o r n i a , 1963. ' 77• Rines, A l i c e . Evaluating Student Progress i n Learning  the Practice of Nursing*! New York: Bureau of Publications, Teachers College, Columbia Univer-s i t y , 1963. 78. Rosen, Albert, and Abraham, Gertrude E. "Evaluation of a Procedure f o r Assessing, the Performance of St a f f Nurses," Nursing Research, 2: 78-82 1963. * 79. Royal Commission on Health Services. Report, V o l . I . Ottawa: Queen's Printer, 1964* 80. Schultz, Frances K. "Evaluation: Signpost or Judgment?," Nursing Outlook, 12, 57-58 (1964). 81. Shetland, Margaret. "Evaluation - a Constructive Process," Publle Health Nursing, 99-101 (1949). 82. Shyen, Ann and others. "Evaluating Public Health Nursing Service to the Maternity Patient and her Family," Nursing Outlook, 11: 56-58 (1963). 83. Simmons, Leo W. and Henderson, V i r g i n i a . Nursing Research: a Survey and Assessment. New York: Appleton-Century-Crofts, 1964. 84* Small, Muriel E. "The Development of a Tool to A s s i s t i n Performance Evaluation i n Public Health Nur-sing," M.A. Thesis, University of Washington, 1955-85» Stephenson, William. "'Correlating Persons Instead of Tests," Character and Personality, 4: 17-24 (1935). * 61 86. Stephenson, William. The Study of Behavior; ft-Technique and Its Methodology. Chicago: University of Chicago Press, 1953* 87. • "Comments on Cronbach and Gleser's Review of: The Study of Behavior Q-Technlque  and I t s Methodology. Psychometries, 19: 331-333 (1954). 88. Symonds, Pe r c i v a l M. "Pupil Evaluation and S e l f -Evaluation," Teachers College Record, 54: 138-149 (1952). 89. Tate, Barbara L. "Evaluating the Nurses' C l i n i c a l Performance," Nursing Outlook, 10: 35-37* (1962). 90. • "Evaluation of C l i n i c a l Performance of the Staff Nurses," Nursing Research, 11: 7-9 (1962). 91. Todd, Frederick J . "An Evaluation of a Continuation Education Program i n Nursing," University of Colorado, Boulder, Colorado, Behavior Research Laboratory Report No. 12, I960. 92. Tschudln, Mary and others. Evaluation i n Basic Nur-sing Education. New York: G.P. Putnam's Sons, 1958. 93* Turabian, Kate L. A Manual for Writers of Term Papers, Theses and Dissertations. Chicago: University of Chicago Press, 1955* 94« Tyler, Louise L. "The Concept of an Ideal Teacher-Student Relationship," The Journal of Educa-t i o n a l Researoh, 58: 112-117 (1964). 95* Wandt, Edwin and Brown, Gerald W. Essentials of Educational Evaluation. New York: Holt, 1957* 96. Whiting, J . Prank. The Nurse-Patient Relationship and  and the Healing Process. New York: American Nurses' Foundation, 1958. 97* • "Q-sort: a Technique f o r Evaluating Percep-Eions of Inter-personal Relationships," Nursing  Research, 4: 70-73 (1955). 62 98. Whiting, J . Prank. "Patients' Needs, Nurses' Needs and the Healing Process," American Journal of  Nursing, 59: 661-665 (1959TI 99. Winer, B.J. S t a t i s t i c a l P r i n c i p l e s i n Experimental Design. New York: McGraw-Hill, 1962. 100. Wittenborn, J.R. "Contribution and Current Status of Q-Methodology." Psychological B u l l e t i n , 58: 132-U+2 (1959). 101. Woodworth, Freda. "A Head Nurses Ideas on Evaluation," Canadian Nurse. 58: 117-119 (1962). — ) 102. Wrightstone, J . Wayne. "Evaluation." Encyclopedia of Eduoational Research, revised ed~ New York: MacMillan Co., 1950. 63 APPENDIX I 1. DIRECTIONS FOR THE TEST FOR TRUTH AND IMPORTANCE _ _ _ _ _ _ _ _ _ _ _ j Research study.: A study of nursing i n s t r u c t o r s 1 attitudes towards evaluation* Background: The f i n a l research instrument to be used i n c o l l e c t i n g the data f o r this study w i l l be a Qr sort of 60 items regarding the functions and eff e c t s of evaluation. Your task: The f i r s t step i s to be sure that the l i s t of items i s r e l a t i v e l y a complete statement of functions and effects of evaluation* These items have been compiled from nursing and educational l i t e r a t u r e and studies. You are requested to do the following three things: 1* Inspect each item to determine i f I t i s a function or e f f e c t of evaluation. I f the statement i s true, e n c i r c l e the T. (true). I f I t i s not a true statement, c i r c l e the F. ( f a l s e ) * 2* Inspect each item a second time to determine i f the function or ef f e c t Is important* I f i t i s a function, then i t may be important to the student, the in s t r u c t o r , the administrator or the community that i t be performed* I f I t i s an e f f e c t , then i t may be one that a f f e c t s students, i n -structors, administrators or the community. I f the item has importance, c i r c l e the I* (important) but i f you believe that i t has no importance, c i r c l e the U. (unimportant). 3* On page 10 you may l i s t any item which you think should be added' to this l i s t . Possibly some of the items marked f a l s e would be acceptable with a simple rewording* Your help i n this area would be p a r t i c u l a r l y important at this stage of the study* 64 2. INSTRUCTIONS FOR VALIDITY OF CATEGORY TEST This t e s t i s one of the steps i n eliminating items i n the preparation of a Q-sort to assess the attitudes of nursing Instructors towards ©valuation. The items printed on 3" x l&n cards are statements about the functions or ef f e c t s of evalua-t i o n . The categories are described as follows: 1. ACHIEVEMENT: Evaluation i s a process that measures the per-formance of a l l students i n a group, at the conclusion of a learning •xperience, with respect to the degree of achievement of s p e c i f i e d learning objectives. 2. PROGRESS: Evaluation i s a process that assesses the be-havior of i n d i v i d u a l students i n a learning s i t u a t i o n i n order to define her learning needs and problems as well as her progress towards achieving s p e c i f i e d learning ob-je c t i v e s . 3. PSYCHOLOGICAL EFFECTS: Evaluation i s a process which influences the motivation, attitudes, feelings and i n t e r a c t i o n of stu-dents and i n s t r u c t o r s . !(.. TEACHING: Evaluation Is a process that influences teaching. 5 . ADMINISTRATION: Evaluation i s a process that influences the administration of a school. PROCEDURE FOR CATEGORY JUDGE 1. On each card i s written a function or e f f e c t of evaluation. 2. You are to read each item and place i t i n the appropriate l a b e l l e d s l o t i n the category box. 3. I f , i n your opinion, the Item does not belong i n any of the f i v e categories, place i t i n the "No category" s l o t . 65 3. DATA SHEET FOR EVALUATION Q-SORT Instructor No._ 1# Years of experience i n nursing service 0 to 2 3 to 5 6 to 10 10 to 20 over 20 2. Years of experience i n nursing education 0 to 2 3 to 5 6 to 10 11 to 20 over 20 3» Type of program taken as preparation to i n s t r u c t None — Post Basic Baccalaureate Diploma Degree Basic Baccalaureate Degree Master's Degree ________ l±. Teaching r e s p o n s i b i l i t i e s are: f u l l time half time les s than h a l f time 5- Administrative r e s p o n s i b i l i t i e s are: f u l l time half time les s than h a l f time _______ 6, I n s t r u c t i o n a l s e t t i n g i s mainly: classroom c l i n i c a l _____ J. Teaching focus i s : S o c i a l Sciences i n Nursing ) Physioal Sciences i n Nursing) Maternal and Child " ) Surgical Nursing ), Psy c h i a t r i c w ) Operating Room Nursing )" Public Health n ) Medical Nursing ) Admini stration) Teaching ) Supervision ) 8 . Type of school i n which you teach: Hospital _____ uni v e r s i t y 9» Have you had a course at u n i v e r s i t y i n tests and measurements? No Yes 10, Do you f e e l that your preparation i n evaluation of content was: Unsatisfactory _______ Adequate _____,.. Excellent ? 11» Do you f e e l that your preparation i n evaluation of performance was: Unsatisfactory ______ Adequate . Excellent ? 12. Do you f e e l that your preparation i n the development and use of r a t i n g scales was: Unsatisfactory Adequate Excellent ? 13* Do you f e e l that your preparation i n the development and use of check l i s t s was: Unsatisfactory Adequate . Excellent ? li|.» Do you f e e l that your preparation i n the development and use of anecdotal note technique was: Unsatisfactory _____ Adequate Excellent ? (cont'd) 66 Data Sheet f o r Evaluation Q-Sort (cont'd) lf>. Do you f e e l that your preparation i n the development and use of the c r i t i c a l incident technique was: Unsatisfactory _______ Adequate Excellent ? 16. When you began i n your present p o s i t i o n , did you f e e l that the orientation to the school's philosophy of evaluation was; Unsatisfactory Adequate Excellent ? 17- When you began i n your present p o s i t i o n , did you f e e l that the o r i e n t a t i o n to the instruments of evaluation used i n the school was: Unsatisfactory Adequate Excellent ? 18» When you have a problem i n evaluation do you consult with: a psychologist ________ a nursing supervisor an educator a head nurse _________ the d i r e c t o r of the school ________ no one a fellow i n s t r u c t o r ________ someone else (Please specify) 67 4» DIRECTIONS FOR ADMINISTRATION OF EVALUATION Q-SORT  Introduction The Evaluation Q-sort t o o l i s designed to assess nursing i n s t r u c t o r s ' opinions of the r e l a t i v e importance of the various functions and effects of evaluation* This tool i s not designed to test knowledge. The returns are anonymous • Up to ten instructors can be tested at a session* S i m i l a r i t y of testing conditions and procedures are impor-tant* Steps ONE Shuffle the ten packs of 5>6 Q>-sort cards TWO Provide each i n s t r u c t o r with: a) a pack of Q>-Sort cards (white) b) a pack of p i l e cards (Yellow) c) a numbered raw tabulation sheet d) a data sheet with the same number as (c) above e) an i n s t r u c t i o n sheet f o r the evaluation Q-Sort f ) work space equal to half of a card table THREE Read the data sheet to the group of ins t r u c t o r s with explanatory comments and have them complete the form* FOUR Read the inst r u c t i o n s f o r the Q-Sort to the instructo r s FIVE Request the instructors to do the sort, f i l l out the raw tabulation sheet and leave t h e i r materials for the Q-Sort administrator to cheek* SIX C o l l e c t the material and mark the number of sub-jects tested. A record of instructors missing the test must be kept. Three attempts w i l l be made to secure sorts from missing persons i n order to complete the sample. 68 £• INSTRUCTIONS FOR THE EVALUATION Q-SORT Evaluation i s an important process i n nursings education. Evaluation has a number of functions and effects which have varying degrees of importance to i n s t r u c t o r s . Your jobs w i l l be one of sorting 56 cards with statements on them regarding evaluation. While you are sorting.cards you should keep the following, question i n mind: Which of these functions and effects of evaluation do you f e e l are of high importance, of medium importance, of low importance, i n your job as an instructor? Here are the steps to follow i n sor t i n g the cards: Step I: Sort the 56 cards into three roughly equal p i l e s of high, medium and low importance. Place the high p i l e on your l e f t , the low p i l e on your r i g h t , with the medium p i l e i n the middle. Step I I : Prom the high p i l e In Step I, sele c t the nine most important items (cards) and place the r e s t i n the medium p i l e . Then from these nine items, se l e c t the three most important items. Then, from these three items select the one most important item. The r e s u l t w i l l be three p i l e s of one, two and s i x Items each which are placed on p i l e cards #1, #2 and #3 r e s p e c t i v e l y . Step I I I : From the low p i l e In Step I, follow the same pro-cedure as above In Step I I ; I.e. sel e c t the nine l e a s t important items, placing the remainder In the medium p i l e . Then from these se l e c t three, then from these se l e c t one l e a s t important"!! TEe r e s u l t w i l l be three p i l e s of one, two and s i x items which are placed on p i l e cards #9, #8 and #7 r e s p e c t i v e l y . Step IV: Separate the medium p i l e of 38 remaining items into three p i l e s of s l i g h t l y more important, medium im-portance and s l i g h t l y l e s s important. Place the s l i g h t l y more important on your l e f t and the s l i g h t l y l e s s important on your r i g h t . When you are f i n i s h e d sorting, you should have 12 items i n the s l i g h t l y more important p i l e , li+ items i n the medium importance p i l e and 12 items i n the s l i g h t l y l e s s Important p i l e to be placed on p i l e cards #4, #5 and #6 r e s p e c t i v e l y . You w i l l then have nine p i l e s of cards i n the f o l l o w i n g . d i s t r i -bution: Number of p i l e : #1 #2 #3_ #4 #5 #6 #7 #8 #9 Number of items: 1 2 6 12 14 12 6 2 1 6. RAW DATA TABULATION SHEET Instructor No* Ik 12 12 -6 6 2 2 1 1 P i l e 1 2 3 4 5 6 7 8 9 4.— 4 — 4.— 4 — 4 — -—>> —?> — ^ — ^ — ^ Most Important Functions and Eff e c t s Least Important Functions and Ef f e c t s 70 APPENDIX II EVALUATION Q-SORT ITEMS Category 1 - Achievement and Progress  Sub-category 1A- Achievement 1« Evaluation i s used to determine the extent to which students are achieving^ the educational goals of the program. 2. Evaluation means grading. 3. Evaluation i s used to compare the performance of groups of students. 4« Evaluation i s used to place students i n categories of achievement. 5>. Evaluation i s used to oompare the performance of groups of students. 6. Evaluation measures achievement with respect to a pre-determined standard. 7. Evaluation i s used to determine current status of students. Sub-category IB- Progress 8. Evaluation Is used by students as a guide to study. 9. Evaluation assesses the extent of changes taking place i n students. 10. Evaluation i s used to determine the progress students are making i n the learning s i t u a t i o n . 11. Evaluation Is used to show a student how she i s pro-gressing. 12. Evaluation i s used to i d e n t i f y p a r t i c u l a r strengths of students. 13. Evaluation assesses p o t e n t i a l f o r further growth In stu-dents • Ii*. Evaluation i s used to locate i n d i v i d u a l learning needs of students. 71 Category 2 - Psychological E f f e c t s 15. Evaluation affects the psychological security of students. 16. Evaluation a f f e c t s the students* attitude toward the con-tent of the learning, experience. 17. Evaluation influences the motivation of students. 18. Evaluation influences adaptation to the learning environ-ment. 19. Evaluation modifies the self-image of students. 20. Evaluation influences the willingness of students to assume r e s p o n s i b i l i t y f o r t h e i r own learning. 21. Evaluation Influences the q u a l i t y of the instruotor-student r e l a t i o n s h i p . 22. Evaluation influences the peer relationships among students. 23. Evaluation a f f e c t s the climate for learning. 2lj.. Evaluation a f f e c t s the students* attitude toward subsequent learning.experience. 2$, Evaluation influences the students' attitude towards t h e i r profession. 26. Evaluation influences the frequency o f instructor-student i n t e r a c t i o n . 27. Evaluation a f f e c t s the communication between inst r u c t o r and student. 28. Evaluation a f f e c t s confidence of students i n a new learning s i t u a t i o n . Category 3 - Teaching 29. Evaluation i s used to i n d i v i d u a l i z e teaching. 30. Evaluation gives indications as to the effectiveness of teaching methods. 31. Evaluation gives indications as to the adequacy of l e a r n -ing experience. 7 2 Category 3 ~ Teaching (cont'd) 3 2 . Evaluation i s used to e f f e c t improvements i n teaching. 3 3 » Evaluation i s used to assess the value of alternate learning, experiences. 3I4.. Evaluation assesses the e f f i c i e n c y of teaching methods. 35. Evaluation i s used to provide an i n s t r u c t o r with feedback on her teaching. 3 6 . Evaluation provides data which guides modifications i n teaching. 3 7 . Evaluation i s used i n determining the s t a r t i n g point f o r i n s t r u e t i o n . 3 8 . Evaluation i s used by i n s t r u c t o r s i n self-evaluation. 3 9 . Evaluation i s used by ins t r u c t o r s to plan modifications i n teaching, s t y l e . lf.0. Evaluation i s used to compare effectiveness of teaching methods. I4.I. Evaluation i s used to investigate the s u i t a b i l i t y of teaching, materials. i+2. Evaluation i s used to analyze the sequence of learning experiences. Category k - Administration ij . 3 . Evaluation i s used i n the s e l e c t i o n of students. Ijif.. Evaluation i s used i n making, the decision to r e t a i n students i n a school. Evaluation i s used In making the decision to require students to withdraw from the school. 1+6. Evaluation i s used to guide curriculum r e v i s i o n . 2j.7» Evaluation i s used to assess the performance of an In-structor. J4.8. Evaluation i s used to analyze performance o f an i n s t r u c -tor. 73 Category k - Admlnistration (cont'd) 49 • Evaluation i s used to assess how well the school i s meeting i t s education objectives. 50. Evaluation indicates the strengths of a program. 51. Evaluation indicates the weaknesses of a program. f>2. Evaluation i s used to appraise needs f o r i n - s e r v i c e education. 53* Evaluation i s used by the d i r e c t o r i n deciding.upon retention of i n s t r u c t o r s . 54* Evaluation i s used by the d i r e c t o r i n deciding upon promotion of Instructors. 55* Evaluation helps administration understand the problems faced by students. 56. Evaluation Is used to control the q u a l i t y of graduating students. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0104652/manifest

Comment

Related Items