Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

British Columbia principals and the evaluation of teaching 1996

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
ubc_1996-0360.pdf [ 7.34MB ]
Metadata
JSON: 1.0055018.json
JSON-LD: 1.0055018+ld.json
RDF/XML (Pretty): 1.0055018.xml
RDF/JSON: 1.0055018+rdf.json
Turtle: 1.0055018+rdf-turtle.txt
N-Triples: 1.0055018+rdf-ntriples.txt
Citation
1.0055018.ris

Full Text

BRITISH COLUMBIA PRINCIPALS AND THE EVALUATION OF TEACHING by WILLIAM EDGAR B.A., The University of Leicester, U.K., 1981 P.G.C.E., North Staffordshire Polytechnic, U.K., 1983 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS in THE FACULTY OF EDUCATION (Centre for the Study of Curriculum and Instruction) We accept t h i s thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA June 1996 © William Edgar, 1996 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of The University of British Columbia Vancouver, Canada 2 * Date \ DE-6 (2/88) i i Abstract The purpose of t h i s study was to investigate the views of B r i t i s h Columbia p r i n c i p a l s with regard to the formal evaluation of teaching. Four major concepts were addressed a) the purpose of evaluation; b) the process of evaluation; c) the need for further p r i n c i p a l t r a i n i n g i n evaluation; and, d) obstacles to carrying out evaluation. The sex of p r i n c i p a l s and years of experience as a p r i n c i p a l were i d e n t i f i e d for further analysis because these variables are absent i n the l i t e r a t u r e on formal evaluation. The data consisted of relevant clauses from a l l 75 B r i t i s h Columbia school d i s t r i c t c o l l e c t i v e agreements and responses to a survey sent to the members of the B r i t i s h Columbia P r i n c i p a l s ' and V i c e - P r i n c i p a l s ' Association. The achieved sample i s 188 p r i n c i p a l s . The findings of t h i s study show the conduct of formal evaluation i s a r e s p o n s i b i l i t y w i l l i n g l y accepted by p r i n c i p a l s and that i t i s a function they consider they carry out well. C o l l e c t i v e agreements say l i t t l e about the purpose of evaluation. The majority of p r i n c i p a l s believe the most important purpose of evaluation i s teacher growth and development. Female p r i n c i p a l s indicate a stronger orientation towards teacher growth and development than males but t h i s difference may also be related to p r i n c i p a l s ' d i f f e r e n t experience l e v e l s . i i i R e l a t i v e l y few evaluations are ca r r i e d out and only a very small proportion re s u l t i n "less than s a t i s f a c t o r y " reports. Evaluations leading to "sat i s f a c t o r y " and "less than s a t i s f a c t o r y " reports are characterised i n very d i f f e r e n t terms by p r i n c i p a l s . Anecdotal responses support the assertion made i n the l i t e r a t u r e that p r i n c i p a l s believe they already know who t h e i r 'weak' teachers are before conducting an evaluation. B r i t i s h Columbia p r i n c i p a l s consider time as the primary obstacle to carrying out formal evaluation. Evaluation cycles and s i t e management r e s p o n s i b i l i t i e s are perceived as the major time consumers. Neither size of s t a f f nor percentage of teaching time were i d e n t i f i e d as s i g n i f i c a n t time ba r r i e r s by the respondents. P r i n c i p a l s do not label themselves as under-trained for the r e s p o n s i b i l i t y of formal evaluator of teaching. Moreover, master's speci a l t y and previous t r a i n i n g are not linked to further t r a i n i n g needs nor to how well p r i n c i p a l s believe they do evaluation. Three p o l i c y recommendations emerge from t h i s study: (1) to re-assess the role of p r i n c i p a l as evaluator i n the l i g h t of t h e i r wider r e s p o n s i b i l i t i e s ; (2) to consider extending the role of formal evaluator to educators other than school-based administrators; and (3) to re-assess the value of formal evaluation as currently practised. i v TABLE OF CONTENTS Abstract i i Table of Contents i v L i s t of Figures and Tables v i i Acknowledgements x Chapter I: Introduction 1 Chapter I I : A Review of the Literature 8 The Role of the P r i n c i p a l 9 Purpose of Evaluation 16 Process of Evaluation 20 Competence of Evaluator 23 Obstacles to Evaluation 28 Summary 31 Chapter I I I : Research Design and Methodology 35 The Framework for the Study 36 Purpose 36 Process 37 Training 37 Obstacles 38 Sex of P r i n c i p a l 38 Years of Experience as P r i n c i p a l 39 Sources of Data 40 Data C o l l e c t i o n Procedures 41 Data Analysis and Presentation 44 Design Limitations 46 Summary 55 Chapter IV: B r i t i s h Columbia School D i s t r i c t C o l l e c t i v e Agreements 56 The Process 56 The Four Phases of a Formal Evaluation of Teaching..58 Formal Evaluation Cycles 59 Evaluation C r i t e r i a 62 The Evaluator and the Evaluatee 63 The I n i t i a t i o n of a Formal Evaluation 64 Responsibility for Conducting a Formal Evaluation of Teaching 65 The Right of Appeal 65 Teacher Entitlement to Professional Development 66 Summary 66 Chapter V: Respondents' Backgrounds, Assignments, and Their Role as Evaluators of Teaching 68 Biographical Information 68 Current Assignment 70 The P r i n c i p a l as a Formal Evaluator of Teaching 72 Should P r i n c i p a l s Do Evaluation? What i s the Purpose? and How Well i s Evaluation Done? 72 In-service Training, Obstacles, and The Four Phases of Evaluation 74 Number of Evaluations and "Less Than Satisfactory" Reports 82 Additional Comments Made by Respondents 84 Summary 89 Chapter VI: Sex of P r i n c i p a l and Years of Experience as P r i n c i p a l 92 Sex of P r i n c i p a l 93 Years of Experience as P r i n c i p a l 98 Summary 103 Chapter VII: Purpose, Training, and Obstacles 105 Evaluation purpose 107 The Need for Further Training .....116 Obstacles to Evaluation 120 Summary 130 Chapter VIII:Discussion, Conclusions, and Recommendations 134 Discussion 134 Purpose 135 Process 140 Training 146 Obstacles. 149 Conclusion 154 Key Findings 157 Recommendations 158 Policy 158 Research 159 v i References 161 Appendices: A: Questionnaire 168 B: Evaluation Phases i n the C o l l e c t i v e Agreements 174 C: Permissable Data i n Evaluation F i n a l Report 175 D: Evaluation C r i t e r i a and Cycles 176 E: School D i s t r i c t Numbers, Names and Sizes .177 F: Sample Evaluation A r t i c l e From a B r i t i s h Columbia School D i s t r i c t C o l l e c t i v e Agreement 178 G: Summary of Response Frequencies 179 v i i L i s t of Figures Figures: 3.1 Comparative D i s t r i b u t i o n of B r i t i s h Columbia Pr i n c i p a l s and Questionnaire Respondents 51 L i s t of Tables Tables: 3.1 Respondents, B r i t i s h Columbia P r i n c i p a l s ' and Vi c e - P r i n c i p a l s ' Association P r i n c i p a l s , and A l l B r i t i s h Columbia Public School P r i n c i p a l s by Sex, Age, School Type,, and Staff Size... 48 3.2 Respondents and A l l B r i t i s h Columbia P r i n c i p a l s by School D i s t r i c t Size 52 3.3 Respondents and B r i t i s h Columbia P r i n c i p a l s by C r i t e r i a and Cycles 53 3.4 C o l l e c t i v e Agreement Wording for Evaluation Cycles 54 4.1 C o l l e c t i v e Agreement Wording on Evaluation Cycles 61 5.1 Respondent Biographical Data 69 5.2 Teaching Load, School Type, and Staff Size 71 5.3 Evaluation Purpose and Quality 73 5.4 Evaluation Training Attendance Since September 1988 74 5.5 Evaluation Training Points Since September 1988...75 5.6 F i r s t , Second, and Third Most Important Obstacles to the Conduct of the Formal Evaluation of Teaching 78 5.7 Factors Present i n Evaluations Leading to "Satisfactory" and "Less Than Satisfactory" Reports 81 v i i i 5.8 Evaluations Conducted and "Less Than Satisfactory" Reports Written Since September 1988 83 5.9 Anecdotal Responses 85 6.1 Sex of P r i n c i p a l by Evaluation Quality, Age, and Master's Specialty 93 6.2 Sex of P r i n c i p a l by School D i s t r i c t Size, School Type, Staff Size, and Teaching Load 95 6.3 Sex of P r i n c i p a l by Evaluation Training Since September 1988 96 6.4 Sex of P r i n c i p a l by Evaluation Training Points Since September 1988 97 6.5 P r i n c i p a l Experience by Evaluation Quality, Age, and Master's Specialty 99 6.6 P r i n c i p a l Experience by School D i s t r i c t Size, School Type, Staff Size, and Teaching Load 100 6.7 P r i n c i p a l Experience by Evaluation Training Since September 1988 102 6.8 P r i n c i p a l Experience by Evaluation Training Points Since September 1988 103 7.1 Sex of P r i n c i p a l by Evaluation Purpose and "Less Than Satisfactory" Reports 109 7.2 P r i n c i p a l Experience by Evaluation Purpose and "Less Than Satisfactory" Reports 112 7.3 P r i n c i p a l s Categorised on the Basis of Evaluation C r i t e r i a by Evaluation Purpose and "Less Than Satisfactory" Reports 114 7.4 Sex of P r i n c i p a l by Years of Experience as P r i n c i p a l 115 7.5 Sex of P r i n c i p a l and Need for Further Training i n Evaluation 117 7.6 P r i n c i p a l Experience and Need for Further Training i n Evaluations Leading to a " Sat i s factory " Report 118 ix 7.7 P r i n c i p a l Experience and Need for Further Training i n Evaluations Leading to a "Less Than Satisfactory" Report 119 7.8 Time Obstacle Statements 124 7.9 Time as an Obstacle and Sex of P r i n c i p a l 126 7.10 Time as an Obstacle and P r i n c i p a l Experience 127 7.11 Time as an Obstacle and P r i n c i p a l s Categorised on the Basis of Evaluation Cycles 128 Acknowledgement s There are a number of individuals and organisations I should l i k e to thank for the help they have given me i n the production of t h i s thesis and the completion of my MA i n Education at UBC. F i r s t l y , without the interest and p r a c t i c a l support offered to me by the executive o f f i c e r s and s t a f f of the B r i t i s h Columbia P r i n c i p a l s ' and V i c e - P r i n c i p a l s ' Association, t h i s thesis would not have been possible. I am also indebted to the 11 people who p i l o t e d the questionnaire and the 267 members of the Association who p a r t i c i p a t e d i n the study and thank them for the kind words and suggestions made i n the course of doing so. Thanks are also due to the s t a f f at the School Finance and Data Management Branch at the BC Ministry of Education, who met a l l my requests for information promptly and c a r e f u l l y . My f i n a l thanks for those ' o f f i c i a l l y ' involved i n t h i s work, go to Arleigh Reichl, i n Education Computer Studies, who assisted me greatly with the s t a t i s t i c a l analysis, my Thesis Committee, Don Fisher, Frank Echols, and Graham Kelsey, and the external reader, Dan Brown. I should also l i k e to extend some special thanks to friends, both i n Vancouver and elsewhere, and i n p a r t i c u l a r to the Lythgoe Family. I am very grateful to June, Len, Shannon and my 'buddy on the block'. Garnet, for t h e i r friendship and for having contributed enormously to making the past two years a pleasurable period for me. My Canadian cousins, the Olive r Family, also helped me to s e t t l e i n and make the t r a n s i t i o n from l i f e i n Great B r i t a i n to l i f e i n Canada so r e l a t i v e l y straightforward. Last, but by no means least, I give a very special thank you to my Mum, Eva Mary Edgar, and my Uncle and Aunt, David and Nola Edgar, for t h e i r moral and p r a c t i c a l support i n t h i s venture on another continent. Without them the whole enterprise would have been very much more d i f f i c u l t . 1 CHAPTER I Introduction In North America the role of formal evaluator of teaching i s generally c a r r i e d out by the school p r i n c i p a l , although superintendents, t h e i r assistants, d i s t r i c t p r i n c i p a l s and, i n some cases, school v i c e - p r i n c i p a l s also perform t h i s r o l e . In B r i t i s h Columbia, formal evaluation i s governed by statute and the provisions l a i d down i n the school d i s t r i c t c o l l e c t i v e agreements drawn up between the lo c a l boards of school trustees and the l o c a l teacher unions a f f i l i a t e d to the B r i t i s h Columbia Teachers' Federation (BCTF). Clause 59 of the Teaching Profession Act 1987 (Province of B r i t i s h Columbia, 1987a), Section 121 (1) reads: "A person appointed as a p r i n c i p a l or v i c e - p r i n c i p a l i n a public school s h a l l , subject to t h i s Act and the regulations:...(c) evaluate teachers under his supervision and report to the board as to his evaluation." The current personnel procedures i n most B r i t i s h Columbia school d i s t r i c t s date back to 1988, following the passing of B i l l s 19, as the Industrial Relations Reform Act (Province of B r i t i s h Columbia, 1987), and 20, as the Teaching Profession Act (Province of B r i t i s h Columbia, 1987a), i n 1987. An important feature of B i l l 19 was to change the former Labour Code of B r i t i s h Columbia "to give more weight to the interests of employers" (Kelsey, Lupini, 2 & Clinton, 1995, p.6). B i l l 20 provided teachers with the option of remaining an 'association' outside the provisions of the new Industrial Relations Reform Act but without the right to s t r i k e or, to become a 'union' within the provisions of the Act and with the right to s t r i k e . Teacher associations i n a l l seventy-five school d i s t r i c t s i n the Province voted to become unions and each subsequently voted to be a f f i l i a t e d to the BCTF. A major consequence of the Teaching Profession Act was the clear d i s t i n c t i o n drawn between teachers, who were now members of d i s t r i c t unions, and administrators who were disallowed union membership. This quasi 'union/management' d i s t i n c t i o n and the r e s p o n s i b i l i t y for evaluation being mainly that of school administrators, highlights the importance of the p r i n c i p a l i n any study of formal evaluation. My interest i n t h i s subject dates back to the mid to late 1980s when teacher appraisal received increasing attention i n England and Wales following the introduction of a series of major government educational i n i t i a t i v e s . These i n i t i a t i v e s addressed the public examination system, the curriculum, and school governance. A process of formal teacher appraisal was intended to address the effectiveness of teaching. The r e s u l t i n g government regulations (HMSO, 1991) and accompanying c i r c u l a r , set out the p r i n c i p l e s which were to be followed by Local Education Authorities (LEAs). In turn, the LEAs were to formulate teacher appraisal guidelines for ind i v i d u a l schools to follow when they developed t h e i r own ' i n s t i t u t i o n s p e c i f i c ' appraisal process. Some important features of these p r i n c i p l e s and guidelines were the emphases on a) professional development; b) career planning; and, c) a supportive, non-threatening process. In t h i s context, as a senior middle manager i n a school with 1,650 pupils on r o l l and 90 s t a f f , I became involved i n helping to construct an appraisal system that would meet government and LEA requirements; and that would be viewed favourably by school s t a f f . Meeting the second of these two c r i t e r i a was p a r t i c u l a r l y important because the perceptions of, and attitudes towards, teacher appraisal by teachers, heavily influence how well the goals of appraisal can be s a t i s f a c t o r i l y attained (Darling-Hammond, 1986: Sergiovanni, 1977, 1991). Primarily, the goals were to maintain high standards of teaching where they existed, and improve standards where necessary. I was able to form a view of appraisal not only i n my capacity as a member of the School Appraisal Committee but also as the Humanities Co-ordinator and Head of the Social Science Department. This p o s i t i o n gave me a dual perspective as one who appraised the s t a f f within my 4 department and as one who, i n turn, was appraised by the head and deputy head teachers. This led me to the conclusion that many teachers perceive appraisal as merely a middle and senior management device for i d e n t i f y i n g poor teachers. As Schonberger (1986) states with reference to Reavis (1978): "national surveys of teachers have tended to show teachers as d i s t r u s t f u l of the supervisory process as t r a d i t i o n a l l y practiced." Furthermore, "Teachers have come to regard supervision with anxiety, fear, suspicion, and resistance" (p.249). However, I also took the view that "teachers, l i k e any other group of professionals, must accept the need to be appraised. But i t must be organised so that the vast majority who do a good job are encouraged" (Edgar, 1991, p.24). In other words, teacher attitudes are important but so i s the q u a l i t y and orientation of the evaluation. It i s important to d i s t i n g u i s h between informal and formal evaluation. The former i s i m p l i c i t l y recognised as occurring i n day-to-day professional i n t e r a c t i o n . The l a t t e r e x p l i c i t l y i d e n t i f i e s the process to be followed and, of c r u c i a l importance, the outcome i s recorded. Therefore, while the f i r s t i s often 'taken as read 1, the second tends to be associated with categorisation as good or bad, and with f i l e d information which can be used and referred to at some time i n the future. The formal evaluation of teaching, governed i n B r i t i s h Columbia by P r o v i n c i a l statute and school d i s t r i c t c o l l e c t i v e agreements, i s the subject of interest i n t h i s t h e s i s . The effectiveness of formal evaluation, as stated above, i s linked to teacher perceptions and attitudes but also depends heavily on the evaluator. Examining formal evaluation from the perspective of the evaluator, within an ends-means framework, highlights four important concepts: a) the reason or purpose p r i n c i p a l s have for conducting evaluations (other than t h e i r contractual o b l i g a t i o n s ) ; b) the evaluation process p r i n c i p a l s have to work with; c) the l e v e l of professional preparation and t r a i n i n g of p r i n c i p a l s ; and, d) the obstacles which may prevent p r i n c i p a l s from f u l f i l l i n g t h i s role to t h e i r desired standard. These four concepts are, of course, highly i n t e r r e l a t e d . While purpose very c l e a r l y relates to the ends of formal evaluation, the process can also convey purposes which may be quite d i f f e r e n t from those formally stated. The extent to which a p a r t i c u l a r factor i n the evaluation process may be seen as an obstacle i s l i k e l y to depend, i n part, on the purpose the evaluator has i n mind and how far the evaluator has been trained for the r o l e . Purpose, assumes p a r t i c u l a r importance because of the widespread teacher d i s t r u s t referred to e a r l i e r . Purpose 6 also has a cle a r p o l i t i c a l dimension. What happens i n public schools i s legitimately part of the public p o l i t i c a l arena. School boards, amongst other bodies, are held to be p u b l i c l y and p o l i t i c a l l y accountable for the perceived standard of education. School p r i n c i p a l s now occupy the ground which l i e s between the public ( i n the form of parents and the school board) and professional educator colleagues who are tr y i n g to provide a service to that public. Therefore, while p r i n c i p a l s have a role as educational leaders and i n s t r u c t i o n a l managers and may well wish to promote professional growth and development, they also have a r e s p o n s i b i l i t y to ensure accountability for the q u a l i t y of the service provided to the public. This, i n turn, may lead to a sense of being 'caught' between two apparently contradictory philosophies. The complexity of the environment i n which p r i n c i p a l s now operate highlights the need to know about the qu a l i t y of professional preparation p r i n c i p a l s receive. A greater understanding of the perceptions of p r i n c i p a l s as evaluators, may also lead to a more s p e c i f i c understanding of the impediments to carrying out the evaluator role. Teaching and learning are the raison d'etre for schools and formal evaluation i s the prescribed means to assess the "classroom s i t u a t i o n " . The success or f a i l u r e of evaluation depends heavily on the objectives p r i n c i p a l s have for 7 evaluating, t h e i r l e v e l of competence i n evaluating, and how far they are able to carry out evaluation unhindered. As Sergiovanni (1991) asserts: "The nature and c h a r a c t e r i s t i c s of evaluation knowledge are determined by the way i n which the supervisor understands them... To understand an evaluation, therefore, one must understand the evaluator" (p.293). Therefore, the purpose of t h i s study i s to e l i c i t the views of B r i t i s h Columbia p r i n c i p a l s with regard to the formal evaluation of teaching i n r e l a t i o n to the four concepts of purpose, process, t r a i n i n g , and obstacles. 8 CHAPTER II A Review of the Literature The l i t e r a t u r e on the formal evaluation of teaching i s extensive but can be organised around four key concepts. These are a) the purpose of evaluation; b) the process of evaluation; c) the competence of the evaluator; and, d) the obstacles to conducting evaluation. These concepts are not mutually exclusive but do provide useful f o c i for four of the sections of t h i s chapter. The concept of purpose relates to the apparently competing needs of the school, as an educational organisation within a wider p o l i t i c a l context, and the needs of teachers. The discussion about process examines the r e l a t i v e positions of p r i n c i p a l s and teachers and how far the purposes of evaluation are met. Competence considers the professional expertise of p r i n c i p a l s as evaluators, including t r a i n i n g , and "obstacles" addresses how time may hinder a p r i n c i p a l ' s capacity to s a t i s f a c t o r i l y carry out the formal evaluation of teaching. The f i r s t section of t h i s chapter makes reference to the wider role of the p r i n c i p a l . The evaluation of teaching i s only one part of a much broader set of p r i n c i p a l r e s p o n s i b i l i t i e s . In p a r t i c u l a r , t h i s opening section includes the issues of i n s t r u c t i o n a l leadership and p r i n c i p a l s ' workload. It also addresses the impact of p r i n c i p a l s ' experience and the sex of p r i n c i p a l s on educational administration. The experience p r i n c i p a l s have as p r i n c i p a l s i s examined because, due to the complexity of the role, 'mastery' of the role of p r i n c i p a l i s l i k e l y to require p r a c t i c e . The sex of p r i n c i p a l s has been addressed because a section of the l i t e r a t u r e on educational administration draws a d i s t i n c t i o n between the professional behaviour of male and female educational managers. The chapter concludes with a summary. The Role of the P r i n c i p a l In t h i s decade, much has been written about the leadership role of the p r i n c i p a l (Rossow, 1990; Sergiovanni, 1991; Sharp & Walter, 1994; Sybouts & Wendel, 1994; Ubben & Hughes, 1992; et a l . ) . This l i t e r a t u r e often refers to the 'effective-schools research' conducted i n the 1970s and 1980s and places considerable emphasis on the importance of the p r i n c i p a l i n bringing about school success. The extent and complexity of the role of p r i n c i p a l i s considerable. Sharp and Walter (1994) i l l u s t r a t e t h i s well as they systematically work through the role of the American p r i n c i p a l 'as school manager1, from "school finance", through the "school f a c i l i t y " , "public r e l a t i o n s " , "personnel r o l e " , "school law", "food services", "student d i s c i p l i n e " , and "pupil transportation", to " p r i n c i p a l as 10 master schedule maker" ( p . v i i ) . They conclude i n t h e i r introduction that "The p r i n c i p a l , whether elementary or secondary, i s the single most important person to a school's success" (p.1). Beck and Murphy (1993) present a series of metaphors that have been associated with the p r i n c i p a l s h i p since the 1920s. Their l i s t of metaphors for the 1990s i s : P r i n c i p a l as leader; as servant; as organisational a r c h i t e c t ; as s o c i a l a r c h i t e c t ; as educator; as moral agent; and, as person i n the community. Beck and Murphy consider the generic role of the manager i n the p o s t - i n d u s t r i a l era and quote Gerding and Serenhuijseur, c i t e d i n Beare (1989, p.19) when they suggest that the 'new manager' w i l l be "a customized version of Indiana Jones: proactive; entrepreneurial; communicating i n various languages; able to i n s p i r e , motivate and persuade subordinates, superiors, colleagues and outside constituents." (p.190). However, Blumberg and Greenfield (1986) remind us that p r i n c i p a l s are people and therefore are bound to range i n effectiveness. They assert that "very few, i f any, can possibly l i v e up to the 'White Knight' image that we hold so dear" (p.232). A l i m i t a t i o n of the l i t e r a t u r e on the role of the p r i n c i p a l i s that i t treats the occupants of t h i s o f f i c e as a r e l a t i v e l y homogeneous group, save for a d i s t i n c t i o n based on 'effective' and 'less e f f e c t i v e ' p r a c t i c e . Alder et a l . (1993, p.4) refer to t h i s tendency and, i n p a r t i c u l a r , the accusation made by Shakeshaft (1987), that the l i t e r a t u r e on school management i s "androcentric" because i t l a r g e l y f a i l s to d i s t i n g u i s h between male and female p r i n c i p a l s . Shakeshaft (1989) i s also quoted i n Pigford and Tonnsen (1993, p.2) as asserting that "the absence of accurate data on women administrators i s by design and i s evidence of a 'conspiracy of si l e n c e ' " . While the l i t e r a t u r e does not refer d i r e c t l y to a d i s t i n c t i o n between men and women p r i n c i p a l s i n the realm of formal evaluation, a r e l a t i v e l y small but int e r e s t i n g l i t e r a t u r e speaks to the issue of gender differences i n school administration generally. A section of t h i s l i t e r a t u r e addresses the kind of data that Shakeshaft suggests i s suppressed. For example, Gross and Trask (1976) and Blumberg and Greenfield (1986) highlight the longer periods women spend as classroom teachers before being promoted into educational administration and that most often women achieve p r i n c i p a l s h i p s i n elementary rather than secondary schools. The l i t e r a t u r e suggests that once women obtain a p r i n c i p a l s h i p they adopt a more c o l l e g i a l and caring approach to the function of school leadership (Alder, Laney & Packer, 1993; Ozga, 1993; Regan & Brooks, 1995; Tibbetts, 1980). 12 Tibbetts (1980), c i t i n g Clement, et a l . (1977), Grambs (1976), and Gross & Trask (1976) c i t e d i n Grambs (1976), makes s p e c i f i c reference to the performance of teachers when she suggests "Data indicate that, on the average, the ca l i b e r of performance of... teachers i n schools administered by women i s found to be of a higher q u a l i t y than i n schools managed by men" (p.176). She goes on to assert, c i t i n g F i s h e l and Pottker (1975), Frasher and Frasher (1979), Grobman and Hines (1956), and Gross and Trask (1964) i n Meskin (1974), that "Women p r i n c i p a l s induce more professional performances and productive behavior from teachers who consequently use more desirable practices, r e s u l t i n g i n higher ratings for teacher performance i n schools with women p r i n c i p a l s " (p.177). Ozga (1993) relates school management to the l i t e r a t u r e on leadership and motivation generally when asserting "leadership i s t y p i c a l l y authoritarian, charismatic or entrepreneurial; motivation i s t y p i c a l l y competitive, and linked to success defined as winning, as beating down the opposition" (p.10). She continues: The beginnings of research on women's management and leadership styles suggest that there are differences from t h i s conventional model (Neville 1988). Women's leadership s t y l e i s less h i e r a r c h i c a l and more democratic. Women, for example, run more c l o s e l y knit schools than do men, and communicate better with teachers, (p.11) 13 Expanding on the description that Ozga provides, Regan and Brooks (1995) make reference to f i v e "Feminist Attributes" (p.25), one of which i s courage. While they acknowledge the caring and collaborative a t t r i b u t e s as well, Regan and Brooks provide an example,of what they mean by courage when they suggest that women "exercise courage i n support of the organization. They take the high road and encourage everyone i n the organization to achieve the high road with them" (p.30). Ozga (1993), c i t i n g B a l l (1987), i d e n t i f i e s two styl e s of management termed 'managerial' and 'interpersonal' and suggests that the f i r s t of these tends to be exhibited more by men and the second more by women (p.31). Ozga (1993), r e f e r r i n g to the 'managerial' s t y l e , quotes B a l l (1987, p.97) when stating " i n theory at least, the roles and r e s p o n s i b i l i t i e s of s t a f f are r e l a t i v e l y fixed and p u b l i c l y recorded" (p.31). The 'interpersonal' s t y l e on the other hand i s characterised by a reliance on personal relationships and face-to-face contact to f u l f i l the rol e . These differences i n st y l e may have consequences for the way formal evaluation i s conducted because they are linked to the way p r i n c i p a l s interact with teachers. For example, these sty l e s may resul t i n rather d i f f e r e n t approaches to evaluation i f male p r i n c i p a l s are more concerned with bureaucratic functions and less concerned with the human 14 context than are female p r i n c i p a l s . In other words, a more 'managerial' approach may place greater importance on the needs of the organisation, whereas an 'interpersonal' s t y l e may give p r i o r i t y to the needs of the employee. A second way of distinguishing between p r i n c i p a l s i s on the basis of experience as a p r i n c i p a l . However, the l i t e r a t u r e has very l i t t l e to say about the role of administrative experience i n determining p r i n c i p a l behaviours. P r i n c i p a l experience i s referred to from time to time (Blumberg & Greenfield, 1986; Rossow, 1990; Ubben & Hughes, 1992; Webster, 1994;) but almost always i n passing or i n terms that take i t s p o s i t i v e 'developmental' e f f e c t for granted. For example, Blumberg and Greenfield (1986), refer to the p o s s i b i l i t y "of better-prepared 'rookie' p r i n c i p a l s who, through p r a c t i c i n g t h e i r c r a f t , can become more s k i l l e d and more e f f e c t i v e over time" (p.239). Rossow (1990) asserts that "The p r i n c i p a l ' s previous experiences w i l l influence his decisions and a c t i v i t i e s " (p.42), but i s unable to give more than an i n t u i t i v e rationale for doing so. Morris et a l . (1984) refer extensively to "P r i n c i p a l i n g and i t s e f f e c t on the p r i n c i p a l " (p.181), but do not i d e n t i f y any research that seeks to i d e n t i f y differences i n the behaviours of p r i n c i p a l s with d i f f e r e n t l e v e l s of experience. Webster (1994) provides a f i n a l example when he states that "It i s possible for experienced p r i n c i p a l s to 15 l i s t . . . a n i n f i n i t e number of s p e c i f i c s k i l l s required i n the p r i n c i p a l s h i p " (p.41), but at no point does he address the issue of experience i n d e t a i l . A possible exception to t h i s truncated or non-existent reference to the e f f e c t s of experience may be provided by Sergiovanni (1991). He refers to Hogben's (1981) work based on Freidson's (1972) examination of the medical profession. According to Sergiovanni, Hogben i d e n t i f i e s four major differences between medical professionals and medical researchers and theoreticians: "Professionals aim at action, not at knowledge...professionals need to believe i n what they are doing as they practice...professionals [rely] on t h e i r own firsthand experiences...the p r a c t i t i o n e r i s very prone to emphasize the idea of indeterminancy or uncertainty" (1991, p.291/292, author's emphases). Sergiovanni maintains that these differences can also be applied to the teaching profession. Even though t h i s issue i s raised i n order to promote the need for p r i n c i p a l s as evaluators to accommodate to the " c l i n i c a l mind" of teachers, i t also i d e n t i f i e s c h a r a c t e r i s t i c s which may apply to p r i n c i p a l s themselves. One c h a r a c t e r i s t i c above, i s that professionals "aim at action" and: i n t h i s process... seek "useful" rather than " i d e a l " knowledge...By taking action, they seek to make sense of the problems they face and to create knowledge i n use. They r e l y heavily on informed i n t u i t i o n to f i l l i n 16 the gaps between what i s known and unknown, (p.291, author 1s emphases) This c l e a r l y highlights the role of experience and Sergiovanni suggests that the 'creation of knowledge i n use' i s necessary because e x i s t i n g theory i s only helpful i n addressing a small minority of the problems professional educators face. Indeed, another c h a r a c t e r i s t i c ascribed to professionals i s a heavy reliance on t h e i r own firsthand experiences and Sergiovanni asserts that "They trust t h e i r own accumulated experiences i n making decisions about practice [ s i c ] than they do abstract p r i n c i p l e s " (p.292). These c h a r a c t e r i s t i c s not only highlight the importance of experience to the behaviours of p r i n c i p a l s they also raise a question about how far educators are amenable to 'external' t r a i n i n g . Purpose of Evaluation The purpose of evaluation has sign i f i c a n c e because the stated objectives are l i k e l y to have an e f f e c t on the perceptions of those who are being evaluated (Airasian, 1993). For example, the stated objectives w i l l provide some ind i c a t i o n of the degree to which the evaluatee i s to be judged, categorised, and given constructive feedback. The purposes presented i n the l i t e r a t u r e tend to vary s l i g h t l y but Harris and Monk (1992) capture the essence of most sources when they quote a 1988 Education Research Service 17 report that stated "teacher evaluation systems...must serve three major purposes: (1) to ensure that a l l teachers are at least minimally competent; (2) to improve further the performance of competent teachers; and (3) to i d e n t i f y and recognize the performance of outstanding teachers" (p.152). Poster and Poster (1993) i d e n t i f y two purposes as.those of 'performance review' and 'staff development review', which they define as follows: Performance review (or appraisal) focuses on the setting of achievable, often r e l a t i v e l y short-term goals. The review gives feedback: on task c l a r i f i c a t i o n through consideration of the employees' understanding of t h e i r objectives set against those of the organisation; and on t r a i n i n g needs as indicated either by shortcomings i n performance or by the demonstration of potential for higher lev e l s of performance. Staff development review (or appraisal) focuses on improving the a b i l i t y of employees to perform t h e i r present or prospective roles, through the i d e n t i f i c a t i o n of personal development needs and the provision of subsequent t r a i n i n g or self-development opportunities. In sum, the former i s concerned with the task, the l a t t e r with the i n d i v i d u a l , ( p . l , authors' emphases) While the d i s t i n c t i o n between these two purposes i s defined as concerns over task and over the i n d i v i d u a l , a somewhat more subtle but related difference i s raised which places emphasis on the needs of the organisation or on the needs of the employee. In r e f e r r i n g to a d i s t i n c t i o n between 'bureaucratic' and 'professional' evaluation, Housego (1989) states that the former " i s meant to serve the needs of the organization for monitoring how adequate the teacher's performance i s " , while the l a t t e r " i s meant to help teachers meet t h e i r needs for support and guidance relevant to improving classroom practice" (p.197). Poster and Poster suggest that t h i s i s a f a l s e dichotomy because the e f f i c i e n t working of an organisation "depends both on the delivery system and on those who deli v e r i t " (p.1-2). However, the teachers may well define evaluation as generally seeking to promote the interests of the organisation at the expense of the i n d i v i d u a l : Instead of encouraging teachers to take control of t h e i r own s t r i v i n g and growth, the externally controlled educational objectives, teaching materials, assignments, and schedules have produced a f e e l i n g of dependence, insecurity, powerlessness, and subservience among teachers (Schonberger, 1986, p.249). In other words, while i t i s important to understand the intended purposes of evaluation, there may well be a difference between what i s stated and what i s perceived. A number of sources make reference to t h i s difference (Allston, Rymhs & Shultz, 1993; Christensen, 1986; Darling- Hammond, 1986; Peterson, 1986; Schonberger, 1986) and they generally present negative attitudes on the part of teachers towards current prac t i c e . These negative attitudes have been linked primarily to the fact that evaluation i s viewed as judgmental, p a r t i c u l a r l y with regard to how far teachers are f u l f i l l i n g t h e i r contractual obligations or performing to a s a t i s f a c t o r y standard (Black, 1993). These teacher perceptions raise a question about the role of the evaluator 19 whose approach w i l l be influenced by the purposes he or she has for formally evaluating. The l i t e r a t u r e suggests that the somewhat managerial and judgmental approach taken by administrators towards evaluation needs to change (Haefele, 1992; Rooney, 1993; Starratt, 1993; Storey & Housego, 1980; Wood, 1992; et al) and advocates a more ' c o l l e g i a l ' form of evaluation. The most negative views are directed towards the c o n t r o l l i n g function ascribed by some teachers to the evaluation process. Evaluation can be characterised by the evaluated as a means for the senior management i n schools to demonstrate t h e i r power and ultimate control over the 'ordinary' teacher, rather than attempting to improve the q u a l i t y of classroom prac t i c e . Referring to studies of instruments used by administrators i n the American public school system, Peterson points out that: these instruments included...compliance with p o l i c i e s ; personal a t t r i b u t e s such as appearance, health, attendance and judgment [ s i c ] ; e x t r a c u r r i c u l a r duties such as record keeping...; and f i n a l l y a few items on the teaching process...The d i s t r e s s i n g discovery of t h i s study i s that as l i t t l e as 5% of the items on one instrument i n the sample focused on teaching. (1985, p.40) Furthermore, Schonberger (1986), highlights what he describes as "pseudo-scientific management practices favored by administrators i n the interest of increasing control, accountability, and e f f i c i e n c y " (p.249). He continues by c i t i n g Withall and Wood (1979), who assert that a number of 20 factors have led to feelings of fear and anxiety i n r e l a t i o n to evaluation, one being "the manner i n which supervisors have tended to project an image of su p e r i o r i t y and omniscience i n i d e n t i f y i n g the strengths and weaknesses of a teacher's performance" (cited i n Schonberger 1986, p.249). Process of Evaluation The evaluation l i t e r a t u r e draws a d i s t i n c t i o n between formative and summative evaluation: In evaluating a teacher's performance, summative evaluation suggests a statement of worth. A judgment i s made about the qu a l i t y of one's teaching...Formative evaluation i s concerned less with judging and rating the teacher than with providing information that helps improve teacher performance. (Sergiovanni, 1977, p.372, i n Schonberger, 1986, p.249) Sergiovanni though, i s once again attaching importance to much more than simply the stated intentions of evaluation and looks at how that purpose i s transmitted through the process of evaluation. The evaluation of teaching tends to be summative rather than formative and tends not to be viewed by teachers i n a p o s i t i v e developmental way. Evaluation can also be seen, by both teachers and p r i n c i p a l s a l i k e , as a r e l a t i v e l y inconsequential 'chore' that has to be performed p e r i o d i c a l l y but which produces l i t t l e of any real value. According to Darling-Hammond (1986) "Teacher evaluation can be u t t e r l y unimportant. In many school d i s t r i c t s i t i s a perfunctory bureaucratic requirement that 21 y i e l d s l i t t l e help for teachers and l i t t l e information on which a school d i s t r i c t can base decisions" (p.531). Some of the l i t e r a t u r e highlights the importance of the po s i t i o n or status of evaluatees i n r e l a t i o n to t h e i r evaluators and concludes that peer evaluation would reduce negative attitudes towards the process. For example, Darling-Hammond (1986) i d e n t i f i e s the need for non- threatening procedures as one of the p r i n c i p l e j u s t i f i c a t i o n s for employing peer evaluation. Her studies seem to indicate that such systems can produce higher l e v e l s of more p o s i t i v e attitudes amongst evaluatees. She also draws attention to the differences amongst teachers and, when r e f e r r i n g to evaluation designs, asserts: Elements that are intended to heighten r e l i a b i l i t y tend to reduce the a b i l i t y of the system to help i n d i v i d u a l teachers improve, since the uniformity of c r i t e r i a and th e i r application...necessarily reduce the f l e x i b i l i t y that would be needed to make evaluation useful to indi v i d u a l teachers with i n d i v i d u a l needs, (p.546) Some research has highlighted the heterogeneous nature of teachers and investigated the orig i n s of phenomena such as powerlessness (Darnell, 1993; and Lusty, 1991). Darnell investigated the attitudes of teachers towards the Texas Teacher Appraisal System i n r e l a t i o n to the self-concept of teachers and t h e i r p o s i t i o n on the 'Career Ladder'. She found general discontent on the part of teachers towards the process and suggests that status within the school may have some influence i n determining such attitudes: Not holding the highest status on the Career Ladder could tend to make Career Ladder II teachers f e e l less adequate than t h e i r peers on Career Ladder I I I . Overall attitudes for the appraiser are p o s i t i v e , but teachers on Career Ladder II indicate a less p o s i t i v e attitude toward the appraiser than do Career Ladder I or III teachers. (1993, Abstract) The potential s i g n i f i c a n c e of the status variable i s supported by Lusty who states that "teachers' opinions on teacher appraisal are c l o s e l y related to t h e i r p o s i t i o n and status within the school" (Abstract). Christensen (1986) refers to three d i f f e r e n t orientations for working with teachers: " d i r e c t i v e , c ollaborative and nondirective", and concludes that these orientations have consequences for the evaluation of teaching: "Research has found that d i f f e r e n t types of teachers need d i f f e r e n t types of supervision... The supervision, therefore, must be oriented to the teacher" (p.23). Sergiovanni (1991), l i n k i n g the process of evaluation to the purpose once more, asserts that "No supervisory system based on a single purpose can succeed over time" (p.284). Antosz (1990), i n her study of teacher evaluation provisions i n selected B r i t i s h Columbia school d i s t r i c t c o l l e c t i v e agreements, concludes that: few B r i t i s h Columbia school d i s t r i c t s have teacher evaluation systems that promote teacher growth and i n s t r u c t i o n a l improvement. Therefore, the majority f a l l short of the l i t e r a t u r e ' s recommended teacher evaluation practices. A l l school d i s t r i c t s studied have summative teacher evaluation systems.(p.116) 23 This finding echoes the concern that the evaluation process i s one dimensional and the purpose transmitted to teachers i s one of accountability. It i s also l i k e l y that many pr i n c i p a l s who f e e l strongly orientated towards the growth and development purpose of evaluation, f i n d themselves working within a system geared more to accountability. Competence of Evaluator Haefele (1992), with reference to what he c a l l s the current " d e f i c i t model" of teacher evaluation, focuses on the role of the p r i n c i p a l i n the process and characterises t h i s role as the "deficient evaluator" (p.337). According to Haefele, recent research (Darling-Hammond, Wise, & Pease, 1983; Huddle, 1985; Lower, 1987; Medley, Coker, & Soar, 1987) indicates that "In general, evaluations performed by p r i n c i p a l s have been found to be poor and imprecise" (p.338). Furthermore, Haefele (1992) c i t e s Scriven (1987) who has highlighted "the questionable a b i l i t y of the p r i n c i p a l to evaluate teachers of subject areas foreign to the p r i n c i p a l ' s background" ( i n Haefele, p.338). Haefele, r e f e r r i n g to other sources (Bridges, 1986; Cangelosi, 1991; Lower, 1987; VanScriver, 1990), concludes that "pri n c i p a l s do not receive much, i f any, rigorous t r a i n i n g i n the rating 24 of teaching performance and other evaluation related s k i l l s " (p.338). Bailey (1984) asserts that "The evaluation of teachers requires incredible amounts of s k i l l and time. Therefore, unfortunately, many administrators f i n d teacher evaluation to be a highly f r u s t r a t i n g endeavour" (p.19). Townsend (1987), r e f e r r i n g to Hunter (1985), warns that school systems which do not provide adequate evaluation t r a i n i n g to school administrators "can expect to encounter serious d i f f i c u l t i e s " (p.26). However, one research study (Page & Page, 1985) suggests that p r i n c i p a l s rate very highly the preparation they receive for "observation of i n s t r u c t i o n " and "evaluation of teachers". At the same time, p r i n c i p a l s i n t h i s study rated both these a c t i v i t i e s as very time consuming and the evaluation of teachers as " d i f f i c u l t " . Wood (1992), from the perspective of n a t u r a l i s t i c inquiry and drawing on sources i n Everhart (1988) and Guba and Lincoln (1981), suggests the deficiency i n many of the evaluations conducted by p r i n c i p a l s i s that observations are not considered i n context and the procedures adopted by many school d i s t r i c t s have "underemphasized the role of the p r i n c i p a l as the 'instrument' of evaluation"(p.52). The importance of context i s also supported by Storey and Housego (1980) and Housego (1989). 25 Under the heading of "Seeing i s Believing...Or Is It?", Wood (1992) asserts that "Administrators and others tend to see what they are prepared to see, and what they already believe" and thus "Indeed, believing i s seeing" (p.53, author's emphasis). Therefore, due to the increasing complexity of the role of p r i n c i p a l and the kinds of pressure on time that Haefele and others refer to above, p r i n c i p a l s have developed the a b i l i t y to operate on, what Wood c a l l s , "automatic p i l o t " (p.56). The issue of pre-judgment i s important for making decisions about who to select for evaluation. A preconception of poor teacher performance would probably lead a p r i n c i p a l to select a teacher for evaluation. However, i f such a preconception e x i s t s , the outcome of the evaluation i s somewhat pre-determined and perhaps flawed as a r e s u l t . A study by Morrow et a l . (1985) may provide a useful guide to the kinds of indicators or ' c r i t e r i a ' p r i n c i p a l s employ when a r r i v i n g at a general assessment of teaching competence. The purpose of the study was to survey the perceptions of p r i n c i p a l s as to the l e v e l of d i f f i c u l t y experienced by th e i r s t a f f with regard to ten "common in s t r u c t i o n a l problems". The ten problems were i d e n t i f i e d from "An extensive l i t e r a t u r e review" (p.387) which included Adams & Martray (1980), Adams (1982), Bartholomew (1974, 26 1976), and Cruickshank (1974). The top f i v e concerns for p r i n c i p a l s i n a l l l e v e l s of school were: "Motivation, getting students interested"; "Providing for ind i v i d u a l differences"; " D i s c i p l i n e , classroom control"; "Organizing and managing the classroom"; and "Testing, grading and promotion of students". Therefore, the ways i n which these kinds of issues 'come to the attention' of p r i n c i p a l s and the conceptions of teaching performance they create are of in t e r e s t . However, t h i s i s based on an assumption, i d e n t i f i e d by Storey and Housego (1980), of " i d e n t i f i a b i l i t y " . They suggest that t h i s means "regardless of the approach used, the personnel being supervised, [and] the supervisor,...act as i f desired outcomes and indicators of e f f e c t i v e practice were known and i d e n t i f i a b l e . " (p.2, authors' emphasis). They go on to raise the question "What are the c r i t e r i a of effectiveness?" (p.3, my emphasis) and subsequently to conclude: Assessment of any kind i s , by d e f i n i t i o n , based on cer t a i n c r i t e r i a . The exp l i c i t n e s s of these c r i t e r i a w i l l vary among organisations. The state of knowledge i n the f i e l d i s l i k e l y to be among the most s i g n i f i c a n t of factors a f f e c t i n g the c l a r i t y , u n i v e r s a l i t y and acceptance of a given set of effectiveness c r i t e r i a , (p.3) The formulation of evaluation c r i t e r i a i s a j o i n t r e s p o n s i b i l i t y of the school d i s t r i c t and p r i n c i p a l i n conjunction with teacher associations. These c r i t e r i a are then communicated through the d i s t r i c t c o l l e c t i v e 27 agreements. However, where the c o l l e c t i v e agreement f a i l s to state c r i t e r i a or make them e x p l i c i t , an onus i s placed on the p r i n c i p a l to codify c r i t e r i a and, i n turn, communicate them to the teacher. This requires that the p r i n c i p a l i s able to codify such c r i t e r i a and, as Storey and Housego imply, t h i s may not always be the case. Bailey (1984) asserts that "Many evaluators lack a systematic and orderly way of diagnosing and analyzing classroom teaching methods". He goes on to promote the use of "classroom teaching s t y l e c l a s s i f i c a t i o n systems...based on the assumption that... teaching styles are not equal" i n order that "the evaluator can i d e n t i f y , c l a s s i f y , and evaluate classroom teaching styles based on t h e i r intended purpose" (p.19). A f i n a l dimension to the consideration of evaluator competence i s provided by Bolton (1980) when he describes twelve "resistances to evaluation by evaluators" (p.27, author's emphasis). These 'resistances' emerged from a series of unstructured conversations with educational administrators over a period of years. They include a) uncertainty about c r i t e r i a , and inter p r e t a t i o n of data; b) fear of an unpleasant reaction which would prevent a relationship conducive to f a c i l i t a t i n g improvement; c) f a i l u r e to see evaluation as linked to the purposes of the evaluator; d) i n a b i l i t y to organise time for adequate 28 observations; e) fear of being held to a commitment to an objective which may take 'additional' time; f) lack of support from higher levels of the organisation; and, g) lack of conviction that evaluation w i l l provide as much payoff as time spent on other a c t i v i t i e s . Bolton provides a varied set of reasons as to why p r i n c i p a l s may be resistant to the role of evaluator and which span the three concepts addressed so far i n t h i s chapter. These reasons also emphasise the importance of the evaluator to the effectiveness of evaluation because they i l l u s t r a t e how far the evaluator i s able to interpret and influence the process. Obstacles to Evaluation Haefele (1992) suggests that lack of time for p r i n c i p a l s to conduct evaluations leads them to hesitate i n giving c r i t i c a l reports because the sample of observations they can carry out i s i n s u f f i c i e n t l y broad ( c i t i n g research from Andrews & Barnes, 1990; Bridges, 1986; Kauchak, Peterson, & D r i s c o l l , 1985; Langlois & Colarusso, 1988; Lower, 1987; Stodolsky, 1988). In r e l a t i o n to "managing time", Pigford and Tonnsen (1993) suggest that some pr i n c i p a l s may have "trouble i n distinguishing between what i s urgent and what i s important" (p.42). They go on to say: 29 Posner quotes Hummel [1967] as he reminds us of the difference: "We l i v e i n constant tension between the urgent and the important. The problem i s that the important tasks r a r e l y must be done today, or even t h i s week. The urgent task c a l l s for instant action" Everard and Morris (1990) t a l k about the " c r i t i c a l d i s t i n c t i o n " between the urgent and the important and that "we must not be lured into the trap of being caught up i n the urgent to the exclusion of the important" (p.123). They suggest that important tasks need to be thought of i n terms of the 'long-term' and 'short-term' and that time must be allocated to the important and kept as c a r e f u l l y as any appointment with a parent or d i s t r i c t o f f i c i a l . Smith and Andrews (1989) highlighted a further d i s t i n c t i o n i n terms of time management when they found that p r i n c i p a l s spent "less time than they thought they should on improving i n s t r u c t i o n and more time on maintaining the school" (p.27). Employing d e f i n i t i o n s for 'average p r i n c i p a l s ' and 'strong i n s t r u c t i o n a l leaders' they also found that 'average p r i n c i p a l s ' did not "implement t h e i r values on a day-to-day basis as they allocate[d] time among the various tasks that must be performed" which "has lead [ s i c ] observers of p r i n c i p a l s ' management practices to conclude that many p r i n c i p a l s are 'building managers' rather than 'instructional leaders', and they should spend less time on building management and more on improving i n s t r u c t i o n " (p.29). However, they go on to say that t h e i r data suggest "pri n c i p a l s who are strong i n s t r u c t i o n a l leaders do not divert time away from building management functions i n favor of i n s t r u c t i o n a l leadership functions". F i n a l l y , i n t h i s regard. Smith and Andrews conclude: These data suggest that p r i n c i p a l s who are strong i n s t r u c t i o n a l leaders implement discretionary time i n such a way that they codify, on a day-to-day basis, the ideals or values of the average p r i n c i p a l . They spend the greatest amount of t h e i r time on educational program improvement a c t i v i t i e s . These data also suggest that i t i s a f a l s e dichotomy to draw the d i s t i n c t i o n between being a strong building manager and a strong i n s t r u c t i o n a l leader, (p.29) Sergiovanni (1991) l i n k s time and purpose when he refers to the "80/20 qual i t y r u l e " . He asserts that "When more than 20 percent of the p r i n c i p a l ' s time and money i s expended i n evaluation for q u a l i t y control or less than 80 percent of the p r i n c i p a l ' s time and money i s spent i n professional improvement, q u a l i t y schooling s u f f e r s " (p.285). Bolton (1980), i n looking at the evaluation of administrators, describes the environment of p r i n c i p a l s as 'problematical' and asserts that an obstacle to the administrator i s "the heavy demand on time made by routine c l e r i c a l and administrative duties" (p.5). He goes on to suggest, c i t i n g E s t o sito et a l . (1975, p.63), that when administrators spend a good deal of t h e i r time on these kinds of routine a c t i v i t i e s or others which do not bring them into contact with teachers "the actual a c t i v i t i e s are 31 incongruent with the major role perceived by c l i e n t s - a helping relationship to them". Bolton concludes that t h i s "becomes an obstacle to harmonious relationships with others" (p.5). This analysis of the time-consuming nature of p r i n c i p a l r e s p o n s i b i l i t i e s i s helpful i n an understanding of the dynamics of the principal-teacher r e l a t i o n s h i p which emerges from the consideration of purpose and process. As a consequence, Bolton also highlights the need to look at the evaluator role of the p r i n c i p a l i n a wider context. F i n a l l y , Beck and Murphy (1993) provide some h i s t o r i c a l perspective to the issue of time as an obstacle when they reveal that i t i s not a new phenomenon. C i t i n g the 1954 Yearbook of the Department of Elementary School P r i n c i p a l s e n t i t l e d "Time for the Job", they quote from the preface which notes that "many p r i n c i p a l s have indicated grave concern about 'lack of time for the job'" ( i n Beck & Murphy, p.56) and explain that the document goes on to suggest ways in which d i f f e r e n t practices by p r i n c i p a l s might help to remedy the problem. Summary Much of the l i t e r a t u r e reveals that negative attitudes exist i n r e l a t i o n to evaluation and, moreover, that educational administrators should seek to develop procedures that w i l l achieve more p o s i t i v e reactions. The research 32 i d e n t i f i e s general explanations of these negative attitudes, such as a sense of powerlessness, and d i s t r u s t about the rea l motives behind evaluation. However, the purposes; from the point of view of the evaluator, do not receive much attention. What does emerge from the l i t e r a t u r e i s a dichotomy of purpose between the growth and development of the teacher and accountability for the q u a l i t y of teaching. This, i n turn, reinforces the image of the p r i n c i p a l caught between two d i f f e r e n t philosophies. Poster and Poster (1993) describe t h i s dichotomy as f a l s e and, i n highl i g h t i n g the importance of the delivery system and those who d e l i v e r i t , could e a s i l y be r e f e r r i n g to the evaluation process and the evaluators. Sergiovanni (1991) draws attention to the implied purpose of evaluation as transmitted through the summative processes employed. Even though the message received by teachers from summative evaluation may well be that organisational needs are being served rather than the needs of teachers, i t raises the question as to whether or not summative evaluation can be formative. Put another way, can the p r i n c i p a l provide a formative experience for the teacher while at the same time producing a summative report? If the answer i s p o s i t i v e , many p r i n c i p a l s may be working with a summative process but with formative purposes i n mind. 33 The l i t e r a t u r e presented i n t h i s chapter would suggest that formative purposes are not at the forefront for many pr i n c i p a l s and, indeed, that t h e i r approach needs to be less managerial and more c o l l e g i a l . This i s q u a l i f i e d by l i t e r a t u r e which suggests that t h i s managerial approach i s more a feature of male than female p r i n c i p a l s h i p . This l i t e r a t u r e describes women p r i n c i p a l s as more competent, as more caring, and, very importantly, as better communicators. This lends considerable importance to the dimension of gender i n a study of formal evaluation because evaluation emerges from the l i t e r a t u r e as a form of communication which involves high l e v e l s of interpersonal s k i l l s . The l i t e r a t u r e regarding the a b i l i t y and competence of school administrators i n t h e i r role as formal evaluators of teaching i s not encouraging. This l i t e r a t u r e suggests considerable room for improvement and Haefele (1992) even employs the term 'deficient evaluator'. Lack of adequate professional preparation or lack of time, due to volume of work, are two major reasons presented for p r i n c i p a l s being unable to perform the function of evaluator s a t i s f a c t o r i l y . However, l i t t l e research exists on the views of evaluators themselves, about t h e i r l e v e l of competence i n the formal evaluation of teaching. Page and Page (1985) suggest that p r i n c i p a l s f e e l well prepared for the role of evaluator. Bolton, on the other hand, presents a l i s t of 'resistances' to evaluation and Darling-Hammond (1986) asserts that many pr i n c i p a l s view evaluation as "u t t e r l y unimportant". The l i t e r a t u r e appears to assume that with greater experience i n the role of p r i n c i p a l the incumbent w i l l develop greater expertise. However, the consideration of issues such as competence i s not s p e c i f i c a l l y related to length of experience and, indeed, l i t t l e d i r e c t reference i s made to p r i n c i p a l experience i n the l i t e r a t u r e . F i n a l l y , the general role of p r i n c i p a l has been widely considered i n the l i t e r a t u r e and what emerges i s a v i v i d picture of a demanding job i n a complex environment. This picture enables an examination of p r i n c i p a l views about the formal evaluation of teaching to take place within a more informed context. CHAPTER III Research Design and Methodology This chapter i s divided into s i x sections: The framework for the study; sources of data; data c o l l e c t i o n procedures; data analysis and presentation; design l i m i t a t i o n s ; and, summary. The framework for the study set out the guiding questions and i d e n t i f i e s the concepts and variables which are given prominence. The sources of data section describes the subjects included i n the study and explains the involvement of the B r i t i s h Columbia P r i n c i p a l s and V i c e - P r i n c i p a l s ' Association (BCPVPA). The t h i r d section describes the two data c o l l e c t i o n procedures employed i n the study: a content analysis of the seventy-five B r i t i s h Columbia school d i s t r i c t c o l l e c t i v e agreements; and a questionnaire sent to a l l the members of the BCPVPA. The data analysis and presentation section explains the methods of analysis used for the questionnaire returns and how the data from these returns have been organised. The penultimate section, design l i m i t a t i o n s , devotes most attention to the consideration of the ge n e r a l i s a b l i t y of the data. The chapter concludes with a summary. 36 The Framework for the Study The evaluation l i t e r a t u r e described i n Chapter II and the concepts which guide t h i s study give r i s e to a number of questions which provide the s t a r t i n g point for the c o l l e c t i o n of data. Each of these questions provides the basis for a part of t h i s section of the chapter: purpose, process, t r a i n i n g , and obstacles. The two f i n a l parts of t h i s section take account of the two variables of sex and years of experience. Purpose What do p r i n c i p a l s believe to be the most important purpose of the formal evaluation of teaching? The answer to t h i s question would probably include both teacher growth and accountability and indeed Sergiovanni (1991) asserts that both are necessary i f the evaluation process i s to have a chance of success. However, i t would be informative to know which of these two i s the more highly regarded by p r i n c i p a l s and, therefore, i t would be necessary to ask p r i n c i p a l s to choose either teacher growth or accountability. Other data are also pertinent to the concept of purpose. For example, knowing whether p r i n c i p a l s carry out the role of evaluator simply because i t i s part of t h e i r contractual obligations or because they believe i t to be a role they should carry out. The extent to which p r i n c i p a l s see the purpose of formal evaluation as a q u a l i t y control measure may also be transmitted through the proportion of ' c r i t i c a l ' reports they write and therefore information about t h i s would be useful. Process With what formal evaluation processes are p r i n c i p a l s i n B r i t i s h Columbia working? Determining the evaluation processes employed i n the Province can be accomplished through the school d i s t r i c t c o l l e c t i v e agreements thus providing an answer to the question above. This would i d e n t i f y the summative/formative nature of evaluation, as well as factors such as the existence of stated c r i t e r i a , and whether or not the p r i n c i p a l has d i s c r e t i o n over choosing teachers to be evaluated. Training What t r a i n i n g have p r i n c i p a l s received i n formal evaluation and what are t h e i r t r a i n i n g needs? This question relates to the issue of professional preparation. It would be informative to know whether p r i n c i p a l s have received s p e c i f i c t r a i n i n g for the role of evaluator and how far they f e e l they need further t r a i n i n g . 38 Obstacles What do p r i n c i p a l s consider to be the most important obstacles to t h e i r carrying out the formal evaluation of teaching? The hindrances to conducting formal evaluation are quite well documented i n the l i t e r a t u r e . It would therefore be i n t e r e s t i n g to compare the views of B r i t i s h Columbia p r i n c i p a l s with t h i s l i t e r a t u r e , p a r t i c u l a r l y with regard to time. A more general question, such as the one above, would avoid the r i s k of leading p r i n c i p a l s into giving time as an obstacle. It would thus be i n t e r e s t i n g to see i f the obstacle of time arises 'naturally'. This question would also be useful i n i d e n t i f y i n g other factors which p r i n c i p a l s consider impede t h e i r conduct of evaluation. Sex of P r i n c i p a l Do differences ex i s t between male and female p r i n c i p a l s with regard to t h e i r views on the formal evaluation of teaching? The l i t e r a t u r e suggests that t h i s i s an important factor i n educational administration but t h i s l i t e r a t u r e does not address the s p e c i f i c issue of formal evaluation. Therefore, t h i s question would be a way of distinguishing between p r i n c i p a l s and provide information hitherto unavailable. 3 9 Years of Experience as P r i n c i p a l Do differences exist between p r i n c i p a l s with d i f f e r e n t l e v e l s of experience with regard to t h e i r views on the formal evaluation of teaching? This question provides a second way i n which to d i s t i n g u i s h between p r i n c i p a l s . The l i t e r a t u r e i s sparse on administrative experience and the behaviours of p r i n c i p a l s . The p a r t i c u l a r complexities of the role of evaluator would suggest that d i f f e r e n t l e v e l s of experience may lead to d i f f e r e n t l e v e l s of competence i n carrying out that r o l e . In order to have an accurate measure of experience, the number of evaluations c a r r i e d out by p r i n c i p a l s and the nature of those evaluations i s necessary, i n addition to the number of years of tenure. Therefore, the study w i l l i d e n t i f y important information about the views of p r i n c i p a l s as evaluators. This information f a l l s into three broad categories of process, purpose, and obstacles. The study w i l l also draw a d i s t i n c t i o n between p r i n c i p a l s based on sex and years of experience as a p r i n c i p a l , while also taking account of the nature of the evaluation process with which p r i n c i p a l s are working. 40 Sources of Data I contacted the B r i t i s h Columbia P r i n c i p a l s ' and Vice- P r i n c i p a l s ' Association (BCPVPA) to e l i c i t t h e i r support for a questionnaire-based study and for access to t h e i r l i b r a r y of B r i t i s h Columbia school d i s t r i c t c o l l e c t i v e agreements. Access to the c o l l e c t i v e agreements was immediately forthcoming and following a number of meetings I had with executive o f f i c e r s of the BCPVPA to discuss the purpose of the f i n a l study, the Association agreed to organise the d i s t r i b u t i o n of a questionnaire to a l l i t s members throughout the Province. Only p r i n c i p a l s were included i n t h i s study because i t i s they who conduct most evaluations. Even when evaluation i s delegated to v i c e - p r i n c i p a l s , anecdotal information suggests that they are given the 'straightforward' evaluations to conduct and therefore w i l l not have the same breadth of experience as p r i n c i p a l s . Therefore, the target population of the survey were the 1,179! p r i n c i p a l members of the BCPVPA who, i n turn, represent 76.9 percent of a l l the school p r i n c i p a l s i n B r i t i s h Columbia. However, a l l 2,430 members of the BCPVPA were sent a questionnaire because the Association were interested i n obtaining data r e l a t i n g to v i c e - p r i n c i p a l s as This i s the most accurate figure that could be obtained from the BCPVPA data f i l e s . 41 well as p r i n c i p a l s . In addition, the l o g i s t i c a l d i f f i c u l t i e s involved for the BCPVPA to post questionnaires to only some of t h e i r members were regarded as too great. In order to be able to l i n k data gathered from the c o l l e c t i v e agreements to data gathered from the questionnaire, i t was necessary to ask respondents the number of t h e i r school d i s t r i c t . While t h i s raised some concern about c o n f i d e n t i a l i t y , i t was considered fundamental to a meaningful set of results and therefore was included i n the section r e l a t i n g to an administrator's current assignment. It i s impossible to say whether or not t h i s question had an ef f e c t on the return rate. Data C o l l e c t i o n Procedures The study included two major data gathering procedures. F i r s t , a detailed study of the sections r e l a t i n g to the formal evaluation of teaching i n the seventy-five B r i t i s h Columbia school d i s t r i c t c o l l e c t i v e agreements and, second, the d i s t r i b u t i o n of a questionnaire to a l l BCPVPA members. A content analysis of a l l seventy-five B r i t i s h Columbia school d i s t r i c t c o l l e c t i v e agreements was c a r r i e d out at the o f f i c e s of the BCPVPA, i n order to extract clauses relevant to the formal evaluation of teaching. I n i t i a l l y , f i v e main questions guided the reading of these c o l l e c t i v e agreements: 42 a) How i s the formal evaluation of teaching c a r r i e d out? b) Who i n i t i a t e s a formal evaluation of teaching? c) Who i s responsible for conducting evaluation? d) Are there stated evaluation c r i t e r i a ? e) Is there a stated evaluation cycle? Common themes emerged from the processes s p e c i f i e d for the formal evaluation of teaching. These included four phases (described i n more d e t a i l i n Chapter IV, p.58), the right of appeal against the process or outcome of an evaluation, the opportunity for the teacher to receive remedial help, and whether or not an e x p l i c i t d i s t i n c t i o n was made between formative and summative evaluation. With regard to formal evaluation c r i t e r i a and cycles, c r i t e r i a and cycles are stated i n quite d i f f e r e n t ways i n d i f f e r e n t c o l l e c t i v e agreements and the frequencies of cycles are by no means always clear cut. For t h i s reason i t became necessary to keep a detailed record of what each contained rather than simply whether or not c r i t e r i a or a cycle existed. With information from the c o l l e c t i v e agreements about the context within which the formal evaluation of teaching takes place i n B r i t i s h Columbia and informed by the review of l i t e r a t u r e , i t was possible to begin formulating a set of questions around the key themes of purpose, time and t r a i n i n g . An important consideration i n formulating two of the questions was to provide a time frame. These relate to the period since September 1988, to take account of the legal change i n relat i o n s h i p between teachers and school- based administrators which occurred as a result of the Teaching Profession Act 1987 (see p . l ) . A draft questionnaire was p i l o t e d with eleven school based administrators who had either recently r e t i r e d , were on the executive committee of the BCPVPA, or were currently on study leave or seconded to other duties. The p i l o t i n g resulted i n some modest but important changes to the questionnaire and the f i n a l version (Appendix A) was then printed and d i s t r i b u t e d by the BCPVPA. A t o t a l of 2,430 questionnaires were d i s t r i b u t e d and, as i s the normal practice for BCPVPA i n i t i a t e d or supported surveys, a return paid envelope was not provided. The questionnaires were posted on or around February 9th, 1996 with a return deadline of February 26th, 1996. A reminder was issued to BCPVPA members v i a t h e i r regional executive o f f i c e r s i n mid-March. However, t h i s resulted i n only an additional three questionnaires being received by the f i n a l deadline of Friday, A p r i l 5th, 1996, bringing the achieved sample to 188 or 15.9 percent of p r i n c i p a l members of the BCPVPA. 44 Data Analysis and Presentation The data from the returned questionnaires were coded and entered onto a data f i l e . For some variables, data were grouped or reformulated i n order to f a c i l i t a t e analysis. Data from question number 7 (see Appendix A.2), which asked for the number of years as a p r i n c i p a l , were merged into four experience categories (1-5, 6-10, 11-15, and 16+). The data from question 16 (see Appendix A.3), which asked for information about past t r a i n i n g i n the formal evaluation of teaching, were reformulated. This reformulation aggregates the d i f f e r e n t durations of t r a i n i n g a respondent could have attended, by using a formula to produce a t o t a l number of t r a i n i n g 'points'. This formula assigned one point for each course of "one day or l e s s " , three points for each course of "between two days and one week", f i v e points for each course of "more than one week but less than one f u l l term", and ten points for each course of "one f u l l u n i v e r s i t y or college term". The rationale for t h i s formula was based on an estimated value for the amount of time spent engaged i n the t r a i n i n g , rather than i t s quali t y , since t h i s was impossible to gauge. The s t a t i s t i c a l analysis was c a r r i e d out using the "SPSS Windows 6.0" program. F i r s t l y , t h i s analysis included the production of frequency summaries for a l l questions, apart from 21 (see Appendix A.6). Secondly, cross tabulations and chi-square analysis were used i n order to break down the respondents into d i f f e r e n t constituent groups and determine the l e v e l of si g n i f i c a n c e of the r e s u l t i n g data. The analysis of question 21 was c a r r i e d out manually by i d e n t i f y i n g themes i n the anecdotal responses and then grouping them into broader categories (see p.84). This approach was also adopted for question 18 about obstacles, i n addition to the coded data on the computer f i l e , i n order to i d e n t i f y the reasons p r i n c i p a l s have for sta t i n g time as a major obstacle. The presentation of the questionnaire data i s i n i t i a l l y i n the form of frequency summaries i n Chapter V, followed by a more detailed breakdown of the data i n Chapters VI and VII. This breakdown involves categorising respondents i n two d i f f e r e n t ways to produce p r o f i l e s based on respondent sex and number of years experience as a p r i n c i p a l . Reference to s t a t i s t i c a l s i g n i f i c a n c e i s included i n Chapter VI and VII i n order to determine the p r o b a b i l i t y that differences found i n these p r o f i l e s occur by chance or whether they are l i k e l y to be found i n the whole B r i t i s h Columbia p r i n c i p a l population. Findings are described as s t a t i s t i c a l l y s i g n i f i c a n t at the p=.05 l e v e l . Where the l e v e l of si g n i f i c a n c e i s p<.05 t h i s i s shown i n parenthesis but no p r o b a b i l i t y value i s given for analyses producing a sig n i f i c a n c e l e v e l of p=>.05. 46 Design Limitations It i s important to note that while school based p r i n c i p a l s carry a considerable part of the r e s p o n s i b i l i t y of formally evaluating teaching, t h i s r e s p o n s i b i l i t y i s not e n t i r e l y t h e i r s . Therefore, t h i s study cannot possibly address the whole formal evaluation scene but only that which relates to the p r i n c i p a l ' s role as evaluator. Furthermore, the subjects i n v i t e d to p a r t i c i p a t e i n the study, as already highlighted, constitute 75.9 percent of the B r i t i s h Columbia p r i n c i p a l population. This 'selection' may have some bearing on the v a l i d i t y of the f i n a l conclusions drawn but t h i s cannot be judged accurately because data r e l a t i n g to the sub-population of approximately 24.1 percent are not available. The t o t a l number of respondents to the questionnaire i s 267 2 of which 188 are school-based p r i n c i p a l s (referred to hereafter as p r i n c i p a l s ) , 70 are v i c e - p r i n c i p a l s and the remaining 9 are d i s t r i c t p r i n c i p a l s . The 188 p r i n c i p a l s represent 15.9 percent of the t o t a l BCPVPA p r i n c i p a l membership (1,179) and, therefore, t h i s constitutes a major ^This l e v e l of response compares well with that of other surveys with BCPVPA members. For example, i n January 1996, the month before the d i s t r i b u t i o n of t h i s questionnaire, a BCPVPA survey asked f o r reactions to the Mi n i s t r y of Education's "Default Plan" on the amalgamation of B r i t i s h Columbia school d i s t r i c t s . This survey on amalgamation e l i c i t e d 194 responses i n t o t a l . It i s cl e a r from t h i s information that p r i n c i p a l s f i n d i t d i f f i c u l t to respond to surveys of t h i s kind, among the many other 'paper exercises' they are asked or required to perform. 47 l i m i t a t i o n to t h i s study. From a methodological standpoint, i t presents d i f f i c u l t i e s for s t a t i s t i c a l analysis, where c e l l sizes may be too small to draw conclusions with confidence and, thus, g e n e r a l i s a b i l i t y of the findings i s a concern. Establishing how far g e n e r a l i s a b i l i t y has been undermined i s d i f f i c u l t with regard to c e r t a i n biographical information. For example, data on years of experience as p r i n c i p a l , post graduate education and specialty, and the percentage of p r i n c i p a l teaching time, were unavailable for either BCPVPA members or B r i t i s h Columbia p r i n c i p a l s as a whole. However, the School Finance and Data Management Branch of the Finance and Administration Department at the B r i t i s h Columbia Ministry of Education was able to provide some information which pertained s p e c i f i c a l l y to p r i n c i p a l s . These Ministry and BCPVPA data indicate that the participants i n t h i s study are si m i l a r to the BCPVPA p r i n c i p a l membership or B r i t i s h Columbia public school p r i n c i p a l s as a whole. Table 3.1 shows the percentages of respondents, BCPVPA p r i n c i p a l s , and B r i t i s h Columbia p r i n c i p a l s as a whole with regard to the variables of sex, age, school type, and s t a f f s i z e . 48 Table 3.1 Respondents, BCPVPA P r i n c i p a l s , and A l l B r i t i s h Columbia Public School P r i n c i p a l s by Sex, Age, School Type, and Staff Size Respondents BCPVPA members B r i t i s h Columbia p r i n c i p a l s Variable n Sex Male Female Age 44 or less 45 to 49 50 to 54 55 or over School type Elementary Secondary Both St a f f s i z e 1 to 9 10 to 19 20 to 29 30 or more 72.7 27.3 21.3 34.6 25.5 18.6 71.8 22.3 5.9 8. 29. 32. 29.7 136 51 40 65 48 35 135 42 11 15 55 60 55 74.5 25.5 78.9 20.1 1.0 878 301 930 237 12 a 73.4 26.6 23.7 32.8 29.3 14.2 28.1 35.6 21.3 15.0 1125 408 363 503 449 218 470 b 596 356 253 a T h i s represents the best a v a i l a b l e information. bThese figures r e l a t e to numbers of schools rather than p r i n c i p a l s . The above Table shows that the percentages of male and female respondents match very c l o s e l y those for BVPVPA p r i n c i p a l s and B r i t i s h Columbia p r i n c i p a l s as a whole. The BCPVPA does not maintain data about the age of i t s members but information from the B r i t i s h Columbia Ministry of Education (Report 2059 - 1995/1996 School Year - Age d i s t r i b u t i o n of Educators by Position Within the School) shows that the age d i s t r i b u t i o n of the respondents i s sim i l a r to the population of a l l B r i t i s h Columbia p r i n c i p a l s . Table 3.1 shows that the largest discrepancy i n the four age categories i s i n the group "55 years or over". However, the pattern of d i s t r i b u t i o n i s the same over the four age groups and the differences are r e l a t i v e l y small. Information about the types of school that p r i n c i p a l s i n the BCPVPA administer was more d i f f i c u l t to ascertain because school descriptions do not always make i t clear whether the student intake i s elementary grades only (K-7), secondary grades only (8-12), or both elementary and secondary grades. However, an approximation was calculated from BCPVPA f i l e s which indicates (as Table 3.1 shows) that once again the respondents quite c l o s e l y match the population as a whole. However, the information which the Ministry of Education made available on teaching s t a f f sizes shows less correspondence between the questionnaire respondents and the whole population of B r i t i s h Columbia p r i n c i p a l s . Indeed, as Table 3.1 shows, there are wide d i s p a r i t i e s between the d i s t r i b u t i o n of respondents and a l l B r i t i s h Columbia p r i n c i p a l s , which i s p a r t i c u l a r l y the case for p r i n c i p a l s with s t a f f s of "1 to 9" and "30 or more". Returns were received from 56 (74.7%) of the 75 B r i t i s h Columbia school d i s t r i c t s . It i s possible to compare the 50 o v e r a l l d i s t r i b u t i o n of respondents i n these school d i s t r i c t s with the o v e r a l l d i s t r i b u t i o n of a l l p r i n c i p a l s i n B r i t i s h Columbia. Figure 3.1 shows the respondents as a percentage of a l l respondents and t h e i r d i s t r i b u t i o n across the 75 B r i t i s h Columbia school d i s t r i c t s . This figure enables a comparison to be made with the d i s t r i b u t i o n of B r i t i s h Columbia p r i n c i p a l s as a whole across a l l school d i s t r i c t s . For example, seven percent of B r i t i s h Columbia p r i n c i p a l s are based i n school d i s t r i c t 36, while ten percent of the respondents to the survey are based there. Figure 3.1 indicates that, generally speaking, where there are concentrations of respondents from p a r t i c u l a r d i s t r i c t s , a corresponding concentration exists among B r i t i s h Columbia p r i n c i p a l s as a whole. However, of the 19 school d i s t r i c t s not represented i n the responses to the questionnaire, one, school d i s t r i c t 22, employs 20 p r i n c i p a l s . A further discrepancy includes the over-representation of respondents from school d i s t r i c t s 7, 60, and 75 (medium sized - see Footnote 3) and 25, 36, and 43 (large). How far these discrepancies may i n t e r f e r e with g e n e r a l i s a b i l i t y can be measured against B r i t i s h Columbia Ministry of Education guidelines for the categorisation of school d i s t r i c t s as "small",  52 "medium" or "large" 3. Table 3.2 reveals a very close s i m i l a r i t y between the percentages of respondents and a l l B r i t i s h Columbia p r i n c i p a l s employed i n the three sizes of school d i s t r i c t referred to above. This comparison suggests that the modest discrepancies highlighted i n Figure 3.1, do not pose a serious threat to g e n e r a l i s a b i l i t y . Table 3.2 Respondents and A l l B r i t i s h Columbia P r i n c i p a l s by School D i s t r i c t Size D i s t r i c t s i z e Respondents B r i t i s h Columbia p r i n c i p a l s n % n Large 51.4 95 51.7 764 Medium 33.0 61 36.2 535 Small 15.7 29 12.2 180 Total 100.1 185 100.1 1479 a a T h i s figure i s d i f f e r e n t from the M i n i s t r y of Education t o t a l of 1533 given i n Table 3.1. It i s a count from the 1994/1995 Public and Independent Schools Book (Province of B r i t i s h Columbia, 1995) and represents the best a v a i l a b l e information. J T h i s categorisation i s based on student enrolments, so that school d i s t r i c t s with 1 to 2,999 students are categorised as "small", those with 3,000 to 14,999 are c l a s s i f i e d as "medium", and those with 15,000 or more student enrolments are defined as "large" (Cherington, 1989, i n Antosz, 1990, p.67 - see Appendix E). 53 Data from the c o l l e c t i v e agreements (Chapter IV), shows that many of the provisions for evaluation are very s i m i l a r . However, a d i s t i n c t i o n i s possible with regard to evaluation c r i t e r i a and evaluation cycles ( i n d i c a t i n g the stated frequency, i f any, for evaluations). Table 3.3 shows that respondents are c l o s e l y representative of a l l B r i t i s h Columbia p r i n c i p a l s i n d i s t r i c t s with stated c r i t e r i a , but less so for p r i n c i p a l s i n d i s t r i c t s with evaluation cycles. Table 3.3 Respondents and B r i t i s h Columbia P r i n c i p a l s by C r i t e r i a and Cycles Respondents B r i t i s h Columbia p r i n c i p a l s Variable % n % n C r i t e r i a No c r i t e r i a C r i t e r i a 33.0 67.0 61 124 34.3 65.7 507 972 Cycle No Cycle Cycle 39.5 60.5 73 112 53.1 46.9 785 694 Furthermore, a comparison can be made on the basis of the way evaluation cycles are described. Table 3.4 shows such a comparison between respondents and a l l B r i t i s h Columbia p r i n c i p a l s and reveals f a i r l y marked differences. However, t h i s i s to be expected as a r e s u l t of the over representation of respondents from d i s t r i c t s with evaluation cycles. Table 3.4 C o l l e c t i v e Agreement Wording for Evaluation Cycles Evaluation cycle C o l l e c t i v e Respondents B r i t i s h Columbia phraseology agreements 3 p r i n c i p a l s n % n % n % "Every" 14 18.7 31 16.8 190 12. 8 "At least every" 15 20.0 41 22.2 278 18. 8 "Not more than one i n " 7 9.3 40 21.6 226 15. 3 Total 36 48.0 112 64.4 694 46. 9 a T o t a l including c o l l e c t i v e agreements without a cycle = 75 Taking the l i m i t a t i o n s into account, the study does provide an opportunity to present and examine the views of p r i n c i p a l s on formal evaluation, i d e n t i f y areas where further investigation would be h e l p f u l , and a r r i v e at a number of conclusions and p o l i c y recommendations. 55 Summary Three themes have been i d e n t i f i e d : Purpose, time, and tr a i n i n g . In addition to these themes the two variables of p r i n c i p a l sex and years of experience are highlighted as possible factors i n determining p r i n c i p a l views. Two data c o l l e c t i o n procedures are used: a content analysis of the clauses r e l a t i n g to formal evaluation i n the B r i t i s h Columbia school d i s t r i c t c o l l e c t i v e agreements; and a questionnaire. The subjects i n v i t e d to p a r t i c i p a t e i n the study were the 1,179 p r i n c i p a l members of the BCPVPA of whom 188 (15.9%) took part. The response rate raises a concern about the g e n e r a l i s a b i l i t y of the findings. However, i n r e l a t i o n to a number of variables including sex, age, school type, d i s t r i c t s i z e , and provision of evaluation c r i t e r i a i n c o l l e c t i v e agreements, the respondents are representative of of B r i t i s h Columbia p r i n c i p a l s as a whole. This i s less so for s t a f f s i z e and provision of evaluation cycles i n c o l l e c t i v e agreements. Staff sizes of "1 to 9" are under represented among the respondents and s t a f f sizes of "30 or more" are over represented. P r i n c i p a l s i n d i s t r i c t s without an evaluation cycle are under represented among the respondents and p r i n c i p a l s i n d i s t r i c t s with an evaluation cycle are over represented. 56 CHAPTER IV B r i t i s h Columbia School D i s t r i c t C o l l e c t i v e Agreements A l l seventy-five B r i t i s h Columbia school d i s t r i c t c o l l e c t i v e agreements contain provision, i n some form or another, for the evaluation (though i n a very few cases not formal evaluation) of teaching. The most current versions of these c o l l e c t i v e agreements were drawn up i n July 1992, to be reviewed i n 1994 or 1995, but remain i n ef f e c t at the time of t h i s study. The f i r s t part of Chapter IV gives an overview of the content analysis of these c o l l e c t i v e agreements with regard to the evaluation process. The second part of the chapter relates to the roles and r e s p o n s i b i l i t i e s of both the evaluator and evaluatee. The chapter concludes with a summary and a sample a r t i c l e from a B r i t i s h Columbia school d i s t r i c t c o l l e c t i v e agreement can be found i n Appendix F. The Process The content analysis of the c o l l e c t i v e agreements reveals that s i x t y - s i x d i s t r i c t s have a very si m i l a r evaluation process which incorporates four phases (see Appendix B). A further two are distinguished only by the fact that they make provision for a shortened process for "highly competent" teachers. Only two of the remaining seven employ a process which ( a l b e i t having phases which are common in some form to other d i s t r i c t s ) i s s u b s t a n t i a l l y d i f f e r e n t from the r e s t . Other features of the process that emerge from the content analysis are a) the provisions, or otherwise, for a regular cycle of formal evaluations (that i s , a fixed period of time within which a teacher's classroom s i t u a t i o n must be formally evaluated); and, b) whether or not there are stated evaluation c r i t e r i a . Most c o l l e c t i v e agreements require that the conclusion to the f i n a l report contain reference to either the term "satisfactory" or "less than s a t i s f a c t o r y " (a very few allow for graded comments such as "excellent", "very good" and so on). The term used indicates the evaluator's summative view of the adequacy or otherwise of the teacher's "classroom s i t u a t i o n " . If a teacher receives three consecutive "less than s a t i s f a c t o r y " reports i n a period normally between twelve and twenty-four months, the teacher i s l i a b l e to dismissal. Reference to the terms formative and summative are not made i n the c o l l e c t i v e agreements, but those which share the features described above are c e r t a i n l y , by implication, summative and more orientated towards accountability than growth and development. 58 The Four Phases of a Formal Evaluation of Teaching The phases that emerge from the c o l l e c t i v e agreements can be described as a) pre-evaluation conference(s); b) classroom observations; c) post-observation conferences; and, d) f i n a l report conference and writing. The pre- evaluation conference (or conferences, since i n a very few c o l l e c t i v e agreements provision i s made for two such conferences) takes place i n order for the p a r t i c i p a n t s to ta l k through the purpose, c r i t e r i a , and timetable for the coming formal evaluation. Generally, the second phase, that of the classroom observations, includes between three and six classroom v i s i t s , which i n most cases are recommended to be for the duration of the whole lesson. The majority of c o l l e c t i v e agreements s t i p u l a t e that a) a post-observation conference should take place within a l i m i t e d time aft e r the observed lesson; and b) the teacher should be provided with an anecdotal statement by the evaluator. If weaknesses were observed the teacher must be apprised of them and given the opportunity to remedy them before the next classroom observation. The fourth phase, writing the f i n a l report, requires the teacher to be given an opportunity to read a draft report and comment upon i t , before the f i n a l report i s written and f i l e d at the school board. Seventeen c o l l e c t i v e agreements s p e c i f i c a l l y disallow the i n c l u s i o n of references 59 to anything other than the data generated from the formal evaluation classroom observations. A further twelve state that classroom observation data should be those used "primarily", "generally", or "normally". However, i n fourteen c o l l e c t i v e agreements the evaluator i s e x p l i c i t l y e n t i t l e d to include aspects of the teacher's work i n the school beyond what was observed i n the classroom v i s i t s . In these school d i s t r i c t s , p r i n c i p a l s can include reference to the teacher's "general contribution", "general performance", "other factual information", "other pertinent information", "other information" or "multiple sources of data". In the remaining thirty-two c o l l e c t i v e agreements no i n d i c a t i o n was given regarding sources of data to be used i n the f i n a l report (see Appendix C). Formal Evaluation Cycles T h i r t y - s i x (48%) c o l l e c t i v e agreements contained terms for an evaluation cycle (see Appendix D). Of the 39 school d i s t r i c t s which make no provision for such a cycle, eight provide for what might be termed 'automatic' evaluation i n c e r t a i n cases. These are i n cases where a teacher i s new to the profession, or to the d i s t r i c t , or has assumed a s i g n i f i c a n t l y d i f f e r e n t assignment. In such cases, the teacher must be evaluated i n his/her f i r s t year (or i n one case the second year for teachers new to the profession). For those d i s t r i c t s , referred to above, that do state a 60 formal evaluation cycle, the frequency of evaluations varies considerably at the extremes (from two year to ten year inter v a l s ) but the vast majority (34 of 36) f a l l somewhere in the range of every three to f i v e years. However, the phraseology used to s t i p u l a t e the frequency of these cycles i s not the same and can be categorised into three types, each of which conveys a somewhat d i f f e r e n t expectation and perhaps, therefore, a d i f f e r e n t l e v e l of r e s p o n s i b i l i t y for the evaluator. These phrases include the provision that a formal evaluation of teaching w i l l be conducted a) "every" stated number of years; b) "at least every" stated number of years and c) "not more than one i n " a stated number of years. This l i n g u i s t i c context i s further complicated by the fact that, i n some cases, the expectation of the frequency of evaluations i s couched i n q u a l i f i e d terms. For example, i n eight c o l l e c t i v e agreements that state a frequency (whether i t be categories a, b, or c above), q u a l i f i c a t i o n s are employed which include "usually", "normally", "unless otherwise agreed", "where practicable" and " i t i s expected". Table 4.1 shows f i r s t l y , reading from l e f t to right, the t o t a l number of school d i s t r i c t c o l l e c t i v e agreements with each of the three forms of wording (a - c above), followed, secondly, by the number that have t h i s wording i n 'unqualified' and unambiguous terms. The t h i r d column 61 indicates the number of c o l l e c t i v e agreements that have each of the three forms of words but i n ' q u a l i f i e d ' terms. F i n a l l y , the fourth column provides the number of B r i t i s h Columbia p r i n c i p a l s whose assignment i s i n school d i s t r i c t s with these three variations of cycle provision. Table 4.1 C o l l e c t i v e Agreement Wording on Evaluation Cycles Wording Total Number with Number with Number of c o l l e c t i v e u n q u a l i f i e d q u a l i f i e d B r i t i s h agreements wording wording Columbia p r i n c i p a l s "Every" 14 11 3 190 "At least every" 15 12 3 278 "Not more than one i n " 7 5 2 226 Total 36 28 8 694 Thus, Table 4.1 shows that the s t i p u l a t i o n "every", i s given i n fourteen c o l l e c t i v e agreements, eleven of which state t h i s i n unqualified terms. The phrase "at least every" i s included i n f i f t e e n c o l l e c t i v e agreements, twelve of which are without q u a l i f i c a t i o n . Seven agreements use "not more than one i n " , of which f i v e are unqualified. The sig n i f i c a n c e of t h i s language springs from the consequences i t i s l i k e l y to have for the frequency of evaluations. The s t i p u l a t i o n "every" allows no room for manoeuvre on the part of p r i n c i p a l s and teachers a l i k e and "at least every" can c l e a r l y mean evaluations take place 62 more often than the stated time period. However, "not more than" provides, by the s t r i c t l e t t e r of the language, unlimited scope for the frequency of evaluations. For example, one evaluation every f i v e years and one every f i f t e e n years both adhere to a cycle of not more than one evaluation every four years (or three, or six and so on) because no evaluation i n a four year period i s not more than one. While the interpretation of such a s t i p u l a t i o n may not be as r a d i c a l as t h i s example suggests, the wording of c o l l e c t i v e agreement provisions and the q u a l i f i c a t i o n s they may contain, c l e a r l y have potential importance for the frequency of evaluation. Evaluation C r i t e r i a Stated c r i t e r i a were found to be present i n 48 (64%) c o l l e c t i v e agreements (see Appendix D). However, when they are present i n a c o l l e c t i v e agreement or referred to as part of some other school d i s t r i c t p o l i c y or document, they can vary markedly i n t h e i r s p e c i f i c i t y . Some provide a great deal of d e t a i l as to exactly what the teacher should be able to demonstrate and the evaluator observe for, while others simply l i s t a set of headings which allows for considerably more interpretation by the parties involved. For example, thirty-two school d i s t r i c t s state t h e i r evaluation c r i t e r i a i n some d e t a i l as either a r t i c l e s i n , or appendices to, the c o l l e c t i v e agreement, or as part of school d i s t r i c t p o l i c y 63 documents. A further seven d i s t r i c t s do state t h e i r evaluation c r i t e r i a i n the c o l l e c t i v e agreements but only as a b r i e f outline or set of general headings covering areas to be commented upon. Such headings include "classroom management" or " i n s t r u c t i o n a l strategies" but do not enter into any d e t a i l as to exactly how an evaluation of "s a t i s f a c t o r y " or "less than s a t i s f a c t o r y " might be arrived at. The remaining nine c o l l e c t i v e agreements contain references including "as the Evaluation Committee recommends" or "to be modified at the school l e v e l " and i n these cases i t was d i f f i c u l t to ascertain the degree of s p e c i f i c i t y employed. The Evaluator and the Evaluatee The content analysis of the c o l l e c t i v e agreements also included taking note of s t i p u l a t i o n s with regard to: a) the i n i t i a t i o n of an evaluation, i f not activated by a regular cycle; b) the personnel responsible for conducting a formal evaluation of teaching; c) the right of a teacher, whose teaching has been the subject of an evaluation, to lodge an appeal against the process and/or outcome; and d) the entitlements a teacher has to professional development opportunities following a "less than s a t i s f a c t o r y " report or an i n d i c a t i o n of weaknesses i n a " s a t i s f a c t o r y " report. 64 The I n i t i a t i o n of a Formal Evaluation The picture painted by school d i s t r i c t c o l l e c t i v e agreements across the Province i s quite a complex one. However, as a general rule, a formal evaluation can be i n i t i a t e d i n d i v i d u a l l y or by some combination of the teacher, the school-based administrative o f f i c e r , or by the school board, through the d i s t r i c t superintendent, assistant superintendent or some other competent board o f f i c i a l . This complex picture i s incomplete because i n 24 c o l l e c t i v e agreements i t i s unclear who i s able to i n i t i a t e an evaluation, other than i n 6 which provide for an evaluation cycle. Of the 51 c o l l e c t i v e agreements that do make some s p e c i f i c statement i n t h i s regard, 24 make reference to the school-based administrative o f f i c e r (which i n a l l cases would mean the p r i n c i p a l , even i f the subsequent evaluation were ca r r i e d out by a v i c e - p r i n c i p a l ) . Forty-three give the right of i n i t i a t i n g an evaluation to the teacher, although, i n a few cases, the agreement of the administrative o f f i c e r or school board i s also required. In four cases the Minister for Education and the B r i t i s h Columbia College of Teachers are also mentioned i n addition to the above par t i e s . 65 Responsibility for Conducting a Formal Evaluation of Teaching The c o l l e c t i v e agreements place most of the r e s p o n s i b i l i t y for formal evaluation on the school-based administrative o f f i c e r . While t h i s r e s p o n s i b i l i t y i s shared with superintendents, t h e i r assistants and i n some cases d i s t r i c t p r i n c i p a l s and directors of personnel, these l a t t e r o f f i c e r s are generally reserved for evaluations where a teacher has already received one "less than s a t i s f a c t o r y " report. Of the 75 school d i s t r i c t c o l l e c t i v e agreements studied, 68 s p e c i f i c a l l y refer to the administrative o f f i c e r or p r i n c i p a l as having t h i s evaluation r e s p o n s i b i l i t y . In the other seven, no reference of any kind was made to the administrative o f f i c e r i n the section r e l a t i n g to the ' evaluation of teaching. The Right of Appeal This i s u n i v e r s a l l y present i n a l l c o l l e c t i v e agreements under the section e n t i t l e d "Grievance Procedure". This usually involves a series of stages ( i n most cases four), each successive one of which i s only reached i f agreement has not been possible at the previous stage. F i n a l l y , there i s provision for a r b i t r a t i o n should agreement prove to be impossible through the grievance procedure. 66 Teacher Entitlement to Professional Development The large majority of c o l l e c t i v e agreements give a teacher who has received, a f i r s t or second "less than s a t i s f a c t o r y " report, an entitlement to professional development opportunities. These generally consist of up to one year's unpaid leave to undertake further t r a i n i n g and/or the o f f e r of a "plan of assistance" which i s to be drawn up by the school p r i n c i p a l or d i s t r i c t superintendent and agreed with the teacher concerned. Summary Generally, the formal evaluation of teaching consists of four phases including a pre-evaluation phase, between three and six classroom observations, post-observation conferences, and a f i n a l report conference. Approximately half of the school d i s t r i c t s i n B r i t i s h Columbia have evaluation cycles but these vary i n length and i n the str i c t n e s s of wording. The other half have no stated frequency of evaluation. Nearly two th i r d s of the school d i s t r i c t s state evaluation c r i t e r i a i n some form. The i n i t i a t i o n of evaluation, i f not by a cycle, i s a right which l i e s predominantly with the teacher and p r i n c i p a l , but can also be exercised by d i s t r i c t and Ministry personnel. However, conduct of formal evaluation i s , i n very large part, the r e s p o n s i b i l i t y of the school 67 p r i n c i p a l . If a teacher's "classroom s i t u a t i o n " i s considered to be d e f i c i e n t , assistance i s generally- ava i l a b l e . Should the teacher be d i s s a t i s f i e d with the process or outcome of the evaluation there i s also provision made for an appeal procedure. CHAPTER V Respondents' Backgrounds, Assignments, and Their Role as Evaluators of Teaching This chapter reports the results obtained from the questionnaire returns. It describes the ov e r a l l summary of response frequencies based on t o t a l v a l i d responses ( i . e . missing cases are not included) to each question and, therefore, not a l l t o t a l numbers of respondents equal 188. If the number of missing cases i s considered high and cannot be accounted for because a question was "not applicable" to a large number of respondents, t h i s fact i s brought to the attention of the reader. A complete summary of response frequencies i s provided i n Appendix G. The chapter i s organised into the three main headings that appeared on the questionnaire. It therefore includes biographical information on the respondents, followed by data r e l a t i n g to t h e i r current assignment and, f i n a l l y , responses with regard to t h e i r r e s p o n s i b i l i t i e s as formal evaluators of teaching. A summary concludes the chapter. Biographical Information Table 5.1 shows that approximately three quarters of the respondents were male and a quarter were female. Four broad age categories were i d e n t i f i e d : 44 years or fewer; 45 to 49; 50 to 54; and 55 years or more. The majority of 69 respondents are i n the middle two age categories with around one f i f t h i n each of the other two. Table 5.1 Respondent Biographical Data Respondents Variable % n Sex Male 72.7 136 Female 27.3 51 Age 44 or fewer 21.3 40 45 to 49 34.6 65 50 to 54 25.5 48 55 or over 18.6 35 Master's s p e c i a l t y Administration 65.7 l l l a Curriculum 14.8 25 Other 19.5 33 Experience as a p r i n c i p a l I to 5 years 31.0 58 6 to 10 31.0 58 II to 15 12.8 24 16 or more 25.1 47 aThere were 19 missing cases i n the returns for t h i s question. The overwhelming majority of respondents, 92.6 percent (n=174), have a master's degree or are currently working on one. Of these, two t h i r d s have a master's i n Educational Administration, while just 14.8 percent (n=25) have t h e i r master's i n Curriculum. The remaining f i f t h have a master's i n an area they described as "other" (these included 70 combined Educational Administration and Curriculum [n=15]; Counselling or Educational Psychology or Special Education [n=10]; subject d i s c i p l i n e [n=5]; and Supervision of Instruction/Teaching Practice [n=3]). Only 5.4 percent (n=10) have a doctoral degree or are currently working on one. The l e v e l of p r i n c i p a l experience i s also categorised into four groups: One to f i v e years; s i x to ten years; eleven to f i f t e e n years; and sixteen years or more. Somewhat less than a t h i r d of the respondents f a l l into each of the f i r s t two groups, with a much smaller proportion i n the "11 to 15 years" group, while a quarter of respondents have 16 or more years of experience as a p r i n c i p a l . Current Assignment Even though only p r i n c i p a l s have been included i n the resu l t s from t h i s survey, over one half have teaching r e s p o n s i b i l i t i e s to some degree. Table 5.2 shows that nearly half of the respondents indicated t h e i r assignment i s fu l l - t i m e administration. The rest are f a i r l y evenly spread across the teaching load categories of 1 to 19 percent; 20 to 39 percent; and 40 percent or more. Elementary p r i n c i p a l s constitute by far the largest group of respondents, while secondary p r i n c i p a l s accounted for around a quarter of the responses, and p r i n c i p a l s from 71 schools which enrol both elementary and secondary grades constituted 5.9 percent (n=ll). Numbers of teaching s t a f f (which respondents were asked to provide as a head count, including the p r i n c i p a l ) within these schools were placed into four groups: 1 to 9 teaching s t a f f ; 10 to 19 s t a f f ; 20 to 29 s t a f f ; and 30 or more s t a f f . The f i r s t of these groups i s by far the smallest, with the other three each at around 30 percent of t o t a l respondents. Table 5.2 Teaching Load, School Type, and Staff Size Respondents Variable % n Percentage teaching Zero 1 to 19 20 to 39 40 or more 44.9 17.3 22.2 15.7 83 32 41 29 Type of school Elementary grades Secondary grades Both 71.8 22.3 5.9 135 42 11 Teaching s t a f f 1 to 9 10' to 19 20 to 29 30 or more 8.1 29.7 32.4 29.7 15 55 60 55 72 The respondents included p r i n c i p a l s from 56 of the 75 B r i t i s h Columbia school d i s t r i c t s . The o v e r a l l d i s t r i b u t i o n of responses from those school d i s t r i c t s i s shown i n Figure 3.1 (p.51). The d i s t r i b u t i o n of responses based on school d i s t r i c t size i s shown i n Table 3.2 (p.52). The P r i n c i p a l as Formal Evaluator of Teaching Should P r i n c i p a l s Do Evaluation? What i s the Purpose? and How Well i s Evaluation Done? Overwhelmingly p r i n c i p a l s expressed the view that the formal evaluation of teaching should be one of t h e i r r e s p o n s i b i l i t i e s , with 96.8 percent (n=181) saying "Yes" to t h i s question. The remainder indicated that they were "not sure". When asked about what they considered to be the most important purpose of formal teacher evaluation, a much greater difference of opinion emerged. However, i t i s important to note that a small number of respondents (six) made i t clear that they found i t impossible to choose between the two main options: a) teacher growth and development; and b) accountability for the qu a l i t y of teaching (respondents were not given the option of choosing both, see p.36). As Table 5.3 shows, of those who could make t h i s choice, the majority opted for "teacher growth and development", while a substantial minority selected 73 "accountability for the q u a l i t y of teaching". Just 3.3 percent (n=6) indicated some other purpose, which included improving communication between administrators and teachers, and providing an opportunity to celebrate excellence i n teaching. Table 5.3 Evaluation Purpose and Quality Respondents Variable % n Evaluation purpose Growth and development 57.1 104 Accountability 39.6 72 Other 3.3 6 Evaluation done Poorly 6.5 12 Adequately 32.6 60 Well/Very well 60.9 112 When asked about how well they did the formal evaluation of teaching, none of the p r i n c i p a l s who responded defined t h e i r execution of formal evaluation as very poor. A small proportion though, expressed the view that they ca r r i e d out t h i s r e s p o n s i b i l i t y poorly. The response of "adequately" was given by a t h i r d of p r i n c i p a l s but a large majority said they did the formal evaluation of teaching either well or very well. 74 In-service Training, Obstacles, and The Four Phases of Evaluation Results from the question about formal evaluation t r a i n i n g show (Table 5.4), of the four categories described i n the questionnaire (a. one day or less ; b. between two days and one week; c. more than one week but less than one f u l l term; d. one f u l l u n i v e r s i t y or college term), generally, at least half of the respondents to each category indicated no attendance since September 1988. Most t r a i n i n g i s of the "one day or les s " or "between two days and one week" va r i e t y . Courses of more than one week have been attended i n much smaller numbers, while a f i f t h have undertaken courses of one f u l l u n i v e r s i t y or college term since September 1988. Table 5.4 Evaluation Training Attendance Since September 1988 Respondents 1 day 2 days/ Less than 1 week/ 1 term or l e s s 1 week more than 1 term Attendances n=188 n=188 n=188 n=188 None 47.8 51.6 87.2 80.9 One 14.4 17.0 6.4 16.5 Two 11.7 17.0 3.2 2.1 Three or more 26.1 14.4 3.2 .5 Total 100.0 100.0 100.0 100.0 75 In order to have some quantifiable means of describing t o t a l t r a i n i n g per respondent, the t r a i n i n g points formula, explained i n Chapter III (p.44), was applied to the data i n Table 5.4. This produces an average number of points per respondent of 8.7, ranging from 0 to 50 at the extremes. Table 5.5 i l l u s t r a t e s the d i s t r i b u t i o n of t r a i n i n g points across f i v e groupings of "1 to 2"; "3 to 4"; "5 to 9"; "10 or more"; and "None". When grouped i n t h i s way i t can be seen that only 2.7 percent (n=5) of respondents have received no t r a i n i n g i n the formal evaluation of teaching since September 1988. Nearly one t h i r d have 1 to 4 points, while just over a t h i r d have "10 or more" (which corresponds, i n the formula referred to above, to a univ e r s i t y or college term course). Table 5.5 Evaluation Training Points Since September 1988 Respondents Training p o i n t s 3 n Percentage Cumulative % None 1 to 2 3 to 4 5 to 9 10 or more 5 30 25 60 68 2.7 15.9 13.3 31.9 36.2 2.7 18.6 31.9 63.8 100.0 aSee page 44 for the formula used to c a l c u l a t e these points. 76 Despite the somewhat limited t r a i n i n g over the past eight years i n the formal evaluation of teaching, there i s no corresponding sense of t h i s being a problem to the p r i n c i p a l s who responded to the survey. For example, t r a i n i n g was mentioned only 9 (2.5%) times (out of a t o t a l of 362 references) as one of the three most important obstacles to carrying out the formal evaluation of teaching. In addition, the majority of p r i n c i p a l s did not believe they required more t r a i n i n g for any of the four phases of the formal evaluation of teaching, other than the "post- observation conferences" and "report writing" phases of an evaluation leading to a "less than s a t i s f a c t o r y report". The question which asked respondents to l i s t , i n rank order, the three most important obstacles to the formal evaluation of teaching, produced sixteen d i f f e r e n t types of obstacle (including 'other') and 362 in d i v i d u a l respondent references (an additional four respondents said there were no obstacles). Time i s by far the most prominent of the sixteen types of obstacle c i t e d and accounts for two thir d s of a l l f i r s t obstacles. It was also the only obstacle to be c i t e d more than once by the same respondent. These multiple references to time were presumably made to emphasise the importance of time. However, i n Table 5.6, time i s only 77 counted once per respondent who referred to i t , even i f that respondent mentioned i t more than once 4. Table 5.6 i s divided into f i v e columns. The f i r s t column shows the f i v e main categories into which obstacle references could be placed. These categories are "Time", "Process", "Individuality", " P o l i t i c a l context", and "Pr i n c i p a l competence". Each of these categories i s made up of one or more types of obstacle. The category t i t l e i s shown f i r s t , as are the data r e l a t i n g to that category. Thus, the f i r s t l i n e of data represents the aggregate data for a l l the types of obstacle at a p a r t i c u l a r l e v e l of importance within that category. For example, i n the "In d i v i d u a l i t y " category the aggregated percentages for "Teacher non-acceptance", "Stress", and "Purpose not agreed" i n the " F i r s t " column, i s 4.8 percent. The next four columns of Table 5.6 show the data r e l a t i n g to the f i v e categories of obstacle referred to above. The f i r s t of these columns, l a b e l l e d " F i r s t " , i d e n t i f i e s the percentage of respondents who c i t e d a most important obstacle (n=184) i n each of the f i v e obstacle categories. For example, 12.0 percent of respondents who c i t e d a most important obstacle, c i t e d " c o l l e c t i v e agreement" which forms part of the "Process" category. The This i s explained i n d e t a i l i n Chapter VII (p.122), as part of the consideration of time as an obstacle to formal evaluation. Table 5.6 F i r s t , Second, and Third Most Important Obstacles to the Conduct of the Formal Evaluation of Teaching Level of importance F i r s t Second Third Combined n=184 n=116 n=66 n=362a Obstacle % Q. % % Time 65.2 20.7 12.1 42.0 Process 18.0 39.7 31.8 27.6 C o l l e c t i v e agreement 12.0 19.8 18.2 15.7 Process 2.2 14.7 12.1 8.0 C r i t e r i a 2.2 3.4 1.5 2.5 Lack of cycle 1.1 .9 — .8 Cycle .5 .9 — .6 I n d i v i d u a l i t y 4.8 13.8 24.3 11.4 Teacher non-acceptance 3.8 9.5 16.7 8.0 Stress .5 4.3 6.1 2.8 Purpose not agreed .5 — 1.5- .6 P o l i t i c a l context 6.5 9.5 , 16.6 9.7 Union 3.8 4.3 9.1 5.2 D i s t r i c t 2.7 5.2 4.5 3.9 Min i s t r y — — 3.0 .6 P r i n c i p a l competence 2.1 8.5 4.5 4.7 Training 1.1 3.4 4.5 2.5 Subject knowledge — 3.4 — 1.1 Lack of experience .5 1.7 .8 P r i n c i p a l biases .5 — — .3 Other 1.1 7.8 10.6 5.0 None 2.2 — — — Total 99.9 100.0 99.9 100.4 a T h i s figure does not include respondents who sa i d "none". 79 next column gives the percentages for second most important obstacles (n=116) and the next shows percentages for the t h i r d most important obstacles (n=66). The l a s t column combines a l l f i r s t , second and t h i r d most important obstacles (n=362), without including those respondents (n=4) who said there were no obstacles to the evaluation of teaching. The process category accounts for just over a quarter of combined obstacle references, the two most prominent parts of which, are the c o l l e c t i v e agreement and the 'process'. Teacher non-acceptance of the process accounts for a quite a large proportion of the other obstacles, as does the p o l i t i c a l context. P r i n c i p a l competence though, does not feature strongly as an obstacle. The importance of time was further borne out by responses to the question regarding the four phases of formal evaluation (Table 5.7, p.81). In r e l a t i o n to an evaluation leading to a "s a t i s f a c t o r y " report, while two thi r d s f e l t that the pre-evaluation conference was "time- consuming", t h i s rose to three quarters or more for the post-observation conference and classroom observations, and an overwhelming 94.1 percent (n=176) for the writing of the f i n a l report. For evaluations leading to a "less than sa t i s f a c t o r y " report the same pattern emerges but with even higher percentages. Four f i f t h s or more regarded the pre- 80 evaluation conference, post-observation conference, and classroom observations as "time-consuming", with an almost unanimous 98.8 percent (n=85) expressing t h i s view about the f i n a l report writing phase. In addition to "time" and "need for t r a i n i n g " , respondents were asked to express a view on two other factors. These were "stress" (for the p r i n c i p a l ) and "complexity". Both were considered less important than the factor of time, although not so markedly when considering evaluations leading to a "less than s a t i s f a c t o r y " report. Table 5.7 shows that, apart from the report writing phase, p r i n c i p a l stress i s not a major factor i n a formal evaluation leading to a "sa t i s f a c t o r y " report. However, the picture i s very d i f f e r e n t when examining evaluations leading to a "less than s a t i s f a c t o r y " report. The pre-evaluation conference i s considered s t r e s s f u l by over half the p r i n c i p a l s , r i s i n g to nearly two third s for the classroom observations, and over 90 percent for the post-observation conference and f i n a l report writing phase. 81 Table 5.7 Factors Present i n Evaluations Leading to "Sa t i s f a c t o r y " and "Less Than S a t i s f a c t o r y " Reports Percentage of respondents agreeing on presence of factor Stress Complexity Time- Need for consuming t r a i n i n g SR a LTSR b SR LTSR SR LTSR SR LTSR Phase n=185 n=86 n=185 n=86 n=187 n=86 n=182 n=82 Pre-evaluation conference(s) 7.0 54.7 26.5 66.3 62.0 79.1 20.8 35.4 Classroom observations 7.0 62.8 51.6 76.7 84.0 89.5 34.1 43.9 Post-observation conferences 25.9 90.7 55.7 91.9 73.1 87.2 35.6 57.3 Writing the f i n a l report 41.1 91.9 75.7 94.2 94.1 98.8 37.2 59.0 a E v a l u a t i o n leading to a " s a t i s f a c t o r y " report. ^Evaluation leading to a "less than s a t i s f a c t o r y " report. Complexity i s also a r e l a t i v e l y unimportant factor when compared to time, although a l i t t l e over half or more of pr i n c i p a l s agree that a l l the phases of an evaluation leading to a "sat i s f a c t o r y " report, apart from the pre- evaluation conference, are complex. As with the other factors, the f i n a l report writing phase receives most agreement with three quarters believing i t to be complex. 82 "Less than s a t i s f a c t o r y " reports are viewed as more complex, with sizeable, i f not substantial, majorities taking t h i s view about a l l four phases, the most s t r i k i n g being the f i n a l report phase. Number of Evaluations and "Less Than Satisfactory" Reports The numbers of formal evaluations of teaching c a r r i e d out by p r i n c i p a l s since September 1988 varied considerably but could be c l a s s i f i e d into four main groups: 1 to 9 evaluations; 10 to 19; 20 to 29; and 30 or more. Table 5.8 shows that a t h i r d of p r i n c i p a l s have done 10 to 19 evaluations, with just over a f i f t h of p r i n c i p a l s f a l l i n g into each of the other three categories. When asked i f they had written a "less than s a t i s f a c t o r y " report i n t h i s period, nearly two thi r d s said they had not. A further quarter have written only one and just 13.5 percent (n=25) have written two or more "less than s a t i s f a c t o r y " reports since September 1988. 83 Table 5.8 Evaluations Conducted and "Less Than S a t i s f a c t o r y " Reports Written Since September 1988 Respondents Variable Q. ~o n Evaluations conducted 1 to 9 22.8 41 10 to 19 32.8 59 20 to 29 21.1 38 30 or more 23.3 42 "Less than s a t i s f a c t o r y " reports written None 61.6 114 One 24.9 46 Two or more 13.5 25 When the data for evaluations conducted and "less than s a t i s f a c t o r y " reports written are aggregated, i t results i n a t o t a l of 110 "less than s a t i s f a c t o r y " reports out of a t o t a l of 3,832 evaluations conducted since September 1988. This means that a "less than s a t i s f a c t o r y " report i s written, on average, once i n every 34.8 evaluations or, put another way, 2.9 percent of a l l evaluations result i n a "less than s a t i s f a c t o r y " report. The t o t a l years of pr i n c i p a l s h i p which a l l respondents have between them i s 84 1,2365. Therefore an average of 3.1 evaluations have been written per year of p r i n c i p a l s h i p over a l l respondents i n the period since September 1988. Additional Comments Made by Respondents The f i n a l question on the questionnaire, number 21, asked respondents i f there was anything they wished to add with regard to formal evaluation. Of the 188 p r i n c i p a l s that responded to the survey, 116 (61.7%) chose to take advantage of t h i s opportunity. These anecdotal data range i n length from one or two sentences to several paragraphs. The f i r s t column of Table 5.9 shows, the f i v e broad themes which emerged from the analysis of these data: "Evaluation purpose", "Inadequate process", " A b i l i t y to evaluate", "Evaluator attitudes", and " P o l i t i c a l context". Within these f i v e broad themes are p a r t i c u l a r types of reference. For example, the theme of "Inadequate process" i s made up of three types of reference. The second column of Table 5.9 shows the number of times each type of reference was made. Because no p r i n c i p a l made the same type of reference more than once, the number of respondents and the number of references are equal. The numbers i n bold type are the aggregate references for that -•Total years of p r i n c i p a l s h i p are based on exact years of exper ience g i v e n i n answer to ques t i on 7 on the q u e s t i o n n a i r e . 85 Table 5.9 Anecdotal Responses Thematic categories Number of No. As a % of References 3 a l l references 1. Evaluation purpose 91 35.5 a. Reserved for poor teachers 16 6.3 b. Growth and development 37 14.6 c. Accountability 18 7.0 d. LTSR b ineffectiveness 20 7.8 2. Inadequate process 60 23.4 a. Generally unsatisfactory 40 15.6 b. Need for peer evaluation 8 3.1 c. Reference to c y c l e / c r i t e r i a 12 4.7 3. A b i l i t y to evaluate 43 16.8 a. Time factor 27 10.5 b. Competence/resolve 16 6.3 4. Evaluator at t i t u d e s 41 16.0 a. Important leadership r o l e 19 7.4 b. P o s i t i v e experience 8 3.0 c. S t r e s s f u l a c t i v i t y 9 3.5 d. Promotes administrator/ teacher understanding 5 2.0 5. P o l i t i c a l context 14 5.5 a. Union/District hindrance 14 5.5 Other 6 2.7 Total 256 99.9 aDoes not include 'other' and i s equal to the number of people who made such a reference. "Less than s a t i s f a c t o r y " report. 86 theme. For example, the three types of reference under "Inadequate process" t o t a l 60 i n d i v i d u a l references. The t h i r d column shows the number of references as a percentage of a l l references made. For example, of the t o t a l of 256 in d i v i d u a l references made i n response to question 21, 60 can be categorised under the theme of "Inadequate process", which represents 23.4 percent of a l l references. The largest number of references (35.5%) relate to the purpose of evaluation. Within t h i s category, apart from "Growth and development" and "Accountability" highlighted i n question 14 (see appendix A.3), 16 respondents (6.3%) suggested formal evaluation of teaching should be reserved for use with poor teachers or those about which the p r i n c i p a l already had cause for concern. Respondent 047 provides a f a i r l y t y p i c a l example when saying "We should re- think the system. The formal evaluation should be reserved for only the 'less than s a t i s f a c t o r y ' teachers." Some respondents express the view that where "less than s a t i s f a c t o r y " reports are written they f a i l to achieve very much. There were 20 (7.8%) such references of which the following i s representative: "[Evaluation] must be focussed on growth - but must be an e f f e c t i v e t o o l i n dismissal when that becomes necessary. I have never heard that teacher evaluations r e s u l t i n g i n 'less than s a t i s f a c t o r y ' reports 87 have been an e f f e c t i v e tool i n dismissing s t a f f " (respondent 122, author's emphases). A further i l l u s t r a t i o n of the d i f f i c u l t i e s some pr i n c i p a l s associate with "less than s a t i s f a c t o r y " reports i s provided by respondent 146: "Less than s a t i s f a c t o r y " evaluations are more s t r e s s f u l because there i s a l l the f a l l o u t - denial, accusation, union grievance, etc...More "less than s a t i s f a c t o r y " reports need to be written, I believe, but the hassles scare admin, o f f . They are intimidated and don't f e e l they can c a l l a spade a spade." The next largest category i s "Inadequate process" which accounts for 60 (23.4%) references. The majority (n=40) expressed a general d i s s a t i s f a c t i o n with the process and also, at times, admitted to a sense of i s o l a t i o n or powerlessness which was echoed to an extent i n a l l the other major categories. The following two extracts give a flavour of the responses i n t h i s regard. The f i r s t i s given by respondent 006 who said: "The area of reporting on the 'marginal teacher' i s the most d i f f i c u l t of a l l . The data i s harder to gather, the teacher i s often immune to professional growth options and the evaluator i s unsure which d i r e c t i o n to go." The second extract comes from respondent 007 who asserts: "The formal evaluation process as i t presently exists i n B.C. i s outdated, s t r e s s f u l , time-consuming, but most importantly ( i n most cases) a t o t a l l y i rrelevant exercise...A.O.'s are i n the embarrassing p o s i t i o n of tryi n g to le g i t i m i z e an a c t i v i t y ( i n i t s present form) that we a l l know i s 'hoop jumping'" 88 In " A b i l i t y to evaluate", time i s referred to on 27 (10.5%) occasions, while the competence and resolve of pr i n c i p a l s to undertake the role of formal evaluator i s mentioned 16 times (6.3%). Competence was referred to i n a number of d i f f e r e n t ways including both p o s i t i v e and negative statements about t r a i n i n g , doubts about the v a l i d i t y of the results an evaluator had produced, lack of s u f f i c i e n t subject knowledge, and simply whether or not the evaluator was doing a good enough job. Two examples include respondent 136 who, af t e r explaining that she had only given "excellent" or "very good" ratings, went on to say "I know that the teachers I rate as Excellent deserve i t but I wonder i f I am right to give so many such a high rati n g . " A second respondent, 187, stated formal evaluation was not an area of concern for her after saying "I f e e l very well trained by my univer s i t y courses, D i s t r i c t inservice and mentoring programs, working with my p r i n c i p a l when I was a VP and Supervising S k i l l s Workshops." Within "Evaluator attitudes", mentioned 41 times (16.0%), the largest sub-category was 'Important leadership r o l e ' with 19 (7.4%) references. These generally t e s t i f y to the b e l i e f that the p r i n c i p a l i s an i n s t r u c t i o n a l manager and the role of formal evaluator i s central to the whole raison d'etre of schools and public education. For example, respondent 122 said "This process can greatly help teachers 89 - b u t . . . [ i t i s ] less valuable than i t ought to be. Solution? = Increased admin time - p r i n c i p a l focus on i n s t r u c t i o n a l leadership not building management.", while respondent 073 asserted that "Instructional leadership i s 'the' most important aspect of our job." F i n a l l y , the ' p o l i t i c a l ' context within which the formal evaluation process must take place was referred to. A l l of these references (n=14), with additional comments about the school board i n three of them, were directed at the hindrance of the B r i t i s h Columbia Teachers' Federation. An i n d i c a t i o n of the feelings expressed i s given i n the comment made by respondent 185 who said "I f i n d i t f r u s t r a t i n g that the Union protects those individuals that c l e a r l y tarnish the reputation of the profession and injure the children we are charged to teach." Summary Around three quarters of respondents to the questionnaire are male and a quarter female. The majority of respondents are between the ages of f o r t y - f i v e and f i f t y - four and most have a master's degree i n Educational Administration. The majority of respondents have between one and ten years of experience as a p r i n c i p a l . Over half the p r i n c i p a l s i n t h i s study have a teaching assignment and they predominantly administer elementary schools with just 90 under a quarter running secondary schools. Nearly two thi r d s have s t a f f s of twenty teachers or more. Overwhelmingly, p r i n c i p a l s i n t h i s study believe they should do formal evaluation and the majority consider the most important purpose of evaluation to be teacher growth and development. They also consider t h i s to be a role they carry out well. The vast majority of respondents have had some recent t r a i n i n g i n the formal evaluation of teaching but few have had extensive t r a i n i n g . Only one f i f t h have undertaken u n i v e r s i t y or college courses with a component on evaluation, since September 1988. Lack of t r a i n i n g does not feature prominently amongst the obstacles to evaluation. By far the most important obstacle i s time. Nearly two t h i r d s of p r i n c i p a l s express t h i s view. The process i s also highlighted i n d i f f e r e n t forms but to a lesser extent than time. Time i s also the most c i t e d factor i n the four phases of a formal evaluation. In the responses about these four phases, the perceptions of p r i n c i p a l s i n r e l a t i o n to evaluations leading to a "satisfactory" report are quite d i f f e r e n t from those leading to a "less than s a t i s f a c t o r y " report. The l a t t e r are considered to be more time-consuming, more s t r e s s f u l and more complex. Few evaluations are written, 3.1 per year of p r i n c i p a l s h i p , and just 2.9 percent re s u l t i n "less than sa t i s f a c t o r y " reports. F i n a l l y , the general data described i n t h i s chapter provide the basis for a more detailed description i n the following two chapters. In Chapter VII the three concepts of purpose, need for further t r a i n i n g , and obstacles give the structure for organising the presentation of findings. Prior to consideration of these three concepts however, introductory data are provided i n Chapter VI i n order to draw a ' p r o f i l e ' of each of two respondent groups based on sex, and experience as a p r i n c i p a l . These p r o f i l e s give a b r i e f description of how well each group considers they do the formal evaluation of teaching, followed by other data gathered from the questionnaire about age, master's specialty, d i s t r i c t size, type of school, s t a f f s i z e , percentage of teaching, and amount of evaluation t r a i n i n g received. 92 CHAPTER VI Sex of P r i n c i p a l and Years of Experience as P r i n c i p a l In t h i s chapter respondents are categorised on the basis of two variables which emerged from the l i t e r a t u r e : Sex of p r i n c i p a l ; and experience as a school p r i n c i p a l . These variables highlight some in t e r e s t i n g differences among p r i n c i p a l s but are also intended to provide f o c i to the description of data i n Chapter VII, with regard to the concepts of purpose, t r a i n i n g , and obstacles. Sex of p r i n c i p a l was chosen because gender differences i n educational administration are claimed i n the l i t e r a t u r e and because of the very human in t e r a c t i v e nature of the formal evaluation process. Experience presented i t s e l f as an i n t e r e s t i n g variable since i t i s rar e l y mentioned i n the l i t e r a t u r e . It might be expected though, that a manager with greater experience would be more practised i n the conduct of p o t e n t i a l l y d i f f i c u l t tasks, such as formal evaluation, than t h e i r less experienced colleagues. The following data therefore present two ' p r o f i l e s ' which include how well p r i n c i p a l s consider they carry out formal evaluation, age, master's specialty, school d i s t r i c t s i z e , type of school, s t a f f s i z e , teaching load, and tr a i n i n g undergone. 9 3 Sex of P r i n c i p a l Table 6.1 shows that l i t t l e difference e x i s t s i n the self-evaluation by male and female respondents as to how well they do formal evaluation. However, male p r i n c i p a l s are more l i k e l y to describe themselves as poor evaluators and female p r i n c i p a l s are more l i k e l y describe themselves as doing formal evaluation either well or very well. Table 6.1 Sex of P r i n c i p a l by Evaluation Quality, Age, and Master's Specialty Percentage of respondents Variable Male Female n=134 n=49 Evaluation done Poorly 8.2 2.0 Adequately 32.8 30.6 Well/Very well 59.0 67.3 n=136 n=51 Age 44 or les s 18.4 29.4 45 to 49 36.8 29.4 50 to 54 25.0 25.5 55 or over 19.9 15.7 n=122 n=47 Master's s p e c i a l t y Administration 67.2 61.7 Curriculum 13.1 19.1 Other 3 19.7 19.1 aSee page 69 for a l i s t of the areas covered by these degrees. 94 The major difference i n age d i s t r i b u t i o n for male and female respondents occurs i n the "44 years or l e s s " category which has a considerably larger proportion of females than males, while the proportions i n the other age categories are somewhat closer. Male p r i n c i p a l s are more l i k e l y to hold a master's degree i n "Educational Administration", while degrees i n "Curriculum" are more l i k e l y to be held by females. Proportions are si m i l a r for degrees i n "other" f i e l d s . Table 6.2 shows that a much larger percentage of female p r i n c i p a l s work i n large school d i s t r i c t s than do males (p<.05), while a larger percentage of males work i n medium and small d i s t r i c t s , although the difference i s not s t a t i s t i c a l l y s i g n i f i c a n t . A s t a t i s t i c a l l y s i g n i f i c a n t difference (p<.05) exists between the proportions of male and female p r i n c i p a l s i n elementary and secondary schools. Elementary schools are more l i k e l y to be administered by women, while p r i n c i p a l s of secondary schools are more l i k e l y to be men. In the case of schools that enrol both elementary and secondary students, the percentages are the same. 95 Table 6.2 Sex of P r i n c i p a l by School D i s t r i c t Size, School Type, St a f f Size, and Teaching Load Percentage of respondents Variable Male Female n=131 n=51 School d i s t r i c t s i z e Large 45.9 64.7 Medium 36.1 25.5 Small 18.0 9.8 n=136 n=51 Type of school* Elementary grades 66.9 84.3 Secondary grades 27.2 9.8 Both 5.9 5.9 n=135 n=49 Teaching s t a f f 1 to 9 8.9 6.1 10 to 19 28.1 34.7 20 to 29 31.9 34.7 30 or more 31.1 24.5 n=134 n=50 Percentage teaching Zero 42.5 50.0 1 to 19 19.4 12.0 20 to 39 25.4 14.0 40 or more 12.7 24.0 *£ < .05. Staf f s izes for male and female respondents are s i m i l a r . However, a c l ear percentage d i f ference ex i s t s i n the teaching time male and female p r i n c i p a l s have as a part of t h e i r assignment. A larger percentage of female respondents have a 100 percent adminis trat ion assignment. 96 but t h i s i s also true for teaching time of 40 percent or more. In the two intervening categories of "1 to 19 percent" and "20 to 39 percent" teaching time, men are represented i n markedly larger proportions than are women. Table 6.3 below shows t r a i n i n g received by male and female respondents since September 1988. Table 6.3 Sex of P r i n c i p a l by Evaluation Training Since September 1988 Percentage of respondents Training duration and number of attendances Male n=136 Female n=51 1 day or l e s s * None One Two Three or more 2 days/1 week None One Two Three or more Less than 1 week/ more than 1 term None One or more 1 term None One or more 47.8 10.3 13.2 28.7 48.6 17.6 19.1 14.7 88.2 11.8 81.6 18.4 47.1 25.5 7.8 19.6 60.7 15.7 11.8 11.8 84.3 15.7 78.4 21.6 *p < .05. 97 A great s i m i l a r i t y exists between male and female respondents with regard to t r a i n i n g received and the only s t a t i s t i c a l difference occurs with courses of one day or less (p<.05): women are represented i n larger percentages for "one attendance" and men are represented i n larger percentages for "two attendances" and "three or more attendances". The average t r a i n i n g points (see p.44) for males and females are almost i d e n t i c a l at 8.7 and 8.6 respectively. Table 6.4 provides a description of the d i s t r i b u t i o n of tr a i n i n g points among male and female p r i n c i p a l s . At the lower end of the points scale the percentage of females i s noticeably larger than for males. For example, over one Table 6.4 Sex of P r i n c i p a l by Evaluation Training Points Since September 1988 Male 3 Female' Training points n=136 n=51 g, *5 Cum. % % Cum. % None 1 to 2 3 to 4 5 to 9 10 or more 2.9 14.0 12.5 35.3 35.3 2.9 16.9 29.4 64.7 100.0 2.0 21.5 15.7 21.6 39.2 2.0 23.5 39.2 60.8 100.0 aAverage number of points = 8.7 ^Average number of points = 8.6 98 f i f t h of female p r i n c i p a l s have "1 to 2" t r a i n i n g points (which equates d i r e c t l y to 1 to 2 days), whereas t h i s applies to only 14 percent (n=19) of male p r i n c i p a l s . At the "3 to 4" points l e v e l , there are somewhat over one t h i r d of females and one quarter of males. Years of Experience as P r i n c i p a l Respondents were categorised into four groups based on t h e i r number of years of experience as a p r i n c i p a l . Table 6.5 shows no clear pattern with regard to how well these experience groups consider they carry out the formal evaluation of teaching. However, quite a large proportion of p r i n c i p a l s with 11 to 15 years of experience say they carry out evaluation poorly, while a r e l a t i v e l y small proportion with 16 years or more experience say t h i s . The age d i s t r i b u t i o n of p r i n c i p a l s when grouped by years of experience follows the predictable pattern that younger p r i n c i p a l s tend to have less experience (p<.05). With regard to master's degree spe c i a l t y , there i s a marked difference between p r i n c i p a l s with 1 to 10 years of experience and those with 11 or more years of experience. The more experienced p r i n c i p a l s have an administration speci a l t y i n higher percentages than t h e i r less experienced counterparts, with a corresponding difference i n curriculum speci a l t y . Master's degree s p e c i a l t i e s i n "other" f i e l d s 99 are held by approximately one f i f t h of p r i n c i p a l s i n a l l the experience categories, apart from "11 to 15 years". Table 6.5 P r i n c i p a l Experience by Evaluation Quality, Age, and Master's Specialty Percentage of respondents Variable 1-5 years 6-10 years 11-15 years 16+ years n=57 n=56 n=24 n=46 Evaluation done Poorly 5.3 7.1 16.7 2.2 Adequately 33.3 32.1 25.0 34.8 Well/Very well 61.4 60.7 58.3 63.0 n=58 n=58 n=24 n=49 Age* 44 or less 39.7 25.9 8.3 — 45 to 49 43.1 41.4 37.5 14.9 50 to 54 17.2 19.0 33.3 38.3 55 or over — 13.8 20.8 46.8 n=56 n=54 n=20 n=39 Master's s p e c i a l t y Administration 60.7 57.4 85.0 74.4 Curriculum 17.9 20.4 5.0 7.7 Other 21.4 22.2 10.0 17.9 *p < .05. Table 6.6 presents the data on school d i s t r i c t s i z e , type of school, teaching s t a f f , and teaching load. The data regarding experience and school d i s t r i c t size reveal no obvious pattern other than declining percentages i n each experience group from large to small d i s t r i c t s . Quite wide differences exist i n the percentages of each experience group that work i n each size of d i s t r i c t . 100 Differences do exist i n the types of school administered by p r i n c i p a l s categorised by experience but they are not s i g n i f i c a n t . The percentage of "16+ years" p r i n c i p a l s that administer elementary schools i s larger than the other three groups, while they are represented i s much smaller proportions i n secondary schools. Table 6.6 P r i n c i p a l Experience by School D i s t r i c t Size, School Type, Sta f f Size, and Teaching Load Percentage of Respondents Variable 1 -5 years 6-10 years 11-15 years 16+ years School d i s t r i c t s i z e n=58 n=56 n=24 n=46 Large 55.2 58.9 37.5 43.5 Medium 32.8 28.6 33.3 39.1 Small 12.1 12.5 29.2 17.4 Type of school n=58 n=58 n=24 n=47 Elementary grades 70.7 67.2 66.7 80.9 Secondary grades 24. 1 29.3 25.0 10.6 Both 5.2 3.4 8.3 8.5 Teaching s t a f f n=55 n=58 n=24 n=47 1 to 9 14.5 3.4 8.3 6.4 10 to 19 36.4 31.0 20.8 25.5 20 to 29 27.3 29.3 37.5 40.4 30 or more 21.8 36.2 33.3 27.7 Percentage teaching n=57 n=57 n=24 n=46 Zero 38.6 45.6 45.8 50.0 1 to 19 17.5 21.1 12.5 15.2 20 to 39 17.5 22.8 29.2 23.9 40 or more 26.3 10.5 12.5 10.9 101 No cl e a r pattern emerges for s t a f f s i z e s . However, p r i n c i p a l s with 1 to 5 years of experience are more heavily represented i n schools with s t a f f s between 1 and 19, while they represent the lowest percentages for schools with s t a f f s of 20 or more. Absence of pattern i s c e r t a i n l y not the case with regard to the teaching r e s p o n s i b i l i t i e s of p r i n c i p a l s within these experience groups. With greater experience comes a greater l i k e l i h o o d of an assignment which consists of 100 percent administration, although p r i n c i p a l s with 11 or more years of experience are s t i l l represented i n sizeable proportions i n the "20 to 39 percent" teaching load category. P r i n c i p a l s with 1 to 5 years experience account for the lowest percentage among experience groups with f u l l administration assignments and the highest percentage with assignments carrying a teaching load of "40 percent or more". Table 6.7 shows the predictable finding that "16+ years" p r i n c i p a l s have attended u n i v e r s i t y or college courses i n very small percentages since September 1988 (p<.05). For courses of "one day or l e s s " and "two days to one week", the percentages for one or more attendances increases as experience increases. Also, much larger percentages of p r i n c i p a l s with 16 or more years experience have attended three or more courses of "one day or l e s s " . However, t r a i n i n g points averages are very s i m i l a r . These 102 averages are 8.6 for p r i n c i p a l s with 1 to 5 years experience, 8.7 for both the "6 to 10 years" and "11 to 15 years" experience groups, and 8.8 for those p r i n c i p a l s with 16 years experience or more. Table 6.7 P r i n c i p a l Experience by Evaluation Training Since September 1988 Training duration and number of attendances Percentage of respondents 1-5 years 6-10 years 11-15 years 16+ years n=58 n=58 n=24 n=47 1 day or less None One Two Three or more 2 days/1 week None One Two Three or more More than 1 week/ less than 1 term None One or more 1 term* None One or more 58.6 12.1 13.8 15.5 58.7 15.5 10.3 15.5 86.2 13.8 72.4 27.6 48.3 15.5 10.3 25.9 55. 1 12.1 20.7 12.1 89.7 10.3 77.6 22.4 41 .7 20.8 16.7 20.8 45.5 29.2 20.8 4.2 83. 16. 75.0 25.0 36.1 12.8 8.5 42.6 42.7 19.1 19.1 19.1 87.2 12.8 97.9 2.1 *£ < . 0 5 . 103 While average t r a i n i n g points may be very similar, Table 6.8 shows that the d i s t r i b u t i o n of points i s not. For example, for "5 to 9" points, there i s a f a i r l y steady increase i n the percentage of respondents as experience increases. Also, the largest percentage without any tr a i n i n g i n formal evaluation, at 6.9 percent (n=4), i s i n the "1-5 years" category. Table 6.8 P r i n c i p a l Experience by Evaluation Training Points Since September 1988 1-5 y e a r s 3 6-10 years 1 3 10-15 y e a r s c 16 + y e a r s d Training points n = 58 n = 58 n=24 n =47 % Cum. % o. "6 Cum. % % Cum.% o ~o Cum. % None 6.9 6.9 1.7 1.7 0.0 0.0 0.0 0.0 1 to 2 13.8 20.7 15.5 17.2 20.8 20.8 17.0 17.0 3 to 4 12.1 32.8 12.1 29.3 12.5 33.3 17.0 34.0 5 to 9 25.8 58.6 32.8 62.1 33.4 66.7 36.2 70.2 10 + 41.4 100.0 37.9 100.0 33.3 100.0 29.8 100.0 aAverage number of points = 8.6 ^Average number of points = 8.7 cAverage number of points = 8.7 ^Average number of points = 8.8 Summary Amongst p r i n c i p a l s categorised by sex, a s t a t i s t i c a l difference exists with type of school administered where women are also over represented i n elementary schools while for males t h i s i s true i n secondary schools. Females are represented to a disproportionately greater extent i n large 104 d i s t r i c t s , while t h i s i s true for males i n medium and small d i s t r i c t s . Teaching load data shows females represented i n higher proportions among p r i n c i p a l s with a 100 percent administration assignment but also for assignments with a teaching load of 40 percent or more. The p r o f i l e s for male and female p r i n c i p a l s with regard to how well they consider they do evaluation, age, master's specialty, numbers of teaching s t a f f , and t r a i n i n g attendance are a l l s i m i l a r . P r i n c i p a l s with 11 or more years of experience are represented i n larger percentages among respondents with a master's degree i n Educational Administration, i n medium and small school d i s t r i c t s , and larger s t a f f s i z e s . P r i n c i p a l s i n the "16+ years" category are also more heavily represented i n elementary schools. With regard to teaching load, the pattern emerges of more experienced p r i n c i p a l s having larger administration assignments than t h e i r less experienced colleagues. Age d i s t r i b u t i o n and uni v e r s i t y or college attendance since September 1988 are s i g n i f i c a n t l y d i f f e r e n t but t h i s i s to be expected because the older p r i n c i p a l s are more l i k e l y to have attended before t h i s date. No pattern was i d e n t i f i e d for how well p r i n c i p a l s with d i f f e r e n t l e v e l s of experience consider they do evaluation. 105 CHAPTER VII Purpose, Training, and Obstacles Three concepts have driven t h i s research from i t s inception through to the analysis. The f i r s t concept i s 'most important purpose' of formal evaluation. As part of purpose, a further element, whether or not "less than s a t i s f a c t o r y " reports have been written, i s also examined. These data provide information about the product of evaluation and therefore may cast further l i g h t on the purposes p r i n c i p a l s have i n mind when they formally evaluate teaching. The second concept i s the 'need for further t r a i n i n g ' . The t h i r d concept i s 'obstacles to carrying out formal evaluation'. Examination of these concepts provides a clearer understanding of why p r i n c i p a l s evaluate and how far they have the preparation and opportunity to evaluate competently. 'Second t i e r ' variables are also selected, where t h i s i s considered appropriate, i n addition to the variables of sex and years of experience described i n Chapter VI. Thus, since the existence or otherwise of evaluation c r i t e r i a may have a bearing on evaluation purpose, t h i s variable i s included i n the consideration of purpose. S i m i l a r l y , the existence or otherwise of an evaluation cycle, the size of s t a f f , and the r a t i o of administration and teaching, may have some determining e f f e c t on the amount of time required 106 or available for evaluation. Therefore, these variables are included i n the consideration of time as an obstacle. Thirdly, the variables of master's spec i a l t y and t r a i n i n g already received may influence the additional t r a i n i n g p r i n c i p a l s believe they need. Therefore, these variables are considered with regard to the theme of further t r a i n i n g required. The i n c l u s i o n of the two d i s t i n c t i o n s : a) between p r i n c i p a l s i n d i s t r i c t s with and without evaluation c r i t e r i a ; and b) between p r i n c i p a l s i n d i s t r i c t s with and without evaluation cycles; emerges from Chapter IV (see p.59-63). For the purposes of analysis, i n t h i s and the following chapter, an assumption i s made that c o l l e c t i v e agreements which s t i p u l a t e a frequency of one formal evaluation "at least every" stated number of years, are un l i k e l y to produce more than one evaluation per member of teaching s t a f f i n that period of time. Therefore, t h i s category has been amalgamated with that of "every" stated number of years. This produces three evaluation cycle types: a) "no cycle"; b) "every/at l e a s t " ; and c) "not more than". Hence, p r i n c i p a l s c l a s s i f i e d by cycle provision are referred to i n the following text as "no cycle", "every/at least", or "not more than" p r i n c i p a l s , and those c l a s s i f i e d by c r i t e r i a provisions are referred to as "no c r i t e r i a " and " c r i t e r i a " p r i n c i p a l s . 107 This chapter generally involves a b i v a r i a t e analysis but on occasions employs multivariate analysis i n order to provide a more sophisticated form of data upon which to base explanations. F i n a l l y , the consideration of the the findings i n t h i s chapter, as with Chapters IV, V, and VI, i s l e f t u n t i l the discussion i n Chapter VIII, where a l l the data gathered i n t h i s study i s drawn together. Evaluation Purpose Both the l i t e r a t u r e and the data presented i n Chapter V highlighted a dichotomy of purpose between teacher growth and development and accountability for the q u a l i t y of teaching. This dichotomy was c a l l e d into question by Poster and Poster (1993) and Sergiovanni (1991) as well as a small number of respondents to the survey who said they were unable to make a choice between these two purposes. However, Table 5.9 (p.85) showed that the largest proportion of anecdotal responses to the survey could be defined under "purpose" and that within t h i s category p r i n c i p a l s continued to d i s t i n g u i s h between growth and accountability. The following anecdotal responses i l l u s t r a t e the mixture of views which p r i n c i p a l s have with regard to the issue of purpose. These views ranged from one p r i n c i p a l who stated that "In t h i s d i s t r i c t a teacher can get a s a t i s f a c t o r y report i n the f i r s t year and never be evaluated 108 again. It's time for a re-focus of purpose." (respondent 047), to another who, aft e r r e f e r r i n g to the purpose of evaluation as personal growth for the teacher, went on: "Samuel Johnson said 'The applause of a single human being i s of great consequence'" (respondent 101), suggesting that an important role of the p r i n c i p a l i n evaluation i s to encourage. The views that these two examples represent found, on occasions, expression i n other comments such as that from respondent 072 who said: "I fir m l y believe i n the more formative, growth oriented philosophy. However, we remain the 'gatekeepers' at t h i s point. I'm not e n t i r e l y convinced that both roles are compatible." F i n a l l y , t h i s dichotomy, reinforced by other data from question 14 (about the purpose of evaluation, see Appendix A.3), was encapsulated by a fourth respondent (138) who asserted: "Question 14, above, gets to the heart of the current dilemma." Table 7.1 shows a s t a t i s t i c a l l y s i g n i f i c a n t difference (p<.05) between male and female p r i n c i p a l s i n r e l a t i o n to the data about the most important purpose of the formal evaluation of teaching. While just over half the male respondents to the questionnaire indicated that the most important purpose was teacher growth and development, nearly three quarters of the female respondents chose t h i s option. A corresponding difference e x i s t s for the option of accountability for the qu a l i t y of teaching. Table 7.1 Sex of P r i n c i p a l by Evaluation Purpose and "Less Than S a t i s f a c t o r y " Reports Percentage of respondents Variable Male Female Evaluation purpose Growth and development Accountability Other n=132 52.3 45.5 2.3 n=49 71.4 22.4 6.1 "Less than s a t i s f a c t o r y " reports written None One Two or more n=133 61.7 21.8 16.5 n=51 62.7 31.4 5.9 A further possible i n d i c a t i o n of purpose i s the propensity to write "less than s a t i s f a c t o r y " reports. The above data reveal that a s i m i l a r proportion of male and female p r i n c i p a l s have never written such a report. However, a somewhat larger percentage of women than men have written one "less than s a t i s f a c t o r y " report since September 1988. The pos i t i o n i s reversed for two or more "less than s a t i s f a c t o r y " reports, where male respondents are i n the majority. However, the larger percentage of male p r i n c i p a l s 110 who have written multiple "less than s a t i s f a c t o r y " reports can be accounted for on the grounds of experience. Most females have been p r i n c i p a l s for ten years or less and t h i s experience group i s responsible for fewer multiple "less than s a t i s f a c t o r y " reports. When "less than s a t i s f a c t o r y " reports written are cross tabulated against the most important purpose for males and females, no s i g n i f i c a n t relationship between male and female respondents i s found. If the number of "less than s a t i s f a c t o r y " reports as a proportion of a l l evaluations written i s compared between males and females, i t shows that 2.8 percent (87 of 3064) of the reports written by men have been "less than s a t i s f a c t o r y " , while t h i s figure i s 3.0 percent (23 of 768) for women. These data are placed i n a more meaningful context when the frequency of evaluations per year of p r i n c i p a l s h i p since September 1988 i s calculated. These data indicate that male p r i n c i p a l s have conducted 3.3 evaluations per year (3064 i n 940 p r i n c i p a l years) i n the above period compared to 2.6 evaluations (768 i n 296 p r i n c i p a l years) by female p r i n c i p a l s . Therefore, women p r i n c i p a l s conduct fewer evaluations per year than men, of which a s l i g h t l y larger proportion re s u l t i n "less than s a t i s f a c t o r y " reports than men. When p r i n c i p a l s are categorised by experience they divide quite noticeably into two 'sub-groups' of 1 to 10 I l l years and 11 years or more, with regard to the most important purpose of formal evaluation. Table 7.2 shows around two t h i r d s of p r i n c i p a l s with 1 to 10 years experience say the most important purpose i s teacher growth and development, while around half of the more experienced p r i n c i p a l s take t h i s view. This i s accompanied by a corresponding difference i n responses in d i c a t i n g the most important purpose i s the accountability for the q u a l i t y of teaching. A prominent (though somewhat predictable) fact to emerge from the data about "less than s a t i s f a c t o r y " reports i s that nearly three quarters of the p r i n c i p a l s with 1 to 5 years of experience have never written such a report. However, they are represented i n s i m i l a r proportions to the other three experience groups for p r i n c i p a l s having written one "less than s a t i s f a c t o r y " report. The "11 to 15 years" group i s also i n t e r e s t i n g because half of a l l these p r i n c i p a l s have written a "less than s a t i s f a c t o r y " report, and nearly a t h i r d have written two or more. This indicates a greater tendency to have written more than one "less than s a t i s f a c t o r y " report than the other three experience groups. However, when a l l experience groups and "less than s a t i s f a c t o r y " reports are cross tabulated there i s no s t a t i s t i c a l s i g n i f i c a n c e . 112 Table 7.2 P r i n c i p a l Experience by Evaluation Purpose and "Less Than S a t i s f a c t o r y " Reports Percentage of respondents Variable 1-5 years 6-10 years 11-15 years 16+ years n=55 n=56 n=24 n=46 Evaluation purpose Growth and development 63.6 62.5 50.0 47.8 Accountability 32.7 33.9 50.0 47.8 Other 3.6 3.6 -- 4.3 n=58 n=58 n=24 n=44 "Less than s a t i s f a c t o r y " reports written None 72.4 58.6 50.0 59.1 One 24.2 24.2 20.8 27.3 Two or more 3.4 17.2 29.2 13.6 When "less than s a t i s f a c t o r y " reports are calculated as a proportion of t o t a l reports written, the "11 to 15 years" group has the highest percentage at 3.6 percent (24 of 673), followed by "6 to 10 years" at 3.1 percent (38 of 1238), "1 to 5 years" at 2.6 percent (18 of 693), and f i n a l l y "16+ years" at 2.4 percent (30 of 1228). P r i n c i p a l s with 1 to 5 years experience have the highest average number of evaluations per year of p r i n c i p a l s h i p since September 1988, at 4.4 (693 i n 158 p r i n c i p a l years). P r i n c i p a l s with 11 to 15 years experience have written 3.1 (673 i n 216 p r i n c i p a l years), those with 16 or more years have written 2.9 (1228 i n 423 p r i n c i p a l years) and the "6 to 10 years" group have 113 the lowest average at 2.8 evaluations per year (1238 i n 439 p r i n c i p a l years) since September 1988. Table 7.3 shows that p r i n c i p a l s i n d i s t r i c t s with c r i t e r i a are more l i k e l y to opt for accountability for the qua l i t y of teaching than are p r i n c i p a l s i n d i s t r i c t s without c r i t e r i a . A correspondingly higher percentage of p r i n c i p a l s i n d i s t r i c t s without c r i t e r i a express the view that teacher growth and development i s the most important purpose. Table 7.3 also shows a s t a t i s t i c a l l y s i g n i f i c a n t difference (p<.05) between p r i n c i p a l s categorised by evaluation c r i t e r i a that have written two or more "less than s a t i s f a c t o r y " reports. One f i f t h of p r i n c i p a l s i n d i s t r i c t s without c r i t e r i a have done so compared to a tenth of the p r i n c i p a l s i n d i s t r i c t s with c r i t e r i a . A corresponding difference exists i n the writing of no "less than s a t i s f a c t o r y " reports. Examining these "less than s a t i s f a c t o r y " reports as a proportion of a l l evaluations written, reveals that 4.2 percent (49 of 1163) of reports written by "no c r i t e r i a " p r i n c i p a l s are "less than s a t i s f a c t o r y " compared to only 2.2 percent (57 of 2585) of " c r i t e r i a " p r i n c i p a l s . However, no s t a t i s t i c a l l y s i g n i f i c a n t relationship emerges when c r i t e r i a and "less than s a t i s f a c t o r y " reports are cross tabulated with purpose. F i n a l l y , "no c r i t e r i a " p r i n c i p a l s have written an average of 2.9 evaluations per year (1163 i n 408 114 p r i n c i p a l years) compared to 3.2 (2585 i n 806 p r i n c i p a l years) for " c r i t e r i a " p r i n c i p a l s . Therefore, "no c r i t e r i a " p r i n c i p a l s conduct fewer evaluations of which a greater proportion re s u l t i n "less than s a t i s f a c t o r y " reports. Table 7.3 P r i n c i p a l s Categorised on the Basis of Evaluation C r i t e r i a by Evaluation Purpose and "Less Than S a t i s f a c t o r y " Reports Percentage of respondents Variable No C r i t e r i a C r i t e r i a n=60 n=119 Evaluation purpose Growth and development 66.7 53.8 Acco u n t a b i l i t y 30.0 43.7 Other 3.3 2.5 n=61 n=121 "Less than s a t i s f a c t o r y " reports written* None 50.8 66.9 One 27.9 24.0 Two or more 21.3 9.1 *2 < .05, Female p r i n c i p a l s have much less experience than male p r i n c i p a l s o v e r a l l (see Table 7.4) and t h i s difference i s s t a t i s t i c a l l y s i g n i f i c a n t (p<.05). Therefore, the sex of pr i n c i p a l s and evaluation purpose were cross tabulated against years of experience as a p r i n c i p a l . This provides a control for experience and, when done, the s t a t i s t i c a l difference that exists between sex and purpose disappears. 115 However, s t a t i s t i c a l s i g n i f i c a n c e remains i n the "1 to 5 years" experience group (p<.05) with men opting for growth and development i n s i g n i f i c a n t l y smaller proportions than women. Table 7.4 Sex of P r i n c i p a l by Years of Experience as P r i n c i p a l Percentage of respondents 1-5 years 6-10 years 11-15 years 16+ years Respondent sex* n=58 n=58 n=24 n=47 Male 26.5 27.2 14.7 31.6 Female 43.1 41.2 7.8 7.8 *£ < .05. From the data presented i n Chapter VI a s i g n i f i c a n t difference also exists i n the type of school male and female p r i n c i p a l s administer (p<.05). This manifested i t s e l f i n terms of males being over represented i n secondary schools while females are over represented i n elementary schools. However, when school type i s cross tabulated with respondent sex and purpose there i s no s t a t i s t i c a l l y s i g n i f i c a n t r e l a t i o n s h i p . 116 The Need For Further Training Findings on the need for further t r a i n i n g should be viewed against a backdrop of general comfort on the part of p r i n c i p a l s about t h e i r l e v e l of competence i n formal evaluation. Training received i s hardly referred to either as an obstacle to formal evaluation or as a comment, pos i t i v e or negative, i n the f i n a l anecdotal section of the survey. Two examples of anecdotal responses were given i n Chapter V (p.88). A t h i r d respondent (075) recognises the ever changing nature of the role of the school p r i n c i p a l and the continual need to upgrade knowledge and s k i l l s : It i s extremely important for p r i n c i p a l s to be current on curriculum and teaching strategies. In t h i s respect, we a l l need "more t r a i n i n g " throughout our career. I have taken available workshops on legal aspects of report writing, but w i l l need more as things change and evolve. A comparison of male and female p r i n c i p a l s with regard to t h e i r need for further t r a i n i n g (Table 7.5), shows very l i t t l e difference. The exceptions are the pre-evaluation phase of an evaluation leading to a " s a t i s f a c t o r y " report and the post-observation and report writing phases of a formal evaluation leading to a "less than s a t i s f a c t o r y " report. However, female p r i n c i p a l s c i t e the need for t r a i n i n g for the pre-evaluation phase of a " s a t i s f a c t o r y " evaluation less than t h e i r male colleagues. In the post- observation phase of a "less than s a t i s f a c t o r y " evaluation, half the male p r i n c i p a l s indicate a need for more t r a i n i n g , whereas t h i s applied to two t h i r d s of female. For the report writing phase, half the males express a need for more tr a i n i n g compared to three quarters of female p r i n c i p a l s . Table 7.5 Sex of P r i n c i p a l and Need f o r Further Training i n Evaluation Percentage agreeing on need for further t r a i n i n g "Satisfactory" "Less than s a t i s f a c t o r y " report report Male Female Male Female Evaluation ~~~~ phase n=133 n=48 n=58 n=23 Pre-evaluation conference(s) Classroom observations Post-observation conferences Writing the f i n a l report 23.1 36.8 34.4 36.6 14.6 27.1 39.6 38.8 34.5 43.1 53.4 52.5 39.1 47.8 65.2 73.9 Table 7.6 presents the data on t r a i n i n g need for d i f f e r e n t experience groups, i n formal evaluations leading to a "satisfactory" report. This Table shows that p r i n c i p a l s with 1 to 5 years of experience indicate, i n much larger percentages, a need for further t r a i n i n g i n a l l four phases of formal evaluation (p<.05) and a trend for p r i n c i p a l s with increasing years of experience to f e e l i n decreasing percentages, that they need further t r a i n i n g . 118 Table 7.6 P r i n c i p a l Experience and Need for Further Training In Evaluations Leading to a "Sat i s f a c t o r y " Report Percentage agreeing on need for further t r a i n i n g 1-5 years 6-10 years 11- 16 years 16+ years Evaluation phase n=55 n=56 n=24 n=47 Pre-evaluation conference!s) 30.9 14.3 12.5 21.3 Classroom observations 45.5 26.8 29.2 32.6 Post-observation conferences 47.3 31.5 37.5 26.1 Writing the f i n a l report 50.0 33.9 30.4 29.8 In the case of evaluations leading to a "less than s a t i s f a c t o r y " report, the d i v i s i o n l i e s between p r i n c i p a l s with 1 to 10 years of experience and those with 11 or more years of experience. By div i d i n g p r i n c i p a l s i n t h i s way, i t emerges from Table 7.7, that respondents with 1 to 10 years of experience say they need more t r a i n i n g than those with 11 years or more. However, none of the above differences are s t a t i s t i c a l l y s i g n i f i c a n t , apart from the pre-evaluation phase for the "1 to 5 years" p r i n c i p a l s (p<.05). As with evaluations leading to a "satisfactory" report, a pattern of decreasing need for t r a i n i n g emerges with increasing numbers of years of experience. 119 Table 7.7 P r i n c i p a l Experience and Need for Further Training i n Evaluations Leading to a "Less Than S a t i s f a c t o r y " Report Percentage agreeing on need for further t r a i n i n g 1-5 years 6-10 years 11- 16 years 16+ years Evaluation phase n=17 n=23 n=16 n=27 Pre-evaluation conference(s) 62.5 26.1 33.3 29.6 Classroom observations 62.5 52.2 26.7 37.0 Post-observation conferences 64.7 69.6 50.0 44.4 Writing the f i n a l report 70.6 73.9 40.0 48.1 Given that master's specialty and previous t r a i n i n g i n formal evaluation might be expected to have some bearing on the extent to which p r i n c i p a l s f e e l the need for further t r a i n i n g , these two variables were f i r s t cross tabulated with t r a i n i n g needs across a l l p r i n c i p a l s . This showed no relationship or s t a t i s t i c a l s i g n i f i c a n c e between master's degree and the need for t r a i n i n g . Furthermore, p r i n c i p a l s with 10 t r a i n i n g points or more are no less l i k e l y to say they need further t r a i n i n g than t h e i r colleagues with fewer t r a i n i n g points. No s t a t i s t i c a l l y s i g n i f i c a n t r e l a t i o n s h i p could be found between these two variables. 120 When the variables of sex and experience were each included i n a cross tabulation with master's speci a l t y and t r a i n i n g need, i t did produce occasional instances of s t a t i s t i c a l s i g n i f i c a n c e . However, c e l l sizes were generally less than f i v e . While t r a i n i n g points cross referenced with t r a i n i n g need and each of the variables of sex and experience was subject to s i m i l a r drawbacks with regard to the size of c e l l s , i t produced a rather more definable pattern. For the observation, post-observation and f i n a l report phases of a "less than s a t i s f a c t o r y " report, there i s a s t a t i s t i c a l l y s i g n i f i c a n t relationship between female p r i n c i p a l s with less than 10 t r a i n i n g points and t h e i r greater need for further t r a i n i n g (p<.05). The same pattern i s then repeated for p r i n c i p a l s with one to f i v e years experience (p<.05). Obstacles to Evaluation By far the most important obstacle to emerge from the survey results was time. Anecdotal responses provide an i n t e r e s t i n g lead into t h i s finding. The majority t e s t i f y to the multitude of tasks p r i n c i p a l s have to do and the d i f f e r e n t roles they are expected to perform. These responses often convey a f e e l i n g of i n s u f f i c i e n t time to devote to what p r i n c i p a l s f e e l are the most important aspects of t h e i r r e s p o n s i b i l i t i e s , one being the evaluation 121 of teaching. The comment made by respondent 073 i s f a i r l y representative: Instructional Leadership i s "the" most important aspect of our job. However, u n t i l t h i s i s recognized by government and by School Boards i n actions as well as rhetoric we w i l l never have the necessary time to do t h i s part well. Eroding administration time i n schools ac t u a l l y erodes the q u a l i t y of education s i g n i f i c a n t l y more than does the r a i s i n g of class size, (author's emphasis) A comment from respondent 093 even goes so far as suggesting the current r e s p o n s i b i l i t i e s of p r i n c i p a l s may need to be separated between p r i n c i p a l s , who would maintain t h e i r function as educational managers, and other administrators who would take on the more bureaucratic r e s p o n s i b i l i t i e s : The increase i n decentralization from d i s t r i c t l e v e l to s i t e based management works against formal evaluations being made an administrative p r i o r i t y due to length of time. If current administrators are expected to continue doing formal evaluations, then other people need to perform the managerial tasks - people not presently i n the system perhaps. A f i n a l comment presents the lack of time and i t s incumbent pressures i n t h e i r starkest form when respondent 061, aft e r l i s t i n g the three main obstacles to the carrying out of the formal evaluation of teaching as "TIME", "TIME", "TIME", went on to say i n response to question 21: "It i s obvious I believe time to be the most s i g n i f i c a n t factor preventing good q u a l i t y assessment." These anecdotal data provide support for the data r e l a t i n g to the most important obstacles to conducting formal evaluation. Time i s given by two thirds of a l l If 122 respondents who c i t e d a most important obstacle. Across a l l f i r s t , second and t h i r d obstacles, "time" was c i t e d by 152 respondents or 42.0 percent of a l l in d i v i d u a l references made. In eight cases p r i n c i p a l s wrote "time", "time", and "time" as t h e i r three most important obstacles. However, i n each case these were recorded only as a f i r s t obstacle and thus constitute one in d i v i d u a l reference rather than three 6. In a further seven cases a l l three obstacles can be defined as time, while i n another 24 cases two of the obstacles can be defined as time. In each of these 39 cases, time i s therefore treated as one in d i v i d u a l reference. However, when counted separately, they bring the t o t a l references to time to 206 and, as shown l a t e r , a l l these references are analysed for what they say about why time i s an obstacle. The t h i r t y - n i n e p r i n c i p a l s , who made multiple references to time, constitute a sub-group which represents 25.7 percent of a l l the respondents who referred to time as an obstacle. However, when an analysis i s c a r r i e d out to discern whether or not these respondents are clustered i n p a r t i c u l a r groups, for example less experienced p r i n c i p a l s , no pattern emerges. For p r i n c i p a l s grouped by sex, experience, and evaluation cycle requirements, the This i s because the main point of i n t e r e s t was the number of p r i n c i p a l s r e f e r r i n g to an obstacle rather than the number of references to that obstacle. 123 proportion of respondents who made multiple time references i s generally between 20 and 25 percent. Among the 206 references i d e n t i f i e d above, 79 were one word statements, 94 gave some elaboration or explanation, and 32 were defined as time obstacles without e x p l i c i t l y s tating the word "time". Therefore, 126 statements (representing the views of 113 respondents, or 74.3 percent of those who gave time as an obstacle) are more elaborative and, as such, provide a r i c h source of explanation for why time i s considered such an important obstacle. Table 7.8 i d e n t i f i e s the two themes (excluding "Other") which emerged from these statements. Both quite evidently have to do with pressure of work but i t i s possible to di s t i n g u i s h between a "Workload" category and a "Process" category. The workload category i s sub-divided into four types of statement, while the process category i s sub- divided twice. The f i r s t two columns of the table show a) the numbers of statements made i n the above two categories; and b) the percentages these numbers represent out of the t o t a l number of statements made. The second two columns show a) the number of respondents who made these types of statement; and b) the percentage these respondents represent of the t o t a l who made time statements. 1 2 4 Table 7.8 Time Obstacle Statements Nature of time pressure Number of statements % Number of respondents % 1.Workload 74 58.7 64 56.6 a.Other p r i o r i t i e s , demands and interruptions 46 36.5 37 32.7 b.Teaching commitments 11 8.7 11 9.7 c.Increased administrative r e s p o n s i b i l i t i e s i n recent years 9 7.1 8 7.1 d.Excessive number of evaluations i n one year 8 6.3 8 7.1 2.Process 43 34.1 40 35.4 a . A b i l i t y to e f f e c t i v e l y carry out the process 29 23.0 28 24.8 b.Observations/Conferencing 14 11.1 12 10.6 Other 9 7.1 9 8.0 Total 126 100.0 113 100.0 Table 7.8 shows that excessive workload i s generally considered an important obstacle by p r i n c i p a l s . The evaluation process also features prominently Almost a l l the statements about time could be placed into one of these two categories. The following two statements highlight the unpredictable nature of the p r i n c i p a l ' s role and the f e e l i n g that there may always be something to unsettle previously made plans: " P r i o r i t i e s - while t h i s component of admin i s 125 important, the urgent needs often displace others -*• 'The tyranny of the urgent 1" (respondent 008). " C r i s i s - both parent and student that take precedence and have to be dealt with 'right now'" (respondent 154). A t h i r d respondent (090) draws attention to the pressure of teaching commitments when saying "Lack of time - I teach .5 and have many, many r e s p o n s i b i l i t i e s besides evaluation of s t a f f " , while another expressed the shortcomings i n the number of evaluations possible: "Time! I should be doing several evaluations a year but can only manage one." (respondent 136). Respondent 142 provides an a l l embracing example of what many others said i n part: 1. other r e s p o n s i b i l i t i e s - meetings, paperwork, c u r r i c u l a r updates, special ed, budgets, behavioural involvements, etc. 2. unexpected interruptions - parents, d i s t r i c t s t a f f , telephone. 3. time commitments - school wide events, performing arts, special projects, assemblies... A further example i l l u s t r a t e s the perception of the pressures imposed by the process: "Time - to develop goals for evaluation process, to observe/collect data, to debrief, revise goals, observe/debrief, revise, write, revise, rewrite!" (088). F i n a l l y , the assertion made by respondent (035) presents a bold statement about a key role of the p r i n c i p a l and the need to address the obstacle of time i f t h i s role i s to be c a r r i e d out e f f e c t i v e l y : "TIME! - i f the 126 p r i n c i p a l , as school leader, i s charged with the r e s p o n s i b i l i t y of a s s i s t i n g teachers to set goals for professional growth, then more admin, release time i s needed." No cl e a r difference between male and female respondents emerges with regard to time (Table 7.9). The data r e l a t i n g to time as a percentage of a l l obstacles referred to i n question 18 of the questionnaire (see Appendix A.3), shows that male p r i n c i p a l s c i t e d "time" 110 times out of a t o t a l of 266 references to obstacles, or 41.4 percent of a l l references. This i s v i r t u a l l y i d e n t i c a l to the 41 references by female p r i n c i p a l s out of a t o t a l of 97, or 42.3 percent of a l l references. Table 7.9 Time as an Obstacle and Sex of P r i n c i p a l Sex of "Time" as most "Time" as a proportion of p r i n c i p a l important obstacle a l l 'obstacle references' Q, "O n a % n b Male 63.2 86 136 41.4 110 266 Female 66.7 34 51 42.3 41 97 a Total number of f i r s t obstacles c i t e d of a l l types. b Total number of f i r s t , second, and t h i r d obstacles c i t e d of a l l types. Table 7.10 shows that as numbers of years of experience increase, p r i n c i p a l s give time as the most important obstacle to conducting formal evaluation i n decreasing 127 percentages. Of p r i n c i p a l s with 1 to 5 years of experience, nearly three quarters put time as t h e i r most important obstacle, while just over half with 16 or more years of experience took t h i s view. Table 7.10 Time as an Obstacle and P r i n c i p a l Experience Years of experience "Time" as most "Time" as a proportion of as a p r i n c i p a l important obstacle a l l 'obstacle references' Q, "6 n a % n b 1 to 5 years 74.1 43 58 43.6 51 117 6 to 10 years 63.8 37 58 40.9 45 110 11 to 15 years 58.3 14 24 42.6 20 47 16 years or more 55.3 26 47 39.3 35 89 a Total number of f i r s t obstacles c i t e d of a l l types. b Total number of f i r s t , second, and t h i r d obstacles c i t e d of a l l types. A s i m i l a r trend can be observed when looking at time as a percentage of t o t a l 'obstacle references' made by each of the experience groups. Once again, i t i s the more experienced p r i n c i p a l s that make fewer references to time as an obstacle than t h e i r less experienced colleagues, a l b e i t by a f a i r l y narrow margin. When evaluation cycle provision i s examined i n r e l a t i o n to time (Table 7.11), just over half the "no cycle" p r i n c i p a l s give time as t h e i r most important obstacle. However, for the "every/at least" p r i n c i p a l s t h i s figure i s 128 nearly three quarters. The "not more than" group also c i t e time as t h e i r most important obstacle to a greater extent than do the "no cycle" p r i n c i p a l s and these differences are s t a t i s t i c a l l y s i g n i f i c a n t (p<.05). When time as a percentage of t o t a l 'obstacle references' i s used as an indicator, the same pattern emerges and, indeed, for the "every/at least" p r i n c i p a l s time amounts to nearly half of a l l t h e i r 'obstacle references'. Table 7.11 Time as an Obstacle and P r i n c i p a l s Categorised on the Basis of Evaluation Cycles C o l l e c t i v e agreement "Time" as the most "Time" as a proportion of provision important obstacle* a l l 'obstacle references' % n a O, o n b "No cycle" 54.8 40 73 38.2 55 144 "Every/At l e a s t " 73.6 53 72 46.3 63 136 "Not more than" 65.0 26 40 39.5 32 81 a Total number of f i r s t obstacles c i t e d of a l l types. b Total number of f i r s t , second, and t h i r d obstacles c i t e d of a l l types. *£ < .05. Furthermore, "no cycle" p r i n c i p a l s have conducted an average of 3.1 evaluations per year of p r i n c i p a l s h i p (1576 i n 502 p r i n c i p a l years) since September 1988, compared to 2.9 (1422 i n 491 p r i n c i p a l years) for "every/at least" and 3.4 (750 i n 221 p r i n c i p a l years) for "not more than" 129 p r i n c i p a l s . These data reveal that "no cycle" and "every/at least" p r i n c i p a l s conduct a si m i l a r number of evaluations per year. F i n a l l y , 3.6 percent (56 of 1576) of the reports written by "no cycle" p r i n c i p a l s have been "less than s a t i s f a c t o r y " , compared to 2.4 percent (34 of 1422) of the "every/at least" p r i n c i p a l s and 2.1 percent (16 of 750) of the "not more than" p r i n c i p a l s . A s t a t i s t i c a l l y s i g n i f i c a n t relationship e x i s t s between teaching load and type of school (p<.05). Elementary p r i n c i p a l s are far more l i k e l y to have teaching assignments of 20 to 39 percent and 40 percent or more, than secondary p r i n c i p a l s . Even so, when time as an obstacle for a l l respondents i s cross referenced against a) the percentage of teaching p r i n c i p a l s do; b) t h e i r type of school; and c) t h e i r s t a f f sizes; no s t a t i s t i c a l l y s i g n i f i c a n t r e l a t i o n s h i p emerges. However, when the percentage of teaching and size of s t a f f were controlled for i n three way cross tabulations with cycle provision and time, s t a t i s t i c a l l y s i g n i f i c a n t relationships were found for "zero" teaching (p<.05) and s t a f f s of "20 to 29" (p<.05) and "30 or more" (p<.05). These data show that p r i n c i p a l s evaluating to a cycle with a 100 percent administration assignment or s t a f f s of "20 to 29" or "30 or more", c i t e time as t h e i r most important obstacle s i g n i f i c a n t l y more than t h e i r "no cycle" colleagues with the same administration assignment and siz e of s t a f f . 130 A s t a t i s t i c a l l y s i g n i f i c a n t r e l a t i o n s h i p e x i s t s between d i s t r i c t size and cycle provision (p<.05). This re l a t i o n s h i p takes the form of large d i s t r i c t s having disproportionately fewer (p<.05) evaluation cycles phrased as "every/at least", while medium d i s t r i c t s have disproportionately more (p<.05). However, when d i s t r i c t size i s cross tabulated with cycle provision and time, the re l a t i o n s h i p for medium d i s t r i c t s completely disappears. While "no cycle" p r i n c i p a l s from large d i s t r i c t s s t i l l under represent time as t h e i r most important obstacle and "every/at least" p r i n c i p a l s from large d i s t r i c t s over represent time, the r e l a t i o n s h i p i s not s t a t i s t i c a l l y s i g n i f i c a n t . A s i g n i f i c a n t difference also exists between cycle provision and experience (p<.05). Because there i s a greater tendency for less experienced p r i n c i p a l s to c i t e time as t h e i r most important obstacle, cycle provision, experience and time were cross tabulated. However, t h i s produces no s t a t i s t i c a l l y s i g n i f i c a n t r e s u l t s . Summary A clear difference exists with regard to the purpose ascribed to formal evaluation by male and female p r i n c i p a l s and t h i s difference i s s t a t i s t i c a l l y s i g n i f i c a n t . A much higher percentage of females than males defined teacher 131 growth and development as the most important purpose, although t h i s i s also the view of the majority of male p r i n c i p a l s . Correspondingly, a much higher percentage of males than females, considered the most important purpose to be accountability for the q u a l i t y of teaching. In addition, female p r i n c i p a l s have conducted s l i g h t l y fewer evaluations per year and written s l i g h t l y more "less than s a t i s f a c t o r y " reports as a percentage of a l l reports written, than t h e i r male counterparts. Among p r i n c i p a l s categorised by experience a pattern also exists with regard to t h e i r views about purpose but i t i s not s t a t i s t i c a l l y s i g n i f i c a n t . P r i n c i p a l s with more than ten years experience assign greater importance to accountability and less to teacher growth and development than do t h e i r less experienced colleagues. Furthermore, p r i n c i p a l s i n the "11 to 15 years" experience category are, to a s t a t i s t i c a l l y s i g n i f i c a n t extent, far more l i k e l y to have written multiple "less than s a t i s f a c t o r y " reports and have the highest percentage of "less than s a t i s f a c t o r y " reports as a percentage of a l l reports. This i s against the backdrop of conducting more evaluations per year than t h e i r colleagues, apart from the "1-5 years" experience group. "No c r i t e r i a " p r i n c i p a l s c i t e teacher growth and development as the most important purpose of evaluation more often than " c r i t e r i a " p r i n c i p a l s , although t h i s i s not 132 s t a t i s t i c a l l y s i g n i f i c a n t . "No c r i t e r i a " p r i n c i p a l s are also more l i k e l y to have written multiple "less than sa t i s f a c t o r y " reports and t h i s i s s t a t i s t i c a l l y s i g n i f i c a n t . This was further borne out by the s u b s t a n t i a l l y higher percentage of evaluations c a r r i e d out by "no c r i t e r i a " p r i n c i p a l s that lead to "less than s a t i s f a c t o r y " reports, while they conduct fewer evaluations o v e r a l l . "No cycle" p r i n c i p a l s are much more l i k e l y to have written multiple "less than s a t i s f a c t o r y " reports and t h i s relationship i s s t a t i s t i c a l l y s i g n i f i c a n t . Furthermore, "no cycle" p r i n c i p a l s write more evaluations per year than t h e i r "every/at least" colleagues, though not as many as the "not more than" p r i n c i p a l s . With regard to respondent sex and time as an obstacle, no marked difference i s i d e n t i f i e d between male and female p r i n c i p a l s . However, time emerged i n percentage terms, as a decreasing obstacle as p r i n c i p a l years of experience increased. A p a r t i c u l a r l y marked difference exists between the "1 to 5 years" group and the r e s t . While "no cycle" p r i n c i p a l s acknowledge time as an important obstacle, they did not express t h i s view to the same extent as t h e i r "with cycle" colleagues and t h i s difference i s s t a t i s t i c a l l y s i g n i f i c a n t . Female p r i n c i p a l s indicated a greater need for t r a i n i n g i n a l l the phases of an evaluation leading to a "less than 133 sa t i s f a c t o r y " report but less need than male p r i n c i p a l s for t r a i n i n g i n the pre-evaluation phase of an evaluation leading to a "satisfactory" report. A si m i l a r trend emerges with regard to experience, where, generally, as experience increases the need for t r a i n i n g decreases for evaluations leading to both "sa t i s f a c t o r y " and "less than s a t i s f a c t o r y " reports. In the case of the "1 to 5 years" experience group, t h i s difference i s s t a t i s t i c a l l y s i g n i f i c a n t for s a t i s f a c t o r y reports. 134 CHAPTER VIII Discussion, Conclusions, and Recommendations This chapter i s divided into three sections: a) discussion; b) conclusions; and c) recommendations. The f i r s t section i s sub-divided into four parts, under the headings of process, purpose, t r a i n i n g , and obstacles and seeks to explain the findings from the study. The second section draws together the main findings of the study and concludes with a l i s t of key findings. The t h i r d section i n t h i s chapter presents recommendations for further research and suggests possible solutions to weaknesses or shortcomings which emerged from the study. Discussion This section seeks to i d e n t i f y explanations for the findings presented i n Chapters IV, V, and VII. As far as possible, explanations are sought by r e l a t i n g the findings from the study to the l i t e r a t u r e presented i n Chapter II. However, at times the l i t e r a t u r e suggests only p a r t i a l explanations. In these cases i n t u i t i v e explanations are offered based on the evidence av a i l a b l e . The section i s divided into four parts: purpose, process, t r a i n i n g , and, obstacles. However, the d i s t i n c t i o n s between these four parts are necessarily blurred because of t h e i r degree of i n t e r - r e l a t i o n s h i p . 135 Purpose The purpose of formal evaluation was a matter of considerable concern to many respondents. The d i f f e r e n t purposes highlighted i n the l i t e r a t u r e (Harris & Monk, 1992; Housego, 1989; Poster & Poster, 1993) are c l e a r l y reconstructed i n the survey responses. The majority of p r i n c i p a l s believe the most important purpose of formal evaluation i s teacher growth and development. However, a sizeable minority of p r i n c i p a l s consider the primary purpose of evaluation to be accountability for the q u a l i t y of teaching. This finds further expression i n the anecdotal responses and, therefore, even though i t i s l i k e l y that the large majority of p r i n c i p a l s would say both of these purposes are important, the above difference of view appears to be a real one. The explanation of t h i s difference of view may be provided by the l i t e r a t u r e which describes the complexity of the p r i n c i p a l role and the d i f f e r e n t stakeholders to whom the p r i n c i p a l i s accountable (Rossow, 1991; Sharp & Walter, 1994; Sybouts & Wendel, 1994). How far the p r i n c i p a l i s influenced by the 'competing' needs of the various stakeholders i n the education system w i l l depend larg e l y on the p r i n c i p a l ' s own personal values and b e l i e f s . The existence of d i f f e r e n t values and b e l i e f s amongst p r i n c i p a l s supports the need to dis t i n g u i s h between them when 136 attempting to explain t h e i r professional views and behaviour. With regard to evaluation purpose, the d i s t i n c t i o n drawn between male and female p r i n c i p a l s and between p r i n c i p a l s with d i f f e r e n t lengths of experience, does produce some intere s t i n g findings. The s t a t i s t i c a l l y s i g n i f i c a n t difference i d e n t i f i e d between male and female p r i n c i p a l s with regard to purpose disappeared when p r i n c i p a l experience was included i n the cross tabulation. However, the data suggest gender i s a factor i n the determination of views about the purpose of evaluation, although i t seems equally l i k e l y that experience has some influence. Trying to e s t a b l i s h whether or not there i s a gender or experience e f f e c t i s problematic because the vast majority of female p r i n c i p a l s have ten years experience or less and t h i s experience group tends to opt for growth and development i n greater percentages than t h e i r more experienced colleagues. Therefore, because the less experienced p r i n c i p a l s are younger and have a more recent univ e r s i t y post-graduate education, the factors of age and greater exposure to 'newer' philosophies pertaining to growth and development may be at work. However, support for a gender explanation i s provided by the differences which exist between male and female p r i n c i p a l s at a l l experience l e v e l s and, indeed, male p r i n c i p a l s i n the "1 to 5 years" experience group c i t e growth and development less 137 than t h e i r female counterparts to a s t a t i s t i c a l l y s i g n i f i c a n t degree. Two important questions are raised here. F i r s t : "Why should greater experience have any association with a greater orientation towards accountability rather than growth and development?" Second: "Why should female p r i n c i p a l s be any more i n c l i n e d to see evaluation as a process of growth and development than male p r i n c i p a l s ? " The l i t e r a t u r e on formal evaluation of teaching provides l i t t l e assistance with the f i r s t question and so i t i s ' i n t u i t i v e ' l o g i c that leads to the rather f a m i l i a r explanation that with more experience comes more cynicism. This straightforward explanation i s made a l l the more appealing when taking into account the views expressed by respondents about the nature of the process. If the f a i r l y negative attitudes expressed are representative, i t i s very l i k e l y that p r i n c i p a l s would develop a degree of "battle weariness" over time. However, t h i s explanation i t s e l f i s based on the assumption that the pursuit of growth and development i s somehow more i d e a l i s t i c than that of accountability. This assumption may very well be f a l s e given that some p r i n c i p a l s wrote with passion about t h e i r b e l i e f i n ridding the teaching profession of those teachers who they f e e l bring harm to the educational well-being of pupils. 138 Another explanation may l i e i n what might be c a l l e d a "culture of accountability". This was epitomised by the existence of school inspectors who, i n the past, were responsible for ensuring the competence of teaching. Anecdotal evidence suggests that i t i s only i n r e l a t i v e l y recent times that notions of growth and development have become more widely accepted. Thus, more experienced p r i n c i p a l s may have had t h e i r views about evaluation shaped i n a rather d i f f e r e n t culture to that which ex i s t s today. An answer to the second question i s c e r t a i n l y offered by the l i t e r a t u r e . Shakeshaft (1987), Alder et a l . (1993), Regan and Brooks (1995) and Ozga (1993) amongst others, have suggested that women adopt a more c o l l e g i a l s t y l e of school management and have a more caring approach to s t a f f within the school than do men. If t h i s i s the case, i t may provide an explanation for the differences observed between men and women with regard to the purpose of formal evaluation, since t h i s more caring d i s p o s i t i o n i s l i k e l y to be better suited to the purpose of growth and development than accountability. However, the l i t e r a t u r e also speaks of the longer periods of time that women p r i n c i p a l s have tended to spend as classroom teachers before they enter the f i e l d of school administration (Gross & Trask, 1976; Blumberg & Greenfield, 1986). This may lead to a greater a f f i n i t y with the l o t of the classroom teacher and a more 'established 139 memory1 of the classroom context than some male p r i n c i p a l s who 'rose through the ranks' more quickly. Furthermore, the l i t e r a t u r e (Blumberg & Greenfield, 1986) and questionnaire data show that women are predominantly p r i n c i p a l s of elementary schools. The questionnaire data also show that elementary p r i n c i p a l s are s i g n i f i c a n t l y more l i k e l y to have a 40 percent or more teaching load as part of t h e i r assignment. This current, day-to-day exposure to the r e a l i t y of the classroom would only serve to reinforce any greater understanding these female p r i n c i p a l s have of the posi t i o n of the classroom teacher. If a greater understanding of the po s i t i o n of the classroom teacher does exist among the generality of women pr i n c i p a l s than among the generality of men, t h i s does not, i n i t s e l f , mean that women p r i n c i p a l s would be less l i k e l y to opt for accountability. Indeed, such an understanding may lead to less tolerance of those whose teaching i s not of a s a t i s f a c t o r y standard. This highlights the marginally greater tendency for women p r i n c i p a l s to write "less than s a t i s f a c t o r y " reports than men, which, at f i r s t glance, would seem to be somewhat at odds with the notion of growth and development. However, the l i t e r a t u r e also refers to the capacity of women i n administration to have a more p r i n c i p l e d stance which re s u l t s i n a more courageous form of leadership (Regan & Brooks, 1995). Bolton's (1980) 140 'evaluator resistances', one of which i s fear of an unpleasant reaction which would prevent a relat i o n s h i p conducive to f a c i l i t a t i n g improvement, may also be pertinent here. If female p r i n c i p a l s are more practised and more confident at the interpersonal s t y l e of management, t h i s i s l i k e l y to also have taught them ways of disagreeing while maintaining a working rela t i o n s h i p . This, i n turn, may lead to less fear of the consequences of a "less than s a t i s f a c t o r y " report than for some male p r i n c i p a l s who are less practised and less s k i l l e d at the art of c o n f l i c t resolution. Process It i s evident from the school d i s t r i c t c o l l e c t i v e agreements, that most of the r e s p o n s i b i l i t y for conducting evaluation l i e s with the school p r i n c i p a l . Furthermore, B r i t i s h Columbia p r i n c i p a l s c l e a r l y believe t h i s i s a r e s p o n s i b i l i t y they should carry out and, to some extent, an important part of t h e i r wider role as i n s t r u c t i o n a l managers or educational leaders. The evaluation process i s summative i n nature and c o l l e c t i v e agreements rarely make s p e c i f i c reference to the purpose of formal evaluation. The f i n a l report i s required to conclude with either a) a statement i n d i c a t i n g that the teacher's 'classroom s i t u a t i o n ' i s "sat i s f a c t o r y " or "less than s a t i s f a c t o r y " ; or, i n a few school d i s t r i c t s , b) a 141 statement of competence l e v e l , for example, "excellent", "very good", and so on. The study c l e a r l y shows that the f i n a l report writing stage i s problematic for p r i n c i p a l s and t h i s i s a l l the more true for reports concluding with a "less than s a t i s f a c t o r y " recommendation. This summative process exists despite the wealth of l i t e r a t u r e (Darling-Hammond, et a l . , 1983; Darling-Hammond, 1986, Sergiovanni, 1977; and others) which describes the negative e f f e c t s such processes have on both the evaluatee and on the evaluator. More s p e c i f i c a l l y , the findings i n Antosz's (1990) study of B r i t i s h Columbia evaluation processes, that most are summative and f a i l to take account of the evaluation l i t e r a t u r e , appear to be as v a l i d today as they were six years ago. This evidence suggests that there are reasons for the existence of a summative process and these reasons can probably be explained best by the l i t e r a t u r e which i d e n t i f i e s the d i f f e r e n t needs of the organisation and of the i n d i v i d u a l (Housego, 1989; and others). Clearly, school boards have to be able to meet the requirements of the Teaching Profession Act 1987 (Province of B r i t i s h Columbia, 1987) and t h i s involves an evaluation report of some kind. However, what appears to be happening i s the production of summative reports by many p r i n c i p a l s who believe growth and development to be the most important purpose. While the 142 data from t h i s study do not provide a c l e a r answer, the anecdotal responses suggest that some p r i n c i p a l s are t r y i n g to provide a formative experience within a summative process (see p.32). A d i s t i n c t i o n has to be drawn between evaluations leading to " s a t i s f a c t o r y " reports and those leading to "less than s a t i s f a c t o r y " ones. A l l the data regarding factors present i n the four stages of a formal evaluation, show "less than s a t i s f a c t o r y " reports to be associated with much greater stress and complexity, as well as a greater requirement of time and need for further t r a i n i n g . Furthermore, since September 1988, close to two t h i r d s of p r i n c i p a l s have never written a "less than s a t i s f a c t o r y " report and a further quarter have written only one. The average number of evaluations conducted per year of p r i n c i p a l s h i p i n t h i s period was 3.1 and the number of "less than s a t i s f a c t o r y " reports written was one per 34.8 evaluations, which equates to 2.9 percent. These data support the assertion made by Haefele (1992), that few "less than s a t i s f a c t o r y " reports are written. There are a number of possible explanations for t h i s phenomenon and Haefele suggests that part of the reason i s lack of time to conduct enough observations upon which to base a "less than s a t i s f a c t o r y " report. Bolton (1980), though, refers to a set of resistances on the part of 143 evaluators. Some of these resistances may explain a pr i n c i p a l ' s d i s i n c l i n a t i o n to write a "less than s a t i s f a c t o r y " report and may also account for the r e l a t i v e l y small number of t o t a l evaluations conducted. For example, uncertainty about c r i t e r i a and inter p r e t a t i o n of data; fear of an unpleasant reaction; i n a b i l i t y to organise time for adequate observations; lack of support at higher l e v e l s of the organisation; and a lack of conviction that evaluation w i l l provide much "payoff". The data from the study provide other possible explanations. For the majority of p r i n c i p a l s the most important purpose of evaluation i s teacher growth and development and many believe the current process to be inadequate and time-consuming. A number of anecdotal responses revealed the d i f f i c u l t i e s involved i n proceeding with a "less than s a t i s f a c t o r y " report and the f e e l i n g that they r a r e l y e f f e c t real change or improvement. The d i s t i n c t i o n made e a r l i e r between evaluations leading to "satisfactory" and "less than s a t i s f a c t o r y " reports, was based on much greater l e v e l s of stress, complexity and time- consumption associated with the l a t t e r . A rationale was presented i n Chapter VII for including "less than s a t i s f a c t o r y " reports written and evaluations conducted as part of the consideration of purpose, since these reports are the product of the evaluation process. 144 However, t h i s study does not support the claim that p r i n c i p a l s who take a d i f f e r e n t view about the most important purpose of formal evaluation, also have a tendency to produce d i f f e r e n t proportions of "less than s a t i s f a c t o r y " reports. In other words, a p r i n c i p a l who i s more orientated towards teacher growth and development seems no less l i k e l y to produce "less than s a t i s f a c t o r y " reports than a p r i n c i p a l who perceives formal evaluation more i n terms of the accountability for the q u a l i t y of teaching. However, i t i s possible that p r i n c i p a l s with d i f f e r e n t views about the most important purpose of evaluation write r e l a t i v e l y few "less than s a t i s f a c t o r y " reports for d i f f e r e n t reasons. Perhaps growth orientated p r i n c i p a l s do regard the writing of a summative "less than s a t i s f a c t o r y " recommendation to be at odds with the concept of growth and development. On the other hand, p r i n c i p a l s who are more i n c l i n e d to want to hold teachers accountable for the q u a l i t y of t h e i r teaching, may be reluctant to use a process which they f e e l i s inadequate i n meeting t h i s objective. Of course, these data are just as l i k e l y to show that p r i n c i p a l s consider the general standard of teaching to be high but, on the rare occasions when i t i s necessary, both growth and development and accountability orientated p r i n c i p a l s are prepared to write "less than s a t i s f a c t o r y " reports. 145 Anecdotal responses show that at least some p r i n c i p a l s already have a notion of who t h e i r weak teachers are before the evaluation i s c a r r i e d out because they were able to suggest that the formal evaluation process should be reserved for such teachers. This supports the contention made by Wood (1992), Housego (1989) and others, that p r i n c i p a l s have preconceptions about the "classroom s i t u a t i o n " of the teachers on t h e i r s t a f f s . It i s impossible to say, from the findings of t h i s study, how well founded these preconceptions are, but they are c l e a r l y a factor i n understanding how p r i n c i p a l s approach t h e i r r e s p o n s i b i l i t i e s as evaluators of teaching. The existence of these preconceptions may explain why p r i n c i p a l s who are not governed by an evaluation cycle produce a greater proportion of "less than s a t i s f a c t o r y " reports. If p r i n c i p a l s are 'freed' from the requirement to evaluate a l l teachers on a c y c l i c a l basis, including the most competent, they may be more i n c l i n e d to focus t h e i r time and attention on the teachers they believe to be less than competent. This, i n turn, would be l i k e l y to lead to the writing of a greater proportion of "less than s a t i s f a c t o r y " reports than by p r i n c i p a l s who are obliged to evaluate a l l teachers on a c y c l i c a l basis. Also, p r i n c i p a l s i n d i s t r i c t s without c r i t e r i a write more "less than s a t i s f a c t o r y " reports than p r i n c i p a l s i n d i s t r i c t s with 146 c r i t e r i a . The explanation for t h i s i s rather speculative but the reason for the above phenomenon may be related to greater freedom once again. In t h i s case, the absence of stated c r i t e r i a allows the p r i n c i p a l an opportunity to ' t a i l o r ' the evaluation to his or her own objectives. In t h i s s i t u a t i o n , any preconceptions the p r i n c i p a l may have about the teaching of a member of s t a f f would be more l i k e l y to manifest themselves i n the evaluation because they would be more able to look for the things they wanted to see. Training P r i n c i p a l s generally believe they do formal evaluation well, there i s no strong i n d i c a t i o n from the survey results that they f e e l inadequately trained, and they generally express l i t t l e need for further t r a i n i n g . However, t h i s o v e r a l l picture i s q u a l i f i e d by the fact that large percentages of respondents expressed a need for further t r a i n i n g i n r e l a t i o n to evaluations leading to a "less than s a t i s f a c t o r y " report. It would seem that t h i s need i s linked to the greater complexity of such evaluations as i l l u s t r a t e d by other data from the survey. For example, several anecdotal responses attested to the increased d i f f i c u l t i e s involved i n evaluations leading to "less than s a t i s f a c t o r y " reports, as highlighted i n the previous section on "process". The report writing phase for both 147 "less than s a t i s f a c t o r y " and "sa t i s f a c t o r y " evaluations i s also characterised by a greater need for t r a i n i n g . The explanation for t h i s i s largely i n t u i t i v e but seems l i k e l y to be associated with the act of recording f i n a l summative recommendations which may then have to be defended. L i t t l e evidence ex i s t s of a l i n k between p r i o r t r a i n i n g i n formal evaluation and the needs expressed for further t r a i n i n g . The exception to t h i s general finding with regard to t r a i n i n g i s among p r i n c i p a l s with ten or less t r a i n i n g points who are either a) female; or b) i n the "1 to 5 years" experience category. These p r i n c i p a l s express a s i g n i f i c a n t l y greater need for t r a i n i n g i n most phases of evaluation. The explanation for both of these groups may be the same, given that many of the female p r i n c i p a l s are also i n the "1 to 5 years" experience category. A p r i n c i p a l with less experience, and i n p a r t i c u l a r less experience of evaluating, w i l l be more l i k e l y to seek evaluation t r a i n i n g than a more experienced p r i n c i p a l who fe e l s well versed i n the role of evaluator. It i s important here to emphasise that because a p r i n c i p a l perceives the need for more t r a i n i n g t h i s does not necessarily imply a lack of confidence on the part of that p r i n c i p a l . Indeed, such a p r i n c i p a l might be very confident and competent, but simply wish to f i l l the gaps they consider ex i s t i n t h e i r knowledge 148 as a result of limited experience. By the same token, a sense of not requiring further t r a i n i n g does not necessarily mean an ind i v i d u a l i s well trained. The fact that no l i n k can be shown between master's degree spec i a l t y and, more p a r t i c u l a r l y , p r i o r t r a i n i n g i n formal evaluation and an expressed need for further t r a i n i n g , i s somewhat d i f f i c u l t to explain by reference to the l i t e r a t u r e . However, Sergiovanni's exploration of Hogben's work on the " c l i n i c a l mind" and the teaching profession may provide some clues. P r i n c i p a l s , l i k e teachers, may be more i n c l i n e d to r e l y on t h e i r own experience than on the ideas generated by educational theoreticians and researchers. In other words, they may believe they learn more by doing than by taking courses. This may be a l l the more l i k e l y , given the intensely personal character of evaluation and the knowledge that no two evaluations are going to be the same. This experiential explanation i s given greater substance by the pattern i n the survey data, already referred to, of decreased need for t r a i n i n g with increased experience. Another important factor i n t h i s 'lack of t r a i n i n g need' phenomenon may be the nature of the t r a i n i n g i t s e l f . However, information r e l a t i n g to the content of evaluation t r a i n i n g does not form part of the data gathered by t h i s study. 149 Obstacles Time featured very heavily amongst the obstacles c i t e d by respondents to the survey. The evaluation process described i n most c o l l e c t i v e agreements t e s t i f i e s , to a greater or lesser extent, to the resources of time t h i s aspect of personnel management i s l i k e l y to consume i f done conscientiously. The planning involved i n the pre- evaluation phase; the s t i p u l a t i o n that classroom observations should be for f u l l lessons and take place on at least three occasions; the need, i n most cases, for the production of a f u l l anecdotal statement at each post- observation conference; and, l a s t l y , the writing of a f i n a l report, amount to a considerable quantity of work. This time pressure on p r i n c i p a l s which emerges from the c o l l e c t i v e agreements and also the l i t e r a t u r e (Haefele, 1992; Pigford & Tonnsen, 1993; Smith & Andrews, 1989; Bolton, 1980; and others) i s borne out and reinforced by the questionnaire returns. Time i s by far the most important obstacle c i t e d , the factor most often i d e n t i f i e d i n the four phases of the evaluation process, and i s a l l the more present for evaluations leading to a "less than sa t i s f a c t o r y " report. Those p r i n c i p a l s who elaborated on time as an obstacle substantiated the impression from the c o l l e c t i v e agreements, of a process which imposes considerable demands on time. In addition, respondents 150 highlighted general demands placed on them i n t h e i r r ole as p r i n c i p a l and together these two data reveal a cl e a r perception held by p r i n c i p a l s of excessive workload and i n s u f f i c i e n t time to meet a l l t h e i r professional p r i o r i t i e s . Therefore, the explanation for time being considered an obstacle seems clear enough, although, curiously, no d i r e c t r e l a t i o n s h i p was found between the percentage of administration time available and time as an obstacle. This was also true for s t a f f s i z e s . A possible explanation for t h i s finding i s that because so many respondents c i t e time as an obstacle these data are bound to include p r i n c i p a l s with a wide range of assignments. Also, a larger administration assignment w i l l not necessarily mean more time available for evaluation, where p r i n c i p a l s have numerous tasks 'bidding' for the 'additional' time. A s t a t i s t i c a l l y s i g n i f i c a n t r e l a t i o n s h i p does exist between evaluation cycles and time. P r i n c i p a l s who do not have to evaluate on a regular cycle are much less l i k e l y to c i t e time as the most important obstacle than t h e i r colleagues. A straightforward explanation of t h i s finding would be that the "no cycle" p r i n c i p a l s do not have the pressure of a c e r t a i n number of evaluations to conduct i n a cer t a i n period of time. This explanation i s given modest support i n the statements made by respondents i n r e l a t i o n to time as an obstacle. Eight respondents referred to 151 evaluation cycles as s p e c i f i c a l l y contributing to time pressures. While t h i s represents only 12.7 percent of a l l "every/at l e a s t " respondents, i t provides some evidence of time pressure imposed by evaluation cycles. The true extent of the contribution evaluation cycles make to a perception of time pressure may also be hidden by the more general references to excessive workload. However, the evidence for a l i n k between evaluation cycles and time i s not conclusive. For example, the data from t h i s study show that p r i n c i p a l s who evaluate on a regular cycle do not conduct any more evaluations per year than "no cycle" p r i n c i p a l s . In other words, the quantity of evaluations conducted i s no greater for p r i n c i p a l s who are required to evaluate on a regular cycle. However, t h i s p a r t i c u l a r finding may i l l u s t r a t e only that, even with the same quantity of work, when an a c t i v i t y i s required to be c a r r i e d out i t i s associated with greater pressure than an a c t i v i t y which involves some element of choice. A further setback to establishing a 'cycle-time' rela t i o n s h i p i s the loss of s t a t i s t i c a l s i g n i f i c a n c e when d i s t r i c t s i z e was incorporated into the equation with cycle provision and time. However, a pattern could s t i l l be observed i n terms of "no cycle" p r i n c i p a l s from large d i s t r i c t s c i t i n g time less often than "every/at l e a s t " p r i n c i p a l s from large d i s t r i c t s . Also, the fact that 152 s i g n i f i c a n t l y fewer large d i s t r i c t s have evaluation cycles may i t s e l f suggest the b e l i e f , on the part of the authors of c o l l e c t i v e agreements i n these d i s t r i c t s , that employing them would place too great a time pressure on t h e i r p r i n c i p a l s . This may be p a r t i c u l a r l y true given that school si z e and thus s t a f f s i z e , tend to be larger i n larger d i s t r i c t s . Although no l i n k exists i n the survey data between s t a f f size and administration time as a proportion of a p r i n c i p a l ' s assignment, when they are coupled with cycle provision a s i g n i f i c a n t r elationship does emerge. Pri n c i p a l s who do not teach or who have small s t a f f s izes, and have no evaluation cycle, c i t e time as an obstacle s i g n i f i c a n t l y less than p r i n c i p a l s i n the same po s i t i o n but who do have an evaluation cycle. Therefore, the importance of an evaluation cycle i s maintained, but i t s e f f e c t i s compounded by teaching r e s p o n s i b i l i t i e s and large s t a f f s . In short, there i s evidence to suggest, i n school d i s t r i c t s where evaluation cycles e x i s t , that t h i s places a greater time pressure on p r i n c i p a l s than i n school d i s t r i c t s where there i s no evaluation cycle. The pattern that emerges of time being a less important obstacle as experience increases, can be explained on both an i n t u i t i v e l e v e l and with reference to Sergiovanni's (1991) consideration of experience. Put simply, i t might be 153 expected that as p r i n c i p a l s become more f a m i l i a r and more practised i n t h e i r role, they would f e e l less 'overwhelmed' by the range of tasks to be done and thus be less prone to see time as an obstacle. They do indeed 'create knowledge i n use 1. However, i t remains a point of interest that differences between experience groups are not c l e a r e r . Time management features quite prominently i n the l i t e r a t u r e (Hummel, 1967; Smith & Andrews, 1989; Sergiovanni, 1991; and others). How far a p r i n c i p a l i s able to make the most e f f e c t i v e use of time i s l i k e l y to influence his or perception of time pressure. Of course time management requires the time manager to have a d e f i n i t i o n of what e f f e c t i v e use of time means. This d e f i n i t i o n requires decisions to be made about which aspects of the p r i n c i p a l ' s r e s p o n s i b i l i t i e s are assigned d i f f e r i n g degrees of p r i o r i t y . It i s at t h i s stage i n any consideration of time management that the issue of 'important' and 'urgent' arises because p r i o r i t y does not necessarily equate with most important. Everard and Morris (1990) highlight the d i s t i n c t i o n between important and urgent matters with regard to establishing p r i o r i t i e s and o f f e r a means by which p r i n c i p a l s can avoid being swept along by continual c r i s i s management. They suggest planned time for the important issues, both on a short and long-term basis. However, a number of the anecdotal responses i n the study describe the d i f f i c u l t y i n planning such time for evaluation: An a c t i v i t y which most p r i n c i p a l s acknowledge as important. The c o r o l l a r y of such comments though, i s that i f c e r t a i n tasks are to be put to one side or completed i n a less rigorous way than p r i n c i p a l s might l i k e , i t i s more acceptable to leave tasks such as the formal evaluation of teaching. If t h i s analysis i s correct, such an attitude must be based on some perception that p r i n c i p a l s have formed about the 'external' value of formal evaluation. Given that formal evaluation of teaching i s one of a p r i n c i p a l ' s contractual obligations, t h i s perception of the value of evaluation must i n part be based on the attitude of the school board. In other words, p r i n c i p a l s must have formed an understanding that the consequences of not evaluating are less severe than those for not doing something else. Conclusion School d i s t r i c t c o l l e c t i v e agreements provide l i t t l e assistance i n s p e c i f i c a l l y determining the purpose of formal evaluation. However, there i s evidence that the way the process i s outlined i n most, i m p l i c i t l y favours an orientation towards accountability. Therefore, the clauses r e l a t i n g to evaluation i n B r i t i s h Columbia school d i s t r i c t c o l l e c t i v e agreements present an austere view of personnel review, apart from a very few d i s t r i c t s where professional growth plans are i n place and the purpose of the process i s c l e a r l y stated as growth and development. P r i n c i p a l s , working within the confines of these c o l l e c t i v e agreements, c l e a r l y view the f i n a l report writing of an evaluation as problematic, as indeed they do the ent i r e process of an evaluation leading to a "less than s a t i s f a c t o r y " report. The study shows that few evaluations are conducted and only a very small percentage of these lead to "less than s a t i s f a c t o r y " reports. Many p r i n c i p a l s are l i k e l y to believe that both teacher growth and development and accountability for the q u a l i t y of teaching are important. However, the vast majority of respondents i n t h i s study were able to d i s t i n g u i s h one as more important than the other when asked to do so. The majority of p r i n c i p a l s believe the most important purpose to be teacher growth and development, which i s a p a r t i c u l a r l y important finding given the summative nature of the evaluation process i n most school d i s t r i c t s . A further d i s t i n c t i o n , with regard to purpose, can be drawn between male and female p r i n c i p a l s and those with d i f f e r e n t l e v e l s of experience. Women are more l i k e l y to opt for growth and development than men; p r i n c i p a l s with more than ten years experience are more l i k e l y to opt for accountability than t h e i r less experienced colleagues. 156 Time emerged as the single most important obstacle to the conduct of evaluation. This perception i s borne of the b e l i e f that p r i n c i p a l s are being asked to perform too many functions r e s u l t i n g i n an i n a b i l i t y to perform some, such as formal evaluation, as well as they would l i k e . However, t h i s view of evaluation i s q u a l i f i e d by the b e l i e f of many pr i n c i p a l s that they s t i l l carry out evaluation well. This study shows that, generally, B r i t i s h Columbia p r i n c i p a l s do not consider that they need further t r a i n i n g i n formal evaluations leading to "satisfactory" reports. However, for f i n a l report writing and evaluations leading to "less than s a t i s f a c t o r y " reports, a greater need for t r a i n i n g i s expressed. The amount of time p r i n c i p a l s have spent i n t r a i n i n g varies considerably but, while i t i s impossible to comment on the q u a l i t y of t r a i n i n g , i t i s cl e a r that on-going t r a i n i n g i n formal evaluation i s available and i s undertaken by p r i n c i p a l s . Returning to the question set out i n the framework for the study i n Chapter III, the following answers can be given: a) The most important purpose of formal evaluation for the majority of p r i n c i p a l s i s teacher growth and development. b) The evaluation process i s l a r g e l y summative and geared more towards the accountability of teaching. c) P r i n c i p a l s have received modest amounts of t r a i n i n g i n formal evaluation and need for further t r a i n i n g i s l i m i t e d to report writing and evaluations leading to "less than s a t i s f a c t o r y " reports. 157 d) The most important obstacle to carrying out formal evaluation i s lack of time. e) More s i m i l a r i t y than difference e x i s t s between the views of men and women p r i n c i p a l s on formal evaluation. f ) More s i m i l a r i t y than difference e x i s t s between the views of experienced and less experienced p r i n c i p a l s on formal evaluation. The objective of the study was to e l i c i t the views of B r i t i s h Columbia p r i n c i p a l s about the formal evaluation of teaching. While i t has achieved t h i s objective, what emerges i s an i n t e r e s t i n g view of the p r i n c i p a l ' s role generally. The issue of time, which i s bound up i n t h i s general view of the role of p r i n c i p a l , implies much about the l e v e l of p r i o r i t y p r i n c i p a l s are w i l l i n g or able to assign to the evaluation of the primary function of schools - teaching and learning. Key Findings 1. The formal evaluation process i n the vast majority of school d i s t r i c t s i s i m p l i c i t l y geared to accountability for the q u a l i t y of teaching. 2. The majority of p r i n c i p a l s consider the most important purpose of formal evaluation to be teacher growth and development. 3. P r i n c i p a l s place time as the most important obstacle to carrying out formal evaluation. Evaluation cycles appear to magnify the problem but there i s no d i r e c t l i n k between s t a f f size or teaching load and time as an 158 obstacle. Clearly p r i n c i p a l s perceive workload as a major contributing factor to time pressures. 4. Post-graduate degree and previous t r a i n i n g have no bearing on the extent to which p r i n c i p a l s f e e l i n need of further t r a i n i n g i n formal evaluation. 5. P r i n c i p a l s who are not required to evaluate on a regular cycle and those who are not bound by stated d i s t r i c t evaluation c r i t e r i a , write "less than s a t i s f a c t o r y " reports more often than p r i n c i p a l s who do have to meet these requirements. 6. Few evaluations are conducted and only a very small percentage res u l t i n "less than s a t i s f a c t o r y " reports. 7. P r i n c i p a l s with less than six years experience express a greater need for further t r a i n i n g i n formal evaluation than t h e i r more experienced colleagues. Recommendations Policy It may be time for the Ministry of Education and in d i v i d u a l school boards to re-assess t h e i r expectations of school p r i n c i p a l s . This re-assessment should focus on the balance between the bureaucratic r e s p o n s i b i l i t i e s of p r i n c i p a l s and t h e i r role as educational leaders and i n s t r u c t i o n a l managers. If t h i s balance has swung too much i n the d i r e c t i o n of bureaucratic functions the result may 159 be a less than f u l l y e f f e c t i v e employment of the expertise and background i n education that p r i n c i p a l s possess. This recommendation does not preclude p r i n c i p a l s also looking again at the p r i o r i t i e s they set for themselves and examining t h e i r time management strategies. If roles and assignments are to be examined i t would be helpful to consider the opportunities available for introducing additional evaluators. Coming from an education system (and a s l i g h t l y d i f f e r e n t school culture) where i t i s e n t i r e l y acceptable for heads of department to 'evaluate' t h e i r departmental colleagues, and given the time pressures that p r i n c i p a l s speak of, spreading the workload of evaluation seems worthy of exploration. F i n a l l y , a re-assessment of the value of formal evaluation, as currently practised, would be timely. If sizeable numbers of p r i n c i p a l s are questioning the value of the process and even greater numbers attest to the d i f f i c u l t y i n carrying out the role of evaluator, a concern i s raised as to how e f f e c t i v e formal evaluation can be i n these circumstances. Research Many of the findings from t h i s study are far from conclusive. More detailed investigation of the evaluation practices of B r i t i s h Columbia p r i n c i p a l s would be very in t e r e s t i n g , e s p e c i a l l y with regard to the i n t e r a c t i o n of 160 informal and formal methods. Further research into the gender difference i n purpose i d e n t i f i e d i n t h i s study i s necessary. F i n a l l y , i t would also be i n t e r e s t i n g to see further study on the s p e c i f i c nature of the evaluation t r a i n i n g offered by school d i s t r i c t s , i t s take up by p r i n c i p a l s , and the extent to which un i v e r s i t y and college master's degree programmes see t h i s aspect of the p r i n c i p a l ' s role as fundamental by making i t required study. 161 References Adams, R. D. (1982). Teacher development: A look at changes i n teacher perception across time. Paper presented at the annual conference of the American Educational Research Association, New York. (ERIC DRS No. ED 214 926). Adams, R. D. & Martray, C. (1980). Correlates of teacher perceived problems. Paper presented at the annual conference of the Mid South Educational Research Association, New Orleans. (ERIC DRS No. ED 195 567). Airasian, P. W. (1993). Teacher assessment: Some issues for p r i n c i p a l s . NASSP B u l l e t i n , 77, (Oct), 55-65. Alder, S., Laney, J., & Packer, M. (1993). Managing women: Feminism and power i n educational management. Buckingham, UK: Open University Press. A l l s t o n , D., Rymhs. R., & Shultz. L. (1993). E f f e c t i v e teaching does make a difference. Alberta Journal of Educational Research, 39, (2), 191-203. Andrews, T., & Barnes, S. (1990). Assessment of teaching. In R. Houston (Ed.), Handbook of research on teacher education. New York: Macmillan. Antosz, J. L. A. (1990). Teacher evaluation i n B r i t i s h Columbia as described i n recent c o l l e c t i v e agreements: A comparative analysis of reported practice as recommended i n the l i t e r a t u r e (Magistral major paper, University of B r i t i s h Columbia, 1990). Bailey, G. D. (1984). An evaluator's guide to diagnosing and analyzing teaching s t y l e s . NASSP B u l l e t i n , 68, (469), 19-25. B a l l , S. (1987). The m i c r o p o l i t i c s of the school: Towards a theory of school organisation. London: Methuen. Bartholomew, B. R. (1974). Teachers' i n s t r u c t i o n a l problems, 1974: NEA survey. Today's Education, 63, (3), 78- 80. Beare, H. (1989, September 25). Educational administration i n the 1990s. Paper presented at the national conference of the Australian Council for Educational Administration, University of New England, Armidale, New South Wales, A u s t r a l i a . 162 Beck, L. G. & Murphy, J. (1993). Understanding the p r i n c i p a l s h i p : Metaphorical themes, 1920s - 1990s. New York: Teachers College Press. Black, S. (1993). How teachers are reshaping evaluation procedures. Educational Leadership, 51, (Oct), 38-42. Blumberg, A. & Greenfield, W. (1986). The e f f e c t i v e p r i n c i p a l (2nd ed.). Newton, MA: A l l y n and Bacon. Bolton, D. L. (1980). Evaluating administrative personnel i n school systems. New York: Teachers College Press. Bridges, E. (1986). The incompetent teacher. Philadelphia, PA: Falmer Press. Cangelosi, J. (1991). Evaluating classroom i n s t r u c t i o n . New York: Longman. Cherington, E. (1989). Information provided on the telephone from the Public Relations Department, B.C. Ministry of Education. Christensen, E. E. (1986). Teacher evaluation - who needs i t ? Roeper Review, 9, (1), 19-23. Clement, J. P., DiBella, C. M., Eckstrom, R. B., & Tobias, S. (1977). No room at the top? American Education, 13, (5), 20-23, 26. Cruickshank, D. R. (1974). Perceived problems of secondary school teachers. Journal of Educational Research, 68, (4), 154-159. Darling-Hammond, L. (1986). A proposal for evaluation i n the teaching profession. Elementary School Journal, 86, (4), 531-551. Darling-Hammond, L., Wise, A., fit Pease, S. (1983). Teacher evaluation i n the organizational context: A review of the l i t e r a t u r e . Review of Educational Research, 53, 285- 328. Darnell, M. A. M. (1993). Self-concept and attitudes of selected Texas public school teachers r e s u l t i n g from the Texas teacher appraisal system (Doctoral d i s s e r t a t i o n , East Texas State University, 1994). Dissertation Abstracts International, 54/08, 2815. 163 Edgar, W. (1991, December 6). Exam results are never referred to. Times Educational Supplement, 3936, 24. Estosito, J. P., Smith, G. E. & Burbank, H. J. (1975). A delineation of the supervisory r o l e . Education, 96, ( F a l l 1975), 63-67. Everard, K. B. & Morris, G. (1990). E f f e c t i v e school management. London: Paul Chapman. Everhart, R. B. (1988). Fieldwork methodology i n educational administration. In Boyan, N. J. (Ed.), Handbook of research on educational administration. New York: Longman. F i s h e l , A. & Pottker, J. (1975). Performance of women pr i n c i p a l s : A review of behavioral and a t t i t u d i n a l studies. Journal of the National Association of Women Deans, Administrators, and Counselors, 38, (3), 110-117. Frasher, J. M. & Frasher, R. S. (1979). Educational administration: A feminine profession. Educational Administration Quarterly, 15, (2), 1-13. Freidson, E. (1972). Profession of medicine: A study of the sociology of applied knowledge. New York: Dodd Mead. Grambs, J. D. (1976). Women and administration: Confrontation or accommodation? Theory Into Practice, 15, (4), 293-300. Grobman, H. & Hines, V. A. (1956). What makes a good p r i n c i p a l ? NASSP B u l l e t i n , 40, (223), 5-16. Gross, N. & Trask, A. E. (1964). Men and women as elementary school p r i n c i p a l s . (Final Report No. 2, Cooperative Research Project No. 853, Graduate School of Education, Harvard University, 1964). Gross, N. & Trask, A. E. (1976). The sex factor and the management of schools. New York: John Wiley and Sons. Guba, E. G., & Lincoln, Y. S. (1981). E f f e c t i v e evaluation. San Francisco: Jossey-Bass. Haefele, D. L. (1992). Evaluating teachers: An altern a t i v e model. Journal of Personnel Evaluation i n Education, 5, 335-345. 164 Harris, B. M. & Monk, J. M. (1992). Personnel administration i n education (3rd e d i t . ) . Toronto: A l l y n and Bacon. Her Majesty's Stationery O f f i c e , (1991). The Education (School Teacher Appraisal) Regulations 1991. London: HMSO. Housego, I. (1989). P r i n c i p a l s ' evaluation of teachers i n a B r i t i s h Columbia school d i s t r i c t : Are the reports professional or bureaucratic documents? The Alberta Journal of Educational Research, 35, (3), 196-216. Hogben, D. (1981). The c l i n i c a l mind: Some implications for educational research and teaching t r a i n i n g . South P a c i f i c Journal of Education, 10, (1). Huddle, G. (1985). Teacher evaluation - How important for e f f e c t i v e schools? Eight messages from research. NASSP B u l l e t i n , 69, (3), 58-63. Hummel, C. (1967). Tyranny of the urgent. Chicago, IL: Intervarsity Press. Hunter, M. (1985). What's wrong with Madeline Hunter? Educational Leadership, 42, (5), 57-60. Kauchak, D., Peterson, K., & D r i s c o l l , A. (1985). An interview study of teachers' attitudes toward teacher evaluation practices. Journal of Research and Development i n Education, 19, 32-37. Kelsey, G., Lupini, D., & Clinton, A. (1995). The ef f e c t s of l e g i s l a t i v e change on the work of B r i t i s h Columbia school superintendents. Report presented to the annual meeting of the B r i t i s h Columbia School Superintendents' Association, Richmond, B r i t i s h Columbia. Langlois, D., & Colarusso, M. (1988). Improving teacher evaluation. The Education Digest, 54, (11), 13-15. Lower, M. (1987). A study of teachers' and p r i n c i p a l s ' perceptions and attitudes toward the evaluation of teachers. Unpublished doctoral d i s s e r t a t i o n . The Ohio State University, Columbus, OH. Lusty, M. G. F. (1991). Teacher appraisal: Teachers' perceptions of an LEA teacher appraisal scheme and i t s implementation [London, teacher perceptions](Doctoral d i s s e r t a t i o n . Open University, 1993). Dissertation Abstracts International, 54/02, 354. Medley, D., Coker, H., fit Soar, R. (1984). Measurement based evaluation of teacher performance: An empirical approach. New York: Longman. Meskin, J. D. (1974). The performance of women school administrators - A review of the l i t e r a t u r e . Administrator's Notebook, 23, (1), 1-4. Morris, V. C , Crowson, R. L., Porter-Gehrie, C. fit Hurwitz, E. (1984). P r i n c i p a l s i n action: The r e a l i t y of managing schools. Columbus, OH: Charles E. M e r r i l l . Morrow, J. E. et a l (1985). Improving teacher effectiveness: Perceptions of p r i n c i p a l s . Education, 105, (4), 385-390. Ne v i l l e , M. (1988). Assessing and teaching language: Literacy and oracy i n schools. Basingstoke, UK: MacMillan Education. Ozga, J. (1993). Women i n educational management. Buckingham, UK: Open University Press. Page, J. A. St Page, F. M. (1985). P r i n c i p a l s ' perceptions of t h e i r role and the perceived effectiveness of th e i r academic preparation. College Student Journal, 19, (1), 2-16. Peterson, D. (1986). Developing teacher evaluation systems with potential for increasing student performance. Educational Research Quarterly, 10, (2), 39-46. Pigford, A. B. fit Tonnsen, S. (1993). Women i n school leadership: Survival and advancement guidebook. Lancaster, PA: Technomic Publishing. Poster, C , St Poster, D. (1993). Teacher appraisal: Training and implementation. London: Routledge. Province of B r i t i s h Columbia (1987). The Industrial Relations Act 1987. V i c t o r i a : Ministry of Education. Province of B r i t i s h Columbia (1987a). The Teaching Profession Act 1987. V i c t o r i a : Ministry of Education. Province of B r i t i s h Columbia (1995). 1994/1995 Public and Independent Schools Book. V i c t o r i a : Ministry of Education. 166 Reavis, C. A. (1978). Teacher improvement through c l i n i c a l supervision. Phi Delta Kappan, 584, Regan, H. B. & Brooks, G. H. (1995). Out of women's experience: creating r e l a t i o n a l leadership. Thousand Oaks, CA: Corwin Press. Rooney, J. (1993). Teacher evaluation: No more "super"vision. Educational Leadership, 51, (Oct), 43-44. Rossow, L. F. (1990). The p r i n c i p a l s h i p : Dimensions i n in s t r u c t i o n a l leadership. Englewood C l i f f s , NJ: Prentice- H a l l . Schonberger, V. L. (1986). The e f f e c t i v e supervision of professional colleagues: S e l f - d i r e c t i o n and professional growth. High School Journal, 69, (4), 248-254. Scriven, M. (1987). V a l i d i t y i n personnel evaluation. Journal of Personnel Evaluation i n Education, 1, 9-23. Sergiovanni, T. J. (1977). C l i n i c a l supervision: A review of the research. Journal of Research and Development i n Education, 9, 21. Sergiovanni, T. J. (1991). The p r i n c i p a l s h i p : A r e f l e c t i v e practice perspective (2nd ed.). Needham Heights, MA: A l l y n and Bacon. Shakeshaft, C. (1987). Women i n educational administration. Newbury Park, CA: Sage. Shakeshaft, C. (1989). Women i n educational administration. Newbury Park, CA: Sage. Sharp, W. L. & Walter, J. K. (1994). The p r i n c i p a l as school manager. Lancaster, PA: Technomic. Smith, W. F. & Andrews, R. L.(1989). Instructional leadership: How p r i n c i p a l s make a difference. Alexandria, VA: Association for Supervision and Curriculum Development. Starratt, R. J. (1993). A modest proposal: Replace supervision with super-vision. The School Administrator, 50, (4), 35. Stodolsky, S. (1988). The subject matters. Chicago: University of Chicago Press. Storey, V. J. & Housego, I. (1980). Personnel supervision: A descriptive framework. The Canadian Administrator, 19, (6), 1-4. Sybouts, W. & Wendel, F. C. (1994). The t r a i n i n g and development of school p r i n c i p a l s : A handbook. Westport, CT: Greenwood Press. Tibbetts, S. (1980). The woman p r i n c i p a l : Superior to the male? In Far rant, P". A. (Ed.), Strategies and attitudes: Women i n educational administration. Washington, DC: National Association of Women Deans, Administrators, and Counselors. Townsend, D. (1987). Components of a model of teacher evaluation. Education Canada, 27, (2), 24-30. Ubben, G. C. & Hughes, L. W. (1992). The p r i n c i p a l : Creative leadership for e f f e c t i v e schools. Needham Heights, MA: A l l y n and Bacon. VanScriver, J. (1990). Teacher dismissals. Phi Delta Kappan, 72, 318-319. Webster, Sr., W. G. (1994). Learner-centered p r i n c i p a l s h i p : The p r i n c i p a l as teacher of teachers. Westport, CT: Praeger. Withall, J. & Wood, F. (1979). Taking the threat out of classroom observation and feedback. Journal of Teacher Education, 30, (1), 55. Wood, C. J. (1992). Toward more e f f e c t i v e teacher evaluation: Lessons from n a t u r a l i s t i c inquiry. NASSP B u l l e t i n , 76, (Mar), 52-59. A p p e n d i x A2 169 6. If you answered "Yes" or "In progress" to 5 above, what is your specialization in? • Educational Administration • Curriculum • Other (please specify) 7. How many years of experience (include the present year as one) do you have as: a) Principal? b) Vice Principal? PART B: CURRENT SCHOOL INFORMATION 8. What is your current Administrative Officer assignment? • Principal • Vice Principal • District Principal 9. What percentage of your official appointment is allocated to each of the following? a) Administration % b) Teaching % c) District % 10. Which of the following best describes your present school? • School enrolling only elementary grades (any grades from K-7) • School enrolling only secondary grades (any grades from 8-12) • School enrolling both elementary and secondary grades • I do not have a school assignment 11. What is the number of your school district? 12. How many teachers, including the principal, do you have on staff? (please report headcount and not FTE) PART C: ADMINISTRATIVE OFFICER AS A FORMAL EVALUATOR OF TEACHING This part of the questionnaire is about the formal evaluation of teaching. Formal evaluation of teaching means the evaluation process which takes place according to the provisions of the district collective agreement and/or legislation. This process results in the writing of a final report concluding that a teacher's classroom situation is either "satisfactory" or "less than satisfactory". 13. The formal evaluation of teaching is part of your responsibilities. Do you think it should be? • Yes • No • Not sure Appendix A3 170 14. What do you consider to be the most important purpose of the formal evaluation of teaching? (please check one response) • Teacher growth and development • Accountability for the quality of teaching • Other (please specify) 15. How well do you carry out the formal evaluation of teaching? Please check the most appropriate description below: Very Poorly Poorly Adequately Well Very Well • • • • • 16. Please indicate the duration and number of any in-service workshops, seminars, university courses (or components thereof), etc. that addressed the formal evaluation of teaching and which you have attended since September 1988: • One day or less Number attended • Between two days and one week Number attended • More than one week but less than one full term Number attended • One full university/college term Number attended 17. Please state, as accurately as possible, the total number of formal evaluations of teaching you have carried out since September 1988, and the number of those that resulted in "satisfactory" reports and "less than satisfactory" reports: Number of Formal Teaching Evaluations Number of "Satis- factory" Reports Number of "Less than satisfactory" Reports 18. Please list, in rank order, what you consider to be the main obstacles (up to a maximum of 3) to your carrying out the formal evaluation of teaching, with # 1 being the greatest obstacle: 1. 2. 3. A p p e n d i x A4 171 19. This question deals with your views on different aspects of the formal evaluation of teaching and asks you to consider evaluations that result in a "satisfactory" report. Almost all collective agreements identify four phases in the formal teaching evaluation process. These are a) the pre-evaluation conference(s); b) classroom observations; c) post-observation conferences; and d) writing the final report. The four parts to this question each give a series of statements relating to these phases. You are asked to indicate your level of agreement with the statements. a) Pre-evaluation conference(s) (to discuss purpose, criteria, time-frame, etc.) For me, I consider this phase: Strongly Disagree Disagree Agree Strongly Agree I) Stressful Jty -Complex I) Time-consuming IV) I need more training b) Classroom observations For me, I consider this phase: Strongly Disagree Disagree Agree Strongly - Agree".' I) Stressful ' 1} Complex' : - " - l).TJrne-consurnlng IV) 1 need more training c) Post-observation conferences For me, 1 consider this phase: Strongly Disagree Disagree Agree Strongly Agree I) Stressful H) Complex I) Time-consuming IV) 1 need more training d) Writing the final report (including any discussions/feedback on draft report, etc.) For me, 1 consider this phase: Strongly Disagree Disagree Agree Strongly Agree'- '- ' I) Stressful I) Complex 1) Time-consuming - IV) 1 need more training Appendix A5 172 20. N.B. Please ignore this question if you have never written a "less than satisfactory" report. This question deals with your views on different aspects of the formal evaluation of teaching and asks you to consider evaluations that result in a "less than satisfactory" report. The four parts to this question each give a series of statements relating to a different phase of formal evaluation. You are asked to indicate your level of agreement with the statements. a) Pre-evaluation conference(s) (to discuss purpose, criteria, time-frame, etc.) For me, 1 consider this phase: Strongly Disagree Disagree Agree Strongly . Agree I) Stressful 11) Complex I) Time-consuming IV) 1 need more training b) Classroom observations For me, 1 consider this phase: Strongly Disagree Disagree Agree -Strongly . .Agree I) Stressful J) Complex I) Time-consuming IV) 1 need more training c) Post-observation conferences For me, 1 consider this phase: Strongly Disagree Disagree Agree Strongly Agree I) Stressful S) Complex I) Time-consuming IV) 1 need more training d) Writing the final report (including any discussions/feedback on draft report etc.) For me, 1 consider this phase: Strongly Disagree Disagree Agree Strongly. Agree I) Stressful -I)' Complex . IH) Time-consuming IV) 1 need more training Appendix A6 173 21. If there are any additional points you would like to make regarding the formal evaluation of teaching please do so in the space below. Thank you for taking the time to participate in this project. Appendix B 174 EVALUATION PHASES IN THE COLLECTIVE AGREEMENTS The table below shows the requirement for c e r t a i n phases i n the formal evaluation of teaching, as contained i n the seventy-five B r i t i s h Columbia school d i s t r i c t c o l l e c t i v e agreements (see key f o r headings and symbols). School D i s t r i c t Pre Evl Ob Pst Ob Fin Rep School D i s t r i c t Pre Svl 0b Pst 0b F i n Rep School D i s t r i c t Pre Evl 0b Pst 0b Fin Rep 1 i i 1̂ 31 X 1 X X 59 i 1̂ i 2 i i 1̂ i L 32 i i \| o 60 i i 1̂ i o 3 i i •i i o 33 2 i i o 61 i i i i 4 i i 34 i i ^ o 62 i i i i 7 i i i <| o 35 i i i o 63NB i i i i 9 i i i i 36 i i i i 64 i i i i o 10 i i i i ° 37 i i i X T 65 i i i o X 11 i i i i 38 i i i 66NB i i 1̂ 12 i i i i o 39 i i i i o 68 i i i] o 13 i i i i o 40NB i i •i 69 i i i o i o 14 i i i i o 41NB i i i L 70 i i i 15 i i i i o 42 i i i i o 71 i i i o 16 i i i 43 i i i o 72 i i i o 17 i i i i o 44 i o i i ° 75 i i 1̂ 18 i i i 45 i i i i o 76 2 i i i o 19 i i i i 46 i i i j o 77 i i i o 21 i i i i o 47 i * i • i 80 i i i i 22 i i i ° 48 i i i i 81 X i X i o 23 i i i \| o 49 i i i o 84 i i i i 24 i i i o i 50 i i i \| o 85 i i i i 26 i i i i o 52 i i i i 86 i i X 27 i i \| o 54 i ) i 87 i i i o 28 i i i i o 55 i i i vJ O 88 i i i i o 29 i i 56 i i i ° 89 i i i v| O 30 i i i VJ O 57 i i i 1 92 i i i o KEY Headings: Pre E vl = Pre-evaluation conference Ob = Classroom observations Pst 0b = Post observation Conference F i n Rep = F i n a l report conference Symbols: = This phase i s stated i n the c o l l e c t i v e agreement x = This phase i s not stated i n the c o l l e c t i v e agreement o = The opportunity for such a meeting must be made av a i l a b l e * = "Process should be agreed" t = Second meeting i s a v a i l a b l e to discuss process i f necessary 0 = More than one, i f necessary T = But "parties should t r y to agree on the report" L = For teachers who receive a "less than s a t i s f a c t o r y " report NB = 4 0 : Provision for 'peer evaluation' 41 : Four step 'professional growth plan' model 63,66: Provision for a 'short' report for excellent teachers Appendix C 175 PERMISSABLE DATA IN EVALUATION FINAL REPORT SOURCE OF DATA COLLECTIVE AGREEMENT SCHOOL DISTRICT NUMBER C l a s s r o o m o b s e r v a t i o n d a t a o n l y 1, 4, 10, 12, 18, 28, 32, 36, 37, 44, 52, 54, 60, 63, 75, 80, 88 TOTAL =17 C l a s s r o o m o b s e r v a t i o n d a t a : • Primarily • P r i n c i p a l l y •Generally • Normally •Not n e c e s s a r i l y 13, 14, 15, 17, 19, 30, 42 89 50, 61 77 40 TOTAL 12 C l a s s r o o m o b s e r v a t i o n d a t a p l u s : •General performance -General con t r i b u t i o n / work of the teacher •Other p e r t i n e n t / f a c t u a l information/material •Other information •Observation of other required duties •Work d i r e c t l y r e l a t e d to teacher's assignment •Multiple sources of data •Not s p e c i f i e d 65, 68, 71, 72 46, 64 49, 85 87, 92 66 21 24 70 TOTAL =14 N o t s t a t e d 2, 3, 7, 9, 11, 22, 23, 26, 27, 29, 31, 33, 34, 35, 38, 39, 41, 43, 45, 47, 48, 55, 56, 57, 59, 62, 69, 76, 81, 84, 86 TOTAL = 32 Appendix D 176 EVALUATION CRITERIA AND CYCLES The table below shows the provision of evaluation cycles and c r i t e r i a as contained i n the seventy-five B r i t i s h Columbia school d i s t r i c t c o l l e c t i v e agreements. S c h o o l D i s t . Cycle C r i t - e r i a . S c h o o l D i s t . Cycle C r i t - e r i a . S c h o o l D i s t . Cycle C r i t - e r i a . 1 i i 31 X X 59 X X 2 i i 32 X 60 i i 3 X X 33 i 61 X X 4 i X 34 X X 62 X 1̂ 7 i X 35 X i 63 i 9 i i 36 i i 64 i i 10 i X 37 i i 65 i i 11 X i 38 i 66 i i 12 i X 39 X i 68 i 13 X i 40 X i 69 i i 14 X i 41 X X 70 i i 15 X i 42 i X 71 i i 16 X X 43 X X 72 X i 17 X X 44 X X 75 i i 18 i X 45 X X 76 i 19 X i 46 X i 77 X X 21 X X 47 X i 80 i i 22 i i 48 X i 81 X X 23 X 49 i i 84 i i 24 X X 50 i i 85 i i 26 X i 52 i X 86 X i 27 i i 54 i X 87 X i 28 X X 55 X i 88 i i 29 X X 56 X i 89 X X 30 X X 57 i i 92 i i KEY i = Stated x = Not stated Appendix E 177 SCHOOL DISTRICT NUMBERS, NAMES, AND SIZES SMALL (0-2,999)* MEDIUM (3,000-14,999)* LARGE (15,000+)* 03 -- Kimberley 01 - Fernie 23 -- Central Okanagan 04 -- Windermere 02 - Cranbrook 24 • - Kamioops 09 • - Castlegar 07 - Nelson 34 • - Abbotsford 10 • - Arrow Lakes 11 - T r a i l 35 -- Langley 12 • - Grand Forks 15 - Penticton 36 -- Surrey 13 • - Kettle V a l l e y 22 - Vernon 37 • - Delta 14 -- Southern Okanagan 27 - Cariboo-Chilcotin 38 -- Richmond 16 • - Keremeos 28 - Quesnel 39 • - Vancouver 17 -- Princeton 33 - Chilliwack 41 -- Burnaby 18 -- Golden 40 - New Westminster 43 • - Coquitlam 19 • - Revelstoke 42 - Maple Ridge 44 • - North Vancouver 21 • - Armstrong-Spallumcheen 45 - West Vancouver 57 -- Prince George 26 • - North Thomson 46 - Sunshine Coast 61 -- Greater V i c t o r i a 29 -- L i l l o o e t 47 - Powell River 68 • - Nanaimo 30 -- South Cariboo 48 - Howe Sound 31 -- Merritt 52 - Prince Rupert 32 -- Hope 54 - Bulkley V a l l e y 49 -- Central Coast 56 - Nechako 50 -- Queen Charlotte 59 - Peace River South 55 -- Burns Lake 60 - Peace River North 64 -- Gulf Islands 62 - Sooke 66 -- Lake Cowichan 63 - Saanich 76 -- Agassiz-Harrison 65 - Cowichan 77 -- Summerland 69 - Qualicum 80 -- Kitimat 70 - Alberni 81 -- Fort Nelson 71 - Courtenay 84 -- Vancouver Island West 72 - Campbell River 85 -- Vancouver Island North 75 - Mission 86 -- Creston-Kaslo 88 - Terrace 87 -- Sti k i n e 89 - Shuswap 92 -- Nisga'a *Student enrolments ( i n d i v i d u a l school d i s t r i c t assignations to d i s t r i c t s i z e are based on 1995 enrolments). Appendix F 178 SAMPLE EVALUATION ARTICLE FROM A BRITISH COLUMBIA SCHOOL DISTRICT COLLECTIVE AGREEMENT Article 5 Evaluation Of Teaching 5.1 Both the [local] Teachers' Association and the Board of School Trustees believe that students are best served when a high quality of classroom instruction and teaching performance is provided and maintained, and adequate assistance for teaching performance is provided. 5.2 All formal reports on the work of a teacher shall be in writing. 5.3 Before commencing observations, the evaluator shall meet with the teacher, discuss the purposes of the evaluation, the approximate time span and schedule of observations, and review the criteria to be applied in the evaluation and report writing process. 5.4 Not less than three (3) nor more than six (6) formal classroom observations which reflect the teacher's assignment, shall be conducted in completing the report process unless otherwise mutually agreed. 5.5 Periods chosen for observation shall be during normal periods of the school year and the teacher shall have the opportunity to select at least one third of the times. a) The evaluator shall provide the teacher with a written anecdotal statement at the end of each lesson observed. 5.6 Reports shall be prepared only by evaluators authorised under the School Act. 5.7 The report shall reflect only the teaching and learning situation within the teacher's responsibility, unless other aspects of the teacher's work are requested to be recognised by the teacher concerned. 5.8 Any written report that is satisfactory and that identifies weaknesses shall include constructive suggestions for improvements. In this case, a teacher may request a plan of assistance from the employer. 5.9 Except in the case of a final, less than satisfactory report, the employer in consultation with the teacher, shall develop a plan of assistance. At this meeting the teacher has the right to be accompanied by a member of the association. 5.10 Except under extraordinary circumstances where a plan of assistance is underway, formal evaluation will be postponed until the plan of assistance is completed. 5.11 The teacher shall be given a draft copy of a report at least forty-eight (48) hours prior to preparation of the final copy. He/she shall have the opportunity of meeting with the evaluator in the company of a member of the association, to discuss the draft. 5.12 The final report shall be filed in the teacher's personnel file. A copy shall be given to the teacher at the time of filing. 5.13 The teacher shall have the right to submit to the evaluator (within one week of receiving the final report) a written commentary on the report which shall be filed with all copies of the report. 179 APPENDIX G SUMMARY OF RESPONSE FREQUENCIES The following summary of response frequencies i s organised i n the same order as questions on the questionnaire. Two abbreviations are used from page 183 on: SR = An evaluation leading to a "satisfactory" report; LTSR = An evaluation leading to a "less than s a t i s f a c t o r y " report. Appendix G 180 R e s p o n d e n t s e x V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t M a l e 1 1 3 6 7 2 . 3 7 2 . 7 7 2 . 7 F e m a l e 2 5 1 2 7 . 1 2 7 . 3 1 0 0 . 0 1 . 5 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 1 8 7 M i s s i n g c a s e s 1 R e s p o n d e n t a g e V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t 3 0 - 3 4 y e a r s 2 2 l . l 1 . 1 1 . 1 3 5 - 3 9 y e a r s 3 5 2 . 7 2 . 7 3 . 7 4 0 - 4 4 y e a r s 4 33 1 7 . 6 1 7 . 6 2 1 . 3 4 5 - 4 9 y e a r s 5 6 5 3 4 . 6 3 4 . 6 5 5 . 9 5 0 - 5 4 y e a r s 6 48 2 5 . 5 2 5 . 5 8 1 . 4 5 5 - 5 9 y e a r s 7 32 17 . 0 1 7 . 0 9 8 . 4 6 0 - 6 5 y e a r s 8 3 1 . 6 1 . 6 1 0 0 . 0 T o t a l . 188 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 1 8 8 M i s s i n g c a s e s 0 M a s t e r s s p e c i a l i s a t i o n V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t A d m i n i s t r a t i o n 1 1 1 1 5 9 . 0 6 5 . 7 6 5 . 7 C u r r i c u l u m 2 2 5 13 . 3 1 4 . 8 8 0 . 5 O t h e r 3 33 1 7 . 6 1 9 . 5 1 0 0 . 0 1 9 10 . 1 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 1 6 9 M i s s i n g c a s e s 1 9 D o c t o r a l s p e c i a l i s a t i o n A d m i n i s t r a t i o n O t h e r V a l i d c a s e s F r e q u e n c y 7 P e r c e n t 3 . 7 . V a l i d P e r c e n t 77 . 8 9 5 . 2 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 M i s s i n g c a s e s 1 7 9 C u m P e r c e n t 77 . 8 1 0 0 . 0 E x p e r i e n c e a s a p r i n c i p a l C u m C u m C u m V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t 1 1 6 9 9 10 1 1 6 62 19 4 2 8 9 2 12 6 15 1 1 3 2 64 2 0 8 4 94 3 1 1 6 2 1 12 6 3 6 7 2 1 6 3 9 7 4 1 0 5 2 6 13 4 2 6 9 22 2 1 98 5 9 5 3 1 14 7 4 73 25 1 1 98 6 1 5 8 3 9 15 4 2 7 5 26 1 1 9 9 7 14 7 4 7 16 8 4 7 9 27 1 1 9 9 8 1 0 5 52 17 7 4 83 3 5 1 1 1 0 0 9 8 4 5 6 18 8 4 87 i d c a s e s 1 8 7 M i s s i n g c a s e s 1 T e a c h i n g l o a d a s a p e r c e n t a g e o f a s s i g n m e n t C u m V a l u e F r e q P e t P e t 0 83 4 5 4 5 3 1 1 4 5 5 6 3 4 9 8 1 1 4 9 10 12 6 5 6 12 5 3 58 13 1 1 5 9 15 2 1 6 0 C u m V a l u e F r e q P e t P e t 16 2 1 6 1 17 2 1 62 2 0 27 15 7 7 22 1 1 7 7 24 1 1 78 2 5 1 1 78 3 0 10 5 84 3 7 1 1 84 M i s s i n g c a s e s 3 C u m V a l u e F r e q P e t P e t 4 0 8 4 8 9 5 0 12 6 95 60 4 2 97 65 1 1 98 70 3 2 9 9 80 1 1 1 0 0 Appendix G 181 T y p e o f s c h o o l V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t E l e m e n t a r y 1 1 3 5 7 1 . 8 7 1 . 8 7 1 . 8 S e c o n d a r y 2 42 2 2 . 3 22 . 3 94 . 1 B o t h 3 1 1 5 . 9 5 . 9 1 0 0 . 0 T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 1 8 8 M i s s i n g c a s e s 0 S t a f f s i z e C u m C u m C u m V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t 2 1 1 1 22 5 3 5 0 4 3 2 1 8 7 4 2 1 2 2 3 6 3 54 44 3 2 8 9 6 3 2 3 24 5 3 5 6 48 1 1 8 9 7 2 1 4 25 1 0 5 62 4 9 1 1 9 0 8 2 1 5 26 5 3 64 5 0 2 1 9 1 9 5 3 8 27 3 2 6 6 52 1 1 9 1 10 3 2 10 28 6 3 6 9 54 1 1 92 1 1 8 4 14 2 9 2 1 7 0 56 1 1 92 12 6 3 1 7 3 0 4 2 72 60 2 1 94 13 6 3 2 1 3 1 4 2 7 5 62 2 1 9 5 14 4 2 23 32 4 2 7 7 63 2 1 9 6 15 10 5 28 33 3 2 78 66 1 1 9 6 16 2 1 2 9 3 5 2 1 7 9 72 1 1 97 17 7 4 33 3 6 2 1 8 1 74 1 1 97 18 8 4 37 38 1 1 8 1 80 1 1 98 19 1 1 38 3 9 2 1 82 81 1 1 98 2 0 10 5 4 3 4 0 2 1 83 90 1 1 9 9 2 1 8 ' 4 48 42 5 3 86 9 9 2 1 1 0 0 V a l i d c a s e s 1 8 5 M i s s i n g c a s e s 3 S h o u l d e v a l u a t i o n b e d o n e b y p r i n c i p a l s ? Y e s N o t s u r e V a l i d c a s e s u e F r e q u e n c y 1 1 8 1 3 . 6 T o t a l M i s s i n g c a s e s P e r c e n t 9 6 . 3 3 . 2 . 5 1 0 0 . 0 V a l i d P e r c e n t 9 6 . 8 3 . 2 M i s s i n g 1 0 0 . 0 C u m P e r c e n t 9 6 . 8 1 0 0 . 0 P u r p o s e o f e v a l u a t i o n G r o w t h a n d d e v e l o p m e n t A c c o u n t a b i l i t y O t h e r V a l i d c a s e s u e F r e q u e n c y 1 1 0 4 2 72 3 6 P e r c e n t 5 5 . 3 3 8 . 3 3 . 2 V a l i d P e r c e n t 3 . 2 M i s s i n g C u m P e r c e n t 5 7 . 1 5 7 . 1 3 9 . 6 . 9 6 . 7 3 . 3 1 0 0 . 0 T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 M i s s i n g c a s e s 6 How w e l l d o y o u d o e v a l u a t i o n ? V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t P o o r l y 2 12 6 . 4 6 . 5 6 . 5 A d e q u a t e l y 3 6 0 3 1 . 9 32 . 6 3 9 . 1 W e l l 4 8 1 4 3 . 1 4 4 . 0 8 3 . 2 V e r y w e l l 5 3 1 1 6 . 5 1 6 . 8 1 0 0 . 0 4 2 . 1 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 1 8 4 M i s s i n g c a s e s 4 N u m b e r o f o n e d a y c o u r s e s s i n c e S e p t e m b e r 1 9 8 8 C u m C u m C u m V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t 0 9 0 48 48 4 1 0 5 85 8 1 1 97 1 2 7 14 62 5 1 5 8 93 9 1 1 98 2 22 12 74 6 6 3 9 6 10 3 2 9 9 3 1 1 6 80 7 1 1 9 7 12 1 1 1 0 0 V a l i d c a s e s 1 8 8 M i s s i n g c a s e s 0 Appendix G 182 N u m b e r o f t w o d a y c o u r s e s s i n c e S e p t e m b e r 1 9 8 8 C u m C u m C u m V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t 0 97 52 52 3 14 7 93 • 6 3 2 1 0 0 1 32 17 69 4 6 3 96 2 32 17 86 5 4 2 98 V a l i d c a s e s 1 8 8 M i s s i n g c a s e s 0 N u m b e r o f o n e w e e k c o u r s e s s i n c e S e p t e m b e r 1 9 8 8 C u m C u m C u m V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t 0 1 6 4 87 87 3 2 1 98 1 0 2 1 1 0 0 1 12 6 94 4 1 1 98 2 6 3 97 5 1 1 99 V a l i d c a s e s 1 8 8 M i s s i n g c a s e s 0 N u m b e r o f o n e t e r m c o u r s e s s i n c e S e p t e m b e r 1 9 8 8 C u m V a l u e F r e q P e t P e t 0 1 5 2 8 1 8 1 1 3 1 16 97 V a l i d c a s e s 1 8 8 C u m V a l u e F r e q P e t P e t 2 4 2 99 3 1 1 1 0 0 M i s s i n g c a s e s 0 N u m b e r o f t r a i n i n g ' p o i n t s ' s i n c e S e p t e m b e r 1 9 8 8 C u m C u m C u m e F r e q P e t P e t V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t 0 5 3 3 10 18 1 0 73 23 1 1 9 5 1 2 1 1 1 14 11 5 3 7 6 25 2 1 9 6 2 9 5 1 9 12 1 0 5 81 2 6 1 1 9 6 3 1 9 1 0 2 9 13 2 1 82 30 2 1 9 7 4 6 3 32 15 9 5 8 7 3 5 1 1 98 5 17 9 4 1 16 3 2 8 9 36 1 1 98 6 26 14 5 5 17 2 1 9 0 4 7 1 1 9 9 7 2 1 5 6 18 4 2 92 5 0 2 1 1 0 0 8 5 3 5 9 2 0 2 1 93 9 1 0 5 64 22 2 1 94 V a l i d c a s e s M i s s i n g c a s e s N u m b e r o f e v a l u a t i o n s c o n d u c t e d s i n c e S e p t e m b e r 1 9 8 8 C u m C u m C u m V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t 0 4 2 2 18 1 0 5 5 5 36 2 1 8 7 1 1 1 3 19 3 2 5 7 37 1 1 88 2 3 2 4 20 8 4 6 1 3 9 1 1 88 3 1 1 5 2 1 3 2 63 4 0 3 2 9 0 4 8 4 9 • 22 1 1 63 42 3 2 9 1 5 8 4 14 23 2 1 64 44 1 1 92 6 8 4 18 24 5 3 67 4 5 1 1 92 7 4 2 2 0 25 1 1 6 73 4 7 1 1 93 8 5 3 23 26 4 2 7 5 5 0 4 2 9 5 9 3 2 24 27 2 1 7 6 54 1 1 9 6 10 8 4 2 9 28 1 1 7 7 55 1 1 9 6 1 1 2 1 3 0 29 1 1 7 7 6 0 2 1 9 7 12 13 7 3 7 3 0 5 3 80 7 0 2 1 98 13 1 1 38 3 1 2 1 81 78 1 1 9 9 14 5 3 4 0 32 3 2 83 94 1 1 9 9 15 14 8 48 3 3 2 1 84 9 9 1 1 1 0 0 16 1 1 48 34 1 1 84 17 2 1 4 9 35 3 2 86 V a l i d c a s e s 1 8 4 M i s s i n g c a s e s 4 N u m b e r o f " l e s s t h a n s a t i s f a c t o r y " r e p o r t s w r i t t e n - s i n c e S e p t e m b e r 1 9 8 8 C u m C u m C u m V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t V a l u e F r e q P e t P e t 0 1 1 4 62 62 3 6 3 98 6 1 1 1 0 0 1 4 6 25 86 4 1 1 99 2 16 9 95 5 1 .1 99 V a l i d c a s e s 1 8 5 M i s s i n g c a s e s Appendix G 183 M o s t i m p o r t a n t o b s t a c l e t o c o n d u c t i n g f o r m a l e v a l u a t i o n V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t • P e r c e n t N o n e 0 4 2 1 2 2 2 . 2 T i m e 1 1 2 0 63 8 6 5 2 6 7 . 4 U n i o n 2 7 3 7 3 8 7 1 . 2 C r i t e r i a 3 4 2 1 2 2 73 . 4 C o l l e c t i v e a g r e e m e n t 4 22 1 1 7 12 0 8 5 . 3 P r o c e s s 5 4 2 1 2 2 8 7 . 5 T e a c h e r a c c e p t a n c e 6 7 3 7 3 8 9 1 . 3 L a c k o f c y c l e 8 2 1 1 1 1 92 . 4 S t r e s s 9 1 5 5 9 2 . 9 D i s t r i c t e x p e c t 1 1 5 2 7 2 7 9 5 . 7 C y c l e 12 1 5 5 9 6 . 2 T r a i n i n g 13 2 1 1 1 1 9 7 . 3 L a c k o f e x p e r n c e 14 1 5 5 9 7 . 8 U n a g r e e d o n p u r p o s e 1 5 1 5 5 9 8 . 4 M y o w n b i a s e s 1 6 1 5 5 9 8 . 9 O t h e r 9 9 2 1 1 1 1 1 0 0 . 0 4 2 1 M i s s i n g T o t a l 1 8 8 1 0 0 0 1 0 0 0 V a l i d c a s e s 1 8 4 M i s s i n g c a s e s 4 S e c o n d m o s t i m p o r t a n t o b s t a c l e t o c o n d u c t i n g f o r m a l e v a l u a t i o n V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t T i m e 1 24 12 8 20 7 2 0 . 7 U n i o n 2 5 2 7 4 3 2 5 . 0 C r i t e r i a 3 4 2 1 3 4 28 . 4 C o l l e c t i v e a g r e e m e n t 4 23 12 2 19 8 4 8 . 3 P r o c e s s 5 17 9 0 14 7 62 . 9 T e a c h e r a c c e p t a n c e 6 1 1 5 9 9 5 72 . 4 S u b j e c t k n o w l e d g e 7 4 2 1 3 4 7 5 . 9 L a c k o f c y c l e 8 1 5 9 . 7 6 . 7 S t r e s s 9 5 2 7 4 3 8 1 . 0 D i s t r i c t e x p e c t 1 1 6 3 2 5 2 8 6 . 2 C y c l e 12 1 5 9 87 . 1 T r a i n i n g 13 4 2 1 3 4 9 0 . 5 L a c k o f e x p e r n c e 14 2 1 1 1 7 9 2 . 2 o t h e r 9 9 9 4 8 7 8 1 0 0 . 0 72 38 3 M i s s i n g T o t a l 1 8 8 1 0 0 0 1 0 0 0 V a l i d c a s e s 1 1 6 M i s s i n g c a s e s 72 T h i r d m o s t i m p o r t a n t o b s t a c l e t o c o n d u c t i n g f o r m a l e v a l u a t i o n V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t T i m e 1 8 4 3 12 1 1 2 . 1 U n i o n 2 6 3 2 9 1 2 1 . 2 C r i t e r i a 3 1 5 l 5 22 . 7 C o l l e c t i v e a g r e e m e n t 4 12 6 4 18 2 . 4 0 . 9 P r o c e s s 5 8 4 3 12 1 5 3 . 0 T e a c h e r a c c e p t a n c e 6 1 1 5 9 1 6 7 6 9 . 7 S t r e s s 9 4 2 1 6 1 7 5 . 8 M i n i s t r y e x p e c t 10 2 1 1 3 0 78 . 8 D i s t r i c t e x p e c t 11 3 1 6 4 5 8 3 . 3 T r a i n i n g 13 3 1 6 4 5 8 7 . 9 U n a g r e e d o n p u r p o s e 15 1 5 1 5 89 . 4 O t h e r 99 7 3 7 10 6 1 0 0 . 0 1 2 2 64 9 M i s s i n g T o t a l 1 8 8 1 0 0 0 1 0 0 0 V a l i d c a s e s 6 6 : M i s s i n g c a s e s 1 2 2 SR - P r e - e v a l u a t i o n c o n f e r e n c e i s s t r e s s f u l V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 83 44 . 1 44 . 6 44 . 6 D i s a g r e e 2 9 0 4 7 . 9 48 . 4 93 . 0 A g r e e 3 13 6 . 9 7 . 0 1 0 0 . 0 2 1 . 1 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 1 8 6 M i s s i n g c a s e s 2 SR - P r e - e v a l u a t i o n c o n f e r e n c e i s c o m p l e x V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 52 2 7 . 7 2 8 . 1 2 8 . 1 D i s a g r e e 2 84 44 . 7 4 5 . 4 7 3 . 5 A g r e e 3 44 23 . 4 2 3 . 8 9 7 . 3 S t r o n g l y a g r e e 4 5 2 . 7 2 . 7 1 0 0 . 0 3 1 . 6 M i s s i n g V a l i d c a s e s 1 8 5 T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 M i s s i n g c a s e s 3 Appendix G 1 SR - P r e - e v a l u a t i o n c o n f e r e n c e i s t i m e - c o n s u m i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 24 12 . 8 12 . 8 12 . 8 D i s a g r e e 2 4 7 2 5 . 0 2 5 . 1 3 8 . 0 A g r e e 3 95 5 0 . 5 5 0 . 8 8 8 . 8 S t r o n g l y a g r e e 4 2 1 1 1 . 2 1 1 . 2 1 0 0 . 0 1 . 5 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s . 1 8 7 M i s s i n g c a s e s 1 SR - P r e - e v a l u a t i o n c o n f e r e n c e r e q u i r e s m o r e t r a i n i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 62 3 3 . 0 3 3 . 9 3 3 . 9 D i s a g r e e 2 83 44 . 1 4 5 . 4 7 9 . 2 A g r e e 3 3 3 1 7 . 6 18 . 0 9 7 . 3 S t r o n g l y a g r e e 4 5 2 . 7 2 . 7 1 0 0 . 0 5 2 . 7 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 1 8 3 M i s s i n g c a s e s 5 S R - C l a s s r o o m o b s e r v a t i o n i s s t r e s s f u l V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 79 42 . 0 42 . 7 42 . 7 D i s a g r e e 2 93 . 4 9 . 5 5 0 . 3 9 3 . 0 A g r e e 3 1 1 5 . 9 5 . 9 9 8 . 9 S t r o n g l y a g r e e 4 2 1 . 1 1 . 1 1 0 0 . 0 3 1 . 6 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 1 8 5 M i s s i n g c a s e s 3 SR - C l a s s r o o m o b s e r v a t i o n i s c o m p l e x V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 4 1 2 1 . 8 2 2 . 3 2 2 . 3 D i s a g r e e 2 48 2 5 . 5 2 6 . 1 4 8 . 4 A g r e e 3 69 36 . 7 3 7 . 5 8 5 . 9 S t r o n g l y a g r e e 4 2 6 1 3 . 8 14 . 1 1 0 0 . 0 4 2 . 1 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 1 8 4 M i s s i n g c a s e s 4 S R - C l a s s r o o m o b s e r v a t i o n i s t i m e - c o n s u m i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 1 1 5 . 9 5 . 9 5 . 9 D i s a g r e e 2 19 10 . 1 1 0 . 2 1 6 . 0 A g r e e 3 95 5 0 . 5 5 0 . 8 6 6 . 8 S t r o n g l y a g r e e 4 62 3 3 . 0 33 . 2 1 0 0 . 0 1 . 5 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 1 8 7 M i s s i n g c a s e s 1 SR - C l a s s r o o m o b s e r v a t i o n r e q u i r e s m o r e t r a i n i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 44 23 . 4 2 4 . 2 2 4 . 2 D i s a g r e e 2 7 6 4 0 . 4 4 1 . 8 6 5 . 9 A g r e e 3 5 5 2 9 . 3 3 0 . 2 9 6 . 2 S t r o n g l y a g r e e 4 7 3 . 7 3 . 8 1 0 0 . 0 6 3 . 2 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 1 8 2 M i s s i n g c a s e s 6 SR - P o s t o b s e r v a t i o n p h a s e i s s t r e s s f u l V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 5 0 2 6 . 6 2 7 . 0 27 . 0 D i s a g r e e 2 87 4 6 . 3 4 7 . 0 74 . 1 A g r e e 3 4 6 2 4 . 5 2 4 . 9 9 8 . 9 S t r o n g l y a g r e e 4 2 1 . 1 1 . 1 1 0 0 . 0 3 1 . 6 M i s s i n g V a l i d c a s e s 1 8 5 T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 M i s s i n g c a s e s 3 Appendix G SR - P o s t o b s e r v a t i o n p h a s e i s c o m p l e x V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 32 17.0 17.3 17.3 D i s a g r e e 2 50 2G.G 27.0 44.3 A g r e e 3 90 47.9 48.6 93.0 S t r o n g l y a g r e e 4 13 6.9 7.0 100.0 3 1.6 M i s s i n g T o t a l 188 100.0 ioo. o V a l i d c a s e s 185 M i s s i n g c a s e s 3 SR - P o s t o b s e r v a t i o n p h a s e i s t i m e - c o n s u m i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 16 8.5 8.6 8.6 D i s a g r e e 2 34 18.1 18.3 26.9 A g r e e 3 98 52.1 52.7 79 . 6 S t r o n g l y a g r e e 4 38 20.2 20.4 100 . 0 2 1.1 M i s s i n g T o t a l 188 100.0 100.0 V a l i d c a s e s 186 M i s s i n g c a s e s 2 SR - P o s t o b s e r v a t i o n p h a s e r e q u i r e s m o r e t r a i n i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 42 22.3 23 .3 23 .3 D i s a g r e e 2 74 39 .4 41.1 64.4 A g r e e 3 55 29.3 30 . 6 95.0 S t r o n g l y a g r e e 4 9 4.8 5.0 100.0 8 4.3 M i s s i n g T o t a l 188 100.0 100.0 V a l i d c a s e s 180 M i s s i n g c a s e s 8 SR - F i n a l r e p o r t w r i t i n g p h a s e i s s t r e s s f u l V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 35 18.6 18 .9 18 .9 D i s a g r e e 2 74 39.4 40.0 58.9 A g r e e 3 61 32 .4 33.0 91.9 S t r o n g l y a g r e e 4 15 8.0 8.1 100.0 3 1.6 M i s s i n g T o t a l 188 100.0 100 .0 V a l i d c a s e s 185 M i s s i n g c a s e s 3 SR - F i n a l r e p o r t w r i t i n g p h a s e i s c o m p l e x V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 20 10.6 10.8 10.8 D i s a g r e e 2 25 13 .3 13.5 24.3 A g r e e 3 106 56.4 57.3 81.6 S t r o n g l y a g r e e 4 34 18.1 18 .4 100.0 3 1.6 M i s s i n g T o t a l 188 100.0 100.0 V a l i d c a s e s 185 M i s s i n g c a s e s 3 SR - F i n a l r e p o r t w r i t i n g p h a s e i s t i m e - c o n s u m i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 5 2.7 2.7 • 2.7 D i s a g r e e 2 6 3.2 3.2 . 5.9 A g r e e 3 83 44 .1 44.4 50.3 S t r o n g l y a g r e e 4 93 49 . 5 49 . 7 100 . 0 1 .5 M i s s i n g T o t a l 188 100 . 0 100.0 V a l i d c a s e s 187 M i s s i n g c a s e s 1 SR - F i n a l r e p o r t w r i t i n g p h a s e r e q u i r e s m o r e t r a i n i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 35 18.6 19.1 19.1 D i s a g r e e 2 80 42 .6 43.7 62.8 A g r e e 3 57 30.3 31.1 94.0 S t r o n g l y a g r e e 4 11 5 5.9 2.7 6.0 M i s s i n g 100.0 V a l i d c a s e s 183 T o t a l 188 100.0 100.0 M i s s i n g c a s e s 5 Appendix G 186 L T S R - P r e - e v a l u a t i o n c o n f e r e n c e i s s t r e s s f u l V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 1 1 5 . 9 1 2 . 8 12 . 8 D i s a g r e e 2 28 14 . 9 3 2 . 6 4 5 . 3 A g r e e 3 33 1 7 . 6 3 8 . 4 83 . 7 S t r o n g l y a g r e e 4 14 7 . 4 1 6 . 3 1 0 0 . 0 1 0 2 5 4 . 3 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 86 M i s s i n g c a s e s 1 0 2 L T S R - P r e - e v a l u a t i o n c o n f e r e n c e i s c o m p l e x V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 9 4 . 8 1 0 . 5 1 0 . 5 ' D i s a g r e e 2 2 0 10 . 6 23 . 3 33 . 7 A g r e e 3 33 1 7 . 6 3 8 . 4 7 2 . 1 S t r o n g l y a g r e e 4 24 12 . 8 2 7 . 9 1 0 0 . 0 1 0 2 5 4 . 3 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 86 M i s s i n g c a s e s 1 0 2 L T S R - P r e - e v a l u a t i o n c o n f e r e n c e i s t i m e - c o n s u m i n g v a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 7 3 . 7 8 . 1 8 . 1 D i s a g r e e 2 1 1 5 . 9 12 . 8 2 0 . 9 A g r e e 3 3 9 2 0 . 7 4 5 . 3 6 6 . 3 S t r o n g l y a g r e e 4 2 9 15 . 4 33 . 7 1 0 0 . 0 1 0 2 5 4 . 3 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 86 M i s s i n g c a s e s 1 0 2 L T S R - P r e - e v a l u a t i o n c o n f e r e n c e r e q u i r e s m o r e t r a i n i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 19 1 0 . 1 2 3 . 2 23 . 2 D i s a g r e e 2 34 1 8 . 1 4 1 . 5 6 4 . 6 A g r e e 3 18 9 . 6 22 . 0 8 6 . 6 S t r o n g l y a g r e e 4 11 5 . 9 13 . 4 1 0 0 . 0 1 0 6 56 . 4 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 82 M i s s i n g c a s e s 1 0 6 L T S R - C l a s s r o o m o b s e r v a t i o n i s s t r e s s f u l V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 8 4 . 3 9 . 3 9 . 3 D i s a g r e e 2 24 1 2 . 8 2 7 . 9 3 7 . 2 A g r e e 3 3 7 1 9 . 7 4 3 . 0 8 0 . 2 S t r o n g l y a g r e e 4 17 9 . 0 1 9 . 8 1 0 0 . 0 1 0 2 5 4 . 3 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 86 M i s s i n g c a s e s 1 0 2 L T S R - C l a s s r o o m o b s e r v a t i o n i s c o m p l e x V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e l 5 2 . 7 5 . 8 5 . 8 D i s a g r e e 2 1 5 8 . 0 1 7 . 4 23 . 3 A g r e e 3 42 2 2 . 3 4 8 . 8 7 2 . 1 S t r o n g l y a g r e e 4 24 1 2 . 8 2 7 . 9 1 0 0 . 0 1 0 2 5 4 . 3 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 86 M i s s i n g c a s e s 1 0 2 L T S R - C l a s s r o o m o b s e r v a t i o n i s t i m e - c o n s u m i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 3 1 . 6 3 . 5 3 . 5 D i s a g r e e 2 6 3 . 2 7 . 0 1 0 . 5 A g r e e 3 3 7 1 9 . 7 4 3 . 0 53 . 5 S t r o n g l y a g r e e 4 4 0 2 1 . 3 4 6 . 5 1 0 0 . 0 1 0 2 5 4 . 3 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 86 M i s s i n g c a s e s 1 0 2 Appendix G 187 L T S R - C l a s s r o o m o b s e r v a t i o n i s r e q u i r e s m o r e t r a i n i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t s t r o n g l y d i s a g r e e 1 18 9 . 6 22 . 0 22 . 0 D i s a g r e e 2 28 14 . 9 3 4 . 1 5 6 . 1 A g r e e 3 28 14 . 9 3 4 . 1 • 9 0 . 2 S t r o n g l y a g r e e 4 8 4 . 3 9 . 8 1 0 0 . 0 1 0 6 5 6 . 4 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 82 M i s s i n g c a s e s 1 0 6 L T S R - P o s t o b s e r v a t i o n p h a s e i s s t r e s s f u l V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 4 2 . 1 4 . 7 4 . 7 D i s a g r e e 2 4 2 . 1 4 . 7 9 . 3 A g r e e 3 34 1 8 . 1 3 9 . 5 4 8 . 8 S t r o n g l y a g r e e 4 . 44 23 . 4 5 1 . 2 1 0 0 . 0 1 0 2 5 4 . 3 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 86 M i s s i n g c a s e s 1 0 2 L T S R - P o s t o b s e r v a t i o n p h a s e i s c o m p l e x V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 3 1 . 6 3 . 5 3 . 5 D i s a g r e e 2 4 2 . 1 4 . 7 8 . 1 A g r e e 3 2 9 15 . 4 3 3 . 7 4 1 . 9 S t r o n g l y a g r e e 4 5 0 2 6 . 6 58 . 1 1 0 0 . 0 1 0 2 54 . 3 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 86 M i s s i n g c a s e s 1 0 2 L T S R - P o s t o b s e r v a t i o n p h a s e i s t i m e - c o n s u m i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 4 2 . 1 4 . 7 4 . 7 D i s a g r e e 2 7 3 . 7 8 . 1 1 2 . 8 A g r e e 3 2 9 15 . 4 3 3 . 7 4 6 . 5 S t r o n g l y a g r e e 4 4 6 2 4 . 5 53 . 5 1 0 0 . 0 1 0 2 5 4 . 3 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s -86 M i s s i n g c a s e s 1 0 2 L T S R - P o s t o b s e r v a t i o n p h a s e r e q u i r e s m o r e t r a i n i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 16 8 . 5 • 1 9 . 5 1 9 . 5 D i s a g r e e 2 19 10 . 1 2 3 . 2 4 2 . 7 A g r e e 3 32 1 7 . 0 3 9 . 0 8 1 . 7 S t r o n g l y a g r e e 4 15 8 . 0 1 8 . 3 1 0 0 . 0 1 0 6 56 . 4 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 • V a l i d c a s e s 82 M i s s i n g c a s e s 1 0 6 L T S R - F i n a l r e p o r t w r i t i n g p h a s e i s s t r e s s f u l v a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 1 . 5 1 . 2 1 . 2 D i s a g r e e 2 6 3 . 2 7 . 0 ' 8 . 1 A g r e e 3 3 6 1 9 . 1 4 1 . 9 5 0 . 0 S t r o n g l y a g r e e 4 43 2 2 . 9 5 0 . 0 1 0 0 . 0 1 0 2 5 4 . 3 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 86 M i s s i n g c a s e s 1 0 2 L T S R - F i n a l r e p o r t w r i t i n g p h a s e i s c o m p l e x V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 1 . 5 1 . 2 1 . 2 D i s a g r e e 2 4 2 . 1 4 . 7 5 . 8 A g r e e 3 2 6 1 3 . 8 3 0 . 2 36 . 0 S t r o n g l y a g r e e 4 5 5 2 9 . 3 64 . 0 1 0 0 . 0 1 0 2 5 4 . 3 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 86 M i s s i n g c a s e s 1 0 2 Appendix G 188 L T S R - F i n a l r e p o r t w r i t i n g p h a s e i s t i m e - c o n s u m i n g v a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 1 . 5 1 . 2 1 . 2 A g r e e 3 28 1 4 . 9 32 . 6 33 . 7 S t r o n g l y a g r e e 4 5 7 3 0 . 3 6 6 . 3 1 0 0 . 0 1 0 2 5 4 . 3 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 8 6 M i s s i n g c a s e s 1 0 2 L T S R - F i n a l r e p o r t w r i t i n g p h a s e r e q u i r e s m o r e t r a i n i n g V a l i d C u m V a l u e F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t S t r o n g l y d i s a g r e e 1 1 5 8 . 0 1 8 . 1 1 8 . 1 D i s a g r e e 2 1 9 1 0 . 1 2 2 . 9 4 1 . 0 A g r e e 3 23 1 2 . 2 2 7 . 7 6 8 . 7 S t r o n g l y a g r e e 4 2 6 1 3 . 8 3 1 . 3 1 0 0 . 0 1 0 5 5 5 . 9 M i s s i n g T o t a l 1 8 8 1 0 0 . 0 1 0 0 . 0 V a l i d c a s e s 83 M i s s i n g c a s e s 1 0 5

Cite

Citation Scheme:

    

Usage Statistics

Country Views Downloads
United States 6 0
France 4 0
China 3 28
Canada 2 0
City Views Downloads
Unknown 6 2
Beijing 3 0
Victoria 2 0
Sunnyvale 2 0
Redmond 1 0
Ashburn 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}

Share

Share to:

Comment

Related Items