Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Faculty members' professional growth in teaching through the summative peer review of teaching and other… Iqbal, Isabeau 2012

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2012_fall_iqbal_isabeau.pdf [ 1.51MB ]
Metadata
JSON: 24-1.0064685.json
JSON-LD: 24-1.0064685-ld.json
RDF/XML (Pretty): 24-1.0064685-rdf.xml
RDF/JSON: 24-1.0064685-rdf.json
Turtle: 24-1.0064685-turtle.txt
N-Triples: 24-1.0064685-rdf-ntriples.txt
Original Record: 24-1.0064685-source.json
Full Text
24-1.0064685-fulltext.txt
Citation
24-1.0064685.ris

Full Text

FACULTY MEMBERS’ PROFESSIONAL GROWTH IN TEACHING THROUGH THE SUMMATIVE PEER REVIEW OF TEACHING AND OTHER DEPARTMENTAL PRACTICES  by Isabeau Anisa Iqbal  M.A., The University of British Columbia, 2004  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE STUDIES (Educational Studies)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)  June 2012  © Isabeau Anisa Iqbal, 2012  ii Abstract This study investigated the ways that summative peer review of teaching contributes to tenure-track faculty members’ professional growth in teaching. It also explored other practices that support or hinder a departmental culture that values teaching.  Using the lens of academic culture, I drew on literature about the peer review of teaching, department culture, and professional growth in academic careers to inform this research. Thirty tenure-track faculty members from six departments and two faculties participated in semi-structured qualitative interviews. Participants were asked about their experiences of summative peer review, how they understood the relationship between peer review and their growth as instructors, and departmental practices that contribute to a culture that values teaching.  Participants had varied and inconsistent experiences of summative peer review of teaching. They reported multiple purposes (evaluative, formative, supplement to the student evaluations of teaching) that frequently conflicted. With few known guidelines that direct peer reviews and insufficient clarity as to their purpose, faculty members conducted summative reviews based on a personal sense of “what was best.” Given the demanding nature of academic careers and an institutional reward system that favours research over teaching, peer reviews were primarily limited to classroom observations and engaged few faculty members in dialogue. Such summative peer reviews appeared to make minimal contribution to professional growth in teaching.  iii The study did find numerous other departmental practices conducive to a culture that values teaching, e.g., informal collegial conversations about teaching and team teaching. Faculty members who partook in these grew as instructors. Results demonstrated that academic values and norms (i.e., collegiality and autonomy), disciplinary traditions pertaining to collaboration, and institutional rewards influenced how faculty members pursued professional growth as teachers.   iv Preface  The protocol for this research study was approved by the UBC Research Ethics Board (Behavioural Research Ethics Board; UBC BREB Number: H10-00389).     v Table of Contents  Abstract ..................................................................................................................................... ii	
   Preface ...................................................................................................................................... iv	
   Table of Contents ...................................................................................................................... v	
   List of Tables ......................................................................................................................... xiii	
   List of Figures ........................................................................................................................ xiv	
   Acknowledgements ................................................................................................................. xv	
   Chapter  1: Introduction ........................................................................................................ 1	
   1.1	
   Presentation of the Problem ..................................................................................... 2	
   1.2	
   Research Purpose ..................................................................................................... 4	
   1.3	
   Research Objectives ................................................................................................. 5	
   1.4	
   Conceptual Framework ............................................................................................ 7	
   1.5	
   Research Significance ............................................................................................ 11	
   1.6	
   Research Questions ................................................................................................ 12	
   1.7	
   Dissertation Structure ............................................................................................. 12	
   Chapter  2: Literature Review ............................................................................................. 13	
   2.1	
   The Peer Review of Teaching ................................................................................ 13	
   2.1.1	
   Peer Review of Teaching and Professional Growth ....................................... 14	
   2.1.1.1	
   Collective Dialogue about Teaching ........................................................ 16	
   2.1.1.2	
   Refining Characteristics of Good Teaching ............................................. 17	
   2.1.2	
   Resistance to Peer Review of Teaching .......................................................... 18	
   2.1.2.1	
   Time ......................................................................................................... 19	
    vi 2.1.2.2	
   Identifying a Suitable Peer ....................................................................... 19	
   2.1.2.3	
   Feedback Challenges ............................................................................... 20	
   2.1.2.4	
   Reliability and Validity Issues ................................................................. 23	
   2.1.2.5	
   Infringement on Autonomy ...................................................................... 26	
   2.2	
   Summary: Peer Review of Teaching Literature ..................................................... 27	
   2.3	
   Department Culture ................................................................................................ 28	
   2.3.1	
   Academic Culture, Values and Norms ............................................................ 29	
   2.3.2	
   The Academic Department and Culture(s) of Teaching ................................. 30	
   2.3.2.1	
   Learning Conversations and Professional Growth in Teaching .............. 32	
   2.3.2.2	
   Departmental Leadership in Teaching ..................................................... 37	
   2.3.3	
   Summary: Department Culture Literature ...................................................... 40	
   2.4	
   Summary ................................................................................................................ 40	
   Chapter  3: Methodology ...................................................................................................... 42	
   3.1	
   Introduction ............................................................................................................ 42	
   3.2	
   Positioning Myself ................................................................................................. 42	
   3.2.1	
   Intellectual Position ........................................................................................ 43	
   3.2.2	
   Professional/Personal Position ........................................................................ 45	
   3.3	
   Changes to the Original Research Questions ......................................................... 46	
   3.4	
   Selecting Faculties, Departments and Faculty Members ....................................... 48	
   3.4.1	
   Institution ........................................................................................................ 49	
   3.4.2	
   Faculties .......................................................................................................... 50	
   3.4.3	
   Departments .................................................................................................... 51	
   3.4.4	
   Defining the Professor .................................................................................... 52	
    vii 3.4.5	
   Recruitment: Beginnings ................................................................................ 54	
   3.4.5.1	
   Challenges Recruiting Assistant Professors and Women ........................ 55	
   3.5	
   Interviews ............................................................................................................... 58	
   3.5.1	
   Data Collection ............................................................................................... 58	
   3.5.2	
   Methodological Issues .................................................................................... 59	
   3.6	
   Data Analysis ......................................................................................................... 62	
   3.7	
   Rigour and Limitations .......................................................................................... 64	
   3.7.1	
   Validity ........................................................................................................... 64	
   3.7.2	
   Reliability ........................................................................................................ 66	
   3.7.3	
   Generalizability ............................................................................................... 67	
   3.8	
   Ethical Considerations ........................................................................................... 69	
   3.8.1	
   Respect ............................................................................................................ 69	
   3.8.2	
   Privacy and Confidentiality ............................................................................ 69	
   3.8.3	
   Informed Consent ............................................................................................ 70	
   3.8.4	
   Institutional Review Board Approval ............................................................. 70	
   Chapter  4: Summative Peer Review of Teaching: Current Purposes and Envisioned Ideals ...................................................................................................................................... 71	
   4.1	
   Chapter Structure ................................................................................................... 71	
   4.2	
   Institutional Culture: Endless Demands, Finite Time ............................................ 72	
   4.2.1	
   Teaching Counts (?) ........................................................................................ 74	
   4.3	
   Summative Peer Review of Teaching: A Sketch ................................................... 78	
   4.3.1	
   Changes Over Time ........................................................................................ 79	
   4.3.2	
   Timing and Frequency .................................................................................... 80	
    viii 4.3.3	
   Selection of Reviewers ................................................................................... 81	
   4.3.4	
   What is Being Reviewed? ............................................................................... 82	
   4.3.5	
   Communication Pre/Post Review ................................................................... 83	
   4.4	
   The Multiple Purposes of Summative Peer Review of Teaching .......................... 84	
   4.4.1	
   Formal Evaluation for Tenure and Promotion ................................................ 84	
   4.4.2	
   Professional Growth in Teaching .................................................................... 88	
   4.4.3	
   Supplement the Student Evaluations of Teaching .......................................... 90	
   4.5	
   No Teaching-Related Professional Growth ........................................................... 92	
   4.5.1	
   Feedback: Non-existent or Unspecific ............................................................ 92	
   4.5.1.1	
   Feedback from the Department Head ...................................................... 95	
   4.5.1.2	
   Why No Feedback? .................................................................................. 97	
   4.5.2	
   Student Evaluations of Teaching Set the Standards ..................................... 103	
   4.5.3	
   Artificial ........................................................................................................ 107	
   4.6	
   Ideal Process ........................................................................................................ 108	
   4.6.1	
   Flexibility/Structure ...................................................................................... 108	
   4.6.2	
   Ongoing, Informal Process ........................................................................... 110	
   4.6.3	
   Feedback Quality and Sharing ...................................................................... 112	
   4.6.4	
   Reviewer-Reviewed Connection .................................................................. 113	
   4.6.4.1	
   Disciplinary Connection ........................................................................ 114	
   4.6.4.2	
   Pedagogical Knowledge ......................................................................... 115	
   4.6.4.3	
   Philosophies of Teaching ....................................................................... 116	
   4.6.4.4	
   Gender .................................................................................................... 116	
   4.6.4.5	
   Rank ....................................................................................................... 118	
    ix 4.6.5	
   More, Sooner ................................................................................................. 118	
   4.7	
   Summary of Findings ........................................................................................... 120	
   4.8	
   Discussion: Summative Peer Review and Professional Growth in Teaching ...... 123	
   4.8.1	
   Unsettling/Honouring Collegiality ................................................................ 124	
   4.8.1.1	
   Feedback Unease ................................................................................... 124	
   4.8.1.2	
   The Primacy of Tenure .......................................................................... 125	
   4.8.1.3	
   Teaching as Private ................................................................................ 126	
   4.8.2	
   Universalism (or We Should Not be Evaluating from a Gut Feeling) .......... 127	
   4.8.3	
   Student Evaluation of Teaching Reigns ........................................................ 129	
   4.8.4	
   Slow Change in Academia ............................................................................ 131	
   Chapter  5: Departmental Practices that Promote or Hinder a Culture that Values Teaching ............................................................................................................................... 135	
   5.1	
   Chapter Structure ................................................................................................. 135	
   5.2	
   Informal Departmental Practices ......................................................................... 137	
   5.2.1	
   Informal Conversations with Colleagues ...................................................... 138	
   5.2.2	
   Collaborative/Individual Teaching Approaches ........................................... 139	
   5.2.3	
   Departmental Composition ........................................................................... 141	
   5.2.4	
   Good Teaching: No Consensus ..................................................................... 145	
   5.2.5	
   Risk-taking and Trust .................................................................................... 146	
   5.2.6	
   Attention to Fairness ..................................................................................... 149	
   5.3	
   Organized Initiatives, Policies and Structures ..................................................... 151	
   5.3.1	
   Recognized Policies and Other Documents .................................................. 151	
   5.3.2	
   Official Go-to People .................................................................................... 154	
    x 5.3.3	
   Effective Science Education Project (ESEd) ................................................ 156	
   5.3.4	
   Teaching Seminars and Retreats ................................................................... 157	
   5.3.5	
   Team Teaching Across Disciplines .............................................................. 159	
   5.3.6	
   Formative Peer Review ................................................................................. 160	
   5.3.7	
   Teaching and the Hiring Process .................................................................. 161	
   5.4	
   The Department Head’s Role in Promoting a Culture that Values Teaching ...... 164	
   5.4.1	
   Teaching and Tenure: Support from the Department Head .......................... 165	
   5.5	
   Summary of Findings ........................................................................................... 170	
   5.6	
   Discussion: Departmental Practices that Support or Hinder a Culture that Values Teaching ........................................................................................................................ 171	
   5.6.1	
   Institutional Culture ...................................................................................... 172	
   5.6.2	
   Departmental Culture .................................................................................... 175	
   5.6.2.1	
   Learning Conversations ......................................................................... 177	
   5.6.3	
   Disciplinary Culture and Conversations ....................................................... 179	
   Chapter  6: Summary, Conclusions, and Recommendations ......................................... 181	
   6.1	
   Summary of the Study ......................................................................................... 181	
   6.1.1	
   Statement of the Problem .............................................................................. 181	
   6.1.2	
   Research Questions ....................................................................................... 183	
   6.1.3	
   Conceptual Framework ................................................................................. 184	
   6.1.4	
   Study Participants and Data Collection ........................................................ 184	
   6.1.5	
   Data Analysis ................................................................................................ 185	
   6.1.6	
   Empirical Summary ...................................................................................... 185	
   6.1.7	
   Limitations .................................................................................................... 189	
    xi 6.2	
   Contributions to Theory ....................................................................................... 190	
   6.2.1	
   Description of the Two Conceptual Models ................................................. 192	
   6.2.1.1	
   Model for Faculty Member Engagement in Summative Peer Review of Teaching and Professional Growth in Teaching (Model A) ................................. 193	
   6.2.1.2	
   Model for Faculty Member Engagement in Departmental Practices that Foster a Culture that Values Teaching and Professional Growth in Teaching (Model B) .............................................................................................................. 197	
   6.3	
   Recommendations for Practice ............................................................................ 201	
   6.3.1	
   Summative Peer Review of Teaching ........................................................... 201	
   6.3.1.1	
   Involve the Department Head ................................................................ 201	
   6.3.1.2	
   Improve the Feedback Process ............................................................... 205	
   6.3.2	
   Fostering a Departmental Culture that Values Teaching .............................. 207	
   6.3.2.1	
   Increase Adoption and Full Integration of Improved Teaching Practices…….. ...................................................................................................... 208	
   6.4	
   Recommendations for Future Research ............................................................... 211	
   6.4.1	
   Research on the Summative Peer Review of Teaching ................................ 211	
   6.4.2	
   Future Research on Departmental Culture that Captures Greater Faculty Member Diversity ..................................................................................................... 213	
   6.4.3	
   Research on the Role of the Department Head in Promoting Professional Growth in Teaching .................................................................................................. 215	
   6.4.4	
   Research on the Role of Pre-tenured Faculty Members’ in Shaping a Culture of Teaching ............................................................................................................... 215	
   6.5	
   Closing Comments ............................................................................................... 217	
    xii References ............................................................................................................................. 219	
   Appendices ............................................................................................................................ 242	
     xiii  List of Tables Table 1: Faculty of Arts: Representation by Rank and Gender in A1, A2, and A3 ............... 52	
   Table 2: Faculty of Science: Representation by Rank and Gender in S1, S2, and S3 ............ 52	
   Table 3: Participant Eligibility Criteria ................................................................................... 55	
   Table 4: Responses to Follow up Email: Reasons for Not Participating ................................ 56	
   Table 5: Final Recruitment from Faculty of Arts (n=15) ....................................................... 57	
   Table 6: Final Recruitment from Faculty of Science (n=15) .................................................. 57	
   Table 7: Categories and Codes (Select Examples) ................................................................. 63	
   Table 8: Informal and Formal Initiatives and Practices that Influence the Culture of Teaching within a Department .............................................................................................................. 163	
   Table 9: Dominant Element of Culture at Play in Models of Professional Growth in Teaching ............................................................................................................................................... 200	
   Table 10: Summary of Recommendations for Practice ........................................................ 210	
   Table 11: Summary of Recommendations for Future Research ........................................... 216	
      xiv List of Figures Figure 1: Conceptual Model for Departmental Practices and Professional Growth in Teaching ................................................................................................................................................... 8	
   Figure 2: Model for Faculty Member Engagement in Summary Peer Review of Teaching and Professional Growth in Teaching (Model A) ........................................................................ 191	
   Figure 3: Model for Faculty Member Engagement in Non-Evaluative Departmental Practices that Foster a Culture that Values Teaching and Professional Growth in Teaching (Model B) ............................................................................................................................................... 192	
      xv Acknowledgements My family, though small, has given me enormous support. Foremost, I thank you, Raúl, for believing in me siempre. I look forward to more silliness, dancing, and adventures. I thank my father for his steady help, my mother for teaching me to be curious and to enjoy my own company, and Diane for her encouragement. I am profoundly grateful to my children who, in many ways, were oblivious to my studies and invited me to play and be creative (when I was inclined to work and worry). Being a mother helped me be a happier, healthier student.  I am appreciative of the guidance provided by my supervisory committee, three individuals I deeply respect. I thank Dr. Gary Poole for his enthusiastic and wonderful mentorship, Dr. Tom Sork for his remarkable ability at (always) finding practical, helpful solutions, and Dr. Amy Scott Metcalfe for introducing me to new ideas and bodies of literature. You were reliable advisors who gave me the right combination of independence and direction.  I give heartfelt thanks to the 30 faculty members who participated in this study and taught me so much. This research was also made possible by the Joseph-Armand Bombardier Doctoral Scholarship from the Social Sciences and Humanities Research Council.  I have benefited from the help of countless other people: my dissertation sisters—Nora, Gen, Tara, Erin, Maryam, Hanae—the bright and funny women I met with weekly to discuss work and other aspects of the PhD journey; the mighty VR warriors and friends on Phinished; Dr. Allison Tom, with whom I shared delightful and fascinating conversations about methodology; the women in my EDST writing group; and my TAG and CTLT colleagues.  xvi  A ton of love goes to my closest friends who have cheered me on: Alejandra, Allison, Barbara, Claire, Erin, Jessie, Julia, Madeline, Michelle, Susan, and Susannah.   1  Chapter  1: Introduction This study examines the ways that summative peer review of teaching and other departmental teaching-related practices contribute to tenure-track professors’ professional growth in teaching. Faculty members’ professional growth in teaching is vital to their work (Clegg, 2003; Gappa, Austin, & Trice, 2007; O’Meara, Terosky & Neumann, 2008) because teaching is a chief responsibility in academic careers (Schuster & Finkelstein, 2006). Continuous learning, which has been described as the “job of a good scholar and teacher” (Schuster & Finkelstein, 2006, p. 78), is considered necessary for faculty members who wish to work productively and creatively in their teaching (Gappa et al., 2007). In addition, the rapidly changing educational context demands ongoing growth. Pressure from the public to improve the learning experience of students, political priorities concerned with augmenting the accountability of higher education institutions, the expectations of an increasingly diverse student population, and advances in technology all require that faculty members regularly engage in ongoing learning (Sorcinelli, Austin, Eddy, & Beach, 2006). When academics grow and learn, they acquire new perspectives, values, skills, and knowledge that they integrate into their practice (Dall’Alba & Sandberg, 2006). Ongoing professional growth allows faculty members to better fulfill their professional teaching roles and responsibilities.  Professional growth, however, is not required solely to fulfill professional obligations. Faculty members care about their learning, and the constructive growth generated through learning matters a great deal to academics; thus, learning contributes strongly to their career satisfaction (Neumann, 2009). Since career satisfaction among academics, which has  2 declined significantly over the past decade (Schuster & Finklestein, 2006), has an impact on career success (Cariaga-Lo, Worthy Dawkins, Enger, Schotter, & Spence, 2010), addressing faculty growth is fundamental to the academic endeavour.  1.1 Presentation of the Problem Professional growth in teaching allows faculty members to better realize their professional teaching roles and responsibilities (Neumann, 2009; O’Meara et al., 2008; Sorcinelli et al., 2006); however, within research-intensive universities, there are many constraints that limit faculty members’ engagement in their own professional growth for teaching (Caffarella & Zinn, 1999; Clegg, 2003; Gappa et al., 2007; Kilgore & Cook, 2007; O’Meara et al., 2008).  One of the main constraints academics face is the priority given to research over teaching. That is, compared to teaching, faculty members receive more positive reinforcement and are more often compensated for their research, via merit pay, promotion, awards, and travel money  (Chalmers, 2010; Hardy & Smith, 2006; Kilgore & Cook; 2007; Rhode, 2006; Serow, 2000). The preference given to research matters, because institutional reward systems are the primary way by which faculty members’ professional work is valued. Consequently, even if much faculty motivation is intrinsic, reward systems have an effect on faculty members’ behaviour (Braxton, 2008; Link, Swan, & Bozeman, 2008; O’Meara, 2011; Schuster & Finkelstein, 2006). Since rewarded activities are deemed more desirable and legitimate, it is unsurprising that academics at research-intensive universities prioritize research over teaching (Link et al., 2008; Milem, Berger, & Dey, 2000), including professional growth activities related to teaching.  3  Faculty members’ engagement in professional growth related to their teaching is further constrained by the escalating instructional and research demands imposed on them (Hardy & Smith, 2006; Schuster & Finklestein, 2006). Over the past decade, the institutional pressures to generate revenue have amplified, as have external calls for increased accountability and proven outcomes. Faculty members have also had to keep up with—and adapt to—new technologies (Summerlee & Christensen Hughes, 2010). The trend of increased work effort, which some suggest is most pronounced at research universities (Schuster & Finklestein, 2006) has resulted in, among other things, faculty members feeling the need to “do more” (Gappa et al., 2007). Academics are putting more time into their profession, and stress is a significant health issue among those in academic careers (Jacobs & Winslow, 2004; Robinson, 2006). Additionally, many faculty members, especially female academics, have disclosed that balancing work pressures and family responsibilities is difficult (Acker & Armenti, 2004; Allan, 2011). The increased workload, which has led to a decline in satisfaction among faculty members, has a bearing on how faculty members allocate their time to fulfill their professional obligations.  If one accepts that professional growth in teaching has important implications for the goals and effectiveness of higher education (Cariago-Lo et al., 2010; Neumann, 2009; Sorcinelli et al., 2006), and that numerous factors within the research-intensive university can support or constrain that growth, then the need for a study that examines the relationship between professional growth and these factors (which comprise the “academic culture,” a term I define in Section 1.4) becomes evident.  4  This dissertation aims to increase our understanding of ways to support faculty members’ engagement in their professional growth as teachers. It does so by examining summative peer review and other departmental practices that contribute to a culture that values teaching. The peer review of teaching has received considerable attention as a mechanism to foster professional growth in teaching, and extensive research has been conducted on the benefits of formative reviews. However, to date there has been little empirical research that examines the relationship between summative peer review and faculty members’ professional growth, in spite of the fact that peer reviews are commonly used to evaluate teaching for the purposes of making tenure and promotion decisions in North American universities (Gravestock & Greenleaf, 2008). Furthermore, since summative peer reviews occur only occasionally, there is a need to examine other departmental practices that may foster growth in teaching. The department is an appropriate site for research into meaningful and far-reaching initiatives because the majority of teaching developments occur as a result of departmental, rather than institution-wide, initiatives (Gibbs, Knapper, & Piccinin, 2008; Quinlan & Åkerlind, 2000; Trowler & Knight, 2000).  1.2 Research Purpose The purpose of this research is to explore practices that encourage professional growth in teaching among tenure-track faculty members working in a research-intensive university. Through the use of qualitative interviews, this study aims to (1) investigate faculty members’ experiences of summative peer review of teaching and to examine how they understand the relationship between summative peer review and professional growth in teaching; and, (2)  5 explore departmental practices that support or hinder a culture that values teaching and learning, including professional growth in teaching.  1.3 Research Objectives The principal objective of this study is to investigate the issue of professional growth in teaching through the lens of academic culture. This work will contribute to a body of scholarly research on faculty careers, peer review of teaching, and departmental cultures of teaching and learning. This study draws on the literatures of departmental culture and the summative and formative peer review of teaching to explore the relationship between certain departmental practices, including summative peer review, and faculty members’ professional growth as teachers. It also attends to the role of collegial interactions in establishing and maintaining departmental practices that promote strong teaching. The empirical aim is to gather the accounts of faculty members about their experiences with summative peer review of teaching and other departmental practices that promote or hinder a culture that values teaching.  The first practical objective of this study is to provide academics and the professionals who work with them—department heads, senior administrators, and educational developers—an expanded perspective on the relationship between departmental practices like the summative peer review of teaching and professional growth in teaching. Although this study cannot reveal the full extent of ways that faculty members grow and learn as teachers within the context of a department, it underscores the need to examine current structures and ways of being that contribute to such learning. I have chosen to examine existing teaching-related  6 practices because I maintain that, given intensifying academic careers, it makes good sense to see how to make use of, as well as enhance and promote, such practices.  The second practical objective is to provide insight into the extent to which a university-level formal report on the summative peer review of teaching influences the practice of peer review. Within higher education, committees work to develop and implement policies and other formal guidelines pertinent to teaching. Typically faculty members do not respond well to such teaching initiatives (Harvey & Kamvounias, 2008; Kezar, 2011) and, when it comes to faculty evaluation and professional enrichment programs, faculty resistance and administrator apathy are two major obstacles to change (Arreola, 2007). Strategies may not be enthusiastically received and adopted because faculty members perceive that they infringe upon their autonomy. Academics may also be overwhelmed by policy overload or resent any bureaucracy that is imposed on teaching (Newton, 2003; Smith, 2011). In addition, the policy-to-practice gap is aggravated by the fact that there are usually scant mechanisms available to implement policies, to monitor their implementation, or to evaluate their impact (Newton, 2003). At the University of Western Canada (UWC) [pseudonym], where this study took place, a working group on the summative peer review of teaching created a document, at the request of the office of the Academic Provost and Vice President, that contained recommendations to guide the development of department-specific procedures and practices. The document was distributed in November 2009, approximately six months prior to the start of my first interview. Thus, the timing of this study allowed me to note whether faculty members made reference to this document or its contents and, if so, how they  7 perceived its relevance. This aspect of my work will be useful for university administrators and others who are involved with creating and implementing policies within their institution.  1.4 Conceptual Framework The conceptual framework for this study (Figure 1) was developed through the contributions of numerous higher education scholars, especially those who have studied academic culture(s) as these pertain to teaching and faculty careers. An academic culture perspective considers the role of values, beliefs, and assumptions in higher education; it examines how these are displayed in attitudes, behavioural norms, rituals, and other symbolic activities (Alvesson & Sveningsson, 2008; Bunch, 2007; Quinlan & Åkerlind, 2000; Trowler, 2008) and how they are maintained and passed along (O’Meara, 2011).  Higher education is frequently described as consisting of various cultures (or subcultures) that include institutional, disciplinary, and departmental cultures. These interconnecting cultures influence numerous academic practices (Clark, 1980, 1984, 1987; Trowler, 2008; Umbach, 2007); although their relative effect on a given practice (e.g., teaching, research, service or other practices) may vary, there is overall consensus that the higher education system is value laden (Clark, 1980, 1987; Gappa et al., 2007; Kezar, 2001). Those values most cherished by faculty members are academic freedom, autonomy, and collegiality (Kezar, 2001; Schuster & Finkelstein, 2006; Walvoord et al., 2000). Attention to values in a study about professional growth and teaching is relevant as these guide academics who, as Schuster and Finklestein (2006) explain, “are among the most likely—given the relative  8 autonomy they enjoy—to ‘act out’ their values in fulfilling their professional responsibilities” (p. 87). Figure 1: Conceptual Model for Departmental Practices and Professional Growth in Teaching  The conceptual framework was further inspired by the work of O’Meara, Terosky, and Neumann (2008) who introduce a framework for faculty professional growth that positions growth as a professional and personal need, in every academic’s life, regardless of his or her career stage. Growth, they maintain, is ongoing, occurs in specific sociocultural and personal contexts, and is influenced by external environments as well as the unique needs of the individual academic. The authors frame their examination of professional growth through four interconnected lenses: learning, agency, professional relationships, and commitments. They make the following main points about each:  9 • Learning involves changes in cognition and is central to faculty members’ work. It happens in different work roles and may occur in interaction with people or be highly personal. Learning is influenced by the changing nature of work and by individuals’ past experiences and identities. • Agency originates from within the individual and refers to a faculty member’s capacity to “construct the contexts of her or his own learning and development in professional and intellectual ways” (O’Meara et al., 2008, p. 28). Agency involves acting with intention and being reflexive about one’s experience. It can be nurtured by the professional community within which a faculty member works, and may be activated by available resources. It is also influenced by broader social structures. • Professional relationships are interactions that provide personal and/or professional support. They become sites of intellectual growth when they prompt, encourage, and affect learning. Professional relationships strengthen a faculty member’s ability to “bring the best of their talents to their work roles” (O’Meara et al., 2008, p. 29) and they enhance faculty motivation, satisfaction, and self-esteem. • Commitments are conscious acts that affirm personal values. They require time, energy, and concrete action. Of these four elements, I was most attuned to professional relationships, since my review of the literature on peer review of teaching had sensitized me to the significant ways in which the candidate-reviewer relationship affects the peer review process. Prior to starting the study, I had learned that the levels of trust and the power differentials between the various individuals involved, and their intentions for peer review, shape participating faculty members’ experiences and resulting outcomes (Blackmore, 2005; Donnelly, 2007). Thus, I  10 surmised that the potential for summative peer review to contribute (or not) to faculty members’ growth in teaching would be influenced by the quality of the professional relationships among the departmental peers who were conducting, evaluating, and communicating about the reviews. Other literature about department cultures and teaching also points to the importance of collegial interactions for faculty members’ learning in their instructor roles.  Figure 1 provides a visual illustration of the conceptual framework for this study. The framework suggests that institutional, disciplinary, and departmental cultures influence the formal and informal practices that can foster a departmental culture which values teaching. It also illustrates that values and norms are constantly operating and influencing faculty member choices with respect to teaching.  The model proposes that institutional, department, and disciplinary cultures and the academic values of collegiality and autonomy will have an impact on faculty members’ participation in professional growth activities related to teaching, including which practices they choose to engage in, how they participate, with whom, and why. For instance, while the existence of a lunchtime seminar on teaching may, in theory, contribute to professional growth, it is possible that a faculty member would not attend because s/he works in a department where peers do not consider time spent on improving teaching as productive, and would thus disapprove of that colleague’s choice of activities. Consequently, the norm in that department is not to participate in lunchtime seminars about teaching.   11 This conceptual model also includes the four aspects of professional growth outlined in O’Meara and colleagues’ framework and, in doing so, assumes that learning, commitment, agency, and professional relationships interconnect to shape how a faculty member pursues his/her professional growth in teaching within a departmental context.  1.5 Research Significance Efforts to enrich and support faculty members’ work, including their growth and learning as teachers, are important to institutional leaders and higher education, as well as to individual faculty members (Neumann, 2009; O’Meara et al., 2008; Sorcinelli et al., 2006). Despite the fact that summative peer review of teaching is widely used in North American universities, there is a paucity of literature on how faculty members experience this process when it is used to inform high-stakes tenure and promotion decisions. As compared to the voluminous literature on formative peer review of teaching, empirical and in-depth information about summative peer review in North America remains limited. This study takes a unique approach to investigating the summative peer review of teaching, as it probes how academics connect the practice of peer review to professional growth in teaching. The research extends its examination of professional growth in academic careers by also considering departmental teaching-related practices.  This research contributes to the scholarship on faculty growth in teaching by providing an account of the ways in which tenure track faculty members currently construct, and seek to develop, their roles as teachers in an environment that favours research. Building on previous  12 work, this research extends an inquiry into the complex relationships among departmental cultures, summative peer review, and professional growth in teaching.  1.6 Research Questions This study addresses the following questions: 1. What are faculty members’ experiences of summative peer review of teaching and how do they understand the relationship between summative peer review and professional growth in teaching? 2. What existing departmental practices support or hinder a culture that values teaching?  1.7 Dissertation Structure There are six chapters in this dissertation. Chapter 1 includes a presentation of the research problem and outlines the study’s purpose, objectives, and significance. It also features the conceptual framework and the research questions. Chapter 2 presents a review of the literature on peer review of teaching and departmental culture. Chapter 3 outlines the design of the study, including context, methods of data collection and analysis, and limitations of the design. Chapter 4 responds to the research question “How do tenure-track faculty members in a research-intensive university perceive the relationship between summative peer review of teaching and professional growth in teaching?” Chapter 5 examines existing departmental practices that support or hinder the culture of teaching. Chapters 4 and 5 each include a discussion of the findings. In Chapter 6, I summarize the study and then discuss implications for theory, practice, and future research.  13 Chapter  2: Literature Review This study draws from and integrates several distinct bodies of research, including the literature on peer review of teaching, departmental culture, and academic culture. In the first section, peer review of teaching, I examine the contribution these reviews make to faculty members’ professional growth in teaching and then describe the reasons some academics resist the process. The second section, literature on department culture, begins with a brief introduction to the concept of academic culture before moving on to the topic of teaching and learning cultures. Throughout the chapter I seek to underscore the relevance of academic values and norms in fostering a peer review practice that contributes to professional growth in teaching and determining a departmental culture that values teaching.  2.1 The Peer Review of Teaching Although some scholars believe peer review of teaching purposes and processes exist along a continuum between formative and summative (Chism, 2007a; Gosling, 2002; Lomas & Nicholls, 2005), the majority draw a sharp distinction, portraying the process as belonging to only one category (Hammersley-Fletcher & Orsmond, 2004; Peel, 2005). In the literature, formative peer review is seen as beneficial for, and supportive of, professional development (Hammersley-Fletcher & Orsmond, 2004). It is hailed as a process that contributes to the enhancement of teaching because it fosters self-reflection, discussion of teaching, and dissemination of best practices (Byrne, Brown, & Challen, 2010; Kell & Annetts, 2009; Shortland, 2010). In contrast, summative peer review of teaching, which is used to aid in making such personnel decisions as promotion, reappointment, tenure, or merit pay (Cavanaugh, 1996; Chism, 2007a), has not been linked to the improvement of teaching  14 (Byrne et al., 2010; Peel, 2005). It has instead been linked to high-stakes evaluations as well as the accountability movement in higher education whereby faculty are increasingly required to measure and quantify their activities in the guise of improving quality and efficiency (Shanahan, 2009). Summative peer review is said to offer few benefits in terms of academic professional growth given that meaningful learning and reflective practice occur most often when academics engage in pedagogical practice for its own sake, not based on an external demand (Byrne et al., 2010; Peel, 2005). In a standard summative process, departmental colleagues assess a faculty member in order to rank or compare that individual within the department. Unlike a formative peer review, where the information is intended for the instructor’s personal/private use, information from summative reviews is open to public inspection (Chism, 2007a).  I turn now to the connection between the practice of peer review and professional growth in teaching, as reported in the literature, and elaborate on some reasons why faculty members resist the practice of peer review of teaching.  2.1.1 Peer Review of Teaching and Professional Growth There are many compelling reasons to support peer review in higher education as a means of promoting professional growth in teaching. As mentioned, the developmental potential of formative peer review has been widely acknowledged in the literature. Reflection on teaching which occurs as reviewers and those reviewed dialogue about teaching, as well as individually, is a central component of the formative model. The act of reflecting on one’s teaching allows individuals to gain insight into the beliefs and intentions that drive, and give  15 justification to, their teaching actions (Pratt, 1997). Reflection helps faculty members uncover their conceptions of teaching; as a result, they may develop a more expansive understanding of their practice. When a colleague confirms these insights, it can aid legitimate new understandings (Peel, 2005). Even when peers are in disagreement over ideas and practices, if the criticism is relevant and instructive, peer review can present an opportunity to challenge assumptions about teaching and question existing actions (Handal, 1999). Thus, as Peel (2005) concludes, reflection when supported by a peer is significant for “transforming understanding and enhancing self-awareness” (p. 498).  Research into the peer review of teaching indicates that the process of critically reflecting upon and discussing teaching can benefit not only the person observed but the observer as well (Bovill, 2008; Donnelly, 2007; Gosling & Ritchie, 2003; Kemp & Gosling, 2000; Lomas & Nicholls, 2005). Gosling and Ritchie (2003), in a follow-up study of regional peer review workshops in the United Kingdom, found that some faculty perceived the opportunity to observe other teachers as equally important to receiving feedback, if not more so. Consequently, Gosling and Ritchie (2003) have recommended replacing the dominant notion of “giver” and “receiver” with a dialogue model in which both the instructor and observer gain mutual benefit from the process. They suggest that guidelines that support the peer review of teaching should explicitly state this benefit. Cosh (1998), on the other hand, found that reflection is more powerfully facilitated when the emphasis is on the observer rather than the observed. She promotes a model of peer review in which the reviewers direct their attention to reflecting on their own teaching so as to promote active self-development.   16  Regardless of whether the emphasis is placed on the reviewer or the person reviewed, peer review of teaching can help academics learn about each other’s teaching practices as well as their own. As faculty members build their understanding of teaching, they can better identify how they wish to develop their practice. Whether they pursue their learning through formal programs, such as the ones offered in university teaching and learning centres, or through self-study and opportunities in their departments, the assumption is that connecting the peer review process to continuous professional learning will contribute to the quality of teaching.  2.1.1.1 Collective Dialogue about Teaching Peer reviews help debunk the notion of teaching as a private activity, one shared with students but not colleagues (Hutchings, 1996). As peers connect with one another and others in their scholarly community, engaging in discussions about teaching will help shift the status of teaching from “private to community property” (Shulman, 1993, p. 6). Shulman (1993), Chism (2007a), and others (i.e., Taylor Huber & Hutchings, 2005) have argued that teaching in higher education will be valued more when it is considered community property and not a solitary and isolated action.  Peer reviews of teaching can create opportunities for dialogue and debate about teaching that extend beyond those that occur between reviewer and person being reviewed (Gosling & Ritchie, 2003; Hammersley-Fletcher & Orsmond, 2004). As faculty members engage in collective discussions about teaching evaluations, methods of teaching, and the challenges of practice, they share ideas and good practice and can report how they have tackled specific  17 teaching challenges (Chism, 2007a; Hutchings, 1996). The process can prompt departmental colleagues to address a wide range of issues pertinent to teaching, at the level of the individual, department, and institution. These “learning conversations” (Byrne et al., 2010, p. 216) may lead to not only professional growth but also an increased sense of collegiality (Byrne et al., 2010; Donnelly, 2007; Lomas & Nicholls, 2005; Peel, 2005).  2.1.1.2 Refining Characteristics of Good Teaching Peer reviews have the potential to help faculty members refine their understandings about the characteristics of good teaching (Chism, 2007a). When colleagues are faced with evaluating peers, they must consider what effective teaching looks like within the context of their department, institution, and discipline. In summative peer review, evaluators must rate and/or comment on a colleague’s teaching. The evaluator, whose report is read by members of the tenure and promotion committee, will presumably be asked to reason his or her rankings and comments. Since characteristics of good teaching are not universally agreed upon (Chism, 2007a; Murphy, MacLaren, & Flynn, 2009; Toth & McKey, 2010b), discussions prompted by peer review may assist faculty members to locally develop those characteristics.  Peer reviews can also contribute to professional growth in teaching, because they enable academics to build on their strengths as evaluators of teaching; existing research has found that peers, as compared to students, are more qualified to assess some characteristics of teaching over others (Chism, 2007a; Fink, 2008). Unlike students, faculty members are qualified to evaluate the following: course goals, content, and organization; instructional materials and methods; appropriateness of assessment and grading practices; commitment to  18 teaching and to student achievement; and professional and ethical behaviour (Chism, 2007a; Courneya, Pratt, & Collins, 2008). On the other hand, students are best at evaluating the quality of student-teacher interactions (Fink, 2008). Thus, information from peer reviews of teaching can add to student evaluations and provide faculty members with broader insights into their own teaching practice.  2.1.2 Resistance to Peer Review of Teaching One of the main barriers to effective peer review has been academics’ unwillingness to engage with the process (Carter, 2008; Lomas & Nicholls, 2005). When the peer review of teaching is seen as a tool for quality assurance and tied to personnel decisions, faculty members may become suspicious of the process (Peel, 2005; Swinglehurt, Russell, & Greenhalgh, 2008). Their resistance may be amplified when the objectives of peer reviews are unclear and/or when academics are unsure what will happen with the information gathered (Hammersley-Fletcher & Ormond, 2004). Faculty members may thus be more receptive to a model that creates dialogue about teaching and learning, and encourages constructive critical feedback and reflection (Swinglehurt et al., 2008). Still, some studies have shown that academics will not engage with the process even in formative peer reviews (Gosling & Ritchie, 2003; Hammersley-Fletcher & Orsmond, 2004). Therefore, it would appear that, irrespective of the model in place, faculty members may be unenthusiastic about participating in peer review for numerous, complex reasons, which will be elaborated in the next section.  19  2.1.2.1 Time One of the principal concerns faculty members have about peer review stems from their perception that it is a time consuming process (Chism, 2007a). The issue of time, however, must be placed in the context within which it operates: an environment where research productivity is rewarded above teaching and academics face escalating career demands, (particularly in the research-intensive university) (Chalmers, 2011; Shuster & Finklestein, 2006). In such an environment, few are eager to immerse themselves in teaching activities that use up more time. Especially when peer review is seen as merely something to “tick-off” their to-do list or, even worse, a “time sink” (Chism, 2007b) academics may approach the process with a sense of complacency (Gosling, 2002). In these cases, they may comply but not fully engage (Hammersley-Fletcher & Ormond, 2004; Shortland, 2004), which diminishes the rigour of the process.  2.1.2.2 Identifying a Suitable Peer Faculty reluctance to the peer review of teaching may also involve questions about observer suitability (Kell & Annetts, 2009). The term “peer” can refer to various relationships within a department (Gosling, 2002). Peers may be defined on the basis of their disciplinary specialization, rank, pedagogical preferences, gender, or race, to name a few. For instance, some faculty members perceive that peer review is more meaningful when conducted by an experienced faculty member who can draw on his/her pedagogical knowledge to contribute to the reflective process (Hammersley-Fletcher & Ormond, 2004). Peel (2005), however, cautions that an experienced faculty member may bring “a variety of experience, baggage,  20 and competence into the classroom” (p. 499). Experience, she warns, does not guarantee a high-quality review.  Some faculty members may prefer a reviewer with a similar disciplinary specialization. When observers understand the subject matter and its delivery, they tend to concentrate on these aspects of teaching. On the other hand, when they are not familiar with course content, they may direct more attention to the student learning experience (Hammersley-Fletcher & Ormond, 2004). Depending on what candidates value in a review, they could prefer someone from the same disciplinary background (or not). A candidate may also be concerned about how a reviewer’s approach to teaching will affect his or her perceptions and appraisals. Indeed, in a study conducted by Courneya, Pratt, and Collins (2008), the authors concluded that reviewers who had an approach to teaching similar to the person being reviewed tended to rate the latter more highly than when their approaches differed. For instance, an evaluator who favoured participatory learning might assign a higher effectiveness rating to an instructor who practices this approach than to one whose dominant approach was lecturing.  2.1.2.3 Feedback Challenges Studies have found that some faculty members resist peer review because they feel ill at ease giving feedback to peers, especially of a critical nature (Braskamp, 2000; Cosh, 1998; Hammersley-Fletcher & Orsmond, 2004; Kemp & Gosling, 2000). Thus, even though collegiality maintains a forceful symbolic function, in practice it may not be readily visible (Lucas, 2006).   21 Massy, Wilger, and Colbeck (1994) are critical of “hollow collegiality” which they describe as a “veneer of civility that pervades faculty interactions” (p. 12). Notwithstanding the fact that collegiality may refer to a democratic network of equal peers (Berquist & Pawlak, 2008), some have drawn attention to the stratified nature of academia (Massy et al., 1994; Roxå, Mårtensson, & Alveteg, 2011) and the abuse of “collegiality codes” by bullies who use them “to threaten and eliminate people they don’t like” (O’Hara, 2009). Massy et al. (1994) along with Becher and Trowler (2001) and Roxå & Mårtensson (2009) have noted that academics are often unwilling to pursue issues that may be divisive or spark debate. Instead, they are “inclined to play safe—to minimize the risk of making professional enemies by not opposing or being critical of colleagues’ views” (Becher & Trowler, 2001, p. 127). The avoidance of such discussions maintains a surface calm but results in a lack of discussion about issues of curricular structure, pedagogical alternatives, and student assessment (Massy et al., 1994). Even so, ambiguities and contradictions can provide meaningful openings for individual and collective development (Trowler & Knight, 2000).  Like the people who give feedback, academics who receive it may not welcome the peer review process because it can include judgments that feel threatening (Blackmore, 2005; Carter, 2008; Chism, 2007a; Cosh, 1998). Faculty members may become defensive to criticism and/or hostile when suggestions of change are made (Cosh, 1998). Chism (2007a) notes that peer reviewers may be more forthcoming when their comments are kept anonymous. Given that peer reviews are, in practice, generally synonymous with classroom observations, complete anonymity may be impossible. A faculty member’s apprehensiveness about giving feedback can intensify when a colleague’s career advancement is at stake. When  22 stakes are high—and even when they are not, the discomfort academics experience may lead them to record only positive comments (Cosh, 1998). In those cases, the peer review process becomes a type of “mutual back-patting, meaningless for genuine staff development” (Cosh, 1998, p. 172).  Resistance may also occur when faculty lack the skills and pedagogical knowledge to provide constructive feedback on teaching. Since routine discussions of teaching are rare among colleagues (Chism, 2007a; Cox, 2004; Gosling, 2002; Massy et al., 1994), faculty members frequently lack the vocabulary to talk about teaching philosophies and approaches. Instead, they may focus their comments on course content as an area they feel qualified to assess (Gosling & Ritchie, 2003). When academics are able to engage in high-quality feedback which includes reflection on teaching, the process is more likely to promote professional growth. The reviewer’s skill, as well as the willingness of the one being reviewed to engage in reflection on his/her own practice, helps ensure the success of this process (Hammersley-Fletcher & Orsmond, 2004). Still, some authors (i.e., Cosh, 1998; Peel, 2005) maintain that when reviewers offer suggestions on how to teach, the emphasis remains on teaching practices developed by others rather than fostering active self-development through reflective practice (Cosh, 1998). “Prescribing ‘how to teach’ does not send out a message, or invitation, for continued (self-directed) learning, and may damage morale,” cautions Peel (2005, p. 493).  Trust between reviewer and person reviewed is another prime consideration in peer review, and lack of it presents a significant source of concern. Academics may feel vulnerable vis-à-  23 vis existing personal or professional rivalries with the reviewer. When faculty members are marginalized on the basis of gender, race, disability, or other factors such as teaching in an area that is not valued, they may worry that bias inherent in the process, instruments, and/or reviewer assumptions will contaminate the results (Chism, 2007a). In summative peer review, where the focus is not necessarily on professional growth and learning, a lack of trust can prevent academics from fully engaging in the process (Gosling, 2002). However, when the person being reviewed “accepts, or even welcomes, the comments of the observer, it can be a powerful learning experience” (Gosling, 2002). Those who have written about formative peer review have repeatedly emphasized the importance of trust and openness in the reviewer-reviewed relationship (Blackmore, 2005; Bovill, 2008; Cosh, 1998; Donnelly, 2007; Gosling, 2002; Shortland, 2010; Toth & McKey, 2010b).  2.1.2.4 Reliability and Validity Issues A common faculty reservation about peer reviews is their uncertain reliability and validity. Given greater confidence in the reliability and validity of peer reviews, faculty members might be more inclined to pay attention to peer evaluation results (Murphy et al., 2009; Yon, Burnap, & Kohut, 2002). In the literature, concerns about the reliability and validity of peer reviews are frequently linked to vague standards for effective teaching, the subjectivity of the process, and the limited amount of information being considered.  24  2.1.2.4.1 Lack of Accepted Standards The validity of peer reviews depends largely on academics’ ability to achieve consensus on what constitutes effective teaching in the local culture (Chism, 2007a). But, effective teaching is difficult to define (Chism, 2007a; Murphy et al., 2009; Peel, 2005; Yon et al., 2002) and there is no universally accepted definition of what comprises an excellent teacher (Arreola, 2007). It has even been claimed that teaching is “impossible to define in a list of criteria” (Cosh, 1998, p. 172). This is not surprising, given that the practice of teaching is a highly complex intellectual endeavour that demands not only disciplinary expertise, but also strong pedagogical knowledge and a thorough understanding of students (Arreola, 2007; Chism, 2007a; Murphy et al., 2009). Faculty members who question the existence of criteria will naturally be concerned about reliability and validity of peer reviews; they may believe that if there is no standard for good teaching, and judgments are biased and arbitrary, there can be no way fair way to measure whether faculty members are meeting expectations. Drawing on the discussion regarding resistance, if we accept that reliability in peer reviews pertains to the consistent evaluation of teaching, it becomes clear that many factors make it difficult to achieve reliable results in peer review. If two reviewers of the same candidate do not share the same content knowledge, approach to teaching, skills at providing feedback, and/or collegial relationship with the candidate, the results of their individual reviews will differ. The fact that most peer reviewers lack training poses another challenge to reliability (Yon et al., 2002).   25 It is noteworthy that vague standards for teaching persist despite a solid body of research (i.e., Arreola, 2007; Chism, 2004; Fink, 2008) that has identified characteristics of effective teaching. This research confirms that standards of performance exist; however, as Chism (2007a) points out, “the setting of standards is more fluid and more situational than it is for less complex activities” (p. 23). Consequently, the application of standards relies on academics’ professional judgment for interpretation. It also requires that faculty members take the time to discuss, debate, and determine these standards and criteria for their department or unit. Doing so is a demanding task that necessitates regular dialogue about teaching. Unfortunately, in many departments there is no culture to support this (Chism, 2007a; Donnelly, 2007). Only when departmental values for teaching have been articulated can peers strive to align them with their evaluations, and such alignment is a fundamental principle of good teaching evaluation (Chism, 2007a).  2.1.2.4.2 Subjectivity/Objectivity Some faculty members resist the peer review of teaching because they perceive it to be a highly subjective process. They argue that evaluation of teaching should be objective, and that peer review would benefit from a systematic means of collecting data, such as checklists and forms based on established criteria (Gosling, 2002). Arreola (2007) points out that such objectivity is impossible since, by definition, evaluation involves applying a judgment to the measurement of data. He proposes aiming for “controlled subjectivity” which he claims can be achieved based on an a priori agreement about the values that will be applied and measured. As discussed, one of the challenges to the peer review of teaching is determining  26 the values associated with excellent, satisfactory, good, and poor teaching, and building the instruments and tools that capture these.  2.1.2.4.3 Limited Information Some academics are critical of the extent to which basing an assessment of teaching on a small number of class visits is reasonable and fair, because classroom teaching represents only a small portion of faculty members’ teaching activities. These include, but are not limited to, time in the classroom, student advising, curriculum development and revision, and administration (Toth & McKey, 2010a). However, peer review of teaching is often equated with classroom observation (Byrne et al., 2010) even though, in theory, it may encompass a review of the instructor’s teaching materials (e.g., syllabi, reading lists, exams), teaching portfolio, teaching-related committee work, student supervision, and student evaluations of teaching (Chism, 2007a; Murphy et al., 2009).  When peer review is based solely on classroom teaching, the reviewed class may not be representative of a faculty member’s overall teaching (Carter, 2008; Chism, 2007a). Furthermore, the presence of the reviewer or a video camera may influence what is being observed, especially when the class is small (Blackmore, 2005; Gosling, 2002).  2.1.2.5 Infringement on Autonomy Resistance to peer review may intensify if faculty members perceive the process as an infringement on their autonomy, a core value in the academic profession (Berquist & Pawlak, 2008; Schuster & Finkelstein, 2006). Since teaching within higher education has a long  27 tradition as a private enterprise, academics can resist “going public” with their teaching (Hutchings, 1996, p. 226). As Berquist & Pawlak (2008) explain: Many faculty members in the collegial culture would take great offense at being asked, let alone required, to accept an observing colleague in their classrooms. Ironically, even though classroom teaching is certainly a public event, it is considered an intimate exchange between faculty member and student. This exchange might be profoundly disrupted if observed and judged by another faculty member. (p. 31) Going public with one’s teaching might lead faculty members to feel more limited about their ability to make autonomous decisions with respect to structuring and managing their work within particular assignments (Gappa et al., 2007). Because academic freedom and autonomy are chief reasons why faculty members experience satisfaction in their profession (Berquist & Pawlak, 2008; Schuster & Finkelstein, 2006), faculty members may prefer to avoid activities that compromise these values.  In addition, faculty members may feel their autonomy is being reduced when they see peer reviews as yet another accountability measure aimed at regulating and monitoring their activities (Ackerman, Gross, & Vigneron, 2009; Hatzipannagos & Lygo-Baker, 2006).  2.2 Summary: Peer Review of Teaching Literature Within the literature, formative peer review of teaching is typically contrasted against summative peer review, each type presented as having distinct purposes. In practice, the differences are frequently blurred, and both processes may contribute to professional growth by promoting dialogue about teaching among departmental peers. Ideally, these evaluations  28 prompt faculty members to establish and clarify characteristics of good teaching for their department through an ongoing process of communication and collaboration.  However, just as formative and summative processes can contribute to professional growth in teaching and help establish a culture of teaching, they can also generate resistance among faculty members. Though some is due to practical issues (i.e.,  reviews based on limited information or which occur infrequently), much of the resistance concerns academic values and norms like collegiality, autonomy, and limited rewards for teaching. These values and norms manifest within and are part of the academic and department cultures to which faculty belong. Since the concept of culture is important to understanding faculty members’ choices and actions, the second part of the literature review will focus on the topics of academic, department, and teaching cultures.  2.3 Department Culture In order to understand how departmental practices can shape the value of teaching and influence faculty members’ engagement in professional growth in teaching, it is essential to recognize the overarching role of academic culture in faculty careers. I therefore begin this section with a short introduction to academic culture and then move to a review of the relevant literature on department culture.  29  2.3.1 Academic Culture, Values and Norms As defined in Chapter 1, academic culture acknowledges that academics are a part of and shaped by interconnected groups tied to their institution, discipline, and department (Clark, 1984; Gappa et al., 2007). As Clark (1984) so aptly describes: Academics are caught up in various matrices, with multiple memberships that shape their work, call upon their loyalties and apportion their authority….the academic belongs simultaneously to a discipline, a field of study, and an enterprise, a specific university or college. (p. 112) Thus, institutional, disciplinary, and department cultures intersect in complex and varied ways to shape faculty careers.  Scholars who study academic culture agree that the system of higher education is value laden, and have identified collegiality and autonomy as dominant values. For the purpose of this study, I presume that: • Collegiality is a value which, when put in practice, provides “opportunities for faculty members to feel that they belong to a mutually respectful community of colleagues who value their unique contributions to their institutions and who are concerned about their overall well-being” (Gappa et al., 2007, p. 142). • Autonomy is a value that pertains to freedom of expression and is a component of academic freedom. Autonomy relates to a faculty member’s right to decide and/or act according to his or her judgment when managing and structuring work (Berquist & Pawlak, 2008; Schuster & Finklestein, 2006).  30 These values and others affect faculty members’ behaviours and attitudes in significant ways (Clark, 1980, 1987; Gappa et al., 2007; Kezar, 2001; O'Meara et al., 2008); consequently, I will refer back to and elaborate on them throughout the dissertation, as I examine the issue of professional growth in teaching.  Like values, norms occupy a central place in academic culture. Braxton (2010) defines norms as beliefs held by a particular group of people about “expected or desired behaviors in a given situation or circumstance” (p. 243). Formal and informal norms develop for behaviours which the majority of the group considers important. Academics internalize these norms which over time shape academic codes of conduct (O’Meara, 2011). In universities and colleges, where faculty members have a lot of autonomy, norms provide guidelines for appropriate behaviours (Braxton, 2010). Norms and values are relevant to a study of academic careers by helping us grasp the relationship between department cultures and faculty members’ choices concerning professional growth in teaching.  2.3.2 The Academic Department and Culture(s) of Teaching Shaped and defined by institutional and disciplinary cultures (Clark, 1987; Lee, 2007; Walvoord et al., 2000), academic departments “are the most salient organizational aspect of higher education for thousands of faculty and millions of students” (Hearn, 2007, p. 224). As the focal point of academic work, they have been described as the “home” within which the academic lives (Lee, 2007). Complex units that tend to be distinct in their organization, policies, standards, and resources, departments are considered the fundamental component of  31 the North American higher education institution (Hearn, 2007; Knight & Trowler, 2000; Lee, 2007).  Within the institution, departments operate fairly independently as they coordinate and manage academic processes. Decisions on course offerings and content, appointing and promoting faculty members and administrative staff, and managing services are normally done at the department level on behalf of the larger institution (Lee, 2007). Because they are accorded significant power in areas that include faculty member promotion and the management of programs and services for professors, what happens in these units can influence faculty members’ behaviours (Hearn, 2007; Lee, 2007). A number of scholars have suggested that the department culture has a significant impact on how faculty members pursue their teaching responsibilities (Lee, 2007; Massy et al., 1994; Trowler & Knight, 2000).  In discussions of teaching and faculty careers, the terms “culture of teaching” or “culture of teaching and learning” are commonly used to refer to the various influences that affect faculty members in their role as instructors. A strong culture of teaching is typically taken to mean a culture that values teaching; in a department culture that values teaching, faculty members will presumably be supported in their professional growth as teachers.  A strong departmental teaching culture (one that values teaching) has distinct characteristics that can include frequent interactions and collaboration among faculty members about teaching, and the presence of an effective department head who is supportive of teaching and  32 communicates a high value placed on that aspect of faculty work. Though other factors also denote a strong teaching culture, i.e., support and commitment from senior administrators, a rigourous evaluation system aligned to promotion and tenure decisions, and the existence of a vibrant educational development program/centre, I have chosen to elaborate on the role of collegial conversations and departmental leadership as these are tied to the main identifiers of a culture of teaching, namely, ongoing involvement of faculty members in all aspects of planning and implementing teaching-related initiatives; the existence of a reward structure that appropriately recognizes faculty teaching and involvement in the scholarship of teaching and learning; and attention to teaching in hiring processes (Bensimon et al., 2000; Braxton, 2008).  2.3.2.1 Learning Conversations and Professional Growth in Teaching Although faculty members can learn and grow as teachers through such structured programs as workshops, learning communities focused on teaching, or formative peer review of teaching programs, the contribution these activities make to professional growth is relatively small (Walvoord et al., 2000). This is due in part to the fact that few faculty members participate in formal, voluntary programs (Altman, 2004). Studies have found that as with many professionals, those in higher education learn by taking part in everyday experiences (Clegg, 2003; Eraut, 2000; Jawitz, 2009; Knight, Tait, & Yorke, 2006). That is, learning about teaching occurs as faculty members plan their work, perform course reviews, serve on committees, grade assignments, advise students, revise their courses, assess and evaluate one another’s work, and so on. Learning that occurs unintentionally, through activities with no set learning objectives, is known as informal learning (Werquin, 2007). It is often contrasted  33 against formal learning, which includes engaging in activities like courses and training workshops attended for the explicit purpose of gaining specific skills, competencies, and knowledge (Werquin, 2007).  The literature indicates that conversations among colleagues about teaching present important opportunities for learning and intellectual growth. These interactions help faculty members expand their understanding of teaching and student learning (Knight et al., 2006; Neumann, 2009; O’Meara et al., 2008; Roxå & Mårtensson, 2009; Trowler & Knight, 2000) and also, more broadly, of the academic profession and their institution (Gappa et al., 2007).  Conversations that foster professional growth in teaching have been described variously in the literature. Roxå and Mårtensson (2009) employ the term “significant conversations” to denote discussions that help university teachers create and maintain their understanding of teaching. In these conversations, academics attempt to make sense of their teaching experiences, discuss their challenges, evaluate their situations, and plan future actions. The authors describe that such discussions characteristically occur privately, among trusted colleagues. Similarly, Byrne, Browne, and Challen (2010) emphasize the value of “learning conversations”—the dialogue between colleagues that enables reflection on teaching. Like Roxå and Mårtensson, Byrne et al., note that when peers share their understanding of teaching-related issues, question assumptions, articulate personal theories (which may include literature on teaching), it can lead to professional growth. Similarly, Haig (2005) affirms that conversations are a key feature in professional learning about teaching.   34 Though faculty members value collegial conversations, several factors can impede these. For example, collegial communication and interactions may be hindered when faculty members feel marginalized because of gender (Haag, 2005; O’Meara et al., 2008). Numerous studies have found that female academics face persistent barriers within their chosen career compared with their male colleagues; they have slower rates of career progression (Barrett & Barrett, 2010; O’Meara et al., 2008; Premeaux & Mondy, 2002) and are less likely to earn tenure and become full professors (Premeaux & Mondy, 2002; Winkler, 2000). In addition, women hold a greater proportion of non-tenure track positions (Gappa et al., 2007; O’Meara et al., 2008), are more involved in service work to the detriment of research productivity (Misra, Hickes Lundquist, Holmes, & Agiomavritis, 2011), and are more frequently challenged by students who dispute their roles as professionals (Kardia & Wright, 2004). Not surprisingly, women experience more overall stress than their male counterparts (Acker & Armenti, 2004; Allan, 2011). These aspects of academic life can have an impact on everyday interactions. Communication among colleagues may also be inhibited because of marginalization due to race, age, and/or appointment type (Gappa et al., 2007) and faculty conflict may be associated with resource allocation and the assignment of instructional loads (Hearn, 2007; Hearn & Anderson, 2002).  Finally, some research has suggested that differences in disciplinary culture can impede communication. Indeed, faculty report they do not have regular opportunities to meet or talk with colleagues in other specializations (Gappa et al., 2007; Hamilton, 2007). Such distinctions among disciplinary culture mean that as disciplinary fragmentation occurs, there may be a shrinking number of peers with whom faculty can engage (Massy et al., 1994).  35 Nevertheless, when collegial conversations about teaching occurred within departments or disciplines, academics were more likely to perceive them as relevant to their practice and personal situation (Quinlan & Åkerlind, 2000). Clearly, disciplinary values, norms, and behaviours have an impact on collegial interactions and faculty teaching (Becher & Trowler, 2001; Healey & Jenkins, 2003; Umbach, 2007).  2.3.2.1.1 Disciplinary Culture, Teaching, and Collaboration Faculty members identify strongly with their discipline (Becher & Trowler, 2001; Healey & Jenkins, 2003; Umbach, 2007). Members of the same disciplinary cultures share common vocabularies, similar norms of scholarly collaboration and codes of ethics, are drawn to certain research methodologies, and belong to the same learned societies (Gizir & Simsek, 2005).  Different disciplinary cultures have their own distinct concepts, methods, and aims that influence the ways in which academics organize and enact their work lives (Becher & Trowler, 2001). The notion of disciplinary groupings is commonly described using the Biglan/Becher typology. Biglan developed a model for categorizing academic disciplines that was elaborated on by Becher in 1989 (Becher & Trowler, 2001). Becher’s model, based on the assumption that different disciplines have distinct cultural characteristics, consists of a four-group classification system: hard-pure, soft-pure, hard-applied, and soft-applied (Becher, 1994). According to research on disciplinary cultures, hard-pure fields such as chemistry and physics tend to be concerned with universals and have a quantitative emphasis (Becher, 1994; Neumann, Parry, & Becher, 2002). In contrast, soft-pure disciplines like anthropology and history are concerned with discovery through interpretation and more often  36 use qualitative approaches (Becher, 1994; Umbach, 2007). Academics in hard-pure disciplines engage more frequently in collaborative practices than do academics in soft-pure disciplines, who tend to pursue individual scholarly interests (Becher & Trowler, 2001). Compared with their “pure” counterparts, hard-applied fields are oriented toward purposive, pragmatic know-how (Becher, 1994).  Since disciplinary	
  cultures have distinct patterns of performing scholarly tasks (Gizir & Simsek, 2005; Hearn & Anderson, 2002; Quinlan & Åkerlind, 2000), disciplinary differences can shape how academics approach teaching (Kreber & Castleden, 2008; Neumann, 2001; Neumann et al., 2002; Umbach, 2007). In hard-pure disciplines, course content is normally “linear, straightforward and uncontentious” (Lindblom-Ylanne, Trigwell, Nevgi, & Ashwin, 2006, p. 287) and courses are tightly structured. Courses in soft-pure disciplines tend to be more loosely organized than those in the hard sciences (Neumann, 2001). When they teach, faculty members in the hard sciences favour learning facts and concepts whereas those in the soft disciplines emphasize the development of cognitive skills (Umbach, 2007). Academics in hard-pure disciplines often teach in teams and there is more agreement among faculty members as to the content and sequencing of the curriculum (Gibbs et al., 2008). In comparison, those in the soft-pure disciplines reportedly have a broad range of views on how the curriculum should be structured, and their teaching is often kept private. Existing research (e.g., Lattuca & Stark, 1994, Neumann, 2001, Umbach, 2007) that has examined the relationship between disciplines and teaching has found that disciplinary differences are tied to teaching preparation time, beliefs about student learning, supervision, assessment and evaluation.  37  Though Becher and Trowler (2001) have noted a substantial variation in communication and collaborative practices even within a particular disciplinary grouping, these categorizations nevertheless offer a useful framework for exploring disciplinary differences.  2.3.2.2 Departmental Leadership in Teaching Appropriate leadership from the department head is a key factor in determining the extent to which a department actively supports teaching (Anderson, Scott, & Coates, 2008; Braxton, 2008; Knight & Trowler, 2000; Massy et al., 1994). Department heads exercise considerable power in their positions and, through their various responsibilities and choices, can encourage a culture of teaching (Braxton, 2008; Gibbs et al., 2008; Knight & Trowler, 2000; Massy et al., 1994). The skills and knowledge required by a department head are complex and extensive, and include knowledge about institutional structure and systems, human resource policies and practices, faculty members’ expertise within the department, curriculum, and information technology. In order to affect the culture of teaching and learning, department heads are also expected to have leadership skills (Berdrow, 2010). Gibbs et al. (2008), in an international study on approaches to departmental leadership in research-intensive universities, found that this role is shaped not only by the way in which heads think about their leadership, but also by the organizational culture of the institution and the discipline within which the heads work.  In a practical sense, the department head can enhance the culture of teaching and learning in a number of ways, thereby contributing to an environment whereby faculty members are  38 inclined to participate in professional growth activities related to their teaching practice. First, they can allocate meaningful rewards to faculty members for their teaching achievements (Braxton, 2008; Gibbs et al., 2008; Massy et al., 1994; O’Meara, 2005). Chairs can explicitly value teaching in appointment, tenure, and promotion decisions and also make visible the value they place on teaching through organizing awards (Gibbs et al., 2008).  Second, department heads can promptly communicate teaching evaluations with faculty members (Del Favero, 2002). These discussions should include consideration of how the results align with departmental criteria for teaching quality as this will allow faculty members to determine professional development needs and give them a clearer understanding of the expectations for tenure and promotion (Bensimon, Ward, & Sanders, 2000).  Third, the department head can ensure that faculty members have the necessary information to advise students (Del Favero, 2002). Advising is a frequently overlooked component in teaching, especially in the case of new professors. Yet the advising relationship can have profound consequences on students’ choice of courses, career, and further education (Bensimmon et al., 2000). Department heads can assist the process by ensuring that relevant course information, degree requirements, suggested coursework progression, and other instructions be shared and readily accessible.  Fourth, because teaching overlaps administrative functions from ordering textbooks to student evaluations, the department head can enhance the climate by communicating  39 available administrative expertise and support for teaching-related activities (Del Favero, 2002).  Fifth, the department head can ensure equal teaching responsibilities are distributed among faculty members. An egalitarian system means that faculty at all ranks teach courses ranging from introductory to advanced and have approximately the same teaching load and opportunities for course rotation (Massy et al., 1994). Finally, department heads can foster a healthy teaching climate and contribute to an outstanding teaching culture when they diagnose teaching problems and mobilize faculty, students, and others to address these in a productive way (Gibbs et al., 2008).  Department heads can take an even more direct approach to encouraging professional growth in teaching by ensuring a teaching orientation for new professors, organizing teaching- focused workshops for all faculty members, and instituting a mentorship program for teaching. Formative peer reviews of teaching may be one means by which faculty members mentor one another; the department head can establish the success of this practice by working to embed it into the culture and directly addressing faculty reservations (Chism, 2007b). Department heads further promote professional growth in teaching when they inform faculty members about the services offered by teaching and learning centres on campus (Bensimon, et a., 2000; Braxton, 2008). In addition to these possibilities, the departmental chair can create opportunities for faculty to collaborate on formal teaching projects (Quinlan & Åkerlind, 2000) or meet regularly to converse about teaching (Braxton, 2008). Since collaboration and conversation among peers occur more often in a department with a  40 collegial atmosphere (Bensimon et al., 2000), department heads need to pay close attention to this aspect of the culture.  2.3.3 Summary: Department Culture Literature Much professional learning about teaching is social and cultural in nature. It occurs largely within the department as academics carry out their daily activities, including interacting with their peers. “Learning opportunities come from the chance to engage with others’ practices and from engaging others with one’s own practices” (Knight et al., 2006, p. 334). As such, it is useful to create many opportunities for collaboration and informal discussions because the sharing and exchange of ideas encourage faculty members to engage with the collective understandings and assumptions within their department.  As it pertains to teaching, the concept of a culture of teaching and learning is useful for examining departmental practices. Learning conversations among peers and organizational leadership—in particular, that coming from the head of the department—are major influences on the departmental culture of teaching and faculty members’ professional growth in teaching. In addition, institutional and disciplinary cultures, with their particular norms and values, occupy a role in shaping the departmental culture of teaching.  2.4 Summary In summary, this review of the literature points to five important ideas that will guide the focus of this dissertation:  41 (1) Departmental culture provides a useful lens for examining practices that promote or hinder faculty members’ professional growth in teaching. (2) Interactions among colleagues can stimulate, shape, and advance learning; within academia, collegiality is a powerful value and this sense of belonging to a community of scholars can enhance job satisfaction and motivation (Gappa et al., 2007; Neumann, 2005; O’Meara et al., 2008).  (3) While much research depicts formative peer review of teaching as distinctly different from summative review, there may be significant overlap between the practices. Summative, not only formative, peer review can foster academic growth and development. (4) Faculty members’ experiences of summative peer review as a mechanism for professional growth in teaching has been an understudied phenomenon. (5) An exploration of faculty members’ everyday practices as they relate to teaching and occur within a department is important for research on teaching and learning.   42 Chapter  3: Methodology 3.1 Introduction The aim of this study is to explore opportunities for professional growth in teaching, both in the context of faculty members’ experiences with summative peer review of teaching and in other departmental practices. In order to explore this issue, I conducted a qualitative study that relied on interviews with tenure-track faculty members working at a research-intensive university. In this chapter I outline the perspectives and decisions that guided this study. In addition, I will 1) clarify my perspectives as a researcher; 2) discuss changes to the original research questions; 3) present the details of the research design, data collection, and analysis; 4) describe the study’s limitations; and 5) outline ways in which I strove for rigour and applied methods to help safeguard the rights, trust, and dignity of the study participants.  3.2 Positioning Myself As a researcher, I conceptualize, design, and conduct the study in ways that reflect my beliefs about the world, and how it should be studied and understood (Mayan, 2009; Schram, 2003). Qualitative research texts emphasize the importance of reflexivity1; good research requires  1     Reflexivity is “the process of being highly attentive to how and why you make decisions and interpretations along the research way, critically examining your personal-researcher role and how this interfaces with all—even the most minute—aspects of the research” (Mayan, 2009, p. 137).  43 one’s beliefs, assumptions, and roles be made explicit in writing (Creswell, 2007) and I will begin by outlining some of these in order to provide a rationale for, and further insight into, my methodology.2  3.2.1 Intellectual Position Based on my understanding of the interpretive paradigm, it is one I associate with most closely. The interpretivist researcher seeks to explore and comprehend the social world using both participants’ understandings and her own. “The researcher’s intent,” writes Creswell, “is to make sense (or interpret) the meanings others have about the world” (2007, p. 21). Looking for meanings and not necessarily causes, the interpretivist acknowledges the inter- relatedness of different aspects of the participants’ lives (Snape & Spencer, 2003). She recognizes that social, cultural, historical, ethnic, political, and other factors influence how people make meaning of their specific situation (Schram, 2003) and therefore seeks a complexity of views among her participants (Creswell, 2007). In the interpretive paradigm, the researcher is also aware that her own background, perspectives, and values shape her research design, interpretations, and findings (Creswell, 2007; MacKenzie & Knipe, 2006; Snape & Spencer, 2003).  2      I understand methodology as the overall approach to research (MacKenzie & Knipe, 2006) connected to the researcher’s paradigm, and comprising the beliefs, values, and theories that shape the researcher’s approach; as such, it influences design, data collection strategies, analysis, and the presentation of one’s findings (Mayan, 2009).  44  In addition to the interpretive paradigm, I also connect with feminist traditions. My original research questions (Section 3.3) clearly point to the fact that I am concerned about, and interested in, gender issues within higher education. In this study, the issue of gender stands out more through its silence than via the participants’ utterances. When I originally designed this study, I anticipated gender would figure more prominently in the findings. One of the chief aims of feminist research is to better understand power relations and the nature of enduring inequalities with respect to gender, so that they may be effectively transformed (Ramazanoglu & Holland, 2002). Feminist researchers have a political and ethical commitment to women that makes them accountable to a community of women who share political and/or moral concerns. Thus, a feminist research methodology investigates questions that arise from women’s lives; such questions, however, are not only about women's lives but also the rest of nature and social relations (Harding, 2004).  While a feminist methodology requires the concept of a community of women, it rejects the notion of an essential woman, one with whom all women will identify (Olesen, 2008). The notion of a universal woman is rather replaced by the idea of a woman who is situated within experiences, interests, and power relations that shape her existence (Olesen, 2008; Ramazanoglu & Holland, 2002). Because a feminist researcher distinguishes numerous differences in women’s experiences, she recognizes that being female doesn’t guarantee access to all women’s lives and knowledge (Olesen, 2008). Furthermore, she comprehends that knowledge is always partial, “in the sense of being ‘not-total’ and in the sense of being ‘not-impartial’” (Ramazanoglu & Holland, 2002, p. 66).  45  Finally, and similar to all qualitative investigations, researchers who adopt a feminist methodology identify the need to critically examine and continuously reflect on the nature of the research process. Consequently, the researcher is attentive to the methods she uses, her participants’ experiences of being subjects of the investigation, whose voices she represents, the way she writes herself into the research, and readers’ reactions to meanings and interpretations of the research. Aware that she has the power to judge between different representations of reality when she decides how to portray the lives and ideas of the individuals whom she researches, the feminist researcher recognizes her role in knowledge production. Reflexivity is a key aspect of a feminist methodology (Fonow & Cook, 2005).  3.2.2 Professional/Personal Position I come to this study as an educational developer, someone whose chosen profession is to support interested members of the higher education community in their growth as teachers. My work with faculty members is inspired by a belief that teaching is important and improving student learning is a worthwhile goal; these beliefs are central to educational development, as is the value placed on addressing individual well-being (Gosling, 2010). Though I maintain a focus on teaching and student learning, I am guided by the conviction that supporting academics’ growth as teachers is facilitated when one recognizes and honours the multiple roles played by faculty as citizens and scholars within their departments, campuses, professions, and wider communities (Sorcinelli et al., 2006). Thus my approach to promoting growth in teaching considers not only the individual, but also the institution and broader context within which academics pursue their careers.  46  I have worked in the field of educational development for over eight years and it was through my involvement with teaching portfolios that I became acquainted with the concept of peer review of teaching. My intention in entering a Ph.D. program was not to work as a faculty member, based on a belief that a demanding career would interfere with family life and the “balance” I seek long-term. This perception could be shaped in part by the fact that I was raised by two former professors. (Given the choice, my father would gladly repeat his career; my mother would not, witness the fact that, in her mid-40s, she resigned from her position as Associate Professor.)  3.3 Changes to the Original Research Questions In the dissertation proposal submitted to the University’s Behavioural Ethics Board in March 2010, I stated my primary research question as “In what ways, and to what extent, do gender, department culture, and disciplinary culture influence pre-tenured and tenured faculty members’ understandings and experiences of the summative peer review of teaching?” I intended to explore faculty experiences of the summative peer review through four inter- related lenses: gender; department culture, in particular as it relates to departmental values, beliefs, and practices about teaching; disciplinary culture, as it relates to disciplinary values, beliefs and practices about teaching; and academic rank.  In the preliminary data analysis stage which coincided with the start of the interviews, I began to anticipate it would be a challenge to distinguish faculty members’ experiences through the four lenses, given the small number of participants. While I recognized that my  47 questions were shifting, I was unable to specify any new questions.  During the writing phase of my dissertation, after I had worked with codes, categories, and themes for many months and had gained additional insights into the study, I revised the original questions included in my proposal, and developed subsidiary questions for the new research components:  1. What are faculty members’ experiences of summative peer review of teaching, and how do they understand the relationship between summative peer review and professional growth in teaching? a.  In what ways do the multiple purposes of summative peer review play out and interconnect? b.  How do academic values and norms, as well as other established practices in higher education, influence the summative peer review process?  2. What existing departmental practices support or hinder a culture that values teaching? a.  How do aspects of institutional and disciplinary culture influence departmental practices that support/hinder a culture that values teaching? b.  What is the relationship between departmental practices that foster a culture that values teaching and professional growth in teaching?  Compared to the original questions, these modified questions align more closely with what the participants and I discussed during the interviews. That my questions transformed throughout the process is common in qualitative research; as Maxwell points out, “qualitative  48 researchers often don’t develop their eventual research questions until they have done a significant amount of data collection and analysis” (2005, p. 65). The original questions provided a frame for the data collection process; but as the various design components interacted (Maxwell, 2005), and I began to undertake analysis, my questions changed.  3.4 Selecting Faculties, Departments and Faculty Members I took a case study approach to this research as this is well-suited to gaining a thorough understanding of a particular situation, and the meaning it may have for all those involved (Merriam, 1998). Case studies are comprehensive descriptions and analyses of a bounded system such as a group, individual, program, activity, process, or event (Creswell, 2008; Merriam, 1998; Yin, 2009), and researchers who use this approach are committed to studying the complexity of the bounded systems they predefine (Thomas, 2011). Merriam (1998) states that the most distinguishing feature of case studies research is the placing of limits on the object of study; consequently, investigators must clearly define boundaries in terms of time, place, components to be studied, or other concrete physical features (Merriam, 2002; Stake, 2005; VanWynsberghe & Khan, 2007; Yin, 2009). The more a case is specific and bounded, the more useful it will be (Stake, 2005). Since not everything about a case can be understood, the researcher must strategically decide which aspects to study and then determine what the reader needs to know, based on the information derived (Stake, 2005). However, defining the limits of a case is not a simple matter. Cases may be embedded in various contexts and have multiple subsections, dimensions, groups, and events, each with its own context (Stake, 2005; Yin, 2009).  49  The case study approach is consistent with an embedded case study, in which I focused on multiple cases within a bounded system (a single institution) (Yin, 2009). I selected two faculties (Arts and Science) and recruited participants from three departments within each faculty. Below I will elaborate on my choice of faculties, departments, and participants.  3.4.1 Institution This study took place at UWC, a research-intensive university in North America. UWC is home to over 2,000 professors (see Section 3.4.4 for definition of professor). According to UWC's 2010 statistics, 30 percent of professors are women and 22 percent are assistant professors.  UWC was chosen for several reasons. As a large university, it provided the chance to select (invite) departmental participation across a wide range of disciplines. Given my original intention of linking faculty members’ experiences to disciplinary culture, having a wide choice of departments would be helpful. Second, UWC’s status as a research-intensive institution made it an interesting site to study teaching because of the multiple and sometimes conflicting demands placed upon professors. Because research is more highly rewarded in decisions about tenure, promotion, and merit, faculty members at such institutions often limit the time and attention devoted to teaching (Chalmers, 2010; Hardy & Smith, 2006; Kilgore & Cook, 2007). Still, universities have begun paying increased attention to teaching (Groccia, 2010; Macfarlane, 2011; O’Meara, 2006) and a study that examined departmental cultures and their connection to teaching in a research-intensive context would be relevant and timely.  50 Finally, site selection was based on the fact that, since UWC is close to home, I would not have to travel—an important factor as it caused less disruption to my family, and therefore me.  3.4.2 Faculties My original research questions were aimed at understanding differences in faculty members’ experiences and understandings of summative peer review through the lines of gender, rank, departmental culture, and disciplinary culture. My recruitment strategy described below reflects this original intent.  To assist the process of faculty and departmental selection, I located institutional statistics posted on UWC's institutional research website. Because detailed department information was only available for Education, Applied Science, Arts, and Science, I narrowed my initial choice to these faculties. I ultimately selected the faculties of Science and Arts as these represented contrasting categories in Becher’s (1994) typology (see also Becher & Trowler, 2001). According to this typology, many disciplines in Science are classified as hard-pure, whereas many in the Arts (social sciences and humanities) are classified as soft-pure. Another criterion that drew me was that Science and the Arts house disciplines outside of my own and represent cultures with which I am unfamiliar. I felt that investigating settings where I had no set ideas or assumptions about “how things work” would be advantageous. While I did not aim to be objective or neutral, I was interested in a context that existed outside the faculty of Education proper.  51  3.4.3 Departments Once I selected the faculties, I identified departments whose faculty members I could invite to participate. Again, I considered only departments where there was statistical information available on the institutional research website. I then narrowed the choice of department based on two main criteria: First, to increase my chances of recruiting individuals from those categories, there must be five or more assistant professors, and the same (or more) with the rank of associate and full professor combined.3 Since my original research sought to compare pre-tenured and tenured professors, the number of faculty members in these categories was relevant.4 Second, less than 20 percent of the faculty in a chosen department should hold the title of instructor or lecturer; it was my belief that a greater percentage of professors in such departments might teach undergraduate and graduate courses compared with departments that had a higher percentage of instructors and lecturers (see section 3.4.4 for information on the distinctions between instructors, sessionals, and professors). Finally, I selected six departments with the following profile:  3     I assumed associate and full professors were tenured but assistant professors were not. 4     Although I have categorized professors at three different ranks in Tables 1 and 2, the study design examined only two categories: pre-tenured and tenured. At the time of design, I (mistakenly) assumed that there was more homogeneity than there actually is in the experiences of tenured professors. What I learned is that the experiences of newly tenured faculty members and full professors appear to be quite different.   52  Table 1: Faculty of Arts: Representation by Rank and Gender in A1, A2, and A3  Department Academic Rank Total  Assistant Professor Associate Professor Full Professor Women Men Women Men Women Men Women Men A1 1 4 7 12 2 5 10 21 A2 1 6 1 2 1 8 3 16 A3 4 5 2 2 3 12 9 19  Table 2: Faculty of Science: Representation by Rank and Gender in S1, S2, and S3  Department Academic Rank Total  Assistant Professor Associate Professor Full Professor Women Men Women Men Women Men Women Men S1 2 5 5 15 2 17 9 37 S2 3 6 2 16 2 19 7 41 S3 1 10 3 12 2 25 6 47  3.4.4 Defining the Professor For this study, I recruited tenure-track professors at the rank of assistant, associate, and full professor, as indicated on their departmental and/or institutional website. Although the dissertation refers to study participants as faculty members, I recognize that the term refers to a broader population than just those with the title of professor. Excluded from the selection  53 process were faculty members listed as “sessional lecturer,”5 “lecturer,” or “instructor.” At UWC, sessional lecturers and lecturers have “term appointments without review.” For the purpose of this study, these were understood to be non-tenure track appointments. Instructors, on the other hand, are hired “with review appointments.” Although instructors hold tenure-track positions, different criteria for promotion and tenure apply to instructors and “research professors.” 6 In particular, the criteria for their tenure and promotion is more heavily weighted on teaching (including educational leadership, curriculum development and other teaching and learning initiatives) for instructors than for those who are appointed as professors. I excluded sessional lecturers and lecturers from the study because, like instructors, whose teaching “counts” for more in career decisions, the literature has reported that the experiences of contingent faculty (lecturers and sessional lecturers in the case of UWC) are significantly different from those with tenure-track appointments (Gappa et al., 2007; O’Meara et al., 2008; Schuster & Finklestein, 2006).7  Where subsequent chapters quote or refer to specific participants, the following designations  5     At UWC, faculty members and others often refer to “sessional lecturers” as “sessionals.” Where it appears in this dissertations, the words “sessional faculty” or “sessional” refer to the “sessional lecturer” appointment. 6     The participants I interviewed commonly used the term “research professor” to distinguish them from other faculty members.  7     Those faculty in nontraditional appointments report less career satisfaction than their tenured colleagues (Gappa et al., 2007; Robinson, 2006), frequently attributed to the fact that they do not enjoy the same degree of academic freedom or job security as those with tenure, shoulder larger teaching loads, and experience feelings of second-class citizenship or marginalization (Gappa et al., 2007; Hamilton, 2007). Contingent faculty also express concerns about their capacity to develop campus-based relationships with colleagues (O’Meara et al., 2008). The result is a limited sense of commitment to the employing institution (Hamilton, 2007; O’Meara et al., 2008).   54 will indicate faculty, department, rank, and gender: • S1, S2, S3: Participants from the Faculty of Science, Departments 1, 2, or 3 • A1, A2, A3: Participants from the Faculty of Arts, Departments 1, 2, or 3 • Full professor (T); Associate professor (t); Assistant professor (a) • Male professor (M); Female professor (F) Thus, a quote or passage followed by “(Irwin, A1aM)” is referring to participant Dr. Irwin, an assistant male professor in the Faculty of Arts, Department 1.  3.4.5 Recruitment: Beginnings Once I identified the assistant, associate, and full professors from each department and collected email addresses, I began the actual recruitment process (see Letter of Initial Contact in Appendix A). Originally I aimed at recruiting a total of 20–24 participants. I started by inviting faculty members in S1 and S2, followed a week later by those in A1 and A2. (I postponed any recruitment from S3 and A3 until I was sure I could not recruit a sufficient number from the other four departments.) If a person responded with a yes or tentative yes, I sent them a short questionnaire (see Faculty Member Questionnaire in Appendix B) to ascertain whether they met the eligibility criteria laid out in Table 3 below.   55 Table 3: Participant Eligibility Criteria  Criteria Assistant Associate Full 1. Has been employed by UWC for 9+ months Yes  Yes Yes 2. Has peer reviewed a colleague on his/her teaching for tenure, promotion and/or reappointment Yes or No Must answer yes to criteria 2 or 3 or both. Must answer yes to criteria 2 or 3 or both. 3. Has been peer reviewed on his/her teaching for tenure, promotion and/or reappointment Yes or No8 Must answer yes to criteria 2 or 3 or both. Must answer yes to criteria 2 or 3 or both.    3.4.5.1 Challenges Recruiting Assistant Professors and Women It did not take long to notice that the greatest number of positive responses came from tenured male professors. In order to increase responses from assistant professors and women, I modified the eligibility criteria for assistant professors (fn. 8) and sent a follow-up email to those women and assistant professors who had not yet responded (Appendix C), indicating I was especially keen to hear from them. I also invited recipients to provide information on why they might not care to participate to help explain the low response rate from certain faculty categories. I heard from 5 individuals whose responses are presented in Table 4.  8     Prior to an official amendment of the Behavioural Ethics Review Board, I wanted to recruit those assistant professors whose teaching had been summatively peer reviewed. After almost a month (and only one assistant professor agreeing to participate), my committee decided that I should expand the criteria to include assistant professors who had not undergone a summative peer review on their teaching, as long as they could offer an opinion about it. Of the final participant selection, only one assistant professor (Irwin, A1aM) was not peer reviewed for tenure, promotion, or reappointment. All others had undergone the process at UWC.  56 Table 4: Responses to Follow up Email: Reasons for Not Participating  Rank, gender and Faculty Email response Summary of reason participant declined Associate, woman, S I would normally say yes, but I am really pressed for time this summer. Lack of time Associate, woman, S I've thought about your request after receiving your first email. Normally I'd like to help, but I really don't have any opinions re: the peer review of teaching. I wasn't even aware that it is going on, despite my being here for XX years. We get the student feedback on our courses (online form results) and that's about it. I think. Except when I went up for tenure and had faculty from my committee come and sit in a few of my classes. However, since their reports were private, I have no idea what they wrote/thought or whether the process worked or not -- well, I got tenure so I suppose it worked for me.  I hope you'll find enough faculty who actually have something to offer in response to your questions, I really don't have any thoughts on this. Does not have anything to contribute despite having been peer reviewed. Full, man, A I have considered your request. In the end, I have decided not to participate in the process, because I have concerns about disclosing confidential information. Concerns about confidentiality. Assistant, woman, S I am afraid that I cannot take part in your project, as this summer is an extremely busy time for me. The main barrier to participation is time - given than I am pre- tenure and have just returned from maternity leave, I have very little of it and I try to make the most with what I have. My suggestion to you is that if you want to attract more participants, you need to come up with a way to make participation in your study beneficial to the participants. I have in a past agreed to such things when it was clear that I was going to learn something useful to myself in the process. Lack of time. No benefit to participating. Associate, woman, A Thank you for your e-mail. I am going to decline to participate as I have had no experience with peer review of teaching. No experience with topic of peer review.  As a result of the follow-up emails, I was able to recruit additional women faculty members, but continued to have a low response rate from assistant professors. I also gained further  57 insight regarding non-participation, specifically that time was a barrier. Some faculty members also stated they had nothing to contribute, while others had concerns about confidentiality and/or could see no benefit from study participation.  Approximately a month into recruitment, I replaced my original plan to recruit an equal number of professors from each category (Arts/Science, Pre/Tenured, Women/Men) with that of recruiting an equal number of professors from Arts and Science, with a participation goal of 30% women (which represents the average ratio of tenure-track female faculty at UWC). By then I had sent two emails to all faculty members in S1, S2, S3, and to those in A1 and A2. After confirming participation from faculty members in A3, I completed my recruitment. Tables 5 and 6 represent the final results: Table 5: Final Recruitment from Faculty of Arts (n=15)  Department # Women Pre- Tenured # Men Pre- Tenured # Women Tenured # Men Tenured A1 0 (1) 2 (4) 0 (9) 1 (17) A2 0 (1) 0 (6) 1 (2) 3 (10) A3 2 (4) 1 (5) 2 (5) 3 (14) Total # 2 3 3 7 Note: Numbers in parentheses are the total number of professors at that rank in the specific department in 2009.   Table 6: Final Recruitment from Faculty of Science (n=15)  Department # Women Pre- Tenured # Men Pre- Tenured # Women Tenured # Men Tenured S1 1 (2) 0 (5) 3 (7) 2 (32) S2 1 (3) 0 (6) 2 (4) 2 (35) S3 0 (1) 1 (10) 1 (5) 2 (37) Total # 2 1 6 6 Note: Numbers in parentheses are the total number of professors at that rank in the specific department in 2009    58 3.5 Interviews Listening and asking questions are central components of a qualitative interview (Creswell, 2007; Fontana & Frey, 2008; Mayan, 2009). Applying these skills in this active and mutually relevant exchange is a difficult task, however. During the interview, as well as before and after, the researcher must establish and maintain trust, practice sensitivity, exercise intellectual judgment, and respond to multiple sensory observations (Fontana & Frey, 2008). Each research engagement thus began with a recruitment email and continued well after I had turned off the audio recorder or put away my pen.  3.5.1 Data Collection Data from this study consisted of 30 semi-structured interviews with professors. Mayan (2009) writes that the use of a semi-structured interview is appropriate when the researcher has a sufficient sense of the phenomenon to develop interview questions, but not so strong as to predict responses. By posing open-ended questions, the researcher aims to give informants space to express meaning in their own words (Brenner, 2006). I chose this method for this study because it had the potential to elicit thoughts, reflections, and ideas about the conditions of professional growth from a faculty member’s perspective.  Interviews lasted between 45–90 minutes, with an average of 60 minutes. All but two took place in the faculty member’s office on campus. The majority of interviews were audio recorded and then transcribed verbatim by a hired transcriptionist (some fillers omitted). Two interviews were not audio recorded upon the participants’ request, and I transcribed four interviews myself. With the understanding that academics have heavy workloads, only one interview was scheduled for each faculty member.  59  In preparation for the interviews, I pilot-tested my protocol with two faculty members in order to detect any weaknesses or flaws in the interview design (Turner, 2010). The value of those conversations (one with a pre-tenured male professor I met through a colleague, the other with a recently retired professor who is a longtime family friend) was immense, and I learned about aspects of summative peer review that I had not ascertained by reading the literature. Following the pilot, I modified some open-ended questions and probes (Barbour, 2008) and created the final protocol for pre-tenured and tenured faculty members  (Appendices D and E, respectively).9 During the actual interviews, the protocol was used as a guide to provide an overall framework. In that way, I could adjust the sequence, direction, and precise wording of my questions in response to any issues and emotions that were raised.  3.5.2 Methodological Issues One of the primary purposes of qualitative interviewing is to derive interpretations from the narratives and statements of respondents (Warren, 2001). The interview is a conversation (Kvale, 1995), meaning an interactional relationship in which researcher and participant are engaged in a process of knowledge production (Fontana & Frey, 2008; Kvale, 1995). As such, interview interactions are shaped by both perspectives, respondent and interviewer, based on their respective histories, gender, age, race, and class (Fontana & Frey, 2008;  9     For example, my first question was originally “Please tell me about your involvement in peer review.” For those who had a lot of experience with peer review, this proved to be a stumbling block, as the person would focus on recalling various experiences, i.e., when they happened and in what order. I later modified this to “Please tell me how peer review takes place in this department,” which allowed participants to relate personal experiences and/or knowledge of the departmental process.  60 Warren, 2001).  Over the course of this study, I became aware of the extent to which my status (i.e., a female graduate student who has never undergone a summative peer review) shaped the interviews. It meant I came to each interview as a learner (Warren, 2001), with no personal experience as a tenure-track faculty member nor with summative peer review. I sensed that research participants related to me in this role, and as a result were patient with my questions and attentive to my interest in learning. While I cannot be sure, I also wondered whether being a graduate student had some influence on the interviews, i.e., motivated faculty members to be interviewed despite their busy schedules. Did they contribute to my research as an extension of their responsibilities as a supervisor/mentor of graduate students? Because they assumed I might become a member of their chosen profession? Or because the topic piqued their interest and was something they wanted to discuss?  After conducting ten interviews with male professors, I began to interview the women, and noticed the effect my gender seemed to have on both sets of participants. Women spoke more frequently about stress in their career, used greater emotion when speaking about their frustrations, and even swore more often. Had I been a male interviewer, I wasn't sure the same conversations would have ensued.  In a chapter entitled “Interviewing Men,” Schwalbe and Wolkomir (2001) use the term baseline threat to describe fundamental interview characteristics (e.g., the interviewer sets the agenda, asks questions, probes for information, etc.) likely to cause both sexes some  61 discomfort. They distinguish these from what they call surplus threats, which are lines of questioning that can threaten the conventional ways men “do gender” in Western culture (West & Zimmerman, 1987). To oversimplify their descriptions, men may distinguish themselves from women by exercising a greater capacity and desire for control and portraying themselves as rational and autonomous in thought and action (Schwalbe & Wolkomir, 2001). The authors point out that questions about gender, or those questions that “put control, autonomy, or rationality into doubt” (p. 206), can be perceived as threatening. Given that such questions were included in my protocol, this dynamic may have been at work in some of my interviews with male professors.  The literature on qualitative interviewing also alerts us to the danger of presuming an interview relationship to be egalitarian (Fontana & Frey, 2008; Warren, 2001). It is often assumed that the researcher holds a position of power since it is s/he who defines the situation by introducing the topic and deciding how to follow up on a respondent’s answers (Kvale, 1995; Warren, 2001). Although I acknowledge authoring power with respect to the final text, when it came to the interviews I felt the participants in this study held the greater power. First, I relied on them for data (and therefore for completion of this project); and second, their status in the academy and society was greater than mine. Whether it be differentials related to social locations such as age, gender or position, the researcher needs to be attentive to how power differences between the interviewee and the researcher play out (Fontana & Frey, 2008; Warren, 2001).  62  3.6 Data Analysis I began the data analysis during data collection, making notes and reflecting on ideas after each interview. I created a face sheet (Grbich, 2007) for each participant, on which I recorded basic facts about the interview: name of participant, time/date/location, and duration; notes on what I could recall; things that surprised me; and issues that I wanted to follow up. The notes were generally typed up before I listened to the recording (i.e., within 1–2 days of the interview), but I occasionally compiled the face sheet only after listening to the interview. Face sheets were embedded in my research journal, an electronic document I began keeping in December, 2008, which I used to track ideas and decisions from discussions with my committee, thoughts generated as I worked through the study, reflections on readings from a qualitative data analysis course, and circumstances and feedback I wanted to record.  Shortly before the interviews were completed, I decided to use a qualitative data analysis software called Atlas.ti. Having never used this type of software before, I was not exactly sure how the process would unfold but trusted that, at the very least, Atlas.ti could help me manage, organize, code, and retrieve data from the interviews. I ended up proceeding as follows: Firstly, I chose three interviews, read through each, line by line, and assigned names to segments (e.g., sentences or paragraphs) of the data. I refer to this process as coding. This allowed me to revisit the interviews that, in some cases, I had not listened to or read for some time. Then, I carefully examined the codes I had created thus far, determining whether they captured the meaning of the text as I understood it. I subsequently modified a number of the codes, refining my code definitions and then re-coding the three interviews. Once satisfied with my ‘new’ codes, I coded the remaining 27 interviews. As I engaged in the coding, some  63 of my codes were further segmented and others were joined (see Table 7 for sample codes and categories). I simultaneously coded and categorized the data, exploring how codes and categories related to one another and remaining attuned to patterns in the data, and “negative cases,” (Mayan, 2009, p. 95) or data that are contrary to what the majority of other participants describe. Finally, after coding each interview, I wrote a detailed summary—an expanded face sheet of sorts—to help me see the interview as a whole again. Table 7: Categories and Codes (Select Examples)  Category Codes Process Communication Data Ideal Knowledge Timing  Reviewer Committee Rank Selection  Tenure and Teaching Administration Honesty Own experience Research vs. teaching Time  The entire process took several months, and in the end, I benefited from Atlas.ti when I was able to retrieve data associated with the codes and categories I had created. Using Microsoft Word, I grouped all the instances within a given code or category into separate documents. For example, for the code “honesty,” I examined each instance where a participant discussed or referred to honesty in terms of the summative peer review of teaching (e.g., whether they felt they could be honest writing teaching reports, sharing feedback, or during committee meetings where colleagues discussed the case). As I continued to create sub-categories, I  64 began to form a loosely detailed outline of themes from my findings. I then proceeded with analysis and writing.  Whereas the account above portrays my analysis as consisting of distinct steps, in reality the process was not so straightforward. I engaged in description, analysis, and interpretation throughout (Wolcott, 1994),10 but my emphasis varied at different times. A large amount of analysis later occurred as I outlined, wrote, and revised my dissertation.  3.7 Rigour and Limitations “Rigor,” writes Mayan (2009) “is demonstrating how and why (through methodology) the findings of a particular inquiry are worth paying attention to” (p. 100). In qualitative studies, the researcher strives for rigour when she keeps accurate records, follows systematic procedures, recognizes the complexity of a situation, and is explicit about her biases (Mayan, 2009; Merriam, 1998).  The limitations of this study can be specified in relation to three attributes of qualitative research: validity, reliability, and generalizability.  3.7.1 Validity Validity in qualitative research refers to whether conclusions, descriptions, and other  10     Drawing on Wolcott (1994), I take “description” to mean recording data, as the researcher originally encounters it. “Analysis” refers to the process whereby the researcher identifies essential features in the study and the relationships between them. During “interpretation,” the researcher extends the analysis, looking for further insights and creating new meaning from the data.  65 accounts can be found in the data (Mayan, 2009). It is not akin to “objective truth” but pertains, rather, to the notion of credibility. Validity is strengthened when a researcher endeavours to achieve methodological coherence—that is, the questions, methods, data, and analytic procedures all “match” (Morse, Barrett, Mayan, Olson, & Spiers, 2002). Validity is augmented when the researcher can furnish evidence that her findings provide an accurate representation of the participants and/or the data (Maxwell, 2005).  In order to augment the validity of my data, I implemented various verification strategies. First, I paid close attention to sampling to ensure that it allowed me to collect “‘rich’ data” (Maxwell, 2005, p. 110). Rich data provides the researcher with detailed descriptions of relevant phenomena (Mayan, 2009). The richness of the data in this study came from the fact that I interviewed 30 faculty members from different disciplinary backgrounds, departments, genders, and ranks. All had been involved in the peer review of teaching and could therefore speak from experience.11 It also resulted because 28 recorded interviews were transcribed verbatim; as such I did not rely on taking notes on only what I felt was significant. I listened to nearly each recorded interview three times: once within three days (almost always) of interviewing the participant, a second time when I meticulously reviewed the transcript upon receiving it and a third time just before sending the transcript back to the participants for verification. In the two cases when the participants asked not to be recorded, I was careful to  11     At the time of the interview, only one individual (Irwin, A1aM) had not taken part in the summative peer review of teaching for tenure or promotion. He had, however, been involved in team teaching, observing a colleague’s class for an entire semester, and other peer review situations at another institution. See also footnote 8.  66 take comprehensive notes of our discussion. Second, I was alert for any negative cases or data that were otherwise incongruous (Maxwell, 2005; Mayan, 2009). I was careful not to ignore these data, pondering their significance and implications and making them public in my findings. A third strategy was respondent verification. This, writes Maxwell (2005), is the most important means of reducing the possibility of misinterpreting what participants have said. In Autumn 2010, each participant was invited to review and modify the interview transcripts; one year later, they were invited to comment on the two findings chapters.  3.7.2 Reliability Reliability pertains to consistency in the data (Merriam, 2002). A qualitative study could be considered reliable if, for example, the same or different observers were assigning instances to the same category on different occasions (Long & Johnson, 2000).  Merriam (2002) suggests that some of the same techniques used to ensure validity, notably member checks and peer review (such as the review of my work by committee members), can be applied to achieve better reliability. In addition, she recommends the use of an audit trail, a detailed account of how data were collected and decisions made throughout the research project. In 2008, I began a research journal to document the evolution of my ideas, keep track of decisions, and note down questions. When I commenced my data collection in the spring of 2010, I used my research journal to write my thoughts and impressions after each interview, and later detailed summaries after coding line by line. I returned to those summaries multiple times, noting links between participants, further queries, and contradictions. As I coded each interview, I kept notes on the development and  67 transformation of my codes and categories. As I advanced my analysis, I worked through many of issues and questions by means of writing.  3.7.3 Generalizability Generalizability “refers to the extent to which one can extend the account of a particular situation or population to other persons, times, or settings than those directly studied” (Maxwell, 1992, p. 293). Flyvbjerg (2006) contests the belief that one cannot generalize from a single case, pointing out that many natural and social science problems have been solved through a detailed examination of an individual case, not by examining large, random samples. Instead, carefully chosen cases, which involve intense observation, can lead to important discoveries that contribute to the collective process of knowledge accumulation, which Flyvbjerg believes is also part of generalization.  In qualitative studies, the concept of internal generalizability is particularly useful. This concept concerns whether the conclusions found within the research sample can be generalized to persons and events that were not observed or interviewed but fall within the same group or institution as the data sample. Internal generalizability is contrasted against external generalizability, which pertains to whether the study findings can be inferred to settings beyond the one sampled (Maxwell, 1992, 2005). Since qualitative studies are not designed to infer findings to a broad population (Maxwell, 2005; Mayan, 2009; Merriam, 2002), external generalizability is a less critical issue in qualitative research. The sample need not be representative of a larger group; instead, participants are selected based on  68 characteristics that allow the researcher to conduct an in-depth exploration of a given issue (Ritchie, Lewis, & Elam, 2003).  For this study, my selection was aimed at capturing a range of experiences across departmental and disciplinary cultures, and across ranks and genders. This choice was driven by a belief that the faculty members I interviewed could provide valuable information about the peer review of teaching, departmental culture of teaching, and professional growth in teaching. Instead of trying to generalize based on a limited sample (one university, 30 participants, a particular moment in time), I drew on the strengths of qualitative research to better understand the phenomena I set out to study.  I recognize that this study is limited in size and scope. All faculty interviewed are from one institution and their numbers are not large. Despite some variation in the departmental and disciplinary cultures, as well as the rank and gender of the respondents, I did not seek to examine differences in race and ethnicity, although these factors have been shown to influence faculty members’ work lives (Gappa et al., 2007; Mamiseishvili, 2010; O’Meara et al., 2008). Furthermore, I conducted only one interview per participant, which likely restricted my ability to probe in depth.   69  3.8 Ethical Considerations This study relied on data collected from human subjects and was conducted in accordance with the standards set out by the UWC Behavioural Research Ethics Board (BREB), which reviewed and approved ethics documents pertaining to the study. Conducting a BREB- approved study entailed taking certain precautions to assure the rights and safety of the study participants. These are described below.  3.8.1 Respect It was my intent to act respectfully and with consideration for my participants throughout the study. Since faculty members were asked to speak about their careers (both positive and negative aspects), I was aware that they might disclose sensitive information and/or recall incidents that would be accompanied by strong emotions. I did my best to respond appropriately, acknowledging participants’ frustrations and reminding them of the steps taken to ensure confidentiality and anonymity (including turning off recording devices at any point). On a practical level, I was mindful of faculty members’ schedules and arranged interviews at times and locations convenient to them; during the interviews, I strove to honour participants’ time constraints.  3.8.2 Privacy and Confidentiality Throughout this study, the privacy and confidentiality of my 30 participants and two pilot participants were treated with the utmost concern. As sole investigator, I was the only person with access to original, identifiable data; any collected data have been secured. In hiring a transcription service, I chose a company that uses a standard protocol for maintaining  70 confidentiality and offers a written confidentiality agreement. To further protect participants’ identities, I generated pseudonyms using an online random name generator, and when writing about participants, attempted to shield their identity by masking research specializations, course titles, and personal characteristics. Although I have used pseudonyms, I did not conceal participant rank and gender, as these elements were relevant to the study. Pseudonyms were also assigned to the institution, and when specific names of units, initiatives, or institutional processes threatened identity, I used generic names.  3.8.3 Informed Consent For their protection, participants were informed of the nature of the study and asked to provide written consent (Appendices F and G). Data collection commenced after a participant had read and signed the consent forms. Participation in this study was voluntary and all participants were free to withdraw at any time.  Participants were allowed to review the full transcript of their interview and make any changes they felt were needed. Once my supervisors had approved an advanced draft of my findings chapters, participants were invited to review the findings in Chapters 4 and 5.  3.8.4 Institutional Review Board Approval In June 2009, I completed the Introductory Tutorial for the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans. I received BREB approval in March, 2010.   71  Chapter  4: Summative Peer Review of Teaching: Current Purposes and Envisioned Ideals 4.1 Chapter Structure In this chapter, I report on the participants’ understandings of and experiences with the summative peer review of teaching, and examine how they perceive the relationship between summative peer review and professional growth in teaching. I begin with a section on academic rewards and the demanding nature of academic careers as all participants made reference to these aspects of institutional culture when they narrated their experiences of summative peer review and spoke about professional growth in teaching. I then turn specifically to the peer review of teaching. First, I briefly outline the peer review process. In doing so, I aim to introduce some of the elements that make up peer review and highlight certain variations and inconsistencies inherent in the summative peer review of teaching practice. Next, I discuss three purposes of peer review. According to the participants, these are: a formal mechanism for evaluation, a means of promoting professional growth in teaching, and a supplement to the student evaluations of teaching. In the following section, I present findings on why the process does not foster professional growth in teaching and examine how the lack of feedback diminishes the potential for learning in the peer review of teaching. I then provide an account of participants’ descriptions of what an ideal peer review of teaching process might entail. I conclude this chapter with an analysis of the findings.  72  4.2 Institutional Culture: Endless Demands, Finite Time When participants refer to their careers in the research-intensive university, they repeatedly raise concerns about the high demands on their time, noting the consuming nature of research and teaching. Assistant and associate professors generally mention this more than full professors do, but the latter are cognizant of the pressures under which their pre-tenured and newly tenured peers operate. The main obstacle to the development of our younger faculty as effective teachers is the tremendous demands made on their time—the high expectations of research and publishing and productivity that are imposed on young faculty simultaneously to the expectation of teaching excellence. Every single junior faculty member could be a wonderful teacher and would want to be a wonderful teacher, and many of them are. But if any of them fall short of their own aspirations to be wonderful teachers, it is only because of the tremendous pressure also to perform at a standard of excellence in their other main responsibility, which is to do research and to publish their findings. (Manfred, A2TM) Though Manfred refers specifically to early career academics’ growth in the above quote, data from the interviews indicate that high expectations for research productivity affect participation in the summative peer review of teaching for faculty members at all career stages, and shape their ability to develop as teachers.  73  All faculty members acknowledge the competing nature of teaching and research, and several talked about the strategic choices they make between the two. That decision-making process can be difficult for people who wish to do their best in both research and teaching. I’m always torn between wanting to be a better teacher but being very, very invested in research and having that be, in some sense, the central part of my life, and then there’s teaching. And yet, if you’re going to do something, you want to do it right. So if I had infinite time to learn about teaching, I’d be involved in all these trenches, but I don’t. So I do what I can. (Ledonne, S1tF) Given the intensity of academic careers, faculty members indicate that they make decisions based on what they perceive to be most useful with respect to their professional and personal goals. You’d like to say we just want to be the best possible we can, but there’s finite resources that we all have to put into this and we have to decide strategically, you know, I already get four hours of sleep a night. Tell me to work harder at my teaching? (Palo, S1TF) Women express with more candour and emotion than their male counterparts the stress they experience in their academic lives as a result of the high demands on their time. “The time thing is a really, really major serious problem. The work/life balance is disastrous. I mean I would quit if I could, because I’m so tired. You know? I’m just so tired. I can’t do more” (Palo, S1TF). Strained by heavy loads, all faculty members want to be clear about the purpose of the activities they engage in and need to believe that these various activities support them in their career growth.   74 4.2.1 Teaching Counts (?) Participants unanimously agree that research productivity, as opposed to strong teaching, is what garners rewards and recognition with respect to career advancement and departmental prestige. A few point out that faculty members are not rewarded for exceptional teaching, and some participants recall being told that teaching awards or high teaching scores would “look bad for tenure” (Nemeth, A2tF); that is, strong teaching is equated to insufficient commitment to research. Consequently—more generally due to the importance of research productivity in tenure and promotion decisions—some tenured faculty members have advised their junior colleagues not to focus very much on teaching prior to tenure. Having a poor research record, faculty members concur, presents a greater impediment to obtaining tenure. In every department in Science, I think that the research definitely comes first. And if your research record is not in place, no way will you be promoted at all; that's it. And also, in considerations of merit review and salary raises, research ranks higher than teaching. (Knauer S2TF) Not only does research matter more in tenure and promotion decisions, but a few participants believe that when it comes to teaching, all faculty members have to do is meet a minimum threshold which in their opinion is easily attainable. If, as these faculty members posit, it is true that “most people teach well enough to get tenure at a research university” (Chen, A3TM), then their energy is best spent on research, since this is where they will reap the greatest rewards.  For a small number of pre-tenured or newly tenured faculty members (Cohen, A3aM; Dawson, S3aM; Fradera, A3tF; Wilks, S1tF), the relative weight of research and teaching on  75 promotion and tenure decisions is not as clear-cut. Cohen, whose teaching has been “really good” but whose “research has been okay,” ponders, “If my research isn’t going to be quite as strong enough, will my strong teaching help carry the day? I don’t know. We’ll see” (Cohen, A1aM). Similarly, Dawson, wonders whether, had he won a Killam teaching award (a prestigious award that recognizes teaching excellence) but only had “okay” research, administrators might feel more compelled to award tenure. He thinks the decision to award tenure might be influenced by the fact that the dean’s office is promoting teaching as important, demonstrated by their support for the Effective Science Education Project (ESEd).12  Two faculty members provide accounts of major changes they made to their courses, motivated in part by a belief that their efforts would be positively regarded by colleagues who vote on tenure and promotion. Wilks (S1tF) recalls, I did think that it would help fill out my teaching portfolio because it shows that, okay, I don’t have breadth of teaching in terms of other courses but it’s certainly not that somebody’s handed me the [name] courses and I’ve used their slides for the last five years and done nothing [laughs]. I’ve done a heck of a lot on the teaching front, or put a lot of energy in. . . . I knew in this department it would be positively seen. So I didn’t think, ‘I’m wasting my time, nobody in this department’s going to recognize  12     The ESEd Project aims at improving undergraduate science education at UWC. Started in 2006, it is a multi-year project that works collaboratively with UWC departments in the Faculty of Science on a range of activities meant to enhance science education.  76 it.’ If anything, I knew they would recognize the effort because they do place an important role on the teaching component. Dawson (S3aM), whose immense concern with student learning prompted him to take on the demanding task of completely redesigning one of his courses, reflects: “I thought it would help my case. . . . Will it help as much or in the right way? I don’t know.” These stories stand out because they contradict the dominant opinion that good teaching is not rewarded as well as strong research. Since, at the time of making the changes, both participants were pre- tenured, it is possible they might not have fully adopted the belief that research productivity matters more than teaching in tenure and promotion decisions. It is noteworthy that, unlike some of the pre-tenured/newly tenured faculty members who wonder about the role of teaching in moving a tenure file forward, tenured faculty are unequivocal about what matters in tenure and promotion decisions: research is what counts.  Despite an overall consensus that research counts more than teaching, several faculty members say that teaching ability is given serious consideration in tenure and promotion decisions. “Teaching is viewed very importantly and it’s very clear that promotions without good teaching are going to be extremely problematic, and therefore every possible effort is made to ensure teaching is good” (Knauer, S2TF). Moretti (S3TF) cites the fact that some faculty members are denied tenure based on a poor teaching record as proof that, institutionally, teaching matters more than it did before: It's been announced and written up in [name of university publication] that people are being denied tenure and promotion based on teaching. So that’s the most important thing. I think that sort of scared people into realizing teaching’s important, because  77 some people now have been denied tenure based on their teaching. And you know, ten or more years ago, that never ever, ever, ever, ever would have happened. 	
   Numerous faculty members, however, speak with cynicism about the university’s claim that teaching matters in tenure and promotion decisions. “Let’s put it that way,” says Maki (A3aF), “nobody’s ever been denied tenure for being a mediocre teacher.” Dawson reinforces this belief: “I would say the prevailing notion is that you can be a bad teacher and if you have excellent research, you will get tenure.” In addition, several associate and full faculty members remark that their teaching has never been peer reviewed since they attained tenure. Others like the faculty members in S1 point out that peer review, which takes place twice a year for pre-tenured faculty members, occurs only once every three years for tenured faculty. If “to care about teaching is to evaluate teaching” (Ludar, A3TM), then faculty members in all departments may perceive that this nonexistence/decrease in peer reviews reflects the lessening importance of teaching as careers advance.  Some faculty members suggest that faculty members would receive more time to develop professionally in this role if the university administrators were serious about improving teaching. “That’s where [UWC] really falls down, trying to do surface fixes rather than really paying attention to what people need to be better teachers in terms of time, energy, and money” (Nemeth, A2tF). The problem at present is that the necessary support is not offered and, as a result, “You just have to make time, so few people do it because it’ll cut into your current teaching quality and preparation or it will cut into your research” (Fradera, A2tF).   78 Having described the institutional context, I now turn to the summative peer review of teaching as it is perceived and practiced within this context.  4.3 Summative Peer Review of Teaching: A Sketch There is variation in how the summative peer review of teaching is understood, implemented, and experienced by individuals across departments within the University. For many, the summative peer review of teaching is synonymous with the classroom observations of teaching; others recognize that it may also include a review of other aspects of teaching, such as an instructor’s course syllabi and assignments. A large number of faculty members understand the summative peer review to consist of both the student evaluations of teaching and the classroom observations of teaching.  According to the participants, the practice of peer review is directed, for the most part, by few or no written guidelines.13 The process is frequently shaped by reviewers’ preferences rather than according to established “rules.” Among the six departments included in the study, only A3 had a written policy on the peer review of teaching at the time of the interviews. Though one might assume that its existence resulted in some consistency in how peer reviews were conducted, this was not the case. Faculty members in A3, like in the other departments, had varying interpretations of how peer reviews of teaching should unfold and vastly different personal experiences of the process.  13     At the time of the interviews, there was a university level report developed by a working group on peer review of teaching. The document, which outlines principles of peer review, had been distributed to department heads by the Office of the Provost and Vice President Academic in November, 2009.  79  Participants comprehend that individual experiences of peer review differ and, consequently, they offered their narratives, not as representative samples of how things work for all individuals within their department, but as their own reflections on their summative peer review experiences. Taken together, the 30 participants’ unique experiences of peer review produce a landscape of peer review analogous to a Sumatran jungle, rather than a Zen garden. To some degree, the process is more unpredictable than it is foreseeable, less developed than desired, and often unplanned.  4.3.1 Changes Over Time The majority of the participants in this study describe the peer review of teaching process as one that has changed over time during their career at the University. Because peer review continues to be in flux— a “work in progress” (Maki, A3aF)—faculty members acknowledge that there is much they do not know about the process. This not knowing includes, but is not limited to, who is selected to do reviews and on what basis, at what career stages peer reviews are conducted, how reviews are conducted, what materials (if any) get considered in the review, and whether there is a departmental policy on the summative peer review of teaching.  Most faculty members are aware, however, that peer reviews of teaching have become more formal over time and cite various factors as evidence of this trend:  80 • Information from peer reviews is used to evaluate teaching in tenure and promotion decisions whereas, in the past, student evaluation of teaching scores were often the only source of data used. • In most departments, there has been an increase in the number of reviews conducted at any given time. Common practice used to be that one reviewer observed one undergraduate classroom session and wrote up a report. Now, in most departments, two reviewers are selected per review and each one is tasked to review an undergraduate and graduate classroom. • Peer reviews of teaching are sometimes considered in making reappointment as well as tenure decisions, whereas the process used to apply only to tenure decisions. • In several departments, the peer review reports are more comprehensive than in the past. Traditionally, they might have incorporated commentaries from the classroom observations. Now they may include an assessment of the faculty member’s teaching materials, a review of his/her statement of teaching philosophy, and a statistical and/or graphical comparison of faculty members' student evaluation of teaching scores. Some faculty members believe these changes reflect an overall institutional shift where the University has begun placing a greater emphasis on teaching.  4.3.2 Timing and Frequency In all the departments included in this study, the peer review of teaching is conducted when assistant professors are being considered for promotion to associate professor. Consequently, peer reviews are typically scheduled so that results are produced in time for the standing committee to evaluate these alongside other data, such as the candidate’s research record and  81 service to the University. In some departments, this is the only time the peer review of teaching is done whereas in others, it is also conducted at reappointment. In S1 and S3, the peer review of teaching is conducted yearly in the pre-tenure years.  Overall, across departments, pre-tenured faculty members are peer reviewed more frequently than their tenured colleagues. None of the new associate professors interviewed in this study have been peer reviewed since attaining that rank and most are not sure whether they will be in the future as part of an ongoing process of evaluation. A number of the tenured faculty members who began their careers in the 1980s or earlier have never been peer reviewed because the process was not used by their department when they were advancing through the professorial ranks and is not currently being used to evaluate full professors.  4.3.3 Selection of Reviewers The department head normally selects reviewers based on several considerations. Across all departments, the department head (or, in some cases, associate head assigned to administer the evaluation of teaching) is typically responsible for selecting reviewers for the peer reviews of teaching. It is the department head, many faculty members think, who has the greatest insight into personality dynamics between faculty members. Therefore, s/he is best able to select reviewers for the classroom observation of teaching and/or decide the composition of the committee that coordinates the peer reviews of teaching. Whether or not the department head has selected the reviewers, faculty members also name the department head as the person they would go to if they had concerns about who had been selected. In most departments, the person who reviews is senior in rank to the candidate but in S1 and A2 tenure track faculty members at all ranks (including instructors) conduct reviews. In some  82 cases, reviewers are selected because their area of content expertise is similar to the candidate’s. In other cases, reviewers are selected because they are available. Several faculty members say they do not think reviewers are chosen on the basis of being a “good teacher,” however a minority thinks the opposite is true. Most of the time, the candidates are told who their reviewers will be and occasionally they are invited to suggest reviewers’ names themselves or given the opportunity to comment on a short list of potential reviewers.  4.3.4 What is Being Reviewed? Participants cite the classroom observations of teaching as the main, and sometimes the only, source of information in the peer review of teaching. As already mentioned, these observations are always of an undergraduate class and, in most departmental processes, a graduate class is also observed. In rare cases, when a faculty member is not teaching during the term when their review is scheduled to take place, a reviewer may observe the candidate teach a lunch-time seminar. When, in addition to a classroom observation, other data are included for review, they may include any of the following: syllabi, assignments, reading lists, exams, a teaching philosophy statement, and a teaching portfolio. Many participants say that it is up to the reviewers to decide what sources of information they will request when they engage in peer review.  83  4.3.5 Communication Pre/Post Review In most departments, there is an unstated understanding that the reviewer shall contact the candidate prior to the review so they can agree on dates for the classroom observation of teaching. In S1, however, the candidate provides a list of suitable dates to a departmental coordinator who then communicates these to the peer review committee; the reviewer, a member of the committee, shows up to the class unannounced. While faculty members in S1 accept this protocol, others in the university find this approach “unthinkable” (Manfred, A2TM). Mutually agreeing upon a date for the observation, Manfred explains, helps ensure that the class being observed is suitable, meaning that there is no exam planned for that day, or movie being shown or guest speaker and so forth. Assistant and new associate professors in departments where the norm is to mutually agree on a date for the observation say they appreciate knowing when to expect a reviewer.  Prior to a classroom observation of teaching, reviewers and candidates rarely meet with one another. A3 provides an exception because the departmental policy requires a meeting, before the first classroom visit, to discuss the candidate’s teaching philosophy, his/her teaching portfolio materials, and specifics of the class, including class and course goals. In addition to A3 faculty members, only Nemeth (A2tF) recalls meeting with the candidate prior to the classroom visit and explains that she did it “more as a courtesy” to help make the candidate feel more comfortable and not because it was a required part of the process.  Although reviewers may be part of a committee, they most frequently conduct the classroom observations on their own. They normally do so without the aid of an instrument or form, except in S1 and A3 where a departmental instrument has been developed and is used.  84 Occasionally, when there are multiple reviewers, the final report is produced collaboratively. Typically, however, each individual reviewer produces a written assessment that s/he later turns in to the head of the peer review committee or department head. Reports may consist of a one-page letter centering on the classroom observation of teaching or, as indicated previously, may include a review of the teaching philosophy statement, graphical/statistical comparisons of peers, and summaries of the student evaluation of teaching scores. Once reviewers complete their classroom observation and/or assess the material (e.g., teaching philosophy, syllabi, assignments, etc.), they may communicate their impressions or share their report with the candidate, but most of the time this step does not occur.  4.4 The Multiple Purposes of Summative Peer Review of Teaching Participants describe the summative peer review of teaching as serving three broad purposes: (a) a formal evaluation procedure conducted to aid in tenure and promotion decisions, (b) a mechanism that contributes to teaching professional growth, and (c) a process to supplement information from the student evaluations of teaching.  4.4.1 Formal Evaluation for Tenure and Promotion The summative peer review of teaching, say the majority of participants, is a formal evaluation conducted to fulfill a requirement for tenure and promotion. Bluntly put, the goal of the evaluation is to determine whether faculty members “should be hired or fired” (Fuentes, A3aM). Summative peer reviews of teaching, thus, are potentially punitive. In several departments, results of peer reviews are used not only to determine how the individual stands “against tenure” (Ludar, A3TM) but also to compare faculty members to  85 one another. In both cases, the evaluation results inform those who are voting on the tenure case and, ideally, the outcomes help administrators build an argument in favour of the faculty member’s career advancement.  Several participants remarked that prior to these critical evaluative episodes, peer reviews of teaching are a way to detect problems that might weaken a person’s future tenure case. When difficulties are identified early in a faculty member’s career, candidates have time to correct them prior to putting forward their files. A candidate’s file is strengthened overall when evaluators see that an individual’s teaching has improved over time, noted some faculty members. Moretti (S3TF) points out that sometimes the very act of making an effort to improve teaching, even if the student evaluation of teaching scores do not increase much, is judged positively by evaluators. As described later in this chapter, some faculty members put less focus on the connection to tenure; for them, the point of identifying problems early on is to offer guidance to instructors and help them develop professionally as teachers.  Faculty members who see the purpose of summative peer review as an essential part of career maintenance and advancement frequently speak about the need to carefully choose their words and language when crafting their reports. They acknowledge being more comfortable documenting constructive suggestions for teaching improvement at reappointment, when the stakes are less high, than at tenure and promotion. Most participants say that constructive suggestions for teaching improvement are not included in the peer evaluation reports for tenure because these might have an “unwarranted negative effect on the person’s circumstance” (Chen, A3TM). Given that these reports are tied to such high-  86 stakes career decisions, faculty members do not want to “pollute” (Hardy, A2tM) their colleagues’ files with comments that might be misconstrued. “It wouldn’t be appropriate to put those things in the summative evaluation because they’re just going to be viewed as negatives when they’re really meant to be helpful, constructive suggestions” (Stromberg, A1TM). When a reviewer thinks a specific candidate deserves tenure, s/he seeks to communicate this clearly. Consequently, for the most part, only positive statements are included in the written report and reviewers who opt to share their constructive feedback usually do so verbally. Parsons (A3TF) explains the need to state unambiguous support for a candidate’s case as follows: You don’t want to put in subtle things that people go, “What’s she really saying there? Does she really mean this?” If you really think the person deserves tenure, you want to say, “I think this person deserves tenure for this, this, and this reason.” Some faculty members, however, specify that if they had serious concerns about a person’s teaching they would note these in a formal report. Documenting poor teaching, according to a few participants, is part of a faculty member’s responsibility toward students. It was because of their colleagues’ strong teaching that they had not formally reported poor teaching, and not because reviewers wanted to intentionally dupe the readers. “It’s not that these letters that are written for the tenure file are filled with lies. They’re not. Most of our colleagues are actually quite good teachers” (Hardy, A2tM). With the exception of three faculty members in S2 who claim that the report on teaching presents a “balanced view” (Hanna, S2tF; Knauer, S2TF; Warr, S2TM), the majority of participants agree that “everything” that goes into a file “is supposed to be good” (Trottier, A2tM).   87 A small number of participants express frustration at the perceived need to frame everything about a person’s case in the positive. That nothing should be open to a negative reading, be it in the summative peer review of teaching reports, the referees’ letters, or the department head’s report, puts a “constraint” (Nemeth, A2tF) on the process and what is communicated therein. The current practice also creates some anxiety among the reviewers who know that “a single negative review can be quite damning” (Abendroth, S1tM). Furthermore, when reviewers feel compelled to include only laudatory comments or to “spin it on the positive side” (Warr, S2TM), it means that the final reports are “highly circumscribed” (Bulmer, A3TM) to the point that they are not useful in any pedagogical sense of improving a persons’ teaching.  Despite the fact that summative peer review is part of the tenure and promotion process, a number of faculty members describe it as a “formality” (Trottier, A2tM), or “technicality” (Hardy, A2tM) that, though required, is largely done in order to have “something to put in the file” (Abendroth, S1tM). Peer reviews, some say, feel like a bureaucratic exercise imposed upon faculty members. Participants with this belief go through the motions but do not think “it really goes anywhere” (Deitz, S1aF). That is, they do not believe the outcomes of peer reviews matter much in decisions about career advancement. One faculty member suspects that many of her colleagues see the peer review as a “waste of time” (Moretti, S3TF). Since faculty members are “all overworked” (Fuentes, A3aM), it is not surprising that some wonder whether the peer review of teaching adds unnecessarily to their workload. The uncertainty comes about because participants cannot readily identify what benefits the process produces and/or because they believe the benefits are too small to justify the amount  88 of time peer reviews consume. Among those who perceive that peer review of teaching is only paying “lip service” (Nemeth, A2tF; Maki, A3aF) to the importance of teaching within the academy—and not truly contributing to the improvement of learning—there is a sense that peer review makes it “look like we value teaching [but] it only exacerbates the problem in some ways because it’s another administrative burden that takes away from our time that we have in the classroom” (Nemeth, A2tF). Others participants, however, say that they have not heard their colleagues express cynicism about the process and report that, on the contrary, most people take it “very seriously” (Woodroof, S2TM).  4.4.2 Professional Growth in Teaching Several faculty members speak about peer review as a process that contributes to teaching improvement and enhanced student learning. It can do so by promoting reflection on teaching, by providing a learning opportunity for the observer, by encouraging faculty member accountability, and by identifying the need for additional support.  Firstly, when candidates receive feedback from their colleagues, this can help them reflect on their teaching. Reviewers may see things that are in a faculty member’s “blind spots” (Maki, A3aF) and their comments can suggest alternative approaches to teaching. Reviewers’ observations may prompt the teacher to re-examine what happened during the class and to consider how to apply the feedback to other classes and teaching situations. “I look on the peer review as being an opportunity for an outsider to suggest things that I might change to make my lectures better” (Abendroth,S1tM). Furthermore, as a few faculty members note,  89 having written—not only oral—feedback is helpful because comments can be revisited over time.  Secondly the peer reviews are connected to professional growth because the reviewer learns when s/he observes a colleagues’ classroom teaching. The act of observing and assessing the instructor stimulates the reviewer to think about his/her own teaching. “One of the things I was struck by is that I got some good ideas for my own teaching from sitting in a colleague’s grad seminar” (Parsons, A3TF). Some reviewers have follow-up conversations with those being reviewed to ask for additional details on a teaching-related activity or approach they observed and want to implement. Faculty members acknowledge that, aside from these occasions, they rarely see their colleagues teach and describe the learning that comes from that as enjoyable and also “considerable” (Ludar, A3TM).  Thirdly, peer reviews encourage faculty members to be accountable to their students because academics are aware that, as part of the process, the reviewer may be evaluating whether the course is aligned with the stated goals outlined in the syllabus and the short description in the academic calendar. Knowing that someone will be checking for alignment, faculty members may be more attentive to ensuring it exists. In addition, when a problem is detected during a review, colleagues “feel an obligation to the students, to everybody that that problem not be allowed to continue” (Quinn, S1TM). Thus, peer reviews promote accountability to students because, when faculty members work to enhance their teaching, the ultimate benefactors are the students.   90 Finally, peer reviews of teaching provide an opportunity to identify whether there is need for additional support and/or professional development among academics whose teaching may be “stale” (Ludar, A3TM). In such cases, the department head can help faculty members determine which opportunities might assist them to expand their background and skills and otherwise address their needs.  4.4.3 Supplement the Student Evaluations of Teaching According to many faculty members, content in the peer review of teaching report aids interpret the student evaluations of teaching results. Comments from peer reviewers help evaluators understand the student scores and may summarize the factors which make a candidate a good teacher or not: They often put words to numbers. Yeah, someone gets a 4.2. How did they get their 4.2? Well, if you read the comments you’ll find they’re very enthusiastic and they really got us excited about stuff or perhaps they were incredibly organized and very clear in what students needed to learn. (Warr, S2TM) Information from peer reviews supplements that from student evaluations and does not normally contradict it. However, when student evaluation scores fall below the acceptable department range, some faculty members suggest that peer reviews can provide additional information that might challenge (or at least explain) low numerical scores. Deitz (S1aF) believes a positive peer review can “offset” a problematic student evaluation (that is, one with low scores). Participants’ accounts indicate that, as long as the student evaluation of teaching scores fall within a range deemed appropriate by the department and by the individual’s own standards, the results are generally accepted. Peer reviews, it would seem,  91 typically play a secondary role to the student evaluations of teaching.  For a minority of faculty members, in particular those who are skeptical of the student evaluation of teaching scores and think the instrument is inadequate, the peer review process is a “much better resource for evaluating faculty teaching” (Cohen, A1aM) than the student evaluation of teaching instrument. Since student ratings and comments can suggest problems where they do not exist, peer reviews of teaching can be corrective of these results. “I think the peer review process is essential, partly because I think that the other information we have about teaching is really inadequate,” asserts Nemeth (A2tF). Thus, according to some faculty members, peer reviews do more than supplement existing student evaluation of teaching scores: they provide a necessary distinct perspective about the instructor’s teaching.  In addition to providing additional insight into the existing student evaluation of teaching scores, or providing a unique perspective on teaching altogether, several faculty members praised reviewers’ comments for being more “precise” (Ledonne, S1tF) and/or “objective” (Dodgson, S2aF) than those from the student evaluations of teaching, which, according to many participants, are left by disgruntled students. Comparing the comments she receives from peer reviewers to those from students, Ledonne says “it’s more bang for the buck, more constructive, more useful” because faculty members, as compared to students, are more articulate in making suggestions for improvement and better able to pinpoint demonstrated qualities of good teaching. Others say the peer reviews are more valid because a faculty member, given his/her teaching experience, is a better judge of whether the candidate is a  92 skillful teacher. This presumes that the faculty member providing the review is, him- or her- self, a capable teacher.  4.5  No Teaching-Related Professional Growth Though some faculty members claim that summative peer review of teaching can foster professional growth in teaching, the majority thinks it does	
  not. Their reasoning falls into any of these three broad categories: feedback on teaching is non-existent or poor, the student evaluations of teaching carry more weight in the evaluation of teaching, and/or the peer review of teaching process is artificial. In this section, I elaborate upon each of these, focusing principally on the lack of quality feedback.  4.5.1 Feedback: Non-existent or Unspecific One of the reasons that summative peer review of teaching does not lead to professional growth in teaching is because the reviewer’s feedback is either non-existent or lacking in specificity. Whether or not the reviewer shares his/her feedback with the person under review, and how that feedback is crafted, depends largely on the individual reviewer. In S1, faculty members who have been reviewed multiple times on their teaching prior to tenure have come to expect that each individual reviewer will decide, or not, to debrief the classroom observation of teaching. “The more conscientious ones do, or the ones that don’t have a meeting right afterwards or something” (Ledonne, S1tF). In other departments, assistant and associate professors who have been reviewed only once are less sure of how the process unfolds as compared to their colleagues who have undergone multiple peer reviews of teaching. Full professors in all departments, however, are aware that there is a lot of  93 variation in how peer reviews are conducted. In the quote that follows, Taylor (S3TM) aptly summarizes common approaches to feedback in peer review across both faculties when he says: “there’s no policy that it [giving feedback] should be done, but no one believes it shouldn’t be done.” Though individuals express a preference for sharing feedback, their stories indicate that it is an infrequent practice.  Sometimes a reviewer shares brief, non-specific feedback with the candidate immediately after a classroom observation of teaching. General statements, such as “it looked great” or “good class,” are standard. That could be the extent of feedback a candidate receives from a peer review. Though statements such as these lack in substance they, at least, let the candidate know that the reviewer perceived what happened positively, note a few participants. Other times, reviewers meet with the person being reviewed to discuss feedback on the classroom observation of teaching. Among the participants who had received verbal feedback in a post-observation meeting, most indicate that it did not contribute meaningfully to their professional growth as teachers. Some say they learned a little bit from the feedback whereas others state the feedback was “largely useless” (Deitz, S1aF). For example, it is common for a candidate to be told, in general terms, that “there are no concerns, things went fine” (Maki, A3aF). Or, they might be informed that they have a distracting habit, such as fiddling with a water bottle and turning their back to the class. A candidate might also be told that students were checking emails or falling asleep in class. This type of “three-sentence conversation,” (Fuentes, A3aM), in which feedback is nonspecific or focused on the “mechanistic” (Hadzik, S2tM) aspects of teaching, is “not particularly helpful” (Dawson, S3aM) for people who wish to improve their teaching, said several participants. Like the  94 verbal feedback, the quality of the written feedback is variable. However, only a small number of individuals spoke about the quality of the written feedback and I suspect that is because most faculty members do not see it, other than those in S1 who receive the completed forms from colleagues who have conducted the classroom observations of teaching.  Some faculty members, including Warren (A3aF) expressed disappointment at the lack of feedback: I was disappointed not to have gotten more feedback on it because it could have been a mentoring opportunity as well. The two people who are doing my peer evaluation are both good teachers and it would have been interesting to have gotten more feedback from them. During the interviews, three female faculty members described their experiences of attempting to get additional feedback from their reviewer and in each case the result was a letdown. Maki (A3aF) recalls asking for clarification and further advice on a minor, constructive comment she received and being told “That’s just how students are. . . . There is not much you can do about that” —a response that fell short of her expectations. Deitz (S1aF), who says that getting verbal feedback from a reviewer “is not standard,” explains that one has to “chase people down” after a classroom observation of teaching in order to get information. She recounts her experience of doing that, and the outcome, as follows: I did that the first couple of times just because I wanted—you know, it’s kind of nerve-racking—and I wanted to see what [the feedback] was. When I discovered that there wasn’t going to be anything useful, then I stopped chasing people down.  95 Similarly, Wilks (S1tF) describes following up with her reviewer on her own initiative, and asking if he could meet with her “because I’m always looking for ways to improve my teaching” in a way that is not “super time-intensive.” Though he initially indicated a willingness to meet with her, and they had some back-and-forth email about a prospective meeting, that meeting never happened. In the end, she concludes: I felt like it was a bit of an imposition because he never came back. So I wonder if it’s just more time for the person actually doing the review. I have no idea what kind of time commitment they’re being told that this is for them and maybe it’s kind of like, “Look I’ve done my thing” and that’s it. That was sort of the sense I had with the second one so I didn’t push it. Both Deitz and Wilks are careful to point out that their respective reviewer is a “good guy” and they do not attribute the absence of a meeting and feedback to a lack of collegiality in their relationship. However, as Deitz points out, her experience has discouraged her from attempting to get additional feedback from reviewers. Although Wilks does not mention whether she will continue to pursue additional opportunities for feedback, one might expect that she will not if she repeatedly ends up being disappointed.  4.5.1.1 Feedback from the Department Head Just as participants note that they rarely receive feedback from those who have reviewed their classroom teaching, they indicate that post-peer review the department head rarely discusses the report with the candidate. When outcomes of the peer review of teaching process are brought up, it is typically in the context of an annual meeting between a pre-tenured faculty member and the department head during which the faculty member’s overall career progress  96 (towards tenure) is discussed. Ludar (A3TM), the only department head among the participants, sees the peer review process as something that either strengthens or weakens a candidate’s career progress. I receive this [peer review report] and this is part of the tenure process and I discuss if it’s good, bad or indifferent, and point out ideally that it is a good report and strengthens their case and how it relates to the student questionnaires and the other mechanisms that we have. (Ludar, A3TM) The point of reviewing teaching yearly with pre-tenured faculty members, he elaborates later in the interview, is to warn people early about poor teaching and to recommend concrete help to a faculty member who needs it. These two points are echoed by others and indicate that the evaluation of teaching matters predominantly as it pertains to tenure and promotion. However, Ludar notes that peer review of teaching is not exclusively linked to decisions about career advancement, but also about “ensuring that our department remains populated by dedicated people for whom the teaching of students matters.” Most faculty members, though they recognize that peer review might contribute to improved teaching and student learning, say that discussions with their department head about peer review have been non- existent or have focused on how it might relate to their career advancement. Deitz (S2aF) doubts that her head even looks at the classroom observation of teaching if a faculty member’s student evaluation of teaching are “good enough.” When peer reviews are dismissed as irrelevant, it strengthens faculty members’ beliefs that student evaluation of teaching scores are of prime importance in the evaluation of teaching.   97 Faculty members whose department heads had acknowledged peer reviews say the feedback from their head was minimal and not helpful for teaching improvement. Maki (A3aF) remembers: “I talked to my head and I said, ‘Can I get any feedback about the review?’ and he said ‘All went well.’ ‘Is there any formal feedback?’ [I asked] ‘No’ [he answered].” Another pre-tenured female faculty recalls the A3 department head reading aloud the report and inviting her response. She described the interaction as awkward, having not had an opportunity to see the report prior to the meeting nor debrief the contents with her reviewers. She recalled being disappointed by some of the feedback but felt inhibited from voicing her reactions because of the hierarchical relationship in place. These stories demonstrate that discussions about peer reviews are not only restricted by the low quality of the feedback, but also by discomfort that arises due to power imbalances between the faculty member and the department head.  4.5.1.2 Why No Feedback? Why is quality feedback not routinely provided to faculty members as part of the peer review process? Although I did not pose this question in the interviews, faculty members ventured explanations which I have categorized in two ways: one, at tenure and promotion, reviewers are focused on evaluation, not on professional growth; and two, because the evaluation of teaching is difficult, reviewers do not know how to generate useful feedback.  4.5.1.2.1 Focus on Evaluation According to many faculty members, at tenure and promotion, peer reviewers are focused on evaluating the candidate, rather than on envisioning how they might help the candidate  98 become a better instructor. In their role as evaluators, some reviewers believe they must keep themselves at “arm’s length” (Woodroof, S1TM) from the candidate; one way of doing so is by not sharing feedback. That judgmental approach can be threatening, and is not conducive to professional growth. “It’s about figuring out how good they are. Are they good enough for what we require of them?” (Nemeth, A2tF). In these cases, the peer review serves as a mechanism to determine whether the candidate is meeting a general threshold that the evaluator judges adequate. Fuentes (A3aM) calls it a “sort of check on ‘Is this person actually trying?’” Others add that the reviewers “probably perceive their role as just looking at things to make sure everything’s okay” (Dawson, S3aM). Finally, some suspect that the reviewers may not see giving feedback as part of their obligation: “As long as you file your report reasonably on time, I think you’ve fulfilled your duties” (Quinn, S1TM).  In A3, where there is an established policy on the peer review of teaching, some faculty members explain the lack of feedback by referring to their departmental policy that, according to them, states that the process is evaluative and is not a mentorship opportunity. Accordingly, an evaluator cannot simultaneously be a mentor and good departmental representative. Thus, in order to fulfill the role of departmental assessor-agent, it is best to separate mentorship from the summative peer review process.  Most faculty members agree that when the summative peer review is being used as a tool for evaluation and to foster professional growth in teaching, the two purposes collide and, typically, neither goal is well accomplished.  99 So, there are two functions of peer review, as I see it. One is a kind of collaborative aspect where you’re trying to help your colleagues get better, solve problems. Let’s just work together, we’re on a team, we are trying to improve all of our teaching, so let’s pool resources to do that, share best practices, problem solved. . . . So there’s that aspect and there’s nothing judgmental about that: it’s process improvement. But then there’s the evaluation side, for marks, and now your peers are required pass judgment. And those two functions are in conflict with each other. (Palo, S1TF) Many faculty members are also vexed by the dual function of peer review. The frustration comes about, in part, because faculty members consider their peers to be both the best individuals to collaborate with to improve their teaching and the most suitable evaluators of their teaching.  4.5.1.2.2 The Evaluation of Teaching is Difficult Another reason quality feedback is not routinely provided relates to the numerous challenges involved with the evaluation of teaching, a topic repeatedly mentioned by the participants. Teaching is difficult to evaluate, the participants explained, because • there are few or no explicit criteria for doing so; • faculty members do not share a common notion of good teaching; • academics may be lacking in pedagogical knowledge and/or competency; and • peer reviews occur infrequently and are based on very little information.  100  4.5.1.2.2.1  Few (or no) Criteria Faculty members see nonspecific criteria as a challenge to the evaluation of teaching. Teaching, they say, is typically evaluated against an unstated standard. Those reviewed feel uncertain about what they are being evaluated on, while reviewers admit they do not necessarily know how to evaluate teaching. This sentiment is shared both by people in departments where a form with criteria is used (A3 and S1), as well as in departments where no such form exists. Several faculty members point out that identifying poor teaching may be easier than identifying the opposite. What we operate by, and this is regrettable, is that we will know when the teaching is not up to par. But we really don’t know when it is up to standard because I don’t think we have thought enough about it. (Ludar, A3TM) Perhaps this is why, in the opinion of some participants, the peer review of teaching serves only to assure colleagues that nothing “egregiously awful” (Palo, S1TF) is going on. Thus, in the absence of standards, the peer review of teaching process aims not to celebrate good teaching, build upon strengths, or even improve weak teaching skills, but only to ensure there are no catastrophes. “I think most of the time, you’re in there to see, ‘Okay, no train wreck’” (Quinn, S1TM). Whereas, in the section above entitled “Focus on evaluation, ” I reported that some faculty members suggest peer reviews act as a “spot check,” to ensure things are “okay,” some findings indicate that the aims of peer review are, indeed, not even that lofty.  101  4.5.1.2.2.2  Good Teaching: No Consistent Opinion Related to the previous point is the fact that there is no consistent opinion among faculty members about what constitutes good teaching. Consequently, faculty members may not be sure how to evaluate teaching that is markedly different from their own. Some describe having to review a colleague as “awkward,” especially when that person is taking an approach unfamiliar to their own. “There’s a reluctance to be overly critical if you see something that you wouldn’t do, because as a reviewer you wonder: ‘Does it work? Does it not work?’ It’s like, ‘I don’t know’” (Quinn, S1TM).  4.5.1.2.2.3  Lack of Pedagogical Expertise A few faculty members state that they lack the pedagogical expertise to suitably judge their colleagues’ teaching and, therefore, consider themselves unqualified for this task. “Yeah, we’re asked to assess teaching, we just do the best we can and from our gut feeling. Well, you shouldn’t be assessing someone’s teaching for tenure from a gut feeling” (Moretti, S3TF). Hadzik (S3tM) is one such person who clearly believes that not having pedagogical expertise is an impediment to his ability to conduct a useful evaluation. As a reviewer, he feels he can only comment on whether students are paying attention and listening, but not on whether they are learning.  A number of participants also remark that they were reviewed by people who are not skillful teachers and reason that someone who is not a good teacher cannot evaluate teaching well. They surmise that one of the reasons feedback on teaching remains focused on the “mechanics” (e.g. neatness of writing, voice level, whether students are awake, etc.) is  102 because faculty members do not possess the pedagogical skills and knowledge to evaluate teaching well. When this is the case, conversations about teaching lack depth and do not address issues of significance. “All these conversations are skirting teaching. . . . It’s like telling someone how to run a company by showing them the photocopier” (Hadzik, S3tM). From the perspective of people committed to improving teaching, feedback focused on the surface qualities of teaching does not adequately fulfill their need for professional growth.  4.5.1.2.2.4  Little to go on Another reason why evaluating teaching is difficult and quality feedback is not routinely provided is that reviewers make judgments about a faculty member’s teaching based on very limited information due in part to the infrequent nature of peer reviews. Unless faculty members are reviewed yearly and post-tenure, a faculty member may only undergo a peer review twice in his/her career: at reappointment and tenure. Furthermore, when peer reviews do occur, they often centre on a small number of classroom observations of teaching.  A candidate’s teaching materials may be considered but these do not normally “do a lot of heavy lifting” (Chen, A3TM) when it comes to the evaluation of teaching. In fact, only two faculty members (Manfred, A2TM; Palo, S2TF) mentioned the benefit of reviewing these at all and their comments were specifically about the usefulness of the teaching philosophy statement as a way to learn about a faculty member’s innovative teaching practices and overall approaches to teaching. Reviewers in the study do not observe laboratories and tutorials, nor do they speak with the faculty members’ advisees or teaching assistants to garner information from a variety of sources about teaching. Combined, this means that  103 reviewers have little information to go on. Several faculty members recognize that, even though classroom teaching is an important activity, there are numerous other practices included under the category of “teaching” that are not included in the review. However, only a small number of faculty members state that they would like peer review to include a broader range of teaching activities.  4.5.2 Student Evaluations of Teaching Set the Standards Participants unanimously agree that for individual faculty members as well as for administrators, student evaluations of teaching remain the most important piece of information for evaluating teaching, even though they perceive that the instrument is flawed and results are biased and not necessarily reliable. Consequently, peer reviews are conducted primarily to fulfill an administrative requirement and not with the intention of fostering professional growth in teaching. Colleagues may use a peer review to have a discussion about teaching, but this occurs infrequently. As I indicated earlier in this chapter, faculty members say they frequently use information from the reviews to help interpret and/or otherwise supplement the student evaluation of teaching scores. In the opinion of most participants, the student evaluations of teaching are taken “very seriously” (Hardy, A2tM) and remain the primary means by which faculty members gage their teaching success, or lack of it.  According to the participants, the reasons the student evaluation of teaching results generally predominate over the peer reviews are threefold. Firstly, results from this instrument are presented as numerical scores and, since faculty members believe that administrators care  104 about numbers, they therefore believe that the results of this instrument are useful to those whose opinions count in tenure and promotion decisions. The numerical scores can also be applied to graphs and statistics to compare faculty members against one another and/or a departmental average. Numbers, several faculty members opine, are quick and easy to read and interpret as compared to the qualitative comments in summative peer review reports. In S1 and S2, a candidate’s tenure file contains graphs and statistical information that plots scores for each course a faculty member teaches against the same course taught by other people and/or other courses at the same level (i.e., 100-level, 200-level, etc.). The ability to compare oneself against a standard (peers, in this case) is referred to as a positive by faculty members in Science. Abendroth (S1tM) recalls being frustrated that he was not told, for the first four years of his appointment, that his student evaluation of teaching were below the department average, especially given that all that information was readily available and statistically compiled. As someone who was, at that time, approaching tenure, he recalls wanting to know where he stood. In S3, Moretti (TF), who has been part of the Dean’s advisory committee for promotion and tenure, speaks with enthusiasm about the plots and graphs in S1, indicating that it would be a useful practice to adopt faculty-wide as it is more objective than the current (non-desirable) subjective ways of presenting information about a candidate’s teaching scores. Striving for objectivity, in peer review or the evaluation of teaching more broadly, is fitting and beneficial according to many faculty members in Science.  It is worth noting that in the Arts departments, the practice of comparing peers does not involve the use of detailed statistics or graphs. In A1, Cohen (aM) relays that the head  105 normally announces, annually at a department meeting, what the average student evaluation of teaching score is faculty-wide. Faculty members can then compare their scores to that average, if they want and in private. In A2, there had been an occasional practice of circulating a summary of scores ranked from highest to lowest. That, explained Nemeth (A2tF), “tended to be badly received as divisive and humiliating for the people who are at the bottom and not particularly helpful,” and therefore is now done more informally, and in private, by the department head when a faculty member’s scores drop below a number deemed acceptable in that unit. In A3, faculty members receive an average score for the 100, 200, 300 and 400-level courses (i.e., the average of all the 400-level courses collectively that term, categorized by full-time faculty and sessional faculty) and they also receive the grade distribution for each course. 	
   A second reason the student evaluations of teaching are considered more important than peer reviews is that most faculty members have a strong sense of what a good numerical score is within the context of their department and, consequently, self-assess using that knowledge. For instance, Fradera (A3tF) describes how receiving information about the departmental student evaluation of teaching scores triggered a sense of competition within her: There is a little bit of competition in the teaching evaluation scores. Every year we get a summary and you can see by that if you’re not up to snuff. The average is 4.5 or higher [on a 5 point scale] and there are a few people who get perfect 5s. Yeah, so there’s sort of intense feeling of peer competition [laughs]. I remember a few times I had 4.2 and I was just destroyed by that. And, some people teach really hard unpopular courses-there’s a bias against them getting high scores—and they get  106 scores of 3 point something. And it must be really hard on those people to get those scores. She goes on to surmise that, although there are no strong departmental initiatives “to do an outstanding, innovative job of teaching,” the peer competition might motivate some faculty members to continuously work hard at their teaching. A small number of other women relayed that they had felt very anxious, even obsessed, when their scores had dropped below a number they considered acceptable. They drew on several resources, such as consulting with colleagues for help and modifying their course, in order to address the source of the problem. In these cases, the student evaluations of teaching scores were a mechanism for self-regulation (and self-inflicted badgering).  Several faculty members cited a precise score (e.g., 4.2 or 3.5) above which nobody in the standing committee would be concerned about a colleague’s teaching. Little discussion of teaching ensues when a candidate has consistently high scores; in these cases, peers seem to agree on a “slam dunk for teaching” (Wilks, S1tF). Good teaching thus requires little discussion in tenure and promotion decisions. Similarly, others speak about an acceptable score, or “magic number” (Ledonne, S1tF), as one that does not alert the “SWAT team…[to] come out [and declare] ‘What are we going to do? This is going to be a problem for tenure’” (Ledonne, S2tF). Individual faculty members have definite ideas about what constitutes a good score within their department, and monitor their results and those of their colleagues’ accordingly. Knowing how one fares against peers matters to many faculty members but not all. In a department where scores are typically high, Chen (A3TM) figures faculty members probably do not care if they are a few points below or above the (high) average. Similarly,  107 Dodgson (S2aF) says she does not consider how she compares to peers as important. But these voices are the outliers in a majority who affirm the importance of student evaluation of teaching scores to assess how they are faring against the norm.  A final reason the student evaluation of teaching scores are considered more important than the peer reviews is that the results are based on the opinion of students who have participated in the whole course, whereas a faculty member has typically seen only one teaching episode.  4.5.3 Artificial A third reason why faculty members do not associate the summative peer review with teaching professional growth is that they view it as an “artificial” situation. Several mentioned that the presence of a reviewer renders the teaching situation unnatural for both the instructor and the students. For the instructor, it alters his/her level of nervousness, which thereby alters teaching. “The very presence of someone watching you teach in some subtle way will affect how you teach” (Manfred, A2TM). A few faculty members whose respective reviewers had come to a small graduate class recall that students had been reluctant to participate at first, even though they had been informed in advance about the fact that a colleague would be present in the classroom. Having someone new in the room made both the students and the instructor more self-conscious. These participants suppose that the presence of the reviewer changes the dynamics that everyone has become accustomed to and, consequently, students are less comfortable contributing to the classroom discussions. Faculty members and others agree that having a reviewer attend an undergraduate class is  108 much less intrusive because a visitor is less likely to be noticed among a large number of students.  4.6 Ideal Process In this section, I draw from the interviews to report on features that would improve the peer review of teaching and/or constitute characteristics of an ideal process. I asked each participant what an ideal peer review of teaching process might entail and, not surprisingly, they had varied responses (see Appendix H for a summary of the range of participant responses). In retrospect, I realize that I should have been more specific in framing the question and also asked in what ways, if at all, an ideal summative and formative peer review of teaching might differ from one another. Even though I did not make the distinction between summative and formative peer review in posing the question, a large number of faculty members pointed out that, ideally, the summative peer review of teaching would not be used for both developmental and evaluative purposes because these goals conflict. When one attempts to use summative peer review to achieve both goals, neither one is well achieved; consequently, affirmed many of the participants, the two goals need to be kept separate if peer review is to serve either function well. “If you’re going to have a reflection on the possibility of growth as a teacher, it has to be part of some other process than tenure and reappointment” (Nemeth, A2tF).  4.6.1 Flexibility/Structure Many participants note the absence of “rules” in peer review. Some celebrate this absence saying that faculty members would perceive strict requirements as an imposition and would  109 therefore resist them. Other faculty members interpret the absence of regulations as something status quo and a regular part of academia. Trottier (A2tM) speaks of his ignorance of the rules in departmental peer review as follows: But I may not be aware of some rules because many of these rules actually are not written [laughs]. At some point, someone comes and says “as part of this department we do such and such.” . . . [and so] we do what the procedure requires us to do. Some faculty members readily accept the flexible approach their department takes to summative peer reviews. They appreciate the freedom they have to conduct the process as they see best and do not want an “across the board” (Stromberg, A1TM) policy or instrument imposed upon them. Peer reviews, some faculty members recognize, are inherently subjective. Others would favour more consistency in the peer review process and believe this could happen if the process were more objective. Faculty members indicate that this could be brought about in a number of ways, the first of which is by articulating the departmental criteria for what constitutes good teaching. The criteria would need to be communicated to the person being reviewed and reviewers so that both parties had clear expectations of what they would be evaluated on and evaluating, respectively. In departments where an instrument with criteria is not routinely used, some faculty members propose that using one could be helpful. They suggest that peers collaboratively develop the instrument based upon common notions of good teaching and that it then be shared with candidates well ahead of time. A second and related way by which more consistency could be brought to a departmental peer review process would be to “share the ground rules for evaluation” (Wilks, S1tF). The ground rules here refer, not only to the specific criteria for evaluating good teaching, but to any other guidelines and instructions such as: (a) at what level of detail will reviewers  110 examine the various criteria?, (b) What can those reviewed expect in terms of feedback?, and (c) How will the review factor into tenure and promotion decisions? A third way that the summative peer review of teaching process might be made more consistent and objective is if it were conducted by a faculty-wide team of trained reviewers, proposes Moretti (S3TF). These individuals would be called upon to carry out peer reviews for the entire faculty. This would help remove bias and bring objectivity to a process, which according to some, is currently too subjective.  4.6.2 Ongoing, Informal Process Many faculty members suggest that it would be ideal for classroom observations of teaching to be done regularly, for everyone, and not just in critical decision periods. When peer reviews are conducted informally—that is, outside of tenure and promotion decisions— reviewers feel more free to give constructive feedback on teaching because there is a lower risk that doing so can have negative repercussions on a faculty member’s career. An ongoing, informal process is more conducive to professional growth in teaching.  Participants mentioned several ways in which informal peer reviews might be carried out. For example, there may be a departmental agreement whereby faculty members peer review one another annually and all that they are held accountable to is sharing feedback and then notifying the department head; no documents would submitted as part of this process. Alternatively, three or four colleagues, from the same department or different ones, might observe one another’s classes and meet once or twice a year to discuss these over lunch. In a third type of formative peer review, pre-tenure faculty members could review one another as  111 this would assure that the focus remain on teaching professional growth and be completely separate from career advancement in the formal sense of the word. Finally, staff from the University-wide teaching and learning centre could be invited to be a part of the informal process. Any of these approaches, or others not mentioned, promotes peers to learn from observing one another’s classroom teaching.  Although many faculty members agree that routine formative peer reviews would be a good thing, some acknowledge it may not occur unless departmental peers collectively agree this practice should become a departmental policy or a university requirement. One reason is that informal peer reviews place additional demands on faculty members’ time. Even though they recognize that peer reviews would be desirable for promoting teaching professional growth and that they would enjoy this type of learning, several participants believe the investment of time would not be worthwhile. Peer observation is a time consuming process that takes valuable time away from other activities. Everyone at this university is overstretched and asked to do too much. And I suspect that while on some level you would get more accurate data if arrangements could be made to sit in on ten lectures instead of two, the amount of time that that would consume and the amount of organizational effort and inconvenience involved would not make it worth it. (Manfred, A2TM) Another reason informal peer reviews might not occur unless they were mandated is that, in the current academic culture, the classroom is frequently perceived as an instructor’s private space and some faculty members, say Trottier (A2tM) and Parsons (A3TF), are reluctant to ask their peers to be invited into that space. “There’s a certain reluctance in academic culture.  112 . . . If I go to my colleague and ask for permission ‘May I sit in?’, I don’t expect a ‘No.’ Certainly the usual answer would be ‘yes,’ but I think there would still be some feeling of discomfort.” Even, as several faculty members indicated, if they are certain their colleagues would accept a peer in their classroom, they hesitate to do so knowing that it would create uneasiness for both the reviewer and the reviewed.  4.6.3 Feedback Quality and Sharing Whether they are observed regularly or infrequently, for summative or formative purposes, faculty members want to engage in conversations about the review with their reviewers. They want follow up to be an explicit part of the process, “rather than something that just happens if they [the reviewers] happen to be conscientious and have time” (Ledonne, S1tF). According to the participants, debriefing the peer review experience, preferably shortly after it has occurred when things are still fresh in the minds of the reviewer and candidate, has three distinct advantages: (a) it lets the person reviewed, who might be nervous about the reviewer’s opinion, know the latter’s impressions; (b) it augments the possibility that the review contributes to enhanced teaching; and, (c) it provides candidates with an opportunity to ask their reviewers questions prior to meeting with their department head.  Those who have had conversations with their reviewers say these are most rewarding and conducive to professional growth when both parties approach the peer review as a learning opportunity. When this is the case, the follow up conversations are carried out in a spirit of reciprocity: those reviewed reflect on their teaching and reviewers share what they learned from the experience. The persons being reviewed prefer feedback that is specific, but that can  113 also apply to teaching situations outside the very classroom that was observed. They also want to have critical feedback, from a knowledgeable reviewer, as they believe this will contribute most to their professional growth. In the following quote, Dawson (S3aM) explains why, for him, this is so important: …because at the end of the day hearing that this looks great is fulfilling, but it doesn’t help you. You’re only helped if you know what it is that you’re not doing right. It’s just like my students. The most important thing for them to know is what they don’t know. Yeah, it’s great to have a pat on the back, but I think learning requires that you confront the things that you don’t understand, or the things that you don’t do well. Dawson, who had completely redesigned one of his courses, goes on to describe how, even though he received some helpful comments from his reviewers, he did not feel that most of them had sufficient knowledge or experience to critically comment on the non-traditional approach to teaching he had adopted and, consequently, he was not able to learn “what is the most important thing I’m not doing.” Although knowing what one does well is also useful, those being reviewed say they appreciate receiving detailed feedback with suggestions for improvement. This aligns well with what some reviewers say they want, which is to provide feedback that is encouraging and constructive.  4.6.4 Reviewer-Reviewed Connection Faculty members have varying opinions about what constitutes an ideal reviewer and comment on qualities that included disciplinary connection, pedagogical knowledge, philosophies of teaching, gender, and rank.   114 4.6.4.1 Disciplinary Connection Faculty members agree that both someone from outside their field as well as a colleague with whom they share a disciplinary connection can contribute meaningfully to peer review. An outsider can comment on many aspects of teaching, such as whether the students are engaged, or not, or whether an instructor is paying more attention to some students than to others. A content expert, however, can better assess whether “concepts are being passed effectively” (Knauer, S2TF). Whereas some say that the value of a peer review is “zero” (Dodgson, S2aF) when their reviewer has no content knowledge, others think it is only important in evaluating a graduate course. Ludar (A3TM) agrees with people who think that there should be some effort to have someone with content expertise and says: “I think it’s preferable to have a person with some deeper competence in the field and it’s also comforting to the professor who increasingly is pretty specialized.” A number of participants believe it would impose an undue burden on those coordinating peer reviews if they had to take into account matching people with a close disciplinary connection.  Wilks (S1tF) raises another point about the reviewer-reviewed connection when she observes that the two people should not be too “intertwined.” In other words, she is not in favour of having her evaluator be someone with whom she collaborates on teaching because that would remove the possibility of an objective review. Numerous Science professors named objectivity as a value to strive for in peer reviews. Related to the goal of objectivity, Hadzik (S3tM) suggests that people from other departments would be ideal reviewers because,  115 lacking content knowledge, they could be “real learners” who could actually comment on what they had learned in the class.14 Certain faculty members propose that peer reviews could be more objective if one were to adopt the following practices to evaluate teaching, some of which were previously mentioned: select expert-educators who can base their judgments on the literature and sound research, use a faculty-wide team of trained reviewers to avoid bias, measure student learning to determine instructor effectiveness, and use pre- established criteria and standards that are known to both the reviewer and person reviewed.  4.6.4.2 Pedagogical Knowledge Several faculty members were explicit about wanting to be reviewed by someone with pedagogical knowledge and mention, for instance, that they value the pedagogical expertise and perspectives of tenure track instructors in their department. Hadzik (S3tM) uses the term “expert educator” to describe his ideal reviewer. The expert researches teaching and learning and can back up her/his critiques and suggestions with data and literature that is “objective.” The ability to do so “counts for a scientist,” he points out. Others state their preference in terms of having a reviewer who has a strong reputation as a teacher and/or is a competent teacher but do not specify on what basis this reputation and/or competency is established.  . 14     In Hadzik’s opinion, the evaluation of teaching would ideally focus on measuring student learning. Since a reviewer attending a single classroom observation of teaching cannot measure student learning, Hadzik proposes that someone who is unfamiliar with the course content can more aptly address the extent to which they, themselves, learned something from the class than someone who knows the content.  116  4.6.4.3 Philosophies of Teaching Some believe that the reviewer and reviewed should share compatible philosophies of teaching. This would allow reviewers to comment knowledgeably about the candidate’s approach to teaching and would be conducive to meaningful follow up. In the context of summative peer review of teaching, matching people based on similar philosophies is of critical importance and, as Nemeth explains (A2tF), not doing so can have a negative consequence: The difficulty of any kind of review process is that a huge amount depends on how compatible the two peoples’ teaching philosophies are. There are people who I might think of as very good teachers who other people in the department might think are doing something that’s really quite inappropriate. So the decision about who does the peer review is absolutely critical in terms of the kind of report that you’re likely to get. Sharing a philosophy of teaching can promote engaged discussion between the reviewer and the person reviewed; the absence of a common philosophy may, on the other hand, lead to a negative evaluation of teaching.  4.6.4.4 Gender When asked whether they thought that gender plays into the peer review of teaching experience or outcomes, the majority of participants said it does not, but some had opinions that contradicted this overarching belief. Had I not posed the question, I intuit that no faculty member would have raised the point of gender on his/her own.   117 According to male faculty members in Science, reviewers do not make a distinction based on gender when they conduct peer reviews; many qualified that statement with a recognition that questions of gender and peer review are tricky to respond to since they had had little experience reviewing women given the small ratio of females to males in Science. Three male faculty members in Arts (Bulmer A3TM; Ludar A3TM; Manfred A2TM) acknowledge that questions of gender are omnipresent and presume that their colleagues, like themselves, are sensitive to issues of diversity and gender when doing peer reviews. Only one male, Stromberg (A1TM), outright says that gender can influence how an evaluator judges a classroom teaching situation. He explains that, in the male-dominated discipline to which he belongs, men expect any individual whose arguments are being challenged to fervently defend themselves. He suspects that a male colleague might poorly judge a female whose behaviour did not meet that norm. Two women (Hannah, S2t; Parsons, A3T) are also emphatic about the role of gender in peer reviews: they reason that, because all social relationships are gendered, gender must “in very subtle ways” (Parsons, A3TF) influence what happens in peer review. Both these women refer to knowing that male students evaluate their female and male instructors differently, and do not think peer review of teaching can be immune to whatever is the cause of this phenomenon.15 A few other women recognize that gender may play into peer reviews especially if the female instructor were teaching a class where gender figured prominently, such as a class on female prostitution or feminist  15     Many faculty members say that, although they do not think gender affects peer reviews in any way, it definitely plays out in the classroom and in interactions with students. They provided accounts of women having to be more intentional about establishing their authority in the classroom and stories of challenging students ‘targeting’ women more than men.  118 organizations. Many faculty members believe that personality plays into peer review much more than gender.  4.6.4.5 Rank Few people comment on the preferred rank of their reviewer, which might be because they have come to expect a reviewer to be of a higher rank. As I wrote previously, in all departments except S1 and A2, only faculty members of a higher rank than the candidate conduct reviews. It is noteworthy that in A2 the department head has, in the past few years, implemented an initiative whereby peers at the same rank—that is, people who will not be voting on the case—can conduct summative reviews. This innovation was implemented, Manfred (A2TM) reckons, for both practical reasons (being able to draw upon a larger number of people helps to coordinate dates and times for reviews) and also to make the process less hierarchical and, therefore, perhaps more comfortable for the person being evaluated. Faculty members are content to be reviewed by tenure track instructors and research professors.  4.6.5 More, Sooner A small number of faculty members would like a greater variety of teaching practices to be included in the summative peer review of teaching. In addition to the observation of an undergraduate lecture or graduate seminar, and reviewing course materials, they name graduate supervision, students’ perspectives, ongoing revisions people make to their courses, problem-based-learning and other group work, teaching assistant training, and tutorials as  119 potentially rich sources of data for peer review. At present, these activities are not incorporated into the evaluation of teaching.  Several Science professors and one Arts professor say that an ideal peer review process would assess the extent to which a faculty member’s teaching facilitates student learning. “You should be evaluated on whether your instruction works rather than the particular mechanism you use to deliver the material” (Palo, S1TF). Measuring student learning, however, is not something faculty members traditionally strive to do in a peer review. Nor was it something academics knew how to do. Instead, reviewers resorted to evaluating aspects of teaching like clarity of syllabus, content, organization, and learning objectives.  As it pertains explicitly to summative peer review, it was suggested that reviews take place early in a faculty member’s career so that people could have the chance to detect—and presumably address—problems well in advance of the tenure decision. Several participants also recommend that peer reviews take place more regularly in the years leading up to tenure so that candidates could be presented with multiple opportunities for feedback. A small number suggest they would like to have the same reviewer over time so that person could comment on their progress and development over the span of several years.  Some faculty members would prefer reviews to occur earlier in the term as this would allow instructors to alter their teaching, provided that feedback from reviews were promptly shared with the person reviewed. A number of candidates say they appreciate having input into the selection of the reviewers and the specific class that will be observed.   120 4.7 Summary of Findings Participants’ experiences and understandings of the purposes of summative peer review of teaching are influenced by institutional culture, namely the demanding nature of academic careers and a reward system that favours research productivity over teaching. Given the existing institutional reward system, as well as limited time and resources, faculty members must make strategic choices about where to expend effort, and the emphasis is usually on research.  At the individual and departmental level, participants vary in their understandings of the summative peer review of teaching and have had diverse personal experiences of the process. Because of this people cannot speak about peer review as a uniform process. Faculty members understand that summative peer review is a formal means of evaluating teaching and is tied to career advancement; however, they differ in their opinions about the strength of that connection. According to some, the connection is weak, a loose knot with fraying string, as it were. For these faculty members, peer review does little except help detect a “train wreck” and/or take time away from other tasks. Other faculty members consider the process, and information derived from it, as crucial and important to the formal evaluation of teaching. Those who maintain there is a weak link between peer reviews and tenure decisions point out that when it comes to the evaluation of teaching, administrators and other decision- makers rely heavily on student evaluations of teaching.  There is a great deal of uncertainty embedded in the peer review of teaching process. Faculty members have questions that range from ‘Which standards and criteria are used in the  121 evaluations?’ to ‘Who is a suitable reviewer?’ Even some fundamental practicalities of the process (i.e., frequency of peer review; whether they will obtain feedback, and in what format and from whom; and whether tenured faculty members are peer-reviewed on their teaching) were unknown to them. Not only are faculty members unsure about the extent to which summative peer review counts toward tenure and promotion, but they are also uncertain which “rules” guide the process. For faculty members in Science, the absence of formal guidelines and criteria for peer review of teaching seems to create more unease than for their counterparts in Arts.  Despite an official University document on peer review, only a few participants mention its existence, and none refer to it in order to justify and explain the ways peer reviews are carried out in their departments. Even in A3, where there is a departmental policy on summative peer review, not all faculty members were familiar with it, and its guidelines are interpreted in various ways by different individuals. Lacking information from their department head or others about the peer review, most faculty members develop their own personal expectations about how the process should unfold.  Many of the faculty members agree that incorporating feedback into the peer review process would be a desirable way to learn and develop as a teacher. Those who share this opinion expressed frustration and disappointment at the lack of feedback they received when they were peer-reviewed for tenure and/or promotion. This in spite of the fact that a large number of participants also felt that when summative peer review of teaching was used to address multiple goals such as formal evaluation and fostering professional growth in teaching,  122 neither was well achieved. This was one of the main contradictions in the findings and something I have come to think of as the “paradox” of summative peer review: on one hand, many faculty members claim that the formative and evaluative functions of peer review should be kept separate, because, when combined, neither can be well achieved and, on the other hand, many express disappointment that the peer review process does not contribute significantly to their growth as teachers. Thus, even though faculty members acknowledge the (primarily) evaluative purpose of summative peer review, they are keen to learn about their teaching through the process.	
  	
    Ultimately the design of any peer review process hinges upon its purpose. When that purpose is not clear, those involved in peer review, whether they be the individual reviewer or candidate, committee members evaluating a tenure application, or university administrators—employ a personal sense of what is best. When this is the case, faculty members’ experiences of summative peer review seem to vary, and be inconsistent within and across departments.  In the ensuing discussion, I use the literature to conduct a final analysis of the major findings pertaining to tenure track professors’ experiences of summative peer review. In doing so, I provide additional background that will support the conclusions and recommendations from this study (see Chapter 6). Although I reported on institutional culture in this chapter’s findings, I shall address it in the discussion in Chapter 5 because the reward system and the demanding nature of academic careers influences the summative peer review, departmental  123 culture, and professional growth in teaching in overlapping ways that are best understood when one acknowledges that they are nested within a broader institutional context.  4.8 Discussion: Summative Peer Review and Professional Growth in Teaching So far, I have described variations and inconsistencies in faculty members’ experiences and understandings of summative peer review. Despite this variability, two predictable aspects of summative peer review emerge. Firstly, for the most part it does not, according to faculty members, promote professional growth in teaching. This confirms research on peer review that ties the formative process to instructor’s self-development through reflection and collective dialogue about teaching, and ties the summative process to institutional issues of accountability (Byrne et al., 2010; Peel, 2005). Secondly, peer reviews are not consistently used to inform tenure and promotion decisions. If, according to the literature, professional growth and evaluation are the primary reasons for conducting peer review, why are these purposes not being fulfilled?  I propose that the summative peer review of teaching does not fulfill its objectives because the process violates, threatens, or otherwise disrupts firmly established academic values and norms and/or other practices upheld in the large, research-intensive North American university. Consequently, faculty members’ resistance to peer review of teaching remains strong, while their engagement in the process continues to be weak.  124  4.8.1 Unsettling/Honouring Collegiality A primary reason for lack of engagement in summative peer review is the ways in which this practice can unsettle and even jeopardize collegial relationships. The summative peer review of teaching involves complicated power relationships. The dynamics at play between individuals (and/or processes/structures) who (that) exercise power and those with less power can upset collegiality.  4.8.1.1 Feedback Unease First, a summative peer review can feel threatening to a reviewer and/or candidate because it requires one or more peers to evaluate a colleague in ways that inform high-stakes career decisions. As discussed in Chapter 2, academics feel uncomfortable about giving and/or receiving negative feedback in peer-review situations (Hammersley-Fletcher & Orsmond, 2004; Kell & Annetts, 2009); this has been tied to an academic disinterest in engaging in teaching-related conversations that could lead to fights and disagreements (Massy et al., 1994; Roxå & Mårtensson, 2009).  This desire to avoid friction with colleagues may partly account for why reviewers do not share feedback with the candidate. That is, even though the post-review conversations have the potential to contribute to the development of teaching by promoting reflection and expanding an understanding of teaching (Peel, 2005; Pratt, 1997), faculty members may judge that doing so poses an unnecessary risk to collegial relationships. Instead, academics may rationalize that peer reviews should be performed at “arm’s length” (Woodroof, S2TM)  125 and like external reviewers’ reports, should not be shared with the candidate. When this approach is taken, faculty members can miss an opportunity to support another’s professional growth, but run less chance of upsetting collegial relationships. 	
   Summative peer reviews of teaching therefore present contrasting opportunities for collegiality. From one point of view, one might expect that in a collegial culture where colleagues presumably behave in ways that support each other’s “well-being” (to use a definition of collegiality borrowed from Gappa et al., 2007), peer reviews would be considered a valued opportunity to learn through observation, reflection, and discussion. From another perspective, peer reviews that do not include feedback may reinforce the notions of mutual respect, per Gappa et al.’s definition of collegiality: when peers respect each other’s autonomy in teaching and withhold critical feedback, they minimize the risk of creating friction with colleagues and thereby preserve and/or honour collegiality.  4.8.1.2 The Primacy of Tenure A related reason why peer reviews may threaten collegial relationships involves the primacy of tenure in faculty careers. The value placed on tenure inhibits faculty members from recording suggestions for constructive feedback in the summative process.  As reported in the findings, though a few participants claimed they endeavoured to present a “balanced” perspective in their peer review reports, the majority said they did not include comments that might be misconstrued in decisions about career advancement. The literature indicates that tenure is highly valued in North American research-intensive universities because it confers job security, power, and prestige (Chait, 2002; Clotfelter, 2002). However,  126 it also indicates that many academics also strongly value learning and growth in their teaching (Gappa et al., 2007; Neumann, 2009; O’Meara et al., 2008). This means that faculty members are squeezed between conflicting values. On one hand, they cherish tenure and promotion while understanding that documenting feedback in official reports might deter a positive review. On the other hand, many are intrinsically motivated to grow as teachers. So, even though faculty members understand why feedback is not a regular part of the summative peer review process, they are bothered by that absence and, as the findings indicate, would appreciate their colleagues’ perspectives and insights about their teaching. This might help explain what I earlier call the paradox of peer review. This conflict arises in part because from a professional growth perspective, sharing constructive feedback would be desirable, but from a tenure/promotion perspective, the same feedback is considered potentially harmful to colleagues. 	
   4.8.1.3 Teaching as Private Summative peer reviews of teaching can unsettle collegial relationships because of the public nature of the classroom observations of teaching; these disrupt the norm of teaching as a private activity16. Teaching in higher educations has often been described as a private endeavour, one that is shared between only an instructor and his/her students (Chism, 2007a; Hutchings, 1996). Unaccustomed to being in each other’s classrooms, with no formal  16      My understanding and use of the words “private” and “public” have been shaped by the work of Pat Huchings (1996) and other scholars (e.g. Daniel Bernstein, Lee Shulman) who, in promoting the scholarship of teaching and learning, have underscored that—outside of the interactions a teacher has with his/her students—the various activities that comprise teaching (planning, assessing, advising, reflecting, and so on) are conducted by the teacher, privately and without the benefit of ongoing collaboration or dialogue with colleagues.  127 guidance on how to proceed, several faculty members described the peer reviews of teaching as “artificial” and found them “nervous-making” (Deitz, S1aF). Reviewers and reviewed alike experienced discomfort with the process. The act of peer review challenges the instructor’s role as an autonomous member of the academy (Chism, 2007a; Lomas & Nicholls, 2005, Swinglehurt et al., 2008). In peer reviews, the faculty member is observed by a colleague in a space where peers traditionally do not interfere. The observation experience is made more intense when tied to high-stakes decisions concerning tenure, promotion, or reappointment. Consequently, the faculty member may feel his/her autonomy, already diminished by the presence of a colleague, could be further jeopardized by the outcome of the peer review process. Even though faculty members commonly engage in peer review to confer judgments on conference papers, presentations, journals, and grant proposals, (Biagioli, 2002; Hamilton, 2007), the public nature of summative reviews, where a peer is present in the classroom and colleagues can read the summative report, runs contrary to anonymous, blind reviews that are the norm in evaluating conference proposals, grants, and other types of scholarly publications.  4.8.2 	
  Universalism (or We Should Not be Evaluating from a Gut Feeling) Resistance to summative peer review may be compounded by the lack of explicit criteria for evaluating teaching, which may disturb the norm of universalism. The norm of universalism involves judging merit based on impersonal and universal criteria rather than such criteria as gender, personality traits, and nationality (Mitroff, 1974; O’Meara, 2011). Universalism presumes the existence of criteria and stipulates the nature of these. If criteria are important to engagement, but faculty members do not believe that criteria for good teaching exist or are  128 unfamiliar with them, academics may experience “hostile resistance” at worst toward peer reviews or, at best, feel a “grudging acceptance” (Arreola, 2007).17 Moretti (S3TF), for example, deplores how the evaluation of teaching typically occurs from a “gut feeling.” Likewise, Quinn (S1TM) recalls the discomfort he experienced when evaluating a peer because he could not confidently ascertain the suitability of his colleague’s approach. These examples are representative of discomfort related to a lack of specific criteria for evaluating the teaching of one’s colleagues.  Another way of thinking about this is that some faculty members find it disturbing to evaluate colleagues’ teaching without established criteria because doing so goes against the norm of academic honesty that is fundamental to academic profession (Clark, 1987). Academic honesty, which refers to the “honest handling of knowledge . . . and fair treatment of colleagues and students, using established universal criteria rather than particularistic judgment” (Clark, 1987, p.133), is similar to the norm of universalism. The subjectivity inherent in the peer review process and its vague, presumably nonexistent criteria can make faculty members feel they are not upholding the value of academic honesty (or not behaving according to the norm of universalism).  In this study, Science professors spoke more often than their Arts counterparts about wanting  17     Arreola (2007) describes how faculty members’ resistance to an integrated faculty evaluation and development program undergoes five stages of resistance starting at “disdainful denial” in Stage 1, moving through “hostile resistance,” “apparent acquiescence,” “attempt to scuttle” (in Stages 2, 3, and 4 respectively) and terminating at “grudging acceptance” in Stage 5. His description is useful for thinking about faculty members’ resistance to new teaching initiatives.   129 an objective peer review process—one with more clearly defined criteria and standards. This is not surprising given that research on disciplinary cultures has found that individuals in hard disciplines are entrenched in a strong tradition of structure, quantification, and coherence (Becher, 1994; Neumann et al., 2002). Consequently, the absence of criteria and the individualistic manner in which peer reviews are conducted were less familiar and therefore potentially more uncomfortable for faculty members in Science. For faculty members in soft-pure disciplines, where traditions are concerned with discovery through interpretation and qualitative approaches are used more often (Becher, 1994; Umbach, 2007), a peer review process that entails ‘no rules’ is more in line with their disciplinary norms and traditions.  4.8.3 Student Evaluation of Teaching Reigns Another reason faculty members did not engage in peer reviews of teaching is the importance placed upon student evaluations of teaching. As described in Chapter 4, faculty members felt department heads and colleagues placed more weight on the student evaluation of teaching scores than on peer review results. In annual review meetings with the department heads, peer reviews were rarely discussed, especially when the student evaluation of teaching scores were acceptable or strong, as determined by a department norm. Likewise, in standing committee meetings, when teaching evaluation scores were high (or acceptable), voting colleagues directed their attention to other aspects of a colleague’s tenure file. As a result, the message about teaching is it will be positively evaluated if student evaluation scores meet or exceed the department norm. That is, teaching was presumed to be good (or good enough) as long as scores were acceptable. As a result, faculty members were uncertain about the extent  130 to which a peer review of teaching influenced tenure and promotion decisions, if at all.  The importance placed upon student evaluations of teaching is unsurprising, given that these instruments have a long history in North American higher education. They were introduced in the late 1960s in response to student demands for public accountability and by the 1980s were established as a “normative-quality measure for administrators” (Ory, 2000, p. 14). To date, they have become the predominant means of evaluating teaching for tenure and promotion (Emery, Kramer, & Tian, 2003).18 The regular use of peer reviews of teaching is, in comparison, relatively recent. Thus, one might justify a faculty reliance on student evaluations of teaching by pointing to the strong role that tradition and history play in restricting change in some higher education institutions (Kezar & Eckel, 2002).  Tradition aside, the reliance on student evaluations of teaching was somewhat perplexing, given that participants described the instrument as flawed and appreciated the alternative and useful perspective provided by peer reviews. The validity of the student evaluations of teaching has been widely questioned because scores can reflect student satisfaction more than learning outcomes (Ackerman et al., 2009; Emery et al., 2003). In addition, student evaluations have been identified as a source of faculty members’ anxiety. Poor results can be disappointing and stressful, especially when academics have expended sincere instructional  18      Within the context of UWC, official policy documents (e.g., the collective agreement or related guidance document used to assist personnel committees or those involved in the decision-making process) do not place more emphasis on the student evaluations of teaching over peer reviews. However, there exists a senate-approved policy on the student evaluations whereas the same is not true for peer reviews. Furthermore, prior to the most recent 2007 Senate Policy on Student Evaluations, there were policies dating back to 1978; this may be taken as evidence that the student evaluations of teaching, as compared to peer reviews, have a longer history.  131 efforts, time, and energy to develop courses (Hodges & Stanton, 2006).  Trout (2000) proposes that instructors accept student evaluations of teaching in part because the practice is so widespread (and therefore perceived as “legitimate”), because they know little about evaluation and its effect on their careers, and because they consider it fair that students be able to evaluate their instructors since the opposite also occurs. I would suggest an additional reason why academics support the use of student evaluations: they value students’ perspective on their teaching because these perspectives can help them improve their courses and modify their teaching. Students’ qualitative comments provide faculty insight into their students’ intellectual development. These insights can contribute to scholarly approaches to teaching and help faculty members make informed choices about their teaching practice (Hodges & Stanton, 2006).  The findings in this study indicate that student evaluations of teaching can also promote competition between faculty members. Motivated to reach or exceed a departmental norm, faculty members may work to modify their teaching and augment their scores. Whereas it has been found that evaluations of teaching normally provide inadequate motivation for faculty members to improve their teaching (Seldin, 1999), those in this study indicated that evaluations triggered a sense of competition that motivated them to improve.  4.8.4 Slow Change in Academia A final finding I wish to analyze relates to the disjuncture between the existence of an official UWC report on summative peer review and the perception that persists among faculty members that there is a lack of guidelines in the peer review of teaching process.  132  The perception about a lack of guidelines may be attributed to an absence of a departmental policy on peer review; with the exception of A3, none of the departments in this study had such a policy. However, at the time of interviews, there was an official university-level report that outlines “principles to guide the development of unit-specific procedures and practices of peer review” and includes implementation guidelines that can be adapted to the specific department and/or faculty context. That report was created at the request of the Office of the Provost and Vice President Academic and circulated in November 2009. By spring 2010, when I began interviewing, it had been, at least in theory, available to departments for six or more months.  Given a university-level report and/or departmental policy with explicit guidelines on peer review, why did summative review operate largely according to reviewers’ preferences, with few guidelines structuring the process? Why was there apparently little formal follow-up on the report’s recommendations? Perhaps the department heads did not send the UWC report on summative peer review to faculty members and/or it was not discussed at department meetings. Since only one participant explicitly mentioned the report, this assumption seems entirely possible. However, it may be that the report was circulated, but for various reasons faculty members chose to ignore it. The issue of whether the summative peer review report was taken up is significant, as it represents a prevalent issue in higher education: the policy- to-practice gap in teaching and learning strategies.  Higher education leaders and policy-makers create policies with the aim of improving an element of teaching practice under the implicit assumption that such change will be beneficial. Implementing large-scale changes, however, is commonly “difficult, painstaking,  133 and protracted” (Kezar, 2011, p. 236), due in part to competing interests among an institution’s administrators, managers, and academics (Newton, 2003). Newton (2003) suggests that academics may resist new policies because they see them as eroding autonomy. That is, when they believe a policy is imposed for managerial purposes and may diminish whatever control they have over that aspect of their work, academics perceive their autonomy is being constrained. Faculty members may also resist new policies because of policy overload due to a voluminous increase of formal strategies to improve teaching and learning (Newton, 2003; Smith, 2011). This too erodes their sense of autonomy.  Kezar (2001) provides additional insight into why the policy-to-practice gap remains prevalent in higher education, pointing to the fact that these institutions have been characterized as organized anarchies. Organized anarchies commonly have unclear and/or ambiguous goals and uncertain preferences (Kezar, 2001), and involvement in decision processes is fluid and outcomes frequently unpredictable. As a result, there may be inconsistencies between stated goals and actions. Ultimately, faculty members learn what such organizations value by observing their actions (Mills & Hyle, 2001).  In the absence of a linear chain of command, it is often uncertain who’s in charge. Formal authority may be attributed to certain positions, but in practice faculty and administrators hold a great deal of authority (Kezar, 2001). In the case of the UWC report on summative peer review, it may be that faculty members judged recommendations unnecessary or did not understand the change or how it would be beneficial (Kezar, 2011).  The characteristics of an organized anarchy are congruent with the context of a collegial system. In such a context—where it is not considered acceptable to apply coercive power  134 over one’s peers (Kezar, 2001)—faculty members are unlikely to employ tactics that ensure peer reviews are done according to specific guidelines, which could explain why faculty members, including those who head peer review initiatives in their departments, do not impose rules on how the process should be conducted. Thus, even if faculty members in a collegial culture are familiar with peer review guidelines, they may not adopt them.  In the next chapter, I report on and discuss findings related to the second research question. In the final chapter of this dissertation, I make recommendations for theory, research, and practice supported by the findings and analyses reported in Chapters 4 and 5 and based on relevant scholarship.  135  Chapter  5: Departmental Practices that Promote or Hinder a Culture that Values Teaching 5.1 Chapter Structure In this chapter, I report on features of departmental culture that support or hinder teaching as described by study participants. Because participants frequently mentioned elements of institutional and disciplinary cultures that had an impact on their departmental experiences, I also incorporate these findings here. This chapter is comprised of four sections: 1) an exploration of informal departmental practices that shape the culture of teaching and learning, summarized in Table 8; 2) an identification of formal departmental practices and initiatives that assist faculty members who want to grow and learn in their teaching, also presented in Table 8; 3) an examination of ways that a department head does or does not support faculty members in developing in their role as teachers; and 4) a discussion of findings related to the second research question. Throughout the chapter, I highlight how professional growth in teaching occurs not only when an individual is internally driven to develop this area, but because of institutional, disciplinary, and departmental practices, beliefs, and structures that support or hinder faculty members’ growth as teachers.  Despite the few rewards that are offered for strong teaching, combined with how departments and individuals are measured by their research productivity, teaching remains a core commitment for study participants. Even without apparent institutional support, my data indicate that participants are continually learning about teaching.  Faculty members provided numerous stories that demonstrate the high value they place on  136 teaching and growing as instructors. Without exception, each individual spoke—at length or in passing—about specific actions s/he had taken to evolve in their professional role as a teacher. For instance, some faculty members redesigned their courses thanks to support from the ESEd, several worked on department-wide endeavours such as implementing a teaching assistant training program or modifying the department’s undergraduate curriculum, and others participated in cross-disciplinary team-teaching. These are just a few examples of initiatives that promote academics’ professional growth in teaching and through which they exemplify their commitment to teaching.  Though participants’ narratives provide evidence of their ongoing involvement in activities that promote professional growth as teachers, I wish to underscore that the majority lamented being unable to pursue such activities more fully. Knowing that research productivity will be favoured in tenure and promotion decisions, faculty members hesitated to engage in time- intensive teaching professional development. For instance, several participants stated they were keen to observe colleagues’ classes to learn more about teaching, but could not find the time to do so. A few noted that they rarely met with their respective mentors because everyone was so busy; others said they lacked the time to participate in workshops at the Centre for Teaching and Learning (CTL) or with the ESEd. As described in Chapter 4, time constraints are tied to expectations of research productivity and a reward system that favours research over teaching. Participants, particularly female faculty, recognized the exigent nature of teaching. Teaching requires a great deal of time and can take “a lot out of you” (Fradera, A3tF). One participant compared teaching to a “black hole,” noting that “you can spend all your energy [on it] and not get what you want” (Maki, A3aF) in terms of rewards  137 like student engagement and outside recognition.  Within the UWC institutional culture, participants’ stories reveal that learning about teaching largely happens because they take action: they pose questions, conduct midterm evaluations, reflect, and get involved in teaching and learning initiatives. Since faculty members’ learning occurs principally in the context of their departments, the next three sections report on departmental practices that influence the culture of teaching and aid faculty members to learn about teaching. Included is a section that deals with the role of the department head. As I will discuss, both informal and formal practices in a given department support teaching—with greater or lesser success. Some of these practices are subtle ways of being that colleagues adopt while others are established programs and initiatives. Together, they comprise an array of approaches that establish the departmental culture of teaching within which faculty members pursue their pedagogical interests.  5.2 Informal Departmental Practices Participants grow as teachers by partaking in various informal departmental practices. Informal practices, in this study, refer to those which are not intentionally planned or coordinated but that influence professional growth and contribute to a departmental culture that values teaching. They can include how faculty members interact with one another, behaviours they adopt uncritically and other actions that collectively might be categorized as “the way we do things around here.” For example, many faculty members gain knowledge about teaching by engaging in spontaneous conversations with their colleagues. Others describe learning through trial and error, saying they have to “make it up as they go along”  138 (Abendroth, S1tM), learn from their mistakes, and “pick it up on their own” (Nemeth, A2tF). Academics also draw from models in their past, such as teachers they admired or supervisors. All of these, and several others upon which I elaborate in this section, are examples of informal ways that faculty members learn about teaching in the context of their department. A summary of these practices can be found in Table 8.  5.2.1 Informal Conversations with Colleagues My findings show that informal conversations with colleagues are one of the main ways that faculty members develop in their roles as teachers. “Most faculty try to improve their teaching to some extent, if they’re interested, by talking to other people, other faculty” (Warr, S2TM). These conversations generally occur on an “ad-hoc continuing basis” (Wilks, S1tF). Faculty members approach their peers about a wide range of issues from the nuts and bolts of teaching to deeper discussions about implications of assignments, readings, and teaching approaches. In their informal conversations, faculty members garner advice about their syllabi, ask how to address a situation specific to a student, and share colleagues’ course materials, to cite only a few examples.  Participants repeatedly emphasize the importance of these conversations to their learning and reveal that the exchanges thrive in an atmosphere where peers have collegial relationships and care about teaching. Given that faculty members from every department, with the exception of S3, perceive that their peers are interested in talking about teaching and use adjectives such as “harmonious,” “friendly,” “collegial,” and “supportive” to describe their department and departmental peers, it is not surprising that many of the participants say there  139 are ongoing informal conversations about teaching in their unit. For example, Dodgson (S2aF), who describes her department as “one of the nicest” where “everyone can talk to everyone,” has asked the “good teachers” about their specific teaching approaches in order to develop her own “tricks” and ideas. Furthermore, the majority of faculty members in this study, with the exception of those I interviewed in S3, speak about their department as a place where people care about, and work hard at, teaching. “And I think most people care a lot. I think it’s—not just care a lot, but work at it and invest a huge amount of time during the teaching term to make sure that things are going well” (Knauer, S2TF). Especially in A2 and A3, participants make reference to “a strong teaching department” (Maki, A3aF) that has many excellent teachers and, in A3, numerous teaching award winners. In a departmental atmosphere that is “cohesive” and perhaps even, according to some, “non-hierarchical,” peers are comfortable approaching one another with questions about teaching. “It’s the kind of culture where if you ask, they’ll answer,” remarks Ledonne about her own department (S1tF).  5.2.2 Collaborative/Individual Teaching Approaches Teaching may be a more collaborative or individual practice. In the Science departments, faculty members more often mention collaborative teaching initiatives than their colleagues in the Arts departments. For example, numerous Science faculty members speak about working with one another to establish links and/or consistency between courses in the undergraduate curriculum. With the assistance of the ESEd, colleagues determine learning goals for courses, put them online in a spreadsheet, and discuss how courses relate to one another, the rationale for their existing course sequence, and how courses contribute to the  140 overall curriculum. Science faculty members also jointly develop core courses that are then taught with a consistent approach. The outcome, explains Abendroth (S1tM), is that “it’s sort of accepted what should be taught, and how it’s going to be taught.” This consistency is considered desirable by the Science participants.  In the Arts departments, participants refer more often than in the Science departments to being able to shape their courses as they judge best. Several participants mention that there is no canonical set of topics for their course and that, consequently, they have a great deal of freedom when it comes to determining course content, readings and assignments. Says Irwin (A1aM) about creating his syllabus: “No one would have told me how to handle the class. It was my class.” Consequently, Irwin adds, he did not ask a previous instructor for his/her syllabus as he did not think it would be useful because of the unique approach each person is likely to take. “Every course is very particular and individualized,” explains Stromberg (A1TM) “and so you have to kind of figure out your personal perspective and then how you’re going to communicate that to students because you want, actually, to be [in] kind of a dialogue between the instructor and the students.” A small number of faculty members in the Arts departments specify that materials from different offerings of the same courses are not expected to have any relationship to each other and, as long as course descriptors do not mislead students, faculty members can design their course as they want. In A2, where there are no course prerequisites, flexibility is at a maximum “because none of us can assume that anybody has learned anything in particular” (Nemeth, A2tF). For the most part, Arts faculty members speak of this approach to course planning positively, suggesting an underlying assumption that this is how things (ought to) work. Two faculty members in A3, however, indicated that some attempts are being made to establish greater cohesion among courses: (a)  141 Bulmer (TM) tells of an ongoing process of undergraduate curricular reform to examine their course offerings and structure as a whole; and (b) Maki (aF) asserts that, as compared to five years ago, there “is a little bit more collective input into the topics that get taught.” She contrasts that to before when “there was very much a sense of you can teach whatever you want.”  Furthermore, or perhaps as a consequence of working more or less collaboratively to shape undergraduate core courses, faculty members express “ownership” over courses differently in the Science and Arts departments. In Science, faculty members refer to undergraduate courses not as their own, but as collective property. “No one owns a course” (Woodroof, S2TM). That is, the same course may be taught by a multiplicity of faculty members and faculty members teach courses that range from elementary to advanced topics. In Arts, participants speak more often of the flexibility they have to shape courses. This broad disciplinary difference, in which there is more apparent collaboration in teaching-related matters in Science, surfaces repeatedly through the interviews and is an element of departmental culture which has bearing on collegial discussions of teaching and, consequently, on professional growth in teaching.  5.2.3 Departmental Composition The composition of a department affects the departmental culture of teaching, and conversations that take place in that unit, in several ways. Firstly, and as it pertains to pre- tenured faculty, these members bring energy and new perspectives to their unit. Hardy (tM), in A2, cites the expansion of their graduate program and the implementation of their teaching  142 assistant training program as examples of projects that have developed largely due to the initiative of the junior faculty. In S3 where, historically, faculty members have “not been so concerned about teaching” (Dawson, S3aM), Moretti (S3TF) notes that—unlike many of her senior colleagues when they were early on in their career—the younger faculty members take a distinct interest in teaching. Similarly, Dawson points out that the younger faculty members have been more involved in redesigning their courses and reflects, about himself and others, that may be so because “we’re young and impetuous” and therefore have had the courage to make such large-scale changes. Secondly, new faculty members support one another in their teaching professional growth. Warren (A3aF), who was hired the same year as four other pre- tenured faculty members, describes how the five new hires experienced a “peer effect” that was conducive to learning. The new faculty members, she explains, “had pipelines” to senior colleagues, all of whom were “very interested in teaching and very interested in making sure we taught well particularly because there was such a big group of us coming in.” New information acquired from senior peers would then be “disseminated” among the pre-tenured faculty members. Another advantage of having a number of colleagues at an early career stage is that they sometimes feel more comfortable going to one another with what they perceive as a “dumb” teaching question. Thirdly, new faculty members, unfamiliar with departmental norms, may alert senior faculty to the need for additional teaching support. For example, in A3, the surge in pre-tenured faculty members prompted people to realize that it would be helpful to formalize some of the procedures and policies that, to date, had been unwritten and informal. The development of a handbook for new instructors and the official establishment of a mini-committee that guides pre-tenured faculty members as they prepare for their annual review with the department head are cited as two examples. Pre-tenured  143 faculty members in A3 express appreciation for some of the measures the department has taken to formalize documents and procedures because it provides them with clear career support. The above three points exemplify that pre-tenured faculty members can shift the departmental teaching culture in important ways.  A second example of department composition that can influence the departmental culture and the extent to which conversations about teaching occur is the ratio of men to women. Specifically, a small ratio of women to men affects female faculty members’ sense of vulnerability; therefore, in a unit where men vastly outnumber women, female faculty may take fewer risks in their teaching—including talking with their male colleagues about teaching. For instance, Fradera (A3tF) remembers being more comfortable approaching other female colleagues with questions about teaching when she started and was one of three women in the department. Similarly, Nemeth (A2tF), who gave a lecture on sexuality and the internet the last time she was peer reviewed, says that she would not have done so earlier in her career, in part because she was one of only two women faculty members, and also because her department was much more conservative in the past. In those days there were a lot of people I would not have given a lecture like that in front of. I would have had to completely recast what I was doing in order to feel that I wouldn’t be vulnerable in a whole range of ways. (Nemeth, A2tF) Parsons (A3TF), who recalls several incidents when young, male students have challenged and tested her authority in the classroom, and who knows this is an ongoing problem for many women, says that this is something she speaks about with her junior female colleagues. She frequently acts as an informal mentor to junior female colleagues, advising them on how  144 to handle challenging teaching-related situations or on other career-related issues. Parsons applauds the fact that, within the context of department meetings, colleagues now discuss gender issues such as the “huge demand placed on them [female faculty members] in terms of female students coming to them for advice that’s unrelated to the particular course.” When female faculty feel less vulnerable and can speak collectively about issues that affect them and their colleagues, this may allow women to pursue their teaching with a greater sense of confidence.  In addition, faculty members cite the physical presence of departmental colleagues as conducive to engaging in discussions about teaching. That is, faculty members who work in their office on campus benefit from each other’s help and provide valuable support to one another as they tackle teaching issues. Knocking on the door of a neighbouring colleague and having informal conversations about teaching in shared physical spaces, such as the lunch room, hallways, or the laboratory, are significant to faculty members’ professional growth.  Finally, a small number of male and female faculty members acknowledge the role of young children in promoting collegial relationships that may, indirectly, foster discussions of teaching. These faculty members refer to the friendships they have made with other colleagues who have young children and say that, as part of these friendships, they socialize and talk about their life, including teaching. Children may also play a role in creating a more harmonious department because they encourage parent-academics to focus on what is most important. “ [Children] exhaust you and [you] don’t always have time for battling [laughs]. You know, you just focus on the most important things” (Hardy, A2tM). Three of the women  145 point out that when faculty members are able to obtain childcare on the university campus19, they are much more likely to work at the office. Children, therefore, connect people to one another and, when childcare is available close by, can connect people to their place of work.  5.2.4  Good Teaching: No Consensus When asked whether departmental colleagues have a common notion of “good teaching,” faculty members say they do not. Collectively, participants employ a wide range of teaching techniques, speaking frequently about “engaging students.” They recognize that there are a variety of ways to be a strong teacher, stating that no single model is appropriate and declaring that striving for uniformity is undesirable. Some faculty members speculate on where divides might lie and why they might exist; they name factors such as age, where the faculty member was educated (e.g. United States, Eastern Europe), and involvement in ESEd as possible reasons for differences in approaches to teaching. For the most part they are not troubled by the fact that colleagues have varying notions of good teaching. In fact, they largely accept—and even welcome—the differences. When asked if there was faculty consensus in A2 as to what good teaching looks like, Manfred (A2TM) answers with a chuckle: I hope not. I hope not. Because there are very many different styles of teaching. There are different objectives of teaching. There are different emphases. I don’t think there is and I don’t think there should be a consensus. I think there is a consensus about  19     According to the UWC Daycare Parent Council website, the waitlist for UWC daycare is more than 1500 children and parents wait an average of 2-3 years before their wait-listed child is offered a daycare spot.  146 what bad teaching is. . . . There’s probably more agreement on what bad practices in teaching are. This quote brings attention to three points commonly raised in the interviews: firstly, and as already mentioned, there is variation in opinion about what constitutes good teaching; secondly, academics believe and accept that there is no singular model of good teaching, and thirdly, given the variation, faculty members find it challenging to identify good teaching, but can more easily identify the opposite. Given this combination of factors, teaching and its evaluation could potentially turn colleagues fiercely against one another. However, in a context where there exist strong collegial relationships, the lack of consensus over teaching does not appear problematic, according to participants’ accounts. At best, it is celebrated and, at worst, it is given “benign neglect” (Maki, A3aF).  5.2.5 Risk-taking and Trust In departments where faculty members believe they can take risks in teaching and not be punished, they may be more likely to experiment and innovate in their teaching. Doing so encourages professional growth in teaching. Says Cohen (A2aM), “If you really just care about getting good, then I would say just go crazy and do stuff.” He admits, however, that taking that approach can be risky, especially for pre-tenure faculty who might worry about whether their teaching (scores) will be good enough for tenure and promotion. Consequently, the easier model is “just do what you did before and tinker with it a little to improve things a little” (Cohen, A1aM). Palo (S1TF), who experienced a troubling dip in her teaching evaluation scores after she made a substantial change to a course, remembers thinking during a discussion with her department head: “Are you going to punish me for trying something  147 innovative and taking risks?” In her case, she was not punished: she asked for support from her department head who connected her to a colleague she trusts and the latter provided her with suggestions for change that she adopted with good results.  In the Science departments, the existence of, and support from, the ESEd also influences faculty members’ willingness to take risks. Numerous faculty members say they have accessed help from ESEd teaching fellows20 to make changes—sometimes large, other times less so—to their courses. Teaching fellows are recognized as pedagogical specialists who can effectively help faculty members redesign their courses, conduct research on student learning, and assist with a multitude of other related teaching challenges and issues at the classroom and departmental level. It is possible, but not necessarily so, that they would not have implemented the changes without support from teaching fellows working with the ESEd. For instance, when Dawson (S3aM) decided to completely redesign his course, he approached one of the ESEd teaching fellows for help; that person, assigned to help another faculty member, was not available. Dawson decided to pursue his project regardless but was eventually given assistance. He recalls: I think what happened was when people realized that I was going in head first, there was enough political will to let [Name of teaching fellow] off the hook for some other tasks and let him help me. Once they could see that I wasn’t joking, I was serious and  20     ESEd teaching fellows are hired by the department to work with faculty members on initiatives that contribute to the improvement of undergraduate science teaching. These individuals have “a combined expertise in the specific departmental disciplines as well as knowledge in relevant science education methodology and research” (UWC’s ESEd website).   148 seriously over my head maybe, that they let [Name of teaching fellow] help me. And then it was a great success, I think.  Irrespective of whom a faculty member approaches, the issue of trust surfaces strongly within the context of professional growth in teaching. A number of faculty members in the Sciences seek help from ESEd teaching fellows because of the pedagogical expertise these individuals have. Other faculty members distinguish more generally between people they would approach and those they would not. There’s at least a half dozen full profs that I would be quite comfortable approaching. I won’t say that I would feel comfortable approaching every person in the department, but there’s no shortage of people that I could talk to, and get very different viewpoints on things. (Abendroth, S1tM) Palo (S1TF) also speaks to the importance of trust when she describes how she addressed a dip in her teaching scores. She sought help from her department head who referred her to a colleague whom she describes as a “professional instructor, considered one of our very best.” Palo applied the advice given to her, not only because she wanted to address the problem, but because she had faith in her colleague’s strong teaching skills and knowledge. Several faculty members draw on and highly regard the pedagogical expertise of tenure-track instructors.  When dealing with teaching challenges, academics take risks to different degrees and may seek help from their colleagues. Decisions about how to proceed and who to approach are based on different rationales; trust, however, is almost always an important consideration.   149 5.2.6 Attention to Fairness Faculty members remark upon several departmental teaching-related practices that demonstrate fairness, an attribute that people believe contributes to collegiality. Collegial relationships, as I have previously indicated, promote pedagogical conversations, which as participants indicate, are one of the principal ways in which they grow and develop as teachers.  One practice related to fairness is that of having an established teaching load. In S1, all participants speak about the existence of a clear formula for the amount of undergraduate and graduate teaching a faculty member is expected to take on yearly, depending on their rank. A second practice that connects to fairness, and mentioned by numerous participants, is that faculty members at all ranks teach courses at all levels. Another is having a say about which courses one would like to teach. For instance, in S1, Ledonne (tF) describes a giant spreadsheet where, for each class in the upcoming three years, a faculty member can indicate their preference on a continuum that ranges from “eager,” to “no way in heck.” The departmental system is further enhanced because details of that schedule (i.e., which course a faculty member will rank as first choice this coming year) are sometimes negotiated ahead of time among faculty members who work in the same laboratory. Ledonne believes S1’s system is a “model of transparency and fairness” because all faculty members are able to make requests, see each other’s preferences, and make updates to these. Dodgson (S2aF) describes a system that resembles the S1 spreadsheet where faculty members indicate their preferences among courses, days of the week and times of the day. All this, she says, is coordinated by “some poor faculty member” and colleagues do not get to see what courses  150 people requested nor what they have been assigned. “But,” she adds “I know from informal conversations that most people get their preferences.” Unlike Ledonne, Dodgson does not enthusiastically refer to S2’s system as one that promotes a sense of equality and fairness among faculty members. She appreciates being able to input her preferences but may speak less enthusiastically because, first of all, the choices and results are not public and/or, second of all, she may not recognize to what degree this formal system, which is not in effect in all departments, has the potential to build collegiality.  In contrast to these practices, “course buyouts,” whereby faculty members are permitted to use funds from a research grant to reduce their teaching load, were noted to deter from fairness. Dawson (S3aM) describes that “there’s a huge internal struggle in the department regarding teaching buyouts.” He continues to say that, in a department where a lot of people have teaching buyouts to do research and others don’t, “there’s a lot of resentment in between those groups. And I think that’s been an ongoing point of friction.” Ledonne (S1tF) notes that the teaching buyouts are “not part of the culture” in S1, no matter how much money a faculty member receives from their regular research grant and she reasons that “that greatly contributes to fairness.” Ludar (A3TM) adds that if a department head allows teaching buyouts, it sends a message to faculty members that they should pay less attention to teaching than to research. About these, he states: “I won’t deal with teaching buyouts. I just say ‘No.’ to them all.” Allowing teaching buyouts, he makes clear, does not contribute to establishing the importance of teaching in a faculty member’s career.  151  5.3 Organized Initiatives, Policies and Structures Numerous formal teaching-related initiatives and practices prompt professional growth in teaching or in other ways influence teaching within a given department. I distinguish formal initiatives from the informal ones (e.g., informal conversations among colleagues, strategies that promote fairness in teaching loads) because the former are typically planned, structured, or in some other way “legitimately” or unequivocally connected to teaching.  Formal structures to address teaching, in the form of committees, policies, seminars, or officially appointed individuals, exist in all departments. They may come about as a result of faculty-wide initiatives (e.g., the decision to make courses more writing-intensive across a given faculty), departmental ones (e.g., student town hall meeting or peer reviews of teaching), or other institutionally supported “movements” (e.g., community service learning). Among the range of departmental initiatives that fall into the formal category (see Table 8), I will be discussing the following: policies and documents, officially appointed resource people, ESEd, seminars and retreats, team-teaching, and formative peer review.  5.3.1 Recognized Policies and Other Documents The majority of participants say that, within their department, there exist few or no written departmental policies and documents pertaining to teaching. At the time of the interviews, only A3 had a formal policy for the peer review of teaching; most, but not all, of the participants from that department knew of its existence. The policy was created in response to a faculty-wide push to formalize peer reviews of teaching and was developed by three  152 faculty members in consultation with all departmental colleagues. In S2, although there is no written policy, a small committee has recently been struck to work out an established protocol for the peer review of teaching. Hanna (S2tF) explains that, previously, the undergraduate chair had coordinated the peer reviews but “he does so much for the department [that] he’s physically not capable of still doing it. So that's why this committee has been founded, because it's something the department feels is very important.” She suspects that part of the reason the committee has been formed is because the department is now so large (close to 100 faculty members and post-doctoral fellows) that, consequently, “the best way to make sure it [peer review of teaching] keeps running smoothly, I think we all agree from our various departmental meetings, was actually just to formalize what we'd been doing anyway” (Hanna, S2tF). Although many other faculty members in departments outside S2 and A3 recognize that there are more demands associated with peer reviews of teaching as compared to before, none specifically mention a need for a policy.  Asked about other departmental documentation that might serve as a teaching resource, faculty members could only list a few. In S1 and S2, an online spreadsheet offers course learning objectives. S2 and S3 have a relatively new initiative where faculty members can upload their teaching resources (syllabi, assignments, exams) to a password-protected departmental website for colleagues to peruse. Both these projects are part of the ESEd. Hadzik (S3tM) also indicates that, at one time, there was an orientation book for new faculty. Since it was coordinated on a voluntary basis, it “didn’t go far.” In A3, there is a handbook for new instructors but not all new faculty are made aware of it. A few faculty members in A1 and A2 mention that syllabi from past courses are kept in a central location (i.e., filing  153 cabinet in the main office and/or on a departmental website) so that faculty members can access these. Finally, in S1, there is a wiki with teaching-related resources that was started by Deitz (S1aF) and to which various people have added material. Unlike the learning goals project and the online repository of materials that are part of the ESEd, the wiki in S1 was initiated without explicit support from another network. The above underscores that projects are more likely to be taken up and persist when at least one person is appointed to champion and advance them.  The above also brings to light general differences in the extent to which materials are gathered and collectively used in Arts compared to Science. For instance, faculty members in S1 and S2 speak frequently about putting their course materials on their personal website. Furthermore, in Science there are more formal practices for sharing teaching-related material than in the Arts; namely, the course material repository and the spreadsheet for learning objectives that have come about because of the ESEd.21 Perhaps because many Science faculty members post their teaching materials online, they also speak more often than academics in Arts about visiting their colleague’s websites to access material when they have had to teach the same course (they also contact these same colleagues in person for material that might be missing from the web). Trottier (tM) in A2, however, explains that he is more likely to search for online syllabi or teaching materials on the websites of colleagues located  21     Given that I only interviewed participants located in departments associated with the initiative, I do not know if similar online repositories exist in other departments. Nor do I know whether these structures exist because of the ESEd, or whether they would have come about regardless. Furthermore, one cannot help but wonder whether faculty members in Arts would put more of their material online if there were an Arts equivalent to the ESEd. I will take some of these issues up in the discussion.  154 at other institutions. That is because there is a greater chance that a colleague elsewhere would be teaching a course similar to his. Within his department, there would be little overlap in approach or content among faculty members even if they were teaching different offerings of the same course. As already pointed out, faculty members in Arts often take a more individual approach to course development and those in the Science more often take a collective one. That is not to say that academics in Arts are not willing to share their material, for they are. However, when it comes to developing their courses, faculty members in Science are more inclined to work collaboratively. This approach is reflected in the fact that there are more multi-section courses with common exams and assignments and faculty members work collaboratively to figure out how courses in a program align with each another.  5.3.2 Official Go-to People The University’s Centre for Teaching and Learning is cited by several faculty members as a place where one can get assistance with teaching. Participants speak about the CTL as a resource for people who are experiencing “teaching problems” and/or as a space to engage in professional development in teaching via workshops and seminars. A number of faculty members note that they do not participate in the CTL’s activities because of limited time and research demands. In the following quote, Warren (A3aF) talks about her interest in taking a certificate program on teaching in higher education, but decides against it in the end because it will take time away from her research: Actually I remember thinking it sounded really cool and I remember thinking it would be great to do it and I’d probably get a lot out of it and it was never going to  155 happen because it wasn’t going to help publish the article I was working on at the time. Although several participants who are aware of the centre have encouraged their colleagues to attend the CTL offerings, only a few have actually used the services themselves and most turn to departmental colleagues and other resources for help.  Faculty members name the following as helpful departmental colleagues to confer with about teaching: course coordinators (Science), tenure-track instructors, and the undergraduate chair. In the Science departments, the course coordinator refers to the individual in charge of coordinating multiple sections of a course. These courses typically have standard midterms, assignments and readings across sections and may have shared labs. Normally not a new faculty member, the course coordinator—or “master custodian” (Ledonne, S1tF) —provides teaching materials and resources and is available to guide faculty members, as necessary. “That’s your big mentor really—the person in charge of the course. . . . They have a big influence on how it’s done in the course” (Woodroof, S2TM). In addition to the course coordinators, faculty members frequently mention tenure-track instructors as key people to consult with because of their teaching expertise. These individuals who are described as “excellent” or “professional” teachers, are well integrated into their respective departments. In S1, where there are more than 10 tenure-track instructors, instructors also handle “a lot of the grunt work” (Abendroth, S1tM) with undergraduate students, including advising. Another person faculty members approach when they have teaching-related questions is the undergraduate chair. Some faculty members recognize this person as one who takes a key role in teaching initiatives (e.g., in A2 the undergraduate chair planned the teaching retreat  156 and in A3 she was one of three people on a committee that developed the summative peer review policy) and regularly puts teaching on the agenda at department meetings. Finally, in the Science departments, all faculty members who are, or have been, involved in the ESEd mention teaching fellows as important resource people.  Faculty members rarely consider mentors, assigned as part of a formal mentorship program, as people to approach with questions about teaching. According to most participants, the mentorship relationship is largely focused on research and career advancement. Only a small number of faculty members occasionally discuss teaching with their mentor.  5.3.3 Effective Science Education Project (ESEd) According to many Science participants, the Effective Science Education Project promotes professional growth in teaching among those who participate in it and has influenced the culture of teaching within the departments where the initiative exists. Faculty members typically voluntarily self-select to take part in ESEd projects. Projects include, but are not limited to the following: course redesign, reading groups, teaching and learning specific research projects, development of learning goals, curriculum revisions, and creation of assessment methods to measure student learning. Teaching fellows, situated in the departments, work with faculty members on the projects.  157  5.3.4 Teaching Seminars and Retreats In S2, there are regular (every 1-2 months) teaching seminars with guest speakers from within and outside the department. Though they are mainly attended by graduate students and post-doctoral fellows, faculty members also participate. The seminars are associated with a teaching assistant development program for which there is a faculty member whose work duties include developing and coordinating the implementation of that program. In A3, informal brown-bag discussions about teaching took place in the past, approximately once a term, over a span of several years. Parsons (A3TF), who had initiated these, suspects that organizing them “fell by the wayside” because it was not a formal assignment that contributed to the service component of her work. In A2, teaching seminars occur irregularly and are normally organized to address a specific concern shared by faculty members; the concern may have been raised at the department’s annual student town hall or at the teaching retreat.  Both A2 and A3 held a departmental retreat within the year prior to the interviews. In A3, the faculty members also discussed the department’s hiring policies at the retreat, whereas in A2, the event focused entirely on teaching. The department retreat in A2, organized by the undergraduate chair, was a two-day event for full-time and sessional faculty; students were also invited to a portion of the event so they could share some of their concerns. Faculty members were asked to cancel their classes and sessional faculty were paid an honorarium to attend—two measures taken to ensure maximum attendance. In the quote below, Manfred (A2TM) gives a brief description of the event and reflects on its significance:  158 And we spent two whole days talking, not primarily about curriculum but about lecturing methods, about how to teach writing effectively, how to deal with special problems that often arise, of issues of cultural sensitivities, issues of students in classes having very different levels of preparation because of the fact that most of our courses don’t have formal prerequisites, etcetera. And it was a wonderful experience because we did a lot of mutual informing and learning about people’s best and worst experiences and so on. It was also a wonderful experience because it was one of those rare occasions where the full-time faculty and the sessional faculty were in a sense interacting as complete equals. Because this is something we all do. We teach [name of department] courses in this department. . . . I think it created an atmosphere in which people realized that how we teach is something the department actually takes very seriously. And that this is not just an adjunct to the other thing that we take very seriously, which is our research programs, but it is a core commitment of the department. We knew that. But it reaffirmed the centrality of teaching to the overall mission of what we do. Thus, as Manfred indicates, the retreat was significant because colleagues had dedicated time to learn from one another about teaching and this helped reconfirm the importance of teaching to their professional lives.  The other faculty members in this department, all of whom spoke positively about the retreat, also noted that it provoked a lot of informal discussions about teaching. That is in part because the faculty members were given assignments prior to the retreat (i.e., they were requested to sit in on two colleagues’ lectures and given a discipline-specific teaching-related  159 article to read) and because, after the retreat, and in response to faculty members’ suggestions that there be more opportunities to talk about teaching, the undergraduate chair organized some non-mandatory department-wide discussions about teaching.  In A3, the one-day retreat was held during UWC’s reading week. Fuentes (A3aM) explains that one of the main purposes was to get people more enthused about teaching by exposing them to some of the teaching techniques their peers are using. Thus, during the half-day portion dedicated to teaching, nine different faculty members presented on topics such as community service learning, simulations, technology and teaching, and in-class debates. Since normally, “you don’t actually find out what your colleagues are doing in class,” the retreat was “a nice way to find that out,” comments Parsons (A3TF). “Everybody,” she continues, “liked hearing what people were doing in their classes and asked lots of good questions.” Although nearly all the faculty members in department A3 mentioned the retreat, they did not connect it to teaching-related initiatives or conversations that have taken place since then, as did the people in A2.  Departmental retreats provide opportunities for faculty members to share and learn about teaching; when retreats prompt follow-up events, such as seminars or additional conversations, the possibilities for growth are extended.  5.3.5 Team Teaching Across Disciplines There are course offerings that are team taught by faculty members from separate departments and different disciplinary backgrounds. Team teaching is taken here to mean  160 two or more faculty members working to develop a course taught to the same class. In team teaching, faculty members attend or participate in a majority of class sessions during the semester. A small number of faculty members express how these experiences have contributed to their learning about teaching. Nemeth (A2tF) describes team teaching as a useful way to learn about “different ways of being in the classroom” and says that her participation in an interdisciplinary teaching program “probably had more impact on shaping me as a teacher than almost anything else.” Irwin (A1aM) describes team teaching as a sort of “constant” peer review because team members discuss teaching with one another on an ongoing basis as they create tests and assignments, review mid-term student evaluations, and go-over exam results. Though not officially team teaching, some faculty in S1 collaborate extensively with colleagues in their sub-field to determine learning objectives and content for individual courses.  5.3.6 Formative Peer Review Only S2 has a regular practice of formative peer review whereby a senior faculty member observes a new faculty member’s classroom teaching within the first three weeks of their first class. All the faculty members interviewed in S2, with one exception, are aware that this is standard practice in their department. Dodgson, the only pre-tenured faculty member I interviewed in the department, remembers the formative feedback as being helpful, but she does not refer to the practice as being part of the departmental protocol. In S1, Abendroth (tM) recalls one “unusual” year during which everyone (i.e., people on the peer review committee and others) had to conduct a formative peer review of a colleague’s teaching. The  161 initiative was dropped and he does not know why. Finally, and as already mentioned, A2 colleagues did classroom observations of teaching as part of their pre-retreat homework.  Though formative peer review is rarely a part of a department’s official policies or functioning, some faculty members have, upon their own initiative, attended their colleagues’ classes. They do so to better learn the content of the class (sometimes because they will be teaching that content themselves and do not feel sufficiently familiar with it) and/or because they want to observe a stellar teacher in action. Faculty members who sit in on their colleague’s classroom may do so for as long as a term, or as short as a single class.  5.3.7 Teaching and the Hiring Process Many faculty members point out that prospective new hires are asked to discuss teaching (i.e., strategies, approach, syllabi) during their interview or are required to teach as part of their application for a tenure-track position. This practice, participants note, can promote discussions of teaching among individuals who evaluate the candidate’s performance. Nemeth (A2tF) points out, however, that she has colleagues who worry that extensive conversations about teaching during the interview process “makes us look like we’re not really concerned about research.” She says there exists a division in the department on that issue, but notes that in the last round of hiring, discussions of teaching figured prominently. Hadzik (S3tM) also expresses unease about the way teaching is incorporated in the hiring process, but his concerns centre around the (questionable) usefulness of the method. In S3, a candidate gives a lecture—always on the same topic—to departmental faculty members. Hadzik deems this is an ineffective method of gauging the candidate’s teaching abilities and  162 admits that he, like several other colleagues, does not normally attend the teaching portion of a candidate’s presentation. He would be more inclined to take an interest in a prospective new hire’s teaching if the candidate were to teach undergraduate students who need not “pretend to be learners.” Students, he explains, could then comment on their learning, unlike faculty members who already know the content well (see also footnote 14). According to Hadzik, the purpose of evaluating teaching in the hiring process ought to be to determine whether the person can promote student learning; since, at present, the focus is not that, Hadzik remains somewhat disengaged. Moretti (S3TF), however, has a somewhat different perspective on the purpose of considering teaching in the hiring process, saying that it is to have an indicator of the candidate’s potential rather than a way to eliminate “lousy teachers.” We grade them, on a scale from zero to ten. Even if they get graded a four, we might still make them an offer based on their excellent research record, because we say, ‘Oh, they’re young and they haven't taught much. They have potential to teach well, even though that was one of the worst things I’ve ever seen in my life.’ [laughs] (Moretti, S3TF) Most often, she continues, the evaluation panel is hoping to get a sense of the candidate’s attitude toward teaching and/or potential to develop as a good teacher. Even if participants disagree on the details of how teaching should be incorporated into a candidate’s application process, they seem to agree that when teaching is considered during the hiring process it indicates that teaching matters in the department. When colleagues evaluate a candidate’s performance or debate how a department may want to demonstrate their commitment to teaching to prospective new hires, I suspect these discussions contribute to learning about teaching.  163  Table 8: Informal and Formal Initiatives and Practices that Influence the Culture of Teaching within a Department  Department Initiatives and Practices Formal Informal Centre for Teaching and Learning (CTL) ü Classroom observation of teaching ü ü Colleagues, Conversations with  ü Colleagues, Physically present  ü Committee work pertaining to teaching ü Course coordinator for multiple sections of a class ü Course development, Collaboration toward ü ü Course, Introduction of new ü ü Courses, Fairly assigned ü ü Curricular reforms/redesigns ü Department composition   ü Department head, Support for teaching from ü ü Department meetings, Discussions of teaching at ü ü Department retreat, Sessions about teaching included in ü Diverse teaching approaches, Acceptance of  ü Documents, Teaching (i.e. handbook for new instructor) ü Effective Science Education Project (ESEd) ü External review ü Hiring, Attention to teaching in ü Instructors, Tenure-track ü Mentorship ü Midterm course assessment ü ü New faculty, Cohort  ü Peer review, Formative ü ü Peer review, Summative ü Policy, Teaching (e.g. peer review of teaching policy) ü Reflection (e.g., on models from one’s past, self-)  ü Resource, Sharing of (e.g., syllabi, wiki, website) ü ü Risk-taking in teaching, support for innovation  ü  164 Department Initiatives and Practices (continued) Formal Informal Self-reflection  ü Seminars, Teaching ü Socializing  ü Student evaluations of teaching ü Student town hall meeting ü Teaching assistant training ü ü Teaching fellows, ESEd ü Team teaching ü ü Trial and error, Learning by  ü Undergraduate chair ü ü Note: Informal practices, in this study, refer to those that are not intentionally planned or coordinated with the purpose of “developing teaching” but that influence faculty members’ professional growth and promote a departmental culture that values teaching. Formal practices refer to planned and structured activities and/or positions that are more “legitimately” or unequivocally tied to teaching.  5.4 The Department Head’s Role in Promoting a Culture that Values Teaching As the participants described the various informal and formal practices within their department, it became evident that the department head has a significant role in establishing and maintaining a culture that values teaching. That individual can do so in overt ways such as establishing committees to address teaching-related issues or by putting into practice university policies pertaining to the improvement of teaching. Many of the ways the department head supports teaching are subtle. And even though only one department head participated in this study, numerous faculty members spoke about ways their head supported—to a greater or lesser degree—their growth as teachers.  In Chapter 4, I described the department head’s role in the summative peer reviews, and reported that most participants have not discussed these reviews in any depth (or at all) with their department head. In the context of summative peer review, the department head  165 contributed little to the professional growth of teaching. Participants’ stories nevertheless highlighted numerous ways that a department head can be involved in a faculty members’ growth as an instructor.  5.4.1 Teaching and Tenure: Support from the Department Head Through various approaches, department heads help academics develop in their professional roles as teachers. The support offered is sometimes directed toward the improvement of teaching and student learning and other times more focused on ensuring career advancement.  The department head occasionally plays a role in helping faculty members when they encounter challenges in their classroom teaching. For example, when students had complained to the department head about Quinn’s (S3TM) course/teaching, the head alerted Quinn and invited him to provide his perspective on what might be going on. Together, they discussed strategies to tackle the teaching complaint. Similarly, Palo (S3TF) consulted with her department head when her teaching scores dropped after she had made a change to one of her courses; she asked her department head to suggest someone who might have tried a similar approach to hers (with success) and he did so. Palo was able to get the help she needed from her colleague and address the issue.  Sometimes a department head attends to a teaching-related situation after the fact in a way that appears to be specifically geared to supporting tenure and promotion as opposed to having an explicit focus on promoting improved teaching and student learning. Returning to the example above, Palo (S3TF), who had had uncharacteristically poor student evaluation  166 results after making a modification to one of her courses, was invited by the head to present an explanation in her portfolio. Doing so would help ensure that peers voting on her case would “feel comfortable that it was indeed an anomaly” and would also provide her department head with “ammunition” to defend the inconsistency to the Dean’s advisory committee on promotion and tenure.  Perceived support from the department head for teaching also comes about when the head does something as ‘basic’ as mentioning that s/he has looked at the student evaluation of teaching scores. Faculty members respond to this positively noting that it gives “a sense that people are paying attention” (Parsons, A3TF). Warr (S2TM) praises a former department head who reviewed student evaluations of teaching annually in a meeting with faculty members: It was a brief time of one of our heads who made an effort to comment on our student evaluations. Now since I always held the student evaluations as being the most important form of evaluation I was very pleased that the head took the effort to look at them. . . . I’m not really convinced that any head before or any head after has made an effort to worry about the teaching evaluations and hence the teaching of the faculty other than at a promotion tenure time. . . . I’ve never heard anything back from him so it’s sort of like, ‘Oh, so have you ever looked?’ I don’t ask that question because that’s their choice, but I just thought it was very nice when it happened. That meant that during that time if your evaluations had been positive you would get some praise. If they had been negative the person would have been pointing out some difficulties.  167 However, teaching rarely figures in faculty members’ discussions with their department head. “I don’t really remember teaching—the teaching component of my portfolio—ever being discussed in those meetings” (Wilks, S1tF). Fradera (A3tF), whose experience is similar to Wilks’, speculates that unless “red flags are coming up” or, in other words, when the faculty member has received negative feedback on their teaching, heads of department have very little information about a faculty members’ teaching. “They don’t really know what’s going on because we don’t have a constant peer evaluation process,” she says. This latter quote captures two points that come up repeatedly in the interviews. Firstly, teaching receives little attention because the primary focus is faculty member’s research productivity and plan. Indeed, two pre-tenured faculty members’ recollect being told by their head, during their annual meeting, to focus on their research and not on their teaching. “I’ve been told to ignore my teaching and focus on my research because my teaching is fine,” says Deitz (S1aF). Similarly, Cohen (A1aM) remembers being told to “spend less time on your teaching; worry about your research.” Secondly, teaching is discussed if it is deemed that a “teaching problem” exists as indicated, typically, by lower than acceptable student evaluation of teaching scores. Teaching, say some tenured faculty members, is given minimal attention by the department head when a faculty member’s teaching is adequate or excellent. Good teaching, one might deduce, is not something worth talking about.  With regards to addressing teaching when it is problematic, some tenured faculty members— but no pre-tenured faculty—refer to UWC’s Collective Agreement (and/or the related guidance document used to assist personnel committees and others involved in the decision- making process) to explain why a department head discusses problematic teaching with a  168 candidate. They point out that the department head is obliged to discuss concerns that might impede a faculty member’s progress to tenure and provide constructive ways to improve the situation. I make this distinction between tenured and pre-tenured academics and the former’s reference to the Collective Agreement (or guidance document) to suggest that it is possible that individuals with longer academic careers might be able to connect departmental practices to existing policies in ways that newer academics are not yet in the habit of doing.  Faculty members, when reflecting on the importance of the department head’s support, frequently mention it in the context of moving a file forward to tenure. Several pre-tenured faculty members note that, as they prepare for tenure, they will go to their department head for details about the process. Some of the tenured faculty members interviewed recall specific “tips” their department head (and/or other tenured colleagues) shared to help them strengthen their tenure case. Tips included: information about the types of courses to teach (i.e., make sure you teach a first year course), advice to aim for breadth in teaching (i.e., teach outside your specialty area), what an acceptable student evaluation of teaching score is, the suggestion to explain any anomalous path in teaching (i.e., why an individual has a dip in their teaching scores; why someone has taught only in their area) and, of course, “think about more journal publications” (Ledonne, S1tF). Faculty members praised the concrete nature of this advice.  Comments from faculty members attest to their belief that the department head has the knowledge to determine what a successful file consists of; s/he also has the power to support  169 (or not) an individual. Referring to meetings with her department head, Maki (A3aF) explains: They are useful in the pre-tenure process because you need to have the head on your side to be put forward. And you get a sense of how strong is the support—are there any reservations? So I think it’s a psychological support. [It is helpful to hear that] “Everything is on track and fine.” Moretti (S3TF) agrees and speaks to the importance of having a dependable and ethical head. As she described the tenure process, Moretti makes clear that the head’s letter in support/denial of a candidate’s tenure, meant to summarize the standing committee’s report and deliberations, should normally be reviewed by the committee members, but committee members do not always see the letter. Sometimes the head “can just subjectively interpret what they want at the meeting, selectively include or not include things . . . and write some bullshitty letter” before sending it up the ranks. Thus, the head’s support is crucial in augmenting the chance of a file moving forward, as is his/her trustworthiness.  In summary, the department head has a potentially significant role in communicating with faculty members about teaching and its evaluation. However, the way in which an individual head might enact this role varies and is “not built into departmental practices in any systematic way” (Nemeth, A2tF). Consequently, the ways in which the head addresses matters to do with teaching vary. Although, by most faculty members’ accounts, teaching rarely figures in discussions with their department head, the participants indicate that this individual is important in setting a tone for the departmental culture of teaching. “That’s the head’s role, to be constantly reading the department to make sure the messages are  170 supportive and realistic. . . . To a degree signing papers and so on, but sending signals at all times about what standards and behaviours are acceptable or unacceptable” (Ludar, A3TM). The department head’s actions, or lack thereof, communicate in important ways what matters; the values he or she promotes transcend many facets of departmental culture.  5.5 Summary of Findings The faculty members in this study are keen to grow in their roles as instructors and are involved in nurturing a departmental culture that values teaching. Many lament the fact that they cannot fully pursue developing as teachers, and attribute this to an institutional reward system that favours research productivity over teaching and a career that places high demands on their time. Though numerous academics express frustration at not being involved with teaching to the extent that they wish, their collective stories demonstrate multiple departmental practices, structures, and “ways of being” that can, and do, support faculty members in their desire to grow and learn as teachers. As the findings in this chapter indicate, when programs and other structures are well supported, by human and other resources collectively and over time, their potential for helping faculty members grow and learn as teachers increased.  In this chapter I have described ways that informal practices and organized departmental initiatives support the enhancement of teaching and noted there are few official departmental documents that guide teaching. I have pointed to the significance of professional learning conversations to instructor growth and highlighted the importance specifically of trust, and more broadly of collegiality, in promoting these conversations. I have also reported on the  171 relevance of disciplinary traditions, namely working collaboratively or individually on teaching, to the prevalence of collegial interactions about teaching.  In Chapter 5, I have elaborated on the previous chapter’s findings with respect to the important role of department head in supporting a culture that values teaching. In ways both subtle and overt, this person signals the importance of teaching to faculty careers.  5.6 Discussion: Departmental Practices that Support or Hinder a Culture that Values Teaching So far, I have reported on informal practices and organized departmental initiatives, including leadership from the department, which support and hinder a culture that values teaching. The catalysts that have spurred these departmental practices and the factors that maintain them differ. The longevity of the initiatives vary, as does the extent to which faculty members have participated in and/or availed themselves of these. In addition, some practices are more directed at professional growth than others. However, because professional growth occurs as academics engage in their day-to-day activities (Billett, 2001; Clegg, 2003; Eraut, 2004; Jawitz, 2009), even teaching practices not labeled as “professional development” may contribute to faculty members’ learning about teaching. And, although I did not set out to inquire about institutional practices per se, the participants discussed them, thus reaffirming the intersections between institutional and departmental culture.  The results of this study demonstrate that responsibility for learning about teaching does not lie solely within the individual, but that it is strongly influenced by context: the institutional, departmental, and disciplinary cultures within which academics work. These cultures operate  172 simultaneously, though not necessarily equally, depending on the situation (Lee, 2007; Umbach, 2007).  5.6.1 Institutional Culture Faculty members’ understanding of the reward system from the context of the research- intensive university influenced their involvement in teaching and its evaluation. The participants, all of whom believed that research productivity garners greater rewards than strong teaching, commonly chose research-oriented activities that led to publication over teaching-related professional development activities. For example, several academics in this study declined to participate in workshops at the CTL, formative peer reviews, and ESEd activities. Others felt they were unable to pursue their own course improvements to the extent they wanted because of high demands on their time and the need to be a productive researcher: “I can’t [make the course modifications], and I’m overwhelmed, and there’s no freaking way,” lamented Ledonne (S1tF) as she talked about “wanting to be a better teacher but being very, very invested in research.” Maki (A3aF) referred to teaching as a “black hole.” As with black holes in space, from which nothing—not even light—escapes, teaching resembles a black hole because the outcomes of one’s effort are unclear. Irrespective of the creativity, time, and resources an instructor devotes to designing and teaching a course, students may be discontent, course objectives may not be met, and scores from student evaluations of teaching may be low. Furthermore, and as described in Chapter 4, faculty members questioned the extent to which outcomes from the summative peer reviews of teaching influence decisions about tenure, promotion, and merit; consequently, their engagement in this process was limited. Given competing demands between research and  173 teaching, and an awareness that a strong research record is favourable in decisions about tenure, promotion, or merit, faculty members opted for activities that would reward them career-wise.  The above findings are in line with studies that have demonstrated that in research-intensive universities, which are positioned at the highest end in the hierarchy of post-secondary institutions, faculty members spend more time on activities related to research and publication than those related to teaching (Link et al., 2008; Milem et al., 2000). Though this study did not track how faculty members allocated their time, many participants spoke about limiting their involvement in teaching-related activities. One reason they prioritized research over teaching is that research productivity is rewarded over teaching (Chalmers, 2010; Serow, 2000). Thus, as confirmed in the literature, the academic reward system acts as a major source of extrinsic motivation (O’Meara, 2011) in influencing how faculty members use their time (Fairweather, 2005; Link et al., 2008). This structural entity, which rewards or penalizes, works to socialize and shape faculty members’ behaviours and is an important cultural force in faculty careers (O’Meara, 2011; Tierney, 1997).  Even though all faculty were clear that research garners greater rewards, several participants expressed beliefs, ranging from tentative to firm, that teaching counted in career decisions. To some extent, this came about because participants identified programs, such as a multimillion dollar initiative to reform undergraduate science (ESEd) and the creation of a community service learning unit, that demonstrated concrete support for teaching by UWC’s senior administration. Many also pointed out that the summative peer review of teaching process had become more formalized over time, and presumed the impetus for this was tied  174 to directives from senior administration. Teaching and student learning also figure prominently in the UWC strategic plan and collective agreement with the Faculty Association. The message touted by UWC is that teaching counts. Its push for teaching is part of a broader trend whereby, in the past two decades, “considerable resources and intellectual attention” (McFarlane, 2011, p. 127) have been devoted to raising the status of teaching in higher education. Still, a number of participants felt that UWC’s claim about valuing teaching was vacant because they were not receiving time and resources to develop professionally in their teaching.  The university’s emphasis on teaching also meant that teaching was being increasingly monitored. Many participants noted that the demands associated with the summative peer reviews had increased compared to three to five years prior to the interviews, and several pointed out that the requirements for the teaching component of a tenure file had “inflated” (Wilks, S1tF). Some faculty members resented the additional demands on their time, especially when they perceived little benefit in terms of their growth and development as teachers. In addition, numerous faculty members in Science talked about how the student evaluations of teaching were used to graphically compare faculty members or courses to one another, another monitoring practice. In Arts as well as in Science, the use of student evaluation of teaching scores to compare people to each other or a department average could create intense stress or a sense of competition. In the literature, these monitoring practices are commonly associated with an accountability agenda in post-secondary institutions where increasingly, faculty members must provide evidence of their performance in all areas, in particular their teaching (Gosling & D’Andrea, 2001; Shanahan, 2009).  175  5.6.2 Departmental Culture In spite of predominant institutional pressures and constraints, faculty members still made choices about how to allocate their time and found ways to cultivate pedagogical interests and commitments. In doing so, they exercised agency over their professional learning activities (Jawitz, 2009; Neumann, Terosky, & Schell, 2009; Serow, 2000). However, factors like motivation, awareness of self, and will were only partly responsible for determining how and if a faculty member pursued her/his instructor role. As discussed elsewhere in this dissertation, institutional culture also shapes faculty members’ engagement in professional growth activities. In addition, the departmental culture, where faculty members participate in routine (and less routine) activities, affects opportunities for growth and learning about teaching.  Jawitz (2009) uses the term “harmonization” to describe a process whereby academics, as they engage over time in workplace practices, adjust how they act upon their agency. Individuals’ choices and motivations with regard to their teaching thus shift as faculty members participate in departmental activities both with peers and by themselves. The findings from this study show that, in a department where teaching is considered a legitimate and worthwhile academic activity, as evidenced by the presence of both formal and informal practices that support teaching, faculty members committed to this aspect of their work are encouraged to discuss teaching, pursue it in creative and diverse ways, and take risks. In some instances, a strong departmental culture of teaching can outweigh an institutional culture that affirms the dominance of research. For example, Wilks (S1tF) made major changes to a course and invested the time to do so because she wanted to improve that course and because she knew it would be “positively seen” by her department, given that they “place  176 some emphasis on teaching.” She was confident that when it came to tenure, her peers would positively evaluate the time and energy she had put into modifying her course. Dawson (S3aM) too invested considerable hours (which he could presumably have devoted to research) to redesign his course. He received departmental support from an ESEd teaching fellow; even without the support, Dawson claims he would have gone ahead with it anyway, because he was convinced that the teaching methods would improve his students’ learning. In both cases, as well as numerous other situations described by participants, it was apparent that the extent to which academics involved themselves in teaching activities and practices depended both on individual and contextual factors: self-will was combined with perceived departmental endorsement and/or concrete departmental resources.  While some departmental practices supported professional growth, others did not. For instance, Warr (S2FM) pointed out that, summative peer review reports in his department are not shared with the candidate, a practice he thought was odd and “stupid,” but which he nevertheless adopted. In addition, several faculty members rarely, if ever, discussed teaching with their mentors. One participant (Cohen, A1aM) felt that doing so might (wrongly) signal an insufficient commitment to research, reasoning that if the mentor-mentee relationship is meant to focus on research (a message most participants had internalized), it was best not to emphasize topics such as teaching. This illustrates how an interest in teaching may be adjusted (or harmonized, to use Jawitz’s term) to match collective departmental practices that run counter to professional growth in teaching and/or asserted the dominance of research. As affirmed in previous studies, faculty members engage with the understanding and assumptions in their own department and, from these, draw inferences about what counts and  177 how to pursue their work (Jawitz, 2009; Knight et al., 2006; Tierney, 1997; Trowler & Knight, 2000).  One who played an important role in fostering practices that could signal the value (or lack thereof) of teaching was the department head. When a department head paid attention to teaching by discussing it in an annual review, investing time and resources for a departmental retreat on teaching, and/or requiring new hires teach as part of their job application process, these practices indicated to faculty members that teaching was an important aspect of their work. The opposite was also true: when a department head neglected to discuss strong evaluations of teaching with individual faculty members, or advised those with an interest in teaching to focus on research, or inconsistently administered the teaching evaluation process, this signaled a low value was placed on teaching. These findings are consistent with the literature, which has shown that the leadership of a department head is key in determining whether and how a department supports teaching (Anderson et al., 2008; Braxton, 2008; Gibbs et al., 2008; Knight & Trowler, 2000; Massy et al., 1994).  5.6.2.1 Learning Conversations One of the most common ways that academics engaged with various departmental understandings and assumptions was through conversation. A pervasive feature in all informal and formal departmental practices that fostered a culture of teaching was how they promoted collegial conversations about teaching. As participants described, these came about in numerous ways. Many—including those that contributed most to their learning—came about informally, in an impromptu fashion. Conversations were also prompted when • new faculty members outwardly questioned existing teaching assumptions and  178 practices; • colleagues worked on campus; • peers collaboratively developed their course and/or team-taught; • faculty members discussed teaching in the hiring process; • departments created teaching-related policies; and • colleagues engaged in dialogue about the peer review of teaching. Based on the study findings, I propose that a main signifier of a culture that values teaching is the presence of conversations that permit faculty members to grow and learn as teachers. The importance of conversations to professional growth has been highlighted by scholars like Byrne et al., (2010), Haig (2005), and Roxå and Mårtensson (2009).  Ultimately, conversations about teaching most often took place in an environment where peers valued and respected one another’s teaching approach and pedagogical knowledge. Trust and a non-threatening climate were key. For example, in the context of summative peer reviews, when the results were potentially punitive, few learning conversations ensued. When, on the other hand, faculty members informally approached colleagues with a teaching challenge or question, learning conversations were more likely to occur. Notably, a lack of consensus about teaching among peers promoted a nonjudgmental climate in departments where relationships were perceived to be collegial. In such departments, faculty members were inclined to believe that regardless of their teaching approach and philosophy, they would be evaluated fairly by their peers. Some noted, however, that in departments where professional and personal rivalries prevailed, the absence of consensus could be problematic: if a candidate in the peer review of teaching did not share the same approach as his/her  179 reviewer and had a strained relationship with that person, the person reviewed might legitimately fear a poor evaluation from a reviewer who was “out to get” him or her. Within a unit, the existing relationships must be conducive to people having conversations.  5.6.3 Disciplinary Culture and Conversations A last finding I wish to examine is the connection between disciplinary culture and collegial discussions of teaching. I reported earlier that when compared with faculty members in Arts, those in Science collaborated more frequently on teaching. In the Science departments, many multi-section courses were jointly developed by several faculty members and implemented with the help of a course coordinator. In addition, academics more often spoke of discussing, debating, and sharing course goals, and team teaching was common. The sense that “no one owns a course” (Woodroof, S2TM) was stronger than in Arts, where faculty members generally developed courses individually and pursued teaching on their own.  These differences confirm that academics in the hard-pure disciplines engage more frequently in collaborative practices than their soft-pure counterparts (Becher & Trowler, 2001; Neumann et al., 2002; Umbach, 2007). A collaboration may be facilitated by the fact that curricula in hard-pure disciplines tends to be linear and hierarchical and comprised of courses that are tightly structured with closely connected concepts and principles (Neumann et al., 2002). When faculty members agree on how courses fit together and why, more frequent conversations about teaching may ensue. On the other hand, in the soft-pure disciplines, where content tends to be “more free-ranging and qualitative, with knowledge- building a formative process” (Neumann et al., p. 408) and courses are more loosely structured (Lattuca & Stark, 1994), faculty members may be less inclined to collaborate, not  180 just because courses are less interdependent but also because collaborative conversations may be seen as time-consuming and spur conflict among peers who disagree on course goals and the ways to achieve them.  In the final chapter of this dissertation I draw on the findings and analyses in Chapters 4 and 5, and from relevant scholarship, to make recommendations for theory, research, and practice.   181  Chapter  6: Summary, Conclusions, and Recommendations This chapter provides a brief summary of the study, a summary of key findings, and a discussion of this study’s contributions to theory. Following the discussion, I consider possible applications of the study findings to summative peer review and to departmental practices that foster a culture that values teaching. In particular, I focus on what improvements in peer review practice might support professional growth in teaching among tenure-track faculty members working in research-intensive universities. I end by outlining suggestions for future research.  6.1 Summary of the Study In what follows, I present a summary of the study including a statement of the research problem and the research questions; a short description of the conceptual framework and research methodology; an empirical summary of the key findings from this research; and, the study limitations.  6.1.1 Statement of the Problem In research-intensive universities, where research productivity is rewarded over teaching and escalating demands impose significant stress on academics (Gappa et al., 2007; Robinson, 2006), faculty members commonly limit their involvement in professional development activities for the improvement of teaching (Kilgore & Cook, 2007). Yet teaching is a chief responsibility in academic careers and central to faculty members’ work. As such, ongoing professional growth in teaching is both necessary and helpful to faculty members as they work to fulfill their professional responsibilities and enhance their teaching roles (Gappa et  182 al., 2007; O’Meara et al., 2008). Prior research has confirmed that a great deal of faculty members’ professional learning about teaching occurs as they participate, within their department, in informal teaching-related activities and other practices that do not intentionally aim to improve teaching (Jawitz, 2009; Knight et al., 2006). These practices, therefore, are especially relevant to a study of faculty members’ professional growth in teaching.  In North American institutions, one teaching-related departmental practice commonly used to evaluate teaching in tenure and promotion decisions is summative peer review of teaching (Gravestock & Greenleaf, 2008). The extent to which summative peer review can contribute to professional growth has been questioned in the literature. Several scholars (e.g., Kell & Annetts, 2009; Peel, 2005; Shortland, 2004) have tied summative peer review to the accountability agenda in higher education which serves to monitor or maintain standards. The summative process is frequently compared to formative peer review, which is said to improve teaching by promoting reflection on, and dialogue about, teaching (Hammersley- Fletcher & Orsmond, 2004). In practice, the formative/summative distinction is not that clear and, consequently, it is preferable to think of peer reviews as existing on a continuum. When one does so, it is conceivable that summative peer review could contribute to professional growth in teaching. However, little research has investigated this aspect of summative peer review in the North American research-intensive university.  In this dissertation I set out to examine faculty members’ experiences of summative peer review and their understandings of how that process contributes to professional growth in  183 teaching. Since summative peer review occurs only periodically in faculty members’ careers, and most faculty members do not perceive it as being conducive to professional growth, a second main objective of this study was to explore other departmental practices that contributed to a culture that values teaching and might, therefore, help faculty members grow in their roles as teachers.  6.1.2 Research Questions The purpose of this research was to explore practices that encourage professional growth in teaching among tenure-track faculty members working in a research-intensive university. The research questions were as follows: 1. What are faculty members’ experiences of summative peer review of teaching and how do they understand the relationship between summative peer review and professional growth in teaching? 1a.  In what ways do the multiple purposes of summative peer review play out and interconnect? 1b.  How do academic values and norms, as well as other established practices in higher education, influence the summative peer review of teaching process? 2. What existing departmental practices support or hinder a culture that values teaching?  184  2a.  How do aspects of institutional and disciplinary culture influence departmental practices that support/hinder a culture that values teaching? 2b.  What is the relationship between departmental practices that foster a culture that values teaching and faculty members’ professional growth in teaching?  6.1.3 Conceptual Framework This study investigated the issue of faculty members’ professional growth in teaching through the lens of academic culture. I drew on scholarship concerning the peer review of teaching and from previous research on institutional, departmental, and disciplinary cultures to develop and conduct the study. O’Meara et al.’s (2008) framework for professional growth in faculty careers provided a useful way of thinking about how faculty members learn and grow in their professional roles.  6.1.4 Study Participants and Data Collection Thirty tenure-track faculty members working at a large, research-intensive Canadian university took part in the study. Half the participants were from the Faculty of Arts and the other half were from the Faculty of Science. For each Faculty, participants were selected from three departments representing hard-pure disciplines (2), hard-applied disciplines (1), and soft-pure disciplines (3). Through the administration of a questionnaire, study participants were purposefully selected to get a range of rank (8 pre-tenured and 22 tenured) and gender (13 females and 17 males).   185 Using a semi-structured interview protocol, I interviewed participants once, either on campus or at a location of their choice, for approximately one hour. Participants were asked to reflect on their experiences of summative peer review: as a candidate, reviewer, or both. I first inquired about the departmental process of peer review and their involvement in and experiences of that process, and about their notions of an ideal peer review process. Subsequent questions focused on what good teaching looks like in their department, how faculty members learn the criteria for good teaching, and departmental support for teaching. These questions generated information about faculty members’ experiences and understanding of the summative peer review and particulars about teaching-related departmental practices.  6.1.5 Data Analysis The interviews were transcribed and subsequently reviewed by myself and those participants who wished to do so. I then developed initial codes that I refined, and sometimes redefined, as I coded the interviews using Atlas.ti. I clustered codes to form themes and sub-themes that helped me make connections between participants and ideas. This process was also facilitated by keeping a research journal in which I recorded reflections. Data analysis continued as I wrote the dissertation.  6.1.6 Empirical Summary Faculty members’ experiences of summative peer review varied, as did their expectations of the purposes and outcomes of the process. Participants built their understandings of peer review based on limited experiences. These may have included experiences of being  186 reviewed, reviewing colleagues, participating in standing committee meetings (where peer review reports may or may not be discussed), and being in meetings with their department head (during which an individual’s peer review reports were rarely given attention). The existence of a university-level document on summative peer review appears to have had little impact on practice or faculty members’ understanding of peer review. In the one department where there was a peer review policy, details of that policy were not known to all participants, and its impact on practice was questionable. People’s individual experiences of peer review still varied a great deal.  Despite the variation and inconsistency in experience, participants identified three purposes for summative peer review: a formal mechanism for evaluation, a means of promoting professional growth in teaching, and a supplement to the student evaluations of teaching. Faculty members did not think that the summative peer review process was successful in achieving the first two objectives, and cited unsuitable reviewers (where lack of suitability was defined as being a poor teacher, and/or having insufficient content/pedagogical knowledge, and being untrained on how to evaluate teaching), ambiguous or nonexistent criteria, the episodic nature of reviews, and few (or no) guidelines for conducting reviews as possible reasons for the lack of success. Furthermore, the majority of participants believed summative peer review failed as an evaluative tool because, firstly, faculty members are unwilling to document suggestions for teaching improvement for fear that these might have a negative effect in tenure and promotion decisions, and secondly, more attention and weight are given to the student evaluation of teaching results.  As it concerned the potential of summative peer review to contribute to professional growth  187 in teaching, participants were of the opinion that this objective was seldom met because of the poor quality of feedback. That is, in most cases the candidates and reviewers did not discuss feedback; in the rare cases when they did, this was usually unspecific and therefore unhelpful to the candidate. Many faculty members expressed disappointment at the absence of feedback from the summative peer review process. Nevertheless, several suggested that the process should not seek to meet multiple objectives because the evaluative functions conflicted with the formative aims of summative peer review.  Although summative peer review, as it is currently practiced, makes minimal or questionable contributions to faculty members’ professional growth as teachers, there are numerous departmental practices that contribute to a culture that values teaching and promote academics’ growth and learning in their roles as instructors. They include formal practices that specifically aim to enhance teaching such as lunchtime seminars on teaching, formative peer reviews, and teaching retreats. More often, however, they consist of informal practices such as conversations among peers about teaching and beliefs about valuing diverse notions of good teaching.  The unplanned learning conversations, which contribute to faculty members’ growth as instructors, normally occur between trusted colleagues who work on campus and value each other’s knowledge and approaches to teaching. They are facilitated by the presence of people who are assigned to positions that emphasize teaching, such as course coordinators, undergraduate chairs, and tenure-track instructors. The conversations are also influenced by disciplinary cultures, in particular whether there is an established tradition of working  188 collectively or individually. Additionally, the department head is an important individual when it comes to establishing a culture that values teaching and where there are frequent collegial conversations about teaching. S/he does so by encouraging collegial relationships among faculty members, appointing committees to address specific departmental teaching issues, discussing teaching with individual faculty members in one-on-one meetings, and supporting faculty-led initiatives for the enhancement of teaching. Faculty members’ commitment to teaching, along with how they exert their agency within their department, also influence their engagement in the various practices that promote growth and learning.  Finally, faculty members’ engagement in summative peer review and other departmental practices that contribute to a culture that values teaching are strongly shaped by the institutional culture. Participants unanimously agreed that research productivity garners greater rewards than teaching and because of this and the high demands on their time, the majority limited the time they spent on teaching (broadly defined) in order to focus on research and publication. However, some wondered about the extent to which teaching counts in tenure and promotion decisions. In a few cases, faculty members invested large amounts of time to redesign a course because they perceived strong departmental support for teaching and reasoned that the time they devoted to teaching would be favourably evaluated in tenure and promotion decisions. Though they acknowledge that the university is placing greater attention on teaching now as compared to the past, several expressed cynicism at the university’s claims that teaching matters.  189  6.1.7 Limitations There are several limitations to this study. First, because the study examined faculty members in a single research-intensive university, it does not consider the experiences of tenure-track faculty members at other types of post-secondary institutions. Especially in institutions where there is more emphasis on teaching and less research activity, one would expect to find differences in how faculty members pursued their professional growth in teaching. Second, faculty members were from a small number of departments in only two faculties and, with the exception of S3, participants described theirs as being strong teaching departments with a collegial culture. Academics from a wider range of departmental cultures, and in particular departments with a weaker culture of teaching, may have had very different experiences than those who volunteered for this study. Third, I assumed that participation in departmental teaching practices would promote professional growth in teaching, but did not measure professional growth in teaching nor observe faculty members’ participation in departmental teaching-related practices. Data collection was based only on participants’ perceptions and self-reported data. Fourth, I made assumptions about the Biglan-Becher typology to which faculty members belonged on the basis of the department they were in. This is because, with one exception, the departments represented in this study were frequently cited as classic examples of that typology in the literature, i.e. “education” is a classic example of a soft-applied discipline. Had I attempted to identify to which specific category (hard-pure/hard-applied; soft-pure/soft-applied) each faculty member should belong, I might have noted different disciplinary trends overall. Fifth, the sample size was relatively small and therefore the study results may not reflect the experiences of a wider  190 range of faculty, in particular, assistant professors and other underrepresented faculty members, including women and faculty members of varying ethnicity and race. Finally, the fact that I was the only person coding the data presents a limitation to the reliability of this study; had there been other people coding, I might have picked up on additional nuances and themes in the data.  6.2 Contributions to Theory A primary purpose of this research was to investigate the issue of faculty members’ professional growth in teaching through the lens of academic culture. Chapter 1 sets forth a preliminary conceptual model (Figure 1) that draws on the literature of institutional, departmental and disciplinary cultures and from O’Meara et al.’s (2008) framework for professional growth in faculty careers.  Based on the findings of this dissertation research, I made several important changes to the preliminary conceptual framework. Notably, I propose two separate conceptual models for the study of faculty cultures and professional growth in teaching, one that pertains to the evaluative practice of summative peer review of teaching, and the other that is more relevant to non-evaluative departmental practices. Whereas the preliminary model did not identify specific aspects of institutional, departmental, and disciplinary culture at play in the relationship between culture and professional growth, the revised models in Figures 2 and 3 do. Thus, the reason for developing two separate models was to highlight the different aspects of academic culture that dominate and manifest in the relationship between summative peer review and professional growth, and in the relationship between non-  191 evaluative departmental practices and growth.  These modifications, which are intended to strengthen the usefulness of these models in future research, are outlined in the sections below. They are also summarized in Table 9.   Figure 2: Model for Faculty Member Engagement in Summary Peer Review of Teaching and Professional Growth in Teaching (Model A)   192    Figure 3: Model for Faculty Member Engagement in Non-Evaluative Departmental Practices that Foster a Culture that Values Teaching and Professional Growth in Teaching (Model B)   6.2.1 Description of the Two Conceptual Models Prior to describing each model and outlining their differences, it is worth pointing out their similarities. First, both models illustrate how institutional, departmental, and disciplinary cultures influence the relationship between departmental teaching practices (evaluative and non-evaluative) and faculty members’ professional growth in teaching. These cultures have been placed at the outer limits of each model, with arrows penetrating the inner boxes, to depict their continuous role in shaping faculty careers. Second, the models portray summative peer review and other departmental practices (the innermost box) occurring within a context where academic norms and values constantly operate. These norms and  193 values are defined by broader overlapping institutional, departmental, and disciplinary cultures. Though values and norms may manifest differently depending on context, they are omnipresent and permeate all aspects of faculty members’ selves and work. Third, models A and B identify the values of autonomy and collegiality as being especially relevant to faculty members’ engagement in departmental practices that may foster professional growth in teaching. Finally, the models illustrate that relationships, agency, commitment, and learning (the four lenses used to explore faculty members’ professional growth in O’Meara et al.’s 2008 conceptual framework) influence and mediate the interactions between departmental teaching practices and professional growth in teaching. That is, faculty members’ growth as instructors may be supported or hindered by the quality of their professional relationships (with colleagues, students, the community and others), their commitment to learning as it pertains to their teaching role, and how they act upon goals that matter to them.  6.2.1.1 Model for Faculty Member Engagement in Summative Peer Review of Teaching and Professional Growth in Teaching (Model A) This model integrates the study findings with other research on faculty cultures to offer a framework for the relationship between professional growth in teaching and summative peer review of teaching.  6.2.1.1.1 Institutional, Departmental and Disciplinary Cultures Model A suggests and identifies specific cultural aspects within institutional, disciplinary, and departmental cultures that have a strong influence on whether and how faculty members grow and learn as instructors through summative peer review. For instance, in an institutional  194 culture where research is rewarded over teaching, faculty members are more inclined to select activities that will augment their research productivity and, as a result, they may not invest time to converse with one another about their experiences of teaching during a summative peer review process. Likewise, in an institutional culture where the student evaluations of teaching are given more attention than peer reviews, faculty members may dismiss peer review of teaching as a formality, and consequently, their engagement in summative peer review may be diminished. Both these aspects of institutional culture hinder faculty members’ opportunities for learning from peer review.  As it pertains to departmental cultures, Model A illustrates how collegial relationships and leadership from the department head shape the extent to which summative peer review does or does not promote growth and learning about teaching. For example, when the reviewer and candidate respect and value each other’s knowledge and teaching approach, they may be more inclined to discuss the peer review experience; the dialogue can contribute to mutual reflection and growth. Along similar lines, if the department head were to use peer review results to help the candidate identify areas of growth in teaching, and acknowledge areas of strength, s/he would signal that the reviews are relevant to a discussion of career development. In these examples, collegiality and leadership in the department support faculty members’ learning through summative peer review.  Finally, as it concerns disciplinary cultures and the summative peer review of teaching, the aspect that appears to have most bearing on professional growth in teaching is the extent to which faculty members desire criteria/standards/objectivity or are comfortable without it. As previously described in this dissertation, the norm of universalism, which relates to the  195 assumption that truth claims will be made on the basis of pre-established, impersonal criteria, is particularly strong in the Sciences. It is not surprising, then, that many participants in the Science departments criticized peer reviews as being highly subjective, and wanted clearer criteria for conducting peer reviews. These faculty members said they would feel better equipped to conduct the reviews and have more confidence in their results if they made use of clear criteria for the evaluation of teaching. This leads me to conclude that they would engage more in the process and consequently learn more from the experience.  In Arts, more faculty members were comfortable conducting reviews without pre-established and detailed criteria.  6.2.1.1.2 Academic Values and Norms Just as there are dominant elements of institutional, departmental, and disciplinary culture in effect in the relationship between growth and summative peer review, so there are particular ways that academic values and norms are manifested.  Autonomy: The study findings indicate that some academics feel uncomfortable when a peer observes their classroom teaching. This may be because peer review disrupts the tradition whereby teaching is a private activity and may threaten a faculty member’s sense of autonomy (Chism, 2007a; Hutchings, 1996).  An element of autonomy that is conducive to growth through summative peer review is an acceptance and valuing of diverse teaching approaches. When faculty members recognize and appreciate that “good teaching” is varied, it provides them with greater opportunity to  196 learn from the peer review process. Furthermore, there is less risk that the candidate will be evaluated poorly because s/he and the reviewer have different notions about what constitutes good teaching.  Collegiality: In the summative peer review of teaching, collegiality is preserved when peers refrain from sharing critical and/or constructive feedback in verbal exchanges post-review and in any formal documentation about the peer review outcomes. In so doing, faculty members miss out on the opportunity to collectively reflect on the summative peer review experience and, consequently, do not benefit from the learning that would be derived from those conversations.  6.2.1.1.3 O’Meara et al.’s Framework for Professional Growth This study demonstrated that, in summative peer review, professional relationships—more so than commitments and agency—strongly influenced the potential for the process to contribute to professional growth. O’Meara et al. describe that interactions among colleagues can “stimulate, facilitate, and shape learning” (2008, p. 29). Based on my findings, the characteristics of the professional relationship especially conducive to learning came when: • A candidate values a reviewer’s (observer, member of the peer review committee, department head) pedagogical and/or content knowledge; • Candidate and reviewer respect each other’s approach to teaching; • A candidate does not perceive that the reviewer is “out to get” him/her; and • Reviewer and candidate discuss teaching, and reflect on the peer review process. However, in the context of summative peer review, the findings also indicated that  197 interactions among colleagues could hinder faculty member learning. The potential for learning might be diminished by the absence of the above characteristics or, as described previously, be reduced because faculty members believed that a more certain way of nurturing and sustaining professional interactions is by refraining from sharing constructive feedback which could, potentially, contribute to growth in teaching. In either case, we see that, as identified by O’Meara et al., professional relationships play a key role in faculty members’ learning.  6.2.1.2 Model for Faculty Member Engagement in Departmental Practices that Foster a Culture that Values Teaching and Professional Growth in Teaching (Model B) Model B in Figure 3 integrates the findings from this study with other research on faculty cultures to offer a framework for the relationship between non-evaluative departmental practices that foster a culture that values teaching and faculty members’ professional growth in teaching. Like Model A, this model identifies specific aspects of culture that play out and determine how faculty members engage in departmental practices that can promote professional growth in teaching.  6.2.1.2.1 Institutional, Departmental and Disciplinary Cultures As in Model A, Model B illustrates that the institutional reward system has a strong bearing on how faculty members pursue teaching-related activities. Knowing that research productivity is favoured over teaching in tenure and promotion decisions, faculty members may limit their involvement in activities that can foster their professional growth in teaching. Nevertheless, those committed to their teaching (like the study participants) find ways to  198 pursue activities that promote their growth as teachers. Involvement in such activities is facilitated when the institution demonstrates its commitment to teaching by providing resources at the faculty and department levels for the improvement of teaching.   The departmental culture also matters for faculty members’ professional growth. Specifically, actions by the department head influence the range of teaching-related practices available and how faculty members perceive them. For instance, a department head can demonstrate that teaching is valued by holding a department retreat focused on teaching, establishing a process for formative peer review, and/or setting up teaching and learning committees or initiatives to address specific issues. In addition, the quality of the relationships among departmental peers has a bearing on how faculty members engage in practices that foster a culture that values teaching. Professional learning conversations are frequent in departments that faculty members describe as collegial and where peers accept varying notions of good teaching. As before, it is interactions between departmental peers and leadership by the head that appear to be most relevant to the relationship between practices and professional growth in teaching.  With respect to disciplinary culture, professional growth is largely affected by whether there is an established tradition of working collectively on teaching issues or whether faculty members tend to pursue their teaching endeavours individually. Because faculty members indicated that they learn a great deal about teaching through conversations with their peers, faculty members who belonged to disciplines with an established tradition of collaboration tended to benefit more often from collective conversations about teaching.  199  6.2.1.2.2 Academic Values and Norms Autonomy: As in Model A, acceptance of and valuing diverse approaches to teaching promotes growth in teaching. In addition, the tradition of working collaboratively or individually on teaching also influences the opportunities for conversations about teaching and the learning that comes from these.  Collegiality: When faculty members participate in non-evaluative departmental practices, they engage in learning conversations about teaching; these foster collegiality among departmental peers. These conversations happen most often between colleagues who trust one another and who value each other’s knowledge and approach to teaching; one might conclude, therefore, that collegial relations must be present in order for faculty members to engage in learning conversations. Collegiality, in Model B, promotes learning through dialogue about teaching.  6.2.1.2.3 O’Meara et al.’s (2008) Framework for Professional Growth Model B illustrates that collegial professional relationships within a department are important in determining faculty members’ engagement in practices that can foster professional growth in teaching. However, unlike Model A, commitment and agency also figure prominently in non-evaluative practices. The findings from this study indicate that faculty members learn and grow professionally as instructors because they are committed to their teaching and find ways to pursue their teaching goals by being involved in a range of departmental practices. This is consistent with O’Meara et al.’s framework that identifies agency, commitments, relationships, and learning as key aspects contributing to professional growth. In the context  200 of summative peer review, agency and commitment manifested less strongly.  Table 9 summarizes which cultural elements dominate in each of the contexts that potentially promote professional growth in teaching. Table 9: Dominant Element of Culture at Play in Models of Professional Growth in Teaching  Cultural Element Growth through Summative Peer Review (Model A) Growth through Non- Evaluative Departmental Practices (Model B)  Dominant aspect Institutional Culture Reward system Student evaluations of teaching Reward system Tangible resources that support teaching Departmental Culture Collegiality Leadership from department head Collegiality Leadership from department head Disciplinary Culture Universalism Individual vs. collaborative approaches to teaching Autonomy Diverse approaches to teaching accepted/valued Teaching as private  Diverse approaches to teaching accepted/valued Individual vs. collaborative approaches to teaching Collegiality Preserve collegiality by not sharing/documenting constructive feedback Foster collegiality through professional learning conversations O’Meara et al.’s (2008) framework for professional growth Professional relationships Professional relationships Agency Commitment Learning   201 6.3 Recommendations for Practice In this section, I make recommendations directed at three areas of practice: enhancing summative peer review, fostering a department culture of teaching and learning, and increasing adoption and full integration of improved teaching practices. The recommendations are summarized in Table 10.  6.3.1 Summative Peer Review of Teaching This study demonstrated that, in the opinion of the participants, summative peer review is neither a useful process for the evaluation of teaching nor for fostering professional growth in teaching. These findings point to the need to improve both functions of summative peer review so that the value of the process is enhanced for academics and administrators alike.  There are multiple levels at which the value of peer review could be reinforced and countless strategies which, if implemented, would result in an improved process. In Appendix H, I present a range of suggestions for an ideal summative peer review of teaching process derived from the study findings. Here, I make recommendations pertaining primarily to the role of the department head and to feedback practices because they are particularly salient to the issue of professional growth. 	
   6.3.1.1 Involve the Department Head The department head, through her/his actions, can communicate strong messages about the value (or lack thereof) of summative peer reviews. S/he can endorse peer review as a meaningful process for evaluation and professional growth when:  202 • s/he discusses the summative peer review outcomes with the candidate in a face-to- face meeting, • s/he demonstrates that both the peer reviews and the student evaluation of teaching scores are given serious consideration in the evaluation of teaching; and • s/he initiates a process whereby faculty members collectively produce departmental guidelines for peer review. Although I name the department head in the following recommendations, I maintain that others in leadership roles, such as the associate head, could substitute for the department head and similar benefits would still ensue.  Recommendation 1: The department head discusses peer review results with the candidate. The majority of the participants had never discussed the peer reviews with their department head. In a few cases the head had briefly referred to peer reviews in an annual meeting, but none of these participants had benefited from meaningful conversation about the results. The absence of dialogue about peer reviews in private meetings and departmental meetings was commonly interpreted by faculty members to mean that the peer reviews were of little to no significance in the evaluation of teaching or for growth.  The department head can signal that peer reviews matter for evaluation and growth when s/he discusses the results with the candidate at an annual meeting or shortly following the peer reviews. Discussions that include an explanation of how the peer review results will be used in tenure, promotion, and merit decisions would help the candidate better understand how the  203 reviews are used in career advancement decisions. If, in addition, the department head asks the candidate to formulate and share a professional growth plan for teaching that considers peer review results and those from the student evaluations of teaching, the candidate would see that this process also serves to promote individual growth.  Recommendation 2: The department head considers peer reviews and student evaluations of teaching more equally. Whether at departmental meetings, standing committee meetings, or in private interactions between the department head and candidate, most participants agree that when it comes to the evaluation of teaching, administrators (and consequently faculty members themselves) give greater consideration to student evaluations than they do peer reviews. Student evaluation of teaching scores are preferentially used over peer reviews to gauge a candidate’s success in teaching and/or to judge how the department, on average, fares in comparison with the faculty as a whole.  Department heads can emphasize the relevance and value of peer reviews by referring not only to the student evaluation of teaching results but also to the reviews in individual and collective discussions about the evaluation of teaching. The head could conceivably gather information from all the peer reviews within a department, anonymize it, and present it to the faculty members so they could then identify issues that need to be addressed at the department level. Dissemination of peer review results would help faculty members appreciate that peer reviews can have broad application and that the process is linked to wider educational development efforts (Hammersley-Fletcher & Orsmond, 2004).  204  In making the above suggestions, I recognize that in order for department heads to shift greater attention to peer reviews, they need to believe that senior administrators and other non-departmental faculty members charged with reviewing a candidate’s file will also value information from the peer reviews. In the current culture, where student evaluations of teaching have strong currency, it is not surprising that a department head who seeks to make a solid case for a candidate will want to communicate using measures (e.g., student evaluations of teaching) that can be easily understood and interpreted by other evaluators.  Recommendation 3: Department head clarifies expectations and enhances awareness of summative peer review process through departmental guidelines. Many participants did not know what to expect from the peer review process or felt their expectations were unmet. The department head can clarify expectations by appointing a committee to develop guidelines for the summative peer review of teaching. These guidelines would help reviewers understand their duties and let candidates know what to expect. They might include such things as how reviewers are selected, timing of reviews, what will be reviewed, and the purpose of any communication, pre- and post-review. Guidelines should also include criteria for good teaching so that evaluators and candidates know what is being evaluated and with what standards. These criteria exist in the literature, but in order to be useful, they must be adapted to a department’s particular values (Chism, 2007a) which may be uncovered via departmental conversations about good teaching. In departments where faculty members resist using a structured form (with criteria) in the peer review process, only broad criteria would be included. To have a greater chance of being accepted, drafts of these  205 guidelines should be circulated among faculty members for feedback and later be presented in department meetings for final acceptance.  The benefits of these criteria and standards would be lost, however, if faculty members were not aware of them or did not use them in the peer review process. Consequently, the department head plays an important role, not only in initiating discussions that result in the creation of guidelines, but also in ensuring that the guidelines are accessible (e.g., posted on the departmental website, included in a policy manual, added to resources given to new faculty members and mentors, circulated via email) and referred to regularly in conversations about teaching.  6.3.1.2 Improve the Feedback Process The findings from this study indicate that if summative peer review is to be used for the enhancement of teaching, the feedback process must be improved.  Recommendation 4: Share written and/or verbal feedback with the candidate. In many cases, participants did not receive feedback on their teaching from either reviewers or their department head. While it is true that the peer review process can prompt self- reflection which leads to teaching improvement (Hatzipanagos & Lygo-Baker, 2006; Lomas & Nicholls, 2005), research has shown that feedback is more effective when the information comes from others as well, and is communicated in a variety of  ways (e.g., written and verbally) (Brinko, 1983; Harms & Roebuck, 2010).  206  Recommendation 5: Ensure that feedback is delivered by a credible source. The feedback literature reports that in order to be effective, feedback is best delivered by someone who is perceived as credible, where credibility encompasses being “knowledgeable enough to make an accurate judgement” (Brinko, 1993, p. 577). The degree to which this is possible relies partly on what the candidate values in a reviewer. The credibility of a reviewer would likely be enhanced if s/he • were familiar with and used departmental criteria for evaluating good teaching • were perceived by the candidate as a good teacher • had content knowledge appropriate to the course being reviewed • had training in peer review and strong communication skills.  Recommendation 6: Explain the value of receiving feedback on content and teaching approaches. Reviewer feedback on both the teaching process and content may help a candidate grow and learn through peer review (Kell & Annetts, 2009; Toth & McKey, 2010a). In this study, some participants clearly valued pedagogical knowledge over content knowledge, whereas others valued the latter over the former. The candidates and reviewers should be educated about the relative merits of receiving feedback on content and teaching approaches so that they can understand—and therefore better appreciate—the value of each.  207  6.3.2 Fostering a Departmental Culture that Values Teaching Recommendation 7: Build and maintain a collegial culture. Participants’ narratives pointed out that they were more likely to converse about teaching and learning with peers whom they trust and whose opinions they value. This was true whether the conversations were with an officially appointed person, such as the undergraduate chair, or with a colleague in a neighbouring office. The foundations upon which trust were built varied depending on the individual; a shared career stage, approach to teaching, disciplinary background, nationality, and gender were all named as factors upon which trust was initially based. Consequently, diversity in a department can provide an environment where faculty members may have more opportunities to connect with others they perceive as trustworthy.  Over time, however, it is the multiple interactions that colleagues have with one another that confirm or refute the perception that a colleague is well-intentioned, respectful, supportive, and worthy of trust. When a department provides multiple opportunities for conversations about teaching, faculty members may be exposed to a wider variety of perspectives about teaching and learning and may learn to value a broader range of opinions, beliefs, and knowledges.  In addition to efforts that faculty members themselves make to develop a respectful, supportive atmosphere, the department head (or associate head or other departmental leader) can also promote collegiality. S/he can do so by publicly recognizing faculty members’ efforts to work together in teaching, research, and service; by including faculty from all ranks in decision-making and other activities to do with department matters (while being mindful  208 of not putting excessive demands on the pre-tenured faculty members’ time); and by holding an annual retreat and/or initiating other activities to encourage relationship building and discussion of commons problems (Bensimon et al., 2000).  6.3.2.1 Increase Adoption and Full Integration of Improved Teaching Practices With one exception, the participants did not refer to the university-level document on summative peer review of teaching. Many acknowledged that there were increased requirements for summative peer review but were not sure from where the impetus for these came. These findings suggest that faculty members were not aware of the university document. Accordingly, one might conclude that this document was having minimal impact on summative peer review practices. The recommendations that follow address the (teaching) policy-practice gap which is a widespread challenge in higher education (Harvey & Kamvounias, 2008; Kezar, 2011; Newton, 2003). They are meant to increase the uptake and integration of improved teaching practices by faculty members and other members of the higher education teaching community.  Recommendation 8: Help faculty members understand how the proposed new practice improves teaching. Three common and related reasons that people do not engage in change practices is that they do not understand the need for change and (consequently) they are not motivated or interested in making a change; additionally, they may perceive the change as threatening. Through deliberation and discussion, however, faculty members can learn and grow their understanding of how a particular practice is relevant to them and meets their professional  209 and personal needs. Informed deliberations can help faculty members challenge current notions, may shape norms and beliefs, and may allow faculty members to exchange ideas and address issues of implementation and motivation (Kezar, 2011).  Recommendation 9: Provide tangible support for sustaining change related to teaching policies. Resources—human, financial, time—are necessary to sustain change. Thus, even when faculty members accept new ways of doing things, they require support. For example, with regard to the adoption of summative peer review practices outlined in the university document, faculty members might benefit from addressing questions and concerns to a coordinator/coordinating committee, receiving training on how to conduct peer reviews, and having time to deliberate about criteria for good teaching. However, as long as an institution continues to reward research above teaching, faculty members will uphold values, practices, and assumptions that prioritize research. Consequently, when administrators attempt to implement policies for the improvement of teaching, faculty members may not broadly adopt these. For teaching and learning policies to shift from being “passively ignored” (Arreola, 2007) to becoming “transformative” (O’Meara, 2011), research-intensive institutions need to better align the reward systems with the institutional rhetoric about the importance of teaching and student learning. Doing so will require a significant shift in values.  210 Table 10: Summary of Recommendations for Practice  Broad area of recommendation Sub-category  Recommendation  Enhance summative peer review of teaching Involve the department head 1. The department head discusses peer review results with the candidate.  2. The department head considers peer reviews and student evaluations of teaching more equally.  3. Department head clarifies expectations and enhances awareness of summative peer review process through departmental guidelines  Improve the feedback process 4. Share written and/or verbal feedback with the candidate.  5. Ensure that feedback is delivered by a credible source.  6. Explain the value of receiving feedback on content and teaching approaches.  Foster a culture that values teaching n/a 7. Build and maintain a collegial culture. Increase adoption and integration of improved teaching practices n/a 8. Help faculty members understand how the proposed new practice improves teaching.  9. Provide tangible support for sustaining change related to teaching policies.   211  6.4 Recommendations for Future Research This study explored faculty members’ experiences and understandings of summative peer review of teaching, and inquired into the relationship between this process and professional growth in teaching. It also incorporated an examination of departmental practices that foster a culture of teaching and promote academics’ growth in their teaching roles. The findings of this study suggest that applying the lens of academic culture is helpful for revealing aspects of institutional, departmental, and disciplinary culture, including academic values and norms, that support or hinder professional growth in teaching.  Results of this study suggest four areas for future research: further study of summative peer review, studies that examine a more diverse faculty member population, studies of the role of the department head in promoting professional growth in teaching, and studies that examine how pre-tenured faculty members’ shape a culture of teaching. All of these (summarized in Table 11) would provide valuable data to continue to develop and test the conceptual framework.  6.4.1 Research on the Summative Peer Review of Teaching The paradox of summative peer review is expressed as a belief among faculty members that the goals of formative and evaluative functions of peer review of teaching should be kept separate (because, when combined, neither can be well achieved) and, concurrently, a disappointment that the summative peer review process does not contribute more significantly to professional growth. While faculty members acknowledged the primarily evaluative purpose of summative peer review, they also hoped to learn about their teaching through the process of peer review. This finding points out the need for additional research in  212 ways to design a feasible summative peer review of teaching process that meets faculty members’ needs for evaluation of teaching and professional growth. There would also be a need to evaluate whether such a process was meeting its goals over time.  A second area of research into summative peer review could be a longitudinal study that would probe the relationship between changes in the institutional culture of teaching and changes in faculty members’ understandings and experiences of peer review. Since many research-intensive institutions are placing an increased emphasis on teaching (Chalmers, 2010; MacFarlane, 2011), a trend also noted by the participants, a longitudinal study might allow for a stronger understanding of the effects of institutional culture (in particular, public statements and reward policies to do with teaching and learning) on faculty members’ perceptions of summative peer review of teaching.  The current study also suggests a third area of research that would involve a more systematic and intentional examination of how the four aspects of professional growth outlined in O’Meara et al.’s (2008) framework (learning, agency, professional relationships, and commitment) manifest in the summative peer review of teaching. In conducting this study, I was especially interested in—and therefore attuned to—displays of collegiality (e.g., professional relationships) and, as a result, may have missed opportunities to probe the other aspects of professional growth. A possible research direction for a study on summative peer review of teaching and professional growth would explicitly seek evidence for all four aspects of the framework.  213  6.4.2 Future Research on Departmental Culture that Captures Greater Faculty Member Diversity When it comes to studying department cultures and teaching, future research focused on the experiences of women and non-tenure track faculty members would expand our understanding of the influence of gender and appointment.  One of the most significant demographic changes among faculty members in the past two decades has been the increasing presence of women (Gappa et al., 2007; Robinson, 2006; Schuster & Finkelstein, 2006). This trend has implications for future studies of professional growth and academic culture. Firstly, women faculty who enter formerly homogenous academic environments often endure more social isolation, have greater difficulty identifying mentors to guide them in their careers (Gappa et al., 2007), and receive less career help from their colleagues (Gersick, Bartunek, & Dutton, 2000; Gibson, 2006). Secondly, there is research that has found that women spend more time teaching (Probert, 2005) and may show a preference for certain teaching approaches and methods (more interaction with students, greater use of diversity, employ collaborative learning techniques more often) compared with their male counterparts (Campbell, 2003, ¶ 10; Umbach, 2006). Thus, women experience academic culture differently than men and may have different inclinations toward teaching. Consequently, future research that focuses on female academics and their professional growth in teaching, or a comparative study of men and women, could provide a wealth of data on how gender, professional growth in teaching, and academic culture intersect.  As it pertains to the broad topic of “diversity,” another well-established trend in North  214 American higher education is the steep increase in positions that are ineligible for tenure (Gappa et al., 2007; Hamilton, 2007; Robinson, 2006; Schuster & Finkelstein, 2006). A second diversity-related area that could be explored is how faculty members with non- traditional appointments learn and grow as teachers through collegial relationships. As previously stated, these relationships can be significant in fostering professional growth in teaching, yet the literature indicates that contingent faculty members are discontent with their capacities to develop campus-based relationships with students and colleagues (O’Meara et al., 2008). An examination into the ways that departmental relationships foster or hinder professional growth in teaching among contingent faculty could be an extension of this study.  Finally, increased diversity in terms of departmental, disciplinary, and institutional representation would help test and develop the conceptual model. The participants in this study (with the exception of those in one Science department) described belonging to strong teaching departments with a productive collegial culture. Future research might use mixed methods, including questionnaires that allow participants to respond to questions anonymously, to reach individuals from a wider range of cultures. Some of these cultures might be considered less strong when it comes to teaching and/or not collegial. Furthermore, while there was some diversity in the disciplinary cultures represented, more attention to faculty members’ disciplinary affiliations would contribute to our understanding of how this aspect of culture influences faculty members’ engagement in summative peer review and other teaching practices.  215  6.4.3 Research on the Role of the Department Head in Promoting Professional Growth in Teaching The results of this study have signaled that the department head has a significant role in promoting professional growth in teaching. Since this study included the perspective of only one department head and did not explicitly set out to ask about the role of the head in promoting a culture of teaching, a future study could focus on the function of the head by interviewing department heads only and/or posing questions that pertained directly to that function. Any such study could be further strengthened if it made connections between how department heads enact their roles and the institutional/disciplinary cultures in which they perform their duties. An additional aspect of this proposed research would involve a close examination of how the department head interacts with university level policies on teaching. That is, future studies could examine how the department head utilizes university policies and/or rhetoric about teaching to strengthen the culture of teaching within his/her department.  6.4.4 Research on the Role of Pre-tenured Faculty Members’ in Shaping a Culture of Teaching An extensive body of research has examined the socialization of new faculty members. This literature typically describes a process whereby new faculty members enter a department and over time learn and adapt to the departmental processes (Jawitz, 2009; Trowler & Knight, 2000). The adaptation process requires new faculty to make multiple adjustments and is frequently characterized as a period of high stress and anxiety (Austin, 2010; Simmons, 2011). Findings from this study indicate that new faculty members located in the same department and hired within the same year or within a few years of one another can shape  216 and change the departmental culture of teaching in important ways. For example, as new faculty members question existing practices, they may spur more senior faculty members to reflect on why these practices exist and whether they need to be revisited. Additionally, new faculty may be more willing to take part in innovative (to the department) practices because they are unaccustomed to the departmental norms. Further investigations into the experiences of pre-tenured faculty members, as they pertain to their influence on the departmental culture of teaching, could expand our understanding of, and provide an alternative perspective on, the power of faculty members at this rank in fostering a departmental culture of teaching. Table 11: Summary of Recommendations for Future Research  Broad area of future research Focused area of future research (worded as a potential question)  Summative peer review of teaching 1. What are the features of a (feasible) summative peer review of teaching process that meets faculty members’ needs for evaluation and professional growth in teaching?  2. How do faculty members’ perceptions of summative peer review change (if at all) over time as an institution puts more emphasis on enhancing teaching?  3. How do learning, agency, commitment and professional relationships manifest and interact with one another in faculty members’ experiences of summative peer review of teaching?  Department culture with a focus on diversity 4. How do women and men’s experiences of professional growth in teaching differ within the context of their department?  5. What are non-tenure track faculty members’ experiences of departmental collegiality and how does this impact upon their professional growth as teachers?  6. In departments where peers do not enjoy a strong sense of collegiality, what is the role of professional relationships in fostering professional growth in teaching?  7. In departments where teaching is devalued, how do faculty members with a commitment to their professional growth as teacher, grow and learn as instructors?   217 Broad area of future research Focused area of future research (worded as a potential question) (continued)  Department culture with a focus on diversity (continued)  8. What is the relationship between disciplinary culture and professional growth in teaching?  Role of the department head 9. What is the role of the department head in fostering professional growth in teaching?  Role of pre- tenured faculty members 10. In what ways do pre-tenured faculty members shape the departmental culture of teaching?   6.5 Closing Comments I began this study with the intention of learning about faculty members’ experiences of summative peer review. Stepping into this conversation catapulted me into wide territory: I soared over the terrain of department culture, was flung into the region of academic norms, and glided into the realm of collegiality.  Perhaps unsurprisingly, it was through my conversations with the individual participants that I grasped the contribution that conversations make to faculty members’ professional learning as teachers. This (seemingly) ordinary and common act has much potential for promoting growth among academics. The presence or absence of these collegial conversations, I discovered, is sometimes contrived, sometimes spontaneous, but always impactful.  Much of what I learned during this study enhanced my appreciation for those individuals who pursue their commitment to teaching within the context of a research-intensive university. I refer here to the individual faculty members, department leaders, university administrators, and educational developers who work to enhance teaching and learning in an  218 environment that does not consistently welcome such initiatives. In my opinion, they are brave and dedicated souls whose efforts, over time, are having some consequence.    219 References Acker, S. & Armenti, C. (2004). Sleepless in academia. Gender in Education, 16(1), 3–24. doi:10.1080/0954025032000170309 Ackerman, D., Gross, B. L., & Vigneron, F. (2009). Peer observation reports and student evaluations of teaching: Who are the experts? Alberta Journal of Educational Research, 55(1), 18–39. Retrieved from http://ajer.synergiesprairies.ca/ ajer/index.php/ ajer/index Allan, E. J. (2011). Women’s status in higher education: Equity matters. ASHE Higher Education Report, 37(1). doi:10.1002/aehe.3701 Altbach, P. (2002). How are faculty faring in other countries? In R. Chait (Ed.), The questions of tenure (pp. 160–181). Cambridge, MA: Harvard University Press. Altman, H. (2004). A baker’s dozen: Dirty lessons I have learned in an academic career. Change, 36(4), 50–53. doi:10.1080/00091380409604977 Alvesson, M., & Sveningsson, S. (2008). Changing organizational culture: Cultural change work in progress. New York, NY: Routledge. Anderson, M., Scott, G., & Coates, H. (2008, September). Leadership challenges for university heads. Paper presented at the British Educational Research Association Conference, Edinburgh, UK. Arreola, R. (2007). Developing a comprehensive faculty evaluation system (3rd ed.). Bolton, MA: Anker Publishing Company, Inc. Austin, A. E. (2010). Expectations and experiences of aspiring and early career academics. In L. McAlpine & G. Åkerlind (Eds.), Becoming an academic: International Perspectives (pp. 18–44). New York, NY: Palgrave Macmillan.  220 Barbour, R. (2008). Introducing qualitative research: A student's guide to the craft of doing qualitative research. Thousand Oaks, CA: Sage. Barrett, L., & Barrett, P. (2010). Women and academic workloads: Career slow lane or cul- de-sac? Higher Education: The International Journal of Higher Education and Educational Planning, 61(2), 141–155. doi:10.1007/s10734-010-9329-3 Becher, T. (1989). Academic tribes and territories: Intellectual enquiry and the cultures of disciplines. Philadelphia, PA: Society for Research into Higher Education & Open University Press. Becher, T. (1994). The significance of disciplinary differences. Studies in Higher Education, 19(2), 151. doi:10.1080/03075079412331382007 Becher, T., & Trowler, P. R. (2001). Academic tribes and territories: Intellectual enquiry and the culture of disciplines (2nd ed.). Philadelphia, PA: Society for Research into Higher Education & Open University Press. Bensimon, E. M., Ward, K., & Sanders, K. (2000). The department chair's role in developing new faculty into teachers and scholars. Bolton, MA: Anker Publishing Company, Inc. Berdrow, I. (2010). King among kings: Understanding the role and responsibilities of the department chair in higher education. Educational Management Administration & Leadership, 38(4), 499-514. doi:10.1177/1741143210368146 Bergquist, W. H., & Pawlak, K. (2008). Engaging the six cultures of the academy. San Francisco, CA: Jossey-Bass. Bernstein, D. J., J. Jonson, & K. L. Smith. (2000). An examination of the implementation of peer review of teaching. New Directions for Teaching and Learning, 83(Fall), 73–86. doi:10.1002/tl.8306  221 Biagioli, M. (2002). From book censorship to academic peer review. Emergences: Journal for the Study of Media & Composite Cultures, 12, 11–45. doi:10.1080/ 1045722022000003435 Billett, S. (2001). Learning through working life: Interdependencies at work. Studies in Continuing Education, 23(1), 19–35. doi:10.1080/01580370120043222 Blackmore, J. (2005). A critical evaluation of peer review via teaching observation within higher education. International Journal of Educational Management, 19(3), 218–232. doi:10.1108/09513540510591002 Borman, K. M., Clarke, C. Cotner, B., & Lee R. (2006). Cross-case analysis. In J. L. Green, G. Camilli, & P. B. Elmore (Eds.), Handbook of complementary methods in education research (pp. 123–139). Mahwah, NJ: Lawrence Erlbaum Associates. Bovill, C. (2008). Peer observation of teaching guidelines. Retrieved from the University of Glasgow Teaching and Learning Centre website: http://www.gla.ac.uk/ services/learningteaching/contactus/drcatherinebovill/ Braskamp, L. (2000). Towards a more holistic approach to assessing faculty as teachers. New Directions for Teaching and Learning, 83(Fall), 19–33. doi:10.1002/tl.8303 Braxton, J. M. (2008). Toward a theory of faculty professional choices in teaching that foster college student success. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (pp. 181–207). New York, NY: Agathon Press. doi:10.1007/978-1- 4020-6959-8_6 Braxton, J. M. (2010). Norms and the work of colleges and universities: Introduction to the special issue—Norms in academia. The Journal of Higher Education, 81(3), 243– 250. doi:10.1353/jhe.0.0097  222 Brenner, M. E. (2006). Interviewing in educational research. In J. L. Green, G. Camilli, & P. B. Elmore (Eds.), Handbook of complementary methods in education research (pp. 357– 370). Mahwah, NJ: Lawrence Erlbaum Associates. Brinko, K. T. (1983). The practice of giving feedback to improve teaching: What is effective? The Journal of Higher Education, 64(5), 574–593. Retrieved from http://www.jstor.org/stable/2959994 Bronstein, P., & Ramalay, J. (2002). Making the persuasive tenure case: Pitfalls and possibilities. In J. Cooper & D. Stevens (Eds.), Tenure in the sacred grove (pp. 31– 55). Albany, NY: SUNY Press. Bunch, K. (2007). Training failure as a consequence of organizational culture. Human Resource Development Review, 6(2), 142–163. doi:10.1177/1534484307299273 Byrne, J., Brown, H., & Challen, D. (2010). Peer development as an alternative to peer observation: a tool to enhance professional development. International Journal for Academic Development, 15(3), 215–228. doi:10.1080/1360144X.2010.497685 Caffarella, R. & Zinn, L. (1999). Professional development for faculty: A conceptual framework for barriers and supports. Innovative Higher Education, 23(4), 241-254. Retrieved from http://www.ingentaconnect.com/content/klu/ ihie/1999/00000023/00000004 Campbell, K. (2003). Personal, political and pedagogical: Female faculty and values-based learning design. Radical Pedagogy, 5(1). Retrieved from http://radicalpedagogy. icaap.org/content/issue5_1/01_campbell.html Cariaga-Lo, L., Worthy Dawkins, P., Enger, R., Schotter, A., & Spence, C. (2010). Supporting the development of the professoriate. Peer Review, 12 (3), 19–22.  223 Retrieved from http://www.aacu.org/peerreview/pr-su10/pr-su10_index.cfm. Carter, V. (2008). Five steps to becoming a better peer reviewer. College Teaching, 56(2), 85–88. doi:10.3200/CTCH.56.2.85-88 Cavanaugh, R. (1996). Formative and summative evaluation in the faculty peer review of teaching. Innovative Higher Education, 20(4), 235–240. doi:10.1007/BF01185798 Chait, R. (2002). Does faculty governance differ at colleges with tenure and colleges without tenure? In R. Chait (Ed.), The questions of tenure (pp. 69–100). Cambridge, MA: Harvard University Press. Chalmers, D. (2011). Progress and challenges to the recognition and reward of the Scholarship of Teaching in higher education. Higher Education Research & Development, 30(1), 25–38. doi:10.1080/07294360.2011.536970 Chism, N. (2004). Characteristics of effective teaching in higher education: Between definitional despair and certainty. Journal of Excellence in College Teaching 15(3), 5–36. Retrieved from http://celt.muohio.edu/ject/index.php Chism, N. (2007a). Peer review of teaching: A sourcebook (2nd ed.). Bolton, MA: Anker Publishing Company, Inc. Chism, N. (2007b). Why introducing or sustaining peer review of teaching is so hard, and what you can do about it. Department Chair: A Resource for Academic Administrators, 18(2), 6–8. doi:10.1002/dch.20017 Clark, B. R. (1980). Academic culture. New Haven, CT: Yale University, Higher Education Research Group. Clark, B. R. (1984). Perspectives on higher education. Berkeley, CA: University of California Press.  224 Clark, B. R. (1987). The academic life: Small worlds, different worlds. Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching. Clegg, S. (2003). Problematising ourselves: Continuing professional development in higher education. International Journal for Academic Development, 8(1), 37–50. doi:10. 1080/1360144042000277928 Clotfelter, C. (2002). Can faculty be induced to relinquish tenure? In R. Chait (Ed.), The questions of tenure (pp. 221–245). Cambridge, MA: Harvard University Press. Cosh, J. (1998). Peer observation in higher education—A reflective approach. Innovations in Education and Teaching International, 35(2), 171–176. doi:10.1080/ 1355800980350211 Courneya, C., Pratt, D. D., & Collins, J. (2008). Through what perspective do we judge the teaching of peers? Teaching and Teacher Education: An International Journal of Research and Studies, 24(1), 69–79. doi:http://dx.doi.org/10.1016/j.tate.2007.01.009, Cox, M. (2004). Introduction to Faculty Learning Communities. New Directions for Teaching and Learning, 97(Spring 2004), 5–23. doi: 10.1002/tl.129 Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage. Creswell, J. W. (2008). Educational research: planning, conducting, and evaluating quantitative and qualitative research. (3rd ed.). Upper Saddle River, NJ: Merrill Prentice Hall. Dall’Alba, G., & Sandberg, J. (2006). Unveiling professional development: A critical review of stage models. Review of Educational Research, 76(3), 383–412. doi:10.3102/ 00346543076003383  225 Del Favero, M. (2002). Linking administrative behavior and student learning: The learning centered academic unit. Peabody Journal of Education, 77(3), 60–84. doi:10.1207/ S15327930PJE7703_4 Donnelly, R. (2007). Perceived impact of peer observation of teaching in higher education. International Journal of Teaching and Learning in Higher Education. 19(2), 117– 129. Retrieved from http://www.isetl.org/ijtlhe/ Egan, T. M. (2008). The relevance of organizational subculture for motivation to transfer learning. Human Resource Development Quarterly, 19(4), 299–322. doi:10.1002/ hrdq.1243 Emery, C., Kramer, T., & Tian, R. (2003). Return to academic standards: A critique of student evaluations of teaching effectiveness. Quality Assurance in Education, 11(1), 37–46. doi:10.1108/09684880310462074 Eraut, M. (2000). Non-formal learning and tacit knowledge in professional work. British Journal of Educational Psychology, 70(1), 113–1310. doi:1348/000709900158001 Fairweather, J. (2002). The ultimate faculty evaluation: Promotion and tenure decisions. New Directions for Institutional Research, 114(Summer), 97–108. doi:10.1002/ir.50 Fink, D. (2008). Evaluating teaching: A new approach to an old problem. To Improve the Academy: Resources for Faculty, Instructional, and Organizational Development, 26, 3–21. Retrieved from http://finkconsulting.info/major-publications/ Flyvbjerg, B. (2006). Five misunderstandings about case-study research. Qualitative Inquiry, 12(2), 219-245. doi:10.1177/1077800405284363 Fonow, M., & Cook, J. (2005). Feminist methodology: New applications in the academy and public polity. Signs, 30(4), 2211–2235. doi:10.1086/428417  226 Fontana, A., & Frey, J. H. (2008). The interview: From neutral stance to political involvement. In N. J. Denzin & Y. S. Lincoln (Eds.), Collecting and interpreting qualitative materials (3rd ed., pp. 115–160). Thousand Oaks, CA: Sage. Gappa, J. M., Austin, A. E., & Trice, A. G. (2007). Rethinking faculty work: Higher education's strategic imperative. San Francisco, CA: Jossey-Bass. Gersick, C. J. G., Bartunek, J. M., & Dutton, J. E (2000). Learning from academia: The importance of relationships in professional life. The Academy of Management Journal, 43(6), 1026–1044.  Retrieved from http://www.jstor.org/stable/1556333 Gibbs, G., Knapper, C., & Piccinin, S. (2008). Disciplinary and contextually appropriate approaches to leadership of teaching in research-intensive academic departments in higher education. Higher Education Quarterly, 62(4), 416–436. doi:10.1111/j.1468- 2273.2008.00402.x Gibson, S. K. (2006). Mentoring of women faculty: The role of organizational politics and culture. Innovative Higher Education, 31(1). doi:10.1007/s10755-006-9007-7 Gizir, S., & Simsek, H. (2005). Communication in academic context. Higher Education, 50(2), 197–221. doi:10.1007/s10734-004-6349-x Gosling, D. (2002). Models of peer observation of teaching. Retrieved from http://www.heacademy.ac.uk/assets/documents/resources/database/id200_Models_of _Peer_Observation_of_Teaching.rtf Gosling, D. (2010). Value commitments and ambivalence in educational development. New Directions for Teaching and Learning, 122(Summer), 91–102. doi:10.1002/tl.401 Gosling, D., & D’Andrea, V. (2001). Quality development: A new concept for higher education. Quality in Higher Education, 7(1), 9–17. doi:10.1080/  227 13538320120045049 Gosling, D., & Ritchie, S. (2003). Research project on peer observation of teaching. Retrieved from The Higher Education Academy Subject Centre for Education ESCalate: http://escalate.ac.uk/resources/peerobservation2/ Gravestock, P., & Greenleaf, E. G. (2008). Overview of tenure and promotion policies across Canada. Retrieved from http://www.malaspina.ca/integratedplanning/documents/ OverviewofTPPoliciesinCanada.pdf Grbich, C. (2007). Qualitative data analysis: An introduction. Thousand Oaks, CA: Sage. Groccia, J. (2010). Why faculty development? Why now? In A. Saroyan & M. Frenay (Eds.), Building teaching capacities in higher education: A comprehensive international model (pp. 1–42). Sterling, VA: Stylus Publishing. Haag, P. (2005). Is collegiality code for hating ethnic, racial, and female faculty at tenure time? Education Digest, 71(1), 57-62. Haig, N. (2005). Everyday conversation as a context for professional learning and Development. International Journal for Academic Development, 10(1), 3–16 doi:10.1080/13601440500099969 Hamilton, N. (2007). Faculty autonomy and obligation. Academe, 93(1), 36-42. Retrieved from http://www.aaup.org/AAUP/pubsres/academe/ Hammersley-Fletcher, L., & Orsmond, P. (2004). Evaluating our peers: Is peer observation a meaningful process? Studies in Higher Education, 29(4), 213–224. doi:10. 1080/0307507042000236380 Handal, G. (1999). Consultation using critical friends. New Directions for Teaching and Learning, 79(Fall), 59–70. doi:10.1002/tl.7907  228 Harding, S. (2004). Rethinking standpoint epistemology: What is ‘strong objectivity’? In S. Nagy Hesse-Biber & M. Yaiser (Eds.), Feminist perspectives on social research (pp. 39–64). New York, NY: Oxford University Press. Hardy, I., & Smith, E. (2006). Contesting tertiary teaching qualifications: An Australian perspective. Teaching in Higher Education, 11(3), 337–350. doi:10.1080/ 13562510600680798 Harms, P., & Britt Roebuck, D. (2010). Teaching the art and craft of giving and receiving feedback. Business Communication Quarterly, 73(4), 413–431. doi:10.1177/ 1080569910385565 Harvey, A., & Kamvounias, P. (2008). Bridging the implementation gap: a teacher-as-learner approach to teaching and learning policy. Higher Education Research & Development, 27(1), 31–41. doi:10.1080/07294360701658716 Hatzipanagos S., & Lygo-Baker, S. (2006). Teaching observations: A meeting of minds? International Journal of Teaching and Learning in Higher Education, 17(2), 97–105. Retrieved from http://www.isetl.org/ijtlhe/ Healey, M., & Jenkins, A. (2003). Discipline-based educational development. In H. Eggins & R. Macdonald (Eds.), The scholarship of academic development (pp. 47–57). Philadelphia, PA: Society for Research into Higher Education & Open University Press. Hearn, J. (2007). Sociological studies of academic departments. In P. J. Gumport (Ed.), Sociology of Higher Education (pp. 222–265). Baltimore, MD: Johns Hopkins University Press.  229 Hearn, J., & Anderson, M. (2002). Conflict in academic departments: An analysis of disputes over faculty promotion and tenure. Research in Higher Education, 43(5), 503–529. doi:10.1023/A:1020197630478 Henkel, M. (2000). Academic identities and policy change in higher education. London, ENG: Jessica Kingsley. Hodges, L., & Stanton, K. (2007). Translating comments on student evaluations into the language of learning. Innovative Higher Education, 31(5), 279–286. doi:10.1007/ s10755-006-9027-3 Huber, M. (2002). Faculty evaluation and the development of academic careers. New Directions for Institutional Research, 114(Summer), 73–83. doi:10.1002/ir.48 Hum, D. (1998). Tenure and pay structures in Canadian universities. Research Monographs in Higher Education (No. 4). Winnipeg, MB: Centre for Higher Education Research and Development. Hutchings, P. (1996). The peer review of teaching: Progress, issues and prospects. Innovative Higher Education, 20(4), 221–234. doi:10.1007/BF01185797 Jacobs, J. A., & Winslow, S. (2004). Overworked faculty: Job stresses and family demands. Annals of the American Academy of Political and Social Science, 596(November), 104–129. doi:10.1177/0002716204268185 Jawitz, J. (2009). Learning in the academic workplace: The harmonization of the collective and the individual habitus. Studies in Higher Education, 34(6), 601–614. doi:10.1080/03075070802556149 Kardia, D., & Wright, M. (2004). Instructor identity: The impact of gender and race on faculty experiences with teaching (CRLT Occasional Paper, No. 19). Retrieved from  230 Center for Research on Learning and Teaching website: www.crlt.umich.edu/ publinks/CRLT_no19.pdf Kell, C., & Annetts, S. (2009). Peer review of teaching embedded practice or policy-holding complacency? Innovations in Education and Teaching International, 46(1), 61–70. doi:10.1080/14703290802646156 Kemp, R., & Gosling, D. (2000). Peer observation of teaching. Retrieved from The Higher Education Academy Subject Centre for Education ESCalate: http://escalate.ac.uk/ resources/peerobservation2/ Kezar, A. (2001). Understanding and facilitating organizational change in the 21st century: Recent research and conceptualizations. ASHE-ERIC Higher Education Report, 28(4). doi:10.1002/aehe.2804 Kezar, A. (2011). What is the best way to achieve broader reach of improved practices in higher education? Innovative Higher Education, 36, 235–247. doi:10.1007/s10755- 011-9174-z Kezar, A., & Eckel, P. D. (2002). The effect of institutional culture on change strategies in higher education: Universal principles or culturally responsive concepts? The Journal of Higher Education, 73(4), 436–460. Retrieved from http://www.jstor.org/stable/ 1558422 Kilgore, D., & Cook, M. D. (2007). “In things and in minds”: Scholarly teachers’ struggles for legitimacy in a research-oriented institution. Journal on Excellence in College Teaching, 18(2), 143–161. Retrieved from: http://celt.muohio.edu/ject/index.php Knight, P., Tait, J., & Yorke, M. (2006). The professional learning of teachers in higher education. Studies in Higher Education, 31(3), 319–339. doi:10.1080/  231 03075070600680786 Knight, P., & Trowler, P. R. (2000). Department-level cultures and the improvement of learning and teaching. Studies in Higher Education, 25(1), 69–83. doi:10.1080/ 030750700116028 Kreber, C., & Castleden, H. (2009). Reflection on teaching and epistemological structure: reflective and critically reflective processes in ‘pure/soft’ and ‘pure/hard’ fields. Higher Education, 57(4), 509–531. doi:10.1007/s10734-008-9158-9 Kvale, S. (1996). InterViews. Thousand Oaks, CA: Sage. Lattuca, L., & Stark, J. (1994). Will disciplinary perspectives impede curricular reform? The Journal of Higher Education, 65(4), 401–426. Retrieved from http://www.jstor.org/ stable/2943853 . Lee, J. L. (2007). The shaping of the departmental culture: Measuring the relative influences of the institution and discipline. Journal of Higher Education Policy & Management, 29(1), 41–55. doi:10.1080/13600800601175771 Lindblom-Ylänne, S., Trigwell, K., Nevgi, A., & Ashwin, P. (2006). How approaches to teaching are affected by discipline and teaching context. Studies in Higher Education, 31(3), 285–298. doi:10.1080/03075070600680539 Link, A. N., Swan, C. A., & Bozeman, B. (2008). A time allocation study of university faculty. Economics of Education Review, 27, 363–374. doi:10.1016/j.econedurev. 2007.04.002 Lomas, L., & Nicholls, G. (2005). Enhancing teaching quality through peer review of learning and teaching. Quality in Higher Education, 11(2), 137–149. doi:10.1080/ 13538320500175118  232 Long, T., & Johnson, M. (2000). Rigour, reliability and validity in qualitative research. Clinical Effectiveness in Nursing, 4(1), 30–37. doi:10.1054/cein.2000.0106, Lucas, L. (2006). The research game in academic life. Maidenhead, UK: Society for Research into Higher Education/Open University Press. Macfarlane, B. (2011). Prizes, pedagogic research and teaching professors: Lowering the status of teaching and learning through bifurcation. Teaching in Higher Education, 16(1), 127–130. doi:10.108013562517.2011.530756 Mackenzie, N., & Knipe, S. (2006). Research dilemmas: Paradigms, methods and methodology. Issues in Educational Research, 16(2), 193–205. Retrieved from http://www.iier.org.au/iier16/2006conts.html Mamiseishvili, K. (2010). Foreign-born women faculty work roles and productivity at research universities in the United States. Higher Education, 60 (2), 139–156. doi:10.1007/s10734-009-9291-0 Massy, W. F., Wilger, A. K., & Colbeck, C. (1994). Overcoming `hollowed' collegiality. Change, 26(4), 10–20. doi:10.1080/00091383.1994.9938496 Maxwell, J. (1992). Understanding and validity in qualitative research. Harvard Educational Review, 63(3), 279-300. Retrieved from http://her.hepg.org/content/ 8323320856251826/ Maxwell, J. A. (2005). Qualitative research design: An interactive approach (Vol. 42, 2nd ed.), Applied Social Research Methods Series. Thousand Oaks, CA: Sage. Mayan, M. J. (2009). Essentials of qualitative inquiry. Walnut Creek, CA: Left Coast Press. Merriam, S. B. (1998). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass.  233 Merriam, S. B. (2002). Qualitative research in practice: Examples for discussion and analysis. San Francisco, CA: Jossey-Bass. Milem, J. F., Berger, J. B., & Dey, E. L. (2000). Faculty time allocation: A study of change over 20 years. The Journal of Higher Education, 71(4), 454–475. Retrieved from http://www.jstor.org/stable/2649148 Mills, M. R., & Hyle, A. D. (2001). No rookies on rookies: Compliance and opportunism in policy implementation. The Journal of Higher Education, 72(4), 453–477. Retrieved from http://www.jstor.org/stable/2672891 Misra, J., Hickes Lundquist, J., Holmes, E., & Agiomavritis, S. (2011). The ivory ceiling of service work. Academe Online, 97(1). Retrieved from http://www.aaup.org/AAUP/ pubsres/academe/2011/JF/ Mitroff, I. I. (1974). Norms and counter-norms in a select group of the Apollo moon scientists: A case study of the ambivalence of scientists. American Sociological Review, 39(4), 579–595. Retrieved from http://www.jstor.org/stable/2094423 Morse, J., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002). Verification strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods, 1(2), 1–19. Retrieved from http://www.ualberta.ca/~ijqm/ Murphy, T., MacLaren, I., & Flynn, S. (2009). Toward a summative system for the assessment of teaching quality in higher education. International Journal of Teaching and Learning in Higher Education, 20(2), 226–236. Retrieved from http://www.isetl. org/ijtlhe/ Neumann, A. (2005). Observations: Taking seriously the topic of learning in studies of faculty work and careers. New Directions for Teaching and Learning, 102(Summer),  234 63–83. doi:10.1002/tl.197 Neumann, A. (2009). Professing to learn: Creating tenured lives and careers in the American research university. Baltimore, MD: The Johns Hopkins University Press. Neumann, R. (2001). Disciplinary differences and university teaching. Studies in Higher Education, 26(2), 135–146. doi:10.1080 /0307507012005207 1 Neumann, R., Parry, S., & Becher, T. (2002). Teaching and learning in their disciplinary contexts: A conceptual analysis. Studies In Higher Education, 27(4), 405–17. doi:10.1080/0307507022000011525 Newton, J. (2003). Implementing an institution-wide learning and teaching strategy: Lessons in managing change. Studies in Higher Education, 28(4), 427–441. doi:10.1080/ 0307507032000122279 O’Hara, R. (2009, May 9). Re: Evaluating collegiality. [Online forum comment]. Retrieved from https://listserv.nd.edu/cgi-bin/wa?A2=ind0905&L=POD&P=R23779. Oleson, V. (2008). Early millennial feminist qualitative research: Challenges and contours. In N. Denzin & Y. Lincoln (Eds.), The landscape of qualitative research (pp. 311–370). London, ENG: Sage. Oloyede, O. (2002). Disciplinary cultures and academic leadership. Society in Transition, 33(1), 116–133. O’Meara, K. (2005). Encouraging multiple forms of scholarship in faculty reward systems: Does it make a difference? Research in Higher Education, 46(5), 479–510. doi:10. 1007/s11162-005-3362-6 O’Meara, K. (2006). Encouraging multiple forms of scholarship in faculty reward systems: Have academic cultures really changed? New Directions for Institutional Research,  235 129 (Spring), 77–95. doi:10.1002/ir.173 O’Meara, K. (2011). Inside the panopticon: Studying academic reward systems. In J. C. Smart & M. B. Paulsen (Eds.). Higher education: Handbook of theory and research (pp. 161–220). New York, NY: Agathon Press. doi:10.1002/aehe.3403 O'Meara, K., Terosky, A., & Neumann, A. (2008). Faculty careers and work lives: A professional growth perspective. ASHE Higher Education Report, 34(3). doi:10.1002/aehe.3403 Ornstein, M., & Stewart, P. (2007). Promotion at Canadian universities: The intersection of gender, discipline, and institution. Canadian Journal of Higher Education, 37(3), 1– 25. Ory, J. C. (2000). Teaching evaluation: Past, present, and future. New Directions for Teaching & Learning, 83(Fall), 13–18. doi:10.1002/tl.8302 Paulsen, M. B. (2002). Evaluating teaching performance. New Directions for Institutional Research, 114(Summer), 5–18. doi:10.1002/ir.42 Peel, D. (2005). Peer observation as a transformatory tool? Teaching in Higher Education, 10(4), 489–504. doi:10.1080/13562510500239125 Pratt, D. (1997). Reconceptualizing the evaluation of teaching in higher education. Higher Education, 34, 23–44. doi:10.1023/A:1003046127941 Premeaux, S. R., & Mondy, R. (2002). Tenure's impact. College Teaching, 50(4), 154. doi:10.1080/87567550209595898 Probert, B. (2005). ‘I just couldn’t fit in’: Gender and unequal outcomes in higher education. Gender, Work and Organization, 12(1), 50–72. doi:0.1111/j.1468-0432.2005.00262.x Quinlan, K., & Åkerlind, G. (2000). Factors affecting departmental peer collaboration for  236 faculty development: two cases in context. Higher Education, 40, 23–53. doi:10.1023/A:1004096306094 Ramazanoglu, C., & Holland, J. (2006). Feminist methodology: Challenges and choices. London, ENG: Sage. Rhode, D. (2006). In pursuit of knowledge: Scholars, status, and academic culture. Stanford, CA: Stanford Law and Politics. Rice, E., & Sorcinelli, M. (2002). Can the tenure process be improved? In R. Chait (Ed.), The questions of tenure (pp. 101–124). Cambridge, MA: Harvard University Press. Rice, E., Sorcinelli, M. D., & Austin, A. E. (2000). Heeding new voices: Academic careers for a new generation. Washington, DC: American Association for Higher Education. Ritchie, J, Lewis, J., & Elam, G. (2003). Designing and selecting samples. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students (pp. 77–107). Thousand Oaks, CA: Sage. Robinson, D. (2006). The status of higher education teaching personnel in Australia, Canada, New Zealand, the United Kingdom, and the United States. Retrieved from Canadian Association of University Teachers website: http://www.caut.ca/en/ publications/reports/ei%20study%20final.pdf Roxå, T., & Mårtensson, K. (2009). Significant conversations and significant networks: Exploring the backstage of the teaching arena. Studies in Higher Education, 34(5), 547-559. doi:10.1080/03075070802597200 Roxå, T., Mårtensson, K., & Alveteg, M. (2011). Understanding and influencing teaching and learning cultures at university: a network approach. Higher Education, 62, 99–111. doi:10.1007/s10734-010-9368-9.  237 Schein, E. (2004). Organizational culture and leadership. San Francisco, CA: John Wiley & Sons. Schram, T. H. (2003). Conceptualizing qualitative inquiry. Columbus, OH: Merrill-Prentice Hall. Schuster, J., & Finkelstein, M. (2006). The American faculty: The restructuring of academic work and careers. Baltimore, MD: The Johns Hopkins University Press. Schwalbe, M., & Wokomir, M. Interviewing men. In J. F. Gubrium & J. A. Holstein (Eds.), Handbook of interview research (pp. 203–219). Thousand Oaks, CA: Sage. Seldin, P. (1999). Changing practices in evaluating teaching: A practical guide to improved faculty performance and promotion/tenure decisions. Bolton, MA: Anker Publishing Company, Inc. Serow, R. (2000). Research and teaching at a research university. Higher Education, 40(4), 449–463. doi:10.1023/A:1004154512833 Shanahan, T. (2009). Accountability initiatives in higher education: An overview of the impetus to accountability, its expressions and implications. Retrieved from http://www.yorku.ca/univsec/senate/committees/exec/reports/2008-2009/ Shanahan%20Paper.pdf Shortland, S. (2004). Peer observation: A tool for staff development or compliance? Journal of Further and Higher Education, 28(2), 219–228. doi:10.1080/ 0309877042000206778 Shortland, S. (2010). Feedback within peer observation: Continuing professional development and unexpected consequences. Innovations in Education and Teaching International, 47(3), 295–304. doi:10.1080/14703297.2010.498181  238 Shulman, L. (1993). Teaching as community property. Change, 25(6), 6–7. doi:10.1080/ 00091383.1993.9938465 Simmons, N. (2011). Caught with their constructs down? Teaching development in the pre- tenure years. International Journal for Academic Development, 16(3), 229–241. doi:10.1080/1360144X.2011.596706 Sorcinelli, M. D., Austin, A. A., Eddy, P. L., & Beach, A. L. (2006). Creating the future of faculty development. Bolton, MA: Anker Publishing Company, Inc. Smith, K. (2011). Cultivating innovative learning and teaching cultures: A question of garden design. Teaching in Higher Education, 16(4), 427–438. doi:10.1080/ 13562517. 2011. 560374 Snape, D., & Spencer, L. (2003). The foundations of qualitative research. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 1–23). Thousand Oaks, CA: Sage. Stake, R. (2005). Qualitative case studies. In N. Denzin & Y. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed., pp. 443–466). Thousand Oaks, CA: Sage. Summerlee, A. J. S., & Christensen Hughes, J. (2010). Pressures for change and the future of university education. In J. Christensen Hughes & J. Mighty (Eds.), Taking stock: Research on teaching and learning in higher education (pp. 243–260). Montreal, PQ: McGill-Queen’s University Press. Suter, W. N. (2006). Introduction to educational research: A critical thinking approach. Thousand Oaks, CA: Sage. Swinglehurst, D., Russell, J., & Greenhalgh, T. (2008). Peer observation of teaching in the online environment: An action research approach. Journal of Computer Assisted  239 Learning , 24(5), 383–393. doi:10.1111/j.1365-2729.2007.00274.x Taylor Huber, M., & Hutchings, P. (2005). The advancement of learning: Building the teaching commons. San Francisco, CA: Jossey-Bass Thomas, G. (2011). A typology for the case study in social science following a review of definition, discourse, and structure. Qualitative Inquiry, 17(6), 511–521. doi:10.1177/ 1077800411409884 Tierney, W. G. (1997). Organizational socialization in higher education. Journal of Higher Education, 68(1), 1–16. Retrieved from http://www.jstor.org/stable/2959934 Toth, K. E., & McKey, C. A. (2010a). Differences in faculty development needs: Implications for educational peer review program design. Canadian Journal of Higher Education, 40(1), 53–68. Retrieved from http://ojs.library.ubc.ca/index.php/ cjhe Toth, K. E., & McKey, C.A. (2010b). Identifying the potential organizational impact of an educational peer review program. International Journal for Academic Development, 15(1), 73–83. doi:10.1080/13601440903529943 Trout, P. (2000). Flunking the test: The dismal record of student evaluations. Academe Online, July-August. Retrieved from http://www.aaup.org/AAUP/pubsres/ academe/2000/JA/Feat/trou.htm Trowler, P. R. (2008). Cultures and change in higher education: Theories and practices. London, ENG: Palgrave Macmillan. Trowler, P. R., & Knight, P. T. (2000). Coming to know in higher education: Theorising faculty entry to new work contexts. Higher Education Research & Development, 19(1), 27–42. doi:10.1080/07294360050020453  240 Turner, D. W., III (2010). Qualitative interview design: A practical guide for novice investigators. The Weekly Qualitative Report, 3(2), 7–13. Retrieved from http://www.nova.edu/ssss/QR/WQR/qid.pdf Umbach, P. D. (2007). Faculty cultures and college teaching. In R. P. Perry & J. C. Smart (Eds.), The scholarship of teaching and learning in higher education (pp. 263–317). Dordrecht, The Netherlands: Springer Verlag. doi:10.1007/1-4020-5742-3_8 VanWynsberghe, R., & Khan, S. (2007). Redefining case study. International Journal of Qualitative Methods, 6(2), 1–10. Retrieved from http://www.ualberta.ca/ ~iiqm/backissues/6_2/vanwynsberghe.pdf Walvoord, B. E., Carey, A. K., Smith, H. L., Soled, S. W., Way, P. K., & Zorn, D. (2000). Academic departments: How they work, how they change. ASHE-ERIC Higher Education Report No. 27(6). doi:10.1002/aehe.3640270801 Warren, J. (2001). Qualitative interviewing. In J. F. H. Gubrium & J. A. Holstein (Eds.), Handbook of interview research: Context & method (pp. 83–101). Thousand Oaks, CA: Sage. Werquin, P. (2007). Terms, concepts and models for analyzing the value of recognition programmes (Declassified OECD Report EDU/EDPC[2007]24). Retrieved Organisation for Economic Co-operation and Development website: http://www. oecd.org/LongAbstract/0,3425,en33873108_33844437_41834712_1_1_1_1,00.html West, C., & Zimmerman, D. H. (1987).  Doing gender. Gender and Society, 1(2), 125151. Retrieved from http://www.jstor.org/stable/189945 Winkler, J. (2000). Faculty reappointment, tenure, and promotion: Barriers for women. Professional Geographer, 52(4), 737–750. doi:10.1111/0033-0124.00262  241 Wolcott, H. F. (1994). Transforming qualitative data: Description, analysis, and interpretation. Thousand Oaks, CA: Sage. Yin, R. K. (2009). Case study research: Design and methods. Los Angeles, CA: Sage. Yon, M., Burnap, C., & Kohut, G. (2002). Evidence of effective teaching: Perceptions of peer reviewers. College Teaching, 50(3), 104–110. doi:10.1080/87567550209595887    242 Appendices Appendix A Letter of Initial Contact Dear Dr. [Insert name],  For my UBC doctoral research, I am conducting a study in which I seek to learn about faculty members’ understandings and experiences of the summative peer review of teaching giving special attention to the role of gender, rank and departmental/disciplinary culture. Given the increasing emphasis the (Name of Institution) is placing on summative peer review of teaching, as demonstrated by the new institutionally-adopted policies for this process, and the limited empirical research into the experiences of faculty members, there is a need to find out more about summative peer review from the perspective of academics.  I am writing to invite you to consider participating. Other full-time tenure track faculty members in your department, and in five other departments across campus, have also been sent this invitation.  Participating in my project should not require much of your time. You and I will complete a single interview, approximately 60-75 minutes in length. I will ask permission to contact you subsequently, should I need clarification on anything you said. Once the interview is transcribed, I will send it to you and invite you to review it (if you agree to participate in the interview but have opted not to be audio-recorded, I will send you my detailed notes); you may make corrections, edits, or choose to delete sections, as you wish. Lastly, if something you said is directly quoted in my dissertation, I will share this section of my draft with you so you may see the context in which it was placed. If you have concerns about my use of your words, we can discuss these.  I will contact you by phone or email in the coming week to see if you are available for and interesting in participating in this study. You may, of course, email me before I contact you. If you are interested, I will ask you a few questions to help me in sample selection; this conversation should take no more than 10 minutes. Because I am recruiting a certain number of tenured/pre-tenured and female/male faculty members, I will continue recruiting until I have the number of participants needed for the study (a maximum of 24). However, even if I have all the participants I need for the study, I will contact all those who reply to this recruitment letter to explain why I no longer am looking for participants and to offer the opportunity to receive the results of the study.  The names of participating individuals will be kept confidential during the research process and in the presentation of the study findings. Participant identities will not be revealed to other study participants or department colleagues. The study has been approved by the UBC Behavioural Ethics Review Board.  I would be happy to talk about the project by phone and/or provide details via email. If there is any other information I can provide to help you make a decision about whether or not to participate, please do not hesitate to ask.  243  I look forward to hearing from you soon.  Sincerely,  Isabeau Iqbal, Doctoral Candidate Department of Educational Studies, UBC Isabeau.iqbal@ubc.ca   244  Appendix B Faculty Member Questionnaire Faculty members’ experiences and understandings of the summative peer review of teaching  Your responses to this questionnaire will be used in planning the study and may be used as part of the final reporting. Your name will not be associated with the final results. Should you wish to include additional details in response to any of the questions, please feel free to do so.  A. Name:  B. Department:  C. With which discipline(s) to you associate most closely?  D. What is your gender?  E. Please indicate how the co-investigator should follow up with you.  ____Phone. Phone number ___________________ ____Email. Email address ____________________  Questions:  1. What is your current title? (i.e. associate professor, full professor, instructor, other. Please be as specific as possible.)  2. Is yours a tenure track appointment?  ¨     Yes      ¨     No  3. Do you currently have tenure? If yes, please indicate the year that you obtained tenured.  ¨     Yes   Year:_____ ¨     No  4. If you are a pre-tenured professor, when (year) do you think you will apply for tenure?   5. When (year) did you begin working in a tenure-track position at (Name of Institution)?     245  6. Has your teaching been evaluated through the summative peer review of teaching? In this questionnaire, the summative peer review of teaching includes, but is not limited to, formal, in-class observations of teaching conducted by a colleague and used for the purpose of making personnel decisions pertaining to re-appointment, promotion, and tenure.  ¨     Yes       ¨     No  If you answered yes to #6, go to 6a. If you answered no to #6, skip to question 7.  6a. For what purpose did you go through the summative peer review of teaching? Choose as many as apply from the list below and indicate whether the review took place at (Name of Institution):  ¨     Reappointment review………………....At (Name of Institution)? Yes/No ¨     Promotion to associate professor………At (Name of Institution)? Yes/No ¨     Promotion to full professor…………….At (Name of Institution)? Yes/No ¨     Other, please specify:    7. Have you ever evaluated a colleague on their teaching for the purpose of helping a committee make a decision about tenure, promotion or re-appointment?  ¨     Yes   (please complete 7a)  ¨     No   7a. If you answered yes to Question 7, did you perform the summative peer review of teaching for a (Name of Institution)  colleague?  ¨     Yes      ¨     No   The end. Thank-you for completing this questionnaire!  Please return this form via email to Isabeau Iqbal (Isabeau.iqbal@ubc.ca) I will call you within one week of receiving this completed questionnaire.     246  Appendix C Follow-up Email to Assistant Professors and Women who had not Responded to Letter of Initial Contact Dear Dr. [Name]  A short while ago, I wrote to ask if you would consider participating in my dissertation research on the peer review of teaching.  I received a high response from tenured male faculty members in the Faculty of Science; but, given my study’s focus on rank and gender, I am keen to include the voices of pre-tenured faculty and women (at all ranks) from the Department of [Name].  If you can offer me 60-75 minutes of your time for an interview, anytime between now and into the Fall, I would be most grateful.  Please contact me via email or by phone (604-822-5811) if you have questions or would like to communicate your decision about participation.  If you prefer to decline participation but would like to provide me with information that can help me better understand the barriers to participating, this would be helpful both as I move forward with recruitment and with analyzing my data.  Thank-you,  Isabeau  Isabeau Iqbal PhD Candidate Department of Educational Studies University of British Columbia  247 	
   	
   Appendix D Interview Protocol for Pre-Tenured Faculty Members Context  • Thank-you for participating in study • Re-introduce study (purpose and what I hope to learn) • Discuss confidentiality • Ask for permission to record • Ask if participant has any questions or concerns • Request participant’s signature on consent form • Provide a copy of the consent form to the participant • (Have copy of ethics approval)  1. Please tell me about your involvement with the peer review of teaching. Probe: • at which point in appointment the review took place • who made up the committee • how the committee composition was determined • how the process unfolded before/during and after • whether and how the individual has been involved as a reviewer in the peer review of teaching  2. What would you say are the primary criteria or expectations for effective teaching in your department? Probe: • extent to which the faculty member associates these criteria with the department and/or discipline to which s/he belongs • how unique s/he perceives these criteria to be to his/her department and/or discipline  3. Do you think you and your senior colleagues share the same expectations about what effective teaching involves? [If not, why?]  4. What sources of information do you have about the criteria for evaluating teaching in this department? Probe extent to which sources include: • hallways conversations • departmental documents • Faculty documents • institutional documents • documents specific to the peer review of teaching    248 5. What do you think needs to happen for people to get a positive evaluation through the summative peer review of teaching? Probe • what the faculty member thinks would be the most relevant/useful/ethical process for peer review of teaching • what s/he believes would be the ideal committee composition • what s/he plans to do in future summative peer reviews of teaching based on her/his experience so far.  6. What do you think can lead to a negative evaluation of teaching through the summative peer review of teaching? Probe • what is considered inadequate teaching • forms of unfairness including interpersonal dynamics • lack of mentorship and other support  7.  Is there anything else you would like to say about your experience with the peer review of teaching process?  Interviewer to verify factual information (Co-investigator will have filled out answers based on the Faculty Member Questionnaire and will check for accuracy verbally):  Interviewee Title/Rank:  Department Affiliation:  Faculty Affiliation:  Discipline:  Thank-you!   Date of interview:   __________________________  Time of interview:  (start) __________________________(end)  Location of interview:  __________________________  Gender of interviewee: ___________________________     249 Appendix E Interview Protocol for Tenured Faculty Members Context • Thank-you for participating in study • Re-introduce study (purpose and what I hope to learn) • Discuss confidentiality • Ask for permission to record • Ask if participant has any questions or concerns • Request participant’s signature on consent form • Provide a copy of the consent form to the participant • (Have copy of ethics approval)  1. Please tell me about your involvement with the peer review of teaching. Probe: • their involvement in committees or otherwise • how--if at all--their understanding and experiences of summative peer review have changed over time, throughout their career • what might have contributed to changes in their perspectives/beliefs/understandings  2. What would you say are the primary criteria or expectations for effective teaching in your department? Probe: • extent to which the faculty member associates these criteria with the department and/or discipline to which s/he belongs • how unique s/he perceives these criteria to be to his/her department and/or discipline  3. Do you think your junior colleagues share the same expectations about what effective teaching involves? If not, why?  4. What sources of information do you have about the criteria for evaluating teaching in this department? Probe extent to which sources include: • hallways conversations • departmental documents • Faculty documents • institutional documents • documents specific to the peer review of teaching    250   5. What would you advise your junior colleagues if you were asked “What should I do to up my chances of getting a positive evaluation of teaching through the summative peer review of teaching?”? Probe • what the faculty member thinks would be the most relevant/useful/ethical process for peer review of teaching • what s/he believes would be the ideal committee composition  6. What do you think can lead to a negative evaluation of teaching through the summative peer review of teaching? Probe • what is considered inadequate teaching • forms of unfairness including interpersonal dynamics • lack of mentorship and other support  7. Is there anything else you would like to say about your experience with the peer review of teaching process?  Interviewer to verify factual information (Co-investigator will have filled out answers based on the Faculty Member Questionnaire and will check for accuracy verbally):  Interviewee Title/Rank:  Department Affiliation:  Faculty Affiliation:  Discipline:  Thank-you!  Date of interview:   __________________________  Time of interview:  (start) __________________________(end)  Location of interview:  __________________________  Gender of interviewee: ___________________________    251  Appendix F Informed Consent for Pre-Tenured Faculty Members Faculty members’ experiences and understandings of the summative peer review of teaching  Principal Investigator:   Thomas Sork, PhD Department of Educational Studies University of British Columbia Phone: 604-822-5702  Email: tom.sork@ubc.ca  Co-Investigator:  Isabeau Iqbal, MA     Doctoral Student     Department of Educational Studies     University of British Columbia     Phone: 604-255-5133  Email: isabeau.iqbal@ubc.ca  Background At the (Name of Institution), as in many other North American research-intensive universities, formal in-class observations of teaching (the summative peer review of teaching) are used, along with other information, to evaluate teaching performance for the purpose of making tenure and promotion decisions.  Although a vast literature on the peer review of teaching recognizes that academic culture impacts upon faculty members’ experiences of peer review, there is limited empirical research into the influence of gender, rank, departmental culture, and disciplinary culture on faculty members’ understandings and experiences of peer review.  Purpose This purpose of this study is to investigate faculty members’ understandings and experiences of the summative peer review of teaching giving special attention to the role of gender, rank and departmental/disciplinary culture.  Given the increasing emphasis (Name of Institution) is placing on summative peer review, as demonstrated by the new institutionally-adopted policies for this process, and the limited empirical research into the experiences of faculty members, there is a need to find out more about summative peer review from the perspective of academics.  Why am I being asked to participate in this research? You are being invited to take part in this research study because you meet the general criteria of being a full-time pre-tenured faculty member whose teaching may or may not have been evaluated through the summative peer review of teaching at least once during your appointment at (Name of Institution).     252 For this study, pre-tenured and tenured faculty members in departments within the Faculties of Arts and Science will be interviewed.  You are being approached in the hope that you will consider making a contribution to our collective understanding of summative peer evaluation of teaching.  Study procedures You will be asked to participate in one (1) in-person interview. The interview will take between 60 to 75 minutes, and you will be asked questions about your experiences and understandings of the summative peer review of teaching. For example, you will be asked about your own process of peer review, the expectations around good teaching in your department, and what an ideal summative peer review of teaching process might entail. With your permission, the interview will be audio-taped to help with the data analysis process. If you consent to participate in the individual interview but do not wish to have the interview audio recorded, your request will be accommodated.  Risks and Benefits One of the risks associated with this study is the possible discomfort discussing your career, teaching, institution, and collegial relationships. There are no other known risks of the proposed research.  If you decide during the interview, or at a later date, that you would like your comments removed from our records, your request will be accommodated.  The benefits of participating in this study include having the opportunity to discuss your experiences and understandings of the summative review of teaching. Doing so will allow you to reflect on your identity as a university teacher in relation to the expectations of your institution, department, discipline, and self.  The results of this research will help us better understand how gender, rank, departmental culture, and disciplinary culture influence academics’ understandings and experiences of the summative peer review of teaching process in a research-intensive environment.  Confidentiality If you consent to participate in this study, your identity will be kept strictly confidential. The completed consent form will be kept separate from data collected in order to protect your identity. All data will be stored in a secure manner in compliance with UBC’s Behavioural Research Ethics Board policies and, as per UBC policy, the files will be kept for five years. The only other people who may have access to the individual data will be my committee members, Dr. Sork, Dr. Poole, and Dr. Metcalfe who are subject to the same terms and conditions of confidentiality outlined in this document.  Compensation There is no monetary incentive for participating in this research.    253 How will results be used? The results of this study will be presented in research reports including: (a) my doctoral dissertation for the Department of Educational Studies at UBC, (b) publications and presentations for professional conferences, and (c) articles for professional journals and/or other publications.  Contact information about the study If you have any questions or desire further information about this study before or during participation, you can contact Isabeau Iqbal at 604-255-5133 or via email at isabeau.iqbal@ubc.ca.  If you have any concerns about your rights as a research subject and/or your experiences while participating in this study, contact the Research Subject Information Line in the University of British Columbia Office of Research Services by e-mail at RSIL@ors.ubc.ca or by phone at 604-822-8598.  Participant consent Your consent is required before you may participate in this study; however, your participation in this study is entirely voluntary. You may end the interview and/or withdraw at any time and without giving any reasons for your decision and without any consequences.  Please select from the statements below and sign under the one that indicates how you would like to participate in this study.  Your signature below indicates that you have received a copy of this consent form for your own records and that you consent to participate in this study and agree to have your interview audio-taped.    Participant’s Signature   Printed name   Date   Your signature below indicates that you have received a copy of this consent form for your own records and that you consent to participate in this study but do not want your interview audio-taped.    Participant’s Signature   Printed name   Date      254  Appendix G Informed Consent for Tenured Faculty Members Faculty members’ experiences and understandings of the summative peer review of teaching  Principal Investigator:   Thomas Sork, PhD Department of Educational Studies University of British Columbia Phone: 604-822-5702  Email: tom.sork@ubc.ca  Co-Investigator:  Isabeau Iqbal, MA     Doctoral Student     Department of Educational Studies     University of British Columbia     Phone: 604-255-5133  Email: isabeau.iqbal@ubc.ca  Background At the (Name of Institution), as in many other North American research-intensive universities, formal in-class observations of teaching (the summative peer review of teaching) are used, along with other information, to evaluate teaching performance for the purpose of making tenure and promotion decisions.  Although a vast literature on the peer review of teaching recognizes that academic culture impacts upon faculty members’ experiences of peer review, there is limited empirical research into the influence of gender, rank, departmental culture, and disciplinary culture on faculty members’ understandings and experiences of peer review.  Purpose This purpose of this study is to investigate faculty members’ understandings and experiences of the summative peer review of teaching giving special attention to the role of gender, rank and departmental/disciplinary culture.  Given the increasing emphasis (Name of Institution) is placing on summative peer review, as demonstrated by the new institutionally-adopted policies for this process, and the limited empirical research into the experiences of faculty members, there is a need to find out more about summative peer review from the perspective of academics.  Why am I being asked to participate in this research? You are being invited to take part in this research study because you meet the general criteria of being a full-time tenured faculty member at (Name of Institution). If you are a Full professor, you will have conducted the summative peer review of teaching once or more at (Name of Institution) for the purpose of making a personnel decision pertaining to tenure. If you are an associate professor, you may or may not have evaluated a colleague through the peer review of teaching.  255   For this study, pre-tenured and tenured faculty members in departments within the Faculties of Arts and Science will be interviewed.  You are being approached in the hope that you will consider making a contribution to our collective understanding of summative peer evaluation of teaching.  Study procedures You will be asked to participate in one (1) in-person interview. The interview will take between 60 to 75 minutes, and you will be asked questions about your experiences and understandings of the summative peer review of teaching. For example, you will be asked about your own process of peer review, what an ideal summative peer review of teaching process might entail, and how you might advise junior colleagues on preparing for a successful peer review of teaching. With your permission, the interview will be audio-taped to help with the data analysis process. If you consent to participate in the individual interview but do not wish to have the interview audio recorded, your request will be accommodated.  Risks and Benefits One of the risks associated with this study is the possible discomfort discussing your career, teaching, institution, and collegial relationships. There are no other known risks of the proposed research.  If you decide during the interview, or at a later date, that you would like your comments removed from our records, your request will be accommodated.  The benefits of participating in this study include having the opportunity to discuss your experiences and understandings of the summative review of teaching. Doing so will allow you to reflect on your identity as a university teacher in relation to the expectations of your institution, department, discipline, and self.  The results of this research will help us better understand how gender, rank, departmental culture, and disciplinary culture influence academics’ understandings and experiences of the summative peer review of teaching process in a research-intensive environment.  Confidentiality If you consent to participate in this study, your identity will be kept strictly confidential. The completed consent form will be kept separate from data collected in order to protect your identity. All data will be stored in a secure manner in compliance with UBC’s Behavioural Research Ethics Board policies and, as per UBC policy, the files will be kept for five years. The only other people who may have access to the individual data will be my committee members, Dr. Sork, Dr. Poole, and Dr. Metcalfe who are subject to the same terms and conditions of confidentiality outlined in this document.  Compensation There is no monetary incentive for participating in this research.  256  How will results be used? The results of this study will be presented in research reports including: (a) my doctoral dissertation for the Department of Educational Studies at UBC, (b) publications and presentations for professional conferences, and (c) articles for professional journals and/or other publications.  Contact information about the study If you have any questions or desire further information about this study before or during participation, you can contact Isabeau Iqbal at 604-255-5133 or via email at isabeau.iqbal@ubc.ca.  If you have any concerns about your rights as a research subject and/or your experiences while participating in this study, contact the Research Subject Information Line in the University of British Columbia Office of Research Services by e-mail at RSIL@ors.ubc.ca or by phone at 604-822-8598.  Participant consent Your consent is required before you may participate in this study; however, your participation in this study is entirely voluntary. You may end the interview and/or withdraw at any time and without giving any reasons for your decision and without any consequences.  Please select from the statements below and sign under the one that indicates how you would like to participate in this study.  Your signature below indicates that you have received a copy of this consent form for your own records and that you consent to participate in this study and to have your interview audio-taped.    Participant’s Signature   Printed name   Date   Your signature below indicates that you have received a copy of this consent form for your own records and that you consent to participate in this study but do not want your interview audio-taped.    Participant’s Signature   Printed name   Date       257  Appendix H Range of Participants’ Responses about what Constitutes an Ideal Summative Peer Review of Teaching Process Purpose of summative peer review (why) • evaluation of teaching for tenure, promotion and/or reappointment • professional growth in teaching • multiple purposes advisable (yes/no) • add to information provided by student evaluations of teaching  Selection of reviewers (who) • by department head • candidate has input • reviewers are located within the department • reviewers are located outside of department  Timing (when) • early in career • regular reviews  Qualities of reviewer (what) • has content knowledge • has pedagogical knowledge • is perceived as a good teacher by the candidate • is trusted by candidate • is well-intentioned • is trained in conducting peer reviews of teaching • accepts and values diverse teaching approaches • candidate and reviewer have similar approaches to teaching • promotes reflection on teaching • senior in rank • rank may be above, below, or equal to candidate  Aspects of teaching reviewed (what) • classroom observation of teaching • teaching philosophy statement, course materials • undergraduate class, graduate class • graduate student supervision • honours student supervision • student learning     258 Communication (how) • process known to candidate and reviewer • candidate and reviewers understand how results will be used in decisions about career advancement • candidate and reviewers understand how results will be used for professional growth • candidate and reviewers understand the relationship between peer reviews and student evaluations of teaching • results are communicated to candidate (by department head, by reviewers) • results communicated in writing and verbally to candidate • reviewer feedback is specific and helps candidate develop as a teacher • candidate and reviewer debrief the peer review process • use of criteria and standards preferred • guidelines for conducting peer reviews preferred • flexible and open process preferred (no criteria/standards; no guidelines) • candidate knows when reviewer will attend class (yes/no) • discussion pre- and post- between candidate and reviewer(s) • balanced view presented (candidate’s strengths and areas for improvement) • ongoing, informal formative reviews incorporated • mindful of faculty members’ time 	
    

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.24.1-0064685/manifest

Comment

Related Items