"Education, Faculty of"@en . "Educational Studies (EDST), Department of"@en . "DSpace"@en . "UBCV"@en . "Iqbal, Isabeau"@en . "2012-06-11T18:06:33Z"@en . "2012"@en . "Doctor of Philosophy - PhD"@en . "University of British Columbia"@en . "This study investigated the ways that summative peer review of teaching contributes to tenure-track faculty members\u00E2\u0080\u0099 professional growth in teaching. It also explored other practices that support or hinder a departmental culture that values teaching. \n\nUsing the lens of academic culture, I drew on literature about the peer review of teaching, department culture, and professional growth in academic careers to inform this research. Thirty tenure-track faculty members from six departments and two faculties participated in semi-structured qualitative interviews. Participants were asked about their experiences of summative peer review, how they understood the relationship between peer review and their growth as instructors, and departmental practices that contribute to a culture that values teaching.\n\nParticipants had varied and inconsistent experiences of summative peer review of teaching. They reported multiple purposes (evaluative, formative, supplement to the student evaluations of teaching) that frequently conflicted. With few known guidelines that direct peer reviews and insufficient clarity as to their purpose, faculty members conducted summative reviews based on a personal sense of \u00E2\u0080\u009Cwhat was best.\u00E2\u0080\u009D Given the demanding nature of academic careers and an institutional reward system that favours research over teaching, peer reviews were primarily limited to classroom observations and engaged few faculty members in dialogue. Such summative peer reviews appeared to make minimal contribution to professional growth in teaching. \nThe study did find numerous other departmental practices conducive to a culture that values teaching, e.g., informal collegial conversations about teaching and team teaching. Faculty members who partook in these grew as instructors. Results demonstrated that academic values and norms (i.e., collegiality and autonomy), disciplinary traditions pertaining to collaboration, and institutional rewards influenced how faculty members pursued professional growth as teachers."@en . "https://circle.library.ubc.ca/rest/handle/2429/42465?expand=metadata"@en . "FACULTY MEMBERS\u00E2\u0080\u0099 PROFESSIONAL GROWTH IN TEACHING THROUGH THE SUMMATIVE PEER REVIEW OF TEACHING AND OTHER DEPARTMENTAL PRACTICES by Isabeau Anisa Iqbal M.A., The University of British Columbia, 2004 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE STUDIES (Educational Studies) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) June 2012 \u00C2\u00A9 Isabeau Anisa Iqbal, 2012 ii Abstract This study investigated the ways that summative peer review of teaching contributes to tenure-track faculty members\u00E2\u0080\u0099 professional growth in teaching. It also explored other practices that support or hinder a departmental culture that values teaching. Using the lens of academic culture, I drew on literature about the peer review of teaching, department culture, and professional growth in academic careers to inform this research. Thirty tenure-track faculty members from six departments and two faculties participated in semi-structured qualitative interviews. Participants were asked about their experiences of summative peer review, how they understood the relationship between peer review and their growth as instructors, and departmental practices that contribute to a culture that values teaching. Participants had varied and inconsistent experiences of summative peer review of teaching. They reported multiple purposes (evaluative, formative, supplement to the student evaluations of teaching) that frequently conflicted. With few known guidelines that direct peer reviews and insufficient clarity as to their purpose, faculty members conducted summative reviews based on a personal sense of \u00E2\u0080\u009Cwhat was best.\u00E2\u0080\u009D Given the demanding nature of academic careers and an institutional reward system that favours research over teaching, peer reviews were primarily limited to classroom observations and engaged few faculty members in dialogue. Such summative peer reviews appeared to make minimal contribution to professional growth in teaching. iii The study did find numerous other departmental practices conducive to a culture that values teaching, e.g., informal collegial conversations about teaching and team teaching. Faculty members who partook in these grew as instructors. Results demonstrated that academic values and norms (i.e., collegiality and autonomy), disciplinary traditions pertaining to collaboration, and institutional rewards influenced how faculty members pursued professional growth as teachers. iv Preface The protocol for this research study was approved by the UBC Research Ethics Board (Behavioural Research Ethics Board; UBC BREB Number: H10-00389). v Table of Contents Abstract ..................................................................................................................................... ii\t\r \u00C2\u00A0 Preface ...................................................................................................................................... iv\t\r \u00C2\u00A0 Table of Contents ...................................................................................................................... v\t\r \u00C2\u00A0 List of Tables ......................................................................................................................... xiii\t\r \u00C2\u00A0 List of Figures ........................................................................................................................ xiv\t\r \u00C2\u00A0 Acknowledgements ................................................................................................................. xv\t\r \u00C2\u00A0 Chapter 1: Introduction ........................................................................................................ 1\t\r \u00C2\u00A0 1.1\t\r \u00C2\u00A0 Presentation of the Problem ..................................................................................... 2\t\r \u00C2\u00A0 1.2\t\r \u00C2\u00A0 Research Purpose ..................................................................................................... 4\t\r \u00C2\u00A0 1.3\t\r \u00C2\u00A0 Research Objectives ................................................................................................. 5\t\r \u00C2\u00A0 1.4\t\r \u00C2\u00A0 Conceptual Framework ............................................................................................ 7\t\r \u00C2\u00A0 1.5\t\r \u00C2\u00A0 Research Significance ............................................................................................ 11\t\r \u00C2\u00A0 1.6\t\r \u00C2\u00A0 Research Questions ................................................................................................ 12\t\r \u00C2\u00A0 1.7\t\r \u00C2\u00A0 Dissertation Structure ............................................................................................. 12\t\r \u00C2\u00A0 Chapter 2: Literature Review ............................................................................................. 13\t\r \u00C2\u00A0 2.1\t\r \u00C2\u00A0 The Peer Review of Teaching ................................................................................ 13\t\r \u00C2\u00A0 2.1.1\t\r \u00C2\u00A0 Peer Review of Teaching and Professional Growth ....................................... 14\t\r \u00C2\u00A0 2.1.1.1\t\r \u00C2\u00A0 Collective Dialogue about Teaching ........................................................ 16\t\r \u00C2\u00A0 2.1.1.2\t\r \u00C2\u00A0 Refining Characteristics of Good Teaching ............................................. 17\t\r \u00C2\u00A0 2.1.2\t\r \u00C2\u00A0 Resistance to Peer Review of Teaching .......................................................... 18\t\r \u00C2\u00A0 2.1.2.1\t\r \u00C2\u00A0 Time ......................................................................................................... 19\t\r \u00C2\u00A0 vi 2.1.2.2\t\r \u00C2\u00A0 Identifying a Suitable Peer ....................................................................... 19\t\r \u00C2\u00A0 2.1.2.3\t\r \u00C2\u00A0 Feedback Challenges ............................................................................... 20\t\r \u00C2\u00A0 2.1.2.4\t\r \u00C2\u00A0 Reliability and Validity Issues ................................................................. 23\t\r \u00C2\u00A0 2.1.2.5\t\r \u00C2\u00A0 Infringement on Autonomy ...................................................................... 26\t\r \u00C2\u00A0 2.2\t\r \u00C2\u00A0 Summary: Peer Review of Teaching Literature ..................................................... 27\t\r \u00C2\u00A0 2.3\t\r \u00C2\u00A0 Department Culture ................................................................................................ 28\t\r \u00C2\u00A0 2.3.1\t\r \u00C2\u00A0 Academic Culture, Values and Norms ............................................................ 29\t\r \u00C2\u00A0 2.3.2\t\r \u00C2\u00A0 The Academic Department and Culture(s) of Teaching ................................. 30\t\r \u00C2\u00A0 2.3.2.1\t\r \u00C2\u00A0 Learning Conversations and Professional Growth in Teaching .............. 32\t\r \u00C2\u00A0 2.3.2.2\t\r \u00C2\u00A0 Departmental Leadership in Teaching ..................................................... 37\t\r \u00C2\u00A0 2.3.3\t\r \u00C2\u00A0 Summary: Department Culture Literature ...................................................... 40\t\r \u00C2\u00A0 2.4\t\r \u00C2\u00A0 Summary ................................................................................................................ 40\t\r \u00C2\u00A0 Chapter 3: Methodology ...................................................................................................... 42\t\r \u00C2\u00A0 3.1\t\r \u00C2\u00A0 Introduction ............................................................................................................ 42\t\r \u00C2\u00A0 3.2\t\r \u00C2\u00A0 Positioning Myself ................................................................................................. 42\t\r \u00C2\u00A0 3.2.1\t\r \u00C2\u00A0 Intellectual Position ........................................................................................ 43\t\r \u00C2\u00A0 3.2.2\t\r \u00C2\u00A0 Professional/Personal Position ........................................................................ 45\t\r \u00C2\u00A0 3.3\t\r \u00C2\u00A0 Changes to the Original Research Questions ......................................................... 46\t\r \u00C2\u00A0 3.4\t\r \u00C2\u00A0 Selecting Faculties, Departments and Faculty Members ....................................... 48\t\r \u00C2\u00A0 3.4.1\t\r \u00C2\u00A0 Institution ........................................................................................................ 49\t\r \u00C2\u00A0 3.4.2\t\r \u00C2\u00A0 Faculties .......................................................................................................... 50\t\r \u00C2\u00A0 3.4.3\t\r \u00C2\u00A0 Departments .................................................................................................... 51\t\r \u00C2\u00A0 3.4.4\t\r \u00C2\u00A0 Defining the Professor .................................................................................... 52\t\r \u00C2\u00A0 vii 3.4.5\t\r \u00C2\u00A0 Recruitment: Beginnings ................................................................................ 54\t\r \u00C2\u00A0 3.4.5.1\t\r \u00C2\u00A0 Challenges Recruiting Assistant Professors and Women ........................ 55\t\r \u00C2\u00A0 3.5\t\r \u00C2\u00A0 Interviews ............................................................................................................... 58\t\r \u00C2\u00A0 3.5.1\t\r \u00C2\u00A0 Data Collection ............................................................................................... 58\t\r \u00C2\u00A0 3.5.2\t\r \u00C2\u00A0 Methodological Issues .................................................................................... 59\t\r \u00C2\u00A0 3.6\t\r \u00C2\u00A0 Data Analysis ......................................................................................................... 62\t\r \u00C2\u00A0 3.7\t\r \u00C2\u00A0 Rigour and Limitations .......................................................................................... 64\t\r \u00C2\u00A0 3.7.1\t\r \u00C2\u00A0 Validity ........................................................................................................... 64\t\r \u00C2\u00A0 3.7.2\t\r \u00C2\u00A0 Reliability ........................................................................................................ 66\t\r \u00C2\u00A0 3.7.3\t\r \u00C2\u00A0 Generalizability ............................................................................................... 67\t\r \u00C2\u00A0 3.8\t\r \u00C2\u00A0 Ethical Considerations ........................................................................................... 69\t\r \u00C2\u00A0 3.8.1\t\r \u00C2\u00A0 Respect ............................................................................................................ 69\t\r \u00C2\u00A0 3.8.2\t\r \u00C2\u00A0 Privacy and Confidentiality ............................................................................ 69\t\r \u00C2\u00A0 3.8.3\t\r \u00C2\u00A0 Informed Consent ............................................................................................ 70\t\r \u00C2\u00A0 3.8.4\t\r \u00C2\u00A0 Institutional Review Board Approval ............................................................. 70\t\r \u00C2\u00A0 Chapter 4: Summative Peer Review of Teaching: Current Purposes and Envisioned Ideals ...................................................................................................................................... 71\t\r \u00C2\u00A0 4.1\t\r \u00C2\u00A0 Chapter Structure ................................................................................................... 71\t\r \u00C2\u00A0 4.2\t\r \u00C2\u00A0 Institutional Culture: Endless Demands, Finite Time ............................................ 72\t\r \u00C2\u00A0 4.2.1\t\r \u00C2\u00A0 Teaching Counts (?) ........................................................................................ 74\t\r \u00C2\u00A0 4.3\t\r \u00C2\u00A0 Summative Peer Review of Teaching: A Sketch ................................................... 78\t\r \u00C2\u00A0 4.3.1\t\r \u00C2\u00A0 Changes Over Time ........................................................................................ 79\t\r \u00C2\u00A0 4.3.2\t\r \u00C2\u00A0 Timing and Frequency .................................................................................... 80\t\r \u00C2\u00A0 viii 4.3.3\t\r \u00C2\u00A0 Selection of Reviewers ................................................................................... 81\t\r \u00C2\u00A0 4.3.4\t\r \u00C2\u00A0 What is Being Reviewed? ............................................................................... 82\t\r \u00C2\u00A0 4.3.5\t\r \u00C2\u00A0 Communication Pre/Post Review ................................................................... 83\t\r \u00C2\u00A0 4.4\t\r \u00C2\u00A0 The Multiple Purposes of Summative Peer Review of Teaching .......................... 84\t\r \u00C2\u00A0 4.4.1\t\r \u00C2\u00A0 Formal Evaluation for Tenure and Promotion ................................................ 84\t\r \u00C2\u00A0 4.4.2\t\r \u00C2\u00A0 Professional Growth in Teaching .................................................................... 88\t\r \u00C2\u00A0 4.4.3\t\r \u00C2\u00A0 Supplement the Student Evaluations of Teaching .......................................... 90\t\r \u00C2\u00A0 4.5\t\r \u00C2\u00A0 No Teaching-Related Professional Growth ........................................................... 92\t\r \u00C2\u00A0 4.5.1\t\r \u00C2\u00A0 Feedback: Non-existent or Unspecific ............................................................ 92\t\r \u00C2\u00A0 4.5.1.1\t\r \u00C2\u00A0 Feedback from the Department Head ...................................................... 95\t\r \u00C2\u00A0 4.5.1.2\t\r \u00C2\u00A0 Why No Feedback? .................................................................................. 97\t\r \u00C2\u00A0 4.5.2\t\r \u00C2\u00A0 Student Evaluations of Teaching Set the Standards ..................................... 103\t\r \u00C2\u00A0 4.5.3\t\r \u00C2\u00A0 Artificial ........................................................................................................ 107\t\r \u00C2\u00A0 4.6\t\r \u00C2\u00A0 Ideal Process ........................................................................................................ 108\t\r \u00C2\u00A0 4.6.1\t\r \u00C2\u00A0 Flexibility/Structure ...................................................................................... 108\t\r \u00C2\u00A0 4.6.2\t\r \u00C2\u00A0 Ongoing, Informal Process ........................................................................... 110\t\r \u00C2\u00A0 4.6.3\t\r \u00C2\u00A0 Feedback Quality and Sharing ...................................................................... 112\t\r \u00C2\u00A0 4.6.4\t\r \u00C2\u00A0 Reviewer-Reviewed Connection .................................................................. 113\t\r \u00C2\u00A0 4.6.4.1\t\r \u00C2\u00A0 Disciplinary Connection ........................................................................ 114\t\r \u00C2\u00A0 4.6.4.2\t\r \u00C2\u00A0 Pedagogical Knowledge ......................................................................... 115\t\r \u00C2\u00A0 4.6.4.3\t\r \u00C2\u00A0 Philosophies of Teaching ....................................................................... 116\t\r \u00C2\u00A0 4.6.4.4\t\r \u00C2\u00A0 Gender .................................................................................................... 116\t\r \u00C2\u00A0 4.6.4.5\t\r \u00C2\u00A0 Rank ....................................................................................................... 118\t\r \u00C2\u00A0 ix 4.6.5\t\r \u00C2\u00A0 More, Sooner ................................................................................................. 118\t\r \u00C2\u00A0 4.7\t\r \u00C2\u00A0 Summary of Findings ........................................................................................... 120\t\r \u00C2\u00A0 4.8\t\r \u00C2\u00A0 Discussion: Summative Peer Review and Professional Growth in Teaching ...... 123\t\r \u00C2\u00A0 4.8.1\t\r \u00C2\u00A0 Unsettling/Honouring Collegiality ................................................................ 124\t\r \u00C2\u00A0 4.8.1.1\t\r \u00C2\u00A0 Feedback Unease ................................................................................... 124\t\r \u00C2\u00A0 4.8.1.2\t\r \u00C2\u00A0 The Primacy of Tenure .......................................................................... 125\t\r \u00C2\u00A0 4.8.1.3\t\r \u00C2\u00A0 Teaching as Private ................................................................................ 126\t\r \u00C2\u00A0 4.8.2\t\r \u00C2\u00A0 Universalism (or We Should Not be Evaluating from a Gut Feeling) .......... 127\t\r \u00C2\u00A0 4.8.3\t\r \u00C2\u00A0 Student Evaluation of Teaching Reigns ........................................................ 129\t\r \u00C2\u00A0 4.8.4\t\r \u00C2\u00A0 Slow Change in Academia ............................................................................ 131\t\r \u00C2\u00A0 Chapter 5: Departmental Practices that Promote or Hinder a Culture that Values Teaching ............................................................................................................................... 135\t\r \u00C2\u00A0 5.1\t\r \u00C2\u00A0 Chapter Structure ................................................................................................. 135\t\r \u00C2\u00A0 5.2\t\r \u00C2\u00A0 Informal Departmental Practices ......................................................................... 137\t\r \u00C2\u00A0 5.2.1\t\r \u00C2\u00A0 Informal Conversations with Colleagues ...................................................... 138\t\r \u00C2\u00A0 5.2.2\t\r \u00C2\u00A0 Collaborative/Individual Teaching Approaches ........................................... 139\t\r \u00C2\u00A0 5.2.3\t\r \u00C2\u00A0 Departmental Composition ........................................................................... 141\t\r \u00C2\u00A0 5.2.4\t\r \u00C2\u00A0 Good Teaching: No Consensus ..................................................................... 145\t\r \u00C2\u00A0 5.2.5\t\r \u00C2\u00A0 Risk-taking and Trust .................................................................................... 146\t\r \u00C2\u00A0 5.2.6\t\r \u00C2\u00A0 Attention to Fairness ..................................................................................... 149\t\r \u00C2\u00A0 5.3\t\r \u00C2\u00A0 Organized Initiatives, Policies and Structures ..................................................... 151\t\r \u00C2\u00A0 5.3.1\t\r \u00C2\u00A0 Recognized Policies and Other Documents .................................................. 151\t\r \u00C2\u00A0 5.3.2\t\r \u00C2\u00A0 Official Go-to People .................................................................................... 154\t\r \u00C2\u00A0 x 5.3.3\t\r \u00C2\u00A0 Effective Science Education Project (ESEd) ................................................ 156\t\r \u00C2\u00A0 5.3.4\t\r \u00C2\u00A0 Teaching Seminars and Retreats ................................................................... 157\t\r \u00C2\u00A0 5.3.5\t\r \u00C2\u00A0 Team Teaching Across Disciplines .............................................................. 159\t\r \u00C2\u00A0 5.3.6\t\r \u00C2\u00A0 Formative Peer Review ................................................................................. 160\t\r \u00C2\u00A0 5.3.7\t\r \u00C2\u00A0 Teaching and the Hiring Process .................................................................. 161\t\r \u00C2\u00A0 5.4\t\r \u00C2\u00A0 The Department Head\u00E2\u0080\u0099s Role in Promoting a Culture that Values Teaching ...... 164\t\r \u00C2\u00A0 5.4.1\t\r \u00C2\u00A0 Teaching and Tenure: Support from the Department Head .......................... 165\t\r \u00C2\u00A0 5.5\t\r \u00C2\u00A0 Summary of Findings ........................................................................................... 170\t\r \u00C2\u00A0 5.6\t\r \u00C2\u00A0 Discussion: Departmental Practices that Support or Hinder a Culture that Values Teaching ........................................................................................................................ 171\t\r \u00C2\u00A0 5.6.1\t\r \u00C2\u00A0 Institutional Culture ...................................................................................... 172\t\r \u00C2\u00A0 5.6.2\t\r \u00C2\u00A0 Departmental Culture .................................................................................... 175\t\r \u00C2\u00A0 5.6.2.1\t\r \u00C2\u00A0 Learning Conversations ......................................................................... 177\t\r \u00C2\u00A0 5.6.3\t\r \u00C2\u00A0 Disciplinary Culture and Conversations ....................................................... 179\t\r \u00C2\u00A0 Chapter 6: Summary, Conclusions, and Recommendations ......................................... 181\t\r \u00C2\u00A0 6.1\t\r \u00C2\u00A0 Summary of the Study ......................................................................................... 181\t\r \u00C2\u00A0 6.1.1\t\r \u00C2\u00A0 Statement of the Problem .............................................................................. 181\t\r \u00C2\u00A0 6.1.2\t\r \u00C2\u00A0 Research Questions ....................................................................................... 183\t\r \u00C2\u00A0 6.1.3\t\r \u00C2\u00A0 Conceptual Framework ................................................................................. 184\t\r \u00C2\u00A0 6.1.4\t\r \u00C2\u00A0 Study Participants and Data Collection ........................................................ 184\t\r \u00C2\u00A0 6.1.5\t\r \u00C2\u00A0 Data Analysis ................................................................................................ 185\t\r \u00C2\u00A0 6.1.6\t\r \u00C2\u00A0 Empirical Summary ...................................................................................... 185\t\r \u00C2\u00A0 6.1.7\t\r \u00C2\u00A0 Limitations .................................................................................................... 189\t\r \u00C2\u00A0 xi 6.2\t\r \u00C2\u00A0 Contributions to Theory ....................................................................................... 190\t\r \u00C2\u00A0 6.2.1\t\r \u00C2\u00A0 Description of the Two Conceptual Models ................................................. 192\t\r \u00C2\u00A0 6.2.1.1\t\r \u00C2\u00A0 Model for Faculty Member Engagement in Summative Peer Review of Teaching and Professional Growth in Teaching (Model A) ................................. 193\t\r \u00C2\u00A0 6.2.1.2\t\r \u00C2\u00A0 Model for Faculty Member Engagement in Departmental Practices that Foster a Culture that Values Teaching and Professional Growth in Teaching (Model B) .............................................................................................................. 197\t\r \u00C2\u00A0 6.3\t\r \u00C2\u00A0 Recommendations for Practice ............................................................................ 201\t\r \u00C2\u00A0 6.3.1\t\r \u00C2\u00A0 Summative Peer Review of Teaching ........................................................... 201\t\r \u00C2\u00A0 6.3.1.1\t\r \u00C2\u00A0 Involve the Department Head ................................................................ 201\t\r \u00C2\u00A0 6.3.1.2\t\r \u00C2\u00A0 Improve the Feedback Process ............................................................... 205\t\r \u00C2\u00A0 6.3.2\t\r \u00C2\u00A0 Fostering a Departmental Culture that Values Teaching .............................. 207\t\r \u00C2\u00A0 6.3.2.1\t\r \u00C2\u00A0 Increase Adoption and Full Integration of Improved Teaching Practices\u00E2\u0080\u00A6\u00E2\u0080\u00A6.. ...................................................................................................... 208\t\r \u00C2\u00A0 6.4\t\r \u00C2\u00A0 Recommendations for Future Research ............................................................... 211\t\r \u00C2\u00A0 6.4.1\t\r \u00C2\u00A0 Research on the Summative Peer Review of Teaching ................................ 211\t\r \u00C2\u00A0 6.4.2\t\r \u00C2\u00A0 Future Research on Departmental Culture that Captures Greater Faculty Member Diversity ..................................................................................................... 213\t\r \u00C2\u00A0 6.4.3\t\r \u00C2\u00A0 Research on the Role of the Department Head in Promoting Professional Growth in Teaching .................................................................................................. 215\t\r \u00C2\u00A0 6.4.4\t\r \u00C2\u00A0 Research on the Role of Pre-tenured Faculty Members\u00E2\u0080\u0099 in Shaping a Culture of Teaching ............................................................................................................... 215\t\r \u00C2\u00A0 6.5\t\r \u00C2\u00A0 Closing Comments ............................................................................................... 217\t\r \u00C2\u00A0 xii References ............................................................................................................................. 219\t\r \u00C2\u00A0 Appendices ............................................................................................................................ 242\t\r \u00C2\u00A0 xiii List of Tables Table 1: Faculty of Arts: Representation by Rank and Gender in A1, A2, and A3 ............... 52\t\r \u00C2\u00A0 Table 2: Faculty of Science: Representation by Rank and Gender in S1, S2, and S3 ............ 52\t\r \u00C2\u00A0 Table 3: Participant Eligibility Criteria ................................................................................... 55\t\r \u00C2\u00A0 Table 4: Responses to Follow up Email: Reasons for Not Participating ................................ 56\t\r \u00C2\u00A0 Table 5: Final Recruitment from Faculty of Arts (n=15) ....................................................... 57\t\r \u00C2\u00A0 Table 6: Final Recruitment from Faculty of Science (n=15) .................................................. 57\t\r \u00C2\u00A0 Table 7: Categories and Codes (Select Examples) ................................................................. 63\t\r \u00C2\u00A0 Table 8: Informal and Formal Initiatives and Practices that Influence the Culture of Teaching within a Department .............................................................................................................. 163\t\r \u00C2\u00A0 Table 9: Dominant Element of Culture at Play in Models of Professional Growth in Teaching ............................................................................................................................................... 200\t\r \u00C2\u00A0 Table 10: Summary of Recommendations for Practice ........................................................ 210\t\r \u00C2\u00A0 Table 11: Summary of Recommendations for Future Research ........................................... 216\t\r \u00C2\u00A0 xiv List of Figures Figure 1: Conceptual Model for Departmental Practices and Professional Growth in Teaching ................................................................................................................................................... 8\t\r \u00C2\u00A0 Figure 2: Model for Faculty Member Engagement in Summary Peer Review of Teaching and Professional Growth in Teaching (Model A) ........................................................................ 191\t\r \u00C2\u00A0 Figure 3: Model for Faculty Member Engagement in Non-Evaluative Departmental Practices that Foster a Culture that Values Teaching and Professional Growth in Teaching (Model B) ............................................................................................................................................... 192\t\r \u00C2\u00A0 xv Acknowledgements My family, though small, has given me enormous support. Foremost, I thank you, Ra\u00C3\u00BAl, for believing in me siempre. I look forward to more silliness, dancing, and adventures. I thank my father for his steady help, my mother for teaching me to be curious and to enjoy my own company, and Diane for her encouragement. I am profoundly grateful to my children who, in many ways, were oblivious to my studies and invited me to play and be creative (when I was inclined to work and worry). Being a mother helped me be a happier, healthier student. I am appreciative of the guidance provided by my supervisory committee, three individuals I deeply respect. I thank Dr. Gary Poole for his enthusiastic and wonderful mentorship, Dr. Tom Sork for his remarkable ability at (always) finding practical, helpful solutions, and Dr. Amy Scott Metcalfe for introducing me to new ideas and bodies of literature. You were reliable advisors who gave me the right combination of independence and direction. I give heartfelt thanks to the 30 faculty members who participated in this study and taught me so much. This research was also made possible by the Joseph-Armand Bombardier Doctoral Scholarship from the Social Sciences and Humanities Research Council. I have benefited from the help of countless other people: my dissertation sisters\u00E2\u0080\u0094Nora, Gen, Tara, Erin, Maryam, Hanae\u00E2\u0080\u0094the bright and funny women I met with weekly to discuss work and other aspects of the PhD journey; the mighty VR warriors and friends on Phinished; Dr. Allison Tom, with whom I shared delightful and fascinating conversations about methodology; the women in my EDST writing group; and my TAG and CTLT colleagues. xvi A ton of love goes to my closest friends who have cheered me on: Alejandra, Allison, Barbara, Claire, Erin, Jessie, Julia, Madeline, Michelle, Susan, and Susannah. 1 Chapter 1: Introduction This study examines the ways that summative peer review of teaching and other departmental teaching-related practices contribute to tenure-track professors\u00E2\u0080\u0099 professional growth in teaching. Faculty members\u00E2\u0080\u0099 professional growth in teaching is vital to their work (Clegg, 2003; Gappa, Austin, & Trice, 2007; O\u00E2\u0080\u0099Meara, Terosky & Neumann, 2008) because teaching is a chief responsibility in academic careers (Schuster & Finkelstein, 2006). Continuous learning, which has been described as the \u00E2\u0080\u009Cjob of a good scholar and teacher\u00E2\u0080\u009D (Schuster & Finkelstein, 2006, p. 78), is considered necessary for faculty members who wish to work productively and creatively in their teaching (Gappa et al., 2007). In addition, the rapidly changing educational context demands ongoing growth. Pressure from the public to improve the learning experience of students, political priorities concerned with augmenting the accountability of higher education institutions, the expectations of an increasingly diverse student population, and advances in technology all require that faculty members regularly engage in ongoing learning (Sorcinelli, Austin, Eddy, & Beach, 2006). When academics grow and learn, they acquire new perspectives, values, skills, and knowledge that they integrate into their practice (Dall\u00E2\u0080\u0099Alba & Sandberg, 2006). Ongoing professional growth allows faculty members to better fulfill their professional teaching roles and responsibilities. Professional growth, however, is not required solely to fulfill professional obligations. Faculty members care about their learning, and the constructive growth generated through learning matters a great deal to academics; thus, learning contributes strongly to their career satisfaction (Neumann, 2009). Since career satisfaction among academics, which has 2 declined significantly over the past decade (Schuster & Finklestein, 2006), has an impact on career success (Cariaga-Lo, Worthy Dawkins, Enger, Schotter, & Spence, 2010), addressing faculty growth is fundamental to the academic endeavour. 1.1 Presentation of the Problem Professional growth in teaching allows faculty members to better realize their professional teaching roles and responsibilities (Neumann, 2009; O\u00E2\u0080\u0099Meara et al., 2008; Sorcinelli et al., 2006); however, within research-intensive universities, there are many constraints that limit faculty members\u00E2\u0080\u0099 engagement in their own professional growth for teaching (Caffarella & Zinn, 1999; Clegg, 2003; Gappa et al., 2007; Kilgore & Cook, 2007; O\u00E2\u0080\u0099Meara et al., 2008). One of the main constraints academics face is the priority given to research over teaching. That is, compared to teaching, faculty members receive more positive reinforcement and are more often compensated for their research, via merit pay, promotion, awards, and travel money (Chalmers, 2010; Hardy & Smith, 2006; Kilgore & Cook; 2007; Rhode, 2006; Serow, 2000). The preference given to research matters, because institutional reward systems are the primary way by which faculty members\u00E2\u0080\u0099 professional work is valued. Consequently, even if much faculty motivation is intrinsic, reward systems have an effect on faculty members\u00E2\u0080\u0099 behaviour (Braxton, 2008; Link, Swan, & Bozeman, 2008; O\u00E2\u0080\u0099Meara, 2011; Schuster & Finkelstein, 2006). Since rewarded activities are deemed more desirable and legitimate, it is unsurprising that academics at research-intensive universities prioritize research over teaching (Link et al., 2008; Milem, Berger, & Dey, 2000), including professional growth activities related to teaching. 3 Faculty members\u00E2\u0080\u0099 engagement in professional growth related to their teaching is further constrained by the escalating instructional and research demands imposed on them (Hardy & Smith, 2006; Schuster & Finklestein, 2006). Over the past decade, the institutional pressures to generate revenue have amplified, as have external calls for increased accountability and proven outcomes. Faculty members have also had to keep up with\u00E2\u0080\u0094and adapt to\u00E2\u0080\u0094new technologies (Summerlee & Christensen Hughes, 2010). The trend of increased work effort, which some suggest is most pronounced at research universities (Schuster & Finklestein, 2006) has resulted in, among other things, faculty members feeling the need to \u00E2\u0080\u009Cdo more\u00E2\u0080\u009D (Gappa et al., 2007). Academics are putting more time into their profession, and stress is a significant health issue among those in academic careers (Jacobs & Winslow, 2004; Robinson, 2006). Additionally, many faculty members, especially female academics, have disclosed that balancing work pressures and family responsibilities is difficult (Acker & Armenti, 2004; Allan, 2011). The increased workload, which has led to a decline in satisfaction among faculty members, has a bearing on how faculty members allocate their time to fulfill their professional obligations. If one accepts that professional growth in teaching has important implications for the goals and effectiveness of higher education (Cariago-Lo et al., 2010; Neumann, 2009; Sorcinelli et al., 2006), and that numerous factors within the research-intensive university can support or constrain that growth, then the need for a study that examines the relationship between professional growth and these factors (which comprise the \u00E2\u0080\u009Cacademic culture,\u00E2\u0080\u009D a term I define in Section 1.4) becomes evident. 4 This dissertation aims to increase our understanding of ways to support faculty members\u00E2\u0080\u0099 engagement in their professional growth as teachers. It does so by examining summative peer review and other departmental practices that contribute to a culture that values teaching. The peer review of teaching has received considerable attention as a mechanism to foster professional growth in teaching, and extensive research has been conducted on the benefits of formative reviews. However, to date there has been little empirical research that examines the relationship between summative peer review and faculty members\u00E2\u0080\u0099 professional growth, in spite of the fact that peer reviews are commonly used to evaluate teaching for the purposes of making tenure and promotion decisions in North American universities (Gravestock & Greenleaf, 2008). Furthermore, since summative peer reviews occur only occasionally, there is a need to examine other departmental practices that may foster growth in teaching. The department is an appropriate site for research into meaningful and far-reaching initiatives because the majority of teaching developments occur as a result of departmental, rather than institution-wide, initiatives (Gibbs, Knapper, & Piccinin, 2008; Quinlan & \u00C3\u0085kerlind, 2000; Trowler & Knight, 2000). 1.2 Research Purpose The purpose of this research is to explore practices that encourage professional growth in teaching among tenure-track faculty members working in a research-intensive university. Through the use of qualitative interviews, this study aims to (1) investigate faculty members\u00E2\u0080\u0099 experiences of summative peer review of teaching and to examine how they understand the relationship between summative peer review and professional growth in teaching; and, (2) 5 explore departmental practices that support or hinder a culture that values teaching and learning, including professional growth in teaching. 1.3 Research Objectives The principal objective of this study is to investigate the issue of professional growth in teaching through the lens of academic culture. This work will contribute to a body of scholarly research on faculty careers, peer review of teaching, and departmental cultures of teaching and learning. This study draws on the literatures of departmental culture and the summative and formative peer review of teaching to explore the relationship between certain departmental practices, including summative peer review, and faculty members\u00E2\u0080\u0099 professional growth as teachers. It also attends to the role of collegial interactions in establishing and maintaining departmental practices that promote strong teaching. The empirical aim is to gather the accounts of faculty members about their experiences with summative peer review of teaching and other departmental practices that promote or hinder a culture that values teaching. The first practical objective of this study is to provide academics and the professionals who work with them\u00E2\u0080\u0094department heads, senior administrators, and educational developers\u00E2\u0080\u0094an expanded perspective on the relationship between departmental practices like the summative peer review of teaching and professional growth in teaching. Although this study cannot reveal the full extent of ways that faculty members grow and learn as teachers within the context of a department, it underscores the need to examine current structures and ways of being that contribute to such learning. I have chosen to examine existing teaching-related 6 practices because I maintain that, given intensifying academic careers, it makes good sense to see how to make use of, as well as enhance and promote, such practices. The second practical objective is to provide insight into the extent to which a university-level formal report on the summative peer review of teaching influences the practice of peer review. Within higher education, committees work to develop and implement policies and other formal guidelines pertinent to teaching. Typically faculty members do not respond well to such teaching initiatives (Harvey & Kamvounias, 2008; Kezar, 2011) and, when it comes to faculty evaluation and professional enrichment programs, faculty resistance and administrator apathy are two major obstacles to change (Arreola, 2007). Strategies may not be enthusiastically received and adopted because faculty members perceive that they infringe upon their autonomy. Academics may also be overwhelmed by policy overload or resent any bureaucracy that is imposed on teaching (Newton, 2003; Smith, 2011). In addition, the policy-to-practice gap is aggravated by the fact that there are usually scant mechanisms available to implement policies, to monitor their implementation, or to evaluate their impact (Newton, 2003). At the University of Western Canada (UWC) [pseudonym], where this study took place, a working group on the summative peer review of teaching created a document, at the request of the office of the Academic Provost and Vice President, that contained recommendations to guide the development of department-specific procedures and practices. The document was distributed in November 2009, approximately six months prior to the start of my first interview. Thus, the timing of this study allowed me to note whether faculty members made reference to this document or its contents and, if so, how they 7 perceived its relevance. This aspect of my work will be useful for university administrators and others who are involved with creating and implementing policies within their institution. 1.4 Conceptual Framework The conceptual framework for this study (Figure 1) was developed through the contributions of numerous higher education scholars, especially those who have studied academic culture(s) as these pertain to teaching and faculty careers. An academic culture perspective considers the role of values, beliefs, and assumptions in higher education; it examines how these are displayed in attitudes, behavioural norms, rituals, and other symbolic activities (Alvesson & Sveningsson, 2008; Bunch, 2007; Quinlan & \u00C3\u0085kerlind, 2000; Trowler, 2008) and how they are maintained and passed along (O\u00E2\u0080\u0099Meara, 2011). Higher education is frequently described as consisting of various cultures (or subcultures) that include institutional, disciplinary, and departmental cultures. These interconnecting cultures influence numerous academic practices (Clark, 1980, 1984, 1987; Trowler, 2008; Umbach, 2007); although their relative effect on a given practice (e.g., teaching, research, service or other practices) may vary, there is overall consensus that the higher education system is value laden (Clark, 1980, 1987; Gappa et al., 2007; Kezar, 2001). Those values most cherished by faculty members are academic freedom, autonomy, and collegiality (Kezar, 2001; Schuster & Finkelstein, 2006; Walvoord et al., 2000). Attention to values in a study about professional growth and teaching is relevant as these guide academics who, as Schuster and Finklestein (2006) explain, \u00E2\u0080\u009Care among the most likely\u00E2\u0080\u0094given the relative 8 autonomy they enjoy\u00E2\u0080\u0094to \u00E2\u0080\u0098act out\u00E2\u0080\u0099 their values in fulfilling their professional responsibilities\u00E2\u0080\u009D (p. 87). Figure 1: Conceptual Model for Departmental Practices and Professional Growth in Teaching The conceptual framework was further inspired by the work of O\u00E2\u0080\u0099Meara, Terosky, and Neumann (2008) who introduce a framework for faculty professional growth that positions growth as a professional and personal need, in every academic\u00E2\u0080\u0099s life, regardless of his or her career stage. Growth, they maintain, is ongoing, occurs in specific sociocultural and personal contexts, and is influenced by external environments as well as the unique needs of the individual academic. The authors frame their examination of professional growth through four interconnected lenses: learning, agency, professional relationships, and commitments. They make the following main points about each: 9 \u00E2\u0080\u00A2 Learning involves changes in cognition and is central to faculty members\u00E2\u0080\u0099 work. It happens in different work roles and may occur in interaction with people or be highly personal. Learning is influenced by the changing nature of work and by individuals\u00E2\u0080\u0099 past experiences and identities. \u00E2\u0080\u00A2 Agency originates from within the individual and refers to a faculty member\u00E2\u0080\u0099s capacity to \u00E2\u0080\u009Cconstruct the contexts of her or his own learning and development in professional and intellectual ways\u00E2\u0080\u009D (O\u00E2\u0080\u0099Meara et al., 2008, p. 28). Agency involves acting with intention and being reflexive about one\u00E2\u0080\u0099s experience. It can be nurtured by the professional community within which a faculty member works, and may be activated by available resources. It is also influenced by broader social structures. \u00E2\u0080\u00A2 Professional relationships are interactions that provide personal and/or professional support. They become sites of intellectual growth when they prompt, encourage, and affect learning. Professional relationships strengthen a faculty member\u00E2\u0080\u0099s ability to \u00E2\u0080\u009Cbring the best of their talents to their work roles\u00E2\u0080\u009D (O\u00E2\u0080\u0099Meara et al., 2008, p. 29) and they enhance faculty motivation, satisfaction, and self-esteem. \u00E2\u0080\u00A2 Commitments are conscious acts that affirm personal values. They require time, energy, and concrete action. Of these four elements, I was most attuned to professional relationships, since my review of the literature on peer review of teaching had sensitized me to the significant ways in which the candidate-reviewer relationship affects the peer review process. Prior to starting the study, I had learned that the levels of trust and the power differentials between the various individuals involved, and their intentions for peer review, shape participating faculty members\u00E2\u0080\u0099 experiences and resulting outcomes (Blackmore, 2005; Donnelly, 2007). Thus, I 10 surmised that the potential for summative peer review to contribute (or not) to faculty members\u00E2\u0080\u0099 growth in teaching would be influenced by the quality of the professional relationships among the departmental peers who were conducting, evaluating, and communicating about the reviews. Other literature about department cultures and teaching also points to the importance of collegial interactions for faculty members\u00E2\u0080\u0099 learning in their instructor roles. Figure 1 provides a visual illustration of the conceptual framework for this study. The framework suggests that institutional, disciplinary, and departmental cultures influence the formal and informal practices that can foster a departmental culture which values teaching. It also illustrates that values and norms are constantly operating and influencing faculty member choices with respect to teaching. The model proposes that institutional, department, and disciplinary cultures and the academic values of collegiality and autonomy will have an impact on faculty members\u00E2\u0080\u0099 participation in professional growth activities related to teaching, including which practices they choose to engage in, how they participate, with whom, and why. For instance, while the existence of a lunchtime seminar on teaching may, in theory, contribute to professional growth, it is possible that a faculty member would not attend because s/he works in a department where peers do not consider time spent on improving teaching as productive, and would thus disapprove of that colleague\u00E2\u0080\u0099s choice of activities. Consequently, the norm in that department is not to participate in lunchtime seminars about teaching. 11 This conceptual model also includes the four aspects of professional growth outlined in O\u00E2\u0080\u0099Meara and colleagues\u00E2\u0080\u0099 framework and, in doing so, assumes that learning, commitment, agency, and professional relationships interconnect to shape how a faculty member pursues his/her professional growth in teaching within a departmental context. 1.5 Research Significance Efforts to enrich and support faculty members\u00E2\u0080\u0099 work, including their growth and learning as teachers, are important to institutional leaders and higher education, as well as to individual faculty members (Neumann, 2009; O\u00E2\u0080\u0099Meara et al., 2008; Sorcinelli et al., 2006). Despite the fact that summative peer review of teaching is widely used in North American universities, there is a paucity of literature on how faculty members experience this process when it is used to inform high-stakes tenure and promotion decisions. As compared to the voluminous literature on formative peer review of teaching, empirical and in-depth information about summative peer review in North America remains limited. This study takes a unique approach to investigating the summative peer review of teaching, as it probes how academics connect the practice of peer review to professional growth in teaching. The research extends its examination of professional growth in academic careers by also considering departmental teaching-related practices. This research contributes to the scholarship on faculty growth in teaching by providing an account of the ways in which tenure track faculty members currently construct, and seek to develop, their roles as teachers in an environment that favours research. Building on previous 12 work, this research extends an inquiry into the complex relationships among departmental cultures, summative peer review, and professional growth in teaching. 1.6 Research Questions This study addresses the following questions: 1. What are faculty members\u00E2\u0080\u0099 experiences of summative peer review of teaching and how do they understand the relationship between summative peer review and professional growth in teaching? 2. What existing departmental practices support or hinder a culture that values teaching? 1.7 Dissertation Structure There are six chapters in this dissertation. Chapter 1 includes a presentation of the research problem and outlines the study\u00E2\u0080\u0099s purpose, objectives, and significance. It also features the conceptual framework and the research questions. Chapter 2 presents a review of the literature on peer review of teaching and departmental culture. Chapter 3 outlines the design of the study, including context, methods of data collection and analysis, and limitations of the design. Chapter 4 responds to the research question \u00E2\u0080\u009CHow do tenure-track faculty members in a research-intensive university perceive the relationship between summative peer review of teaching and professional growth in teaching?\u00E2\u0080\u009D Chapter 5 examines existing departmental practices that support or hinder the culture of teaching. Chapters 4 and 5 each include a discussion of the findings. In Chapter 6, I summarize the study and then discuss implications for theory, practice, and future research. 13 Chapter 2: Literature Review This study draws from and integrates several distinct bodies of research, including the literature on peer review of teaching, departmental culture, and academic culture. In the first section, peer review of teaching, I examine the contribution these reviews make to faculty members\u00E2\u0080\u0099 professional growth in teaching and then describe the reasons some academics resist the process. The second section, literature on department culture, begins with a brief introduction to the concept of academic culture before moving on to the topic of teaching and learning cultures. Throughout the chapter I seek to underscore the relevance of academic values and norms in fostering a peer review practice that contributes to professional growth in teaching and determining a departmental culture that values teaching. 2.1 The Peer Review of Teaching Although some scholars believe peer review of teaching purposes and processes exist along a continuum between formative and summative (Chism, 2007a; Gosling, 2002; Lomas & Nicholls, 2005), the majority draw a sharp distinction, portraying the process as belonging to only one category (Hammersley-Fletcher & Orsmond, 2004; Peel, 2005). In the literature, formative peer review is seen as beneficial for, and supportive of, professional development (Hammersley-Fletcher & Orsmond, 2004). It is hailed as a process that contributes to the enhancement of teaching because it fosters self-reflection, discussion of teaching, and dissemination of best practices (Byrne, Brown, & Challen, 2010; Kell & Annetts, 2009; Shortland, 2010). In contrast, summative peer review of teaching, which is used to aid in making such personnel decisions as promotion, reappointment, tenure, or merit pay (Cavanaugh, 1996; Chism, 2007a), has not been linked to the improvement of teaching 14 (Byrne et al., 2010; Peel, 2005). It has instead been linked to high-stakes evaluations as well as the accountability movement in higher education whereby faculty are increasingly required to measure and quantify their activities in the guise of improving quality and efficiency (Shanahan, 2009). Summative peer review is said to offer few benefits in terms of academic professional growth given that meaningful learning and reflective practice occur most often when academics engage in pedagogical practice for its own sake, not based on an external demand (Byrne et al., 2010; Peel, 2005). In a standard summative process, departmental colleagues assess a faculty member in order to rank or compare that individual within the department. Unlike a formative peer review, where the information is intended for the instructor\u00E2\u0080\u0099s personal/private use, information from summative reviews is open to public inspection (Chism, 2007a). I turn now to the connection between the practice of peer review and professional growth in teaching, as reported in the literature, and elaborate on some reasons why faculty members resist the practice of peer review of teaching. 2.1.1 Peer Review of Teaching and Professional Growth There are many compelling reasons to support peer review in higher education as a means of promoting professional growth in teaching. As mentioned, the developmental potential of formative peer review has been widely acknowledged in the literature. Reflection on teaching which occurs as reviewers and those reviewed dialogue about teaching, as well as individually, is a central component of the formative model. The act of reflecting on one\u00E2\u0080\u0099s teaching allows individuals to gain insight into the beliefs and intentions that drive, and give 15 justification to, their teaching actions (Pratt, 1997). Reflection helps faculty members uncover their conceptions of teaching; as a result, they may develop a more expansive understanding of their practice. When a colleague confirms these insights, it can aid legitimate new understandings (Peel, 2005). Even when peers are in disagreement over ideas and practices, if the criticism is relevant and instructive, peer review can present an opportunity to challenge assumptions about teaching and question existing actions (Handal, 1999). Thus, as Peel (2005) concludes, reflection when supported by a peer is significant for \u00E2\u0080\u009Ctransforming understanding and enhancing self-awareness\u00E2\u0080\u009D (p. 498). Research into the peer review of teaching indicates that the process of critically reflecting upon and discussing teaching can benefit not only the person observed but the observer as well (Bovill, 2008; Donnelly, 2007; Gosling & Ritchie, 2003; Kemp & Gosling, 2000; Lomas & Nicholls, 2005). Gosling and Ritchie (2003), in a follow-up study of regional peer review workshops in the United Kingdom, found that some faculty perceived the opportunity to observe other teachers as equally important to receiving feedback, if not more so. Consequently, Gosling and Ritchie (2003) have recommended replacing the dominant notion of \u00E2\u0080\u009Cgiver\u00E2\u0080\u009D and \u00E2\u0080\u009Creceiver\u00E2\u0080\u009D with a dialogue model in which both the instructor and observer gain mutual benefit from the process. They suggest that guidelines that support the peer review of teaching should explicitly state this benefit. Cosh (1998), on the other hand, found that reflection is more powerfully facilitated when the emphasis is on the observer rather than the observed. She promotes a model of peer review in which the reviewers direct their attention to reflecting on their own teaching so as to promote active self-development. 16 Regardless of whether the emphasis is placed on the reviewer or the person reviewed, peer review of teaching can help academics learn about each other\u00E2\u0080\u0099s teaching practices as well as their own. As faculty members build their understanding of teaching, they can better identify how they wish to develop their practice. Whether they pursue their learning through formal programs, such as the ones offered in university teaching and learning centres, or through self-study and opportunities in their departments, the assumption is that connecting the peer review process to continuous professional learning will contribute to the quality of teaching. 2.1.1.1 Collective Dialogue about Teaching Peer reviews help debunk the notion of teaching as a private activity, one shared with students but not colleagues (Hutchings, 1996). As peers connect with one another and others in their scholarly community, engaging in discussions about teaching will help shift the status of teaching from \u00E2\u0080\u009Cprivate to community property\u00E2\u0080\u009D (Shulman, 1993, p. 6). Shulman (1993), Chism (2007a), and others (i.e., Taylor Huber & Hutchings, 2005) have argued that teaching in higher education will be valued more when it is considered community property and not a solitary and isolated action. Peer reviews of teaching can create opportunities for dialogue and debate about teaching that extend beyond those that occur between reviewer and person being reviewed (Gosling & Ritchie, 2003; Hammersley-Fletcher & Orsmond, 2004). As faculty members engage in collective discussions about teaching evaluations, methods of teaching, and the challenges of practice, they share ideas and good practice and can report how they have tackled specific 17 teaching challenges (Chism, 2007a; Hutchings, 1996). The process can prompt departmental colleagues to address a wide range of issues pertinent to teaching, at the level of the individual, department, and institution. These \u00E2\u0080\u009Clearning conversations\u00E2\u0080\u009D (Byrne et al., 2010, p. 216) may lead to not only professional growth but also an increased sense of collegiality (Byrne et al., 2010; Donnelly, 2007; Lomas & Nicholls, 2005; Peel, 2005). 2.1.1.2 Refining Characteristics of Good Teaching Peer reviews have the potential to help faculty members refine their understandings about the characteristics of good teaching (Chism, 2007a). When colleagues are faced with evaluating peers, they must consider what effective teaching looks like within the context of their department, institution, and discipline. In summative peer review, evaluators must rate and/or comment on a colleague\u00E2\u0080\u0099s teaching. The evaluator, whose report is read by members of the tenure and promotion committee, will presumably be asked to reason his or her rankings and comments. Since characteristics of good teaching are not universally agreed upon (Chism, 2007a; Murphy, MacLaren, & Flynn, 2009; Toth & McKey, 2010b), discussions prompted by peer review may assist faculty members to locally develop those characteristics. Peer reviews can also contribute to professional growth in teaching, because they enable academics to build on their strengths as evaluators of teaching; existing research has found that peers, as compared to students, are more qualified to assess some characteristics of teaching over others (Chism, 2007a; Fink, 2008). Unlike students, faculty members are qualified to evaluate the following: course goals, content, and organization; instructional materials and methods; appropriateness of assessment and grading practices; commitment to 18 teaching and to student achievement; and professional and ethical behaviour (Chism, 2007a; Courneya, Pratt, & Collins, 2008). On the other hand, students are best at evaluating the quality of student-teacher interactions (Fink, 2008). Thus, information from peer reviews of teaching can add to student evaluations and provide faculty members with broader insights into their own teaching practice. 2.1.2 Resistance to Peer Review of Teaching One of the main barriers to effective peer review has been academics\u00E2\u0080\u0099 unwillingness to engage with the process (Carter, 2008; Lomas & Nicholls, 2005). When the peer review of teaching is seen as a tool for quality assurance and tied to personnel decisions, faculty members may become suspicious of the process (Peel, 2005; Swinglehurt, Russell, & Greenhalgh, 2008). Their resistance may be amplified when the objectives of peer reviews are unclear and/or when academics are unsure what will happen with the information gathered (Hammersley-Fletcher & Ormond, 2004). Faculty members may thus be more receptive to a model that creates dialogue about teaching and learning, and encourages constructive critical feedback and reflection (Swinglehurt et al., 2008). Still, some studies have shown that academics will not engage with the process even in formative peer reviews (Gosling & Ritchie, 2003; Hammersley-Fletcher & Orsmond, 2004). Therefore, it would appear that, irrespective of the model in place, faculty members may be unenthusiastic about participating in peer review for numerous, complex reasons, which will be elaborated in the next section. 19 2.1.2.1 Time One of the principal concerns faculty members have about peer review stems from their perception that it is a time consuming process (Chism, 2007a). The issue of time, however, must be placed in the context within which it operates: an environment where research productivity is rewarded above teaching and academics face escalating career demands, (particularly in the research-intensive university) (Chalmers, 2011; Shuster & Finklestein, 2006). In such an environment, few are eager to immerse themselves in teaching activities that use up more time. Especially when peer review is seen as merely something to \u00E2\u0080\u009Ctick-off\u00E2\u0080\u009D their to-do list or, even worse, a \u00E2\u0080\u009Ctime sink\u00E2\u0080\u009D (Chism, 2007b) academics may approach the process with a sense of complacency (Gosling, 2002). In these cases, they may comply but not fully engage (Hammersley-Fletcher & Ormond, 2004; Shortland, 2004), which diminishes the rigour of the process. 2.1.2.2 Identifying a Suitable Peer Faculty reluctance to the peer review of teaching may also involve questions about observer suitability (Kell & Annetts, 2009). The term \u00E2\u0080\u009Cpeer\u00E2\u0080\u009D can refer to various relationships within a department (Gosling, 2002). Peers may be defined on the basis of their disciplinary specialization, rank, pedagogical preferences, gender, or race, to name a few. For instance, some faculty members perceive that peer review is more meaningful when conducted by an experienced faculty member who can draw on his/her pedagogical knowledge to contribute to the reflective process (Hammersley-Fletcher & Ormond, 2004). Peel (2005), however, cautions that an experienced faculty member may bring \u00E2\u0080\u009Ca variety of experience, baggage, 20 and competence into the classroom\u00E2\u0080\u009D (p. 499). Experience, she warns, does not guarantee a high-quality review. Some faculty members may prefer a reviewer with a similar disciplinary specialization. When observers understand the subject matter and its delivery, they tend to concentrate on these aspects of teaching. On the other hand, when they are not familiar with course content, they may direct more attention to the student learning experience (Hammersley-Fletcher & Ormond, 2004). Depending on what candidates value in a review, they could prefer someone from the same disciplinary background (or not). A candidate may also be concerned about how a reviewer\u00E2\u0080\u0099s approach to teaching will affect his or her perceptions and appraisals. Indeed, in a study conducted by Courneya, Pratt, and Collins (2008), the authors concluded that reviewers who had an approach to teaching similar to the person being reviewed tended to rate the latter more highly than when their approaches differed. For instance, an evaluator who favoured participatory learning might assign a higher effectiveness rating to an instructor who practices this approach than to one whose dominant approach was lecturing. 2.1.2.3 Feedback Challenges Studies have found that some faculty members resist peer review because they feel ill at ease giving feedback to peers, especially of a critical nature (Braskamp, 2000; Cosh, 1998; Hammersley-Fletcher & Orsmond, 2004; Kemp & Gosling, 2000). Thus, even though collegiality maintains a forceful symbolic function, in practice it may not be readily visible (Lucas, 2006). 21 Massy, Wilger, and Colbeck (1994) are critical of \u00E2\u0080\u009Chollow collegiality\u00E2\u0080\u009D which they describe as a \u00E2\u0080\u009Cveneer of civility that pervades faculty interactions\u00E2\u0080\u009D (p. 12). Notwithstanding the fact that collegiality may refer to a democratic network of equal peers (Berquist & Pawlak, 2008), some have drawn attention to the stratified nature of academia (Massy et al., 1994; Rox\u00C3\u00A5, M\u00C3\u00A5rtensson, & Alveteg, 2011) and the abuse of \u00E2\u0080\u009Ccollegiality codes\u00E2\u0080\u009D by bullies who use them \u00E2\u0080\u009Cto threaten and eliminate people they don\u00E2\u0080\u0099t like\u00E2\u0080\u009D (O\u00E2\u0080\u0099Hara, 2009). Massy et al. (1994) along with Becher and Trowler (2001) and Rox\u00C3\u00A5 & M\u00C3\u00A5rtensson (2009) have noted that academics are often unwilling to pursue issues that may be divisive or spark debate. Instead, they are \u00E2\u0080\u009Cinclined to play safe\u00E2\u0080\u0094to minimize the risk of making professional enemies by not opposing or being critical of colleagues\u00E2\u0080\u0099 views\u00E2\u0080\u009D (Becher & Trowler, 2001, p. 127). The avoidance of such discussions maintains a surface calm but results in a lack of discussion about issues of curricular structure, pedagogical alternatives, and student assessment (Massy et al., 1994). Even so, ambiguities and contradictions can provide meaningful openings for individual and collective development (Trowler & Knight, 2000). Like the people who give feedback, academics who receive it may not welcome the peer review process because it can include judgments that feel threatening (Blackmore, 2005; Carter, 2008; Chism, 2007a; Cosh, 1998). Faculty members may become defensive to criticism and/or hostile when suggestions of change are made (Cosh, 1998). Chism (2007a) notes that peer reviewers may be more forthcoming when their comments are kept anonymous. Given that peer reviews are, in practice, generally synonymous with classroom observations, complete anonymity may be impossible. A faculty member\u00E2\u0080\u0099s apprehensiveness about giving feedback can intensify when a colleague\u00E2\u0080\u0099s career advancement is at stake. When 22 stakes are high\u00E2\u0080\u0094and even when they are not, the discomfort academics experience may lead them to record only positive comments (Cosh, 1998). In those cases, the peer review process becomes a type of \u00E2\u0080\u009Cmutual back-patting, meaningless for genuine staff development\u00E2\u0080\u009D (Cosh, 1998, p. 172). Resistance may also occur when faculty lack the skills and pedagogical knowledge to provide constructive feedback on teaching. Since routine discussions of teaching are rare among colleagues (Chism, 2007a; Cox, 2004; Gosling, 2002; Massy et al., 1994), faculty members frequently lack the vocabulary to talk about teaching philosophies and approaches. Instead, they may focus their comments on course content as an area they feel qualified to assess (Gosling & Ritchie, 2003). When academics are able to engage in high-quality feedback which includes reflection on teaching, the process is more likely to promote professional growth. The reviewer\u00E2\u0080\u0099s skill, as well as the willingness of the one being reviewed to engage in reflection on his/her own practice, helps ensure the success of this process (Hammersley-Fletcher & Orsmond, 2004). Still, some authors (i.e., Cosh, 1998; Peel, 2005) maintain that when reviewers offer suggestions on how to teach, the emphasis remains on teaching practices developed by others rather than fostering active self-development through reflective practice (Cosh, 1998). \u00E2\u0080\u009CPrescribing \u00E2\u0080\u0098how to teach\u00E2\u0080\u0099 does not send out a message, or invitation, for continued (self-directed) learning, and may damage morale,\u00E2\u0080\u009D cautions Peel (2005, p. 493). Trust between reviewer and person reviewed is another prime consideration in peer review, and lack of it presents a significant source of concern. Academics may feel vulnerable vis-\u00C3\u00A0- 23 vis existing personal or professional rivalries with the reviewer. When faculty members are marginalized on the basis of gender, race, disability, or other factors such as teaching in an area that is not valued, they may worry that bias inherent in the process, instruments, and/or reviewer assumptions will contaminate the results (Chism, 2007a). In summative peer review, where the focus is not necessarily on professional growth and learning, a lack of trust can prevent academics from fully engaging in the process (Gosling, 2002). However, when the person being reviewed \u00E2\u0080\u009Caccepts, or even welcomes, the comments of the observer, it can be a powerful learning experience\u00E2\u0080\u009D (Gosling, 2002). Those who have written about formative peer review have repeatedly emphasized the importance of trust and openness in the reviewer-reviewed relationship (Blackmore, 2005; Bovill, 2008; Cosh, 1998; Donnelly, 2007; Gosling, 2002; Shortland, 2010; Toth & McKey, 2010b). 2.1.2.4 Reliability and Validity Issues A common faculty reservation about peer reviews is their uncertain reliability and validity. Given greater confidence in the reliability and validity of peer reviews, faculty members might be more inclined to pay attention to peer evaluation results (Murphy et al., 2009; Yon, Burnap, & Kohut, 2002). In the literature, concerns about the reliability and validity of peer reviews are frequently linked to vague standards for effective teaching, the subjectivity of the process, and the limited amount of information being considered. 24 2.1.2.4.1 Lack of Accepted Standards The validity of peer reviews depends largely on academics\u00E2\u0080\u0099 ability to achieve consensus on what constitutes effective teaching in the local culture (Chism, 2007a). But, effective teaching is difficult to define (Chism, 2007a; Murphy et al., 2009; Peel, 2005; Yon et al., 2002) and there is no universally accepted definition of what comprises an excellent teacher (Arreola, 2007). It has even been claimed that teaching is \u00E2\u0080\u009Cimpossible to define in a list of criteria\u00E2\u0080\u009D (Cosh, 1998, p. 172). This is not surprising, given that the practice of teaching is a highly complex intellectual endeavour that demands not only disciplinary expertise, but also strong pedagogical knowledge and a thorough understanding of students (Arreola, 2007; Chism, 2007a; Murphy et al., 2009). Faculty members who question the existence of criteria will naturally be concerned about reliability and validity of peer reviews; they may believe that if there is no standard for good teaching, and judgments are biased and arbitrary, there can be no way fair way to measure whether faculty members are meeting expectations. Drawing on the discussion regarding resistance, if we accept that reliability in peer reviews pertains to the consistent evaluation of teaching, it becomes clear that many factors make it difficult to achieve reliable results in peer review. If two reviewers of the same candidate do not share the same content knowledge, approach to teaching, skills at providing feedback, and/or collegial relationship with the candidate, the results of their individual reviews will differ. The fact that most peer reviewers lack training poses another challenge to reliability (Yon et al., 2002). 25 It is noteworthy that vague standards for teaching persist despite a solid body of research (i.e., Arreola, 2007; Chism, 2004; Fink, 2008) that has identified characteristics of effective teaching. This research confirms that standards of performance exist; however, as Chism (2007a) points out, \u00E2\u0080\u009Cthe setting of standards is more fluid and more situational than it is for less complex activities\u00E2\u0080\u009D (p. 23). Consequently, the application of standards relies on academics\u00E2\u0080\u0099 professional judgment for interpretation. It also requires that faculty members take the time to discuss, debate, and determine these standards and criteria for their department or unit. Doing so is a demanding task that necessitates regular dialogue about teaching. Unfortunately, in many departments there is no culture to support this (Chism, 2007a; Donnelly, 2007). Only when departmental values for teaching have been articulated can peers strive to align them with their evaluations, and such alignment is a fundamental principle of good teaching evaluation (Chism, 2007a). 2.1.2.4.2 Subjectivity/Objectivity Some faculty members resist the peer review of teaching because they perceive it to be a highly subjective process. They argue that evaluation of teaching should be objective, and that peer review would benefit from a systematic means of collecting data, such as checklists and forms based on established criteria (Gosling, 2002). Arreola (2007) points out that such objectivity is impossible since, by definition, evaluation involves applying a judgment to the measurement of data. He proposes aiming for \u00E2\u0080\u009Ccontrolled subjectivity\u00E2\u0080\u009D which he claims can be achieved based on an a priori agreement about the values that will be applied and measured. As discussed, one of the challenges to the peer review of teaching is determining 26 the values associated with excellent, satisfactory, good, and poor teaching, and building the instruments and tools that capture these. 2.1.2.4.3 Limited Information Some academics are critical of the extent to which basing an assessment of teaching on a small number of class visits is reasonable and fair, because classroom teaching represents only a small portion of faculty members\u00E2\u0080\u0099 teaching activities. These include, but are not limited to, time in the classroom, student advising, curriculum development and revision, and administration (Toth & McKey, 2010a). However, peer review of teaching is often equated with classroom observation (Byrne et al., 2010) even though, in theory, it may encompass a review of the instructor\u00E2\u0080\u0099s teaching materials (e.g., syllabi, reading lists, exams), teaching portfolio, teaching-related committee work, student supervision, and student evaluations of teaching (Chism, 2007a; Murphy et al., 2009). When peer review is based solely on classroom teaching, the reviewed class may not be representative of a faculty member\u00E2\u0080\u0099s overall teaching (Carter, 2008; Chism, 2007a). Furthermore, the presence of the reviewer or a video camera may influence what is being observed, especially when the class is small (Blackmore, 2005; Gosling, 2002). 2.1.2.5 Infringement on Autonomy Resistance to peer review may intensify if faculty members perceive the process as an infringement on their autonomy, a core value in the academic profession (Berquist & Pawlak, 2008; Schuster & Finkelstein, 2006). Since teaching within higher education has a long 27 tradition as a private enterprise, academics can resist \u00E2\u0080\u009Cgoing public\u00E2\u0080\u009D with their teaching (Hutchings, 1996, p. 226). As Berquist & Pawlak (2008) explain: Many faculty members in the collegial culture would take great offense at being asked, let alone required, to accept an observing colleague in their classrooms. Ironically, even though classroom teaching is certainly a public event, it is considered an intimate exchange between faculty member and student. This exchange might be profoundly disrupted if observed and judged by another faculty member. (p. 31) Going public with one\u00E2\u0080\u0099s teaching might lead faculty members to feel more limited about their ability to make autonomous decisions with respect to structuring and managing their work within particular assignments (Gappa et al., 2007). Because academic freedom and autonomy are chief reasons why faculty members experience satisfaction in their profession (Berquist & Pawlak, 2008; Schuster & Finkelstein, 2006), faculty members may prefer to avoid activities that compromise these values. In addition, faculty members may feel their autonomy is being reduced when they see peer reviews as yet another accountability measure aimed at regulating and monitoring their activities (Ackerman, Gross, & Vigneron, 2009; Hatzipannagos & Lygo-Baker, 2006). 2.2 Summary: Peer Review of Teaching Literature Within the literature, formative peer review of teaching is typically contrasted against summative peer review, each type presented as having distinct purposes. In practice, the differences are frequently blurred, and both processes may contribute to professional growth by promoting dialogue about teaching among departmental peers. Ideally, these evaluations 28 prompt faculty members to establish and clarify characteristics of good teaching for their department through an ongoing process of communication and collaboration. However, just as formative and summative processes can contribute to professional growth in teaching and help establish a culture of teaching, they can also generate resistance among faculty members. Though some is due to practical issues (i.e., reviews based on limited information or which occur infrequently), much of the resistance concerns academic values and norms like collegiality, autonomy, and limited rewards for teaching. These values and norms manifest within and are part of the academic and department cultures to which faculty belong. Since the concept of culture is important to understanding faculty members\u00E2\u0080\u0099 choices and actions, the second part of the literature review will focus on the topics of academic, department, and teaching cultures. 2.3 Department Culture In order to understand how departmental practices can shape the value of teaching and influence faculty members\u00E2\u0080\u0099 engagement in professional growth in teaching, it is essential to recognize the overarching role of academic culture in faculty careers. I therefore begin this section with a short introduction to academic culture and then move to a review of the relevant literature on department culture. 29 2.3.1 Academic Culture, Values and Norms As defined in Chapter 1, academic culture acknowledges that academics are a part of and shaped by interconnected groups tied to their institution, discipline, and department (Clark, 1984; Gappa et al., 2007). As Clark (1984) so aptly describes: Academics are caught up in various matrices, with multiple memberships that shape their work, call upon their loyalties and apportion their authority\u00E2\u0080\u00A6.the academic belongs simultaneously to a discipline, a field of study, and an enterprise, a specific university or college. (p. 112) Thus, institutional, disciplinary, and department cultures intersect in complex and varied ways to shape faculty careers. Scholars who study academic culture agree that the system of higher education is value laden, and have identified collegiality and autonomy as dominant values. For the purpose of this study, I presume that: \u00E2\u0080\u00A2 Collegiality is a value which, when put in practice, provides \u00E2\u0080\u009Copportunities for faculty members to feel that they belong to a mutually respectful community of colleagues who value their unique contributions to their institutions and who are concerned about their overall well-being\u00E2\u0080\u009D (Gappa et al., 2007, p. 142). \u00E2\u0080\u00A2 Autonomy is a value that pertains to freedom of expression and is a component of academic freedom. Autonomy relates to a faculty member\u00E2\u0080\u0099s right to decide and/or act according to his or her judgment when managing and structuring work (Berquist & Pawlak, 2008; Schuster & Finklestein, 2006). 30 These values and others affect faculty members\u00E2\u0080\u0099 behaviours and attitudes in significant ways (Clark, 1980, 1987; Gappa et al., 2007; Kezar, 2001; O'Meara et al., 2008); consequently, I will refer back to and elaborate on them throughout the dissertation, as I examine the issue of professional growth in teaching. Like values, norms occupy a central place in academic culture. Braxton (2010) defines norms as beliefs held by a particular group of people about \u00E2\u0080\u009Cexpected or desired behaviors in a given situation or circumstance\u00E2\u0080\u009D (p. 243). Formal and informal norms develop for behaviours which the majority of the group considers important. Academics internalize these norms which over time shape academic codes of conduct (O\u00E2\u0080\u0099Meara, 2011). In universities and colleges, where faculty members have a lot of autonomy, norms provide guidelines for appropriate behaviours (Braxton, 2010). Norms and values are relevant to a study of academic careers by helping us grasp the relationship between department cultures and faculty members\u00E2\u0080\u0099 choices concerning professional growth in teaching. 2.3.2 The Academic Department and Culture(s) of Teaching Shaped and defined by institutional and disciplinary cultures (Clark, 1987; Lee, 2007; Walvoord et al., 2000), academic departments \u00E2\u0080\u009Care the most salient organizational aspect of higher education for thousands of faculty and millions of students\u00E2\u0080\u009D (Hearn, 2007, p. 224). As the focal point of academic work, they have been described as the \u00E2\u0080\u009Chome\u00E2\u0080\u009D within which the academic lives (Lee, 2007). Complex units that tend to be distinct in their organization, policies, standards, and resources, departments are considered the fundamental component of 31 the North American higher education institution (Hearn, 2007; Knight & Trowler, 2000; Lee, 2007). Within the institution, departments operate fairly independently as they coordinate and manage academic processes. Decisions on course offerings and content, appointing and promoting faculty members and administrative staff, and managing services are normally done at the department level on behalf of the larger institution (Lee, 2007). Because they are accorded significant power in areas that include faculty member promotion and the management of programs and services for professors, what happens in these units can influence faculty members\u00E2\u0080\u0099 behaviours (Hearn, 2007; Lee, 2007). A number of scholars have suggested that the department culture has a significant impact on how faculty members pursue their teaching responsibilities (Lee, 2007; Massy et al., 1994; Trowler & Knight, 2000). In discussions of teaching and faculty careers, the terms \u00E2\u0080\u009Cculture of teaching\u00E2\u0080\u009D or \u00E2\u0080\u009Cculture of teaching and learning\u00E2\u0080\u009D are commonly used to refer to the various influences that affect faculty members in their role as instructors. A strong culture of teaching is typically taken to mean a culture that values teaching; in a department culture that values teaching, faculty members will presumably be supported in their professional growth as teachers. A strong departmental teaching culture (one that values teaching) has distinct characteristics that can include frequent interactions and collaboration among faculty members about teaching, and the presence of an effective department head who is supportive of teaching and 32 communicates a high value placed on that aspect of faculty work. Though other factors also denote a strong teaching culture, i.e., support and commitment from senior administrators, a rigourous evaluation system aligned to promotion and tenure decisions, and the existence of a vibrant educational development program/centre, I have chosen to elaborate on the role of collegial conversations and departmental leadership as these are tied to the main identifiers of a culture of teaching, namely, ongoing involvement of faculty members in all aspects of planning and implementing teaching-related initiatives; the existence of a reward structure that appropriately recognizes faculty teaching and involvement in the scholarship of teaching and learning; and attention to teaching in hiring processes (Bensimon et al., 2000; Braxton, 2008). 2.3.2.1 Learning Conversations and Professional Growth in Teaching Although faculty members can learn and grow as teachers through such structured programs as workshops, learning communities focused on teaching, or formative peer review of teaching programs, the contribution these activities make to professional growth is relatively small (Walvoord et al., 2000). This is due in part to the fact that few faculty members participate in formal, voluntary programs (Altman, 2004). Studies have found that as with many professionals, those in higher education learn by taking part in everyday experiences (Clegg, 2003; Eraut, 2000; Jawitz, 2009; Knight, Tait, & Yorke, 2006). That is, learning about teaching occurs as faculty members plan their work, perform course reviews, serve on committees, grade assignments, advise students, revise their courses, assess and evaluate one another\u00E2\u0080\u0099s work, and so on. Learning that occurs unintentionally, through activities with no set learning objectives, is known as informal learning (Werquin, 2007). It is often contrasted 33 against formal learning, which includes engaging in activities like courses and training workshops attended for the explicit purpose of gaining specific skills, competencies, and knowledge (Werquin, 2007). The literature indicates that conversations among colleagues about teaching present important opportunities for learning and intellectual growth. These interactions help faculty members expand their understanding of teaching and student learning (Knight et al., 2006; Neumann, 2009; O\u00E2\u0080\u0099Meara et al., 2008; Rox\u00C3\u00A5 & M\u00C3\u00A5rtensson, 2009; Trowler & Knight, 2000) and also, more broadly, of the academic profession and their institution (Gappa et al., 2007). Conversations that foster professional growth in teaching have been described variously in the literature. Rox\u00C3\u00A5 and M\u00C3\u00A5rtensson (2009) employ the term \u00E2\u0080\u009Csignificant conversations\u00E2\u0080\u009D to denote discussions that help university teachers create and maintain their understanding of teaching. In these conversations, academics attempt to make sense of their teaching experiences, discuss their challenges, evaluate their situations, and plan future actions. The authors describe that such discussions characteristically occur privately, among trusted colleagues. Similarly, Byrne, Browne, and Challen (2010) emphasize the value of \u00E2\u0080\u009Clearning conversations\u00E2\u0080\u009D\u00E2\u0080\u0094the dialogue between colleagues that enables reflection on teaching. Like Rox\u00C3\u00A5 and M\u00C3\u00A5rtensson, Byrne et al., note that when peers share their understanding of teaching-related issues, question assumptions, articulate personal theories (which may include literature on teaching), it can lead to professional growth. Similarly, Haig (2005) affirms that conversations are a key feature in professional learning about teaching. 34 Though faculty members value collegial conversations, several factors can impede these. For example, collegial communication and interactions may be hindered when faculty members feel marginalized because of gender (Haag, 2005; O\u00E2\u0080\u0099Meara et al., 2008). Numerous studies have found that female academics face persistent barriers within their chosen career compared with their male colleagues; they have slower rates of career progression (Barrett & Barrett, 2010; O\u00E2\u0080\u0099Meara et al., 2008; Premeaux & Mondy, 2002) and are less likely to earn tenure and become full professors (Premeaux & Mondy, 2002; Winkler, 2000). In addition, women hold a greater proportion of non-tenure track positions (Gappa et al., 2007; O\u00E2\u0080\u0099Meara et al., 2008), are more involved in service work to the detriment of research productivity (Misra, Hickes Lundquist, Holmes, & Agiomavritis, 2011), and are more frequently challenged by students who dispute their roles as professionals (Kardia & Wright, 2004). Not surprisingly, women experience more overall stress than their male counterparts (Acker & Armenti, 2004; Allan, 2011). These aspects of academic life can have an impact on everyday interactions. Communication among colleagues may also be inhibited because of marginalization due to race, age, and/or appointment type (Gappa et al., 2007) and faculty conflict may be associated with resource allocation and the assignment of instructional loads (Hearn, 2007; Hearn & Anderson, 2002). Finally, some research has suggested that differences in disciplinary culture can impede communication. Indeed, faculty report they do not have regular opportunities to meet or talk with colleagues in other specializations (Gappa et al., 2007; Hamilton, 2007). Such distinctions among disciplinary culture mean that as disciplinary fragmentation occurs, there may be a shrinking number of peers with whom faculty can engage (Massy et al., 1994). 35 Nevertheless, when collegial conversations about teaching occurred within departments or disciplines, academics were more likely to perceive them as relevant to their practice and personal situation (Quinlan & \u00C3\u0085kerlind, 2000). Clearly, disciplinary values, norms, and behaviours have an impact on collegial interactions and faculty teaching (Becher & Trowler, 2001; Healey & Jenkins, 2003; Umbach, 2007). 2.3.2.1.1 Disciplinary Culture, Teaching, and Collaboration Faculty members identify strongly with their discipline (Becher & Trowler, 2001; Healey & Jenkins, 2003; Umbach, 2007). Members of the same disciplinary cultures share common vocabularies, similar norms of scholarly collaboration and codes of ethics, are drawn to certain research methodologies, and belong to the same learned societies (Gizir & Simsek, 2005). Different disciplinary cultures have their own distinct concepts, methods, and aims that influence the ways in which academics organize and enact their work lives (Becher & Trowler, 2001). The notion of disciplinary groupings is commonly described using the Biglan/Becher typology. Biglan developed a model for categorizing academic disciplines that was elaborated on by Becher in 1989 (Becher & Trowler, 2001). Becher\u00E2\u0080\u0099s model, based on the assumption that different disciplines have distinct cultural characteristics, consists of a four-group classification system: hard-pure, soft-pure, hard-applied, and soft-applied (Becher, 1994). According to research on disciplinary cultures, hard-pure fields such as chemistry and physics tend to be concerned with universals and have a quantitative emphasis (Becher, 1994; Neumann, Parry, & Becher, 2002). In contrast, soft-pure disciplines like anthropology and history are concerned with discovery through interpretation and more often 36 use qualitative approaches (Becher, 1994; Umbach, 2007). Academics in hard-pure disciplines engage more frequently in collaborative practices than do academics in soft-pure disciplines, who tend to pursue individual scholarly interests (Becher & Trowler, 2001). Compared with their \u00E2\u0080\u009Cpure\u00E2\u0080\u009D counterparts, hard-applied fields are oriented toward purposive, pragmatic know-how (Becher, 1994). Since disciplinary\t\r \u00C2\u00A0cultures have distinct patterns of performing scholarly tasks (Gizir & Simsek, 2005; Hearn & Anderson, 2002; Quinlan & \u00C3\u0085kerlind, 2000), disciplinary differences can shape how academics approach teaching (Kreber & Castleden, 2008; Neumann, 2001; Neumann et al., 2002; Umbach, 2007). In hard-pure disciplines, course content is normally \u00E2\u0080\u009Clinear, straightforward and uncontentious\u00E2\u0080\u009D (Lindblom-Ylanne, Trigwell, Nevgi, & Ashwin, 2006, p. 287) and courses are tightly structured. Courses in soft-pure disciplines tend to be more loosely organized than those in the hard sciences (Neumann, 2001). When they teach, faculty members in the hard sciences favour learning facts and concepts whereas those in the soft disciplines emphasize the development of cognitive skills (Umbach, 2007). Academics in hard-pure disciplines often teach in teams and there is more agreement among faculty members as to the content and sequencing of the curriculum (Gibbs et al., 2008). In comparison, those in the soft-pure disciplines reportedly have a broad range of views on how the curriculum should be structured, and their teaching is often kept private. Existing research (e.g., Lattuca & Stark, 1994, Neumann, 2001, Umbach, 2007) that has examined the relationship between disciplines and teaching has found that disciplinary differences are tied to teaching preparation time, beliefs about student learning, supervision, assessment and evaluation. 37 Though Becher and Trowler (2001) have noted a substantial variation in communication and collaborative practices even within a particular disciplinary grouping, these categorizations nevertheless offer a useful framework for exploring disciplinary differences. 2.3.2.2 Departmental Leadership in Teaching Appropriate leadership from the department head is a key factor in determining the extent to which a department actively supports teaching (Anderson, Scott, & Coates, 2008; Braxton, 2008; Knight & Trowler, 2000; Massy et al., 1994). Department heads exercise considerable power in their positions and, through their various responsibilities and choices, can encourage a culture of teaching (Braxton, 2008; Gibbs et al., 2008; Knight & Trowler, 2000; Massy et al., 1994). The skills and knowledge required by a department head are complex and extensive, and include knowledge about institutional structure and systems, human resource policies and practices, faculty members\u00E2\u0080\u0099 expertise within the department, curriculum, and information technology. In order to affect the culture of teaching and learning, department heads are also expected to have leadership skills (Berdrow, 2010). Gibbs et al. (2008), in an international study on approaches to departmental leadership in research-intensive universities, found that this role is shaped not only by the way in which heads think about their leadership, but also by the organizational culture of the institution and the discipline within which the heads work. In a practical sense, the department head can enhance the culture of teaching and learning in a number of ways, thereby contributing to an environment whereby faculty members are 38 inclined to participate in professional growth activities related to their teaching practice. First, they can allocate meaningful rewards to faculty members for their teaching achievements (Braxton, 2008; Gibbs et al., 2008; Massy et al., 1994; O\u00E2\u0080\u0099Meara, 2005). Chairs can explicitly value teaching in appointment, tenure, and promotion decisions and also make visible the value they place on teaching through organizing awards (Gibbs et al., 2008). Second, department heads can promptly communicate teaching evaluations with faculty members (Del Favero, 2002). These discussions should include consideration of how the results align with departmental criteria for teaching quality as this will allow faculty members to determine professional development needs and give them a clearer understanding of the expectations for tenure and promotion (Bensimon, Ward, & Sanders, 2000). Third, the department head can ensure that faculty members have the necessary information to advise students (Del Favero, 2002). Advising is a frequently overlooked component in teaching, especially in the case of new professors. Yet the advising relationship can have profound consequences on students\u00E2\u0080\u0099 choice of courses, career, and further education (Bensimmon et al., 2000). Department heads can assist the process by ensuring that relevant course information, degree requirements, suggested coursework progression, and other instructions be shared and readily accessible. Fourth, because teaching overlaps administrative functions from ordering textbooks to student evaluations, the department head can enhance the climate by communicating 39 available administrative expertise and support for teaching-related activities (Del Favero, 2002). Fifth, the department head can ensure equal teaching responsibilities are distributed among faculty members. An egalitarian system means that faculty at all ranks teach courses ranging from introductory to advanced and have approximately the same teaching load and opportunities for course rotation (Massy et al., 1994). Finally, department heads can foster a healthy teaching climate and contribute to an outstanding teaching culture when they diagnose teaching problems and mobilize faculty, students, and others to address these in a productive way (Gibbs et al., 2008). Department heads can take an even more direct approach to encouraging professional growth in teaching by ensuring a teaching orientation for new professors, organizing teaching- focused workshops for all faculty members, and instituting a mentorship program for teaching. Formative peer reviews of teaching may be one means by which faculty members mentor one another; the department head can establish the success of this practice by working to embed it into the culture and directly addressing faculty reservations (Chism, 2007b). Department heads further promote professional growth in teaching when they inform faculty members about the services offered by teaching and learning centres on campus (Bensimon, et a., 2000; Braxton, 2008). In addition to these possibilities, the departmental chair can create opportunities for faculty to collaborate on formal teaching projects (Quinlan & \u00C3\u0085kerlind, 2000) or meet regularly to converse about teaching (Braxton, 2008). Since collaboration and conversation among peers occur more often in a department with a 40 collegial atmosphere (Bensimon et al., 2000), department heads need to pay close attention to this aspect of the culture. 2.3.3 Summary: Department Culture Literature Much professional learning about teaching is social and cultural in nature. It occurs largely within the department as academics carry out their daily activities, including interacting with their peers. \u00E2\u0080\u009CLearning opportunities come from the chance to engage with others\u00E2\u0080\u0099 practices and from engaging others with one\u00E2\u0080\u0099s own practices\u00E2\u0080\u009D (Knight et al., 2006, p. 334). As such, it is useful to create many opportunities for collaboration and informal discussions because the sharing and exchange of ideas encourage faculty members to engage with the collective understandings and assumptions within their department. As it pertains to teaching, the concept of a culture of teaching and learning is useful for examining departmental practices. Learning conversations among peers and organizational leadership\u00E2\u0080\u0094in particular, that coming from the head of the department\u00E2\u0080\u0094are major influences on the departmental culture of teaching and faculty members\u00E2\u0080\u0099 professional growth in teaching. In addition, institutional and disciplinary cultures, with their particular norms and values, occupy a role in shaping the departmental culture of teaching. 2.4 Summary In summary, this review of the literature points to five important ideas that will guide the focus of this dissertation: 41 (1) Departmental culture provides a useful lens for examining practices that promote or hinder faculty members\u00E2\u0080\u0099 professional growth in teaching. (2) Interactions among colleagues can stimulate, shape, and advance learning; within academia, collegiality is a powerful value and this sense of belonging to a community of scholars can enhance job satisfaction and motivation (Gappa et al., 2007; Neumann, 2005; O\u00E2\u0080\u0099Meara et al., 2008). (3) While much research depicts formative peer review of teaching as distinctly different from summative review, there may be significant overlap between the practices. Summative, not only formative, peer review can foster academic growth and development. (4) Faculty members\u00E2\u0080\u0099 experiences of summative peer review as a mechanism for professional growth in teaching has been an understudied phenomenon. (5) An exploration of faculty members\u00E2\u0080\u0099 everyday practices as they relate to teaching and occur within a department is important for research on teaching and learning. 42 Chapter 3: Methodology 3.1 Introduction The aim of this study is to explore opportunities for professional growth in teaching, both in the context of faculty members\u00E2\u0080\u0099 experiences with summative peer review of teaching and in other departmental practices. In order to explore this issue, I conducted a qualitative study that relied on interviews with tenure-track faculty members working at a research-intensive university. In this chapter I outline the perspectives and decisions that guided this study. In addition, I will 1) clarify my perspectives as a researcher; 2) discuss changes to the original research questions; 3) present the details of the research design, data collection, and analysis; 4) describe the study\u00E2\u0080\u0099s limitations; and 5) outline ways in which I strove for rigour and applied methods to help safeguard the rights, trust, and dignity of the study participants. 3.2 Positioning Myself As a researcher, I conceptualize, design, and conduct the study in ways that reflect my beliefs about the world, and how it should be studied and understood (Mayan, 2009; Schram, 2003). Qualitative research texts emphasize the importance of reflexivity1; good research requires 1 Reflexivity is \u00E2\u0080\u009Cthe process of being highly attentive to how and why you make decisions and interpretations along the research way, critically examining your personal-researcher role and how this interfaces with all\u00E2\u0080\u0094even the most minute\u00E2\u0080\u0094aspects of the research\u00E2\u0080\u009D (Mayan, 2009, p. 137). 43 one\u00E2\u0080\u0099s beliefs, assumptions, and roles be made explicit in writing (Creswell, 2007) and I will begin by outlining some of these in order to provide a rationale for, and further insight into, my methodology.2 3.2.1 Intellectual Position Based on my understanding of the interpretive paradigm, it is one I associate with most closely. The interpretivist researcher seeks to explore and comprehend the social world using both participants\u00E2\u0080\u0099 understandings and her own. \u00E2\u0080\u009CThe researcher\u00E2\u0080\u0099s intent,\u00E2\u0080\u009D writes Creswell, \u00E2\u0080\u009Cis to make sense (or interpret) the meanings others have about the world\u00E2\u0080\u009D (2007, p. 21). Looking for meanings and not necessarily causes, the interpretivist acknowledges the inter- relatedness of different aspects of the participants\u00E2\u0080\u0099 lives (Snape & Spencer, 2003). She recognizes that social, cultural, historical, ethnic, political, and other factors influence how people make meaning of their specific situation (Schram, 2003) and therefore seeks a complexity of views among her participants (Creswell, 2007). In the interpretive paradigm, the researcher is also aware that her own background, perspectives, and values shape her research design, interpretations, and findings (Creswell, 2007; MacKenzie & Knipe, 2006; Snape & Spencer, 2003). 2 I understand methodology as the overall approach to research (MacKenzie & Knipe, 2006) connected to the researcher\u00E2\u0080\u0099s paradigm, and comprising the beliefs, values, and theories that shape the researcher\u00E2\u0080\u0099s approach; as such, it influences design, data collection strategies, analysis, and the presentation of one\u00E2\u0080\u0099s findings (Mayan, 2009). 44 In addition to the interpretive paradigm, I also connect with feminist traditions. My original research questions (Section 3.3) clearly point to the fact that I am concerned about, and interested in, gender issues within higher education. In this study, the issue of gender stands out more through its silence than via the participants\u00E2\u0080\u0099 utterances. When I originally designed this study, I anticipated gender would figure more prominently in the findings. One of the chief aims of feminist research is to better understand power relations and the nature of enduring inequalities with respect to gender, so that they may be effectively transformed (Ramazanoglu & Holland, 2002). Feminist researchers have a political and ethical commitment to women that makes them accountable to a community of women who share political and/or moral concerns. Thus, a feminist research methodology investigates questions that arise from women\u00E2\u0080\u0099s lives; such questions, however, are not only about women's lives but also the rest of nature and social relations (Harding, 2004). While a feminist methodology requires the concept of a community of women, it rejects the notion of an essential woman, one with whom all women will identify (Olesen, 2008). The notion of a universal woman is rather replaced by the idea of a woman who is situated within experiences, interests, and power relations that shape her existence (Olesen, 2008; Ramazanoglu & Holland, 2002). Because a feminist researcher distinguishes numerous differences in women\u00E2\u0080\u0099s experiences, she recognizes that being female doesn\u00E2\u0080\u0099t guarantee access to all women\u00E2\u0080\u0099s lives and knowledge (Olesen, 2008). Furthermore, she comprehends that knowledge is always partial, \u00E2\u0080\u009Cin the sense of being \u00E2\u0080\u0098not-total\u00E2\u0080\u0099 and in the sense of being \u00E2\u0080\u0098not-impartial\u00E2\u0080\u0099\u00E2\u0080\u009D (Ramazanoglu & Holland, 2002, p. 66). 45 Finally, and similar to all qualitative investigations, researchers who adopt a feminist methodology identify the need to critically examine and continuously reflect on the nature of the research process. Consequently, the researcher is attentive to the methods she uses, her participants\u00E2\u0080\u0099 experiences of being subjects of the investigation, whose voices she represents, the way she writes herself into the research, and readers\u00E2\u0080\u0099 reactions to meanings and interpretations of the research. Aware that she has the power to judge between different representations of reality when she decides how to portray the lives and ideas of the individuals whom she researches, the feminist researcher recognizes her role in knowledge production. Reflexivity is a key aspect of a feminist methodology (Fonow & Cook, 2005). 3.2.2 Professional/Personal Position I come to this study as an educational developer, someone whose chosen profession is to support interested members of the higher education community in their growth as teachers. My work with faculty members is inspired by a belief that teaching is important and improving student learning is a worthwhile goal; these beliefs are central to educational development, as is the value placed on addressing individual well-being (Gosling, 2010). Though I maintain a focus on teaching and student learning, I am guided by the conviction that supporting academics\u00E2\u0080\u0099 growth as teachers is facilitated when one recognizes and honours the multiple roles played by faculty as citizens and scholars within their departments, campuses, professions, and wider communities (Sorcinelli et al., 2006). Thus my approach to promoting growth in teaching considers not only the individual, but also the institution and broader context within which academics pursue their careers. 46 I have worked in the field of educational development for over eight years and it was through my involvement with teaching portfolios that I became acquainted with the concept of peer review of teaching. My intention in entering a Ph.D. program was not to work as a faculty member, based on a belief that a demanding career would interfere with family life and the \u00E2\u0080\u009Cbalance\u00E2\u0080\u009D I seek long-term. This perception could be shaped in part by the fact that I was raised by two former professors. (Given the choice, my father would gladly repeat his career; my mother would not, witness the fact that, in her mid-40s, she resigned from her position as Associate Professor.) 3.3 Changes to the Original Research Questions In the dissertation proposal submitted to the University\u00E2\u0080\u0099s Behavioural Ethics Board in March 2010, I stated my primary research question as \u00E2\u0080\u009CIn what ways, and to what extent, do gender, department culture, and disciplinary culture influence pre-tenured and tenured faculty members\u00E2\u0080\u0099 understandings and experiences of the summative peer review of teaching?\u00E2\u0080\u009D I intended to explore faculty experiences of the summative peer review through four inter- related lenses: gender; department culture, in particular as it relates to departmental values, beliefs, and practices about teaching; disciplinary culture, as it relates to disciplinary values, beliefs and practices about teaching; and academic rank. In the preliminary data analysis stage which coincided with the start of the interviews, I began to anticipate it would be a challenge to distinguish faculty members\u00E2\u0080\u0099 experiences through the four lenses, given the small number of participants. While I recognized that my 47 questions were shifting, I was unable to specify any new questions. During the writing phase of my dissertation, after I had worked with codes, categories, and themes for many months and had gained additional insights into the study, I revised the original questions included in my proposal, and developed subsidiary questions for the new research components: 1. What are faculty members\u00E2\u0080\u0099 experiences of summative peer review of teaching, and how do they understand the relationship between summative peer review and professional growth in teaching? a. In what ways do the multiple purposes of summative peer review play out and interconnect? b. How do academic values and norms, as well as other established practices in higher education, influence the summative peer review process? 2. What existing departmental practices support or hinder a culture that values teaching? a. How do aspects of institutional and disciplinary culture influence departmental practices that support/hinder a culture that values teaching? b. What is the relationship between departmental practices that foster a culture that values teaching and professional growth in teaching? Compared to the original questions, these modified questions align more closely with what the participants and I discussed during the interviews. That my questions transformed throughout the process is common in qualitative research; as Maxwell points out, \u00E2\u0080\u009Cqualitative 48 researchers often don\u00E2\u0080\u0099t develop their eventual research questions until they have done a significant amount of data collection and analysis\u00E2\u0080\u009D (2005, p. 65). The original questions provided a frame for the data collection process; but as the various design components interacted (Maxwell, 2005), and I began to undertake analysis, my questions changed. 3.4 Selecting Faculties, Departments and Faculty Members I took a case study approach to this research as this is well-suited to gaining a thorough understanding of a particular situation, and the meaning it may have for all those involved (Merriam, 1998). Case studies are comprehensive descriptions and analyses of a bounded system such as a group, individual, program, activity, process, or event (Creswell, 2008; Merriam, 1998; Yin, 2009), and researchers who use this approach are committed to studying the complexity of the bounded systems they predefine (Thomas, 2011). Merriam (1998) states that the most distinguishing feature of case studies research is the placing of limits on the object of study; consequently, investigators must clearly define boundaries in terms of time, place, components to be studied, or other concrete physical features (Merriam, 2002; Stake, 2005; VanWynsberghe & Khan, 2007; Yin, 2009). The more a case is specific and bounded, the more useful it will be (Stake, 2005). Since not everything about a case can be understood, the researcher must strategically decide which aspects to study and then determine what the reader needs to know, based on the information derived (Stake, 2005). However, defining the limits of a case is not a simple matter. Cases may be embedded in various contexts and have multiple subsections, dimensions, groups, and events, each with its own context (Stake, 2005; Yin, 2009). 49 The case study approach is consistent with an embedded case study, in which I focused on multiple cases within a bounded system (a single institution) (Yin, 2009). I selected two faculties (Arts and Science) and recruited participants from three departments within each faculty. Below I will elaborate on my choice of faculties, departments, and participants. 3.4.1 Institution This study took place at UWC, a research-intensive university in North America. UWC is home to over 2,000 professors (see Section 3.4.4 for definition of professor). According to UWC's 2010 statistics, 30 percent of professors are women and 22 percent are assistant professors. UWC was chosen for several reasons. As a large university, it provided the chance to select (invite) departmental participation across a wide range of disciplines. Given my original intention of linking faculty members\u00E2\u0080\u0099 experiences to disciplinary culture, having a wide choice of departments would be helpful. Second, UWC\u00E2\u0080\u0099s status as a research-intensive institution made it an interesting site to study teaching because of the multiple and sometimes conflicting demands placed upon professors. Because research is more highly rewarded in decisions about tenure, promotion, and merit, faculty members at such institutions often limit the time and attention devoted to teaching (Chalmers, 2010; Hardy & Smith, 2006; Kilgore & Cook, 2007). Still, universities have begun paying increased attention to teaching (Groccia, 2010; Macfarlane, 2011; O\u00E2\u0080\u0099Meara, 2006) and a study that examined departmental cultures and their connection to teaching in a research-intensive context would be relevant and timely. 50 Finally, site selection was based on the fact that, since UWC is close to home, I would not have to travel\u00E2\u0080\u0094an important factor as it caused less disruption to my family, and therefore me. 3.4.2 Faculties My original research questions were aimed at understanding differences in faculty members\u00E2\u0080\u0099 experiences and understandings of summative peer review through the lines of gender, rank, departmental culture, and disciplinary culture. My recruitment strategy described below reflects this original intent. To assist the process of faculty and departmental selection, I located institutional statistics posted on UWC's institutional research website. Because detailed department information was only available for Education, Applied Science, Arts, and Science, I narrowed my initial choice to these faculties. I ultimately selected the faculties of Science and Arts as these represented contrasting categories in Becher\u00E2\u0080\u0099s (1994) typology (see also Becher & Trowler, 2001). According to this typology, many disciplines in Science are classified as hard-pure, whereas many in the Arts (social sciences and humanities) are classified as soft-pure. Another criterion that drew me was that Science and the Arts house disciplines outside of my own and represent cultures with which I am unfamiliar. I felt that investigating settings where I had no set ideas or assumptions about \u00E2\u0080\u009Chow things work\u00E2\u0080\u009D would be advantageous. While I did not aim to be objective or neutral, I was interested in a context that existed outside the faculty of Education proper. 51 3.4.3 Departments Once I selected the faculties, I identified departments whose faculty members I could invite to participate. Again, I considered only departments where there was statistical information available on the institutional research website. I then narrowed the choice of department based on two main criteria: First, to increase my chances of recruiting individuals from those categories, there must be five or more assistant professors, and the same (or more) with the rank of associate and full professor combined.3 Since my original research sought to compare pre-tenured and tenured professors, the number of faculty members in these categories was relevant.4 Second, less than 20 percent of the faculty in a chosen department should hold the title of instructor or lecturer; it was my belief that a greater percentage of professors in such departments might teach undergraduate and graduate courses compared with departments that had a higher percentage of instructors and lecturers (see section 3.4.4 for information on the distinctions between instructors, sessionals, and professors). Finally, I selected six departments with the following profile: 3 I assumed associate and full professors were tenured but assistant professors were not. 4 Although I have categorized professors at three different ranks in Tables 1 and 2, the study design examined only two categories: pre-tenured and tenured. At the time of design, I (mistakenly) assumed that there was more homogeneity than there actually is in the experiences of tenured professors. What I learned is that the experiences of newly tenured faculty members and full professors appear to be quite different. 52 Table 1: Faculty of Arts: Representation by Rank and Gender in A1, A2, and A3 Department Academic Rank Total Assistant Professor Associate Professor Full Professor Women Men Women Men Women Men Women Men A1 1 4 7 12 2 5 10 21 A2 1 6 1 2 1 8 3 16 A3 4 5 2 2 3 12 9 19 Table 2: Faculty of Science: Representation by Rank and Gender in S1, S2, and S3 Department Academic Rank Total Assistant Professor Associate Professor Full Professor Women Men Women Men Women Men Women Men S1 2 5 5 15 2 17 9 37 S2 3 6 2 16 2 19 7 41 S3 1 10 3 12 2 25 6 47 3.4.4 Defining the Professor For this study, I recruited tenure-track professors at the rank of assistant, associate, and full professor, as indicated on their departmental and/or institutional website. Although the dissertation refers to study participants as faculty members, I recognize that the term refers to a broader population than just those with the title of professor. Excluded from the selection 53 process were faculty members listed as \u00E2\u0080\u009Csessional lecturer,\u00E2\u0080\u009D5 \u00E2\u0080\u009Clecturer,\u00E2\u0080\u009D or \u00E2\u0080\u009Cinstructor.\u00E2\u0080\u009D At UWC, sessional lecturers and lecturers have \u00E2\u0080\u009Cterm appointments without review.\u00E2\u0080\u009D For the purpose of this study, these were understood to be non-tenure track appointments. Instructors, on the other hand, are hired \u00E2\u0080\u009Cwith review appointments.\u00E2\u0080\u009D Although instructors hold tenure-track positions, different criteria for promotion and tenure apply to instructors and \u00E2\u0080\u009Cresearch professors.\u00E2\u0080\u009D 6 In particular, the criteria for their tenure and promotion is more heavily weighted on teaching (including educational leadership, curriculum development and other teaching and learning initiatives) for instructors than for those who are appointed as professors. I excluded sessional lecturers and lecturers from the study because, like instructors, whose teaching \u00E2\u0080\u009Ccounts\u00E2\u0080\u009D for more in career decisions, the literature has reported that the experiences of contingent faculty (lecturers and sessional lecturers in the case of UWC) are significantly different from those with tenure-track appointments (Gappa et al., 2007; O\u00E2\u0080\u0099Meara et al., 2008; Schuster & Finklestein, 2006).7 Where subsequent chapters quote or refer to specific participants, the following designations 5 At UWC, faculty members and others often refer to \u00E2\u0080\u009Csessional lecturers\u00E2\u0080\u009D as \u00E2\u0080\u009Csessionals.\u00E2\u0080\u009D Where it appears in this dissertations, the words \u00E2\u0080\u009Csessional faculty\u00E2\u0080\u009D or \u00E2\u0080\u009Csessional\u00E2\u0080\u009D refer to the \u00E2\u0080\u009Csessional lecturer\u00E2\u0080\u009D appointment. 6 The participants I interviewed commonly used the term \u00E2\u0080\u009Cresearch professor\u00E2\u0080\u009D to distinguish them from other faculty members. 7 Those faculty in nontraditional appointments report less career satisfaction than their tenured colleagues (Gappa et al., 2007; Robinson, 2006), frequently attributed to the fact that they do not enjoy the same degree of academic freedom or job security as those with tenure, shoulder larger teaching loads, and experience feelings of second-class citizenship or marginalization (Gappa et al., 2007; Hamilton, 2007). Contingent faculty also express concerns about their capacity to develop campus-based relationships with colleagues (O\u00E2\u0080\u0099Meara et al., 2008). The result is a limited sense of commitment to the employing institution (Hamilton, 2007; O\u00E2\u0080\u0099Meara et al., 2008). 54 will indicate faculty, department, rank, and gender: \u00E2\u0080\u00A2 S1, S2, S3: Participants from the Faculty of Science, Departments 1, 2, or 3 \u00E2\u0080\u00A2 A1, A2, A3: Participants from the Faculty of Arts, Departments 1, 2, or 3 \u00E2\u0080\u00A2 Full professor (T); Associate professor (t); Assistant professor (a) \u00E2\u0080\u00A2 Male professor (M); Female professor (F) Thus, a quote or passage followed by \u00E2\u0080\u009C(Irwin, A1aM)\u00E2\u0080\u009D is referring to participant Dr. Irwin, an assistant male professor in the Faculty of Arts, Department 1. 3.4.5 Recruitment: Beginnings Once I identified the assistant, associate, and full professors from each department and collected email addresses, I began the actual recruitment process (see Letter of Initial Contact in Appendix A). Originally I aimed at recruiting a total of 20\u00E2\u0080\u009324 participants. I started by inviting faculty members in S1 and S2, followed a week later by those in A1 and A2. (I postponed any recruitment from S3 and A3 until I was sure I could not recruit a sufficient number from the other four departments.) If a person responded with a yes or tentative yes, I sent them a short questionnaire (see Faculty Member Questionnaire in Appendix B) to ascertain whether they met the eligibility criteria laid out in Table 3 below. 55 Table 3: Participant Eligibility Criteria Criteria Assistant Associate Full 1. Has been employed by UWC for 9+ months Yes Yes Yes 2. Has peer reviewed a colleague on his/her teaching for tenure, promotion and/or reappointment Yes or No Must answer yes to criteria 2 or 3 or both. Must answer yes to criteria 2 or 3 or both. 3. Has been peer reviewed on his/her teaching for tenure, promotion and/or reappointment Yes or No8 Must answer yes to criteria 2 or 3 or both. Must answer yes to criteria 2 or 3 or both. 3.4.5.1 Challenges Recruiting Assistant Professors and Women It did not take long to notice that the greatest number of positive responses came from tenured male professors. In order to increase responses from assistant professors and women, I modified the eligibility criteria for assistant professors (fn. 8) and sent a follow-up email to those women and assistant professors who had not yet responded (Appendix C), indicating I was especially keen to hear from them. I also invited recipients to provide information on why they might not care to participate to help explain the low response rate from certain faculty categories. I heard from 5 individuals whose responses are presented in Table 4. 8 Prior to an official amendment of the Behavioural Ethics Review Board, I wanted to recruit those assistant professors whose teaching had been summatively peer reviewed. After almost a month (and only one assistant professor agreeing to participate), my committee decided that I should expand the criteria to include assistant professors who had not undergone a summative peer review on their teaching, as long as they could offer an opinion about it. Of the final participant selection, only one assistant professor (Irwin, A1aM) was not peer reviewed for tenure, promotion, or reappointment. All others had undergone the process at UWC. 56 Table 4: Responses to Follow up Email: Reasons for Not Participating Rank, gender and Faculty Email response Summary of reason participant declined Associate, woman, S I would normally say yes, but I am really pressed for time this summer. Lack of time Associate, woman, S I've thought about your request after receiving your first email. Normally I'd like to help, but I really don't have any opinions re: the peer review of teaching. I wasn't even aware that it is going on, despite my being here for XX years. We get the student feedback on our courses (online form results) and that's about it. I think. Except when I went up for tenure and had faculty from my committee come and sit in a few of my classes. However, since their reports were private, I have no idea what they wrote/thought or whether the process worked or not -- well, I got tenure so I suppose it worked for me. I hope you'll find enough faculty who actually have something to offer in response to your questions, I really don't have any thoughts on this. Does not have anything to contribute despite having been peer reviewed. Full, man, A I have considered your request. In the end, I have decided not to participate in the process, because I have concerns about disclosing confidential information. Concerns about confidentiality. Assistant, woman, S I am afraid that I cannot take part in your project, as this summer is an extremely busy time for me. The main barrier to participation is time - given than I am pre- tenure and have just returned from maternity leave, I have very little of it and I try to make the most with what I have. My suggestion to you is that if you want to attract more participants, you need to come up with a way to make participation in your study beneficial to the participants. I have in a past agreed to such things when it was clear that I was going to learn something useful to myself in the process. Lack of time. No benefit to participating. Associate, woman, A Thank you for your e-mail. I am going to decline to participate as I have had no experience with peer review of teaching. No experience with topic of peer review. As a result of the follow-up emails, I was able to recruit additional women faculty members, but continued to have a low response rate from assistant professors. I also gained further 57 insight regarding non-participation, specifically that time was a barrier. Some faculty members also stated they had nothing to contribute, while others had concerns about confidentiality and/or could see no benefit from study participation. Approximately a month into recruitment, I replaced my original plan to recruit an equal number of professors from each category (Arts/Science, Pre/Tenured, Women/Men) with that of recruiting an equal number of professors from Arts and Science, with a participation goal of 30% women (which represents the average ratio of tenure-track female faculty at UWC). By then I had sent two emails to all faculty members in S1, S2, S3, and to those in A1 and A2. After confirming participation from faculty members in A3, I completed my recruitment. Tables 5 and 6 represent the final results: Table 5: Final Recruitment from Faculty of Arts (n=15) Department # Women Pre- Tenured # Men Pre- Tenured # Women Tenured # Men Tenured A1 0 (1) 2 (4) 0 (9) 1 (17) A2 0 (1) 0 (6) 1 (2) 3 (10) A3 2 (4) 1 (5) 2 (5) 3 (14) Total # 2 3 3 7 Note: Numbers in parentheses are the total number of professors at that rank in the specific department in 2009. Table 6: Final Recruitment from Faculty of Science (n=15) Department # Women Pre- Tenured # Men Pre- Tenured # Women Tenured # Men Tenured S1 1 (2) 0 (5) 3 (7) 2 (32) S2 1 (3) 0 (6) 2 (4) 2 (35) S3 0 (1) 1 (10) 1 (5) 2 (37) Total # 2 1 6 6 Note: Numbers in parentheses are the total number of professors at that rank in the specific department in 2009 58 3.5 Interviews Listening and asking questions are central components of a qualitative interview (Creswell, 2007; Fontana & Frey, 2008; Mayan, 2009). Applying these skills in this active and mutually relevant exchange is a difficult task, however. During the interview, as well as before and after, the researcher must establish and maintain trust, practice sensitivity, exercise intellectual judgment, and respond to multiple sensory observations (Fontana & Frey, 2008). Each research engagement thus began with a recruitment email and continued well after I had turned off the audio recorder or put away my pen. 3.5.1 Data Collection Data from this study consisted of 30 semi-structured interviews with professors. Mayan (2009) writes that the use of a semi-structured interview is appropriate when the researcher has a sufficient sense of the phenomenon to develop interview questions, but not so strong as to predict responses. By posing open-ended questions, the researcher aims to give informants space to express meaning in their own words (Brenner, 2006). I chose this method for this study because it had the potential to elicit thoughts, reflections, and ideas about the conditions of professional growth from a faculty member\u00E2\u0080\u0099s perspective. Interviews lasted between 45\u00E2\u0080\u009390 minutes, with an average of 60 minutes. All but two took place in the faculty member\u00E2\u0080\u0099s office on campus. The majority of interviews were audio recorded and then transcribed verbatim by a hired transcriptionist (some fillers omitted). Two interviews were not audio recorded upon the participants\u00E2\u0080\u0099 request, and I transcribed four interviews myself. With the understanding that academics have heavy workloads, only one interview was scheduled for each faculty member. 59 In preparation for the interviews, I pilot-tested my protocol with two faculty members in order to detect any weaknesses or flaws in the interview design (Turner, 2010). The value of those conversations (one with a pre-tenured male professor I met through a colleague, the other with a recently retired professor who is a longtime family friend) was immense, and I learned about aspects of summative peer review that I had not ascertained by reading the literature. Following the pilot, I modified some open-ended questions and probes (Barbour, 2008) and created the final protocol for pre-tenured and tenured faculty members (Appendices D and E, respectively).9 During the actual interviews, the protocol was used as a guide to provide an overall framework. In that way, I could adjust the sequence, direction, and precise wording of my questions in response to any issues and emotions that were raised. 3.5.2 Methodological Issues One of the primary purposes of qualitative interviewing is to derive interpretations from the narratives and statements of respondents (Warren, 2001). The interview is a conversation (Kvale, 1995), meaning an interactional relationship in which researcher and participant are engaged in a process of knowledge production (Fontana & Frey, 2008; Kvale, 1995). As such, interview interactions are shaped by both perspectives, respondent and interviewer, based on their respective histories, gender, age, race, and class (Fontana & Frey, 2008; 9 For example, my first question was originally \u00E2\u0080\u009CPlease tell me about your involvement in peer review.\u00E2\u0080\u009D For those who had a lot of experience with peer review, this proved to be a stumbling block, as the person would focus on recalling various experiences, i.e., when they happened and in what order. I later modified this to \u00E2\u0080\u009CPlease tell me how peer review takes place in this department,\u00E2\u0080\u009D which allowed participants to relate personal experiences and/or knowledge of the departmental process. 60 Warren, 2001). Over the course of this study, I became aware of the extent to which my status (i.e., a female graduate student who has never undergone a summative peer review) shaped the interviews. It meant I came to each interview as a learner (Warren, 2001), with no personal experience as a tenure-track faculty member nor with summative peer review. I sensed that research participants related to me in this role, and as a result were patient with my questions and attentive to my interest in learning. While I cannot be sure, I also wondered whether being a graduate student had some influence on the interviews, i.e., motivated faculty members to be interviewed despite their busy schedules. Did they contribute to my research as an extension of their responsibilities as a supervisor/mentor of graduate students? Because they assumed I might become a member of their chosen profession? Or because the topic piqued their interest and was something they wanted to discuss? After conducting ten interviews with male professors, I began to interview the women, and noticed the effect my gender seemed to have on both sets of participants. Women spoke more frequently about stress in their career, used greater emotion when speaking about their frustrations, and even swore more often. Had I been a male interviewer, I wasn't sure the same conversations would have ensued. In a chapter entitled \u00E2\u0080\u009CInterviewing Men,\u00E2\u0080\u009D Schwalbe and Wolkomir (2001) use the term baseline threat to describe fundamental interview characteristics (e.g., the interviewer sets the agenda, asks questions, probes for information, etc.) likely to cause both sexes some 61 discomfort. They distinguish these from what they call surplus threats, which are lines of questioning that can threaten the conventional ways men \u00E2\u0080\u009Cdo gender\u00E2\u0080\u009D in Western culture (West & Zimmerman, 1987). To oversimplify their descriptions, men may distinguish themselves from women by exercising a greater capacity and desire for control and portraying themselves as rational and autonomous in thought and action (Schwalbe & Wolkomir, 2001). The authors point out that questions about gender, or those questions that \u00E2\u0080\u009Cput control, autonomy, or rationality into doubt\u00E2\u0080\u009D (p. 206), can be perceived as threatening. Given that such questions were included in my protocol, this dynamic may have been at work in some of my interviews with male professors. The literature on qualitative interviewing also alerts us to the danger of presuming an interview relationship to be egalitarian (Fontana & Frey, 2008; Warren, 2001). It is often assumed that the researcher holds a position of power since it is s/he who defines the situation by introducing the topic and deciding how to follow up on a respondent\u00E2\u0080\u0099s answers (Kvale, 1995; Warren, 2001). Although I acknowledge authoring power with respect to the final text, when it came to the interviews I felt the participants in this study held the greater power. First, I relied on them for data (and therefore for completion of this project); and second, their status in the academy and society was greater than mine. Whether it be differentials related to social locations such as age, gender or position, the researcher needs to be attentive to how power differences between the interviewee and the researcher play out (Fontana & Frey, 2008; Warren, 2001). 62 3.6 Data Analysis I began the data analysis during data collection, making notes and reflecting on ideas after each interview. I created a face sheet (Grbich, 2007) for each participant, on which I recorded basic facts about the interview: name of participant, time/date/location, and duration; notes on what I could recall; things that surprised me; and issues that I wanted to follow up. The notes were generally typed up before I listened to the recording (i.e., within 1\u00E2\u0080\u00932 days of the interview), but I occasionally compiled the face sheet only after listening to the interview. Face sheets were embedded in my research journal, an electronic document I began keeping in December, 2008, which I used to track ideas and decisions from discussions with my committee, thoughts generated as I worked through the study, reflections on readings from a qualitative data analysis course, and circumstances and feedback I wanted to record. Shortly before the interviews were completed, I decided to use a qualitative data analysis software called Atlas.ti. Having never used this type of software before, I was not exactly sure how the process would unfold but trusted that, at the very least, Atlas.ti could help me manage, organize, code, and retrieve data from the interviews. I ended up proceeding as follows: Firstly, I chose three interviews, read through each, line by line, and assigned names to segments (e.g., sentences or paragraphs) of the data. I refer to this process as coding. This allowed me to revisit the interviews that, in some cases, I had not listened to or read for some time. Then, I carefully examined the codes I had created thus far, determining whether they captured the meaning of the text as I understood it. I subsequently modified a number of the codes, refining my code definitions and then re-coding the three interviews. Once satisfied with my \u00E2\u0080\u0098new\u00E2\u0080\u0099 codes, I coded the remaining 27 interviews. As I engaged in the coding, some 63 of my codes were further segmented and others were joined (see Table 7 for sample codes and categories). I simultaneously coded and categorized the data, exploring how codes and categories related to one another and remaining attuned to patterns in the data, and \u00E2\u0080\u009Cnegative cases,\u00E2\u0080\u009D (Mayan, 2009, p. 95) or data that are contrary to what the majority of other participants describe. Finally, after coding each interview, I wrote a detailed summary\u00E2\u0080\u0094an expanded face sheet of sorts\u00E2\u0080\u0094to help me see the interview as a whole again. Table 7: Categories and Codes (Select Examples) Category Codes Process Communication Data Ideal Knowledge Timing Reviewer Committee Rank Selection Tenure and Teaching Administration Honesty Own experience Research vs. teaching Time The entire process took several months, and in the end, I benefited from Atlas.ti when I was able to retrieve data associated with the codes and categories I had created. Using Microsoft Word, I grouped all the instances within a given code or category into separate documents. For example, for the code \u00E2\u0080\u009Chonesty,\u00E2\u0080\u009D I examined each instance where a participant discussed or referred to honesty in terms of the summative peer review of teaching (e.g., whether they felt they could be honest writing teaching reports, sharing feedback, or during committee meetings where colleagues discussed the case). As I continued to create sub-categories, I 64 began to form a loosely detailed outline of themes from my findings. I then proceeded with analysis and writing. Whereas the account above portrays my analysis as consisting of distinct steps, in reality the process was not so straightforward. I engaged in description, analysis, and interpretation throughout (Wolcott, 1994),10 but my emphasis varied at different times. A large amount of analysis later occurred as I outlined, wrote, and revised my dissertation. 3.7 Rigour and Limitations \u00E2\u0080\u009CRigor,\u00E2\u0080\u009D writes Mayan (2009) \u00E2\u0080\u009Cis demonstrating how and why (through methodology) the findings of a particular inquiry are worth paying attention to\u00E2\u0080\u009D (p. 100). In qualitative studies, the researcher strives for rigour when she keeps accurate records, follows systematic procedures, recognizes the complexity of a situation, and is explicit about her biases (Mayan, 2009; Merriam, 1998). The limitations of this study can be specified in relation to three attributes of qualitative research: validity, reliability, and generalizability. 3.7.1 Validity Validity in qualitative research refers to whether conclusions, descriptions, and other 10 Drawing on Wolcott (1994), I take \u00E2\u0080\u009Cdescription\u00E2\u0080\u009D to mean recording data, as the researcher originally encounters it. \u00E2\u0080\u009CAnalysis\u00E2\u0080\u009D refers to the process whereby the researcher identifies essential features in the study and the relationships between them. During \u00E2\u0080\u009Cinterpretation,\u00E2\u0080\u009D the researcher extends the analysis, looking for further insights and creating new meaning from the data. 65 accounts can be found in the data (Mayan, 2009). It is not akin to \u00E2\u0080\u009Cobjective truth\u00E2\u0080\u009D but pertains, rather, to the notion of credibility. Validity is strengthened when a researcher endeavours to achieve methodological coherence\u00E2\u0080\u0094that is, the questions, methods, data, and analytic procedures all \u00E2\u0080\u009Cmatch\u00E2\u0080\u009D (Morse, Barrett, Mayan, Olson, & Spiers, 2002). Validity is augmented when the researcher can furnish evidence that her findings provide an accurate representation of the participants and/or the data (Maxwell, 2005). In order to augment the validity of my data, I implemented various verification strategies. First, I paid close attention to sampling to ensure that it allowed me to collect \u00E2\u0080\u009C\u00E2\u0080\u0098rich\u00E2\u0080\u0099 data\u00E2\u0080\u009D (Maxwell, 2005, p. 110). Rich data provides the researcher with detailed descriptions of relevant phenomena (Mayan, 2009). The richness of the data in this study came from the fact that I interviewed 30 faculty members from different disciplinary backgrounds, departments, genders, and ranks. All had been involved in the peer review of teaching and could therefore speak from experience.11 It also resulted because 28 recorded interviews were transcribed verbatim; as such I did not rely on taking notes on only what I felt was significant. I listened to nearly each recorded interview three times: once within three days (almost always) of interviewing the participant, a second time when I meticulously reviewed the transcript upon receiving it and a third time just before sending the transcript back to the participants for verification. In the two cases when the participants asked not to be recorded, I was careful to 11 At the time of the interview, only one individual (Irwin, A1aM) had not taken part in the summative peer review of teaching for tenure or promotion. He had, however, been involved in team teaching, observing a colleague\u00E2\u0080\u0099s class for an entire semester, and other peer review situations at another institution. See also footnote 8. 66 take comprehensive notes of our discussion. Second, I was alert for any negative cases or data that were otherwise incongruous (Maxwell, 2005; Mayan, 2009). I was careful not to ignore these data, pondering their significance and implications and making them public in my findings. A third strategy was respondent verification. This, writes Maxwell (2005), is the most important means of reducing the possibility of misinterpreting what participants have said. In Autumn 2010, each participant was invited to review and modify the interview transcripts; one year later, they were invited to comment on the two findings chapters. 3.7.2 Reliability Reliability pertains to consistency in the data (Merriam, 2002). A qualitative study could be considered reliable if, for example, the same or different observers were assigning instances to the same category on different occasions (Long & Johnson, 2000). Merriam (2002) suggests that some of the same techniques used to ensure validity, notably member checks and peer review (such as the review of my work by committee members), can be applied to achieve better reliability. In addition, she recommends the use of an audit trail, a detailed account of how data were collected and decisions made throughout the research project. In 2008, I began a research journal to document the evolution of my ideas, keep track of decisions, and note down questions. When I commenced my data collection in the spring of 2010, I used my research journal to write my thoughts and impressions after each interview, and later detailed summaries after coding line by line. I returned to those summaries multiple times, noting links between participants, further queries, and contradictions. As I coded each interview, I kept notes on the development and 67 transformation of my codes and categories. As I advanced my analysis, I worked through many of issues and questions by means of writing. 3.7.3 Generalizability Generalizability \u00E2\u0080\u009Crefers to the extent to which one can extend the account of a particular situation or population to other persons, times, or settings than those directly studied\u00E2\u0080\u009D (Maxwell, 1992, p. 293). Flyvbjerg (2006) contests the belief that one cannot generalize from a single case, pointing out that many natural and social science problems have been solved through a detailed examination of an individual case, not by examining large, random samples. Instead, carefully chosen cases, which involve intense observation, can lead to important discoveries that contribute to the collective process of knowledge accumulation, which Flyvbjerg believes is also part of generalization. In qualitative studies, the concept of internal generalizability is particularly useful. This concept concerns whether the conclusions found within the research sample can be generalized to persons and events that were not observed or interviewed but fall within the same group or institution as the data sample. Internal generalizability is contrasted against external generalizability, which pertains to whether the study findings can be inferred to settings beyond the one sampled (Maxwell, 1992, 2005). Since qualitative studies are not designed to infer findings to a broad population (Maxwell, 2005; Mayan, 2009; Merriam, 2002), external generalizability is a less critical issue in qualitative research. The sample need not be representative of a larger group; instead, participants are selected based on 68 characteristics that allow the researcher to conduct an in-depth exploration of a given issue (Ritchie, Lewis, & Elam, 2003). For this study, my selection was aimed at capturing a range of experiences across departmental and disciplinary cultures, and across ranks and genders. This choice was driven by a belief that the faculty members I interviewed could provide valuable information about the peer review of teaching, departmental culture of teaching, and professional growth in teaching. Instead of trying to generalize based on a limited sample (one university, 30 participants, a particular moment in time), I drew on the strengths of qualitative research to better understand the phenomena I set out to study. I recognize that this study is limited in size and scope. All faculty interviewed are from one institution and their numbers are not large. Despite some variation in the departmental and disciplinary cultures, as well as the rank and gender of the respondents, I did not seek to examine differences in race and ethnicity, although these factors have been shown to influence faculty members\u00E2\u0080\u0099 work lives (Gappa et al., 2007; Mamiseishvili, 2010; O\u00E2\u0080\u0099Meara et al., 2008). Furthermore, I conducted only one interview per participant, which likely restricted my ability to probe in depth. 69 3.8 Ethical Considerations This study relied on data collected from human subjects and was conducted in accordance with the standards set out by the UWC Behavioural Research Ethics Board (BREB), which reviewed and approved ethics documents pertaining to the study. Conducting a BREB- approved study entailed taking certain precautions to assure the rights and safety of the study participants. These are described below. 3.8.1 Respect It was my intent to act respectfully and with consideration for my participants throughout the study. Since faculty members were asked to speak about their careers (both positive and negative aspects), I was aware that they might disclose sensitive information and/or recall incidents that would be accompanied by strong emotions. I did my best to respond appropriately, acknowledging participants\u00E2\u0080\u0099 frustrations and reminding them of the steps taken to ensure confidentiality and anonymity (including turning off recording devices at any point). On a practical level, I was mindful of faculty members\u00E2\u0080\u0099 schedules and arranged interviews at times and locations convenient to them; during the interviews, I strove to honour participants\u00E2\u0080\u0099 time constraints. 3.8.2 Privacy and Confidentiality Throughout this study, the privacy and confidentiality of my 30 participants and two pilot participants were treated with the utmost concern. As sole investigator, I was the only person with access to original, identifiable data; any collected data have been secured. In hiring a transcription service, I chose a company that uses a standard protocol for maintaining 70 confidentiality and offers a written confidentiality agreement. To further protect participants\u00E2\u0080\u0099 identities, I generated pseudonyms using an online random name generator, and when writing about participants, attempted to shield their identity by masking research specializations, course titles, and personal characteristics. Although I have used pseudonyms, I did not conceal participant rank and gender, as these elements were relevant to the study. Pseudonyms were also assigned to the institution, and when specific names of units, initiatives, or institutional processes threatened identity, I used generic names. 3.8.3 Informed Consent For their protection, participants were informed of the nature of the study and asked to provide written consent (Appendices F and G). Data collection commenced after a participant had read and signed the consent forms. Participation in this study was voluntary and all participants were free to withdraw at any time. Participants were allowed to review the full transcript of their interview and make any changes they felt were needed. Once my supervisors had approved an advanced draft of my findings chapters, participants were invited to review the findings in Chapters 4 and 5. 3.8.4 Institutional Review Board Approval In June 2009, I completed the Introductory Tutorial for the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans. I received BREB approval in March, 2010. 71 Chapter 4: Summative Peer Review of Teaching: Current Purposes and Envisioned Ideals 4.1 Chapter Structure In this chapter, I report on the participants\u00E2\u0080\u0099 understandings of and experiences with the summative peer review of teaching, and examine how they perceive the relationship between summative peer review and professional growth in teaching. I begin with a section on academic rewards and the demanding nature of academic careers as all participants made reference to these aspects of institutional culture when they narrated their experiences of summative peer review and spoke about professional growth in teaching. I then turn specifically to the peer review of teaching. First, I briefly outline the peer review process. In doing so, I aim to introduce some of the elements that make up peer review and highlight certain variations and inconsistencies inherent in the summative peer review of teaching practice. Next, I discuss three purposes of peer review. According to the participants, these are: a formal mechanism for evaluation, a means of promoting professional growth in teaching, and a supplement to the student evaluations of teaching. In the following section, I present findings on why the process does not foster professional growth in teaching and examine how the lack of feedback diminishes the potential for learning in the peer review of teaching. I then provide an account of participants\u00E2\u0080\u0099 descriptions of what an ideal peer review of teaching process might entail. I conclude this chapter with an analysis of the findings. 72 4.2 Institutional Culture: Endless Demands, Finite Time When participants refer to their careers in the research-intensive university, they repeatedly raise concerns about the high demands on their time, noting the consuming nature of research and teaching. Assistant and associate professors generally mention this more than full professors do, but the latter are cognizant of the pressures under which their pre-tenured and newly tenured peers operate. The main obstacle to the development of our younger faculty as effective teachers is the tremendous demands made on their time\u00E2\u0080\u0094the high expectations of research and publishing and productivity that are imposed on young faculty simultaneously to the expectation of teaching excellence. Every single junior faculty member could be a wonderful teacher and would want to be a wonderful teacher, and many of them are. But if any of them fall short of their own aspirations to be wonderful teachers, it is only because of the tremendous pressure also to perform at a standard of excellence in their other main responsibility, which is to do research and to publish their findings. (Manfred, A2TM) Though Manfred refers specifically to early career academics\u00E2\u0080\u0099 growth in the above quote, data from the interviews indicate that high expectations for research productivity affect participation in the summative peer review of teaching for faculty members at all career stages, and shape their ability to develop as teachers. 73 All faculty members acknowledge the competing nature of teaching and research, and several talked about the strategic choices they make between the two. That decision-making process can be difficult for people who wish to do their best in both research and teaching. I\u00E2\u0080\u0099m always torn between wanting to be a better teacher but being very, very invested in research and having that be, in some sense, the central part of my life, and then there\u00E2\u0080\u0099s teaching. And yet, if you\u00E2\u0080\u0099re going to do something, you want to do it right. So if I had infinite time to learn about teaching, I\u00E2\u0080\u0099d be involved in all these trenches, but I don\u00E2\u0080\u0099t. So I do what I can. (Ledonne, S1tF) Given the intensity of academic careers, faculty members indicate that they make decisions based on what they perceive to be most useful with respect to their professional and personal goals. You\u00E2\u0080\u0099d like to say we just want to be the best possible we can, but there\u00E2\u0080\u0099s finite resources that we all have to put into this and we have to decide strategically, you know, I already get four hours of sleep a night. Tell me to work harder at my teaching? (Palo, S1TF) Women express with more candour and emotion than their male counterparts the stress they experience in their academic lives as a result of the high demands on their time. \u00E2\u0080\u009CThe time thing is a really, really major serious problem. The work/life balance is disastrous. I mean I would quit if I could, because I\u00E2\u0080\u0099m so tired. You know? I\u00E2\u0080\u0099m just so tired. I can\u00E2\u0080\u0099t do more\u00E2\u0080\u009D (Palo, S1TF). Strained by heavy loads, all faculty members want to be clear about the purpose of the activities they engage in and need to believe that these various activities support them in their career growth. 74 4.2.1 Teaching Counts (?) Participants unanimously agree that research productivity, as opposed to strong teaching, is what garners rewards and recognition with respect to career advancement and departmental prestige. A few point out that faculty members are not rewarded for exceptional teaching, and some participants recall being told that teaching awards or high teaching scores would \u00E2\u0080\u009Clook bad for tenure\u00E2\u0080\u009D (Nemeth, A2tF); that is, strong teaching is equated to insufficient commitment to research. Consequently\u00E2\u0080\u0094more generally due to the importance of research productivity in tenure and promotion decisions\u00E2\u0080\u0094some tenured faculty members have advised their junior colleagues not to focus very much on teaching prior to tenure. Having a poor research record, faculty members concur, presents a greater impediment to obtaining tenure. In every department in Science, I think that the research definitely comes first. And if your research record is not in place, no way will you be promoted at all; that's it. And also, in considerations of merit review and salary raises, research ranks higher than teaching. (Knauer S2TF) Not only does research matter more in tenure and promotion decisions, but a few participants believe that when it comes to teaching, all faculty members have to do is meet a minimum threshold which in their opinion is easily attainable. If, as these faculty members posit, it is true that \u00E2\u0080\u009Cmost people teach well enough to get tenure at a research university\u00E2\u0080\u009D (Chen, A3TM), then their energy is best spent on research, since this is where they will reap the greatest rewards. For a small number of pre-tenured or newly tenured faculty members (Cohen, A3aM; Dawson, S3aM; Fradera, A3tF; Wilks, S1tF), the relative weight of research and teaching on 75 promotion and tenure decisions is not as clear-cut. Cohen, whose teaching has been \u00E2\u0080\u009Creally good\u00E2\u0080\u009D but whose \u00E2\u0080\u009Cresearch has been okay,\u00E2\u0080\u009D ponders, \u00E2\u0080\u009CIf my research isn\u00E2\u0080\u0099t going to be quite as strong enough, will my strong teaching help carry the day? I don\u00E2\u0080\u0099t know. We\u00E2\u0080\u0099ll see\u00E2\u0080\u009D (Cohen, A1aM). Similarly, Dawson, wonders whether, had he won a Killam teaching award (a prestigious award that recognizes teaching excellence) but only had \u00E2\u0080\u009Cokay\u00E2\u0080\u009D research, administrators might feel more compelled to award tenure. He thinks the decision to award tenure might be influenced by the fact that the dean\u00E2\u0080\u0099s office is promoting teaching as important, demonstrated by their support for the Effective Science Education Project (ESEd).12 Two faculty members provide accounts of major changes they made to their courses, motivated in part by a belief that their efforts would be positively regarded by colleagues who vote on tenure and promotion. Wilks (S1tF) recalls, I did think that it would help fill out my teaching portfolio because it shows that, okay, I don\u00E2\u0080\u0099t have breadth of teaching in terms of other courses but it\u00E2\u0080\u0099s certainly not that somebody\u00E2\u0080\u0099s handed me the [name] courses and I\u00E2\u0080\u0099ve used their slides for the last five years and done nothing [laughs]. I\u00E2\u0080\u0099ve done a heck of a lot on the teaching front, or put a lot of energy in. . . . I knew in this department it would be positively seen. So I didn\u00E2\u0080\u0099t think, \u00E2\u0080\u0098I\u00E2\u0080\u0099m wasting my time, nobody in this department\u00E2\u0080\u0099s going to recognize 12 The ESEd Project aims at improving undergraduate science education at UWC. Started in 2006, it is a multi-year project that works collaboratively with UWC departments in the Faculty of Science on a range of activities meant to enhance science education. 76 it.\u00E2\u0080\u0099 If anything, I knew they would recognize the effort because they do place an important role on the teaching component. Dawson (S3aM), whose immense concern with student learning prompted him to take on the demanding task of completely redesigning one of his courses, reflects: \u00E2\u0080\u009CI thought it would help my case. . . . Will it help as much or in the right way? I don\u00E2\u0080\u0099t know.\u00E2\u0080\u009D These stories stand out because they contradict the dominant opinion that good teaching is not rewarded as well as strong research. Since, at the time of making the changes, both participants were pre- tenured, it is possible they might not have fully adopted the belief that research productivity matters more than teaching in tenure and promotion decisions. It is noteworthy that, unlike some of the pre-tenured/newly tenured faculty members who wonder about the role of teaching in moving a tenure file forward, tenured faculty are unequivocal about what matters in tenure and promotion decisions: research is what counts. Despite an overall consensus that research counts more than teaching, several faculty members say that teaching ability is given serious consideration in tenure and promotion decisions. \u00E2\u0080\u009CTeaching is viewed very importantly and it\u00E2\u0080\u0099s very clear that promotions without good teaching are going to be extremely problematic, and therefore every possible effort is made to ensure teaching is good\u00E2\u0080\u009D (Knauer, S2TF). Moretti (S3TF) cites the fact that some faculty members are denied tenure based on a poor teaching record as proof that, institutionally, teaching matters more than it did before: It's been announced and written up in [name of university publication] that people are being denied tenure and promotion based on teaching. So that\u00E2\u0080\u0099s the most important thing. I think that sort of scared people into realizing teaching\u00E2\u0080\u0099s important, because 77 some people now have been denied tenure based on their teaching. And you know, ten or more years ago, that never ever, ever, ever, ever would have happened. \t\r \u00C2\u00A0 Numerous faculty members, however, speak with cynicism about the university\u00E2\u0080\u0099s claim that teaching matters in tenure and promotion decisions. \u00E2\u0080\u009CLet\u00E2\u0080\u0099s put it that way,\u00E2\u0080\u009D says Maki (A3aF), \u00E2\u0080\u009Cnobody\u00E2\u0080\u0099s ever been denied tenure for being a mediocre teacher.\u00E2\u0080\u009D Dawson reinforces this belief: \u00E2\u0080\u009CI would say the prevailing notion is that you can be a bad teacher and if you have excellent research, you will get tenure.\u00E2\u0080\u009D In addition, several associate and full faculty members remark that their teaching has never been peer reviewed since they attained tenure. Others like the faculty members in S1 point out that peer review, which takes place twice a year for pre-tenured faculty members, occurs only once every three years for tenured faculty. If \u00E2\u0080\u009Cto care about teaching is to evaluate teaching\u00E2\u0080\u009D (Ludar, A3TM), then faculty members in all departments may perceive that this nonexistence/decrease in peer reviews reflects the lessening importance of teaching as careers advance. Some faculty members suggest that faculty members would receive more time to develop professionally in this role if the university administrators were serious about improving teaching. \u00E2\u0080\u009CThat\u00E2\u0080\u0099s where [UWC] really falls down, trying to do surface fixes rather than really paying attention to what people need to be better teachers in terms of time, energy, and money\u00E2\u0080\u009D (Nemeth, A2tF). The problem at present is that the necessary support is not offered and, as a result, \u00E2\u0080\u009CYou just have to make time, so few people do it because it\u00E2\u0080\u0099ll cut into your current teaching quality and preparation or it will cut into your research\u00E2\u0080\u009D (Fradera, A2tF). 78 Having described the institutional context, I now turn to the summative peer review of teaching as it is perceived and practiced within this context. 4.3 Summative Peer Review of Teaching: A Sketch There is variation in how the summative peer review of teaching is understood, implemented, and experienced by individuals across departments within the University. For many, the summative peer review of teaching is synonymous with the classroom observations of teaching; others recognize that it may also include a review of other aspects of teaching, such as an instructor\u00E2\u0080\u0099s course syllabi and assignments. A large number of faculty members understand the summative peer review to consist of both the student evaluations of teaching and the classroom observations of teaching. According to the participants, the practice of peer review is directed, for the most part, by few or no written guidelines.13 The process is frequently shaped by reviewers\u00E2\u0080\u0099 preferences rather than according to established \u00E2\u0080\u009Crules.\u00E2\u0080\u009D Among the six departments included in the study, only A3 had a written policy on the peer review of teaching at the time of the interviews. Though one might assume that its existence resulted in some consistency in how peer reviews were conducted, this was not the case. Faculty members in A3, like in the other departments, had varying interpretations of how peer reviews of teaching should unfold and vastly different personal experiences of the process. 13 At the time of the interviews, there was a university level report developed by a working group on peer review of teaching. The document, which outlines principles of peer review, had been distributed to department heads by the Office of the Provost and Vice President Academic in November, 2009. 79 Participants comprehend that individual experiences of peer review differ and, consequently, they offered their narratives, not as representative samples of how things work for all individuals within their department, but as their own reflections on their summative peer review experiences. Taken together, the 30 participants\u00E2\u0080\u0099 unique experiences of peer review produce a landscape of peer review analogous to a Sumatran jungle, rather than a Zen garden. To some degree, the process is more unpredictable than it is foreseeable, less developed than desired, and often unplanned. 4.3.1 Changes Over Time The majority of the participants in this study describe the peer review of teaching process as one that has changed over time during their career at the University. Because peer review continues to be in flux\u00E2\u0080\u0094 a \u00E2\u0080\u009Cwork in progress\u00E2\u0080\u009D (Maki, A3aF)\u00E2\u0080\u0094faculty members acknowledge that there is much they do not know about the process. This not knowing includes, but is not limited to, who is selected to do reviews and on what basis, at what career stages peer reviews are conducted, how reviews are conducted, what materials (if any) get considered in the review, and whether there is a departmental policy on the summative peer review of teaching. Most faculty members are aware, however, that peer reviews of teaching have become more formal over time and cite various factors as evidence of this trend: 80 \u00E2\u0080\u00A2 Information from peer reviews is used to evaluate teaching in tenure and promotion decisions whereas, in the past, student evaluation of teaching scores were often the only source of data used. \u00E2\u0080\u00A2 In most departments, there has been an increase in the number of reviews conducted at any given time. Common practice used to be that one reviewer observed one undergraduate classroom session and wrote up a report. Now, in most departments, two reviewers are selected per review and each one is tasked to review an undergraduate and graduate classroom. \u00E2\u0080\u00A2 Peer reviews of teaching are sometimes considered in making reappointment as well as tenure decisions, whereas the process used to apply only to tenure decisions. \u00E2\u0080\u00A2 In several departments, the peer review reports are more comprehensive than in the past. Traditionally, they might have incorporated commentaries from the classroom observations. Now they may include an assessment of the faculty member\u00E2\u0080\u0099s teaching materials, a review of his/her statement of teaching philosophy, and a statistical and/or graphical comparison of faculty members' student evaluation of teaching scores. Some faculty members believe these changes reflect an overall institutional shift where the University has begun placing a greater emphasis on teaching. 4.3.2 Timing and Frequency In all the departments included in this study, the peer review of teaching is conducted when assistant professors are being considered for promotion to associate professor. Consequently, peer reviews are typically scheduled so that results are produced in time for the standing committee to evaluate these alongside other data, such as the candidate\u00E2\u0080\u0099s research record and 81 service to the University. In some departments, this is the only time the peer review of teaching is done whereas in others, it is also conducted at reappointment. In S1 and S3, the peer review of teaching is conducted yearly in the pre-tenure years. Overall, across departments, pre-tenured faculty members are peer reviewed more frequently than their tenured colleagues. None of the new associate professors interviewed in this study have been peer reviewed since attaining that rank and most are not sure whether they will be in the future as part of an ongoing process of evaluation. A number of the tenured faculty members who began their careers in the 1980s or earlier have never been peer reviewed because the process was not used by their department when they were advancing through the professorial ranks and is not currently being used to evaluate full professors. 4.3.3 Selection of Reviewers The department head normally selects reviewers based on several considerations. Across all departments, the department head (or, in some cases, associate head assigned to administer the evaluation of teaching) is typically responsible for selecting reviewers for the peer reviews of teaching. It is the department head, many faculty members think, who has the greatest insight into personality dynamics between faculty members. Therefore, s/he is best able to select reviewers for the classroom observation of teaching and/or decide the composition of the committee that coordinates the peer reviews of teaching. Whether or not the department head has selected the reviewers, faculty members also name the department head as the person they would go to if they had concerns about who had been selected. In most departments, the person who reviews is senior in rank to the candidate but in S1 and A2 tenure track faculty members at all ranks (including instructors) conduct reviews. In some 82 cases, reviewers are selected because their area of content expertise is similar to the candidate\u00E2\u0080\u0099s. In other cases, reviewers are selected because they are available. Several faculty members say they do not think reviewers are chosen on the basis of being a \u00E2\u0080\u009Cgood teacher,\u00E2\u0080\u009D however a minority thinks the opposite is true. Most of the time, the candidates are told who their reviewers will be and occasionally they are invited to suggest reviewers\u00E2\u0080\u0099 names themselves or given the opportunity to comment on a short list of potential reviewers. 4.3.4 What is Being Reviewed? Participants cite the classroom observations of teaching as the main, and sometimes the only, source of information in the peer review of teaching. As already mentioned, these observations are always of an undergraduate class and, in most departmental processes, a graduate class is also observed. In rare cases, when a faculty member is not teaching during the term when their review is scheduled to take place, a reviewer may observe the candidate teach a lunch-time seminar. When, in addition to a classroom observation, other data are included for review, they may include any of the following: syllabi, assignments, reading lists, exams, a teaching philosophy statement, and a teaching portfolio. Many participants say that it is up to the reviewers to decide what sources of information they will request when they engage in peer review. 83 4.3.5 Communication Pre/Post Review In most departments, there is an unstated understanding that the reviewer shall contact the candidate prior to the review so they can agree on dates for the classroom observation of teaching. In S1, however, the candidate provides a list of suitable dates to a departmental coordinator who then communicates these to the peer review committee; the reviewer, a member of the committee, shows up to the class unannounced. While faculty members in S1 accept this protocol, others in the university find this approach \u00E2\u0080\u009Cunthinkable\u00E2\u0080\u009D (Manfred, A2TM). Mutually agreeing upon a date for the observation, Manfred explains, helps ensure that the class being observed is suitable, meaning that there is no exam planned for that day, or movie being shown or guest speaker and so forth. Assistant and new associate professors in departments where the norm is to mutually agree on a date for the observation say they appreciate knowing when to expect a reviewer. Prior to a classroom observation of teaching, reviewers and candidates rarely meet with one another. A3 provides an exception because the departmental policy requires a meeting, before the first classroom visit, to discuss the candidate\u00E2\u0080\u0099s teaching philosophy, his/her teaching portfolio materials, and specifics of the class, including class and course goals. In addition to A3 faculty members, only Nemeth (A2tF) recalls meeting with the candidate prior to the classroom visit and explains that she did it \u00E2\u0080\u009Cmore as a courtesy\u00E2\u0080\u009D to help make the candidate feel more comfortable and not because it was a required part of the process. Although reviewers may be part of a committee, they most frequently conduct the classroom observations on their own. They normally do so without the aid of an instrument or form, except in S1 and A3 where a departmental instrument has been developed and is used. 84 Occasionally, when there are multiple reviewers, the final report is produced collaboratively. Typically, however, each individual reviewer produces a written assessment that s/he later turns in to the head of the peer review committee or department head. Reports may consist of a one-page letter centering on the classroom observation of teaching or, as indicated previously, may include a review of the teaching philosophy statement, graphical/statistical comparisons of peers, and summaries of the student evaluation of teaching scores. Once reviewers complete their classroom observation and/or assess the material (e.g., teaching philosophy, syllabi, assignments, etc.), they may communicate their impressions or share their report with the candidate, but most of the time this step does not occur. 4.4 The Multiple Purposes of Summative Peer Review of Teaching Participants describe the summative peer review of teaching as serving three broad purposes: (a) a formal evaluation procedure conducted to aid in tenure and promotion decisions, (b) a mechanism that contributes to teaching professional growth, and (c) a process to supplement information from the student evaluations of teaching. 4.4.1 Formal Evaluation for Tenure and Promotion The summative peer review of teaching, say the majority of participants, is a formal evaluation conducted to fulfill a requirement for tenure and promotion. Bluntly put, the goal of the evaluation is to determine whether faculty members \u00E2\u0080\u009Cshould be hired or fired\u00E2\u0080\u009D (Fuentes, A3aM). Summative peer reviews of teaching, thus, are potentially punitive. In several departments, results of peer reviews are used not only to determine how the individual stands \u00E2\u0080\u009Cagainst tenure\u00E2\u0080\u009D (Ludar, A3TM) but also to compare faculty members to 85 one another. In both cases, the evaluation results inform those who are voting on the tenure case and, ideally, the outcomes help administrators build an argument in favour of the faculty member\u00E2\u0080\u0099s career advancement. Several participants remarked that prior to these critical evaluative episodes, peer reviews of teaching are a way to detect problems that might weaken a person\u00E2\u0080\u0099s future tenure case. When difficulties are identified early in a faculty member\u00E2\u0080\u0099s career, candidates have time to correct them prior to putting forward their files. A candidate\u00E2\u0080\u0099s file is strengthened overall when evaluators see that an individual\u00E2\u0080\u0099s teaching has improved over time, noted some faculty members. Moretti (S3TF) points out that sometimes the very act of making an effort to improve teaching, even if the student evaluation of teaching scores do not increase much, is judged positively by evaluators. As described later in this chapter, some faculty members put less focus on the connection to tenure; for them, the point of identifying problems early on is to offer guidance to instructors and help them develop professionally as teachers. Faculty members who see the purpose of summative peer review as an essential part of career maintenance and advancement frequently speak about the need to carefully choose their words and language when crafting their reports. They acknowledge being more comfortable documenting constructive suggestions for teaching improvement at reappointment, when the stakes are less high, than at tenure and promotion. Most participants say that constructive suggestions for teaching improvement are not included in the peer evaluation reports for tenure because these might have an \u00E2\u0080\u009Cunwarranted negative effect on the person\u00E2\u0080\u0099s circumstance\u00E2\u0080\u009D (Chen, A3TM). Given that these reports are tied to such high- 86 stakes career decisions, faculty members do not want to \u00E2\u0080\u009Cpollute\u00E2\u0080\u009D (Hardy, A2tM) their colleagues\u00E2\u0080\u0099 files with comments that might be misconstrued. \u00E2\u0080\u009CIt wouldn\u00E2\u0080\u0099t be appropriate to put those things in the summative evaluation because they\u00E2\u0080\u0099re just going to be viewed as negatives when they\u00E2\u0080\u0099re really meant to be helpful, constructive suggestions\u00E2\u0080\u009D (Stromberg, A1TM). When a reviewer thinks a specific candidate deserves tenure, s/he seeks to communicate this clearly. Consequently, for the most part, only positive statements are included in the written report and reviewers who opt to share their constructive feedback usually do so verbally. Parsons (A3TF) explains the need to state unambiguous support for a candidate\u00E2\u0080\u0099s case as follows: You don\u00E2\u0080\u0099t want to put in subtle things that people go, \u00E2\u0080\u009CWhat\u00E2\u0080\u0099s she really saying there? Does she really mean this?\u00E2\u0080\u009D If you really think the person deserves tenure, you want to say, \u00E2\u0080\u009CI think this person deserves tenure for this, this, and this reason.\u00E2\u0080\u009D Some faculty members, however, specify that if they had serious concerns about a person\u00E2\u0080\u0099s teaching they would note these in a formal report. Documenting poor teaching, according to a few participants, is part of a faculty member\u00E2\u0080\u0099s responsibility toward students. It was because of their colleagues\u00E2\u0080\u0099 strong teaching that they had not formally reported poor teaching, and not because reviewers wanted to intentionally dupe the readers. \u00E2\u0080\u009CIt\u00E2\u0080\u0099s not that these letters that are written for the tenure file are filled with lies. They\u00E2\u0080\u0099re not. Most of our colleagues are actually quite good teachers\u00E2\u0080\u009D (Hardy, A2tM). With the exception of three faculty members in S2 who claim that the report on teaching presents a \u00E2\u0080\u009Cbalanced view\u00E2\u0080\u009D (Hanna, S2tF; Knauer, S2TF; Warr, S2TM), the majority of participants agree that \u00E2\u0080\u009Ceverything\u00E2\u0080\u009D that goes into a file \u00E2\u0080\u009Cis supposed to be good\u00E2\u0080\u009D (Trottier, A2tM). 87 A small number of participants express frustration at the perceived need to frame everything about a person\u00E2\u0080\u0099s case in the positive. That nothing should be open to a negative reading, be it in the summative peer review of teaching reports, the referees\u00E2\u0080\u0099 letters, or the department head\u00E2\u0080\u0099s report, puts a \u00E2\u0080\u009Cconstraint\u00E2\u0080\u009D (Nemeth, A2tF) on the process and what is communicated therein. The current practice also creates some anxiety among the reviewers who know that \u00E2\u0080\u009Ca single negative review can be quite damning\u00E2\u0080\u009D (Abendroth, S1tM). Furthermore, when reviewers feel compelled to include only laudatory comments or to \u00E2\u0080\u009Cspin it on the positive side\u00E2\u0080\u009D (Warr, S2TM), it means that the final reports are \u00E2\u0080\u009Chighly circumscribed\u00E2\u0080\u009D (Bulmer, A3TM) to the point that they are not useful in any pedagogical sense of improving a persons\u00E2\u0080\u0099 teaching. Despite the fact that summative peer review is part of the tenure and promotion process, a number of faculty members describe it as a \u00E2\u0080\u009Cformality\u00E2\u0080\u009D (Trottier, A2tM), or \u00E2\u0080\u009Ctechnicality\u00E2\u0080\u009D (Hardy, A2tM) that, though required, is largely done in order to have \u00E2\u0080\u009Csomething to put in the file\u00E2\u0080\u009D (Abendroth, S1tM). Peer reviews, some say, feel like a bureaucratic exercise imposed upon faculty members. Participants with this belief go through the motions but do not think \u00E2\u0080\u009Cit really goes anywhere\u00E2\u0080\u009D (Deitz, S1aF). That is, they do not believe the outcomes of peer reviews matter much in decisions about career advancement. One faculty member suspects that many of her colleagues see the peer review as a \u00E2\u0080\u009Cwaste of time\u00E2\u0080\u009D (Moretti, S3TF). Since faculty members are \u00E2\u0080\u009Call overworked\u00E2\u0080\u009D (Fuentes, A3aM), it is not surprising that some wonder whether the peer review of teaching adds unnecessarily to their workload. The uncertainty comes about because participants cannot readily identify what benefits the process produces and/or because they believe the benefits are too small to justify the amount 88 of time peer reviews consume. Among those who perceive that peer review of teaching is only paying \u00E2\u0080\u009Clip service\u00E2\u0080\u009D (Nemeth, A2tF; Maki, A3aF) to the importance of teaching within the academy\u00E2\u0080\u0094and not truly contributing to the improvement of learning\u00E2\u0080\u0094there is a sense that peer review makes it \u00E2\u0080\u009Clook like we value teaching [but] it only exacerbates the problem in some ways because it\u00E2\u0080\u0099s another administrative burden that takes away from our time that we have in the classroom\u00E2\u0080\u009D (Nemeth, A2tF). Others participants, however, say that they have not heard their colleagues express cynicism about the process and report that, on the contrary, most people take it \u00E2\u0080\u009Cvery seriously\u00E2\u0080\u009D (Woodroof, S2TM). 4.4.2 Professional Growth in Teaching Several faculty members speak about peer review as a process that contributes to teaching improvement and enhanced student learning. It can do so by promoting reflection on teaching, by providing a learning opportunity for the observer, by encouraging faculty member accountability, and by identifying the need for additional support. Firstly, when candidates receive feedback from their colleagues, this can help them reflect on their teaching. Reviewers may see things that are in a faculty member\u00E2\u0080\u0099s \u00E2\u0080\u009Cblind spots\u00E2\u0080\u009D (Maki, A3aF) and their comments can suggest alternative approaches to teaching. Reviewers\u00E2\u0080\u0099 observations may prompt the teacher to re-examine what happened during the class and to consider how to apply the feedback to other classes and teaching situations. \u00E2\u0080\u009CI look on the peer review as being an opportunity for an outsider to suggest things that I might change to make my lectures better\u00E2\u0080\u009D (Abendroth,S1tM). Furthermore, as a few faculty members note, 89 having written\u00E2\u0080\u0094not only oral\u00E2\u0080\u0094feedback is helpful because comments can be revisited over time. Secondly the peer reviews are connected to professional growth because the reviewer learns when s/he observes a colleagues\u00E2\u0080\u0099 classroom teaching. The act of observing and assessing the instructor stimulates the reviewer to think about his/her own teaching. \u00E2\u0080\u009COne of the things I was struck by is that I got some good ideas for my own teaching from sitting in a colleague\u00E2\u0080\u0099s grad seminar\u00E2\u0080\u009D (Parsons, A3TF). Some reviewers have follow-up conversations with those being reviewed to ask for additional details on a teaching-related activity or approach they observed and want to implement. Faculty members acknowledge that, aside from these occasions, they rarely see their colleagues teach and describe the learning that comes from that as enjoyable and also \u00E2\u0080\u009Cconsiderable\u00E2\u0080\u009D (Ludar, A3TM). Thirdly, peer reviews encourage faculty members to be accountable to their students because academics are aware that, as part of the process, the reviewer may be evaluating whether the course is aligned with the stated goals outlined in the syllabus and the short description in the academic calendar. Knowing that someone will be checking for alignment, faculty members may be more attentive to ensuring it exists. In addition, when a problem is detected during a review, colleagues \u00E2\u0080\u009Cfeel an obligation to the students, to everybody that that problem not be allowed to continue\u00E2\u0080\u009D (Quinn, S1TM). Thus, peer reviews promote accountability to students because, when faculty members work to enhance their teaching, the ultimate benefactors are the students. 90 Finally, peer reviews of teaching provide an opportunity to identify whether there is need for additional support and/or professional development among academics whose teaching may be \u00E2\u0080\u009Cstale\u00E2\u0080\u009D (Ludar, A3TM). In such cases, the department head can help faculty members determine which opportunities might assist them to expand their background and skills and otherwise address their needs. 4.4.3 Supplement the Student Evaluations of Teaching According to many faculty members, content in the peer review of teaching report aids interpret the student evaluations of teaching results. Comments from peer reviewers help evaluators understand the student scores and may summarize the factors which make a candidate a good teacher or not: They often put words to numbers. Yeah, someone gets a 4.2. How did they get their 4.2? Well, if you read the comments you\u00E2\u0080\u0099ll find they\u00E2\u0080\u0099re very enthusiastic and they really got us excited about stuff or perhaps they were incredibly organized and very clear in what students needed to learn. (Warr, S2TM) Information from peer reviews supplements that from student evaluations and does not normally contradict it. However, when student evaluation scores fall below the acceptable department range, some faculty members suggest that peer reviews can provide additional information that might challenge (or at least explain) low numerical scores. Deitz (S1aF) believes a positive peer review can \u00E2\u0080\u009Coffset\u00E2\u0080\u009D a problematic student evaluation (that is, one with low scores). Participants\u00E2\u0080\u0099 accounts indicate that, as long as the student evaluation of teaching scores fall within a range deemed appropriate by the department and by the individual\u00E2\u0080\u0099s own standards, the results are generally accepted. Peer reviews, it would seem, 91 typically play a secondary role to the student evaluations of teaching. For a minority of faculty members, in particular those who are skeptical of the student evaluation of teaching scores and think the instrument is inadequate, the peer review process is a \u00E2\u0080\u009Cmuch better resource for evaluating faculty teaching\u00E2\u0080\u009D (Cohen, A1aM) than the student evaluation of teaching instrument. Since student ratings and comments can suggest problems where they do not exist, peer reviews of teaching can be corrective of these results. \u00E2\u0080\u009CI think the peer review process is essential, partly because I think that the other information we have about teaching is really inadequate,\u00E2\u0080\u009D asserts Nemeth (A2tF). Thus, according to some faculty members, peer reviews do more than supplement existing student evaluation of teaching scores: they provide a necessary distinct perspective about the instructor\u00E2\u0080\u0099s teaching. In addition to providing additional insight into the existing student evaluation of teaching scores, or providing a unique perspective on teaching altogether, several faculty members praised reviewers\u00E2\u0080\u0099 comments for being more \u00E2\u0080\u009Cprecise\u00E2\u0080\u009D (Ledonne, S1tF) and/or \u00E2\u0080\u009Cobjective\u00E2\u0080\u009D (Dodgson, S2aF) than those from the student evaluations of teaching, which, according to many participants, are left by disgruntled students. Comparing the comments she receives from peer reviewers to those from students, Ledonne says \u00E2\u0080\u009Cit\u00E2\u0080\u0099s more bang for the buck, more constructive, more useful\u00E2\u0080\u009D because faculty members, as compared to students, are more articulate in making suggestions for improvement and better able to pinpoint demonstrated qualities of good teaching. Others say the peer reviews are more valid because a faculty member, given his/her teaching experience, is a better judge of whether the candidate is a 92 skillful teacher. This presumes that the faculty member providing the review is, him- or her- self, a capable teacher. 4.5 No Teaching-Related Professional Growth Though some faculty members claim that summative peer review of teaching can foster professional growth in teaching, the majority thinks it does\t\r \u00C2\u00A0not. Their reasoning falls into any of these three broad categories: feedback on teaching is non-existent or poor, the student evaluations of teaching carry more weight in the evaluation of teaching, and/or the peer review of teaching process is artificial. In this section, I elaborate upon each of these, focusing principally on the lack of quality feedback. 4.5.1 Feedback: Non-existent or Unspecific One of the reasons that summative peer review of teaching does not lead to professional growth in teaching is because the reviewer\u00E2\u0080\u0099s feedback is either non-existent or lacking in specificity. Whether or not the reviewer shares his/her feedback with the person under review, and how that feedback is crafted, depends largely on the individual reviewer. In S1, faculty members who have been reviewed multiple times on their teaching prior to tenure have come to expect that each individual reviewer will decide, or not, to debrief the classroom observation of teaching. \u00E2\u0080\u009CThe more conscientious ones do, or the ones that don\u00E2\u0080\u0099t have a meeting right afterwards or something\u00E2\u0080\u009D (Ledonne, S1tF). In other departments, assistant and associate professors who have been reviewed only once are less sure of how the process unfolds as compared to their colleagues who have undergone multiple peer reviews of teaching. Full professors in all departments, however, are aware that there is a lot of 93 variation in how peer reviews are conducted. In the quote that follows, Taylor (S3TM) aptly summarizes common approaches to feedback in peer review across both faculties when he says: \u00E2\u0080\u009Cthere\u00E2\u0080\u0099s no policy that it [giving feedback] should be done, but no one believes it shouldn\u00E2\u0080\u0099t be done.\u00E2\u0080\u009D Though individuals express a preference for sharing feedback, their stories indicate that it is an infrequent practice. Sometimes a reviewer shares brief, non-specific feedback with the candidate immediately after a classroom observation of teaching. General statements, such as \u00E2\u0080\u009Cit looked great\u00E2\u0080\u009D or \u00E2\u0080\u009Cgood class,\u00E2\u0080\u009D are standard. That could be the extent of feedback a candidate receives from a peer review. Though statements such as these lack in substance they, at least, let the candidate know that the reviewer perceived what happened positively, note a few participants. Other times, reviewers meet with the person being reviewed to discuss feedback on the classroom observation of teaching. Among the participants who had received verbal feedback in a post-observation meeting, most indicate that it did not contribute meaningfully to their professional growth as teachers. Some say they learned a little bit from the feedback whereas others state the feedback was \u00E2\u0080\u009Clargely useless\u00E2\u0080\u009D (Deitz, S1aF). For example, it is common for a candidate to be told, in general terms, that \u00E2\u0080\u009Cthere are no concerns, things went fine\u00E2\u0080\u009D (Maki, A3aF). Or, they might be informed that they have a distracting habit, such as fiddling with a water bottle and turning their back to the class. A candidate might also be told that students were checking emails or falling asleep in class. This type of \u00E2\u0080\u009Cthree-sentence conversation,\u00E2\u0080\u009D (Fuentes, A3aM), in which feedback is nonspecific or focused on the \u00E2\u0080\u009Cmechanistic\u00E2\u0080\u009D (Hadzik, S2tM) aspects of teaching, is \u00E2\u0080\u009Cnot particularly helpful\u00E2\u0080\u009D (Dawson, S3aM) for people who wish to improve their teaching, said several participants. Like the 94 verbal feedback, the quality of the written feedback is variable. However, only a small number of individuals spoke about the quality of the written feedback and I suspect that is because most faculty members do not see it, other than those in S1 who receive the completed forms from colleagues who have conducted the classroom observations of teaching. Some faculty members, including Warren (A3aF) expressed disappointment at the lack of feedback: I was disappointed not to have gotten more feedback on it because it could have been a mentoring opportunity as well. The two people who are doing my peer evaluation are both good teachers and it would have been interesting to have gotten more feedback from them. During the interviews, three female faculty members described their experiences of attempting to get additional feedback from their reviewer and in each case the result was a letdown. Maki (A3aF) recalls asking for clarification and further advice on a minor, constructive comment she received and being told \u00E2\u0080\u009CThat\u00E2\u0080\u0099s just how students are. . . . There is not much you can do about that\u00E2\u0080\u009D \u00E2\u0080\u0094a response that fell short of her expectations. Deitz (S1aF), who says that getting verbal feedback from a reviewer \u00E2\u0080\u009Cis not standard,\u00E2\u0080\u009D explains that one has to \u00E2\u0080\u009Cchase people down\u00E2\u0080\u009D after a classroom observation of teaching in order to get information. She recounts her experience of doing that, and the outcome, as follows: I did that the first couple of times just because I wanted\u00E2\u0080\u0094you know, it\u00E2\u0080\u0099s kind of nerve-racking\u00E2\u0080\u0094and I wanted to see what [the feedback] was. When I discovered that there wasn\u00E2\u0080\u0099t going to be anything useful, then I stopped chasing people down. 95 Similarly, Wilks (S1tF) describes following up with her reviewer on her own initiative, and asking if he could meet with her \u00E2\u0080\u009Cbecause I\u00E2\u0080\u0099m always looking for ways to improve my teaching\u00E2\u0080\u009D in a way that is not \u00E2\u0080\u009Csuper time-intensive.\u00E2\u0080\u009D Though he initially indicated a willingness to meet with her, and they had some back-and-forth email about a prospective meeting, that meeting never happened. In the end, she concludes: I felt like it was a bit of an imposition because he never came back. So I wonder if it\u00E2\u0080\u0099s just more time for the person actually doing the review. I have no idea what kind of time commitment they\u00E2\u0080\u0099re being told that this is for them and maybe it\u00E2\u0080\u0099s kind of like, \u00E2\u0080\u009CLook I\u00E2\u0080\u0099ve done my thing\u00E2\u0080\u009D and that\u00E2\u0080\u0099s it. That was sort of the sense I had with the second one so I didn\u00E2\u0080\u0099t push it. Both Deitz and Wilks are careful to point out that their respective reviewer is a \u00E2\u0080\u009Cgood guy\u00E2\u0080\u009D and they do not attribute the absence of a meeting and feedback to a lack of collegiality in their relationship. However, as Deitz points out, her experience has discouraged her from attempting to get additional feedback from reviewers. Although Wilks does not mention whether she will continue to pursue additional opportunities for feedback, one might expect that she will not if she repeatedly ends up being disappointed. 4.5.1.1 Feedback from the Department Head Just as participants note that they rarely receive feedback from those who have reviewed their classroom teaching, they indicate that post-peer review the department head rarely discusses the report with the candidate. When outcomes of the peer review of teaching process are brought up, it is typically in the context of an annual meeting between a pre-tenured faculty member and the department head during which the faculty member\u00E2\u0080\u0099s overall career progress 96 (towards tenure) is discussed. Ludar (A3TM), the only department head among the participants, sees the peer review process as something that either strengthens or weakens a candidate\u00E2\u0080\u0099s career progress. I receive this [peer review report] and this is part of the tenure process and I discuss if it\u00E2\u0080\u0099s good, bad or indifferent, and point out ideally that it is a good report and strengthens their case and how it relates to the student questionnaires and the other mechanisms that we have. (Ludar, A3TM) The point of reviewing teaching yearly with pre-tenured faculty members, he elaborates later in the interview, is to warn people early about poor teaching and to recommend concrete help to a faculty member who needs it. These two points are echoed by others and indicate that the evaluation of teaching matters predominantly as it pertains to tenure and promotion. However, Ludar notes that peer review of teaching is not exclusively linked to decisions about career advancement, but also about \u00E2\u0080\u009Censuring that our department remains populated by dedicated people for whom the teaching of students matters.\u00E2\u0080\u009D Most faculty members, though they recognize that peer review might contribute to improved teaching and student learning, say that discussions with their department head about peer review have been non- existent or have focused on how it might relate to their career advancement. Deitz (S2aF) doubts that her head even looks at the classroom observation of teaching if a faculty member\u00E2\u0080\u0099s student evaluation of teaching are \u00E2\u0080\u009Cgood enough.\u00E2\u0080\u009D When peer reviews are dismissed as irrelevant, it strengthens faculty members\u00E2\u0080\u0099 beliefs that student evaluation of teaching scores are of prime importance in the evaluation of teaching. 97 Faculty members whose department heads had acknowledged peer reviews say the feedback from their head was minimal and not helpful for teaching improvement. Maki (A3aF) remembers: \u00E2\u0080\u009CI talked to my head and I said, \u00E2\u0080\u0098Can I get any feedback about the review?\u00E2\u0080\u0099 and he said \u00E2\u0080\u0098All went well.\u00E2\u0080\u0099 \u00E2\u0080\u0098Is there any formal feedback?\u00E2\u0080\u0099 [I asked] \u00E2\u0080\u0098No\u00E2\u0080\u0099 [he answered].\u00E2\u0080\u009D Another pre-tenured female faculty recalls the A3 department head reading aloud the report and inviting her response. She described the interaction as awkward, having not had an opportunity to see the report prior to the meeting nor debrief the contents with her reviewers. She recalled being disappointed by some of the feedback but felt inhibited from voicing her reactions because of the hierarchical relationship in place. These stories demonstrate that discussions about peer reviews are not only restricted by the low quality of the feedback, but also by discomfort that arises due to power imbalances between the faculty member and the department head. 4.5.1.2 Why No Feedback? Why is quality feedback not routinely provided to faculty members as part of the peer review process? Although I did not pose this question in the interviews, faculty members ventured explanations which I have categorized in two ways: one, at tenure and promotion, reviewers are focused on evaluation, not on professional growth; and two, because the evaluation of teaching is difficult, reviewers do not know how to generate useful feedback. 4.5.1.2.1 Focus on Evaluation According to many faculty members, at tenure and promotion, peer reviewers are focused on evaluating the candidate, rather than on envisioning how they might help the candidate 98 become a better instructor. In their role as evaluators, some reviewers believe they must keep themselves at \u00E2\u0080\u009Carm\u00E2\u0080\u0099s length\u00E2\u0080\u009D (Woodroof, S1TM) from the candidate; one way of doing so is by not sharing feedback. That judgmental approach can be threatening, and is not conducive to professional growth. \u00E2\u0080\u009CIt\u00E2\u0080\u0099s about figuring out how good they are. Are they good enough for what we require of them?\u00E2\u0080\u009D (Nemeth, A2tF). In these cases, the peer review serves as a mechanism to determine whether the candidate is meeting a general threshold that the evaluator judges adequate. Fuentes (A3aM) calls it a \u00E2\u0080\u009Csort of check on \u00E2\u0080\u0098Is this person actually trying?\u00E2\u0080\u0099\u00E2\u0080\u009D Others add that the reviewers \u00E2\u0080\u009Cprobably perceive their role as just looking at things to make sure everything\u00E2\u0080\u0099s okay\u00E2\u0080\u009D (Dawson, S3aM). Finally, some suspect that the reviewers may not see giving feedback as part of their obligation: \u00E2\u0080\u009CAs long as you file your report reasonably on time, I think you\u00E2\u0080\u0099ve fulfilled your duties\u00E2\u0080\u009D (Quinn, S1TM). In A3, where there is an established policy on the peer review of teaching, some faculty members explain the lack of feedback by referring to their departmental policy that, according to them, states that the process is evaluative and is not a mentorship opportunity. Accordingly, an evaluator cannot simultaneously be a mentor and good departmental representative. Thus, in order to fulfill the role of departmental assessor-agent, it is best to separate mentorship from the summative peer review process. Most faculty members agree that when the summative peer review is being used as a tool for evaluation and to foster professional growth in teaching, the two purposes collide and, typically, neither goal is well accomplished. 99 So, there are two functions of peer review, as I see it. One is a kind of collaborative aspect where you\u00E2\u0080\u0099re trying to help your colleagues get better, solve problems. Let\u00E2\u0080\u0099s just work together, we\u00E2\u0080\u0099re on a team, we are trying to improve all of our teaching, so let\u00E2\u0080\u0099s pool resources to do that, share best practices, problem solved. . . . So there\u00E2\u0080\u0099s that aspect and there\u00E2\u0080\u0099s nothing judgmental about that: it\u00E2\u0080\u0099s process improvement. But then there\u00E2\u0080\u0099s the evaluation side, for marks, and now your peers are required pass judgment. And those two functions are in conflict with each other. (Palo, S1TF) Many faculty members are also vexed by the dual function of peer review. The frustration comes about, in part, because faculty members consider their peers to be both the best individuals to collaborate with to improve their teaching and the most suitable evaluators of their teaching. 4.5.1.2.2 The Evaluation of Teaching is Difficult Another reason quality feedback is not routinely provided relates to the numerous challenges involved with the evaluation of teaching, a topic repeatedly mentioned by the participants. Teaching is difficult to evaluate, the participants explained, because \u00E2\u0080\u00A2 there are few or no explicit criteria for doing so; \u00E2\u0080\u00A2 faculty members do not share a common notion of good teaching; \u00E2\u0080\u00A2 academics may be lacking in pedagogical knowledge and/or competency; and \u00E2\u0080\u00A2 peer reviews occur infrequently and are based on very little information. 100 4.5.1.2.2.1 Few (or no) Criteria Faculty members see nonspecific criteria as a challenge to the evaluation of teaching. Teaching, they say, is typically evaluated against an unstated standard. Those reviewed feel uncertain about what they are being evaluated on, while reviewers admit they do not necessarily know how to evaluate teaching. This sentiment is shared both by people in departments where a form with criteria is used (A3 and S1), as well as in departments where no such form exists. Several faculty members point out that identifying poor teaching may be easier than identifying the opposite. What we operate by, and this is regrettable, is that we will know when the teaching is not up to par. But we really don\u00E2\u0080\u0099t know when it is up to standard because I don\u00E2\u0080\u0099t think we have thought enough about it. (Ludar, A3TM) Perhaps this is why, in the opinion of some participants, the peer review of teaching serves only to assure colleagues that nothing \u00E2\u0080\u009Cegregiously awful\u00E2\u0080\u009D (Palo, S1TF) is going on. Thus, in the absence of standards, the peer review of teaching process aims not to celebrate good teaching, build upon strengths, or even improve weak teaching skills, but only to ensure there are no catastrophes. \u00E2\u0080\u009CI think most of the time, you\u00E2\u0080\u0099re in there to see, \u00E2\u0080\u0098Okay, no train wreck\u00E2\u0080\u0099\u00E2\u0080\u009D (Quinn, S1TM). Whereas, in the section above entitled \u00E2\u0080\u009CFocus on evaluation, \u00E2\u0080\u009D I reported that some faculty members suggest peer reviews act as a \u00E2\u0080\u009Cspot check,\u00E2\u0080\u009D to ensure things are \u00E2\u0080\u009Cokay,\u00E2\u0080\u009D some findings indicate that the aims of peer review are, indeed, not even that lofty. 101 4.5.1.2.2.2 Good Teaching: No Consistent Opinion Related to the previous point is the fact that there is no consistent opinion among faculty members about what constitutes good teaching. Consequently, faculty members may not be sure how to evaluate teaching that is markedly different from their own. Some describe having to review a colleague as \u00E2\u0080\u009Cawkward,\u00E2\u0080\u009D especially when that person is taking an approach unfamiliar to their own. \u00E2\u0080\u009CThere\u00E2\u0080\u0099s a reluctance to be overly critical if you see something that you wouldn\u00E2\u0080\u0099t do, because as a reviewer you wonder: \u00E2\u0080\u0098Does it work? Does it not work?\u00E2\u0080\u0099 It\u00E2\u0080\u0099s like, \u00E2\u0080\u0098I don\u00E2\u0080\u0099t know\u00E2\u0080\u0099\u00E2\u0080\u009D (Quinn, S1TM). 4.5.1.2.2.3 Lack of Pedagogical Expertise A few faculty members state that they lack the pedagogical expertise to suitably judge their colleagues\u00E2\u0080\u0099 teaching and, therefore, consider themselves unqualified for this task. \u00E2\u0080\u009CYeah, we\u00E2\u0080\u0099re asked to assess teaching, we just do the best we can and from our gut feeling. Well, you shouldn\u00E2\u0080\u0099t be assessing someone\u00E2\u0080\u0099s teaching for tenure from a gut feeling\u00E2\u0080\u009D (Moretti, S3TF). Hadzik (S3tM) is one such person who clearly believes that not having pedagogical expertise is an impediment to his ability to conduct a useful evaluation. As a reviewer, he feels he can only comment on whether students are paying attention and listening, but not on whether they are learning. A number of participants also remark that they were reviewed by people who are not skillful teachers and reason that someone who is not a good teacher cannot evaluate teaching well. They surmise that one of the reasons feedback on teaching remains focused on the \u00E2\u0080\u009Cmechanics\u00E2\u0080\u009D (e.g. neatness of writing, voice level, whether students are awake, etc.) is 102 because faculty members do not possess the pedagogical skills and knowledge to evaluate teaching well. When this is the case, conversations about teaching lack depth and do not address issues of significance. \u00E2\u0080\u009CAll these conversations are skirting teaching. . . . It\u00E2\u0080\u0099s like telling someone how to run a company by showing them the photocopier\u00E2\u0080\u009D (Hadzik, S3tM). From the perspective of people committed to improving teaching, feedback focused on the surface qualities of teaching does not adequately fulfill their need for professional growth. 4.5.1.2.2.4 Little to go on Another reason why evaluating teaching is difficult and quality feedback is not routinely provided is that reviewers make judgments about a faculty member\u00E2\u0080\u0099s teaching based on very limited information due in part to the infrequent nature of peer reviews. Unless faculty members are reviewed yearly and post-tenure, a faculty member may only undergo a peer review twice in his/her career: at reappointment and tenure. Furthermore, when peer reviews do occur, they often centre on a small number of classroom observations of teaching. A candidate\u00E2\u0080\u0099s teaching materials may be considered but these do not normally \u00E2\u0080\u009Cdo a lot of heavy lifting\u00E2\u0080\u009D (Chen, A3TM) when it comes to the evaluation of teaching. In fact, only two faculty members (Manfred, A2TM; Palo, S2TF) mentioned the benefit of reviewing these at all and their comments were specifically about the usefulness of the teaching philosophy statement as a way to learn about a faculty member\u00E2\u0080\u0099s innovative teaching practices and overall approaches to teaching. Reviewers in the study do not observe laboratories and tutorials, nor do they speak with the faculty members\u00E2\u0080\u0099 advisees or teaching assistants to garner information from a variety of sources about teaching. Combined, this means that 103 reviewers have little information to go on. Several faculty members recognize that, even though classroom teaching is an important activity, there are numerous other practices included under the category of \u00E2\u0080\u009Cteaching\u00E2\u0080\u009D that are not included in the review. However, only a small number of faculty members state that they would like peer review to include a broader range of teaching activities. 4.5.2 Student Evaluations of Teaching Set the Standards Participants unanimously agree that for individual faculty members as well as for administrators, student evaluations of teaching remain the most important piece of information for evaluating teaching, even though they perceive that the instrument is flawed and results are biased and not necessarily reliable. Consequently, peer reviews are conducted primarily to fulfill an administrative requirement and not with the intention of fostering professional growth in teaching. Colleagues may use a peer review to have a discussion about teaching, but this occurs infrequently. As I indicated earlier in this chapter, faculty members say they frequently use information from the reviews to help interpret and/or otherwise supplement the student evaluation of teaching scores. In the opinion of most participants, the student evaluations of teaching are taken \u00E2\u0080\u009Cvery seriously\u00E2\u0080\u009D (Hardy, A2tM) and remain the primary means by which faculty members gage their teaching success, or lack of it. According to the participants, the reasons the student evaluation of teaching results generally predominate over the peer reviews are threefold. Firstly, results from this instrument are presented as numerical scores and, since faculty members believe that administrators care 104 about numbers, they therefore believe that the results of this instrument are useful to those whose opinions count in tenure and promotion decisions. The numerical scores can also be applied to graphs and statistics to compare faculty members against one another and/or a departmental average. Numbers, several faculty members opine, are quick and easy to read and interpret as compared to the qualitative comments in summative peer review reports. In S1 and S2, a candidate\u00E2\u0080\u0099s tenure file contains graphs and statistical information that plots scores for each course a faculty member teaches against the same course taught by other people and/or other courses at the same level (i.e., 100-level, 200-level, etc.). The ability to compare oneself against a standard (peers, in this case) is referred to as a positive by faculty members in Science. Abendroth (S1tM) recalls being frustrated that he was not told, for the first four years of his appointment, that his student evaluation of teaching were below the department average, especially given that all that information was readily available and statistically compiled. As someone who was, at that time, approaching tenure, he recalls wanting to know where he stood. In S3, Moretti (TF), who has been part of the Dean\u00E2\u0080\u0099s advisory committee for promotion and tenure, speaks with enthusiasm about the plots and graphs in S1, indicating that it would be a useful practice to adopt faculty-wide as it is more objective than the current (non-desirable) subjective ways of presenting information about a candidate\u00E2\u0080\u0099s teaching scores. Striving for objectivity, in peer review or the evaluation of teaching more broadly, is fitting and beneficial according to many faculty members in Science. It is worth noting that in the Arts departments, the practice of comparing peers does not involve the use of detailed statistics or graphs. In A1, Cohen (aM) relays that the head 105 normally announces, annually at a department meeting, what the average student evaluation of teaching score is faculty-wide. Faculty members can then compare their scores to that average, if they want and in private. In A2, there had been an occasional practice of circulating a summary of scores ranked from highest to lowest. That, explained Nemeth (A2tF), \u00E2\u0080\u009Ctended to be badly received as divisive and humiliating for the people who are at the bottom and not particularly helpful,\u00E2\u0080\u009D and therefore is now done more informally, and in private, by the department head when a faculty member\u00E2\u0080\u0099s scores drop below a number deemed acceptable in that unit. In A3, faculty members receive an average score for the 100, 200, 300 and 400-level courses (i.e., the average of all the 400-level courses collectively that term, categorized by full-time faculty and sessional faculty) and they also receive the grade distribution for each course. \t\r \u00C2\u00A0 A second reason the student evaluations of teaching are considered more important than peer reviews is that most faculty members have a strong sense of what a good numerical score is within the context of their department and, consequently, self-assess using that knowledge. For instance, Fradera (A3tF) describes how receiving information about the departmental student evaluation of teaching scores triggered a sense of competition within her: There is a little bit of competition in the teaching evaluation scores. Every year we get a summary and you can see by that if you\u00E2\u0080\u0099re not up to snuff. The average is 4.5 or higher [on a 5 point scale] and there are a few people who get perfect 5s. Yeah, so there\u00E2\u0080\u0099s sort of intense feeling of peer competition [laughs]. I remember a few times I had 4.2 and I was just destroyed by that. And, some people teach really hard unpopular courses-there\u00E2\u0080\u0099s a bias against them getting high scores\u00E2\u0080\u0094and they get 106 scores of 3 point something. And it must be really hard on those people to get those scores. She goes on to surmise that, although there are no strong departmental initiatives \u00E2\u0080\u009Cto do an outstanding, innovative job of teaching,\u00E2\u0080\u009D the peer competition might motivate some faculty members to continuously work hard at their teaching. A small number of other women relayed that they had felt very anxious, even obsessed, when their scores had dropped below a number they considered acceptable. They drew on several resources, such as consulting with colleagues for help and modifying their course, in order to address the source of the problem. In these cases, the student evaluations of teaching scores were a mechanism for self-regulation (and self-inflicted badgering). Several faculty members cited a precise score (e.g., 4.2 or 3.5) above which nobody in the standing committee would be concerned about a colleague\u00E2\u0080\u0099s teaching. Little discussion of teaching ensues when a candidate has consistently high scores; in these cases, peers seem to agree on a \u00E2\u0080\u009Cslam dunk for teaching\u00E2\u0080\u009D (Wilks, S1tF). Good teaching thus requires little discussion in tenure and promotion decisions. Similarly, others speak about an acceptable score, or \u00E2\u0080\u009Cmagic number\u00E2\u0080\u009D (Ledonne, S1tF), as one that does not alert the \u00E2\u0080\u009CSWAT team\u00E2\u0080\u00A6[to] come out [and declare] \u00E2\u0080\u0098What are we going to do? This is going to be a problem for tenure\u00E2\u0080\u0099\u00E2\u0080\u009D (Ledonne, S2tF). Individual faculty members have definite ideas about what constitutes a good score within their department, and monitor their results and those of their colleagues\u00E2\u0080\u0099 accordingly. Knowing how one fares against peers matters to many faculty members but not all. In a department where scores are typically high, Chen (A3TM) figures faculty members probably do not care if they are a few points below or above the (high) average. Similarly, 107 Dodgson (S2aF) says she does not consider how she compares to peers as important. But these voices are the outliers in a majority who affirm the importance of student evaluation of teaching scores to assess how they are faring against the norm. A final reason the student evaluation of teaching scores are considered more important than the peer reviews is that the results are based on the opinion of students who have participated in the whole course, whereas a faculty member has typically seen only one teaching episode. 4.5.3 Artificial A third reason why faculty members do not associate the summative peer review with teaching professional growth is that they view it as an \u00E2\u0080\u009Cartificial\u00E2\u0080\u009D situation. Several mentioned that the presence of a reviewer renders the teaching situation unnatural for both the instructor and the students. For the instructor, it alters his/her level of nervousness, which thereby alters teaching. \u00E2\u0080\u009CThe very presence of someone watching you teach in some subtle way will affect how you teach\u00E2\u0080\u009D (Manfred, A2TM). A few faculty members whose respective reviewers had come to a small graduate class recall that students had been reluctant to participate at first, even though they had been informed in advance about the fact that a colleague would be present in the classroom. Having someone new in the room made both the students and the instructor more self-conscious. These participants suppose that the presence of the reviewer changes the dynamics that everyone has become accustomed to and, consequently, students are less comfortable contributing to the classroom discussions. Faculty members and others agree that having a reviewer attend an undergraduate class is 108 much less intrusive because a visitor is less likely to be noticed among a large number of students. 4.6 Ideal Process In this section, I draw from the interviews to report on features that would improve the peer review of teaching and/or constitute characteristics of an ideal process. I asked each participant what an ideal peer review of teaching process might entail and, not surprisingly, they had varied responses (see Appendix H for a summary of the range of participant responses). In retrospect, I realize that I should have been more specific in framing the question and also asked in what ways, if at all, an ideal summative and formative peer review of teaching might differ from one another. Even though I did not make the distinction between summative and formative peer review in posing the question, a large number of faculty members pointed out that, ideally, the summative peer review of teaching would not be used for both developmental and evaluative purposes because these goals conflict. When one attempts to use summative peer review to achieve both goals, neither one is well achieved; consequently, affirmed many of the participants, the two goals need to be kept separate if peer review is to serve either function well. \u00E2\u0080\u009CIf you\u00E2\u0080\u0099re going to have a reflection on the possibility of growth as a teacher, it has to be part of some other process than tenure and reappointment\u00E2\u0080\u009D (Nemeth, A2tF). 4.6.1 Flexibility/Structure Many participants note the absence of \u00E2\u0080\u009Crules\u00E2\u0080\u009D in peer review. Some celebrate this absence saying that faculty members would perceive strict requirements as an imposition and would 109 therefore resist them. Other faculty members interpret the absence of regulations as something status quo and a regular part of academia. Trottier (A2tM) speaks of his ignorance of the rules in departmental peer review as follows: But I may not be aware of some rules because many of these rules actually are not written [laughs]. At some point, someone comes and says \u00E2\u0080\u009Cas part of this department we do such and such.\u00E2\u0080\u009D . . . [and so] we do what the procedure requires us to do. Some faculty members readily accept the flexible approach their department takes to summative peer reviews. They appreciate the freedom they have to conduct the process as they see best and do not want an \u00E2\u0080\u009Cacross the board\u00E2\u0080\u009D (Stromberg, A1TM) policy or instrument imposed upon them. Peer reviews, some faculty members recognize, are inherently subjective. Others would favour more consistency in the peer review process and believe this could happen if the process were more objective. Faculty members indicate that this could be brought about in a number of ways, the first of which is by articulating the departmental criteria for what constitutes good teaching. The criteria would need to be communicated to the person being reviewed and reviewers so that both parties had clear expectations of what they would be evaluated on and evaluating, respectively. In departments where an instrument with criteria is not routinely used, some faculty members propose that using one could be helpful. They suggest that peers collaboratively develop the instrument based upon common notions of good teaching and that it then be shared with candidates well ahead of time. A second and related way by which more consistency could be brought to a departmental peer review process would be to \u00E2\u0080\u009Cshare the ground rules for evaluation\u00E2\u0080\u009D (Wilks, S1tF). The ground rules here refer, not only to the specific criteria for evaluating good teaching, but to any other guidelines and instructions such as: (a) at what level of detail will reviewers 110 examine the various criteria?, (b) What can those reviewed expect in terms of feedback?, and (c) How will the review factor into tenure and promotion decisions? A third way that the summative peer review of teaching process might be made more consistent and objective is if it were conducted by a faculty-wide team of trained reviewers, proposes Moretti (S3TF). These individuals would be called upon to carry out peer reviews for the entire faculty. This would help remove bias and bring objectivity to a process, which according to some, is currently too subjective. 4.6.2 Ongoing, Informal Process Many faculty members suggest that it would be ideal for classroom observations of teaching to be done regularly, for everyone, and not just in critical decision periods. When peer reviews are conducted informally\u00E2\u0080\u0094that is, outside of tenure and promotion decisions\u00E2\u0080\u0094 reviewers feel more free to give constructive feedback on teaching because there is a lower risk that doing so can have negative repercussions on a faculty member\u00E2\u0080\u0099s career. An ongoing, informal process is more conducive to professional growth in teaching. Participants mentioned several ways in which informal peer reviews might be carried out. For example, there may be a departmental agreement whereby faculty members peer review one another annually and all that they are held accountable to is sharing feedback and then notifying the department head; no documents would submitted as part of this process. Alternatively, three or four colleagues, from the same department or different ones, might observe one another\u00E2\u0080\u0099s classes and meet once or twice a year to discuss these over lunch. In a third type of formative peer review, pre-tenure faculty members could review one another as 111 this would assure that the focus remain on teaching professional growth and be completely separate from career advancement in the formal sense of the word. Finally, staff from the University-wide teaching and learning centre could be invited to be a part of the informal process. Any of these approaches, or others not mentioned, promotes peers to learn from observing one another\u00E2\u0080\u0099s classroom teaching. Although many faculty members agree that routine formative peer reviews would be a good thing, some acknowledge it may not occur unless departmental peers collectively agree this practice should become a departmental policy or a university requirement. One reason is that informal peer reviews place additional demands on faculty members\u00E2\u0080\u0099 time. Even though they recognize that peer reviews would be desirable for promoting teaching professional growth and that they would enjoy this type of learning, several participants believe the investment of time would not be worthwhile. Peer observation is a time consuming process that takes valuable time away from other activities. Everyone at this university is overstretched and asked to do too much. And I suspect that while on some level you would get more accurate data if arrangements could be made to sit in on ten lectures instead of two, the amount of time that that would consume and the amount of organizational effort and inconvenience involved would not make it worth it. (Manfred, A2TM) Another reason informal peer reviews might not occur unless they were mandated is that, in the current academic culture, the classroom is frequently perceived as an instructor\u00E2\u0080\u0099s private space and some faculty members, say Trottier (A2tM) and Parsons (A3TF), are reluctant to ask their peers to be invited into that space. \u00E2\u0080\u009CThere\u00E2\u0080\u0099s a certain reluctance in academic culture. 112 . . . If I go to my colleague and ask for permission \u00E2\u0080\u0098May I sit in?\u00E2\u0080\u0099, I don\u00E2\u0080\u0099t expect a \u00E2\u0080\u0098No.\u00E2\u0080\u0099 Certainly the usual answer would be \u00E2\u0080\u0098yes,\u00E2\u0080\u0099 but I think there would still be some feeling of discomfort.\u00E2\u0080\u009D Even, as several faculty members indicated, if they are certain their colleagues would accept a peer in their classroom, they hesitate to do so knowing that it would create uneasiness for both the reviewer and the reviewed. 4.6.3 Feedback Quality and Sharing Whether they are observed regularly or infrequently, for summative or formative purposes, faculty members want to engage in conversations about the review with their reviewers. They want follow up to be an explicit part of the process, \u00E2\u0080\u009Crather than something that just happens if they [the reviewers] happen to be conscientious and have time\u00E2\u0080\u009D (Ledonne, S1tF). According to the participants, debriefing the peer review experience, preferably shortly after it has occurred when things are still fresh in the minds of the reviewer and candidate, has three distinct advantages: (a) it lets the person reviewed, who might be nervous about the reviewer\u00E2\u0080\u0099s opinion, know the latter\u00E2\u0080\u0099s impressions; (b) it augments the possibility that the review contributes to enhanced teaching; and, (c) it provides candidates with an opportunity to ask their reviewers questions prior to meeting with their department head. Those who have had conversations with their reviewers say these are most rewarding and conducive to professional growth when both parties approach the peer review as a learning opportunity. When this is the case, the follow up conversations are carried out in a spirit of reciprocity: those reviewed reflect on their teaching and reviewers share what they learned from the experience. The persons being reviewed prefer feedback that is specific, but that can 113 also apply to teaching situations outside the very classroom that was observed. They also want to have critical feedback, from a knowledgeable reviewer, as they believe this will contribute most to their professional growth. In the following quote, Dawson (S3aM) explains why, for him, this is so important: \u00E2\u0080\u00A6because at the end of the day hearing that this looks great is fulfilling, but it doesn\u00E2\u0080\u0099t help you. You\u00E2\u0080\u0099re only helped if you know what it is that you\u00E2\u0080\u0099re not doing right. It\u00E2\u0080\u0099s just like my students. The most important thing for them to know is what they don\u00E2\u0080\u0099t know. Yeah, it\u00E2\u0080\u0099s great to have a pat on the back, but I think learning requires that you confront the things that you don\u00E2\u0080\u0099t understand, or the things that you don\u00E2\u0080\u0099t do well. Dawson, who had completely redesigned one of his courses, goes on to describe how, even though he received some helpful comments from his reviewers, he did not feel that most of them had sufficient knowledge or experience to critically comment on the non-traditional approach to teaching he had adopted and, consequently, he was not able to learn \u00E2\u0080\u009Cwhat is the most important thing I\u00E2\u0080\u0099m not doing.\u00E2\u0080\u009D Although knowing what one does well is also useful, those being reviewed say they appreciate receiving detailed feedback with suggestions for improvement. This aligns well with what some reviewers say they want, which is to provide feedback that is encouraging and constructive. 4.6.4 Reviewer-Reviewed Connection Faculty members have varying opinions about what constitutes an ideal reviewer and comment on qualities that included disciplinary connection, pedagogical knowledge, philosophies of teaching, gender, and rank. 114 4.6.4.1 Disciplinary Connection Faculty members agree that both someone from outside their field as well as a colleague with whom they share a disciplinary connection can contribute meaningfully to peer review. An outsider can comment on many aspects of teaching, such as whether the students are engaged, or not, or whether an instructor is paying more attention to some students than to others. A content expert, however, can better assess whether \u00E2\u0080\u009Cconcepts are being passed effectively\u00E2\u0080\u009D (Knauer, S2TF). Whereas some say that the value of a peer review is \u00E2\u0080\u009Czero\u00E2\u0080\u009D (Dodgson, S2aF) when their reviewer has no content knowledge, others think it is only important in evaluating a graduate course. Ludar (A3TM) agrees with people who think that there should be some effort to have someone with content expertise and says: \u00E2\u0080\u009CI think it\u00E2\u0080\u0099s preferable to have a person with some deeper competence in the field and it\u00E2\u0080\u0099s also comforting to the professor who increasingly is pretty specialized.\u00E2\u0080\u009D A number of participants believe it would impose an undue burden on those coordinating peer reviews if they had to take into account matching people with a close disciplinary connection. Wilks (S1tF) raises another point about the reviewer-reviewed connection when she observes that the two people should not be too \u00E2\u0080\u009Cintertwined.\u00E2\u0080\u009D In other words, she is not in favour of having her evaluator be someone with whom she collaborates on teaching because that would remove the possibility of an objective review. Numerous Science professors named objectivity as a value to strive for in peer reviews. Related to the goal of objectivity, Hadzik (S3tM) suggests that people from other departments would be ideal reviewers because, 115 lacking content knowledge, they could be \u00E2\u0080\u009Creal learners\u00E2\u0080\u009D who could actually comment on what they had learned in the class.14 Certain faculty members propose that peer reviews could be more objective if one were to adopt the following practices to evaluate teaching, some of which were previously mentioned: select expert-educators who can base their judgments on the literature and sound research, use a faculty-wide team of trained reviewers to avoid bias, measure student learning to determine instructor effectiveness, and use pre- established criteria and standards that are known to both the reviewer and person reviewed. 4.6.4.2 Pedagogical Knowledge Several faculty members were explicit about wanting to be reviewed by someone with pedagogical knowledge and mention, for instance, that they value the pedagogical expertise and perspectives of tenure track instructors in their department. Hadzik (S3tM) uses the term \u00E2\u0080\u009Cexpert educator\u00E2\u0080\u009D to describe his ideal reviewer. The expert researches teaching and learning and can back up her/his critiques and suggestions with data and literature that is \u00E2\u0080\u009Cobjective.\u00E2\u0080\u009D The ability to do so \u00E2\u0080\u009Ccounts for a scientist,\u00E2\u0080\u009D he points out. Others state their preference in terms of having a reviewer who has a strong reputation as a teacher and/or is a competent teacher but do not specify on what basis this reputation and/or competency is established. . 14 In Hadzik\u00E2\u0080\u0099s opinion, the evaluation of teaching would ideally focus on measuring student learning. Since a reviewer attending a single classroom observation of teaching cannot measure student learning, Hadzik proposes that someone who is unfamiliar with the course content can more aptly address the extent to which they, themselves, learned something from the class than someone who knows the content. 116 4.6.4.3 Philosophies of Teaching Some believe that the reviewer and reviewed should share compatible philosophies of teaching. This would allow reviewers to comment knowledgeably about the candidate\u00E2\u0080\u0099s approach to teaching and would be conducive to meaningful follow up. In the context of summative peer review of teaching, matching people based on similar philosophies is of critical importance and, as Nemeth explains (A2tF), not doing so can have a negative consequence: The difficulty of any kind of review process is that a huge amount depends on how compatible the two peoples\u00E2\u0080\u0099 teaching philosophies are. There are people who I might think of as very good teachers who other people in the department might think are doing something that\u00E2\u0080\u0099s really quite inappropriate. So the decision about who does the peer review is absolutely critical in terms of the kind of report that you\u00E2\u0080\u0099re likely to get. Sharing a philosophy of teaching can promote engaged discussion between the reviewer and the person reviewed; the absence of a common philosophy may, on the other hand, lead to a negative evaluation of teaching. 4.6.4.4 Gender When asked whether they thought that gender plays into the peer review of teaching experience or outcomes, the majority of participants said it does not, but some had opinions that contradicted this overarching belief. Had I not posed the question, I intuit that no faculty member would have raised the point of gender on his/her own. 117 According to male faculty members in Science, reviewers do not make a distinction based on gender when they conduct peer reviews; many qualified that statement with a recognition that questions of gender and peer review are tricky to respond to since they had had little experience reviewing women given the small ratio of females to males in Science. Three male faculty members in Arts (Bulmer A3TM; Ludar A3TM; Manfred A2TM) acknowledge that questions of gender are omnipresent and presume that their colleagues, like themselves, are sensitive to issues of diversity and gender when doing peer reviews. Only one male, Stromberg (A1TM), outright says that gender can influence how an evaluator judges a classroom teaching situation. He explains that, in the male-dominated discipline to which he belongs, men expect any individual whose arguments are being challenged to fervently defend themselves. He suspects that a male colleague might poorly judge a female whose behaviour did not meet that norm. Two women (Hannah, S2t; Parsons, A3T) are also emphatic about the role of gender in peer reviews: they reason that, because all social relationships are gendered, gender must \u00E2\u0080\u009Cin very subtle ways\u00E2\u0080\u009D (Parsons, A3TF) influence what happens in peer review. Both these women refer to knowing that male students evaluate their female and male instructors differently, and do not think peer review of teaching can be immune to whatever is the cause of this phenomenon.15 A few other women recognize that gender may play into peer reviews especially if the female instructor were teaching a class where gender figured prominently, such as a class on female prostitution or feminist 15 Many faculty members say that, although they do not think gender affects peer reviews in any way, it definitely plays out in the classroom and in interactions with students. They provided accounts of women having to be more intentional about establishing their authority in the classroom and stories of challenging students \u00E2\u0080\u0098targeting\u00E2\u0080\u0099 women more than men. 118 organizations. Many faculty members believe that personality plays into peer review much more than gender. 4.6.4.5 Rank Few people comment on the preferred rank of their reviewer, which might be because they have come to expect a reviewer to be of a higher rank. As I wrote previously, in all departments except S1 and A2, only faculty members of a higher rank than the candidate conduct reviews. It is noteworthy that in A2 the department head has, in the past few years, implemented an initiative whereby peers at the same rank\u00E2\u0080\u0094that is, people who will not be voting on the case\u00E2\u0080\u0094can conduct summative reviews. This innovation was implemented, Manfred (A2TM) reckons, for both practical reasons (being able to draw upon a larger number of people helps to coordinate dates and times for reviews) and also to make the process less hierarchical and, therefore, perhaps more comfortable for the person being evaluated. Faculty members are content to be reviewed by tenure track instructors and research professors. 4.6.5 More, Sooner A small number of faculty members would like a greater variety of teaching practices to be included in the summative peer review of teaching. In addition to the observation of an undergraduate lecture or graduate seminar, and reviewing course materials, they name graduate supervision, students\u00E2\u0080\u0099 perspectives, ongoing revisions people make to their courses, problem-based-learning and other group work, teaching assistant training, and tutorials as 119 potentially rich sources of data for peer review. At present, these activities are not incorporated into the evaluation of teaching. Several Science professors and one Arts professor say that an ideal peer review process would assess the extent to which a faculty member\u00E2\u0080\u0099s teaching facilitates student learning. \u00E2\u0080\u009CYou should be evaluated on whether your instruction works rather than the particular mechanism you use to deliver the material\u00E2\u0080\u009D (Palo, S1TF). Measuring student learning, however, is not something faculty members traditionally strive to do in a peer review. Nor was it something academics knew how to do. Instead, reviewers resorted to evaluating aspects of teaching like clarity of syllabus, content, organization, and learning objectives. As it pertains explicitly to summative peer review, it was suggested that reviews take place early in a faculty member\u00E2\u0080\u0099s career so that people could have the chance to detect\u00E2\u0080\u0094and presumably address\u00E2\u0080\u0094problems well in advance of the tenure decision. Several participants also recommend that peer reviews take place more regularly in the years leading up to tenure so that candidates could be presented with multiple opportunities for feedback. A small number suggest they would like to have the same reviewer over time so that person could comment on their progress and development over the span of several years. Some faculty members would prefer reviews to occur earlier in the term as this would allow instructors to alter their teaching, provided that feedback from reviews were promptly shared with the person reviewed. A number of candidates say they appreciate having input into the selection of the reviewers and the specific class that will be observed. 120 4.7 Summary of Findings Participants\u00E2\u0080\u0099 experiences and understandings of the purposes of summative peer review of teaching are influenced by institutional culture, namely the demanding nature of academic careers and a reward system that favours research productivity over teaching. Given the existing institutional reward system, as well as limited time and resources, faculty members must make strategic choices about where to expend effort, and the emphasis is usually on research. At the individual and departmental level, participants vary in their understandings of the summative peer review of teaching and have had diverse personal experiences of the process. Because of this people cannot speak about peer review as a uniform process. Faculty members understand that summative peer review is a formal means of evaluating teaching and is tied to career advancement; however, they differ in their opinions about the strength of that connection. According to some, the connection is weak, a loose knot with fraying string, as it were. For these faculty members, peer review does little except help detect a \u00E2\u0080\u009Ctrain wreck\u00E2\u0080\u009D and/or take time away from other tasks. Other faculty members consider the process, and information derived from it, as crucial and important to the formal evaluation of teaching. Those who maintain there is a weak link between peer reviews and tenure decisions point out that when it comes to the evaluation of teaching, administrators and other decision- makers rely heavily on student evaluations of teaching. There is a great deal of uncertainty embedded in the peer review of teaching process. Faculty members have questions that range from \u00E2\u0080\u0098Which standards and criteria are used in the 121 evaluations?\u00E2\u0080\u0099 to \u00E2\u0080\u0098Who is a suitable reviewer?\u00E2\u0080\u0099 Even some fundamental practicalities of the process (i.e., frequency of peer review; whether they will obtain feedback, and in what format and from whom; and whether tenured faculty members are peer-reviewed on their teaching) were unknown to them. Not only are faculty members unsure about the extent to which summative peer review counts toward tenure and promotion, but they are also uncertain which \u00E2\u0080\u009Crules\u00E2\u0080\u009D guide the process. For faculty members in Science, the absence of formal guidelines and criteria for peer review of teaching seems to create more unease than for their counterparts in Arts. Despite an official University document on peer review, only a few participants mention its existence, and none refer to it in order to justify and explain the ways peer reviews are carried out in their departments. Even in A3, where there is a departmental policy on summative peer review, not all faculty members were familiar with it, and its guidelines are interpreted in various ways by different individuals. Lacking information from their department head or others about the peer review, most faculty members develop their own personal expectations about how the process should unfold. Many of the faculty members agree that incorporating feedback into the peer review process would be a desirable way to learn and develop as a teacher. Those who share this opinion expressed frustration and disappointment at the lack of feedback they received when they were peer-reviewed for tenure and/or promotion. This in spite of the fact that a large number of participants also felt that when summative peer review of teaching was used to address multiple goals such as formal evaluation and fostering professional growth in teaching, 122 neither was well achieved. This was one of the main contradictions in the findings and something I have come to think of as the \u00E2\u0080\u009Cparadox\u00E2\u0080\u009D of summative peer review: on one hand, many faculty members claim that the formative and evaluative functions of peer review should be kept separate, because, when combined, neither can be well achieved and, on the other hand, many express disappointment that the peer review process does not contribute significantly to their growth as teachers. Thus, even though faculty members acknowledge the (primarily) evaluative purpose of summative peer review, they are keen to learn about their teaching through the process.\t\r \u00C2\u00A0\t\r \u00C2\u00A0 Ultimately the design of any peer review process hinges upon its purpose. When that purpose is not clear, those involved in peer review, whether they be the individual reviewer or candidate, committee members evaluating a tenure application, or university administrators\u00E2\u0080\u0094employ a personal sense of what is best. When this is the case, faculty members\u00E2\u0080\u0099 experiences of summative peer review seem to vary, and be inconsistent within and across departments. In the ensuing discussion, I use the literature to conduct a final analysis of the major findings pertaining to tenure track professors\u00E2\u0080\u0099 experiences of summative peer review. In doing so, I provide additional background that will support the conclusions and recommendations from this study (see Chapter 6). Although I reported on institutional culture in this chapter\u00E2\u0080\u0099s findings, I shall address it in the discussion in Chapter 5 because the reward system and the demanding nature of academic careers influences the summative peer review, departmental 123 culture, and professional growth in teaching in overlapping ways that are best understood when one acknowledges that they are nested within a broader institutional context. 4.8 Discussion: Summative Peer Review and Professional Growth in Teaching So far, I have described variations and inconsistencies in faculty members\u00E2\u0080\u0099 experiences and understandings of summative peer review. Despite this variability, two predictable aspects of summative peer review emerge. Firstly, for the most part it does not, according to faculty members, promote professional growth in teaching. This confirms research on peer review that ties the formative process to instructor\u00E2\u0080\u0099s self-development through reflection and collective dialogue about teaching, and ties the summative process to institutional issues of accountability (Byrne et al., 2010; Peel, 2005). Secondly, peer reviews are not consistently used to inform tenure and promotion decisions. If, according to the literature, professional growth and evaluation are the primary reasons for conducting peer review, why are these purposes not being fulfilled? I propose that the summative peer review of teaching does not fulfill its objectives because the process violates, threatens, or otherwise disrupts firmly established academic values and norms and/or other practices upheld in the large, research-intensive North American university. Consequently, faculty members\u00E2\u0080\u0099 resistance to peer review of teaching remains strong, while their engagement in the process continues to be weak. 124 4.8.1 Unsettling/Honouring Collegiality A primary reason for lack of engagement in summative peer review is the ways in which this practice can unsettle and even jeopardize collegial relationships. The summative peer review of teaching involves complicated power relationships. The dynamics at play between individuals (and/or processes/structures) who (that) exercise power and those with less power can upset collegiality. 4.8.1.1 Feedback Unease First, a summative peer review can feel threatening to a reviewer and/or candidate because it requires one or more peers to evaluate a colleague in ways that inform high-stakes career decisions. As discussed in Chapter 2, academics feel uncomfortable about giving and/or receiving negative feedback in peer-review situations (Hammersley-Fletcher & Orsmond, 2004; Kell & Annetts, 2009); this has been tied to an academic disinterest in engaging in teaching-related conversations that could lead to fights and disagreements (Massy et al., 1994; Rox\u00C3\u00A5 & M\u00C3\u00A5rtensson, 2009). This desire to avoid friction with colleagues may partly account for why reviewers do not share feedback with the candidate. That is, even though the post-review conversations have the potential to contribute to the development of teaching by promoting reflection and expanding an understanding of teaching (Peel, 2005; Pratt, 1997), faculty members may judge that doing so poses an unnecessary risk to collegial relationships. Instead, academics may rationalize that peer reviews should be performed at \u00E2\u0080\u009Carm\u00E2\u0080\u0099s length\u00E2\u0080\u009D (Woodroof, S2TM) 125 and like external reviewers\u00E2\u0080\u0099 reports, should not be shared with the candidate. When this approach is taken, faculty members can miss an opportunity to support another\u00E2\u0080\u0099s professional growth, but run less chance of upsetting collegial relationships. \t\r \u00C2\u00A0 Summative peer reviews of teaching therefore present contrasting opportunities for collegiality. From one point of view, one might expect that in a collegial culture where colleagues presumably behave in ways that support each other\u00E2\u0080\u0099s \u00E2\u0080\u009Cwell-being\u00E2\u0080\u009D (to use a definition of collegiality borrowed from Gappa et al., 2007), peer reviews would be considered a valued opportunity to learn through observation, reflection, and discussion. From another perspective, peer reviews that do not include feedback may reinforce the notions of mutual respect, per Gappa et al.\u00E2\u0080\u0099s definition of collegiality: when peers respect each other\u00E2\u0080\u0099s autonomy in teaching and withhold critical feedback, they minimize the risk of creating friction with colleagues and thereby preserve and/or honour collegiality. 4.8.1.2 The Primacy of Tenure A related reason why peer reviews may threaten collegial relationships involves the primacy of tenure in faculty careers. The value placed on tenure inhibits faculty members from recording suggestions for constructive feedback in the summative process. As reported in the findings, though a few participants claimed they endeavoured to present a \u00E2\u0080\u009Cbalanced\u00E2\u0080\u009D perspective in their peer review reports, the majority said they did not include comments that might be misconstrued in decisions about career advancement. The literature indicates that tenure is highly valued in North American research-intensive universities because it confers job security, power, and prestige (Chait, 2002; Clotfelter, 2002). However, 126 it also indicates that many academics also strongly value learning and growth in their teaching (Gappa et al., 2007; Neumann, 2009; O\u00E2\u0080\u0099Meara et al., 2008). This means that faculty members are squeezed between conflicting values. On one hand, they cherish tenure and promotion while understanding that documenting feedback in official reports might deter a positive review. On the other hand, many are intrinsically motivated to grow as teachers. So, even though faculty members understand why feedback is not a regular part of the summative peer review process, they are bothered by that absence and, as the findings indicate, would appreciate their colleagues\u00E2\u0080\u0099 perspectives and insights about their teaching. This might help explain what I earlier call the paradox of peer review. This conflict arises in part because from a professional growth perspective, sharing constructive feedback would be desirable, but from a tenure/promotion perspective, the same feedback is considered potentially harmful to colleagues. \t\r \u00C2\u00A0 4.8.1.3 Teaching as Private Summative peer reviews of teaching can unsettle collegial relationships because of the public nature of the classroom observations of teaching; these disrupt the norm of teaching as a private activity16. Teaching in higher educations has often been described as a private endeavour, one that is shared between only an instructor and his/her students (Chism, 2007a; Hutchings, 1996). Unaccustomed to being in each other\u00E2\u0080\u0099s classrooms, with no formal 16 My understanding and use of the words \u00E2\u0080\u009Cprivate\u00E2\u0080\u009D and \u00E2\u0080\u009Cpublic\u00E2\u0080\u009D have been shaped by the work of Pat Huchings (1996) and other scholars (e.g. Daniel Bernstein, Lee Shulman) who, in promoting the scholarship of teaching and learning, have underscored that\u00E2\u0080\u0094outside of the interactions a teacher has with his/her students\u00E2\u0080\u0094the various activities that comprise teaching (planning, assessing, advising, reflecting, and so on) are conducted by the teacher, privately and without the benefit of ongoing collaboration or dialogue with colleagues. 127 guidance on how to proceed, several faculty members described the peer reviews of teaching as \u00E2\u0080\u009Cartificial\u00E2\u0080\u009D and found them \u00E2\u0080\u009Cnervous-making\u00E2\u0080\u009D (Deitz, S1aF). Reviewers and reviewed alike experienced discomfort with the process. The act of peer review challenges the instructor\u00E2\u0080\u0099s role as an autonomous member of the academy (Chism, 2007a; Lomas & Nicholls, 2005, Swinglehurt et al., 2008). In peer reviews, the faculty member is observed by a colleague in a space where peers traditionally do not interfere. The observation experience is made more intense when tied to high-stakes decisions concerning tenure, promotion, or reappointment. Consequently, the faculty member may feel his/her autonomy, already diminished by the presence of a colleague, could be further jeopardized by the outcome of the peer review process. Even though faculty members commonly engage in peer review to confer judgments on conference papers, presentations, journals, and grant proposals, (Biagioli, 2002; Hamilton, 2007), the public nature of summative reviews, where a peer is present in the classroom and colleagues can read the summative report, runs contrary to anonymous, blind reviews that are the norm in evaluating conference proposals, grants, and other types of scholarly publications. 4.8.2 \t\r \u00C2\u00A0Universalism (or We Should Not be Evaluating from a Gut Feeling) Resistance to summative peer review may be compounded by the lack of explicit criteria for evaluating teaching, which may disturb the norm of universalism. The norm of universalism involves judging merit based on impersonal and universal criteria rather than such criteria as gender, personality traits, and nationality (Mitroff, 1974; O\u00E2\u0080\u0099Meara, 2011). Universalism presumes the existence of criteria and stipulates the nature of these. If criteria are important to engagement, but faculty members do not believe that criteria for good teaching exist or are 128 unfamiliar with them, academics may experience \u00E2\u0080\u009Chostile resistance\u00E2\u0080\u009D at worst toward peer reviews or, at best, feel a \u00E2\u0080\u009Cgrudging acceptance\u00E2\u0080\u009D (Arreola, 2007).17 Moretti (S3TF), for example, deplores how the evaluation of teaching typically occurs from a \u00E2\u0080\u009Cgut feeling.\u00E2\u0080\u009D Likewise, Quinn (S1TM) recalls the discomfort he experienced when evaluating a peer because he could not confidently ascertain the suitability of his colleague\u00E2\u0080\u0099s approach. These examples are representative of discomfort related to a lack of specific criteria for evaluating the teaching of one\u00E2\u0080\u0099s colleagues. Another way of thinking about this is that some faculty members find it disturbing to evaluate colleagues\u00E2\u0080\u0099 teaching without established criteria because doing so goes against the norm of academic honesty that is fundamental to academic profession (Clark, 1987). Academic honesty, which refers to the \u00E2\u0080\u009Chonest handling of knowledge . . . and fair treatment of colleagues and students, using established universal criteria rather than particularistic judgment\u00E2\u0080\u009D (Clark, 1987, p.133), is similar to the norm of universalism. The subjectivity inherent in the peer review process and its vague, presumably nonexistent criteria can make faculty members feel they are not upholding the value of academic honesty (or not behaving according to the norm of universalism). In this study, Science professors spoke more often than their Arts counterparts about wanting 17 Arreola (2007) describes how faculty members\u00E2\u0080\u0099 resistance to an integrated faculty evaluation and development program undergoes five stages of resistance starting at \u00E2\u0080\u009Cdisdainful denial\u00E2\u0080\u009D in Stage 1, moving through \u00E2\u0080\u009Chostile resistance,\u00E2\u0080\u009D \u00E2\u0080\u009Capparent acquiescence,\u00E2\u0080\u009D \u00E2\u0080\u009Cattempt to scuttle\u00E2\u0080\u009D (in Stages 2, 3, and 4 respectively) and terminating at \u00E2\u0080\u009Cgrudging acceptance\u00E2\u0080\u009D in Stage 5. His description is useful for thinking about faculty members\u00E2\u0080\u0099 resistance to new teaching initiatives. 129 an objective peer review process\u00E2\u0080\u0094one with more clearly defined criteria and standards. This is not surprising given that research on disciplinary cultures has found that individuals in hard disciplines are entrenched in a strong tradition of structure, quantification, and coherence (Becher, 1994; Neumann et al., 2002). Consequently, the absence of criteria and the individualistic manner in which peer reviews are conducted were less familiar and therefore potentially more uncomfortable for faculty members in Science. For faculty members in soft-pure disciplines, where traditions are concerned with discovery through interpretation and qualitative approaches are used more often (Becher, 1994; Umbach, 2007), a peer review process that entails \u00E2\u0080\u0098no rules\u00E2\u0080\u0099 is more in line with their disciplinary norms and traditions. 4.8.3 Student Evaluation of Teaching Reigns Another reason faculty members did not engage in peer reviews of teaching is the importance placed upon student evaluations of teaching. As described in Chapter 4, faculty members felt department heads and colleagues placed more weight on the student evaluation of teaching scores than on peer review results. In annual review meetings with the department heads, peer reviews were rarely discussed, especially when the student evaluation of teaching scores were acceptable or strong, as determined by a department norm. Likewise, in standing committee meetings, when teaching evaluation scores were high (or acceptable), voting colleagues directed their attention to other aspects of a colleague\u00E2\u0080\u0099s tenure file. As a result, the message about teaching is it will be positively evaluated if student evaluation scores meet or exceed the department norm. That is, teaching was presumed to be good (or good enough) as long as scores were acceptable. As a result, faculty members were uncertain about the extent 130 to which a peer review of teaching influenced tenure and promotion decisions, if at all. The importance placed upon student evaluations of teaching is unsurprising, given that these instruments have a long history in North American higher education. They were introduced in the late 1960s in response to student demands for public accountability and by the 1980s were established as a \u00E2\u0080\u009Cnormative-quality measure for administrators\u00E2\u0080\u009D (Ory, 2000, p. 14). To date, they have become the predominant means of evaluating teaching for tenure and promotion (Emery, Kramer, & Tian, 2003).18 The regular use of peer reviews of teaching is, in comparison, relatively recent. Thus, one might justify a faculty reliance on student evaluations of teaching by pointing to the strong role that tradition and history play in restricting change in some higher education institutions (Kezar & Eckel, 2002). Tradition aside, the reliance on student evaluations of teaching was somewhat perplexing, given that participants described the instrument as flawed and appreciated the alternative and useful perspective provided by peer reviews. The validity of the student evaluations of teaching has been widely questioned because scores can reflect student satisfaction more than learning outcomes (Ackerman et al., 2009; Emery et al., 2003). In addition, student evaluations have been identified as a source of faculty members\u00E2\u0080\u0099 anxiety. Poor results can be disappointing and stressful, especially when academics have expended sincere instructional 18 Within the context of UWC, official policy documents (e.g., the collective agreement or related guidance document used to assist personnel committees or those involved in the decision-making process) do not place more emphasis on the student evaluations of teaching over peer reviews. However, there exists a senate-approved policy on the student evaluations whereas the same is not true for peer reviews. Furthermore, prior to the most recent 2007 Senate Policy on Student Evaluations, there were policies dating back to 1978; this may be taken as evidence that the student evaluations of teaching, as compared to peer reviews, have a longer history. 131 efforts, time, and energy to develop courses (Hodges & Stanton, 2006). Trout (2000) proposes that instructors accept student evaluations of teaching in part because the practice is so widespread (and therefore perceived as \u00E2\u0080\u009Clegitimate\u00E2\u0080\u009D), because they know little about evaluation and its effect on their careers, and because they consider it fair that students be able to evaluate their instructors since the opposite also occurs. I would suggest an additional reason why academics support the use of student evaluations: they value students\u00E2\u0080\u0099 perspective on their teaching because these perspectives can help them improve their courses and modify their teaching. Students\u00E2\u0080\u0099 qualitative comments provide faculty insight into their students\u00E2\u0080\u0099 intellectual development. These insights can contribute to scholarly approaches to teaching and help faculty members make informed choices about their teaching practice (Hodges & Stanton, 2006). The findings in this study indicate that student evaluations of teaching can also promote competition between faculty members. Motivated to reach or exceed a departmental norm, faculty members may work to modify their teaching and augment their scores. Whereas it has been found that evaluations of teaching normally provide inadequate motivation for faculty members to improve their teaching (Seldin, 1999), those in this study indicated that evaluations triggered a sense of competition that motivated them to improve. 4.8.4 Slow Change in Academia A final finding I wish to analyze relates to the disjuncture between the existence of an official UWC report on summative peer review and the perception that persists among faculty members that there is a lack of guidelines in the peer review of teaching process. 132 The perception about a lack of guidelines may be attributed to an absence of a departmental policy on peer review; with the exception of A3, none of the departments in this study had such a policy. However, at the time of interviews, there was an official university-level report that outlines \u00E2\u0080\u009Cprinciples to guide the development of unit-specific procedures and practices of peer review\u00E2\u0080\u009D and includes implementation guidelines that can be adapted to the specific department and/or faculty context. That report was created at the request of the Office of the Provost and Vice President Academic and circulated in November 2009. By spring 2010, when I began interviewing, it had been, at least in theory, available to departments for six or more months. Given a university-level report and/or departmental policy with explicit guidelines on peer review, why did summative review operate largely according to reviewers\u00E2\u0080\u0099 preferences, with few guidelines structuring the process? Why was there apparently little formal follow-up on the report\u00E2\u0080\u0099s recommendations? Perhaps the department heads did not send the UWC report on summative peer review to faculty members and/or it was not discussed at department meetings. Since only one participant explicitly mentioned the report, this assumption seems entirely possible. However, it may be that the report was circulated, but for various reasons faculty members chose to ignore it. The issue of whether the summative peer review report was taken up is significant, as it represents a prevalent issue in higher education: the policy- to-practice gap in teaching and learning strategies. Higher education leaders and policy-makers create policies with the aim of improving an element of teaching practice under the implicit assumption that such change will be beneficial. Implementing large-scale changes, however, is commonly \u00E2\u0080\u009Cdifficult, painstaking, 133 and protracted\u00E2\u0080\u009D (Kezar, 2011, p. 236), due in part to competing interests among an institution\u00E2\u0080\u0099s administrators, managers, and academics (Newton, 2003). Newton (2003) suggests that academics may resist new policies because they see them as eroding autonomy. That is, when they believe a policy is imposed for managerial purposes and may diminish whatever control they have over that aspect of their work, academics perceive their autonomy is being constrained. Faculty members may also resist new policies because of policy overload due to a voluminous increase of formal strategies to improve teaching and learning (Newton, 2003; Smith, 2011). This too erodes their sense of autonomy. Kezar (2001) provides additional insight into why the policy-to-practice gap remains prevalent in higher education, pointing to the fact that these institutions have been characterized as organized anarchies. Organized anarchies commonly have unclear and/or ambiguous goals and uncertain preferences (Kezar, 2001), and involvement in decision processes is fluid and outcomes frequently unpredictable. As a result, there may be inconsistencies between stated goals and actions. Ultimately, faculty members learn what such organizations value by observing their actions (Mills & Hyle, 2001). In the absence of a linear chain of command, it is often uncertain who\u00E2\u0080\u0099s in charge. Formal authority may be attributed to certain positions, but in practice faculty and administrators hold a great deal of authority (Kezar, 2001). In the case of the UWC report on summative peer review, it may be that faculty members judged recommendations unnecessary or did not understand the change or how it would be beneficial (Kezar, 2011). The characteristics of an organized anarchy are congruent with the context of a collegial system. In such a context\u00E2\u0080\u0094where it is not considered acceptable to apply coercive power 134 over one\u00E2\u0080\u0099s peers (Kezar, 2001)\u00E2\u0080\u0094faculty members are unlikely to employ tactics that ensure peer reviews are done according to specific guidelines, which could explain why faculty members, including those who head peer review initiatives in their departments, do not impose rules on how the process should be conducted. Thus, even if faculty members in a collegial culture are familiar with peer review guidelines, they may not adopt them. In the next chapter, I report on and discuss findings related to the second research question. In the final chapter of this dissertation, I make recommendations for theory, research, and practice supported by the findings and analyses reported in Chapters 4 and 5 and based on relevant scholarship. 135 Chapter 5: Departmental Practices that Promote or Hinder a Culture that Values Teaching 5.1 Chapter Structure In this chapter, I report on features of departmental culture that support or hinder teaching as described by study participants. Because participants frequently mentioned elements of institutional and disciplinary cultures that had an impact on their departmental experiences, I also incorporate these findings here. This chapter is comprised of four sections: 1) an exploration of informal departmental practices that shape the culture of teaching and learning, summarized in Table 8; 2) an identification of formal departmental practices and initiatives that assist faculty members who want to grow and learn in their teaching, also presented in Table 8; 3) an examination of ways that a department head does or does not support faculty members in developing in their role as teachers; and 4) a discussion of findings related to the second research question. Throughout the chapter, I highlight how professional growth in teaching occurs not only when an individual is internally driven to develop this area, but because of institutional, disciplinary, and departmental practices, beliefs, and structures that support or hinder faculty members\u00E2\u0080\u0099 growth as teachers. Despite the few rewards that are offered for strong teaching, combined with how departments and individuals are measured by their research productivity, teaching remains a core commitment for study participants. Even without apparent institutional support, my data indicate that participants are continually learning about teaching. Faculty members provided numerous stories that demonstrate the high value they place on 136 teaching and growing as instructors. Without exception, each individual spoke\u00E2\u0080\u0094at length or in passing\u00E2\u0080\u0094about specific actions s/he had taken to evolve in their professional role as a teacher. For instance, some faculty members redesigned their courses thanks to support from the ESEd, several worked on department-wide endeavours such as implementing a teaching assistant training program or modifying the department\u00E2\u0080\u0099s undergraduate curriculum, and others participated in cross-disciplinary team-teaching. These are just a few examples of initiatives that promote academics\u00E2\u0080\u0099 professional growth in teaching and through which they exemplify their commitment to teaching. Though participants\u00E2\u0080\u0099 narratives provide evidence of their ongoing involvement in activities that promote professional growth as teachers, I wish to underscore that the majority lamented being unable to pursue such activities more fully. Knowing that research productivity will be favoured in tenure and promotion decisions, faculty members hesitated to engage in time- intensive teaching professional development. For instance, several participants stated they were keen to observe colleagues\u00E2\u0080\u0099 classes to learn more about teaching, but could not find the time to do so. A few noted that they rarely met with their respective mentors because everyone was so busy; others said they lacked the time to participate in workshops at the Centre for Teaching and Learning (CTL) or with the ESEd. As described in Chapter 4, time constraints are tied to expectations of research productivity and a reward system that favours research over teaching. Participants, particularly female faculty, recognized the exigent nature of teaching. Teaching requires a great deal of time and can take \u00E2\u0080\u009Ca lot out of you\u00E2\u0080\u009D (Fradera, A3tF). One participant compared teaching to a \u00E2\u0080\u009Cblack hole,\u00E2\u0080\u009D noting that \u00E2\u0080\u009Cyou can spend all your energy [on it] and not get what you want\u00E2\u0080\u009D (Maki, A3aF) in terms of rewards 137 like student engagement and outside recognition. Within the UWC institutional culture, participants\u00E2\u0080\u0099 stories reveal that learning about teaching largely happens because they take action: they pose questions, conduct midterm evaluations, reflect, and get involved in teaching and learning initiatives. Since faculty members\u00E2\u0080\u0099 learning occurs principally in the context of their departments, the next three sections report on departmental practices that influence the culture of teaching and aid faculty members to learn about teaching. Included is a section that deals with the role of the department head. As I will discuss, both informal and formal practices in a given department support teaching\u00E2\u0080\u0094with greater or lesser success. Some of these practices are subtle ways of being that colleagues adopt while others are established programs and initiatives. Together, they comprise an array of approaches that establish the departmental culture of teaching within which faculty members pursue their pedagogical interests. 5.2 Informal Departmental Practices Participants grow as teachers by partaking in various informal departmental practices. Informal practices, in this study, refer to those which are not intentionally planned or coordinated but that influence professional growth and contribute to a departmental culture that values teaching. They can include how faculty members interact with one another, behaviours they adopt uncritically and other actions that collectively might be categorized as \u00E2\u0080\u009Cthe way we do things around here.\u00E2\u0080\u009D For example, many faculty members gain knowledge about teaching by engaging in spontaneous conversations with their colleagues. Others describe learning through trial and error, saying they have to \u00E2\u0080\u009Cmake it up as they go along\u00E2\u0080\u009D 138 (Abendroth, S1tM), learn from their mistakes, and \u00E2\u0080\u009Cpick it up on their own\u00E2\u0080\u009D (Nemeth, A2tF). Academics also draw from models in their past, such as teachers they admired or supervisors. All of these, and several others upon which I elaborate in this section, are examples of informal ways that faculty members learn about teaching in the context of their department. A summary of these practices can be found in Table 8. 5.2.1 Informal Conversations with Colleagues My findings show that informal conversations with colleagues are one of the main ways that faculty members develop in their roles as teachers. \u00E2\u0080\u009CMost faculty try to improve their teaching to some extent, if they\u00E2\u0080\u0099re interested, by talking to other people, other faculty\u00E2\u0080\u009D (Warr, S2TM). These conversations generally occur on an \u00E2\u0080\u009Cad-hoc continuing basis\u00E2\u0080\u009D (Wilks, S1tF). Faculty members approach their peers about a wide range of issues from the nuts and bolts of teaching to deeper discussions about implications of assignments, readings, and teaching approaches. In their informal conversations, faculty members garner advice about their syllabi, ask how to address a situation specific to a student, and share colleagues\u00E2\u0080\u0099 course materials, to cite only a few examples. Participants repeatedly emphasize the importance of these conversations to their learning and reveal that the exchanges thrive in an atmosphere where peers have collegial relationships and care about teaching. Given that faculty members from every department, with the exception of S3, perceive that their peers are interested in talking about teaching and use adjectives such as \u00E2\u0080\u009Charmonious,\u00E2\u0080\u009D \u00E2\u0080\u009Cfriendly,\u00E2\u0080\u009D \u00E2\u0080\u009Ccollegial,\u00E2\u0080\u009D and \u00E2\u0080\u009Csupportive\u00E2\u0080\u009D to describe their department and departmental peers, it is not surprising that many of the participants say there 139 are ongoing informal conversations about teaching in their unit. For example, Dodgson (S2aF), who describes her department as \u00E2\u0080\u009Cone of the nicest\u00E2\u0080\u009D where \u00E2\u0080\u009Ceveryone can talk to everyone,\u00E2\u0080\u009D has asked the \u00E2\u0080\u009Cgood teachers\u00E2\u0080\u009D about their specific teaching approaches in order to develop her own \u00E2\u0080\u009Ctricks\u00E2\u0080\u009D and ideas. Furthermore, the majority of faculty members in this study, with the exception of those I interviewed in S3, speak about their department as a place where people care about, and work hard at, teaching. \u00E2\u0080\u009CAnd I think most people care a lot. I think it\u00E2\u0080\u0099s\u00E2\u0080\u0094not just care a lot, but work at it and invest a huge amount of time during the teaching term to make sure that things are going well\u00E2\u0080\u009D (Knauer, S2TF). Especially in A2 and A3, participants make reference to \u00E2\u0080\u009Ca strong teaching department\u00E2\u0080\u009D (Maki, A3aF) that has many excellent teachers and, in A3, numerous teaching award winners. In a departmental atmosphere that is \u00E2\u0080\u009Ccohesive\u00E2\u0080\u009D and perhaps even, according to some, \u00E2\u0080\u009Cnon-hierarchical,\u00E2\u0080\u009D peers are comfortable approaching one another with questions about teaching. \u00E2\u0080\u009CIt\u00E2\u0080\u0099s the kind of culture where if you ask, they\u00E2\u0080\u0099ll answer,\u00E2\u0080\u009D remarks Ledonne about her own department (S1tF). 5.2.2 Collaborative/Individual Teaching Approaches Teaching may be a more collaborative or individual practice. In the Science departments, faculty members more often mention collaborative teaching initiatives than their colleagues in the Arts departments. For example, numerous Science faculty members speak about working with one another to establish links and/or consistency between courses in the undergraduate curriculum. With the assistance of the ESEd, colleagues determine learning goals for courses, put them online in a spreadsheet, and discuss how courses relate to one another, the rationale for their existing course sequence, and how courses contribute to the 140 overall curriculum. Science faculty members also jointly develop core courses that are then taught with a consistent approach. The outcome, explains Abendroth (S1tM), is that \u00E2\u0080\u009Cit\u00E2\u0080\u0099s sort of accepted what should be taught, and how it\u00E2\u0080\u0099s going to be taught.\u00E2\u0080\u009D This consistency is considered desirable by the Science participants. In the Arts departments, participants refer more often than in the Science departments to being able to shape their courses as they judge best. Several participants mention that there is no canonical set of topics for their course and that, consequently, they have a great deal of freedom when it comes to determining course content, readings and assignments. Says Irwin (A1aM) about creating his syllabus: \u00E2\u0080\u009CNo one would have told me how to handle the class. It was my class.\u00E2\u0080\u009D Consequently, Irwin adds, he did not ask a previous instructor for his/her syllabus as he did not think it would be useful because of the unique approach each person is likely to take. \u00E2\u0080\u009CEvery course is very particular and individualized,\u00E2\u0080\u009D explains Stromberg (A1TM) \u00E2\u0080\u009Cand so you have to kind of figure out your personal perspective and then how you\u00E2\u0080\u0099re going to communicate that to students because you want, actually, to be [in] kind of a dialogue between the instructor and the students.\u00E2\u0080\u009D A small number of faculty members in the Arts departments specify that materials from different offerings of the same courses are not expected to have any relationship to each other and, as long as course descriptors do not mislead students, faculty members can design their course as they want. In A2, where there are no course prerequisites, flexibility is at a maximum \u00E2\u0080\u009Cbecause none of us can assume that anybody has learned anything in particular\u00E2\u0080\u009D (Nemeth, A2tF). For the most part, Arts faculty members speak of this approach to course planning positively, suggesting an underlying assumption that this is how things (ought to) work. Two faculty members in A3, however, indicated that some attempts are being made to establish greater cohesion among courses: (a) 141 Bulmer (TM) tells of an ongoing process of undergraduate curricular reform to examine their course offerings and structure as a whole; and (b) Maki (aF) asserts that, as compared to five years ago, there \u00E2\u0080\u009Cis a little bit more collective input into the topics that get taught.\u00E2\u0080\u009D She contrasts that to before when \u00E2\u0080\u009Cthere was very much a sense of you can teach whatever you want.\u00E2\u0080\u009D Furthermore, or perhaps as a consequence of working more or less collaboratively to shape undergraduate core courses, faculty members express \u00E2\u0080\u009Cownership\u00E2\u0080\u009D over courses differently in the Science and Arts departments. In Science, faculty members refer to undergraduate courses not as their own, but as collective property. \u00E2\u0080\u009CNo one owns a course\u00E2\u0080\u009D (Woodroof, S2TM). That is, the same course may be taught by a multiplicity of faculty members and faculty members teach courses that range from elementary to advanced topics. In Arts, participants speak more often of the flexibility they have to shape courses. This broad disciplinary difference, in which there is more apparent collaboration in teaching-related matters in Science, surfaces repeatedly through the interviews and is an element of departmental culture which has bearing on collegial discussions of teaching and, consequently, on professional growth in teaching. 5.2.3 Departmental Composition The composition of a department affects the departmental culture of teaching, and conversations that take place in that unit, in several ways. Firstly, and as it pertains to pre- tenured faculty, these members bring energy and new perspectives to their unit. Hardy (tM), in A2, cites the expansion of their graduate program and the implementation of their teaching 142 assistant training program as examples of projects that have developed largely due to the initiative of the junior faculty. In S3 where, historically, faculty members have \u00E2\u0080\u009Cnot been so concerned about teaching\u00E2\u0080\u009D (Dawson, S3aM), Moretti (S3TF) notes that\u00E2\u0080\u0094unlike many of her senior colleagues when they were early on in their career\u00E2\u0080\u0094the younger faculty members take a distinct interest in teaching. Similarly, Dawson points out that the younger faculty members have been more involved in redesigning their courses and reflects, about himself and others, that may be so because \u00E2\u0080\u009Cwe\u00E2\u0080\u0099re young and impetuous\u00E2\u0080\u009D and therefore have had the courage to make such large-scale changes. Secondly, new faculty members support one another in their teaching professional growth. Warren (A3aF), who was hired the same year as four other pre- tenured faculty members, describes how the five new hires experienced a \u00E2\u0080\u009Cpeer effect\u00E2\u0080\u009D that was conducive to learning. The new faculty members, she explains, \u00E2\u0080\u009Chad pipelines\u00E2\u0080\u009D to senior colleagues, all of whom were \u00E2\u0080\u009Cvery interested in teaching and very interested in making sure we taught well particularly because there was such a big group of us coming in.\u00E2\u0080\u009D New information acquired from senior peers would then be \u00E2\u0080\u009Cdisseminated\u00E2\u0080\u009D among the pre-tenured faculty members. Another advantage of having a number of colleagues at an early career stage is that they sometimes feel more comfortable going to one another with what they perceive as a \u00E2\u0080\u009Cdumb\u00E2\u0080\u009D teaching question. Thirdly, new faculty members, unfamiliar with departmental norms, may alert senior faculty to the need for additional teaching support. For example, in A3, the surge in pre-tenured faculty members prompted people to realize that it would be helpful to formalize some of the procedures and policies that, to date, had been unwritten and informal. The development of a handbook for new instructors and the official establishment of a mini-committee that guides pre-tenured faculty members as they prepare for their annual review with the department head are cited as two examples. Pre-tenured 143 faculty members in A3 express appreciation for some of the measures the department has taken to formalize documents and procedures because it provides them with clear career support. The above three points exemplify that pre-tenured faculty members can shift the departmental teaching culture in important ways. A second example of department composition that can influence the departmental culture and the extent to which conversations about teaching occur is the ratio of men to women. Specifically, a small ratio of women to men affects female faculty members\u00E2\u0080\u0099 sense of vulnerability; therefore, in a unit where men vastly outnumber women, female faculty may take fewer risks in their teaching\u00E2\u0080\u0094including talking with their male colleagues about teaching. For instance, Fradera (A3tF) remembers being more comfortable approaching other female colleagues with questions about teaching when she started and was one of three women in the department. Similarly, Nemeth (A2tF), who gave a lecture on sexuality and the internet the last time she was peer reviewed, says that she would not have done so earlier in her career, in part because she was one of only two women faculty members, and also because her department was much more conservative in the past. In those days there were a lot of people I would not have given a lecture like that in front of. I would have had to completely recast what I was doing in order to feel that I wouldn\u00E2\u0080\u0099t be vulnerable in a whole range of ways. (Nemeth, A2tF) Parsons (A3TF), who recalls several incidents when young, male students have challenged and tested her authority in the classroom, and who knows this is an ongoing problem for many women, says that this is something she speaks about with her junior female colleagues. She frequently acts as an informal mentor to junior female colleagues, advising them on how 144 to handle challenging teaching-related situations or on other career-related issues. Parsons applauds the fact that, within the context of department meetings, colleagues now discuss gender issues such as the \u00E2\u0080\u009Chuge demand placed on them [female faculty members] in terms of female students coming to them for advice that\u00E2\u0080\u0099s unrelated to the particular course.\u00E2\u0080\u009D When female faculty feel less vulnerable and can speak collectively about issues that affect them and their colleagues, this may allow women to pursue their teaching with a greater sense of confidence. In addition, faculty members cite the physical presence of departmental colleagues as conducive to engaging in discussions about teaching. That is, faculty members who work in their office on campus benefit from each other\u00E2\u0080\u0099s help and provide valuable support to one another as they tackle teaching issues. Knocking on the door of a neighbouring colleague and having informal conversations about teaching in shared physical spaces, such as the lunch room, hallways, or the laboratory, are significant to faculty members\u00E2\u0080\u0099 professional growth. Finally, a small number of male and female faculty members acknowledge the role of young children in promoting collegial relationships that may, indirectly, foster discussions of teaching. These faculty members refer to the friendships they have made with other colleagues who have young children and say that, as part of these friendships, they socialize and talk about their life, including teaching. Children may also play a role in creating a more harmonious department because they encourage parent-academics to focus on what is most important. \u00E2\u0080\u009C [Children] exhaust you and [you] don\u00E2\u0080\u0099t always have time for battling [laughs]. You know, you just focus on the most important things\u00E2\u0080\u009D (Hardy, A2tM). Three of the women 145 point out that when faculty members are able to obtain childcare on the university campus19, they are much more likely to work at the office. Children, therefore, connect people to one another and, when childcare is available close by, can connect people to their place of work. 5.2.4 Good Teaching: No Consensus When asked whether departmental colleagues have a common notion of \u00E2\u0080\u009Cgood teaching,\u00E2\u0080\u009D faculty members say they do not. Collectively, participants employ a wide range of teaching techniques, speaking frequently about \u00E2\u0080\u009Cengaging students.\u00E2\u0080\u009D They recognize that there are a variety of ways to be a strong teacher, stating that no single model is appropriate and declaring that striving for uniformity is undesirable. Some faculty members speculate on where divides might lie and why they might exist; they name factors such as age, where the faculty member was educated (e.g. United States, Eastern Europe), and involvement in ESEd as possible reasons for differences in approaches to teaching. For the most part they are not troubled by the fact that colleagues have varying notions of good teaching. In fact, they largely accept\u00E2\u0080\u0094and even welcome\u00E2\u0080\u0094the differences. When asked if there was faculty consensus in A2 as to what good teaching looks like, Manfred (A2TM) answers with a chuckle: I hope not. I hope not. Because there are very many different styles of teaching. There are different objectives of teaching. There are different emphases. I don\u00E2\u0080\u0099t think there is and I don\u00E2\u0080\u0099t think there should be a consensus. I think there is a consensus about 19 According to the UWC Daycare Parent Council website, the waitlist for UWC daycare is more than 1500 children and parents wait an average of 2-3 years before their wait-listed child is offered a daycare spot. 146 what bad teaching is. . . . There\u00E2\u0080\u0099s probably more agreement on what bad practices in teaching are. This quote brings attention to three points commonly raised in the interviews: firstly, and as already mentioned, there is variation in opinion about what constitutes good teaching; secondly, academics believe and accept that there is no singular model of good teaching, and thirdly, given the variation, faculty members find it challenging to identify good teaching, but can more easily identify the opposite. Given this combination of factors, teaching and its evaluation could potentially turn colleagues fiercely against one another. However, in a context where there exist strong collegial relationships, the lack of consensus over teaching does not appear problematic, according to participants\u00E2\u0080\u0099 accounts. At best, it is celebrated and, at worst, it is given \u00E2\u0080\u009Cbenign neglect\u00E2\u0080\u009D (Maki, A3aF). 5.2.5 Risk-taking and Trust In departments where faculty members believe they can take risks in teaching and not be punished, they may be more likely to experiment and innovate in their teaching. Doing so encourages professional growth in teaching. Says Cohen (A2aM), \u00E2\u0080\u009CIf you really just care about getting good, then I would say just go crazy and do stuff.\u00E2\u0080\u009D He admits, however, that taking that approach can be risky, especially for pre-tenure faculty who might worry about whether their teaching (scores) will be good enough for tenure and promotion. Consequently, the easier model is \u00E2\u0080\u009Cjust do what you did before and tinker with it a little to improve things a little\u00E2\u0080\u009D (Cohen, A1aM). Palo (S1TF), who experienced a troubling dip in her teaching evaluation scores after she made a substantial change to a course, remembers thinking during a discussion with her department head: \u00E2\u0080\u009CAre you going to punish me for trying something 147 innovative and taking risks?\u00E2\u0080\u009D In her case, she was not punished: she asked for support from her department head who connected her to a colleague she trusts and the latter provided her with suggestions for change that she adopted with good results. In the Science departments, the existence of, and support from, the ESEd also influences faculty members\u00E2\u0080\u0099 willingness to take risks. Numerous faculty members say they have accessed help from ESEd teaching fellows20 to make changes\u00E2\u0080\u0094sometimes large, other times less so\u00E2\u0080\u0094to their courses. Teaching fellows are recognized as pedagogical specialists who can effectively help faculty members redesign their courses, conduct research on student learning, and assist with a multitude of other related teaching challenges and issues at the classroom and departmental level. It is possible, but not necessarily so, that they would not have implemented the changes without support from teaching fellows working with the ESEd. For instance, when Dawson (S3aM) decided to completely redesign his course, he approached one of the ESEd teaching fellows for help; that person, assigned to help another faculty member, was not available. Dawson decided to pursue his project regardless but was eventually given assistance. He recalls: I think what happened was when people realized that I was going in head first, there was enough political will to let [Name of teaching fellow] off the hook for some other tasks and let him help me. Once they could see that I wasn\u00E2\u0080\u0099t joking, I was serious and 20 ESEd teaching fellows are hired by the department to work with faculty members on initiatives that contribute to the improvement of undergraduate science teaching. These individuals have \u00E2\u0080\u009Ca combined expertise in the specific departmental disciplines as well as knowledge in relevant science education methodology and research\u00E2\u0080\u009D (UWC\u00E2\u0080\u0099s ESEd website). 148 seriously over my head maybe, that they let [Name of teaching fellow] help me. And then it was a great success, I think. Irrespective of whom a faculty member approaches, the issue of trust surfaces strongly within the context of professional growth in teaching. A number of faculty members in the Sciences seek help from ESEd teaching fellows because of the pedagogical expertise these individuals have. Other faculty members distinguish more generally between people they would approach and those they would not. There\u00E2\u0080\u0099s at least a half dozen full profs that I would be quite comfortable approaching. I won\u00E2\u0080\u0099t say that I would feel comfortable approaching every person in the department, but there\u00E2\u0080\u0099s no shortage of people that I could talk to, and get very different viewpoints on things. (Abendroth, S1tM) Palo (S1TF) also speaks to the importance of trust when she describes how she addressed a dip in her teaching scores. She sought help from her department head who referred her to a colleague whom she describes as a \u00E2\u0080\u009Cprofessional instructor, considered one of our very best.\u00E2\u0080\u009D Palo applied the advice given to her, not only because she wanted to address the problem, but because she had faith in her colleague\u00E2\u0080\u0099s strong teaching skills and knowledge. Several faculty members draw on and highly regard the pedagogical expertise of tenure-track instructors. When dealing with teaching challenges, academics take risks to different degrees and may seek help from their colleagues. Decisions about how to proceed and who to approach are based on different rationales; trust, however, is almost always an important consideration. 149 5.2.6 Attention to Fairness Faculty members remark upon several departmental teaching-related practices that demonstrate fairness, an attribute that people believe contributes to collegiality. Collegial relationships, as I have previously indicated, promote pedagogical conversations, which as participants indicate, are one of the principal ways in which they grow and develop as teachers. One practice related to fairness is that of having an established teaching load. In S1, all participants speak about the existence of a clear formula for the amount of undergraduate and graduate teaching a faculty member is expected to take on yearly, depending on their rank. A second practice that connects to fairness, and mentioned by numerous participants, is that faculty members at all ranks teach courses at all levels. Another is having a say about which courses one would like to teach. For instance, in S1, Ledonne (tF) describes a giant spreadsheet where, for each class in the upcoming three years, a faculty member can indicate their preference on a continuum that ranges from \u00E2\u0080\u009Ceager,\u00E2\u0080\u009D to \u00E2\u0080\u009Cno way in heck.\u00E2\u0080\u009D The departmental system is further enhanced because details of that schedule (i.e., which course a faculty member will rank as first choice this coming year) are sometimes negotiated ahead of time among faculty members who work in the same laboratory. Ledonne believes S1\u00E2\u0080\u0099s system is a \u00E2\u0080\u009Cmodel of transparency and fairness\u00E2\u0080\u009D because all faculty members are able to make requests, see each other\u00E2\u0080\u0099s preferences, and make updates to these. Dodgson (S2aF) describes a system that resembles the S1 spreadsheet where faculty members indicate their preferences among courses, days of the week and times of the day. All this, she says, is coordinated by \u00E2\u0080\u009Csome poor faculty member\u00E2\u0080\u009D and colleagues do not get to see what courses 150 people requested nor what they have been assigned. \u00E2\u0080\u009CBut,\u00E2\u0080\u009D she adds \u00E2\u0080\u009CI know from informal conversations that most people get their preferences.\u00E2\u0080\u009D Unlike Ledonne, Dodgson does not enthusiastically refer to S2\u00E2\u0080\u0099s system as one that promotes a sense of equality and fairness among faculty members. She appreciates being able to input her preferences but may speak less enthusiastically because, first of all, the choices and results are not public and/or, second of all, she may not recognize to what degree this formal system, which is not in effect in all departments, has the potential to build collegiality. In contrast to these practices, \u00E2\u0080\u009Ccourse buyouts,\u00E2\u0080\u009D whereby faculty members are permitted to use funds from a research grant to reduce their teaching load, were noted to deter from fairness. Dawson (S3aM) describes that \u00E2\u0080\u009Cthere\u00E2\u0080\u0099s a huge internal struggle in the department regarding teaching buyouts.\u00E2\u0080\u009D He continues to say that, in a department where a lot of people have teaching buyouts to do research and others don\u00E2\u0080\u0099t, \u00E2\u0080\u009Cthere\u00E2\u0080\u0099s a lot of resentment in between those groups. And I think that\u00E2\u0080\u0099s been an ongoing point of friction.\u00E2\u0080\u009D Ledonne (S1tF) notes that the teaching buyouts are \u00E2\u0080\u009Cnot part of the culture\u00E2\u0080\u009D in S1, no matter how much money a faculty member receives from their regular research grant and she reasons that \u00E2\u0080\u009Cthat greatly contributes to fairness.\u00E2\u0080\u009D Ludar (A3TM) adds that if a department head allows teaching buyouts, it sends a message to faculty members that they should pay less attention to teaching than to research. About these, he states: \u00E2\u0080\u009CI won\u00E2\u0080\u0099t deal with teaching buyouts. I just say \u00E2\u0080\u0098No.\u00E2\u0080\u0099 to them all.\u00E2\u0080\u009D Allowing teaching buyouts, he makes clear, does not contribute to establishing the importance of teaching in a faculty member\u00E2\u0080\u0099s career. 151 5.3 Organized Initiatives, Policies and Structures Numerous formal teaching-related initiatives and practices prompt professional growth in teaching or in other ways influence teaching within a given department. I distinguish formal initiatives from the informal ones (e.g., informal conversations among colleagues, strategies that promote fairness in teaching loads) because the former are typically planned, structured, or in some other way \u00E2\u0080\u009Clegitimately\u00E2\u0080\u009D or unequivocally connected to teaching. Formal structures to address teaching, in the form of committees, policies, seminars, or officially appointed individuals, exist in all departments. They may come about as a result of faculty-wide initiatives (e.g., the decision to make courses more writing-intensive across a given faculty), departmental ones (e.g., student town hall meeting or peer reviews of teaching), or other institutionally supported \u00E2\u0080\u009Cmovements\u00E2\u0080\u009D (e.g., community service learning). Among the range of departmental initiatives that fall into the formal category (see Table 8), I will be discussing the following: policies and documents, officially appointed resource people, ESEd, seminars and retreats, team-teaching, and formative peer review. 5.3.1 Recognized Policies and Other Documents The majority of participants say that, within their department, there exist few or no written departmental policies and documents pertaining to teaching. At the time of the interviews, only A3 had a formal policy for the peer review of teaching; most, but not all, of the participants from that department knew of its existence. The policy was created in response to a faculty-wide push to formalize peer reviews of teaching and was developed by three 152 faculty members in consultation with all departmental colleagues. In S2, although there is no written policy, a small committee has recently been struck to work out an established protocol for the peer review of teaching. Hanna (S2tF) explains that, previously, the undergraduate chair had coordinated the peer reviews but \u00E2\u0080\u009Che does so much for the department [that] he\u00E2\u0080\u0099s physically not capable of still doing it. So that's why this committee has been founded, because it's something the department feels is very important.\u00E2\u0080\u009D She suspects that part of the reason the committee has been formed is because the department is now so large (close to 100 faculty members and post-doctoral fellows) that, consequently, \u00E2\u0080\u009Cthe best way to make sure it [peer review of teaching] keeps running smoothly, I think we all agree from our various departmental meetings, was actually just to formalize what we'd been doing anyway\u00E2\u0080\u009D (Hanna, S2tF). Although many other faculty members in departments outside S2 and A3 recognize that there are more demands associated with peer reviews of teaching as compared to before, none specifically mention a need for a policy. Asked about other departmental documentation that might serve as a teaching resource, faculty members could only list a few. In S1 and S2, an online spreadsheet offers course learning objectives. S2 and S3 have a relatively new initiative where faculty members can upload their teaching resources (syllabi, assignments, exams) to a password-protected departmental website for colleagues to peruse. Both these projects are part of the ESEd. Hadzik (S3tM) also indicates that, at one time, there was an orientation book for new faculty. Since it was coordinated on a voluntary basis, it \u00E2\u0080\u009Cdidn\u00E2\u0080\u0099t go far.\u00E2\u0080\u009D In A3, there is a handbook for new instructors but not all new faculty are made aware of it. A few faculty members in A1 and A2 mention that syllabi from past courses are kept in a central location (i.e., filing 153 cabinet in the main office and/or on a departmental website) so that faculty members can access these. Finally, in S1, there is a wiki with teaching-related resources that was started by Deitz (S1aF) and to which various people have added material. Unlike the learning goals project and the online repository of materials that are part of the ESEd, the wiki in S1 was initiated without explicit support from another network. The above underscores that projects are more likely to be taken up and persist when at least one person is appointed to champion and advance them. The above also brings to light general differences in the extent to which materials are gathered and collectively used in Arts compared to Science. For instance, faculty members in S1 and S2 speak frequently about putting their course materials on their personal website. Furthermore, in Science there are more formal practices for sharing teaching-related material than in the Arts; namely, the course material repository and the spreadsheet for learning objectives that have come about because of the ESEd.21 Perhaps because many Science faculty members post their teaching materials online, they also speak more often than academics in Arts about visiting their colleague\u00E2\u0080\u0099s websites to access material when they have had to teach the same course (they also contact these same colleagues in person for material that might be missing from the web). Trottier (tM) in A2, however, explains that he is more likely to search for online syllabi or teaching materials on the websites of colleagues located 21 Given that I only interviewed participants located in departments associated with the initiative, I do not know if similar online repositories exist in other departments. Nor do I know whether these structures exist because of the ESEd, or whether they would have come about regardless. Furthermore, one cannot help but wonder whether faculty members in Arts would put more of their material online if there were an Arts equivalent to the ESEd. I will take some of these issues up in the discussion. 154 at other institutions. That is because there is a greater chance that a colleague elsewhere would be teaching a course similar to his. Within his department, there would be little overlap in approach or content among faculty members even if they were teaching different offerings of the same course. As already pointed out, faculty members in Arts often take a more individual approach to course development and those in the Science more often take a collective one. That is not to say that academics in Arts are not willing to share their material, for they are. However, when it comes to developing their courses, faculty members in Science are more inclined to work collaboratively. This approach is reflected in the fact that there are more multi-section courses with common exams and assignments and faculty members work collaboratively to figure out how courses in a program align with each another. 5.3.2 Official Go-to People The University\u00E2\u0080\u0099s Centre for Teaching and Learning is cited by several faculty members as a place where one can get assistance with teaching. Participants speak about the CTL as a resource for people who are experiencing \u00E2\u0080\u009Cteaching problems\u00E2\u0080\u009D and/or as a space to engage in professional development in teaching via workshops and seminars. A number of faculty members note that they do not participate in the CTL\u00E2\u0080\u0099s activities because of limited time and research demands. In the following quote, Warren (A3aF) talks about her interest in taking a certificate program on teaching in higher education, but decides against it in the end because it will take time away from her research: Actually I remember thinking it sounded really cool and I remember thinking it would be great to do it and I\u00E2\u0080\u0099d probably get a lot out of it and it was never going to 155 happen because it wasn\u00E2\u0080\u0099t going to help publish the article I was working on at the time. Although several participants who are aware of the centre have encouraged their colleagues to attend the CTL offerings, only a few have actually used the services themselves and most turn to departmental colleagues and other resources for help. Faculty members name the following as helpful departmental colleagues to confer with about teaching: course coordinators (Science), tenure-track instructors, and the undergraduate chair. In the Science departments, the course coordinator refers to the individual in charge of coordinating multiple sections of a course. These courses typically have standard midterms, assignments and readings across sections and may have shared labs. Normally not a new faculty member, the course coordinator\u00E2\u0080\u0094or \u00E2\u0080\u009Cmaster custodian\u00E2\u0080\u009D (Ledonne, S1tF) \u00E2\u0080\u0094provides teaching materials and resources and is available to guide faculty members, as necessary. \u00E2\u0080\u009CThat\u00E2\u0080\u0099s your big mentor really\u00E2\u0080\u0094the person in charge of the course. . . . They have a big influence on how it\u00E2\u0080\u0099s done in the course\u00E2\u0080\u009D (Woodroof, S2TM). In addition to the course coordinators, faculty members frequently mention tenure-track instructors as key people to consult with because of their teaching expertise. These individuals who are described as \u00E2\u0080\u009Cexcellent\u00E2\u0080\u009D or \u00E2\u0080\u009Cprofessional\u00E2\u0080\u009D teachers, are well integrated into their respective departments. In S1, where there are more than 10 tenure-track instructors, instructors also handle \u00E2\u0080\u009Ca lot of the grunt work\u00E2\u0080\u009D (Abendroth, S1tM) with undergraduate students, including advising. Another person faculty members approach when they have teaching-related questions is the undergraduate chair. Some faculty members recognize this person as one who takes a key role in teaching initiatives (e.g., in A2 the undergraduate chair planned the teaching retreat 156 and in A3 she was one of three people on a committee that developed the summative peer review policy) and regularly puts teaching on the agenda at department meetings. Finally, in the Science departments, all faculty members who are, or have been, involved in the ESEd mention teaching fellows as important resource people. Faculty members rarely consider mentors, assigned as part of a formal mentorship program, as people to approach with questions about teaching. According to most participants, the mentorship relationship is largely focused on research and career advancement. Only a small number of faculty members occasionally discuss teaching with their mentor. 5.3.3 Effective Science Education Project (ESEd) According to many Science participants, the Effective Science Education Project promotes professional growth in teaching among those who participate in it and has influenced the culture of teaching within the departments where the initiative exists. Faculty members typically voluntarily self-select to take part in ESEd projects. Projects include, but are not limited to the following: course redesign, reading groups, teaching and learning specific research projects, development of learning goals, curriculum revisions, and creation of assessment methods to measure student learning. Teaching fellows, situated in the departments, work with faculty members on the projects. 157 5.3.4 Teaching Seminars and Retreats In S2, there are regular (every 1-2 months) teaching seminars with guest speakers from within and outside the department. Though they are mainly attended by graduate students and post-doctoral fellows, faculty members also participate. The seminars are associated with a teaching assistant development program for which there is a faculty member whose work duties include developing and coordinating the implementation of that program. In A3, informal brown-bag discussions about teaching took place in the past, approximately once a term, over a span of several years. Parsons (A3TF), who had initiated these, suspects that organizing them \u00E2\u0080\u009Cfell by the wayside\u00E2\u0080\u009D because it was not a formal assignment that contributed to the service component of her work. In A2, teaching seminars occur irregularly and are normally organized to address a specific concern shared by faculty members; the concern may have been raised at the department\u00E2\u0080\u0099s annual student town hall or at the teaching retreat. Both A2 and A3 held a departmental retreat within the year prior to the interviews. In A3, the faculty members also discussed the department\u00E2\u0080\u0099s hiring policies at the retreat, whereas in A2, the event focused entirely on teaching. The department retreat in A2, organized by the undergraduate chair, was a two-day event for full-time and sessional faculty; students were also invited to a portion of the event so they could share some of their concerns. Faculty members were asked to cancel their classes and sessional faculty were paid an honorarium to attend\u00E2\u0080\u0094two measures taken to ensure maximum attendance. In the quote below, Manfred (A2TM) gives a brief description of the event and reflects on its significance: 158 And we spent two whole days talking, not primarily about curriculum but about lecturing methods, about how to teach writing effectively, how to deal with special problems that often arise, of issues of cultural sensitivities, issues of students in classes having very different levels of preparation because of the fact that most of our courses don\u00E2\u0080\u0099t have formal prerequisites, etcetera. And it was a wonderful experience because we did a lot of mutual informing and learning about people\u00E2\u0080\u0099s best and worst experiences and so on. It was also a wonderful experience because it was one of those rare occasions where the full-time faculty and the sessional faculty were in a sense interacting as complete equals. Because this is something we all do. We teach [name of department] courses in this department. . . . I think it created an atmosphere in which people realized that how we teach is something the department actually takes very seriously. And that this is not just an adjunct to the other thing that we take very seriously, which is our research programs, but it is a core commitment of the department. We knew that. But it reaffirmed the centrality of teaching to the overall mission of what we do. Thus, as Manfred indicates, the retreat was significant because colleagues had dedicated time to learn from one another about teaching and this helped reconfirm the importance of teaching to their professional lives. The other faculty members in this department, all of whom spoke positively about the retreat, also noted that it provoked a lot of informal discussions about teaching. That is in part because the faculty members were given assignments prior to the retreat (i.e., they were requested to sit in on two colleagues\u00E2\u0080\u0099 lectures and given a discipline-specific teaching-related 159 article to read) and because, after the retreat, and in response to faculty members\u00E2\u0080\u0099 suggestions that there be more opportunities to talk about teaching, the undergraduate chair organized some non-mandatory department-wide discussions about teaching. In A3, the one-day retreat was held during UWC\u00E2\u0080\u0099s reading week. Fuentes (A3aM) explains that one of the main purposes was to get people more enthused about teaching by exposing them to some of the teaching techniques their peers are using. Thus, during the half-day portion dedicated to teaching, nine different faculty members presented on topics such as community service learning, simulations, technology and teaching, and in-class debates. Since normally, \u00E2\u0080\u009Cyou don\u00E2\u0080\u0099t actually find out what your colleagues are doing in class,\u00E2\u0080\u009D the retreat was \u00E2\u0080\u009Ca nice way to find that out,\u00E2\u0080\u009D comments Parsons (A3TF). \u00E2\u0080\u009CEverybody,\u00E2\u0080\u009D she continues, \u00E2\u0080\u009Cliked hearing what people were doing in their classes and asked lots of good questions.\u00E2\u0080\u009D Although nearly all the faculty members in department A3 mentioned the retreat, they did not connect it to teaching-related initiatives or conversations that have taken place since then, as did the people in A2. Departmental retreats provide opportunities for faculty members to share and learn about teaching; when retreats prompt follow-up events, such as seminars or additional conversations, the possibilities for growth are extended. 5.3.5 Team Teaching Across Disciplines There are course offerings that are team taught by faculty members from separate departments and different disciplinary backgrounds. Team teaching is taken here to mean 160 two or more faculty members working to develop a course taught to the same class. In team teaching, faculty members attend or participate in a majority of class sessions during the semester. A small number of faculty members express how these experiences have contributed to their learning about teaching. Nemeth (A2tF) describes team teaching as a useful way to learn about \u00E2\u0080\u009Cdifferent ways of being in the classroom\u00E2\u0080\u009D and says that her participation in an interdisciplinary teaching program \u00E2\u0080\u009Cprobably had more impact on shaping me as a teacher than almost anything else.\u00E2\u0080\u009D Irwin (A1aM) describes team teaching as a sort of \u00E2\u0080\u009Cconstant\u00E2\u0080\u009D peer review because team members discuss teaching with one another on an ongoing basis as they create tests and assignments, review mid-term student evaluations, and go-over exam results. Though not officially team teaching, some faculty in S1 collaborate extensively with colleagues in their sub-field to determine learning objectives and content for individual courses. 5.3.6 Formative Peer Review Only S2 has a regular practice of formative peer review whereby a senior faculty member observes a new faculty member\u00E2\u0080\u0099s classroom teaching within the first three weeks of their first class. All the faculty members interviewed in S2, with one exception, are aware that this is standard practice in their department. Dodgson, the only pre-tenured faculty member I interviewed in the department, remembers the formative feedback as being helpful, but she does not refer to the practice as being part of the departmental protocol. In S1, Abendroth (tM) recalls one \u00E2\u0080\u009Cunusual\u00E2\u0080\u009D year during which everyone (i.e., people on the peer review committee and others) had to conduct a formative peer review of a colleague\u00E2\u0080\u0099s teaching. The 161 initiative was dropped and he does not know why. Finally, and as already mentioned, A2 colleagues did classroom observations of teaching as part of their pre-retreat homework. Though formative peer review is rarely a part of a department\u00E2\u0080\u0099s official policies or functioning, some faculty members have, upon their own initiative, attended their colleagues\u00E2\u0080\u0099 classes. They do so to better learn the content of the class (sometimes because they will be teaching that content themselves and do not feel sufficiently familiar with it) and/or because they want to observe a stellar teacher in action. Faculty members who sit in on their colleague\u00E2\u0080\u0099s classroom may do so for as long as a term, or as short as a single class. 5.3.7 Teaching and the Hiring Process Many faculty members point out that prospective new hires are asked to discuss teaching (i.e., strategies, approach, syllabi) during their interview or are required to teach as part of their application for a tenure-track position. This practice, participants note, can promote discussions of teaching among individuals who evaluate the candidate\u00E2\u0080\u0099s performance. Nemeth (A2tF) points out, however, that she has colleagues who worry that extensive conversations about teaching during the interview process \u00E2\u0080\u009Cmakes us look like we\u00E2\u0080\u0099re not really concerned about research.\u00E2\u0080\u009D She says there exists a division in the department on that issue, but notes that in the last round of hiring, discussions of teaching figured prominently. Hadzik (S3tM) also expresses unease about the way teaching is incorporated in the hiring process, but his concerns centre around the (questionable) usefulness of the method. In S3, a candidate gives a lecture\u00E2\u0080\u0094always on the same topic\u00E2\u0080\u0094to departmental faculty members. Hadzik deems this is an ineffective method of gauging the candidate\u00E2\u0080\u0099s teaching abilities and 162 admits that he, like several other colleagues, does not normally attend the teaching portion of a candidate\u00E2\u0080\u0099s presentation. He would be more inclined to take an interest in a prospective new hire\u00E2\u0080\u0099s teaching if the candidate were to teach undergraduate students who need not \u00E2\u0080\u009Cpretend to be learners.\u00E2\u0080\u009D Students, he explains, could then comment on their learning, unlike faculty members who already know the content well (see also footnote 14). According to Hadzik, the purpose of evaluating teaching in the hiring process ought to be to determine whether the person can promote student learning; since, at present, the focus is not that, Hadzik remains somewhat disengaged. Moretti (S3TF), however, has a somewhat different perspective on the purpose of considering teaching in the hiring process, saying that it is to have an indicator of the candidate\u00E2\u0080\u0099s potential rather than a way to eliminate \u00E2\u0080\u009Clousy teachers.\u00E2\u0080\u009D We grade them, on a scale from zero to ten. Even if they get graded a four, we might still make them an offer based on their excellent research record, because we say, \u00E2\u0080\u0098Oh, they\u00E2\u0080\u0099re young and they haven't taught much. They have potential to teach well, even though that was one of the worst things I\u00E2\u0080\u0099ve ever seen in my life.\u00E2\u0080\u0099 [laughs] (Moretti, S3TF) Most often, she continues, the evaluation panel is hoping to get a sense of the candidate\u00E2\u0080\u0099s attitude toward teaching and/or potential to develop as a good teacher. Even if participants disagree on the details of how teaching should be incorporated into a candidate\u00E2\u0080\u0099s application process, they seem to agree that when teaching is considered during the hiring process it indicates that teaching matters in the department. When colleagues evaluate a candidate\u00E2\u0080\u0099s performance or debate how a department may want to demonstrate their commitment to teaching to prospective new hires, I suspect these discussions contribute to learning about teaching. 163 Table 8: Informal and Formal Initiatives and Practices that Influence the Culture of Teaching within a Department Department Initiatives and Practices Formal Informal Centre for Teaching and Learning (CTL) \u00C3\u00BC\u00EF\u0083\u00BC Classroom observation of teaching \u00C3\u00BC\u00EF\u0083\u00BC \u00C3\u00BC\u00EF\u0083\u00BC Colleagues, Conversations with \u00C3\u00BC\u00EF\u0083\u00BC Colleagues, Physically present \u00C3\u00BC\u00EF\u0083\u00BC Committee work pertaining to teaching \u00C3\u00BC\u00EF\u0083\u00BC Course coordinator for multiple sections of a class \u00C3\u00BC\u00EF\u0083\u00BC Course development, Collaboration toward \u00C3\u00BC\u00EF\u0083\u00BC \u00C3\u00BC\u00EF\u0083\u00BC Course, Introduction of new \u00C3\u00BC\u00EF\u0083\u00BC \u00C3\u00BC\u00EF\u0083\u00BC Courses, Fairly assigned \u00C3\u00BC\u00EF\u0083\u00BC \u00C3\u00BC\u00EF\u0083\u00BC Curricular reforms/redesigns \u00C3\u00BC\u00EF\u0083\u00BC Department composition \u00C3\u00BC\u00EF\u0083\u00BC Department head, Support for teaching from \u00C3\u00BC\u00EF\u0083\u00BC \u00C3\u00BC\u00EF\u0083\u00BC Department meetings, Discussions of teaching at \u00C3\u00BC\u00EF\u0083\u00BC \u00C3\u00BC\u00EF\u0083\u00BC Department retreat, Sessions about teaching included in \u00C3\u00BC\u00EF\u0083\u00BC Diverse teaching approaches, Acceptance of \u00C3\u00BC\u00EF\u0083\u00BC Documents, Teaching (i.e. handbook for new instructor) \u00C3\u00BC\u00EF\u0083\u00BC Effective Science Education Project (ESEd) \u00C3\u00BC\u00EF\u0083\u00BC External review \u00C3\u00BC\u00EF\u0083\u00BC Hiring, Attention to teaching in \u00C3\u00BC\u00EF\u0083\u00BC Instructors, Tenure-track \u00C3\u00BC\u00EF\u0083\u00BC Mentorship \u00C3\u00BC\u00EF\u0083\u00BC Midterm course assessment \u00C3\u00BC\u00EF\u0083\u00BC \u00C3\u00BC\u00EF\u0083\u00BC New faculty, Cohort \u00C3\u00BC\u00EF\u0083\u00BC Peer review, Formative \u00C3\u00BC\u00EF\u0083\u00BC \u00C3\u00BC\u00EF\u0083\u00BC Peer review, Summative \u00C3\u00BC\u00EF\u0083\u00BC Policy, Teaching (e.g. peer review of teaching policy) \u00C3\u00BC\u00EF\u0083\u00BC Reflection (e.g., on models from one\u00E2\u0080\u0099s past, self-) \u00C3\u00BC\u00EF\u0083\u00BC Resource, Sharing of (e.g., syllabi, wiki, website) \u00C3\u00BC\u00EF\u0083\u00BC \u00C3\u00BC\u00EF\u0083\u00BC Risk-taking in teaching, support for innovation \u00C3\u00BC\u00EF\u0083\u00BC 164 Department Initiatives and Practices (continued) Formal Informal Self-reflection \u00C3\u00BC\u00EF\u0083\u00BC Seminars, Teaching \u00C3\u00BC\u00EF\u0083\u00BC Socializing \u00C3\u00BC\u00EF\u0083\u00BC Student evaluations of teaching \u00C3\u00BC\u00EF\u0083\u00BC Student town hall meeting \u00C3\u00BC\u00EF\u0083\u00BC Teaching assistant training \u00C3\u00BC\u00EF\u0083\u00BC \u00C3\u00BC\u00EF\u0083\u00BC Teaching fellows, ESEd \u00C3\u00BC\u00EF\u0083\u00BC Team teaching \u00C3\u00BC\u00EF\u0083\u00BC \u00C3\u00BC\u00EF\u0083\u00BC Trial and error, Learning by \u00C3\u00BC\u00EF\u0083\u00BC Undergraduate chair \u00C3\u00BC\u00EF\u0083\u00BC \u00C3\u00BC\u00EF\u0083\u00BC Note: Informal practices, in this study, refer to those that are not intentionally planned or coordinated with the purpose of \u00E2\u0080\u009Cdeveloping teaching\u00E2\u0080\u009D but that influence faculty members\u00E2\u0080\u0099 professional growth and promote a departmental culture that values teaching. Formal practices refer to planned and structured activities and/or positions that are more \u00E2\u0080\u009Clegitimately\u00E2\u0080\u009D or unequivocally tied to teaching. 5.4 The Department Head\u00E2\u0080\u0099s Role in Promoting a Culture that Values Teaching As the participants described the various informal and formal practices within their department, it became evident that the department head has a significant role in establishing and maintaining a culture that values teaching. That individual can do so in overt ways such as establishing committees to address teaching-related issues or by putting into practice university policies pertaining to the improvement of teaching. Many of the ways the department head supports teaching are subtle. And even though only one department head participated in this study, numerous faculty members spoke about ways their head supported\u00E2\u0080\u0094to a greater or lesser degree\u00E2\u0080\u0094their growth as teachers. In Chapter 4, I described the department head\u00E2\u0080\u0099s role in the summative peer reviews, and reported that most participants have not discussed these reviews in any depth (or at all) with their department head. In the context of summative peer review, the department head 165 contributed little to the professional growth of teaching. Participants\u00E2\u0080\u0099 stories nevertheless highlighted numerous ways that a department head can be involved in a faculty members\u00E2\u0080\u0099 growth as an instructor. 5.4.1 Teaching and Tenure: Support from the Department Head Through various approaches, department heads help academics develop in their professional roles as teachers. The support offered is sometimes directed toward the improvement of teaching and student learning and other times more focused on ensuring career advancement. The department head occasionally plays a role in helping faculty members when they encounter challenges in their classroom teaching. For example, when students had complained to the department head about Quinn\u00E2\u0080\u0099s (S3TM) course/teaching, the head alerted Quinn and invited him to provide his perspective on what might be going on. Together, they discussed strategies to tackle the teaching complaint. Similarly, Palo (S3TF) consulted with her department head when her teaching scores dropped after she had made a change to one of her courses; she asked her department head to suggest someone who might have tried a similar approach to hers (with success) and he did so. Palo was able to get the help she needed from her colleague and address the issue. Sometimes a department head attends to a teaching-related situation after the fact in a way that appears to be specifically geared to supporting tenure and promotion as opposed to having an explicit focus on promoting improved teaching and student learning. Returning to the example above, Palo (S3TF), who had had uncharacteristically poor student evaluation 166 results after making a modification to one of her courses, was invited by the head to present an explanation in her portfolio. Doing so would help ensure that peers voting on her case would \u00E2\u0080\u009Cfeel comfortable that it was indeed an anomaly\u00E2\u0080\u009D and would also provide her department head with \u00E2\u0080\u009Cammunition\u00E2\u0080\u009D to defend the inconsistency to the Dean\u00E2\u0080\u0099s advisory committee on promotion and tenure. Perceived support from the department head for teaching also comes about when the head does something as \u00E2\u0080\u0098basic\u00E2\u0080\u0099 as mentioning that s/he has looked at the student evaluation of teaching scores. Faculty members respond to this positively noting that it gives \u00E2\u0080\u009Ca sense that people are paying attention\u00E2\u0080\u009D (Parsons, A3TF). Warr (S2TM) praises a former department head who reviewed student evaluations of teaching annually in a meeting with faculty members: It was a brief time of one of our heads who made an effort to comment on our student evaluations. Now since I always held the student evaluations as being the most important form of evaluation I was very pleased that the head took the effort to look at them. . . . I\u00E2\u0080\u0099m not really convinced that any head before or any head after has made an effort to worry about the teaching evaluations and hence the teaching of the faculty other than at a promotion tenure time. . . . I\u00E2\u0080\u0099ve never heard anything back from him so it\u00E2\u0080\u0099s sort of like, \u00E2\u0080\u0098Oh, so have you ever looked?\u00E2\u0080\u0099 I don\u00E2\u0080\u0099t ask that question because that\u00E2\u0080\u0099s their choice, but I just thought it was very nice when it happened. That meant that during that time if your evaluations had been positive you would get some praise. If they had been negative the person would have been pointing out some difficulties. 167 However, teaching rarely figures in faculty members\u00E2\u0080\u0099 discussions with their department head. \u00E2\u0080\u009CI don\u00E2\u0080\u0099t really remember teaching\u00E2\u0080\u0094the teaching component of my portfolio\u00E2\u0080\u0094ever being discussed in those meetings\u00E2\u0080\u009D (Wilks, S1tF). Fradera (A3tF), whose experience is similar to Wilks\u00E2\u0080\u0099, speculates that unless \u00E2\u0080\u009Cred flags are coming up\u00E2\u0080\u009D or, in other words, when the faculty member has received negative feedback on their teaching, heads of department have very little information about a faculty members\u00E2\u0080\u0099 teaching. \u00E2\u0080\u009CThey don\u00E2\u0080\u0099t really know what\u00E2\u0080\u0099s going on because we don\u00E2\u0080\u0099t have a constant peer evaluation process,\u00E2\u0080\u009D she says. This latter quote captures two points that come up repeatedly in the interviews. Firstly, teaching receives little attention because the primary focus is faculty member\u00E2\u0080\u0099s research productivity and plan. Indeed, two pre-tenured faculty members\u00E2\u0080\u0099 recollect being told by their head, during their annual meeting, to focus on their research and not on their teaching. \u00E2\u0080\u009CI\u00E2\u0080\u0099ve been told to ignore my teaching and focus on my research because my teaching is fine,\u00E2\u0080\u009D says Deitz (S1aF). Similarly, Cohen (A1aM) remembers being told to \u00E2\u0080\u009Cspend less time on your teaching; worry about your research.\u00E2\u0080\u009D Secondly, teaching is discussed if it is deemed that a \u00E2\u0080\u009Cteaching problem\u00E2\u0080\u009D exists as indicated, typically, by lower than acceptable student evaluation of teaching scores. Teaching, say some tenured faculty members, is given minimal attention by the department head when a faculty member\u00E2\u0080\u0099s teaching is adequate or excellent. Good teaching, one might deduce, is not something worth talking about. With regards to addressing teaching when it is problematic, some tenured faculty members\u00E2\u0080\u0094 but no pre-tenured faculty\u00E2\u0080\u0094refer to UWC\u00E2\u0080\u0099s Collective Agreement (and/or the related guidance document used to assist personnel committees and others involved in the decision- making process) to explain why a department head discusses problematic teaching with a 168 candidate. They point out that the department head is obliged to discuss concerns that might impede a faculty member\u00E2\u0080\u0099s progress to tenure and provide constructive ways to improve the situation. I make this distinction between tenured and pre-tenured academics and the former\u00E2\u0080\u0099s reference to the Collective Agreement (or guidance document) to suggest that it is possible that individuals with longer academic careers might be able to connect departmental practices to existing policies in ways that newer academics are not yet in the habit of doing. Faculty members, when reflecting on the importance of the department head\u00E2\u0080\u0099s support, frequently mention it in the context of moving a file forward to tenure. Several pre-tenured faculty members note that, as they prepare for tenure, they will go to their department head for details about the process. Some of the tenured faculty members interviewed recall specific \u00E2\u0080\u009Ctips\u00E2\u0080\u009D their department head (and/or other tenured colleagues) shared to help them strengthen their tenure case. Tips included: information about the types of courses to teach (i.e., make sure you teach a first year course), advice to aim for breadth in teaching (i.e., teach outside your specialty area), what an acceptable student evaluation of teaching score is, the suggestion to explain any anomalous path in teaching (i.e., why an individual has a dip in their teaching scores; why someone has taught only in their area) and, of course, \u00E2\u0080\u009Cthink about more journal publications\u00E2\u0080\u009D (Ledonne, S1tF). Faculty members praised the concrete nature of this advice. Comments from faculty members attest to their belief that the department head has the knowledge to determine what a successful file consists of; s/he also has the power to support 169 (or not) an individual. Referring to meetings with her department head, Maki (A3aF) explains: They are useful in the pre-tenure process because you need to have the head on your side to be put forward. And you get a sense of how strong is the support\u00E2\u0080\u0094are there any reservations? So I think it\u00E2\u0080\u0099s a psychological support. [It is helpful to hear that] \u00E2\u0080\u009CEverything is on track and fine.\u00E2\u0080\u009D Moretti (S3TF) agrees and speaks to the importance of having a dependable and ethical head. As she described the tenure process, Moretti makes clear that the head\u00E2\u0080\u0099s letter in support/denial of a candidate\u00E2\u0080\u0099s tenure, meant to summarize the standing committee\u00E2\u0080\u0099s report and deliberations, should normally be reviewed by the committee members, but committee members do not always see the letter. Sometimes the head \u00E2\u0080\u009Ccan just subjectively interpret what they want at the meeting, selectively include or not include things . . . and write some bullshitty letter\u00E2\u0080\u009D before sending it up the ranks. Thus, the head\u00E2\u0080\u0099s support is crucial in augmenting the chance of a file moving forward, as is his/her trustworthiness. In summary, the department head has a potentially significant role in communicating with faculty members about teaching and its evaluation. However, the way in which an individual head might enact this role varies and is \u00E2\u0080\u009Cnot built into departmental practices in any systematic way\u00E2\u0080\u009D (Nemeth, A2tF). Consequently, the ways in which the head addresses matters to do with teaching vary. Although, by most faculty members\u00E2\u0080\u0099 accounts, teaching rarely figures in discussions with their department head, the participants indicate that this individual is important in setting a tone for the departmental culture of teaching. \u00E2\u0080\u009CThat\u00E2\u0080\u0099s the head\u00E2\u0080\u0099s role, to be constantly reading the department to make sure the messages are 170 supportive and realistic. . . . To a degree signing papers and so on, but sending signals at all times about what standards and behaviours are acceptable or unacceptable\u00E2\u0080\u009D (Ludar, A3TM). The department head\u00E2\u0080\u0099s actions, or lack thereof, communicate in important ways what matters; the values he or she promotes transcend many facets of departmental culture. 5.5 Summary of Findings The faculty members in this study are keen to grow in their roles as instructors and are involved in nurturing a departmental culture that values teaching. Many lament the fact that they cannot fully pursue developing as teachers, and attribute this to an institutional reward system that favours research productivity over teaching and a career that places high demands on their time. Though numerous academics express frustration at not being involved with teaching to the extent that they wish, their collective stories demonstrate multiple departmental practices, structures, and \u00E2\u0080\u009Cways of being\u00E2\u0080\u009D that can, and do, support faculty members in their desire to grow and learn as teachers. As the findings in this chapter indicate, when programs and other structures are well supported, by human and other resources collectively and over time, their potential for helping faculty members grow and learn as teachers increased. In this chapter I have described ways that informal practices and organized departmental initiatives support the enhancement of teaching and noted there are few official departmental documents that guide teaching. I have pointed to the significance of professional learning conversations to instructor growth and highlighted the importance specifically of trust, and more broadly of collegiality, in promoting these conversations. I have also reported on the 171 relevance of disciplinary traditions, namely working collaboratively or individually on teaching, to the prevalence of collegial interactions about teaching. In Chapter 5, I have elaborated on the previous chapter\u00E2\u0080\u0099s findings with respect to the important role of department head in supporting a culture that values teaching. In ways both subtle and overt, this person signals the importance of teaching to faculty careers. 5.6 Discussion: Departmental Practices that Support or Hinder a Culture that Values Teaching So far, I have reported on informal practices and organized departmental initiatives, including leadership from the department, which support and hinder a culture that values teaching. The catalysts that have spurred these departmental practices and the factors that maintain them differ. The longevity of the initiatives vary, as does the extent to which faculty members have participated in and/or availed themselves of these. In addition, some practices are more directed at professional growth than others. However, because professional growth occurs as academics engage in their day-to-day activities (Billett, 2001; Clegg, 2003; Eraut, 2004; Jawitz, 2009), even teaching practices not labeled as \u00E2\u0080\u009Cprofessional development\u00E2\u0080\u009D may contribute to faculty members\u00E2\u0080\u0099 learning about teaching. And, although I did not set out to inquire about institutional practices per se, the participants discussed them, thus reaffirming the intersections between institutional and departmental culture. The results of this study demonstrate that responsibility for learning about teaching does not lie solely within the individual, but that it is strongly influenced by context: the institutional, departmental, and disciplinary cultures within which academics work. These cultures operate 172 simultaneously, though not necessarily equally, depending on the situation (Lee, 2007; Umbach, 2007). 5.6.1 Institutional Culture Faculty members\u00E2\u0080\u0099 understanding of the reward system from the context of the research- intensive university influenced their involvement in teaching and its evaluation. The participants, all of whom believed that research productivity garners greater rewards than strong teaching, commonly chose research-oriented activities that led to publication over teaching-related professional development activities. For example, several academics in this study declined to participate in workshops at the CTL, formative peer reviews, and ESEd activities. Others felt they were unable to pursue their own course improvements to the extent they wanted because of high demands on their time and the need to be a productive researcher: \u00E2\u0080\u009CI can\u00E2\u0080\u0099t [make the course modifications], and I\u00E2\u0080\u0099m overwhelmed, and there\u00E2\u0080\u0099s no freaking way,\u00E2\u0080\u009D lamented Ledonne (S1tF) as she talked about \u00E2\u0080\u009Cwanting to be a better teacher but being very, very invested in research.\u00E2\u0080\u009D Maki (A3aF) referred to teaching as a \u00E2\u0080\u009Cblack hole.\u00E2\u0080\u009D As with black holes in space, from which nothing\u00E2\u0080\u0094not even light\u00E2\u0080\u0094escapes, teaching resembles a black hole because the outcomes of one\u00E2\u0080\u0099s effort are unclear. Irrespective of the creativity, time, and resources an instructor devotes to designing and teaching a course, students may be discontent, course objectives may not be met, and scores from student evaluations of teaching may be low. Furthermore, and as described in Chapter 4, faculty members questioned the extent to which outcomes from the summative peer reviews of teaching influence decisions about tenure, promotion, and merit; consequently, their engagement in this process was limited. Given competing demands between research and 173 teaching, and an awareness that a strong research record is favourable in decisions about tenure, promotion, or merit, faculty members opted for activities that would reward them career-wise. The above findings are in line with studies that have demonstrated that in research-intensive universities, which are positioned at the highest end in the hierarchy of post-secondary institutions, faculty members spend more time on activities related to research and publication than those related to teaching (Link et al., 2008; Milem et al., 2000). Though this study did not track how faculty members allocated their time, many participants spoke about limiting their involvement in teaching-related activities. One reason they prioritized research over teaching is that research productivity is rewarded over teaching (Chalmers, 2010; Serow, 2000). Thus, as confirmed in the literature, the academic reward system acts as a major source of extrinsic motivation (O\u00E2\u0080\u0099Meara, 2011) in influencing how faculty members use their time (Fairweather, 2005; Link et al., 2008). This structural entity, which rewards or penalizes, works to socialize and shape faculty members\u00E2\u0080\u0099 behaviours and is an important cultural force in faculty careers (O\u00E2\u0080\u0099Meara, 2011; Tierney, 1997). Even though all faculty were clear that research garners greater rewards, several participants expressed beliefs, ranging from tentative to firm, that teaching counted in career decisions. To some extent, this came about because participants identified programs, such as a multimillion dollar initiative to reform undergraduate science (ESEd) and the creation of a community service learning unit, that demonstrated concrete support for teaching by UWC\u00E2\u0080\u0099s senior administration. Many also pointed out that the summative peer review of teaching process had become more formalized over time, and presumed the impetus for this was tied 174 to directives from senior administration. Teaching and student learning also figure prominently in the UWC strategic plan and collective agreement with the Faculty Association. The message touted by UWC is that teaching counts. Its push for teaching is part of a broader trend whereby, in the past two decades, \u00E2\u0080\u009Cconsiderable resources and intellectual attention\u00E2\u0080\u009D (McFarlane, 2011, p. 127) have been devoted to raising the status of teaching in higher education. Still, a number of participants felt that UWC\u00E2\u0080\u0099s claim about valuing teaching was vacant because they were not receiving time and resources to develop professionally in their teaching. The university\u00E2\u0080\u0099s emphasis on teaching also meant that teaching was being increasingly monitored. Many participants noted that the demands associated with the summative peer reviews had increased compared to three to five years prior to the interviews, and several pointed out that the requirements for the teaching component of a tenure file had \u00E2\u0080\u009Cinflated\u00E2\u0080\u009D (Wilks, S1tF). Some faculty members resented the additional demands on their time, especially when they perceived little benefit in terms of their growth and development as teachers. In addition, numerous faculty members in Science talked about how the student evaluations of teaching were used to graphically compare faculty members or courses to one another, another monitoring practice. In Arts as well as in Science, the use of student evaluation of teaching scores to compare people to each other or a department average could create intense stress or a sense of competition. In the literature, these monitoring practices are commonly associated with an accountability agenda in post-secondary institutions where increasingly, faculty members must provide evidence of their performance in all areas, in particular their teaching (Gosling & D\u00E2\u0080\u0099Andrea, 2001; Shanahan, 2009). 175 5.6.2 Departmental Culture In spite of predominant institutional pressures and constraints, faculty members still made choices about how to allocate their time and found ways to cultivate pedagogical interests and commitments. In doing so, they exercised agency over their professional learning activities (Jawitz, 2009; Neumann, Terosky, & Schell, 2009; Serow, 2000). However, factors like motivation, awareness of self, and will were only partly responsible for determining how and if a faculty member pursued her/his instructor role. As discussed elsewhere in this dissertation, institutional culture also shapes faculty members\u00E2\u0080\u0099 engagement in professional growth activities. In addition, the departmental culture, where faculty members participate in routine (and less routine) activities, affects opportunities for growth and learning about teaching. Jawitz (2009) uses the term \u00E2\u0080\u009Charmonization\u00E2\u0080\u009D to describe a process whereby academics, as they engage over time in workplace practices, adjust how they act upon their agency. Individuals\u00E2\u0080\u0099 choices and motivations with regard to their teaching thus shift as faculty members participate in departmental activities both with peers and by themselves. The findings from this study show that, in a department where teaching is considered a legitimate and worthwhile academic activity, as evidenced by the presence of both formal and informal practices that support teaching, faculty members committed to this aspect of their work are encouraged to discuss teaching, pursue it in creative and diverse ways, and take risks. In some instances, a strong departmental culture of teaching can outweigh an institutional culture that affirms the dominance of research. For example, Wilks (S1tF) made major changes to a course and invested the time to do so because she wanted to improve that course and because she knew it would be \u00E2\u0080\u009Cpositively seen\u00E2\u0080\u009D by her department, given that they \u00E2\u0080\u009Cplace 176 some emphasis on teaching.\u00E2\u0080\u009D She was confident that when it came to tenure, her peers would positively evaluate the time and energy she had put into modifying her course. Dawson (S3aM) too invested considerable hours (which he could presumably have devoted to research) to redesign his course. He received departmental support from an ESEd teaching fellow; even without the support, Dawson claims he would have gone ahead with it anyway, because he was convinced that the teaching methods would improve his students\u00E2\u0080\u0099 learning. In both cases, as well as numerous other situations described by participants, it was apparent that the extent to which academics involved themselves in teaching activities and practices depended both on individual and contextual factors: self-will was combined with perceived departmental endorsement and/or concrete departmental resources. While some departmental practices supported professional growth, others did not. For instance, Warr (S2FM) pointed out that, summative peer review reports in his department are not shared with the candidate, a practice he thought was odd and \u00E2\u0080\u009Cstupid,\u00E2\u0080\u009D but which he nevertheless adopted. In addition, several faculty members rarely, if ever, discussed teaching with their mentors. One participant (Cohen, A1aM) felt that doing so might (wrongly) signal an insufficient commitment to research, reasoning that if the mentor-mentee relationship is meant to focus on research (a message most participants had internalized), it was best not to emphasize topics such as teaching. This illustrates how an interest in teaching may be adjusted (or harmonized, to use Jawitz\u00E2\u0080\u0099s term) to match collective departmental practices that run counter to professional growth in teaching and/or asserted the dominance of research. As affirmed in previous studies, faculty members engage with the understanding and assumptions in their own department and, from these, draw inferences about what counts and 177 how to pursue their work (Jawitz, 2009; Knight et al., 2006; Tierney, 1997; Trowler & Knight, 2000). One who played an important role in fostering practices that could signal the value (or lack thereof) of teaching was the department head. When a department head paid attention to teaching by discussing it in an annual review, investing time and resources for a departmental retreat on teaching, and/or requiring new hires teach as part of their job application process, these practices indicated to faculty members that teaching was an important aspect of their work. The opposite was also true: when a department head neglected to discuss strong evaluations of teaching with individual faculty members, or advised those with an interest in teaching to focus on research, or inconsistently administered the teaching evaluation process, this signaled a low value was placed on teaching. These findings are consistent with the literature, which has shown that the leadership of a department head is key in determining whether and how a department supports teaching (Anderson et al., 2008; Braxton, 2008; Gibbs et al., 2008; Knight & Trowler, 2000; Massy et al., 1994). 5.6.2.1 Learning Conversations One of the most common ways that academics engaged with various departmental understandings and assumptions was through conversation. A pervasive feature in all informal and formal departmental practices that fostered a culture of teaching was how they promoted collegial conversations about teaching. As participants described, these came about in numerous ways. Many\u00E2\u0080\u0094including those that contributed most to their learning\u00E2\u0080\u0094came about informally, in an impromptu fashion. Conversations were also prompted when \u00E2\u0080\u00A2 new faculty members outwardly questioned existing teaching assumptions and 178 practices; \u00E2\u0080\u00A2 colleagues worked on campus; \u00E2\u0080\u00A2 peers collaboratively developed their course and/or team-taught; \u00E2\u0080\u00A2 faculty members discussed teaching in the hiring process; \u00E2\u0080\u00A2 departments created teaching-related policies; and \u00E2\u0080\u00A2 colleagues engaged in dialogue about the peer review of teaching. Based on the study findings, I propose that a main signifier of a culture that values teaching is the presence of conversations that permit faculty members to grow and learn as teachers. The importance of conversations to professional growth has been highlighted by scholars like Byrne et al., (2010), Haig (2005), and Rox\u00C3\u00A5 and M\u00C3\u00A5rtensson (2009). Ultimately, conversations about teaching most often took place in an environment where peers valued and respected one another\u00E2\u0080\u0099s teaching approach and pedagogical knowledge. Trust and a non-threatening climate were key. For example, in the context of summative peer reviews, when the results were potentially punitive, few learning conversations ensued. When, on the other hand, faculty members informally approached colleagues with a teaching challenge or question, learning conversations were more likely to occur. Notably, a lack of consensus about teaching among peers promoted a nonjudgmental climate in departments where relationships were perceived to be collegial. In such departments, faculty members were inclined to believe that regardless of their teaching approach and philosophy, they would be evaluated fairly by their peers. Some noted, however, that in departments where professional and personal rivalries prevailed, the absence of consensus could be problematic: if a candidate in the peer review of teaching did not share the same approach as his/her 179 reviewer and had a strained relationship with that person, the person reviewed might legitimately fear a poor evaluation from a reviewer who was \u00E2\u0080\u009Cout to get\u00E2\u0080\u009D him or her. Within a unit, the existing relationships must be conducive to people having conversations. 5.6.3 Disciplinary Culture and Conversations A last finding I wish to examine is the connection between disciplinary culture and collegial discussions of teaching. I reported earlier that when compared with faculty members in Arts, those in Science collaborated more frequently on teaching. In the Science departments, many multi-section courses were jointly developed by several faculty members and implemented with the help of a course coordinator. In addition, academics more often spoke of discussing, debating, and sharing course goals, and team teaching was common. The sense that \u00E2\u0080\u009Cno one owns a course\u00E2\u0080\u009D (Woodroof, S2TM) was stronger than in Arts, where faculty members generally developed courses individually and pursued teaching on their own. These differences confirm that academics in the hard-pure disciplines engage more frequently in collaborative practices than their soft-pure counterparts (Becher & Trowler, 2001; Neumann et al., 2002; Umbach, 2007). A collaboration may be facilitated by the fact that curricula in hard-pure disciplines tends to be linear and hierarchical and comprised of courses that are tightly structured with closely connected concepts and principles (Neumann et al., 2002). When faculty members agree on how courses fit together and why, more frequent conversations about teaching may ensue. On the other hand, in the soft-pure disciplines, where content tends to be \u00E2\u0080\u009Cmore free-ranging and qualitative, with knowledge- building a formative process\u00E2\u0080\u009D (Neumann et al., p. 408) and courses are more loosely structured (Lattuca & Stark, 1994), faculty members may be less inclined to collaborate, not 180 just because courses are less interdependent but also because collaborative conversations may be seen as time-consuming and spur conflict among peers who disagree on course goals and the ways to achieve them. In the final chapter of this dissertation I draw on the findings and analyses in Chapters 4 and 5, and from relevant scholarship, to make recommendations for theory, research, and practice. 181 Chapter 6: Summary, Conclusions, and Recommendations This chapter provides a brief summary of the study, a summary of key findings, and a discussion of this study\u00E2\u0080\u0099s contributions to theory. Following the discussion, I consider possible applications of the study findings to summative peer review and to departmental practices that foster a culture that values teaching. In particular, I focus on what improvements in peer review practice might support professional growth in teaching among tenure-track faculty members working in research-intensive universities. I end by outlining suggestions for future research. 6.1 Summary of the Study In what follows, I present a summary of the study including a statement of the research problem and the research questions; a short description of the conceptual framework and research methodology; an empirical summary of the key findings from this research; and, the study limitations. 6.1.1 Statement of the Problem In research-intensive universities, where research productivity is rewarded over teaching and escalating demands impose significant stress on academics (Gappa et al., 2007; Robinson, 2006), faculty members commonly limit their involvement in professional development activities for the improvement of teaching (Kilgore & Cook, 2007). Yet teaching is a chief responsibility in academic careers and central to faculty members\u00E2\u0080\u0099 work. As such, ongoing professional growth in teaching is both necessary and helpful to faculty members as they work to fulfill their professional responsibilities and enhance their teaching roles (Gappa et 182 al., 2007; O\u00E2\u0080\u0099Meara et al., 2008). Prior research has confirmed that a great deal of faculty members\u00E2\u0080\u0099 professional learning about teaching occurs as they participate, within their department, in informal teaching-related activities and other practices that do not intentionally aim to improve teaching (Jawitz, 2009; Knight et al., 2006). These practices, therefore, are especially relevant to a study of faculty members\u00E2\u0080\u0099 professional growth in teaching. In North American institutions, one teaching-related departmental practice commonly used to evaluate teaching in tenure and promotion decisions is summative peer review of teaching (Gravestock & Greenleaf, 2008). The extent to which summative peer review can contribute to professional growth has been questioned in the literature. Several scholars (e.g., Kell & Annetts, 2009; Peel, 2005; Shortland, 2004) have tied summative peer review to the accountability agenda in higher education which serves to monitor or maintain standards. The summative process is frequently compared to formative peer review, which is said to improve teaching by promoting reflection on, and dialogue about, teaching (Hammersley- Fletcher & Orsmond, 2004). In practice, the formative/summative distinction is not that clear and, consequently, it is preferable to think of peer reviews as existing on a continuum. When one does so, it is conceivable that summative peer review could contribute to professional growth in teaching. However, little research has investigated this aspect of summative peer review in the North American research-intensive university. In this dissertation I set out to examine faculty members\u00E2\u0080\u0099 experiences of summative peer review and their understandings of how that process contributes to professional growth in 183 teaching. Since summative peer review occurs only periodically in faculty members\u00E2\u0080\u0099 careers, and most faculty members do not perceive it as being conducive to professional growth, a second main objective of this study was to explore other departmental practices that contributed to a culture that values teaching and might, therefore, help faculty members grow in their roles as teachers. 6.1.2 Research Questions The purpose of this research was to explore practices that encourage professional growth in teaching among tenure-track faculty members working in a research-intensive university. The research questions were as follows: 1. What are faculty members\u00E2\u0080\u0099 experiences of summative peer review of teaching and how do they understand the relationship between summative peer review and professional growth in teaching? 1a. In what ways do the multiple purposes of summative peer review play out and interconnect? 1b. How do academic values and norms, as well as other established practices in higher education, influence the summative peer review of teaching process? 2. What existing departmental practices support or hinder a culture that values teaching? 184 2a. How do aspects of institutional and disciplinary culture influence departmental practices that support/hinder a culture that values teaching? 2b. What is the relationship between departmental practices that foster a culture that values teaching and faculty members\u00E2\u0080\u0099 professional growth in teaching? 6.1.3 Conceptual Framework This study investigated the issue of faculty members\u00E2\u0080\u0099 professional growth in teaching through the lens of academic culture. I drew on scholarship concerning the peer review of teaching and from previous research on institutional, departmental, and disciplinary cultures to develop and conduct the study. O\u00E2\u0080\u0099Meara et al.\u00E2\u0080\u0099s (2008) framework for professional growth in faculty careers provided a useful way of thinking about how faculty members learn and grow in their professional roles. 6.1.4 Study Participants and Data Collection Thirty tenure-track faculty members working at a large, research-intensive Canadian university took part in the study. Half the participants were from the Faculty of Arts and the other half were from the Faculty of Science. For each Faculty, participants were selected from three departments representing hard-pure disciplines (2), hard-applied disciplines (1), and soft-pure disciplines (3). Through the administration of a questionnaire, study participants were purposefully selected to get a range of rank (8 pre-tenured and 22 tenured) and gender (13 females and 17 males). 185 Using a semi-structured interview protocol, I interviewed participants once, either on campus or at a location of their choice, for approximately one hour. Participants were asked to reflect on their experiences of summative peer review: as a candidate, reviewer, or both. I first inquired about the departmental process of peer review and their involvement in and experiences of that process, and about their notions of an ideal peer review process. Subsequent questions focused on what good teaching looks like in their department, how faculty members learn the criteria for good teaching, and departmental support for teaching. These questions generated information about faculty members\u00E2\u0080\u0099 experiences and understanding of the summative peer review and particulars about teaching-related departmental practices. 6.1.5 Data Analysis The interviews were transcribed and subsequently reviewed by myself and those participants who wished to do so. I then developed initial codes that I refined, and sometimes redefined, as I coded the interviews using Atlas.ti. I clustered codes to form themes and sub-themes that helped me make connections between participants and ideas. This process was also facilitated by keeping a research journal in which I recorded reflections. Data analysis continued as I wrote the dissertation. 6.1.6 Empirical Summary Faculty members\u00E2\u0080\u0099 experiences of summative peer review varied, as did their expectations of the purposes and outcomes of the process. Participants built their understandings of peer review based on limited experiences. These may have included experiences of being 186 reviewed, reviewing colleagues, participating in standing committee meetings (where peer review reports may or may not be discussed), and being in meetings with their department head (during which an individual\u00E2\u0080\u0099s peer review reports were rarely given attention). The existence of a university-level document on summative peer review appears to have had little impact on practice or faculty members\u00E2\u0080\u0099 understanding of peer review. In the one department where there was a peer review policy, details of that policy were not known to all participants, and its impact on practice was questionable. People\u00E2\u0080\u0099s individual experiences of peer review still varied a great deal. Despite the variation and inconsistency in experience, participants identified three purposes for summative peer review: a formal mechanism for evaluation, a means of promoting professional growth in teaching, and a supplement to the student evaluations of teaching. Faculty members did not think that the summative peer review process was successful in achieving the first two objectives, and cited unsuitable reviewers (where lack of suitability was defined as being a poor teacher, and/or having insufficient content/pedagogical knowledge, and being untrained on how to evaluate teaching), ambiguous or nonexistent criteria, the episodic nature of reviews, and few (or no) guidelines for conducting reviews as possible reasons for the lack of success. Furthermore, the majority of participants believed summative peer review failed as an evaluative tool because, firstly, faculty members are unwilling to document suggestions for teaching improvement for fear that these might have a negative effect in tenure and promotion decisions, and secondly, more attention and weight are given to the student evaluation of teaching results. As it concerned the potential of summative peer review to contribute to professional growth 187 in teaching, participants were of the opinion that this objective was seldom met because of the poor quality of feedback. That is, in most cases the candidates and reviewers did not discuss feedback; in the rare cases when they did, this was usually unspecific and therefore unhelpful to the candidate. Many faculty members expressed disappointment at the absence of feedback from the summative peer review process. Nevertheless, several suggested that the process should not seek to meet multiple objectives because the evaluative functions conflicted with the formative aims of summative peer review. Although summative peer review, as it is currently practiced, makes minimal or questionable contributions to faculty members\u00E2\u0080\u0099 professional growth as teachers, there are numerous departmental practices that contribute to a culture that values teaching and promote academics\u00E2\u0080\u0099 growth and learning in their roles as instructors. They include formal practices that specifically aim to enhance teaching such as lunchtime seminars on teaching, formative peer reviews, and teaching retreats. More often, however, they consist of informal practices such as conversations among peers about teaching and beliefs about valuing diverse notions of good teaching. The unplanned learning conversations, which contribute to faculty members\u00E2\u0080\u0099 growth as instructors, normally occur between trusted colleagues who work on campus and value each other\u00E2\u0080\u0099s knowledge and approaches to teaching. They are facilitated by the presence of people who are assigned to positions that emphasize teaching, such as course coordinators, undergraduate chairs, and tenure-track instructors. The conversations are also influenced by disciplinary cultures, in particular whether there is an established tradition of working 188 collectively or individually. Additionally, the department head is an important individual when it comes to establishing a culture that values teaching and where there are frequent collegial conversations about teaching. S/he does so by encouraging collegial relationships among faculty members, appointing committees to address specific departmental teaching issues, discussing teaching with individual faculty members in one-on-one meetings, and supporting faculty-led initiatives for the enhancement of teaching. Faculty members\u00E2\u0080\u0099 commitment to teaching, along with how they exert their agency within their department, also influence their engagement in the various practices that promote growth and learning. Finally, faculty members\u00E2\u0080\u0099 engagement in summative peer review and other departmental practices that contribute to a culture that values teaching are strongly shaped by the institutional culture. Participants unanimously agreed that research productivity garners greater rewards than teaching and because of this and the high demands on their time, the majority limited the time they spent on teaching (broadly defined) in order to focus on research and publication. However, some wondered about the extent to which teaching counts in tenure and promotion decisions. In a few cases, faculty members invested large amounts of time to redesign a course because they perceived strong departmental support for teaching and reasoned that the time they devoted to teaching would be favourably evaluated in tenure and promotion decisions. Though they acknowledge that the university is placing greater attention on teaching now as compared to the past, several expressed cynicism at the university\u00E2\u0080\u0099s claims that teaching matters. 189 6.1.7 Limitations There are several limitations to this study. First, because the study examined faculty members in a single research-intensive university, it does not consider the experiences of tenure-track faculty members at other types of post-secondary institutions. Especially in institutions where there is more emphasis on teaching and less research activity, one would expect to find differences in how faculty members pursued their professional growth in teaching. Second, faculty members were from a small number of departments in only two faculties and, with the exception of S3, participants described theirs as being strong teaching departments with a collegial culture. Academics from a wider range of departmental cultures, and in particular departments with a weaker culture of teaching, may have had very different experiences than those who volunteered for this study. Third, I assumed that participation in departmental teaching practices would promote professional growth in teaching, but did not measure professional growth in teaching nor observe faculty members\u00E2\u0080\u0099 participation in departmental teaching-related practices. Data collection was based only on participants\u00E2\u0080\u0099 perceptions and self-reported data. Fourth, I made assumptions about the Biglan-Becher typology to which faculty members belonged on the basis of the department they were in. This is because, with one exception, the departments represented in this study were frequently cited as classic examples of that typology in the literature, i.e. \u00E2\u0080\u009Ceducation\u00E2\u0080\u009D is a classic example of a soft-applied discipline. Had I attempted to identify to which specific category (hard-pure/hard-applied; soft-pure/soft-applied) each faculty member should belong, I might have noted different disciplinary trends overall. Fifth, the sample size was relatively small and therefore the study results may not reflect the experiences of a wider 190 range of faculty, in particular, assistant professors and other underrepresented faculty members, including women and faculty members of varying ethnicity and race. Finally, the fact that I was the only person coding the data presents a limitation to the reliability of this study; had there been other people coding, I might have picked up on additional nuances and themes in the data. 6.2 Contributions to Theory A primary purpose of this research was to investigate the issue of faculty members\u00E2\u0080\u0099 professional growth in teaching through the lens of academic culture. Chapter 1 sets forth a preliminary conceptual model (Figure 1) that draws on the literature of institutional, departmental and disciplinary cultures and from O\u00E2\u0080\u0099Meara et al.\u00E2\u0080\u0099s (2008) framework for professional growth in faculty careers. Based on the findings of this dissertation research, I made several important changes to the preliminary conceptual framework. Notably, I propose two separate conceptual models for the study of faculty cultures and professional growth in teaching, one that pertains to the evaluative practice of summative peer review of teaching, and the other that is more relevant to non-evaluative departmental practices. Whereas the preliminary model did not identify specific aspects of institutional, departmental, and disciplinary culture at play in the relationship between culture and professional growth, the revised models in Figures 2 and 3 do. Thus, the reason for developing two separate models was to highlight the different aspects of academic culture that dominate and manifest in the relationship between summative peer review and professional growth, and in the relationship between non- 191 evaluative departmental practices and growth. These modifications, which are intended to strengthen the usefulness of these models in future research, are outlined in the sections below. They are also summarized in Table 9. Figure 2: Model for Faculty Member Engagement in Summary Peer Review of Teaching and Professional Growth in Teaching (Model A) 192 Figure 3: Model for Faculty Member Engagement in Non-Evaluative Departmental Practices that Foster a Culture that Values Teaching and Professional Growth in Teaching (Model B) 6.2.1 Description of the Two Conceptual Models Prior to describing each model and outlining their differences, it is worth pointing out their similarities. First, both models illustrate how institutional, departmental, and disciplinary cultures influence the relationship between departmental teaching practices (evaluative and non-evaluative) and faculty members\u00E2\u0080\u0099 professional growth in teaching. These cultures have been placed at the outer limits of each model, with arrows penetrating the inner boxes, to depict their continuous role in shaping faculty careers. Second, the models portray summative peer review and other departmental practices (the innermost box) occurring within a context where academic norms and values constantly operate. These norms and 193 values are defined by broader overlapping institutional, departmental, and disciplinary cultures. Though values and norms may manifest differently depending on context, they are omnipresent and permeate all aspects of faculty members\u00E2\u0080\u0099 selves and work. Third, models A and B identify the values of autonomy and collegiality as being especially relevant to faculty members\u00E2\u0080\u0099 engagement in departmental practices that may foster professional growth in teaching. Finally, the models illustrate that relationships, agency, commitment, and learning (the four lenses used to explore faculty members\u00E2\u0080\u0099 professional growth in O\u00E2\u0080\u0099Meara et al.\u00E2\u0080\u0099s 2008 conceptual framework) influence and mediate the interactions between departmental teaching practices and professional growth in teaching. That is, faculty members\u00E2\u0080\u0099 growth as instructors may be supported or hindered by the quality of their professional relationships (with colleagues, students, the community and others), their commitment to learning as it pertains to their teaching role, and how they act upon goals that matter to them. 6.2.1.1 Model for Faculty Member Engagement in Summative Peer Review of Teaching and Professional Growth in Teaching (Model A) This model integrates the study findings with other research on faculty cultures to offer a framework for the relationship between professional growth in teaching and summative peer review of teaching. 6.2.1.1.1 Institutional, Departmental and Disciplinary Cultures Model A suggests and identifies specific cultural aspects within institutional, disciplinary, and departmental cultures that have a strong influence on whether and how faculty members grow and learn as instructors through summative peer review. For instance, in an institutional 194 culture where research is rewarded over teaching, faculty members are more inclined to select activities that will augment their research productivity and, as a result, they may not invest time to converse with one another about their experiences of teaching during a summative peer review process. Likewise, in an institutional culture where the student evaluations of teaching are given more attention than peer reviews, faculty members may dismiss peer review of teaching as a formality, and consequently, their engagement in summative peer review may be diminished. Both these aspects of institutional culture hinder faculty members\u00E2\u0080\u0099 opportunities for learning from peer review. As it pertains to departmental cultures, Model A illustrates how collegial relationships and leadership from the department head shape the extent to which summative peer review does or does not promote growth and learning about teaching. For example, when the reviewer and candidate respect and value each other\u00E2\u0080\u0099s knowledge and teaching approach, they may be more inclined to discuss the peer review experience; the dialogue can contribute to mutual reflection and growth. Along similar lines, if the department head were to use peer review results to help the candidate identify areas of growth in teaching, and acknowledge areas of strength, s/he would signal that the reviews are relevant to a discussion of career development. In these examples, collegiality and leadership in the department support faculty members\u00E2\u0080\u0099 learning through summative peer review. Finally, as it concerns disciplinary cultures and the summative peer review of teaching, the aspect that appears to have most bearing on professional growth in teaching is the extent to which faculty members desire criteria/standards/objectivity or are comfortable without it. As previously described in this dissertation, the norm of universalism, which relates to the 195 assumption that truth claims will be made on the basis of pre-established, impersonal criteria, is particularly strong in the Sciences. It is not surprising, then, that many participants in the Science departments criticized peer reviews as being highly subjective, and wanted clearer criteria for conducting peer reviews. These faculty members said they would feel better equipped to conduct the reviews and have more confidence in their results if they made use of clear criteria for the evaluation of teaching. This leads me to conclude that they would engage more in the process and consequently learn more from the experience. In Arts, more faculty members were comfortable conducting reviews without pre-established and detailed criteria. 6.2.1.1.2 Academic Values and Norms Just as there are dominant elements of institutional, departmental, and disciplinary culture in effect in the relationship between growth and summative peer review, so there are particular ways that academic values and norms are manifested. Autonomy: The study findings indicate that some academics feel uncomfortable when a peer observes their classroom teaching. This may be because peer review disrupts the tradition whereby teaching is a private activity and may threaten a faculty member\u00E2\u0080\u0099s sense of autonomy (Chism, 2007a; Hutchings, 1996). An element of autonomy that is conducive to growth through summative peer review is an acceptance and valuing of diverse teaching approaches. When faculty members recognize and appreciate that \u00E2\u0080\u009Cgood teaching\u00E2\u0080\u009D is varied, it provides them with greater opportunity to 196 learn from the peer review process. Furthermore, there is less risk that the candidate will be evaluated poorly because s/he and the reviewer have different notions about what constitutes good teaching. Collegiality: In the summative peer review of teaching, collegiality is preserved when peers refrain from sharing critical and/or constructive feedback in verbal exchanges post-review and in any formal documentation about the peer review outcomes. In so doing, faculty members miss out on the opportunity to collectively reflect on the summative peer review experience and, consequently, do not benefit from the learning that would be derived from those conversations. 6.2.1.1.3 O\u00E2\u0080\u0099Meara et al.\u00E2\u0080\u0099s Framework for Professional Growth This study demonstrated that, in summative peer review, professional relationships\u00E2\u0080\u0094more so than commitments and agency\u00E2\u0080\u0094strongly influenced the potential for the process to contribute to professional growth. O\u00E2\u0080\u0099Meara et al. describe that interactions among colleagues can \u00E2\u0080\u009Cstimulate, facilitate, and shape learning\u00E2\u0080\u009D (2008, p. 29). Based on my findings, the characteristics of the professional relationship especially conducive to learning came when: \u00E2\u0080\u00A2 A candidate values a reviewer\u00E2\u0080\u0099s (observer, member of the peer review committee, department head) pedagogical and/or content knowledge; \u00E2\u0080\u00A2 Candidate and reviewer respect each other\u00E2\u0080\u0099s approach to teaching; \u00E2\u0080\u00A2 A candidate does not perceive that the reviewer is \u00E2\u0080\u009Cout to get\u00E2\u0080\u009D him/her; and \u00E2\u0080\u00A2 Reviewer and candidate discuss teaching, and reflect on the peer review process. However, in the context of summative peer review, the findings also indicated that 197 interactions among colleagues could hinder faculty member learning. The potential for learning might be diminished by the absence of the above characteristics or, as described previously, be reduced because faculty members believed that a more certain way of nurturing and sustaining professional interactions is by refraining from sharing constructive feedback which could, potentially, contribute to growth in teaching. In either case, we see that, as identified by O\u00E2\u0080\u0099Meara et al., professional relationships play a key role in faculty members\u00E2\u0080\u0099 learning. 6.2.1.2 Model for Faculty Member Engagement in Departmental Practices that Foster a Culture that Values Teaching and Professional Growth in Teaching (Model B) Model B in Figure 3 integrates the findings from this study with other research on faculty cultures to offer a framework for the relationship between non-evaluative departmental practices that foster a culture that values teaching and faculty members\u00E2\u0080\u0099 professional growth in teaching. Like Model A, this model identifies specific aspects of culture that play out and determine how faculty members engage in departmental practices that can promote professional growth in teaching. 6.2.1.2.1 Institutional, Departmental and Disciplinary Cultures As in Model A, Model B illustrates that the institutional reward system has a strong bearing on how faculty members pursue teaching-related activities. Knowing that research productivity is favoured over teaching in tenure and promotion decisions, faculty members may limit their involvement in activities that can foster their professional growth in teaching. Nevertheless, those committed to their teaching (like the study participants) find ways to 198 pursue activities that promote their growth as teachers. Involvement in such activities is facilitated when the institution demonstrates its commitment to teaching by providing resources at the faculty and department levels for the improvement of teaching. The departmental culture also matters for faculty members\u00E2\u0080\u0099 professional growth. Specifically, actions by the department head influence the range of teaching-related practices available and how faculty members perceive them. For instance, a department head can demonstrate that teaching is valued by holding a department retreat focused on teaching, establishing a process for formative peer review, and/or setting up teaching and learning committees or initiatives to address specific issues. In addition, the quality of the relationships among departmental peers has a bearing on how faculty members engage in practices that foster a culture that values teaching. Professional learning conversations are frequent in departments that faculty members describe as collegial and where peers accept varying notions of good teaching. As before, it is interactions between departmental peers and leadership by the head that appear to be most relevant to the relationship between practices and professional growth in teaching. With respect to disciplinary culture, professional growth is largely affected by whether there is an established tradition of working collectively on teaching issues or whether faculty members tend to pursue their teaching endeavours individually. Because faculty members indicated that they learn a great deal about teaching through conversations with their peers, faculty members who belonged to disciplines with an established tradition of collaboration tended to benefit more often from collective conversations about teaching. 199 6.2.1.2.2 Academic Values and Norms Autonomy: As in Model A, acceptance of and valuing diverse approaches to teaching promotes growth in teaching. In addition, the tradition of working collaboratively or individually on teaching also influences the opportunities for conversations about teaching and the learning that comes from these. Collegiality: When faculty members participate in non-evaluative departmental practices, they engage in learning conversations about teaching; these foster collegiality among departmental peers. These conversations happen most often between colleagues who trust one another and who value each other\u00E2\u0080\u0099s knowledge and approach to teaching; one might conclude, therefore, that collegial relations must be present in order for faculty members to engage in learning conversations. Collegiality, in Model B, promotes learning through dialogue about teaching. 6.2.1.2.3 O\u00E2\u0080\u0099Meara et al.\u00E2\u0080\u0099s (2008) Framework for Professional Growth Model B illustrates that collegial professional relationships within a department are important in determining faculty members\u00E2\u0080\u0099 engagement in practices that can foster professional growth in teaching. However, unlike Model A, commitment and agency also figure prominently in non-evaluative practices. The findings from this study indicate that faculty members learn and grow professionally as instructors because they are committed to their teaching and find ways to pursue their teaching goals by being involved in a range of departmental practices. This is consistent with O\u00E2\u0080\u0099Meara et al.\u00E2\u0080\u0099s framework that identifies agency, commitments, relationships, and learning as key aspects contributing to professional growth. In the context 200 of summative peer review, agency and commitment manifested less strongly. Table 9 summarizes which cultural elements dominate in each of the contexts that potentially promote professional growth in teaching. Table 9: Dominant Element of Culture at Play in Models of Professional Growth in Teaching Cultural Element Growth through Summative Peer Review (Model A) Growth through Non- Evaluative Departmental Practices (Model B) Dominant aspect Institutional Culture Reward system Student evaluations of teaching Reward system Tangible resources that support teaching Departmental Culture Collegiality Leadership from department head Collegiality Leadership from department head Disciplinary Culture Universalism Individual vs. collaborative approaches to teaching Autonomy Diverse approaches to teaching accepted/valued Teaching as private Diverse approaches to teaching accepted/valued Individual vs. collaborative approaches to teaching Collegiality Preserve collegiality by not sharing/documenting constructive feedback Foster collegiality through professional learning conversations O\u00E2\u0080\u0099Meara et al.\u00E2\u0080\u0099s (2008) framework for professional growth Professional relationships Professional relationships Agency Commitment Learning 201 6.3 Recommendations for Practice In this section, I make recommendations directed at three areas of practice: enhancing summative peer review, fostering a department culture of teaching and learning, and increasing adoption and full integration of improved teaching practices. The recommendations are summarized in Table 10. 6.3.1 Summative Peer Review of Teaching This study demonstrated that, in the opinion of the participants, summative peer review is neither a useful process for the evaluation of teaching nor for fostering professional growth in teaching. These findings point to the need to improve both functions of summative peer review so that the value of the process is enhanced for academics and administrators alike. There are multiple levels at which the value of peer review could be reinforced and countless strategies which, if implemented, would result in an improved process. In Appendix H, I present a range of suggestions for an ideal summative peer review of teaching process derived from the study findings. Here, I make recommendations pertaining primarily to the role of the department head and to feedback practices because they are particularly salient to the issue of professional growth. \t\r \u00C2\u00A0 6.3.1.1 Involve the Department Head The department head, through her/his actions, can communicate strong messages about the value (or lack thereof) of summative peer reviews. S/he can endorse peer review as a meaningful process for evaluation and professional growth when: 202 \u00E2\u0080\u00A2 s/he discusses the summative peer review outcomes with the candidate in a face-to- face meeting, \u00E2\u0080\u00A2 s/he demonstrates that both the peer reviews and the student evaluation of teaching scores are given serious consideration in the evaluation of teaching; and \u00E2\u0080\u00A2 s/he initiates a process whereby faculty members collectively produce departmental guidelines for peer review. Although I name the department head in the following recommendations, I maintain that others in leadership roles, such as the associate head, could substitute for the department head and similar benefits would still ensue. Recommendation 1: The department head discusses peer review results with the candidate. The majority of the participants had never discussed the peer reviews with their department head. In a few cases the head had briefly referred to peer reviews in an annual meeting, but none of these participants had benefited from meaningful conversation about the results. The absence of dialogue about peer reviews in private meetings and departmental meetings was commonly interpreted by faculty members to mean that the peer reviews were of little to no significance in the evaluation of teaching or for growth. The department head can signal that peer reviews matter for evaluation and growth when s/he discusses the results with the candidate at an annual meeting or shortly following the peer reviews. Discussions that include an explanation of how the peer review results will be used in tenure, promotion, and merit decisions would help the candidate better understand how the 203 reviews are used in career advancement decisions. If, in addition, the department head asks the candidate to formulate and share a professional growth plan for teaching that considers peer review results and those from the student evaluations of teaching, the candidate would see that this process also serves to promote individual growth. Recommendation 2: The department head considers peer reviews and student evaluations of teaching more equally. Whether at departmental meetings, standing committee meetings, or in private interactions between the department head and candidate, most participants agree that when it comes to the evaluation of teaching, administrators (and consequently faculty members themselves) give greater consideration to student evaluations than they do peer reviews. Student evaluation of teaching scores are preferentially used over peer reviews to gauge a candidate\u00E2\u0080\u0099s success in teaching and/or to judge how the department, on average, fares in comparison with the faculty as a whole. Department heads can emphasize the relevance and value of peer reviews by referring not only to the student evaluation of teaching results but also to the reviews in individual and collective discussions about the evaluation of teaching. The head could conceivably gather information from all the peer reviews within a department, anonymize it, and present it to the faculty members so they could then identify issues that need to be addressed at the department level. Dissemination of peer review results would help faculty members appreciate that peer reviews can have broad application and that the process is linked to wider educational development efforts (Hammersley-Fletcher & Orsmond, 2004). 204 In making the above suggestions, I recognize that in order for department heads to shift greater attention to peer reviews, they need to believe that senior administrators and other non-departmental faculty members charged with reviewing a candidate\u00E2\u0080\u0099s file will also value information from the peer reviews. In the current culture, where student evaluations of teaching have strong currency, it is not surprising that a department head who seeks to make a solid case for a candidate will want to communicate using measures (e.g., student evaluations of teaching) that can be easily understood and interpreted by other evaluators. Recommendation 3: Department head clarifies expectations and enhances awareness of summative peer review process through departmental guidelines. Many participants did not know what to expect from the peer review process or felt their expectations were unmet. The department head can clarify expectations by appointing a committee to develop guidelines for the summative peer review of teaching. These guidelines would help reviewers understand their duties and let candidates know what to expect. They might include such things as how reviewers are selected, timing of reviews, what will be reviewed, and the purpose of any communication, pre- and post-review. Guidelines should also include criteria for good teaching so that evaluators and candidates know what is being evaluated and with what standards. These criteria exist in the literature, but in order to be useful, they must be adapted to a department\u00E2\u0080\u0099s particular values (Chism, 2007a) which may be uncovered via departmental conversations about good teaching. In departments where faculty members resist using a structured form (with criteria) in the peer review process, only broad criteria would be included. To have a greater chance of being accepted, drafts of these 205 guidelines should be circulated among faculty members for feedback and later be presented in department meetings for final acceptance. The benefits of these criteria and standards would be lost, however, if faculty members were not aware of them or did not use them in the peer review process. Consequently, the department head plays an important role, not only in initiating discussions that result in the creation of guidelines, but also in ensuring that the guidelines are accessible (e.g., posted on the departmental website, included in a policy manual, added to resources given to new faculty members and mentors, circulated via email) and referred to regularly in conversations about teaching. 6.3.1.2 Improve the Feedback Process The findings from this study indicate that if summative peer review is to be used for the enhancement of teaching, the feedback process must be improved. Recommendation 4: Share written and/or verbal feedback with the candidate. In many cases, participants did not receive feedback on their teaching from either reviewers or their department head. While it is true that the peer review process can prompt self- reflection which leads to teaching improvement (Hatzipanagos & Lygo-Baker, 2006; Lomas & Nicholls, 2005), research has shown that feedback is more effective when the information comes from others as well, and is communicated in a variety of ways (e.g., written and verbally) (Brinko, 1983; Harms & Roebuck, 2010). 206 Recommendation 5: Ensure that feedback is delivered by a credible source. The feedback literature reports that in order to be effective, feedback is best delivered by someone who is perceived as credible, where credibility encompasses being \u00E2\u0080\u009Cknowledgeable enough to make an accurate judgement\u00E2\u0080\u009D (Brinko, 1993, p. 577). The degree to which this is possible relies partly on what the candidate values in a reviewer. The credibility of a reviewer would likely be enhanced if s/he \u00E2\u0080\u00A2 were familiar with and used departmental criteria for evaluating good teaching \u00E2\u0080\u00A2 were perceived by the candidate as a good teacher \u00E2\u0080\u00A2 had content knowledge appropriate to the course being reviewed \u00E2\u0080\u00A2 had training in peer review and strong communication skills. Recommendation 6: Explain the value of receiving feedback on content and teaching approaches. Reviewer feedback on both the teaching process and content may help a candidate grow and learn through peer review (Kell & Annetts, 2009; Toth & McKey, 2010a). In this study, some participants clearly valued pedagogical knowledge over content knowledge, whereas others valued the latter over the former. The candidates and reviewers should be educated about the relative merits of receiving feedback on content and teaching approaches so that they can understand\u00E2\u0080\u0094and therefore better appreciate\u00E2\u0080\u0094the value of each. 207 6.3.2 Fostering a Departmental Culture that Values Teaching Recommendation 7: Build and maintain a collegial culture. Participants\u00E2\u0080\u0099 narratives pointed out that they were more likely to converse about teaching and learning with peers whom they trust and whose opinions they value. This was true whether the conversations were with an officially appointed person, such as the undergraduate chair, or with a colleague in a neighbouring office. The foundations upon which trust were built varied depending on the individual; a shared career stage, approach to teaching, disciplinary background, nationality, and gender were all named as factors upon which trust was initially based. Consequently, diversity in a department can provide an environment where faculty members may have more opportunities to connect with others they perceive as trustworthy. Over time, however, it is the multiple interactions that colleagues have with one another that confirm or refute the perception that a colleague is well-intentioned, respectful, supportive, and worthy of trust. When a department provides multiple opportunities for conversations about teaching, faculty members may be exposed to a wider variety of perspectives about teaching and learning and may learn to value a broader range of opinions, beliefs, and knowledges. In addition to efforts that faculty members themselves make to develop a respectful, supportive atmosphere, the department head (or associate head or other departmental leader) can also promote collegiality. S/he can do so by publicly recognizing faculty members\u00E2\u0080\u0099 efforts to work together in teaching, research, and service; by including faculty from all ranks in decision-making and other activities to do with department matters (while being mindful 208 of not putting excessive demands on the pre-tenured faculty members\u00E2\u0080\u0099 time); and by holding an annual retreat and/or initiating other activities to encourage relationship building and discussion of commons problems (Bensimon et al., 2000). 6.3.2.1 Increase Adoption and Full Integration of Improved Teaching Practices With one exception, the participants did not refer to the university-level document on summative peer review of teaching. Many acknowledged that there were increased requirements for summative peer review but were not sure from where the impetus for these came. These findings suggest that faculty members were not aware of the university document. Accordingly, one might conclude that this document was having minimal impact on summative peer review practices. The recommendations that follow address the (teaching) policy-practice gap which is a widespread challenge in higher education (Harvey & Kamvounias, 2008; Kezar, 2011; Newton, 2003). They are meant to increase the uptake and integration of improved teaching practices by faculty members and other members of the higher education teaching community. Recommendation 8: Help faculty members understand how the proposed new practice improves teaching. Three common and related reasons that people do not engage in change practices is that they do not understand the need for change and (consequently) they are not motivated or interested in making a change; additionally, they may perceive the change as threatening. Through deliberation and discussion, however, faculty members can learn and grow their understanding of how a particular practice is relevant to them and meets their professional 209 and personal needs. Informed deliberations can help faculty members challenge current notions, may shape norms and beliefs, and may allow faculty members to exchange ideas and address issues of implementation and motivation (Kezar, 2011). Recommendation 9: Provide tangible support for sustaining change related to teaching policies. Resources\u00E2\u0080\u0094human, financial, time\u00E2\u0080\u0094are necessary to sustain change. Thus, even when faculty members accept new ways of doing things, they require support. For example, with regard to the adoption of summative peer review practices outlined in the university document, faculty members might benefit from addressing questions and concerns to a coordinator/coordinating committee, receiving training on how to conduct peer reviews, and having time to deliberate about criteria for good teaching. However, as long as an institution continues to reward research above teaching, faculty members will uphold values, practices, and assumptions that prioritize research. Consequently, when administrators attempt to implement policies for the improvement of teaching, faculty members may not broadly adopt these. For teaching and learning policies to shift from being \u00E2\u0080\u009Cpassively ignored\u00E2\u0080\u009D (Arreola, 2007) to becoming \u00E2\u0080\u009Ctransformative\u00E2\u0080\u009D (O\u00E2\u0080\u0099Meara, 2011), research-intensive institutions need to better align the reward systems with the institutional rhetoric about the importance of teaching and student learning. Doing so will require a significant shift in values. 210 Table 10: Summary of Recommendations for Practice Broad area of recommendation Sub-category Recommendation Enhance summative peer review of teaching Involve the department head 1. The department head discusses peer review results with the candidate. 2. The department head considers peer reviews and student evaluations of teaching more equally. 3. Department head clarifies expectations and enhances awareness of summative peer review process through departmental guidelines Improve the feedback process 4. Share written and/or verbal feedback with the candidate. 5. Ensure that feedback is delivered by a credible source. 6. Explain the value of receiving feedback on content and teaching approaches. Foster a culture that values teaching n/a 7. Build and maintain a collegial culture. Increase adoption and integration of improved teaching practices n/a 8. Help faculty members understand how the proposed new practice improves teaching. 9. Provide tangible support for sustaining change related to teaching policies. 211 6.4 Recommendations for Future Research This study explored faculty members\u00E2\u0080\u0099 experiences and understandings of summative peer review of teaching, and inquired into the relationship between this process and professional growth in teaching. It also incorporated an examination of departmental practices that foster a culture of teaching and promote academics\u00E2\u0080\u0099 growth in their teaching roles. The findings of this study suggest that applying the lens of academic culture is helpful for revealing aspects of institutional, departmental, and disciplinary culture, including academic values and norms, that support or hinder professional growth in teaching. Results of this study suggest four areas for future research: further study of summative peer review, studies that examine a more diverse faculty member population, studies of the role of the department head in promoting professional growth in teaching, and studies that examine how pre-tenured faculty members\u00E2\u0080\u0099 shape a culture of teaching. All of these (summarized in Table 11) would provide valuable data to continue to develop and test the conceptual framework. 6.4.1 Research on the Summative Peer Review of Teaching The paradox of summative peer review is expressed as a belief among faculty members that the goals of formative and evaluative functions of peer review of teaching should be kept separate (because, when combined, neither can be well achieved) and, concurrently, a disappointment that the summative peer review process does not contribute more significantly to professional growth. While faculty members acknowledged the primarily evaluative purpose of summative peer review, they also hoped to learn about their teaching through the process of peer review. This finding points out the need for additional research in 212 ways to design a feasible summative peer review of teaching process that meets faculty members\u00E2\u0080\u0099 needs for evaluation of teaching and professional growth. There would also be a need to evaluate whether such a process was meeting its goals over time. A second area of research into summative peer review could be a longitudinal study that would probe the relationship between changes in the institutional culture of teaching and changes in faculty members\u00E2\u0080\u0099 understandings and experiences of peer review. Since many research-intensive institutions are placing an increased emphasis on teaching (Chalmers, 2010; MacFarlane, 2011), a trend also noted by the participants, a longitudinal study might allow for a stronger understanding of the effects of institutional culture (in particular, public statements and reward policies to do with teaching and learning) on faculty members\u00E2\u0080\u0099 perceptions of summative peer review of teaching. The current study also suggests a third area of research that would involve a more systematic and intentional examination of how the four aspects of professional growth outlined in O\u00E2\u0080\u0099Meara et al.\u00E2\u0080\u0099s (2008) framework (learning, agency, professional relationships, and commitment) manifest in the summative peer review of teaching. In conducting this study, I was especially interested in\u00E2\u0080\u0094and therefore attuned to\u00E2\u0080\u0094displays of collegiality (e.g., professional relationships) and, as a result, may have missed opportunities to probe the other aspects of professional growth. A possible research direction for a study on summative peer review of teaching and professional growth would explicitly seek evidence for all four aspects of the framework. 213 6.4.2 Future Research on Departmental Culture that Captures Greater Faculty Member Diversity When it comes to studying department cultures and teaching, future research focused on the experiences of women and non-tenure track faculty members would expand our understanding of the influence of gender and appointment. One of the most significant demographic changes among faculty members in the past two decades has been the increasing presence of women (Gappa et al., 2007; Robinson, 2006; Schuster & Finkelstein, 2006). This trend has implications for future studies of professional growth and academic culture. Firstly, women faculty who enter formerly homogenous academic environments often endure more social isolation, have greater difficulty identifying mentors to guide them in their careers (Gappa et al., 2007), and receive less career help from their colleagues (Gersick, Bartunek, & Dutton, 2000; Gibson, 2006). Secondly, there is research that has found that women spend more time teaching (Probert, 2005) and may show a preference for certain teaching approaches and methods (more interaction with students, greater use of diversity, employ collaborative learning techniques more often) compared with their male counterparts (Campbell, 2003, \u00C2\u00B6 10; Umbach, 2006). Thus, women experience academic culture differently than men and may have different inclinations toward teaching. Consequently, future research that focuses on female academics and their professional growth in teaching, or a comparative study of men and women, could provide a wealth of data on how gender, professional growth in teaching, and academic culture intersect. As it pertains to the broad topic of \u00E2\u0080\u009Cdiversity,\u00E2\u0080\u009D another well-established trend in North 214 American higher education is the steep increase in positions that are ineligible for tenure (Gappa et al., 2007; Hamilton, 2007; Robinson, 2006; Schuster & Finkelstein, 2006). A second diversity-related area that could be explored is how faculty members with non- traditional appointments learn and grow as teachers through collegial relationships. As previously stated, these relationships can be significant in fostering professional growth in teaching, yet the literature indicates that contingent faculty members are discontent with their capacities to develop campus-based relationships with students and colleagues (O\u00E2\u0080\u0099Meara et al., 2008). An examination into the ways that departmental relationships foster or hinder professional growth in teaching among contingent faculty could be an extension of this study. Finally, increased diversity in terms of departmental, disciplinary, and institutional representation would help test and develop the conceptual model. The participants in this study (with the exception of those in one Science department) described belonging to strong teaching departments with a productive collegial culture. Future research might use mixed methods, including questionnaires that allow participants to respond to questions anonymously, to reach individuals from a wider range of cultures. Some of these cultures might be considered less strong when it comes to teaching and/or not collegial. Furthermore, while there was some diversity in the disciplinary cultures represented, more attention to faculty members\u00E2\u0080\u0099 disciplinary affiliations would contribute to our understanding of how this aspect of culture influences faculty members\u00E2\u0080\u0099 engagement in summative peer review and other teaching practices. 215 6.4.3 Research on the Role of the Department Head in Promoting Professional Growth in Teaching The results of this study have signaled that the department head has a significant role in promoting professional growth in teaching. Since this study included the perspective of only one department head and did not explicitly set out to ask about the role of the head in promoting a culture of teaching, a future study could focus on the function of the head by interviewing department heads only and/or posing questions that pertained directly to that function. Any such study could be further strengthened if it made connections between how department heads enact their roles and the institutional/disciplinary cultures in which they perform their duties. An additional aspect of this proposed research would involve a close examination of how the department head interacts with university level policies on teaching. That is, future studies could examine how the department head utilizes university policies and/or rhetoric about teaching to strengthen the culture of teaching within his/her department. 6.4.4 Research on the Role of Pre-tenured Faculty Members\u00E2\u0080\u0099 in Shaping a Culture of Teaching An extensive body of research has examined the socialization of new faculty members. This literature typically describes a process whereby new faculty members enter a department and over time learn and adapt to the departmental processes (Jawitz, 2009; Trowler & Knight, 2000). The adaptation process requires new faculty to make multiple adjustments and is frequently characterized as a period of high stress and anxiety (Austin, 2010; Simmons, 2011). Findings from this study indicate that new faculty members located in the same department and hired within the same year or within a few years of one another can shape 216 and change the departmental culture of teaching in important ways. For example, as new faculty members question existing practices, they may spur more senior faculty members to reflect on why these practices exist and whether they need to be revisited. Additionally, new faculty may be more willing to take part in innovative (to the department) practices because they are unaccustomed to the departmental norms. Further investigations into the experiences of pre-tenured faculty members, as they pertain to their influence on the departmental culture of teaching, could expand our understanding of, and provide an alternative perspective on, the power of faculty members at this rank in fostering a departmental culture of teaching. Table 11: Summary of Recommendations for Future Research Broad area of future research Focused area of future research (worded as a potential question) Summative peer review of teaching 1. What are the features of a (feasible) summative peer review of teaching process that meets faculty members\u00E2\u0080\u0099 needs for evaluation and professional growth in teaching? 2. How do faculty members\u00E2\u0080\u0099 perceptions of summative peer review change (if at all) over time as an institution puts more emphasis on enhancing teaching? 3. How do learning, agency, commitment and professional relationships manifest and interact with one another in faculty members\u00E2\u0080\u0099 experiences of summative peer review of teaching? Department culture with a focus on diversity 4. How do women and men\u00E2\u0080\u0099s experiences of professional growth in teaching differ within the context of their department? 5. What are non-tenure track faculty members\u00E2\u0080\u0099 experiences of departmental collegiality and how does this impact upon their professional growth as teachers? 6. In departments where peers do not enjoy a strong sense of collegiality, what is the role of professional relationships in fostering professional growth in teaching? 7. In departments where teaching is devalued, how do faculty members with a commitment to their professional growth as teacher, grow and learn as instructors? 217 Broad area of future research Focused area of future research (worded as a potential question) (continued) Department culture with a focus on diversity (continued) 8. What is the relationship between disciplinary culture and professional growth in teaching? Role of the department head 9. What is the role of the department head in fostering professional growth in teaching? Role of pre- tenured faculty members 10. In what ways do pre-tenured faculty members shape the departmental culture of teaching? 6.5 Closing Comments I began this study with the intention of learning about faculty members\u00E2\u0080\u0099 experiences of summative peer review. Stepping into this conversation catapulted me into wide territory: I soared over the terrain of department culture, was flung into the region of academic norms, and glided into the realm of collegiality. Perhaps unsurprisingly, it was through my conversations with the individual participants that I grasped the contribution that conversations make to faculty members\u00E2\u0080\u0099 professional learning as teachers. This (seemingly) ordinary and common act has much potential for promoting growth among academics. The presence or absence of these collegial conversations, I discovered, is sometimes contrived, sometimes spontaneous, but always impactful. Much of what I learned during this study enhanced my appreciation for those individuals who pursue their commitment to teaching within the context of a research-intensive university. I refer here to the individual faculty members, department leaders, university administrators, and educational developers who work to enhance teaching and learning in an 218 environment that does not consistently welcome such initiatives. In my opinion, they are brave and dedicated souls whose efforts, over time, are having some consequence. 219 References Acker, S. & Armenti, C. (2004). Sleepless in academia. Gender in Education, 16(1), 3\u00E2\u0080\u009324. doi:10.1080/0954025032000170309 Ackerman, D., Gross, B. L., & Vigneron, F. (2009). Peer observation reports and student evaluations of teaching: Who are the experts? Alberta Journal of Educational Research, 55(1), 18\u00E2\u0080\u009339. Retrieved from http://ajer.synergiesprairies.ca/ ajer/index.php/ ajer/index Allan, E. J. (2011). Women\u00E2\u0080\u0099s status in higher education: Equity matters. ASHE Higher Education Report, 37(1). doi:10.1002/aehe.3701 Altbach, P. (2002). How are faculty faring in other countries? In R. Chait (Ed.), The questions of tenure (pp. 160\u00E2\u0080\u0093181). Cambridge, MA: Harvard University Press. Altman, H. (2004). A baker\u00E2\u0080\u0099s dozen: Dirty lessons I have learned in an academic career. Change, 36(4), 50\u00E2\u0080\u009353. doi:10.1080/00091380409604977 Alvesson, M., & Sveningsson, S. (2008). Changing organizational culture: Cultural change work in progress. New York, NY: Routledge. Anderson, M., Scott, G., & Coates, H. (2008, September). Leadership challenges for university heads. Paper presented at the British Educational Research Association Conference, Edinburgh, UK. Arreola, R. (2007). Developing a comprehensive faculty evaluation system (3rd ed.). Bolton, MA: Anker Publishing Company, Inc. Austin, A. E. (2010). Expectations and experiences of aspiring and early career academics. In L. McAlpine & G. \u00C3\u0085kerlind (Eds.), Becoming an academic: International Perspectives (pp. 18\u00E2\u0080\u009344). New York, NY: Palgrave Macmillan. 220 Barbour, R. (2008). Introducing qualitative research: A student's guide to the craft of doing qualitative research. Thousand Oaks, CA: Sage. Barrett, L., & Barrett, P. (2010). Women and academic workloads: Career slow lane or cul- de-sac? Higher Education: The International Journal of Higher Education and Educational Planning, 61(2), 141\u00E2\u0080\u0093155. doi:10.1007/s10734-010-9329-3 Becher, T. (1989). Academic tribes and territories: Intellectual enquiry and the cultures of disciplines. Philadelphia, PA: Society for Research into Higher Education & Open University Press. Becher, T. (1994). The significance of disciplinary differences. Studies in Higher Education, 19(2), 151. doi:10.1080/03075079412331382007 Becher, T., & Trowler, P. R. (2001). Academic tribes and territories: Intellectual enquiry and the culture of disciplines (2nd ed.). Philadelphia, PA: Society for Research into Higher Education & Open University Press. Bensimon, E. M., Ward, K., & Sanders, K. (2000). The department chair's role in developing new faculty into teachers and scholars. Bolton, MA: Anker Publishing Company, Inc. Berdrow, I. (2010). King among kings: Understanding the role and responsibilities of the department chair in higher education. Educational Management Administration & Leadership, 38(4), 499-514. doi:10.1177/1741143210368146 Bergquist, W. H., & Pawlak, K. (2008). Engaging the six cultures of the academy. San Francisco, CA: Jossey-Bass. Bernstein, D. J., J. Jonson, & K. L. Smith. (2000). An examination of the implementation of peer review of teaching. New Directions for Teaching and Learning, 83(Fall), 73\u00E2\u0080\u009386. doi:10.1002/tl.8306 221 Biagioli, M. (2002). From book censorship to academic peer review. Emergences: Journal for the Study of Media & Composite Cultures, 12, 11\u00E2\u0080\u009345. doi:10.1080/ 1045722022000003435 Billett, S. (2001). Learning through working life: Interdependencies at work. Studies in Continuing Education, 23(1), 19\u00E2\u0080\u009335. doi:10.1080/01580370120043222 Blackmore, J. (2005). A critical evaluation of peer review via teaching observation within higher education. International Journal of Educational Management, 19(3), 218\u00E2\u0080\u0093232. doi:10.1108/09513540510591002 Borman, K. M., Clarke, C. Cotner, B., & Lee R. (2006). Cross-case analysis. In J. L. Green, G. Camilli, & P. B. Elmore (Eds.), Handbook of complementary methods in education research (pp. 123\u00E2\u0080\u0093139). Mahwah, NJ: Lawrence Erlbaum Associates. Bovill, C. (2008). Peer observation of teaching guidelines. Retrieved from the University of Glasgow Teaching and Learning Centre website: http://www.gla.ac.uk/ services/learningteaching/contactus/drcatherinebovill/ Braskamp, L. (2000). Towards a more holistic approach to assessing faculty as teachers. New Directions for Teaching and Learning, 83(Fall), 19\u00E2\u0080\u009333. doi:10.1002/tl.8303 Braxton, J. M. (2008). Toward a theory of faculty professional choices in teaching that foster college student success. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (pp. 181\u00E2\u0080\u0093207). New York, NY: Agathon Press. doi:10.1007/978-1- 4020-6959-8_6 Braxton, J. M. (2010). Norms and the work of colleges and universities: Introduction to the special issue\u00E2\u0080\u0094Norms in academia. The Journal of Higher Education, 81(3), 243\u00E2\u0080\u0093 250. doi:10.1353/jhe.0.0097 222 Brenner, M. E. (2006). Interviewing in educational research. In J. L. Green, G. Camilli, & P. B. Elmore (Eds.), Handbook of complementary methods in education research (pp. 357\u00E2\u0080\u0093 370). Mahwah, NJ: Lawrence Erlbaum Associates. Brinko, K. T. (1983). The practice of giving feedback to improve teaching: What is effective? The Journal of Higher Education, 64(5), 574\u00E2\u0080\u0093593. Retrieved from http://www.jstor.org/stable/2959994 Bronstein, P., & Ramalay, J. (2002). Making the persuasive tenure case: Pitfalls and possibilities. In J. Cooper & D. Stevens (Eds.), Tenure in the sacred grove (pp. 31\u00E2\u0080\u0093 55). Albany, NY: SUNY Press. Bunch, K. (2007). Training failure as a consequence of organizational culture. Human Resource Development Review, 6(2), 142\u00E2\u0080\u0093163. doi:10.1177/1534484307299273 Byrne, J., Brown, H., & Challen, D. (2010). Peer development as an alternative to peer observation: a tool to enhance professional development. International Journal for Academic Development, 15(3), 215\u00E2\u0080\u0093228. doi:10.1080/1360144X.2010.497685 Caffarella, R. & Zinn, L. (1999). Professional development for faculty: A conceptual framework for barriers and supports. Innovative Higher Education, 23(4), 241-254. Retrieved from http://www.ingentaconnect.com/content/klu/ ihie/1999/00000023/00000004 Campbell, K. (2003). Personal, political and pedagogical: Female faculty and values-based learning design. Radical Pedagogy, 5(1). Retrieved from http://radicalpedagogy. icaap.org/content/issue5_1/01_campbell.html Cariaga-Lo, L., Worthy Dawkins, P., Enger, R., Schotter, A., & Spence, C. (2010). Supporting the development of the professoriate. Peer Review, 12 (3), 19\u00E2\u0080\u009322. 223 Retrieved from http://www.aacu.org/peerreview/pr-su10/pr-su10_index.cfm. Carter, V. (2008). Five steps to becoming a better peer reviewer. College Teaching, 56(2), 85\u00E2\u0080\u009388. doi:10.3200/CTCH.56.2.85-88 Cavanaugh, R. (1996). Formative and summative evaluation in the faculty peer review of teaching. Innovative Higher Education, 20(4), 235\u00E2\u0080\u0093240. doi:10.1007/BF01185798 Chait, R. (2002). Does faculty governance differ at colleges with tenure and colleges without tenure? In R. Chait (Ed.), The questions of tenure (pp. 69\u00E2\u0080\u0093100). Cambridge, MA: Harvard University Press. Chalmers, D. (2011). Progress and challenges to the recognition and reward of the Scholarship of Teaching in higher education. Higher Education Research & Development, 30(1), 25\u00E2\u0080\u009338. doi:10.1080/07294360.2011.536970 Chism, N. (2004). Characteristics of effective teaching in higher education: Between definitional despair and certainty. Journal of Excellence in College Teaching 15(3), 5\u00E2\u0080\u009336. Retrieved from http://celt.muohio.edu/ject/index.php Chism, N. (2007a). Peer review of teaching: A sourcebook (2nd ed.). Bolton, MA: Anker Publishing Company, Inc. Chism, N. (2007b). Why introducing or sustaining peer review of teaching is so hard, and what you can do about it. Department Chair: A Resource for Academic Administrators, 18(2), 6\u00E2\u0080\u00938. doi:10.1002/dch.20017 Clark, B. R. (1980). Academic culture. New Haven, CT: Yale University, Higher Education Research Group. Clark, B. R. (1984). Perspectives on higher education. Berkeley, CA: University of California Press. 224 Clark, B. R. (1987). The academic life: Small worlds, different worlds. Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching. Clegg, S. (2003). Problematising ourselves: Continuing professional development in higher education. International Journal for Academic Development, 8(1), 37\u00E2\u0080\u009350. doi:10. 1080/1360144042000277928 Clotfelter, C. (2002). Can faculty be induced to relinquish tenure? In R. Chait (Ed.), The questions of tenure (pp. 221\u00E2\u0080\u0093245). Cambridge, MA: Harvard University Press. Cosh, J. (1998). Peer observation in higher education\u00E2\u0080\u0094A reflective approach. Innovations in Education and Teaching International, 35(2), 171\u00E2\u0080\u0093176. doi:10.1080/ 1355800980350211 Courneya, C., Pratt, D. D., & Collins, J. (2008). Through what perspective do we judge the teaching of peers? Teaching and Teacher Education: An International Journal of Research and Studies, 24(1), 69\u00E2\u0080\u009379. doi:http://dx.doi.org/10.1016/j.tate.2007.01.009, Cox, M. (2004). Introduction to Faculty Learning Communities. New Directions for Teaching and Learning, 97(Spring 2004), 5\u00E2\u0080\u009323. doi: 10.1002/tl.129 Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage. Creswell, J. W. (2008). Educational research: planning, conducting, and evaluating quantitative and qualitative research. (3rd ed.). Upper Saddle River, NJ: Merrill Prentice Hall. Dall\u00E2\u0080\u0099Alba, G., & Sandberg, J. (2006). Unveiling professional development: A critical review of stage models. Review of Educational Research, 76(3), 383\u00E2\u0080\u0093412. doi:10.3102/ 00346543076003383 225 Del Favero, M. (2002). Linking administrative behavior and student learning: The learning centered academic unit. Peabody Journal of Education, 77(3), 60\u00E2\u0080\u009384. doi:10.1207/ S15327930PJE7703_4 Donnelly, R. (2007). Perceived impact of peer observation of teaching in higher education. International Journal of Teaching and Learning in Higher Education. 19(2), 117\u00E2\u0080\u0093 129. Retrieved from http://www.isetl.org/ijtlhe/ Egan, T. M. (2008). The relevance of organizational subculture for motivation to transfer learning. Human Resource Development Quarterly, 19(4), 299\u00E2\u0080\u0093322. doi:10.1002/ hrdq.1243 Emery, C., Kramer, T., & Tian, R. (2003). Return to academic standards: A critique of student evaluations of teaching effectiveness. Quality Assurance in Education, 11(1), 37\u00E2\u0080\u009346. doi:10.1108/09684880310462074 Eraut, M. (2000). Non-formal learning and tacit knowledge in professional work. British Journal of Educational Psychology, 70(1), 113\u00E2\u0080\u00931310. doi:1348/000709900158001 Fairweather, J. (2002). The ultimate faculty evaluation: Promotion and tenure decisions. New Directions for Institutional Research, 114(Summer), 97\u00E2\u0080\u0093108. doi:10.1002/ir.50 Fink, D. (2008). Evaluating teaching: A new approach to an old problem. To Improve the Academy: Resources for Faculty, Instructional, and Organizational Development, 26, 3\u00E2\u0080\u009321. Retrieved from http://finkconsulting.info/major-publications/ Flyvbjerg, B. (2006). Five misunderstandings about case-study research. Qualitative Inquiry, 12(2), 219-245. doi:10.1177/1077800405284363 Fonow, M., & Cook, J. (2005). Feminist methodology: New applications in the academy and public polity. Signs, 30(4), 2211\u00E2\u0080\u00932235. doi:10.1086/428417 226 Fontana, A., & Frey, J. H. (2008). The interview: From neutral stance to political involvement. In N. J. Denzin & Y. S. Lincoln (Eds.), Collecting and interpreting qualitative materials (3rd ed., pp. 115\u00E2\u0080\u0093160). Thousand Oaks, CA: Sage. Gappa, J. M., Austin, A. E., & Trice, A. G. (2007). Rethinking faculty work: Higher education's strategic imperative. San Francisco, CA: Jossey-Bass. Gersick, C. J. G., Bartunek, J. M., & Dutton, J. E (2000). Learning from academia: The importance of relationships in professional life. The Academy of Management Journal, 43(6), 1026\u00E2\u0080\u00931044. Retrieved from http://www.jstor.org/stable/1556333 Gibbs, G., Knapper, C., & Piccinin, S. (2008). Disciplinary and contextually appropriate approaches to leadership of teaching in research-intensive academic departments in higher education. Higher Education Quarterly, 62(4), 416\u00E2\u0080\u0093436. doi:10.1111/j.1468- 2273.2008.00402.x Gibson, S. K. (2006). Mentoring of women faculty: The role of organizational politics and culture. Innovative Higher Education, 31(1). doi:10.1007/s10755-006-9007-7 Gizir, S., & Simsek, H. (2005). Communication in academic context. Higher Education, 50(2), 197\u00E2\u0080\u0093221. doi:10.1007/s10734-004-6349-x Gosling, D. (2002). Models of peer observation of teaching. Retrieved from http://www.heacademy.ac.uk/assets/documents/resources/database/id200_Models_of _Peer_Observation_of_Teaching.rtf Gosling, D. (2010). Value commitments and ambivalence in educational development. New Directions for Teaching and Learning, 122(Summer), 91\u00E2\u0080\u0093102. doi:10.1002/tl.401 Gosling, D., & D\u00E2\u0080\u0099Andrea, V. (2001). Quality development: A new concept for higher education. Quality in Higher Education, 7(1), 9\u00E2\u0080\u009317. doi:10.1080/ 227 13538320120045049 Gosling, D., & Ritchie, S. (2003). Research project on peer observation of teaching. Retrieved from The Higher Education Academy Subject Centre for Education ESCalate: http://escalate.ac.uk/resources/peerobservation2/ Gravestock, P., & Greenleaf, E. G. (2008). Overview of tenure and promotion policies across Canada. Retrieved from http://www.malaspina.ca/integratedplanning/documents/ OverviewofTPPoliciesinCanada.pdf Grbich, C. (2007). Qualitative data analysis: An introduction. Thousand Oaks, CA: Sage. Groccia, J. (2010). Why faculty development? Why now? In A. Saroyan & M. Frenay (Eds.), Building teaching capacities in higher education: A comprehensive international model (pp. 1\u00E2\u0080\u009342). Sterling, VA: Stylus Publishing. Haag, P. (2005). Is collegiality code for hating ethnic, racial, and female faculty at tenure time? Education Digest, 71(1), 57-62. Haig, N. (2005). Everyday conversation as a context for professional learning and Development. International Journal for Academic Development, 10(1), 3\u00E2\u0080\u009316 doi:10.1080/13601440500099969 Hamilton, N. (2007). Faculty autonomy and obligation. Academe, 93(1), 36-42. Retrieved from http://www.aaup.org/AAUP/pubsres/academe/ Hammersley-Fletcher, L., & Orsmond, P. (2004). Evaluating our peers: Is peer observation a meaningful process? Studies in Higher Education, 29(4), 213\u00E2\u0080\u0093224. doi:10. 1080/0307507042000236380 Handal, G. (1999). Consultation using critical friends. New Directions for Teaching and Learning, 79(Fall), 59\u00E2\u0080\u009370. doi:10.1002/tl.7907 228 Harding, S. (2004). Rethinking standpoint epistemology: What is \u00E2\u0080\u0098strong objectivity\u00E2\u0080\u0099? In S. Nagy Hesse-Biber & M. Yaiser (Eds.), Feminist perspectives on social research (pp. 39\u00E2\u0080\u009364). New York, NY: Oxford University Press. Hardy, I., & Smith, E. (2006). Contesting tertiary teaching qualifications: An Australian perspective. Teaching in Higher Education, 11(3), 337\u00E2\u0080\u0093350. doi:10.1080/ 13562510600680798 Harms, P., & Britt Roebuck, D. (2010). Teaching the art and craft of giving and receiving feedback. Business Communication Quarterly, 73(4), 413\u00E2\u0080\u0093431. doi:10.1177/ 1080569910385565 Harvey, A., & Kamvounias, P. (2008). Bridging the implementation gap: a teacher-as-learner approach to teaching and learning policy. Higher Education Research & Development, 27(1), 31\u00E2\u0080\u009341. doi:10.1080/07294360701658716 Hatzipanagos S., & Lygo-Baker, S. (2006). Teaching observations: A meeting of minds? International Journal of Teaching and Learning in Higher Education, 17(2), 97\u00E2\u0080\u0093105. Retrieved from http://www.isetl.org/ijtlhe/ Healey, M., & Jenkins, A. (2003). Discipline-based educational development. In H. Eggins & R. Macdonald (Eds.), The scholarship of academic development (pp. 47\u00E2\u0080\u009357). Philadelphia, PA: Society for Research into Higher Education & Open University Press. Hearn, J. (2007). Sociological studies of academic departments. In P. J. Gumport (Ed.), Sociology of Higher Education (pp. 222\u00E2\u0080\u0093265). Baltimore, MD: Johns Hopkins University Press. 229 Hearn, J., & Anderson, M. (2002). Conflict in academic departments: An analysis of disputes over faculty promotion and tenure. Research in Higher Education, 43(5), 503\u00E2\u0080\u0093529. doi:10.1023/A:1020197630478 Henkel, M. (2000). Academic identities and policy change in higher education. London, ENG: Jessica Kingsley. Hodges, L., & Stanton, K. (2007). Translating comments on student evaluations into the language of learning. Innovative Higher Education, 31(5), 279\u00E2\u0080\u0093286. doi:10.1007/ s10755-006-9027-3 Huber, M. (2002). Faculty evaluation and the development of academic careers. New Directions for Institutional Research, 114(Summer), 73\u00E2\u0080\u009383. doi:10.1002/ir.48 Hum, D. (1998). Tenure and pay structures in Canadian universities. Research Monographs in Higher Education (No. 4). Winnipeg, MB: Centre for Higher Education Research and Development. Hutchings, P. (1996). The peer review of teaching: Progress, issues and prospects. Innovative Higher Education, 20(4), 221\u00E2\u0080\u0093234. doi:10.1007/BF01185797 Jacobs, J. A., & Winslow, S. (2004). Overworked faculty: Job stresses and family demands. Annals of the American Academy of Political and Social Science, 596(November), 104\u00E2\u0080\u0093129. doi:10.1177/0002716204268185 Jawitz, J. (2009). Learning in the academic workplace: The harmonization of the collective and the individual habitus. Studies in Higher Education, 34(6), 601\u00E2\u0080\u0093614. doi:10.1080/03075070802556149 Kardia, D., & Wright, M. (2004). Instructor identity: The impact of gender and race on faculty experiences with teaching (CRLT Occasional Paper, No. 19). Retrieved from 230 Center for Research on Learning and Teaching website: www.crlt.umich.edu/ publinks/CRLT_no19.pdf Kell, C., & Annetts, S. (2009). Peer review of teaching embedded practice or policy-holding complacency? Innovations in Education and Teaching International, 46(1), 61\u00E2\u0080\u009370. doi:10.1080/14703290802646156 Kemp, R., & Gosling, D. (2000). Peer observation of teaching. Retrieved from The Higher Education Academy Subject Centre for Education ESCalate: http://escalate.ac.uk/ resources/peerobservation2/ Kezar, A. (2001). Understanding and facilitating organizational change in the 21st century: Recent research and conceptualizations. ASHE-ERIC Higher Education Report, 28(4). doi:10.1002/aehe.2804 Kezar, A. (2011). What is the best way to achieve broader reach of improved practices in higher education? Innovative Higher Education, 36, 235\u00E2\u0080\u0093247. doi:10.1007/s10755- 011-9174-z Kezar, A., & Eckel, P. D. (2002). The effect of institutional culture on change strategies in higher education: Universal principles or culturally responsive concepts? The Journal of Higher Education, 73(4), 436\u00E2\u0080\u0093460. Retrieved from http://www.jstor.org/stable/ 1558422 Kilgore, D., & Cook, M. D. (2007). \u00E2\u0080\u009CIn things and in minds\u00E2\u0080\u009D: Scholarly teachers\u00E2\u0080\u0099 struggles for legitimacy in a research-oriented institution. Journal on Excellence in College Teaching, 18(2), 143\u00E2\u0080\u0093161. Retrieved from: http://celt.muohio.edu/ject/index.php Knight, P., Tait, J., & Yorke, M. (2006). The professional learning of teachers in higher education. Studies in Higher Education, 31(3), 319\u00E2\u0080\u0093339. doi:10.1080/ 231 03075070600680786 Knight, P., & Trowler, P. R. (2000). Department-level cultures and the improvement of learning and teaching. Studies in Higher Education, 25(1), 69\u00E2\u0080\u009383. doi:10.1080/ 030750700116028 Kreber, C., & Castleden, H. (2009). Reflection on teaching and epistemological structure: reflective and critically reflective processes in \u00E2\u0080\u0098pure/soft\u00E2\u0080\u0099 and \u00E2\u0080\u0098pure/hard\u00E2\u0080\u0099 fields. Higher Education, 57(4), 509\u00E2\u0080\u0093531. doi:10.1007/s10734-008-9158-9 Kvale, S. (1996). InterViews. Thousand Oaks, CA: Sage. Lattuca, L., & Stark, J. (1994). Will disciplinary perspectives impede curricular reform? The Journal of Higher Education, 65(4), 401\u00E2\u0080\u0093426. Retrieved from http://www.jstor.org/ stable/2943853 . Lee, J. L. (2007). The shaping of the departmental culture: Measuring the relative influences of the institution and discipline. Journal of Higher Education Policy & Management, 29(1), 41\u00E2\u0080\u009355. doi:10.1080/13600800601175771 Lindblom-Yl\u00C3\u00A4nne, S., Trigwell, K., Nevgi, A., & Ashwin, P. (2006). How approaches to teaching are affected by discipline and teaching context. Studies in Higher Education, 31(3), 285\u00E2\u0080\u0093298. doi:10.1080/03075070600680539 Link, A. N., Swan, C. A., & Bozeman, B. (2008). A time allocation study of university faculty. Economics of Education Review, 27, 363\u00E2\u0080\u0093374. doi:10.1016/j.econedurev. 2007.04.002 Lomas, L., & Nicholls, G. (2005). Enhancing teaching quality through peer review of learning and teaching. Quality in Higher Education, 11(2), 137\u00E2\u0080\u0093149. doi:10.1080/ 13538320500175118 232 Long, T., & Johnson, M. (2000). Rigour, reliability and validity in qualitative research. Clinical Effectiveness in Nursing, 4(1), 30\u00E2\u0080\u009337. doi:10.1054/cein.2000.0106, Lucas, L. (2006). The research game in academic life. Maidenhead, UK: Society for Research into Higher Education/Open University Press. Macfarlane, B. (2011). Prizes, pedagogic research and teaching professors: Lowering the status of teaching and learning through bifurcation. Teaching in Higher Education, 16(1), 127\u00E2\u0080\u0093130. doi:10.108013562517.2011.530756 Mackenzie, N., & Knipe, S. (2006). Research dilemmas: Paradigms, methods and methodology. Issues in Educational Research, 16(2), 193\u00E2\u0080\u0093205. Retrieved from http://www.iier.org.au/iier16/2006conts.html Mamiseishvili, K. (2010). Foreign-born women faculty work roles and productivity at research universities in the United States. Higher Education, 60 (2), 139\u00E2\u0080\u0093156. doi:10.1007/s10734-009-9291-0 Massy, W. F., Wilger, A. K., & Colbeck, C. (1994). Overcoming `hollowed' collegiality. Change, 26(4), 10\u00E2\u0080\u009320. doi:10.1080/00091383.1994.9938496 Maxwell, J. (1992). Understanding and validity in qualitative research. Harvard Educational Review, 63(3), 279-300. Retrieved from http://her.hepg.org/content/ 8323320856251826/ Maxwell, J. A. (2005). Qualitative research design: An interactive approach (Vol. 42, 2nd ed.), Applied Social Research Methods Series. Thousand Oaks, CA: Sage. Mayan, M. J. (2009). Essentials of qualitative inquiry. Walnut Creek, CA: Left Coast Press. Merriam, S. B. (1998). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass. 233 Merriam, S. B. (2002). Qualitative research in practice: Examples for discussion and analysis. San Francisco, CA: Jossey-Bass. Milem, J. F., Berger, J. B., & Dey, E. L. (2000). Faculty time allocation: A study of change over 20 years. The Journal of Higher Education, 71(4), 454\u00E2\u0080\u0093475. Retrieved from http://www.jstor.org/stable/2649148 Mills, M. R., & Hyle, A. D. (2001). No rookies on rookies: Compliance and opportunism in policy implementation. The Journal of Higher Education, 72(4), 453\u00E2\u0080\u0093477. Retrieved from http://www.jstor.org/stable/2672891 Misra, J., Hickes Lundquist, J., Holmes, E., & Agiomavritis, S. (2011). The ivory ceiling of service work. Academe Online, 97(1). Retrieved from http://www.aaup.org/AAUP/ pubsres/academe/2011/JF/ Mitroff, I. I. (1974). Norms and counter-norms in a select group of the Apollo moon scientists: A case study of the ambivalence of scientists. American Sociological Review, 39(4), 579\u00E2\u0080\u0093595. Retrieved from http://www.jstor.org/stable/2094423 Morse, J., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002). Verification strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods, 1(2), 1\u00E2\u0080\u009319. Retrieved from http://www.ualberta.ca/~ijqm/ Murphy, T., MacLaren, I., & Flynn, S. (2009). Toward a summative system for the assessment of teaching quality in higher education. International Journal of Teaching and Learning in Higher Education, 20(2), 226\u00E2\u0080\u0093236. Retrieved from http://www.isetl. org/ijtlhe/ Neumann, A. (2005). Observations: Taking seriously the topic of learning in studies of faculty work and careers. New Directions for Teaching and Learning, 102(Summer), 234 63\u00E2\u0080\u009383. doi:10.1002/tl.197 Neumann, A. (2009). Professing to learn: Creating tenured lives and careers in the American research university. Baltimore, MD: The Johns Hopkins University Press. Neumann, R. (2001). Disciplinary differences and university teaching. Studies in Higher Education, 26(2), 135\u00E2\u0080\u0093146. doi:10.1080 /0307507012005207 1 Neumann, R., Parry, S., & Becher, T. (2002). Teaching and learning in their disciplinary contexts: A conceptual analysis. Studies In Higher Education, 27(4), 405\u00E2\u0080\u009317. doi:10.1080/0307507022000011525 Newton, J. (2003). Implementing an institution-wide learning and teaching strategy: Lessons in managing change. Studies in Higher Education, 28(4), 427\u00E2\u0080\u0093441. doi:10.1080/ 0307507032000122279 O\u00E2\u0080\u0099Hara, R. (2009, May 9). Re: Evaluating collegiality. [Online forum comment]. Retrieved from https://listserv.nd.edu/cgi-bin/wa?A2=ind0905&L=POD&P=R23779. Oleson, V. (2008). Early millennial feminist qualitative research: Challenges and contours. In N. Denzin & Y. Lincoln (Eds.), The landscape of qualitative research (pp. 311\u00E2\u0080\u0093370). London, ENG: Sage. Oloyede, O. (2002). Disciplinary cultures and academic leadership. Society in Transition, 33(1), 116\u00E2\u0080\u0093133. O\u00E2\u0080\u0099Meara, K. (2005). Encouraging multiple forms of scholarship in faculty reward systems: Does it make a difference? Research in Higher Education, 46(5), 479\u00E2\u0080\u0093510. doi:10. 1007/s11162-005-3362-6 O\u00E2\u0080\u0099Meara, K. (2006). Encouraging multiple forms of scholarship in faculty reward systems: Have academic cultures really changed? New Directions for Institutional Research, 235 129 (Spring), 77\u00E2\u0080\u009395. doi:10.1002/ir.173 O\u00E2\u0080\u0099Meara, K. (2011). Inside the panopticon: Studying academic reward systems. In J. C. Smart & M. B. Paulsen (Eds.). Higher education: Handbook of theory and research (pp. 161\u00E2\u0080\u0093220). New York, NY: Agathon Press. doi:10.1002/aehe.3403 O'Meara, K., Terosky, A., & Neumann, A. (2008). Faculty careers and work lives: A professional growth perspective. ASHE Higher Education Report, 34(3). doi:10.1002/aehe.3403 Ornstein, M., & Stewart, P. (2007). Promotion at Canadian universities: The intersection of gender, discipline, and institution. Canadian Journal of Higher Education, 37(3), 1\u00E2\u0080\u0093 25. Ory, J. C. (2000). Teaching evaluation: Past, present, and future. New Directions for Teaching & Learning, 83(Fall), 13\u00E2\u0080\u009318. doi:10.1002/tl.8302 Paulsen, M. B. (2002). Evaluating teaching performance. New Directions for Institutional Research, 114(Summer), 5\u00E2\u0080\u009318. doi:10.1002/ir.42 Peel, D. (2005). Peer observation as a transformatory tool? Teaching in Higher Education, 10(4), 489\u00E2\u0080\u0093504. doi:10.1080/13562510500239125 Pratt, D. (1997). Reconceptualizing the evaluation of teaching in higher education. Higher Education, 34, 23\u00E2\u0080\u009344. doi:10.1023/A:1003046127941 Premeaux, S. R., & Mondy, R. (2002). Tenure's impact. College Teaching, 50(4), 154. doi:10.1080/87567550209595898 Probert, B. (2005). \u00E2\u0080\u0098I just couldn\u00E2\u0080\u0099t fit in\u00E2\u0080\u0099: Gender and unequal outcomes in higher education. Gender, Work and Organization, 12(1), 50\u00E2\u0080\u009372. doi:0.1111/j.1468-0432.2005.00262.x Quinlan, K., & \u00C3\u0085kerlind, G. (2000). Factors affecting departmental peer collaboration for 236 faculty development: two cases in context. Higher Education, 40, 23\u00E2\u0080\u009353. doi:10.1023/A:1004096306094 Ramazanoglu, C., & Holland, J. (2006). Feminist methodology: Challenges and choices. London, ENG: Sage. Rhode, D. (2006). In pursuit of knowledge: Scholars, status, and academic culture. Stanford, CA: Stanford Law and Politics. Rice, E., & Sorcinelli, M. (2002). Can the tenure process be improved? In R. Chait (Ed.), The questions of tenure (pp. 101\u00E2\u0080\u0093124). Cambridge, MA: Harvard University Press. Rice, E., Sorcinelli, M. D., & Austin, A. E. (2000). Heeding new voices: Academic careers for a new generation. Washington, DC: American Association for Higher Education. Ritchie, J, Lewis, J., & Elam, G. (2003). Designing and selecting samples. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students (pp. 77\u00E2\u0080\u0093107). Thousand Oaks, CA: Sage. Robinson, D. (2006). The status of higher education teaching personnel in Australia, Canada, New Zealand, the United Kingdom, and the United States. Retrieved from Canadian Association of University Teachers website: http://www.caut.ca/en/ publications/reports/ei%20study%20final.pdf Rox\u00C3\u00A5, T., & M\u00C3\u00A5rtensson, K. (2009). Significant conversations and significant networks: Exploring the backstage of the teaching arena. Studies in Higher Education, 34(5), 547-559. doi:10.1080/03075070802597200 Rox\u00C3\u00A5, T., M\u00C3\u00A5rtensson, K., & Alveteg, M. (2011). Understanding and influencing teaching and learning cultures at university: a network approach. Higher Education, 62, 99\u00E2\u0080\u0093111. doi:10.1007/s10734-010-9368-9. 237 Schein, E. (2004). Organizational culture and leadership. San Francisco, CA: John Wiley & Sons. Schram, T. H. (2003). Conceptualizing qualitative inquiry. Columbus, OH: Merrill-Prentice Hall. Schuster, J., & Finkelstein, M. (2006). The American faculty: The restructuring of academic work and careers. Baltimore, MD: The Johns Hopkins University Press. Schwalbe, M., & Wokomir, M. Interviewing men. In J. F. Gubrium & J. A. Holstein (Eds.), Handbook of interview research (pp. 203\u00E2\u0080\u0093219). Thousand Oaks, CA: Sage. Seldin, P. (1999). Changing practices in evaluating teaching: A practical guide to improved faculty performance and promotion/tenure decisions. Bolton, MA: Anker Publishing Company, Inc. Serow, R. (2000). Research and teaching at a research university. Higher Education, 40(4), 449\u00E2\u0080\u0093463. doi:10.1023/A:1004154512833 Shanahan, T. (2009). Accountability initiatives in higher education: An overview of the impetus to accountability, its expressions and implications. Retrieved from http://www.yorku.ca/univsec/senate/committees/exec/reports/2008-2009/ Shanahan%20Paper.pdf Shortland, S. (2004). Peer observation: A tool for staff development or compliance? Journal of Further and Higher Education, 28(2), 219\u00E2\u0080\u0093228. doi:10.1080/ 0309877042000206778 Shortland, S. (2010). Feedback within peer observation: Continuing professional development and unexpected consequences. Innovations in Education and Teaching International, 47(3), 295\u00E2\u0080\u0093304. doi:10.1080/14703297.2010.498181 238 Shulman, L. (1993). Teaching as community property. Change, 25(6), 6\u00E2\u0080\u00937. doi:10.1080/ 00091383.1993.9938465 Simmons, N. (2011). Caught with their constructs down? Teaching development in the pre- tenure years. International Journal for Academic Development, 16(3), 229\u00E2\u0080\u0093241. doi:10.1080/1360144X.2011.596706 Sorcinelli, M. D., Austin, A. A., Eddy, P. L., & Beach, A. L. (2006). Creating the future of faculty development. Bolton, MA: Anker Publishing Company, Inc. Smith, K. (2011). Cultivating innovative learning and teaching cultures: A question of garden design. Teaching in Higher Education, 16(4), 427\u00E2\u0080\u0093438. doi:10.1080/ 13562517. 2011. 560374 Snape, D., & Spencer, L. (2003). The foundations of qualitative research. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 1\u00E2\u0080\u009323). Thousand Oaks, CA: Sage. Stake, R. (2005). Qualitative case studies. In N. Denzin & Y. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed., pp. 443\u00E2\u0080\u0093466). Thousand Oaks, CA: Sage. Summerlee, A. J. S., & Christensen Hughes, J. (2010). Pressures for change and the future of university education. In J. Christensen Hughes & J. Mighty (Eds.), Taking stock: Research on teaching and learning in higher education (pp. 243\u00E2\u0080\u0093260). Montreal, PQ: McGill-Queen\u00E2\u0080\u0099s University Press. Suter, W. N. (2006). Introduction to educational research: A critical thinking approach. Thousand Oaks, CA: Sage. Swinglehurst, D., Russell, J., & Greenhalgh, T. (2008). Peer observation of teaching in the online environment: An action research approach. Journal of Computer Assisted 239 Learning , 24(5), 383\u00E2\u0080\u0093393. doi:10.1111/j.1365-2729.2007.00274.x Taylor Huber, M., & Hutchings, P. (2005). The advancement of learning: Building the teaching commons. San Francisco, CA: Jossey-Bass Thomas, G. (2011). A typology for the case study in social science following a review of definition, discourse, and structure. Qualitative Inquiry, 17(6), 511\u00E2\u0080\u0093521. doi:10.1177/ 1077800411409884 Tierney, W. G. (1997). Organizational socialization in higher education. Journal of Higher Education, 68(1), 1\u00E2\u0080\u009316. Retrieved from http://www.jstor.org/stable/2959934 Toth, K. E., & McKey, C. A. (2010a). Differences in faculty development needs: Implications for educational peer review program design. Canadian Journal of Higher Education, 40(1), 53\u00E2\u0080\u009368. Retrieved from http://ojs.library.ubc.ca/index.php/ cjhe Toth, K. E., & McKey, C.A. (2010b). Identifying the potential organizational impact of an educational peer review program. International Journal for Academic Development, 15(1), 73\u00E2\u0080\u009383. doi:10.1080/13601440903529943 Trout, P. (2000). Flunking the test: The dismal record of student evaluations. Academe Online, July-August. Retrieved from http://www.aaup.org/AAUP/pubsres/ academe/2000/JA/Feat/trou.htm Trowler, P. R. (2008). Cultures and change in higher education: Theories and practices. London, ENG: Palgrave Macmillan. Trowler, P. R., & Knight, P. T. (2000). Coming to know in higher education: Theorising faculty entry to new work contexts. Higher Education Research & Development, 19(1), 27\u00E2\u0080\u009342. doi:10.1080/07294360050020453 240 Turner, D. W., III (2010). Qualitative interview design: A practical guide for novice investigators. The Weekly Qualitative Report, 3(2), 7\u00E2\u0080\u009313. Retrieved from http://www.nova.edu/ssss/QR/WQR/qid.pdf Umbach, P. D. (2007). Faculty cultures and college teaching. In R. P. Perry & J. C. Smart (Eds.), The scholarship of teaching and learning in higher education (pp. 263\u00E2\u0080\u0093317). Dordrecht, The Netherlands: Springer Verlag. doi:10.1007/1-4020-5742-3_8 VanWynsberghe, R., & Khan, S. (2007). Redefining case study. International Journal of Qualitative Methods, 6(2), 1\u00E2\u0080\u009310. Retrieved from http://www.ualberta.ca/ ~iiqm/backissues/6_2/vanwynsberghe.pdf Walvoord, B. E., Carey, A. K., Smith, H. L., Soled, S. W., Way, P. K., & Zorn, D. (2000). Academic departments: How they work, how they change. ASHE-ERIC Higher Education Report No. 27(6). doi:10.1002/aehe.3640270801 Warren, J. (2001). Qualitative interviewing. In J. F. H. Gubrium & J. A. Holstein (Eds.), Handbook of interview research: Context & method (pp. 83\u00E2\u0080\u0093101). Thousand Oaks, CA: Sage. Werquin, P. (2007). Terms, concepts and models for analyzing the value of recognition programmes (Declassified OECD Report EDU/EDPC[2007]24). Retrieved Organisation for Economic Co-operation and Development website: http://www. oecd.org/LongAbstract/0,3425,en33873108_33844437_41834712_1_1_1_1,00.html West, C., & Zimmerman, D. H. (1987). Doing gender. Gender and Society, 1(2), 125151. Retrieved from http://www.jstor.org/stable/189945 Winkler, J. (2000). Faculty reappointment, tenure, and promotion: Barriers for women. Professional Geographer, 52(4), 737\u00E2\u0080\u0093750. doi:10.1111/0033-0124.00262 241 Wolcott, H. F. (1994). Transforming qualitative data: Description, analysis, and interpretation. Thousand Oaks, CA: Sage. Yin, R. K. (2009). Case study research: Design and methods. Los Angeles, CA: Sage. Yon, M., Burnap, C., & Kohut, G. (2002). Evidence of effective teaching: Perceptions of peer reviewers. College Teaching, 50(3), 104\u00E2\u0080\u0093110. doi:10.1080/87567550209595887 242 Appendices Appendix A Letter of Initial Contact Dear Dr. [Insert name], For my UBC doctoral research, I am conducting a study in which I seek to learn about faculty members\u00E2\u0080\u0099 understandings and experiences of the summative peer review of teaching giving special attention to the role of gender, rank and departmental/disciplinary culture. Given the increasing emphasis the (Name of Institution) is placing on summative peer review of teaching, as demonstrated by the new institutionally-adopted policies for this process, and the limited empirical research into the experiences of faculty members, there is a need to find out more about summative peer review from the perspective of academics. I am writing to invite you to consider participating. Other full-time tenure track faculty members in your department, and in five other departments across campus, have also been sent this invitation. Participating in my project should not require much of your time. You and I will complete a single interview, approximately 60-75 minutes in length. I will ask permission to contact you subsequently, should I need clarification on anything you said. Once the interview is transcribed, I will send it to you and invite you to review it (if you agree to participate in the interview but have opted not to be audio-recorded, I will send you my detailed notes); you may make corrections, edits, or choose to delete sections, as you wish. Lastly, if something you said is directly quoted in my dissertation, I will share this section of my draft with you so you may see the context in which it was placed. If you have concerns about my use of your words, we can discuss these. I will contact you by phone or email in the coming week to see if you are available for and interesting in participating in this study. You may, of course, email me before I contact you. If you are interested, I will ask you a few questions to help me in sample selection; this conversation should take no more than 10 minutes. Because I am recruiting a certain number of tenured/pre-tenured and female/male faculty members, I will continue recruiting until I have the number of participants needed for the study (a maximum of 24). However, even if I have all the participants I need for the study, I will contact all those who reply to this recruitment letter to explain why I no longer am looking for participants and to offer the opportunity to receive the results of the study. The names of participating individuals will be kept confidential during the research process and in the presentation of the study findings. Participant identities will not be revealed to other study participants or department colleagues. The study has been approved by the UBC Behavioural Ethics Review Board. I would be happy to talk about the project by phone and/or provide details via email. If there is any other information I can provide to help you make a decision about whether or not to participate, please do not hesitate to ask. 243 I look forward to hearing from you soon. Sincerely, Isabeau Iqbal, Doctoral Candidate Department of Educational Studies, UBC Isabeau.iqbal@ubc.ca 244 Appendix B Faculty Member Questionnaire Faculty members\u00E2\u0080\u0099 experiences and understandings of the summative peer review of teaching Your responses to this questionnaire will be used in planning the study and may be used as part of the final reporting. Your name will not be associated with the final results. Should you wish to include additional details in response to any of the questions, please feel free to do so. A. Name: B. Department: C. With which discipline(s) to you associate most closely? D. What is your gender? E. Please indicate how the co-investigator should follow up with you. ____Phone. Phone number ___________________ ____Email. Email address ____________________ Questions: 1. What is your current title? (i.e. associate professor, full professor, instructor, other. Please be as specific as possible.) 2. Is yours a tenure track appointment? \u00C2\u00A8\u00EF\u0082\u00A8 Yes \u00C2\u00A8\u00EF\u0082\u00A8 No 3. Do you currently have tenure? If yes, please indicate the year that you obtained tenured. \u00C2\u00A8\u00EF\u0082\u00A8 Yes Year:_____ \u00C2\u00A8\u00EF\u0082\u00A8 No 4. If you are a pre-tenured professor, when (year) do you think you will apply for tenure? 5. When (year) did you begin working in a tenure-track position at (Name of Institution)? 245 6. Has your teaching been evaluated through the summative peer review of teaching? In this questionnaire, the summative peer review of teaching includes, but is not limited to, formal, in-class observations of teaching conducted by a colleague and used for the purpose of making personnel decisions pertaining to re-appointment, promotion, and tenure. \u00C2\u00A8\u00EF\u0082\u00A8 Yes \u00C2\u00A8\u00EF\u0082\u00A8 No If you answered yes to #6, go to 6a. If you answered no to #6, skip to question 7. 6a. For what purpose did you go through the summative peer review of teaching? Choose as many as apply from the list below and indicate whether the review took place at (Name of Institution): \u00C2\u00A8\u00EF\u0082\u00A8 Reappointment review\u00E2\u0080\u00A6\u00E2\u0080\u00A6\u00E2\u0080\u00A6\u00E2\u0080\u00A6\u00E2\u0080\u00A6\u00E2\u0080\u00A6....At (Name of Institution)? Yes/No \u00C2\u00A8\u00EF\u0082\u00A8 Promotion to associate professor\u00E2\u0080\u00A6\u00E2\u0080\u00A6\u00E2\u0080\u00A6At (Name of Institution)? Yes/No \u00C2\u00A8\u00EF\u0082\u00A8 Promotion to full professor\u00E2\u0080\u00A6\u00E2\u0080\u00A6\u00E2\u0080\u00A6\u00E2\u0080\u00A6\u00E2\u0080\u00A6.At (Name of Institution)? Yes/No \u00C2\u00A8\u00EF\u0082\u00A8 Other, please specify: 7. Have you ever evaluated a colleague on their teaching for the purpose of helping a committee make a decision about tenure, promotion or re-appointment? \u00C2\u00A8\u00EF\u0082\u00A8 Yes (please complete 7a) \u00C2\u00A8\u00EF\u0082\u00A8 No 7a. If you answered yes to Question 7, did you perform the summative peer review of teaching for a (Name of Institution) colleague? \u00C2\u00A8\u00EF\u0082\u00A8 Yes \u00C2\u00A8\u00EF\u0082\u00A8 No The end. Thank-you for completing this questionnaire! Please return this form via email to Isabeau Iqbal (Isabeau.iqbal@ubc.ca) I will call you within one week of receiving this completed questionnaire. 246 Appendix C Follow-up Email to Assistant Professors and Women who had not Responded to Letter of Initial Contact Dear Dr. [Name] A short while ago, I wrote to ask if you would consider participating in my dissertation research on the peer review of teaching. I received a high response from tenured male faculty members in the Faculty of Science; but, given my study\u00E2\u0080\u0099s focus on rank and gender, I am keen to include the voices of pre-tenured faculty and women (at all ranks) from the Department of [Name]. If you can offer me 60-75 minutes of your time for an interview, anytime between now and into the Fall, I would be most grateful. Please contact me via email or by phone (604-822-5811) if you have questions or would like to communicate your decision about participation. If you prefer to decline participation but would like to provide me with information that can help me better understand the barriers to participating, this would be helpful both as I move forward with recruitment and with analyzing my data. Thank-you, Isabeau Isabeau Iqbal PhD Candidate Department of Educational Studies University of British Columbia 247 \t\r \u00C2\u00A0 \t\r \u00C2\u00A0 Appendix D Interview Protocol for Pre-Tenured Faculty Members Context \u00E2\u0080\u00A2 Thank-you for participating in study \u00E2\u0080\u00A2 Re-introduce study (purpose and what I hope to learn) \u00E2\u0080\u00A2 Discuss confidentiality \u00E2\u0080\u00A2 Ask for permission to record \u00E2\u0080\u00A2 Ask if participant has any questions or concerns \u00E2\u0080\u00A2 Request participant\u00E2\u0080\u0099s signature on consent form \u00E2\u0080\u00A2 Provide a copy of the consent form to the participant \u00E2\u0080\u00A2 (Have copy of ethics approval) 1. Please tell me about your involvement with the peer review of teaching. Probe: \u00E2\u0080\u00A2 at which point in appointment the review took place \u00E2\u0080\u00A2 who made up the committee \u00E2\u0080\u00A2 how the committee composition was determined \u00E2\u0080\u00A2 how the process unfolded before/during and after \u00E2\u0080\u00A2 whether and how the individual has been involved as a reviewer in the peer review of teaching 2. What would you say are the primary criteria or expectations for effective teaching in your department? Probe: \u00E2\u0080\u00A2 extent to which the faculty member associates these criteria with the department and/or discipline to which s/he belongs \u00E2\u0080\u00A2 how unique s/he perceives these criteria to be to his/her department and/or discipline 3. Do you think you and your senior colleagues share the same expectations about what effective teaching involves? [If not, why?] 4. What sources of information do you have about the criteria for evaluating teaching in this department? Probe extent to which sources include: \u00E2\u0080\u00A2 hallways conversations \u00E2\u0080\u00A2 departmental documents \u00E2\u0080\u00A2 Faculty documents \u00E2\u0080\u00A2 institutional documents \u00E2\u0080\u00A2 documents specific to the peer review of teaching 248 5. What do you think needs to happen for people to get a positive evaluation through the summative peer review of teaching? Probe \u00E2\u0080\u00A2 what the faculty member thinks would be the most relevant/useful/ethical process for peer review of teaching \u00E2\u0080\u00A2 what s/he believes would be the ideal committee composition \u00E2\u0080\u00A2 what s/he plans to do in future summative peer reviews of teaching based on her/his experience so far. 6. What do you think can lead to a negative evaluation of teaching through the summative peer review of teaching? Probe \u00E2\u0080\u00A2 what is considered inadequate teaching \u00E2\u0080\u00A2 forms of unfairness including interpersonal dynamics \u00E2\u0080\u00A2 lack of mentorship and other support 7. Is there anything else you would like to say about your experience with the peer review of teaching process? Interviewer to verify factual information (Co-investigator will have filled out answers based on the Faculty Member Questionnaire and will check for accuracy verbally): Interviewee Title/Rank: Department Affiliation: Faculty Affiliation: Discipline: Thank-you! Date of interview: __________________________ Time of interview: (start) __________________________(end) Location of interview: __________________________ Gender of interviewee: ___________________________ 249 Appendix E Interview Protocol for Tenured Faculty Members Context \u00E2\u0080\u00A2 Thank-you for participating in study \u00E2\u0080\u00A2 Re-introduce study (purpose and what I hope to learn) \u00E2\u0080\u00A2 Discuss confidentiality \u00E2\u0080\u00A2 Ask for permission to record \u00E2\u0080\u00A2 Ask if participant has any questions or concerns \u00E2\u0080\u00A2 Request participant\u00E2\u0080\u0099s signature on consent form \u00E2\u0080\u00A2 Provide a copy of the consent form to the participant \u00E2\u0080\u00A2 (Have copy of ethics approval) 1. Please tell me about your involvement with the peer review of teaching. Probe: \u00E2\u0080\u00A2 their involvement in committees or otherwise \u00E2\u0080\u00A2 how--if at all--their understanding and experiences of summative peer review have changed over time, throughout their career \u00E2\u0080\u00A2 what might have contributed to changes in their perspectives/beliefs/understandings 2. What would you say are the primary criteria or expectations for effective teaching in your department? Probe: \u00E2\u0080\u00A2 extent to which the faculty member associates these criteria with the department and/or discipline to which s/he belongs \u00E2\u0080\u00A2 how unique s/he perceives these criteria to be to his/her department and/or discipline 3. Do you think your junior colleagues share the same expectations about what effective teaching involves? If not, why? 4. What sources of information do you have about the criteria for evaluating teaching in this department? Probe extent to which sources include: \u00E2\u0080\u00A2 hallways conversations \u00E2\u0080\u00A2 departmental documents \u00E2\u0080\u00A2 Faculty documents \u00E2\u0080\u00A2 institutional documents \u00E2\u0080\u00A2 documents specific to the peer review of teaching 250 5. What would you advise your junior colleagues if you were asked \u00E2\u0080\u009CWhat should I do to up my chances of getting a positive evaluation of teaching through the summative peer review of teaching?\u00E2\u0080\u009D? Probe \u00E2\u0080\u00A2 what the faculty member thinks would be the most relevant/useful/ethical process for peer review of teaching \u00E2\u0080\u00A2 what s/he believes would be the ideal committee composition 6. What do you think can lead to a negative evaluation of teaching through the summative peer review of teaching? Probe \u00E2\u0080\u00A2 what is considered inadequate teaching \u00E2\u0080\u00A2 forms of unfairness including interpersonal dynamics \u00E2\u0080\u00A2 lack of mentorship and other support 7. Is there anything else you would like to say about your experience with the peer review of teaching process? Interviewer to verify factual information (Co-investigator will have filled out answers based on the Faculty Member Questionnaire and will check for accuracy verbally): Interviewee Title/Rank: Department Affiliation: Faculty Affiliation: Discipline: Thank-you! Date of interview: __________________________ Time of interview: (start) __________________________(end) Location of interview: __________________________ Gender of interviewee: ___________________________ 251 Appendix F Informed Consent for Pre-Tenured Faculty Members Faculty members\u00E2\u0080\u0099 experiences and understandings of the summative peer review of teaching Principal Investigator: Thomas Sork, PhD Department of Educational Studies University of British Columbia Phone: 604-822-5702 Email: tom.sork@ubc.ca Co-Investigator: Isabeau Iqbal, MA Doctoral Student Department of Educational Studies University of British Columbia Phone: 604-255-5133 Email: isabeau.iqbal@ubc.ca Background At the (Name of Institution), as in many other North American research-intensive universities, formal in-class observations of teaching (the summative peer review of teaching) are used, along with other information, to evaluate teaching performance for the purpose of making tenure and promotion decisions. Although a vast literature on the peer review of teaching recognizes that academic culture impacts upon faculty members\u00E2\u0080\u0099 experiences of peer review, there is limited empirical research into the influence of gender, rank, departmental culture, and disciplinary culture on faculty members\u00E2\u0080\u0099 understandings and experiences of peer review. Purpose This purpose of this study is to investigate faculty members\u00E2\u0080\u0099 understandings and experiences of the summative peer review of teaching giving special attention to the role of gender, rank and departmental/disciplinary culture. Given the increasing emphasis (Name of Institution) is placing on summative peer review, as demonstrated by the new institutionally-adopted policies for this process, and the limited empirical research into the experiences of faculty members, there is a need to find out more about summative peer review from the perspective of academics. Why am I being asked to participate in this research? You are being invited to take part in this research study because you meet the general criteria of being a full-time pre-tenured faculty member whose teaching may or may not have been evaluated through the summative peer review of teaching at least once during your appointment at (Name of Institution). 252 For this study, pre-tenured and tenured faculty members in departments within the Faculties of Arts and Science will be interviewed. You are being approached in the hope that you will consider making a contribution to our collective understanding of summative peer evaluation of teaching. Study procedures You will be asked to participate in one (1) in-person interview. The interview will take between 60 to 75 minutes, and you will be asked questions about your experiences and understandings of the summative peer review of teaching. For example, you will be asked about your own process of peer review, the expectations around good teaching in your department, and what an ideal summative peer review of teaching process might entail. With your permission, the interview will be audio-taped to help with the data analysis process. If you consent to participate in the individual interview but do not wish to have the interview audio recorded, your request will be accommodated. Risks and Benefits One of the risks associated with this study is the possible discomfort discussing your career, teaching, institution, and collegial relationships. There are no other known risks of the proposed research. If you decide during the interview, or at a later date, that you would like your comments removed from our records, your request will be accommodated. The benefits of participating in this study include having the opportunity to discuss your experiences and understandings of the summative review of teaching. Doing so will allow you to reflect on your identity as a university teacher in relation to the expectations of your institution, department, discipline, and self. The results of this research will help us better understand how gender, rank, departmental culture, and disciplinary culture influence academics\u00E2\u0080\u0099 understandings and experiences of the summative peer review of teaching process in a research-intensive environment. Confidentiality If you consent to participate in this study, your identity will be kept strictly confidential. The completed consent form will be kept separate from data collected in order to protect your identity. All data will be stored in a secure manner in compliance with UBC\u00E2\u0080\u0099s Behavioural Research Ethics Board policies and, as per UBC policy, the files will be kept for five years. The only other people who may have access to the individual data will be my committee members, Dr. Sork, Dr. Poole, and Dr. Metcalfe who are subject to the same terms and conditions of confidentiality outlined in this document. Compensation There is no monetary incentive for participating in this research. 253 How will results be used? The results of this study will be presented in research reports including: (a) my doctoral dissertation for the Department of Educational Studies at UBC, (b) publications and presentations for professional conferences, and (c) articles for professional journals and/or other publications. Contact information about the study If you have any questions or desire further information about this study before or during participation, you can contact Isabeau Iqbal at 604-255-5133 or via email at isabeau.iqbal@ubc.ca. If you have any concerns about your rights as a research subject and/or your experiences while participating in this study, contact the Research Subject Information Line in the University of British Columbia Office of Research Services by e-mail at RSIL@ors.ubc.ca or by phone at 604-822-8598. Participant consent Your consent is required before you may participate in this study; however, your participation in this study is entirely voluntary. You may end the interview and/or withdraw at any time and without giving any reasons for your decision and without any consequences. Please select from the statements below and sign under the one that indicates how you would like to participate in this study. Your signature below indicates that you have received a copy of this consent form for your own records and that you consent to participate in this study and agree to have your interview audio-taped. Participant\u00E2\u0080\u0099s Signature Printed name Date Your signature below indicates that you have received a copy of this consent form for your own records and that you consent to participate in this study but do not want your interview audio-taped. Participant\u00E2\u0080\u0099s Signature Printed name Date 254 Appendix G Informed Consent for Tenured Faculty Members Faculty members\u00E2\u0080\u0099 experiences and understandings of the summative peer review of teaching Principal Investigator: Thomas Sork, PhD Department of Educational Studies University of British Columbia Phone: 604-822-5702 Email: tom.sork@ubc.ca Co-Investigator: Isabeau Iqbal, MA Doctoral Student Department of Educational Studies University of British Columbia Phone: 604-255-5133 Email: isabeau.iqbal@ubc.ca Background At the (Name of Institution), as in many other North American research-intensive universities, formal in-class observations of teaching (the summative peer review of teaching) are used, along with other information, to evaluate teaching performance for the purpose of making tenure and promotion decisions. Although a vast literature on the peer review of teaching recognizes that academic culture impacts upon faculty members\u00E2\u0080\u0099 experiences of peer review, there is limited empirical research into the influence of gender, rank, departmental culture, and disciplinary culture on faculty members\u00E2\u0080\u0099 understandings and experiences of peer review. Purpose This purpose of this study is to investigate faculty members\u00E2\u0080\u0099 understandings and experiences of the summative peer review of teaching giving special attention to the role of gender, rank and departmental/disciplinary culture. Given the increasing emphasis (Name of Institution) is placing on summative peer review, as demonstrated by the new institutionally-adopted policies for this process, and the limited empirical research into the experiences of faculty members, there is a need to find out more about summative peer review from the perspective of academics. Why am I being asked to participate in this research? You are being invited to take part in this research study because you meet the general criteria of being a full-time tenured faculty member at (Name of Institution). If you are a Full professor, you will have conducted the summative peer review of teaching once or more at (Name of Institution) for the purpose of making a personnel decision pertaining to tenure. If you are an associate professor, you may or may not have evaluated a colleague through the peer review of teaching. 255 For this study, pre-tenured and tenured faculty members in departments within the Faculties of Arts and Science will be interviewed. You are being approached in the hope that you will consider making a contribution to our collective understanding of summative peer evaluation of teaching. Study procedures You will be asked to participate in one (1) in-person interview. The interview will take between 60 to 75 minutes, and you will be asked questions about your experiences and understandings of the summative peer review of teaching. For example, you will be asked about your own process of peer review, what an ideal summative peer review of teaching process might entail, and how you might advise junior colleagues on preparing for a successful peer review of teaching. With your permission, the interview will be audio-taped to help with the data analysis process. If you consent to participate in the individual interview but do not wish to have the interview audio recorded, your request will be accommodated. Risks and Benefits One of the risks associated with this study is the possible discomfort discussing your career, teaching, institution, and collegial relationships. There are no other known risks of the proposed research. If you decide during the interview, or at a later date, that you would like your comments removed from our records, your request will be accommodated. The benefits of participating in this study include having the opportunity to discuss your experiences and understandings of the summative review of teaching. Doing so will allow you to reflect on your identity as a university teacher in relation to the expectations of your institution, department, discipline, and self. The results of this research will help us better understand how gender, rank, departmental culture, and disciplinary culture influence academics\u00E2\u0080\u0099 understandings and experiences of the summative peer review of teaching process in a research-intensive environment. Confidentiality If you consent to participate in this study, your identity will be kept strictly confidential. The completed consent form will be kept separate from data collected in order to protect your identity. All data will be stored in a secure manner in compliance with UBC\u00E2\u0080\u0099s Behavioural Research Ethics Board policies and, as per UBC policy, the files will be kept for five years. The only other people who may have access to the individual data will be my committee members, Dr. Sork, Dr. Poole, and Dr. Metcalfe who are subject to the same terms and conditions of confidentiality outlined in this document. Compensation There is no monetary incentive for participating in this research. 256 How will results be used? The results of this study will be presented in research reports including: (a) my doctoral dissertation for the Department of Educational Studies at UBC, (b) publications and presentations for professional conferences, and (c) articles for professional journals and/or other publications. Contact information about the study If you have any questions or desire further information about this study before or during participation, you can contact Isabeau Iqbal at 604-255-5133 or via email at isabeau.iqbal@ubc.ca. If you have any concerns about your rights as a research subject and/or your experiences while participating in this study, contact the Research Subject Information Line in the University of British Columbia Office of Research Services by e-mail at RSIL@ors.ubc.ca or by phone at 604-822-8598. Participant consent Your consent is required before you may participate in this study; however, your participation in this study is entirely voluntary. You may end the interview and/or withdraw at any time and without giving any reasons for your decision and without any consequences. Please select from the statements below and sign under the one that indicates how you would like to participate in this study. Your signature below indicates that you have received a copy of this consent form for your own records and that you consent to participate in this study and to have your interview audio-taped. Participant\u00E2\u0080\u0099s Signature Printed name Date Your signature below indicates that you have received a copy of this consent form for your own records and that you consent to participate in this study but do not want your interview audio-taped. Participant\u00E2\u0080\u0099s Signature Printed name Date 257 Appendix H Range of Participants\u00E2\u0080\u0099 Responses about what Constitutes an Ideal Summative Peer Review of Teaching Process Purpose of summative peer review (why) \u00E2\u0080\u00A2 evaluation of teaching for tenure, promotion and/or reappointment \u00E2\u0080\u00A2 professional growth in teaching \u00E2\u0080\u00A2 multiple purposes advisable (yes/no) \u00E2\u0080\u00A2 add to information provided by student evaluations of teaching Selection of reviewers (who) \u00E2\u0080\u00A2 by department head \u00E2\u0080\u00A2 candidate has input \u00E2\u0080\u00A2 reviewers are located within the department \u00E2\u0080\u00A2 reviewers are located outside of department Timing (when) \u00E2\u0080\u00A2 early in career \u00E2\u0080\u00A2 regular reviews Qualities of reviewer (what) \u00E2\u0080\u00A2 has content knowledge \u00E2\u0080\u00A2 has pedagogical knowledge \u00E2\u0080\u00A2 is perceived as a good teacher by the candidate \u00E2\u0080\u00A2 is trusted by candidate \u00E2\u0080\u00A2 is well-intentioned \u00E2\u0080\u00A2 is trained in conducting peer reviews of teaching \u00E2\u0080\u00A2 accepts and values diverse teaching approaches \u00E2\u0080\u00A2 candidate and reviewer have similar approaches to teaching \u00E2\u0080\u00A2 promotes reflection on teaching \u00E2\u0080\u00A2 senior in rank \u00E2\u0080\u00A2 rank may be above, below, or equal to candidate Aspects of teaching reviewed (what) \u00E2\u0080\u00A2 classroom observation of teaching \u00E2\u0080\u00A2 teaching philosophy statement, course materials \u00E2\u0080\u00A2 undergraduate class, graduate class \u00E2\u0080\u00A2 graduate student supervision \u00E2\u0080\u00A2 honours student supervision \u00E2\u0080\u00A2 student learning 258 Communication (how) \u00E2\u0080\u00A2 process known to candidate and reviewer \u00E2\u0080\u00A2 candidate and reviewers understand how results will be used in decisions about career advancement \u00E2\u0080\u00A2 candidate and reviewers understand how results will be used for professional growth \u00E2\u0080\u00A2 candidate and reviewers understand the relationship between peer reviews and student evaluations of teaching \u00E2\u0080\u00A2 results are communicated to candidate (by department head, by reviewers) \u00E2\u0080\u00A2 results communicated in writing and verbally to candidate \u00E2\u0080\u00A2 reviewer feedback is specific and helps candidate develop as a teacher \u00E2\u0080\u00A2 candidate and reviewer debrief the peer review process \u00E2\u0080\u00A2 use of criteria and standards preferred \u00E2\u0080\u00A2 guidelines for conducting peer reviews preferred \u00E2\u0080\u00A2 flexible and open process preferred (no criteria/standards; no guidelines) \u00E2\u0080\u00A2 candidate knows when reviewer will attend class (yes/no) \u00E2\u0080\u00A2 discussion pre- and post- between candidate and reviewer(s) \u00E2\u0080\u00A2 balanced view presented (candidate\u00E2\u0080\u0099s strengths and areas for improvement) \u00E2\u0080\u00A2 ongoing, informal formative reviews incorporated \u00E2\u0080\u00A2 mindful of faculty members\u00E2\u0080\u0099 time \t\r \u00C2\u00A0 "@en . "Thesis/Dissertation"@en . "2012-11"@en . "10.14288/1.0064685"@en . "eng"@en . "Educational Studies"@en . "Vancouver : University of British Columbia Library"@en . "University of British Columbia"@en . "Attribution-NonCommercial-NoDerivatives 4.0 International"@en . "http://creativecommons.org/licenses/by-nc-nd/4.0/"@en . "Graduate"@en . "Faculty members' professional growth in teaching through the summative peer review of teaching and other departmental practices"@en . "Text"@en . "http://hdl.handle.net/2429/42465"@en .