Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Exploring the transformative potential of public sector innovation labs : assembling a cabinet of curiousities Cole, Lindsay 2021

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
24-ubc_2021_may_cole_lindsay.pdf [ 4.98MB ]
Metadata
JSON: 24-1.0396909.json
JSON-LD: 24-1.0396909-ld.json
RDF/XML (Pretty): 24-1.0396909-rdf.xml
RDF/JSON: 24-1.0396909-rdf.json
Turtle: 24-1.0396909-turtle.txt
N-Triples: 24-1.0396909-rdf-ntriples.txt
Original Record: 24-1.0396909-source.json
Full Text
24-1.0396909-fulltext.txt
Citation
24-1.0396909.ris

Full Text

  EXPLORING THE TRANSFORMATIVE POTENTIAL OF PUBLIC SECTOR INNOVATION LABS: ASSEMBLING A CABINET OF CURIOUSITIES  by Lindsay Cole M.A. Royal Roads University, 2003 B.Sc. University of Victoria, 2000    A DISSERTATION SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Interdisciplinary Studies)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) April 2021      Lindsay Cole, 2021        ii   The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the dissertation entitled:  Exploring the transformative potential of public sector innovation labs: assembling a cabinet of curiousities.  submitted by Lindsay Cole  in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Interdisciplinary Studies  Examining Committee: Moura Quayle, Vice-Provost and Associate Vice-President, Academic Affairs, UBC Co-supervisor Robert VanWynsberghe, Associate Professor, Educational Studies, UBC Co-supervisor  Mark Cabaj, President, Here 2 There Consulting Supervisory Committee Member  Heather Campbell, Director, School of Community and Regional Planning, UBC University Examiner Stephen Sheppard, Director, Collaborative for Advanced Landscape Planning, UBC University Examiner     iii  Abstract There is a proliferation of Public Sector Innovation Labs (PSI labs) around the world, with estimates that more than 500 now exist. They are under-studied as a contemporary innovation construct, making them a potent area for study. The definition of a PSI lab is still contested, although they commonly describe their relationship with government, their topics of focus, and the techniques that they use. The goal or purpose of innovation pursued by these PSI labs is often not described, though most efforts tend to focus on finding efficiencies and improving services for users and tend to operate within dominant governance paradigms. In this dissertation I explore the potential for PSI labs to imagine and catalyze transformative and emergent version of innovation, working at the intersections of personal, organizational and systems scales.  I use a critical qualitative research bricolage, pulling together methodologies that challenge Western ways of knowing, allow for multiple truths to coexist, and invite a researcher with an active role in the research questions. Participatory action research (PAR) and constructivist grounded theory (CGT) are the backbone research methodologies in this bricolage. Research is conducted with co-researchers from three different PSI lab action research sites in Canada, and through interviews with expert practitioners in Canada and Europe.  This dissertation opens up a cabinet of curiousities, rather than proposing definitive conclusions. It is grounded in interdisciplinary theory and has an ambition to be useful and accessible for practitioners. One part of the cabinet offers up a framework to more strongly theorize the work of PSI labs. A second part describes systemic interventions to create stronger enabling conditions for transformative and emergent innovation. A third section focuses on building transformative innovation learning infrastructure and practices. A thread that connects each section considers ways that we might think about measuring and evaluating the impacts and outcomes of PSI labs. Together, this cabinet of curiousities offers researchers and practitioners a plurality of ways to think about a transformative and emergent approach to PSI, and also what PSI might need to become in this time of urgent and complex challenges facing the public sector.     iv  Lay Summary Governments are facing increasing pressure to address complex challenges like climate change, growing inequity, affordability, and many others. Innovation in the public sector is quite urgently and desperately needed, and governments must begin to adapt the paradigms, processes, systems, structures, and tools of their trade to better serve those they are responsible and accountable to. But how might an organization best known for its reliability, consistency, and slow rate of change become more innovative?  There is a global proliferation of Public Sector Innovation Labs (PSI labs) responding to this pressure, although this particular innovation catalyst is still very much in development. This research supports the thinking and practice of these PSI labs, with an aim to increasing their impacts by constructing theory from action research in four areas: strengthening the theorization of public sector innovation; identifying leverage points for systemic interventions for transformative innovation; co-creating transformative learning infrastructure; and evaluation.        v  Preface The identification, design, carrying out, and writing of this dissertation is my original research, with support and mentorship from Moura Quayle, Rob VanWynsberghe, and Mark Cabaj as committee members.  Financial support has been provided by the University of British Columbia (UBC) Four Year Fellowship, the UBC Faculty of Graduate Studies Graduate Award, the UBC Public Scholars Initiative, and the SSHRC Doctoral Fellowship. During this research I was employed by the City of Vancouver, and as such was able to become deeply entangled as an action researcher at this site.  A participatory action research methodology was used, and thus 85 co-researchers have actively engaged with, contributed to, and co-created this research. All data collection, analysis, and writing was completed by me.  Moura Quayle (co-lead), Lily Raphael (research assistant), and Sanmini Koffi (research assistant) were co-designers and co-facilitators of the Solutions Lab community of practice in 2019, and Lily Raphael was also co-designer and co-facilitator of the community of practice in 2018. Maggie Low (research assistant) supported developmental evaluation of the first iteration of the Solutions Lab in the summer of 2017. Their thinking has greatly contributed to this dissertation, and in particular Chapter 6 and Appendix B. The 2019 community of practice was partially funded by a SSHRC Partner Engage Grant, with Moura Quayle as the Principal Investigator.  All original graphics, and graphics adapted from the work of others, were created by Lily Raphael. Lily has been a thought partner in this work and co-imagined how to visualize all of the ideas captured in the original graphics produced through this research. Dawn Smith edited the final dissertation.  Work-in-progress was published as four blog posts on Medium and one report shared on the City of Vancouver website, in order to engage in public scholarship activities and share early    vi  thinking with co-researchers and others. Portions of this writing have been edited and included within this dissertation.  A paper about theorizing public sector innovation labs was presented at the European School of Social Innovation PhD workshop in October, 2019. Portions of this paper have been adapted and re-written into the dissertation, in particular Chapter 4.  It is the authors intention to revise and submit Chapters 4, 5 and 6 for publication in journals, and the chapters have been written with this in mind.  As this dissertation research was coming to a close, the next iteration of transformative innovation learning work was beginning. A SSHRC New Frontiers Research Fund project to build a learning journey for people working in and with Canadian cities on complex decolonization, equity, and climate change challenges began in July 2020. The discussion with this research team, including Maggie Low, Rob VanWynsberghe, Mumbi Maina, Kyla Pascal, Lily Raphael, and Moura Quayle, informed thinking in Chapter 6.  This research received Behavioural Ethics Review Board approval (H17-02282), with Moura Quayle as the Principal Investigator, in April 2018. Sanmini Koffi provided French translation of the consent form and interview questions.       vii  Table of Contents Abstract ............................................................................................................................................... iii Lay Summary ...................................................................................................................................... iv Preface.................................................................................................................................................. v Table of Contents ............................................................................................................................. vii List of Tables .................................................................................................................................... xiii List of Figures ................................................................................................................................... xiv Acknowledgments ......................................................................................................................... xvii Chapter 1: Introduction .................................................................................................................... 1 1.1 Research Context ................................................................................................................... 2 1.2 Research Questions ............................................................................................................... 5 1.3 Overview of Research Methodology and Approach ......................................................... 6 1.4 Dissertation Structure and Chapter Overview .................................................................... 8 1.5 Significance of Contributions ............................................................................................ 11 Chapter 2: Review of Literature................................................................................................... 13 2.1 Exploring Public Sector Innovation ................................................................................... 13 2.1.1 Governance Paradigms and their Relationship to Innovation .................................... 14 2.1.2 Definitions of Public Sector Innovation ......................................................................... 17 2.1.3 Types of Public Sector Innovation ................................................................................. 21 2.1.4 Enabling Conditions for Public Sector Innovation ....................................................... 22 2.1.5 Barriers to Public Sector Innovation .............................................................................. 28 2.2 Public Sector Innovation Labs ........................................................................................... 35 2.2.1 Typologies ........................................................................................................................ 35 2.2.2 Design Approaches ......................................................................................................... 42 2.2.3 Social Innovation Approaches ....................................................................................... 47 2.2.4 Innovation Capacities ...................................................................................................... 53 2.2.5 Evaluation ......................................................................................................................... 56 2.3 Change vs. Transformation Orientation to Innovation ................................................... 65    viii  2.3.1 Personal Transformation ................................................................................................. 66 2.3.2 Organizational Culture Transformation ........................................................................ 73 2.3.3 Systems Transformation .................................................................................................. 79 2.3.4 Connecting Transformation Literature and PSI Lab Practice ...................................... 85 2.4 Analysis of the Literature .................................................................................................... 88 2.4.1 Defining Innovation Purpose .......................................................................................... 90 2.4.2 Theorization ...................................................................................................................... 92 2.4.3 Ambitious Systemic Change .......................................................................................... 94 2.4.4 Innovation Infrastructures and Capacities .................................................................... 95 2.4.5 Power ................................................................................................................................ 97 2.4.6 Durability .......................................................................................................................... 99 2.4.7 Impact and Evaluation ................................................................................................... 100  2.5 In Summary ........................................................................................................................ 104  Chapter 3: Research Methodology ........................................................................................... 108 3.1 Positionality ........................................................................................................................ 110  3.1.1 Individual Positionality .................................................................................................. 110  3.1.2 Role as a ‘Pracademic’ ................................................................................................... 111 3.1.3 Sustainability and Equity ............................................................................................... 112 3.1.4 Decolonization ............................................................................................................... 113  3.1.5 Transformative Innovation ............................................................................................ 113 3.2 Research Bricolage ........................................................................................................... 114 3.2.1 Participatory Action Research ...................................................................................... 115 3.2.2 Constructivist Grounded Theory ................................................................................. 116 3.2.3 CGT and Analysis ........................................................................................................... 118  3.2.4 Modes of Thinking for Qualitative Research .............................................................. 120 3.2.5 Approach to Coding and Analysis ............................................................................... 122  3.3 Summary of Research Sites, Activities, and Timeline .................................................... 129    ix  3.3.1 Research Sites ................................................................................................................ 130  3.3.2 Expert Interviews............................................................................................................ 135 3.3.3 Research Activities and Timeline ................................................................................. 136  3.4 Closing Thoughts on Research Methodology ............................................................... 137 Chapter 4: Stronger Theorization of Public Sector Innovation Labs ................................. 139 4.1 Introduction ....................................................................................................................... 139  4.2 Current Approaches to Theorization of Public Sector Innovation Labs ..................... 140 4.3 The Problem with this Approach to Theorization .......................................................... 142 4.4 Action Research Journey .................................................................................................. 143 4.4.1 City of Vancouver Solutions Lab Action Research ..................................................... 143  4.4.2 Hub and LIUM Action Research ................................................................................... 148  4.5 Expert Interviews and Literature Review ........................................................................ 149 4.5.1 Expert Interviews............................................................................................................ 150 4.5.2 Literature Review Findings ........................................................................................... 151 4.6 A Framework for Stronger Theorization of PSI labs ...................................................... 154 4.7 Action Research Return of Theorizing Innovation Flower ............................................ 160 4.7.1 How the Framework Can Help to More Strongly Theorize PSI ................................ 161 4.7.2 How the Framework Can Be Used to Support Practice ............................................ 162 4.7.3 Upgrades and Changes to Future Iterations of the Framework .............................. 163 4.8 Conclusion ......................................................................................................................... 164  Chapter 5: Systemic Interventions for Transformative Innovation .................................... 166 5.1 Introduction ....................................................................................................................... 166  5.2 Enabling Conditions and Barriers for Public Sector Innovation .................................. 167  5.2.1 Enabling Conditions ...................................................................................................... 168  5.2.2 Barriers ............................................................................................................................ 170 5.3 Expansion of Possibilities from Literature to Reframe Enabling Conditions for PSI .. 172 5.3.1 Governance Paradigms................................................................................................. 172 5.3.2 Change, Transformation, and Emergence ................................................................. 174     x  5.3.3 Dominant Narratives and Potential Reframes About Enabling Conditions for PSI 175 5.3.4 Ways of Thinking About Systemic Interventions ........................................................ 177 5.4 Paradigms, Patterns, and Activities Toward Transformative and Emergent Innovation  ................................................................................................................................................... 181  5.4.1 Paradigms ....................................................................................................................... 181 5.4.2 Patterns ........................................................................................................................... 183  5.4.3 Systemic Intervention Activities ................................................................................... 188  5.5 Discussion .......................................................................................................................... 190  5.6 Conclusion ......................................................................................................................... 192  Chapter 6: Transformative Innovation Learning and Public Sector Innovation Labs .... 194 6.1 Introduction ....................................................................................................................... 194  6.2 Promising Ideas from Literature and Practice ................................................................ 197 6.2.1 Competencies, Capacities, and Capabilities .............................................................. 198 6.2.2 Transformative Learning ............................................................................................... 200 6.2.3 Innovation Learning Infrastructures ............................................................................. 203  6.2.4 Evaluating Innovation Learning ................................................................................... 206 6.3 Action Research with the City of Vancouver Solutions Lab .......................................... 210 6.4 Ten Practices for Transformative Innovation Learning ................................................. 214 6.5 Discussion .......................................................................................................................... 219  6.5.1 What About Competencies? ........................................................................................ 219  6.5.2 Personal Level of Learning ........................................................................................... 219  6.5.3 Gravitational Pull of Participants to This Experience ................................................. 220 6.5.4 Evaluating Transformative Innovation Learning ......................................................... 221 6.5.5 Implications for Future Research ................................................................................. 221 6.6 Conclusion ......................................................................................................................... 222  7. Conclusion ................................................................................................................................. 224  7.1 Summary of Research Findings and Contributions....................................................... 226 7.1.1 Stronger Theorization of PSI labs ................................................................................. 227    xi  7.1.2 Systemic Interventions for Transformative Innovation .............................................. 229 7.1.3 Transformative Innovation Learning ............................................................................ 231 7.2 Strengths and Limitations of this Study .......................................................................... 233 7.2.1 Interdisciplinarity ........................................................................................................... 233  7.2.2 Methodology .................................................................................................................. 234  7.2.3 Data Sources .................................................................................................................. 235 7.2.4 Evaluation ....................................................................................................................... 235  7.3 Invitations for Key Actors in Public Sector Innovation .................................................. 236 7.3.1 Invitations for Public Sector Innovation Lab Practitioners ......................................... 236 7.3.2 Invitations for Researchers of Public Sector Innovation and Labs ........................... 239 7.4 Future Research and Practice .......................................................................................... 241 7.5 In Closing ........................................................................................................................... 245  References ...................................................................................................................................... 248 Appendix A: Rich Description of First Cycle Action Research and Theory Building ...... 262 A.1 Introduction ....................................................................................................................... 262  A.2 First Cycle of Action Research and Theory Building .................................................... 264 A.2.1 Solutions Lab (SLab) 1.0 Design Brief ......................................................................... 265 A.2.2 Solutions Lab 1.0 Activities and Outcomes ................................................................ 267  A.3 Themes from Early Expert Interviews ............................................................................. 269  A.3.1 Definitions of Innovation in the Public Sector ............................................................ 271  A.3.2 Maturity of the Landscape ............................................................................................ 273 A.3.3 Understanding and Evaluating Impact ....................................................................... 276 A.4 Solutions Lab 1.0 Reframes and Successes ................................................................... 279  A.5 Coming to the First Solutions Lab Theory of Change .................................................. 286 A.6 Conclusion ......................................................................................................................... 290  Appendix B: Rich Description of the Solutions Lab Community of Practice.................... 291 B.1 Introduction ....................................................................................................................... 291  B.2 What Problems Was the Community of Practice Trying to Solve? .............................. 291 B.3 Rich Description of the Solutions Lab Community of Practice .................................... 294    xii  B.3.1 Prototypes in 2018 ........................................................................................................ 294 B.3.2 Community of Practice in 2019 .................................................................................... 295  B.3.3 Solutions Lab Community of Practice Toolkit ............................................................ 300 B.4 Reflection and Learning into the Next Iterations ........................................................... 302 B.5 Conclusion ......................................................................................................................... 304       xiii   List of Tables  Table 1: Public sector innovation types and associated evaluative questions (OECD, 2018)……………………………………………………………………………………………………62 Table 2: Moving to a version of collective impact 3.0 (Cabaj & Weaver, 2016, p.3)…………77 Table 3: Emerging shifts in PSI labs…………………………………………………………………88 Table 4: Summary of research activities and timeline……………………………………………131 Table 5: Leverage points for high impact and systemic interventions for transformative  and emergent innovation…………………………………………………………………………….177 Table 6: PSI lab practices where systemic leverage points for transformative and  emergent innovation could be applied….………………………………………………………...182 Table 7: Types of value and evaluative questions for learning and capacity building  activities (Wenger et al., 2011)………………………………………………………………………200 Table 8: Practices, and corresponding capacities and capabilities, for transformative innovation learning…………………………………………………………………………………...208 Table 9: Learning design for SLab CoP…………………………………………………………….287                xiv  List of Figures  Figure 1: Constructivist grounded theory and participatory action research methodology…………………………………………………………………………………….  7 Figure 2: Weaving together CGT and PAR in this dissertation…………………………… 8 Figure 3: Principles of Innovation (Nesta, 2018)……………………………………………. 40 Figure 4: Innovation methods and tools (Nesta, 2018)……………………………………. 41 Figure 5: Innovation lab methods (Brookfield Institute, 2018,p.8)………………………. 42 Figure 6: Design principles mapped to design model (Jones, 2014, p. 108)………….. 46 Figure 7: Cynefin framework (Snowdon, 2020)…………………………………………...... 49 Figure 8: Seven stages in a social innovation process (Leurs and & Roberts, 2018)………………………………………………………………………………………………  50 Figure 9: Theory U (Adapted from Scharmer,2016)……………………………………….. 52 Figure 10: Culture change impact framework (Nesta, 2018)……………………………... 61 Figure 11: Facets  of innovation (Adapted from the OECD OPSI,2018b)………………. 62 Figure 12: Example of a psychograph showing different developmental lines in the  individual interior, or ‘I’ quadrant (Wilber, 2005, p.28)…………………………………….  69 Figure 13: AQAL Framework showing an example of developmental stages for the transpersonal developmental line (Wilber, 2005)…………………………………………..  70 Figure 14: The productive zone of disequilibrium (Adapted from Heifetz, 1994)……... 73 Figure 15: Comparing cooperation, coordination, and collaboration (ARACY, 2013)………………………………………………………………………………………………  77 Figure 16: Elements of a community of practice (Adapted from Wenger et al., 2002)………………………………………………………………………………………………  79 Figure 17: The adaptive cycle (Gunderson and & Holling,2002)………………………… 80 Figure 18: Points of leverage in a system (adapted from Meadows, 2008)……..……… 82    xv  Figure 19: Multi-level sustainability transitions (adapted from Geels, 2002)……...……. 83 Figure 20: Two Loops Model (Adapted from Wheatley & Frieze, no date)…………….. 84 Figure 21: Three Horizons Model of innovation (Sharpe et al., 2016)……………………...........................................................................................................  84 Figure 22: Gartner’s hype cycle (accessed Dec. 2018)…………………………………….. 99 Figure 23: Constructivist Grounded Theory and Participatory Action Research methodology………………………………………………………………………………..…...  109 Figure 24: Constructivist grounded theory process map (Tweed & Charmaz, 2012)………………………………………………………………………………………………  119 Figure 25: From data, to codes, to categories, to themes, to theory  (Saldaña, 2016, p. 14)…………………………………………………………………………...  120 Figure 26: Illustrated example of what first cycle coding looked like.…………………... 126 Figure 27: In process photo of creating assemblages from coded data………………………………………………………………………………………………..  129 Figure 28:  SLab 3.0 Theory of Change……………………………………………………… 147 Figure 29: Theorizing Innovation Flower……………………………………………………. 155 Figure 30: Paradigm and patterns for transformative and emergent innovation……… 182 Figure 31: PSI lab activities where leverage points for transformative and emergent innovation might be applied…………………………………………………………………..  188 Figure 32: Nesta’s competency framework for public sector innovation (Nesta, 2018)………………………………………………………………………………………………  200 Figure 33: The productive zone of disequilibrium (Adapted from Heifetz, 1994)……... 202 Figure 34: Conceptual model for generative learning designs  (Yorks and & Nicolaides, 2013,p.14)…………………..……………………………………  205 Figure 35: Culture change impact framework (Nesta, 2018)………………..……………. 209    xvi  Figure 36: Ten practices for transformative innovation learning…………………………. 214 Figure 37: Theorizing innovation flower…………………………………………………….. 228 Figure 38: Paradigm and patterns for transformative and emergent innovation……… 230 Figure 39: Ten practices for transformative innovation learning ………………………… 232 Figure 40: Facets of innovation (Adapted from the OECD OPSI, 2018b). ……………... 263 Figure 41: Solutions Lab 3.0 theory of change……………………………………………... 289 Figure 42: Solutions Lab 2019 Community of Practice Learning Commitments……………………………………………………………………………………  298 Figure 43: Solutions Lab storytelling canvas, created for SLab by Naomi Devine………............................................................................................................................  301       xvii  Acknowledgments As I write this, the earth is approaching the winter solstice of 2020, my favourite moment to mark each year. We are collectively coming around the biggest of the big bends, and in the Northern hemisphere we’re beginning a longed-for return to the light. From here on, each day gets just a little bit longer, at least until we head back around again for another trip. I usually mark this occasion with a gathering of loved ones, comforting food and drink, and a fire in the yard to put our words of release and hope into. But this year… This year is a collective experience of mass disruption, uncertainty, and so many other things. So instead of a joyful and loving solstice gathering, I’m finding much joy and love in thinking about all of those that I’m grateful for in supporting my journey with this immense dissertation project.  Gratitude for Maathai Cole, and his high fives, words of encouragement, beach walk breaks, and so much patience. Gratitude for my family, and in particular my matriarchs Sue Van and Dorothy Boutwell, who’ve modeled resilience and fortitude since before I can remember. Gratitude for the circle of women that have held me up during this journey in all of the ways that we do for each other – Naomi Devine, Maggie Low, Lily Raphael, Olive Dempsey, Dawn Smith, Moura Quayle, Amanda Mitchell, Keltie Craig, Ali Grant, Nadia Carvalho, Jackie Kanyuk, Leah Alcuitas, Laura Piersol, Lisa Gibson, Stacy Barter, Sanmini Koffi, and Jen Brummitt. Gratitude for the friends and colleagues who’ve been generous with their hearts, minds, and experiences with working toward equity, decolonization, and justice, I’ve learned so much from you Jason Hsieh, Nadia Carvalho, Alexander Dirksen, Naomi Mumbi Maina, Skylar Sage, and Aslam Bulbulia. Gratitude for each and every one of my co-researchers who have put any and all of the good into this work - I hope that I have offered back something that encourages and emboldens you. Gratitude for Moura Quayle, Rob VanWynsberghe, and Mark Cabaj for lovingly and openly guiding me, while also encouraging me to find my own path. Finally, so much gratitude for the lands, waters, and kin of this shishálh swiya in which I’m blessed to make my home.    1  Chapter 1: Introduction         2  The idea for constructing this dissertation as a cabinet of curiousities came to me as lots of (I think) good ideas do – while I was washing the dishes. I have a little window over the kitchen sink that looks out on the backyard, and I usually look out there while doing the washing up. Just to the left of the window is this shadow box. It was made by my mom and dad at some point in the 1970’s (in case you couldn’t tell the vintage from the colour of the vegetable peeler). They were post high-school, pre-children, broke, and on an adventure far away from friends and family. For Christmas gifts one year they decided to get some wood and glass, and rustle up some things that they had around the house to make these boxes to send home as gifts. This one was passed along to me by my mom’s then- and now best friend. One day, while washing dishes and deep into considering methodology options and choices for this research, I looked up at this beauty with its 40+ year old peanuts and beans and the idea of assembling this dissertation as a cabinet of curiousities came to be.  I want to reframe the construct for my purposes here, as the cabinets of old were exclusive affairs reserved for the wealthy, and often full of things taken/stolen from other cultures and then placed out of context. These cabinets had a preference for what was seen as strange and exotic according to an outsiders’ view. Our cabinet is full of extraordinary things hidden within what often seems quite ordinary, and takes an entirely insider view. It is co-constructed with many others, and open to (re)interpretation. Although I’ve assembled it into patterns and shapes that are interesting and pleasing to me, I’ve also tried to make it so that you can move things around to make your own meanings. There are many nooks that we take glimpses into that have whole other worlds and possibilities existing beyond them, and that I have not yet followed. So, this cabinet is also an invitation to open that next drawer or door and see what surprises are still to be revealed. Let’s begin.  1.1 Research Context When I came to work in the public sector in 2010, it was with a conscious choice to move into an organization and role that I thought would offer different – and more powerful – levers for change in sustainability work. Until this point, I’d been active in the nonprofit and (what is now called) social enterprise sectors for about ten years and was feeling the limits of advising on    3  policy from the outside. So, I joined the City of Vancouver in 2010 to lead the planning and public engagement work for the Greenest City Action Plan, and to see what it was like to be on the inside. I have had several different roles within- and alongside the City since then, always focused on environment and social policy work, and with a growing curiousity and urgency about the role of the public sector in significant, ambitious, and transformative sustainability work. In 2014, a wise colleague responsible for designing the Healthy City for All Strategy took inspiration from the local, national, and global social innovation movements and her own experiences of stuck social systems. She nested an action into this strategy to build a staff hub within the City of Vancouver, where we could work differently on urgent, complex, interdisciplinary challenges. This planted the seed from which the City of Vancouver Solutions Lab (SLab) grew starting in the fall of 2016, and I jumped in with both feet.  In my early conversations, thinking, and research into contemporary public sector innovation labs and hubs I could see a bigger possibility for this work with the City of Vancouver to inform and support thinking, research, action, and impact of the larger field than could be realized through the job description alone. I asked Moura, Rob, and later Mark if they would consider supporting a Ph.D. journey into the potential for public sector innovation labs (PSI labs) to provoke transformative innovation on complex sustainability challenges. They all said yes, as did 85 action co-researchers and public innovators, and the fruits of these shared labours are contained within what you are about to read.   There is a great deal more about public sector innovation, the particular contemporary construct of innovation labs, and the contributions that researchers and practitioners are making to thinking and action in this space contained in Chapter 2. However, I will provide some brief context here as we open up this cabinet of curiousities as an orientation to how and why I’ve constructed it in the way that I have. I began my research by reading about the work of the early and well-known public sector innovation labs (PSI labs), including MindLab in Denmark, Helsinki Design Lab (both now closed), Policy Lab UK, and about a dozen others. As I deepened my learning about their work and practices, and as I began to do some early interviews with expert practitioners, I had a hunch that the field did not have strong and explicit definitions of innovation, nor did we have clear and ambitious goals or directionality to our innovation work in many cases. It also appeared as though researchers were most    4  interested in defining, describing, comparing, and codifying PSI labs, and with a focus on their methods, and that deep entanglement of researchers and practitioners into shared questions of interest was not yet happening.  The purpose and approach of many PSI labs is often described as a need to innovate, improve practice, and add public value by bringing design, creativity, and user-centeredness to the complex challenges of government (Carstensen & Bason, 2012; Lewis et al., 2019; McGann et al., 2018; Puttick et al. 2014; Tõnurist et al., 2017). Gryszkiewicz et al. (2016) capture the essence of a lab as “…a semi-autonomous organization that engages diverse participants on a long-term basis – in open collaboration for the purposes of creating, elaborating, and prototyping radical solutions to open-ended systemic challenges” (p. 84). My early hunch was confirmed for me later when I read the results of a systematic literature review of 181 public sector innovation (PSI) articles by de Vries et al. (2016). They found that 76% of the studies they reviewed did not define innovation, and that 35% did not name the specific goals for their innovation work. In some ways this finding makes a great deal of sense. PSI labs are the grain of sand in the oyster shell, constantly causing discomfort. Or perhaps another analogy is of the unknown virus or bacteria that kicks an organizational immune system into gear. Moore (2017) describes an idea of social innovations as proto-regimes, working within, alongside, or outside of the dominant regime that they are trying to change, and having distinct relationships to/with this dominant regime depending on their relative position, and how bound they are, to it. PSI labs have to work so hard just to exist/persist and to respond to the many barriers that make their work difficult, and all while working on some of the most complex, high-pressure, and often politically contentious challenges. Taking time to more strongly theorize and evaluate their work feels like a luxury that they cannot afford.  For me, coming into this researcher-practitioner role to design, convene, and evaluate the impacts of a PSI lab while holding a strong sustainability and equity orientation, this lack of a more ambitious definition, directionality, and values-base to ‘innovation’ was surprising. Perhaps PSI labs were continuing to operate within dominant paradigms of governance and systems and structures of power rather than challenging them, and/or imagining other possibilities. Perhaps there was useful thinking that I could explore to see if/what/how a PSI    5  lab that holds a more disruptive and transformative intent might be constructed. This curiousity and impulse led me into exploring a broader field of literature about ways to think about transformation, and also to a refinement of my research questions.  1.2 Research Questions When I set out on this journey, my initial research questions were: How might a deeper understanding and integration of personal, organizational, and systems transformation practices into PSI labs add to their effectiveness in achieving meaningful results on complex sustainability challenges? Are these practices interrelated and/or integrated, or are there fundamental differences that may make them incompatible?   The early literature review, along with my own experiences as researcher-practitioner with the first iteration of the City of Vancouver Solutions Lab and early expert interviews, led to a refinement and fuller expression of these initial research questions. I understand these reframed questions as four fractals that, when taken together, draw from theory and practice to generate a useful assemblage of ideas that point toward the potential for PSI labs to catalyze significant and positive impacts toward sustainability and equity. These fractals/research questions that shape this dissertation are:  How might we more strongly theorize the purpose for PSI labs, and why might this be important?  How might public sector innovation labs systemically intervene in complex public sector challenges in ways that create stronger enabling conditions for transformative and emergent innovation?  What if PSI labs took a transformative innovation learning approach to their work in order to enable and unleash innovation within and amongst individual, organizational, and ecosystem-scale actors?  How might we measure, evaluate, and tell stories of transformative change resulting from PSI lab interventions?     6  1.3 Overview of Research Methodology and Approach The cabinet of curiousities metaphor came from a methodological idea sparked by Freeman and MacLure. Freeman’s (2017) description of a diagrammatical mode of thinking for qualitative research “seeks to disrupt conventional ways of thinking… and asks that we look beyond the familiar narrative construction of a story and transverse core aspects of its telling in a way that creates new assemblages” (p. 9). MacLure’s (2013) work critiques the processes of coding qualitative data and invites the researcher to follow wonder and emergence in this analytic process. Together these ideas created the cabinet of curiousities metaphor for my research, which methodologically came together as a critical, qualitative research bricolage.  Kincheloe et al. (2017) say that “a critical research bricolage attempts to create an equitable research field and disallows a proclamation to correctness, validity, truth and the tacit axis of Western power through traditional research… Without proclaiming a canonical and singular method, the critical bricolage allows the researcher to become a participant and the participant to become a researcher” (p.253-4). This bricolage incorporated a collection of methodologies that challenge Western ways of knowing, allow for multiple truths to coexist, invite a researcher with an active role in the research questions, and co-create and engage community in knowledge production. A lineage of researchers in Indigenous methodologies and anti-oppression theory inform this bricolage, and offer alternatives to the research practices deemed ‘valid’ by academic institutions and that continue to be dominated by white, male, colonizing, straight, and able-bodied worldviews that reproduce very narrow ways of knowing and being (brown, 2017; Kincheloe et al., 2017; Kovach, 2009; Simpson, 2017; Smith, 2016; Strega & Brown, 2015). The research questions explored in this dissertation required a methodology that enabled seeing outside of the dominant paradigms that shape PSI and lab discourse, and invited divergence, openness, and non-closure. This is of particular importance for this study because the research and practices of PSI labs tend to implicitly operate within these dominant paradigms, thus limiting the different ways that this research question might be interpreted, and what might be revealed through this inquiry.   Participatory action research (PAR) and constructivist grounded theory (CGT) are the backbone research methodologies in this bricolage (Figure 1). PAR and CGT are appropriate    7  for community-engaged social innovation inquiry as they can handle researchers who hold standpoints, a social justice orientation and perspectives, and a desire to produce radical, democratizing transformation, as long as these are transparent, made explicit, and are a part of a reflective process (Charmaz, 2014; Denzin & Lincoln, 2017). These methodologies do not require a neutral, objective observer, and invite the researchers’ perspectives on data and analysis as relevant to the course of inquiry. Applied and action-oriented knowledge generation and mobilization is built in, along with transparent and open co-production of knowledge with/by those who are most impacted by the challenges being researched.    Figure 1: Constructivist grounded theory and participatory action research methodology  This dialogue between PAR for theory testing, and CGT for middle range theory building was engaged with in three cycles between October 2016 – December 2020 (Charmaz, 2014; Charmaz, 2017a; Charmaz, 2017b; Kemmis, 2008; Merton, 1968; Reason & Bradbury, 2008; Strauss & Corbin, 1990; Swantz, 2008). Literature review informed the construction of theory as well as action research practice and interventions. Participatory action co-researchers were connected with the primary research site at the City of Vancouver Solutions Lab, secondary action research sites at the Laboratoire d’innovation urbain de Montreal and the British Columbia Government Public Service Innovation Hub. Additional expert interviews with    8  practitioners from PSI labs and network-serving organizations were also conducted, totaling 85 co-researchers from 25 organizations in seven countries contributing to this work. The data from interviews, observation, evaluation, and reflection generated rich qualitative data to code, analyze, and assemble into patterns that shaped the theory construction. Figure 2 describes how this research methodology worked in practice, showing cycles of divergence and convergence around findings. Figure 2 also orients the reader to how the non-linear construction of the doors and drawers of this cabinet of curiousities connect and relate to one another from my point of view, with some wayfinding about where this content is written up so that the reader can also find their own provocations, patterns, and curiousities. The construction of this cabinet of curiousities, and the fractals that you might find inside, are shared next.  Figure 2: Weaving together CGT and PAR in this dissertation  1.4 Dissertation Structure and Chapter Overview This dissertation was conceived as a hybrid between a traditional and paper-based structure. It is traditional in that it includes a robust literature review and description of research methodology. It is paper-based in that Chapters 4-6 are written to be somewhat stand-alone,    9  and they take the reader quite directly and succinctly to the theory generated through the research instead of including a great deal of the raw data and coding process that you might expect from a more traditional approach to a dissertation.   Chapter 2 opens an interdisciplinary literature review full of wonders. The first section explores different facets of public sector innovation. Dominant paradigms of governance and how each conceives of innovation are explored first, including different definitions of public sector innovation, as well as enabling conditions and barriers. A review of academic and practitioner literature about public sector innovation labs follows, discussing typologies of PSI labs, the different methods that they draw from, how innovation capacities are considered by PSI labs, and how their impacts are evaluated. The literature review then moves into the possible differences between a change and transformation orientation to innovation, and includes literature interested in transformation at the personal, organizational culture, and systems levels. Chapter 2 concludes with a critical analysis of the state of PSI lab research and practice and suggests seven areas of particular relevance for this dissertation.  Chapter 3 describes the research methodologies used in my doctoral work in some detail. I position myself in the research, and then describe each aspect of the critical research bricolage. My approach to coding and analysis is described, as are details about the three action research sites and expert interviews. This chapter closes with a summary of my research activities and timeline.  Chapter 4 focuses on the research question concerned with more strongly theorizing the work of PSI labs. It pulls together threads from the literature to critique the current state of research and practice, and then shares some detail from the action research learning journey and expert interviews relating to theorization. A framework for stronger theorization of PSI labs is proposed and described, with feedback from action co-researchers shared to describe how this framework might be used and improved in future.  Chapter 5 follows the research question that is interested in systemic interventions for transformative innovation. When a PSI lab is interested in catalyzing transformation through its work, how might it go about doing this? Literature that explores enabling conditions and    10  barriers for public sector innovation is brought in, and then four ways to expand beyond these dominant research and practice frames are described. A framework that offers a paradigm for transformative public sector innovation, elaborated with a set of nine patterns to move away from and move toward, is described. A set of 20 systemic interventions for public sector innovators to harness in order to catalyze these movements away and toward are provided, before the chapter discusses implications for this framework.  Chapter 6 focuses on transformative innovation learning in public sector innovation labs. It first draws from literature and practice that focuses on dimensions of innovation learning, and then describes action research from the Solutions Lab. Ten practices for transformative innovation learning are proposed and described in rich detail drawing from action research data. The chapter concludes by discussing the potential role of PSI labs in innovation learning.  Chapter 7 concludes this dissertation. It reflects on what this study offers to PSI lab research and practice, as well as to other types of labs and intentional innovation interventions, social innovators, and social entrepreneurs interested in transformative work.  The chapter points to future research questions and applied learning possibilities. I share some personal reflections and remaining questions, and the sections of this cabinet of curiousities that were not fully opened up.   Appendices A and B offer rich descriptions of action research activities and the learning that resulted. Appendix A describes the action and evaluation from the first iterations of the Solutions Lab, with an eye to how this experience resulted in a stronger theorization of this work based on iteratively layering research and application. This experience formed the foundation for Chapter 4’s focus and approach to more strongly theorizing PSI labs. Appendix B is a rich description of the transformative innovation learning work of the Solutions Lab, focused on describing the design, facilitation, and learning from the community of practice.      11  1.5 Significance of Contributions The challenges facing public sector organizations, and society generally, continue to become more urgent and acute. These challenges are increasingly complex, characterized by ambiguity, non-linearity, uncertainty, and volatility. We now have many decades of evidence telling us that human and ecological health and wellbeing is compromised, particularly for people who have been systemically and structurally oppressed, and for non-human kin that remain largely unrepresented. The go-to systems, structures, and processes of governments should adapt in response, and this pressure is widely recognized. But how might we do this? Why aren’t we doing this at the speed and magnitude that these challenges demand? And, of particular interest in this inquiry, when/if we do ‘innovate’ in response, will we continue to uphold the systems of oppression and exploitation that are baked in to the ways in which colonial governments operate, or will this also be transformed? The existing systems and structures are perfectly designed for the outcomes that result, so public sector and social innovators must stay attuned to who benefits from the system remaining as-is, and if/how we might be complicit in that through our ‘innovation’ work.  This research aims to contribute to these big questions through sharpening our thinking about the particular construct of innovation labs and making the lab construct more coherent at this moment in time. This is a field-in-motion, and it is rapidly iterating and learning. By drawing from the direct experiences of action co-researchers working in this field, I hope that this research offers a useful landing point for some of the big questions in this moment. I hope that it contributes to ever-increasing impact of social innovators working on the biggest, most complex civic and public sector challenges that we currently face. Labs offer some provocative and useful ways to conceptualize ‘unblocking the drain’ in contexts beyond public sector organizations as well, for example in education and business. They provide a mechanism, or an opportunity space, to rehearse other ways to be/do/feel as intrapreneurs working to change these stuck legacy organizations from within, and to learn/teach the system and the people that co-create it that they can make other choices. This research also articulates the ‘it’ of an innovation lab in ways that can lead to stronger evaluation thinking and processes in the near future, which is a perennial challenge of labs. Over time we may    12  find that labs are a transient concept/construct, and that they are a step in a collective movement toward something that we do not yet see or know.      13  Chapter 2: Review of Literature  This chapter reviews both academic and practitioner-oriented literature, which was assembled by taking guidance from both my research questions and positionality. This is an interdisciplinary study with many potential bodies of work that could inform this inquiry. It is very likely that readers will have their own ideas about fields of literature that relate to these research questions and wonder why they are not included. My intent here is to both set the stage for an inquiry into public sector innovation, and also invite some thinking, provocation, and inspiration from interdisciplinary sources that can inform the more specific interest in the transformative potential of public sector innovation labs (PSI labs). I hope and expect that the theory constructed in chapters 4-6 of this dissertation will continue to be tested and refined over time, and that an expansion into different bodies of literature as well as increased depth of exploration in the fields included here will be an important part of this process.  This literature review explores public sector innovation (PSI), including governance paradigms, definitions and types of PSI, and enabling conditions and barriers. It then dives into the more specific form of PSI labs, considering how they are defined, their typologies, and design and social innovation approaches to these labs. Innovation capacities and evaluation of PSI labs is also explored. There is then a foray into literature about personal, organizational culture, and systems transformation and how this relates to strongly theorizing a transformative approach to PSI labs. The chapter closes with an analysis of the literature as it relates to PSI lab theory and practice, addressing seven important themes that are particularly important for this research.  2.1 Exploring Public Sector Innovation This section explores how public sector innovation (PSI) is considered in three different paradigms of Western governance, including traditional public administration, New Public Management, and network and collaborative governance. A variety of ways that the literature defines PSI, ascribes PSI typologies, and identifies drivers and barriers are discussed.    14   2.1.1 Governance Paradigms and their Relationship to Innovation Recent scholars have tracked the evolution of innovation in the public sector in a few different ways, with some common themes and threads. This literature is from distinctly European and North American paradigms and practices of governance. Lewis et al. (2017) offer a useful framework to organize thinking for the purposes of this course of study and describe three paradigms of governance relevant to studies of innovation: 1) traditional public administration; 2) New Public Management; and 3) network governance. Sørensen and Torfing (2011) examine literature from the fields of public administration, planning, and economic innovation and find a similar three stage arc across these fields: 1) traditional, bureaucratic and expert forms of public management; 2) economic, choice-oriented, and inter-organizational collaborative forms; and 3) citizen-engaged, networked, flexible and adaptive forms of public sector management. The categorization from Lewis et al. (2017) and Sørensen and Torfing (2011) is used here as a structure for describing and defining how these different governance paradigms shape conceptualizations of PSI.  2.1.1.1 Traditional Public Administration Traditional public administration, also described as Weberian bureaucracy and hierarchical forms of governance, dominated Western conceptions of governance until the 1970’s. It is characterized by hierarchy, division of labour, rules, rigidity, and expert- and career orientation (Bason, 2017a; Bason 2017b; Boukamel & Emery, 2017; Lewis et al., 2017; Mulgan, 2009; Sørensen & Torfing, 2011). There was a perception that there was no real need for innovation in the public sector during this time, and that the focus was instead on providing stability, certainty, and fairness in the provision of impersonal public sector rules, policy, processes, and services (Bason, 2017b).  2.1.1.2 New Public Management  New Public Management (NPM) is described as a managerial and market-driven period from the 1970’s and into the early 2000’s, and that still dominates much of the thinking that shapes    15  public sector innovation today (Bason, 2017b; Boukamel & Emery, 2017; Hartley et al., 2013; Lewis et al., 2017; Mulgan, 2009; Sørensen & Torfing, 2011). It is characterized by approaches developed by and for the business sector and applied to the public sector, including efficiency, effectiveness, performance management, competition, and privatization. Bason (2017b) calls NPM a scientific paradigm of management dominated by analytical, rational, and optimization-oriented approaches that are focused on problem solving. Hartley et al. (2013) describe two attributes of NPM that can drive innovation, including: competition as a means to encourage efforts to do more with less; and strategic management practices focused on performance and results.   Hartley et al. (2013) identify three barriers to innovation that can be caused by a NPM approach, including: an increase in bureaucratic rules as a result of focusing on performance and risk management; an unhelpful fixation about finding best practice instead of next practice; and prioritizing efficiency and standardization instead of enhanced effectiveness of policies and systems. A critique of an NPM approach to PSI is that a direct application of business approaches to the public sector does not tend to work due to the unique purpose, opportunities, and constraints in this context. Although drawing from private sector approaches to innovation might be helpful, and the sectors do have commonalities (e.g. larger organizations of different types may share the attribute of being slow to act and prone to incremental change), the public sector has a fundamentally different purpose. It must balance multiple bottom lines, negotiate diverse points of view, and provide a wide range of governance-related activities. Innovation in the public sector warrants its own set of processes, practices, and theoretical underpinnings.  2.1.1.3 Network and Collaborative Governance What Lewis et al. (2017) call network governance is one name for a more contemporary governance paradigm within which PSI is currently being conceptualized. Variations of this paradigm-in-development include New Public Governance (NPG), post-NPM, neo-Weberian governance, design approaches, co-creation, co-production, and collaborative governance. Together these can be understood as contemporary expressions of how PSI is being formed and theorized and includes the 2000’s through to present day. Lewis et al. (2017) characterize    16  network governance as being of particular importance for PSI because this form considers a diversity of different points of view, mobilization of resources, flexibility of information flows, and trust building, all of which are viewed as being important for innovation. Boukamel and Emery (2017) call this phase post-NPM and characterize it as being about the creation of public value, enhancing democracy, transparency, and multi-disciplinarity, and clearly acknowledging that innovation is required in the public sector.   Hartley et al. (2013) articulate two additional relevant governance paradigms as neo-Weberian and collaborative. Together they include the features of paying attention to different forms of leadership, inter- and intra-organizational coordination and collaboration, entrepreneurial approaches, citizen-centric behaviours, and trust-based accountabilities. Bason (2017b) explores design approaches to PSI including new forms of citizen involvement, collaborative innovation, and empowering citizens to play an active role in the design and delivery of services. Hartley et al. (2013) argue that collaboration is the most promising, although still nascent, approach to public sector innovation because it conceptualizes how innovations are produced as including the roles of a range of different stakeholders throughout the process.   These three broad paradigms demonstrate the importance of context in shaping the understanding and definition of innovation. The construct of innovation is shaped by political, fiscal, social, ecological, and other issues as they rise and fall in their relative importance, and as different actors and sectors rise and fall in their ability and agency to influence public sector thinking and action. Hartley et al. (2013) note that each of these paradigms shapes what is meant by PSI in useful and illuminating ways and may be more or less appropriate in different contexts, and on different types of governance challenges. Traditional bureaucracy tends to bring efficiency, predictability, reliability, procedural fairness, and the reduction of randomness and unpredictability to government and society. NPM brings approaches from business deep into the public sector, including ideals of efficiency, competition, privatization, and entrepreneurial activity. Both of these paradigms still have relevance and are widely used in modern governance. As the third paradigm, called network and collaborative governance here, emerges, it is becoming clearer that what is missing from earlier and still dominant paradigms are the theories and practices that help governments to respond to contemporary    17  pressures that are very different than what has been seen before. The nature of complex, emergent, systems-oriented challenges facing governments, and the increasing expectations of citizens for transparent and collaborative governance processes, compels researchers and practitioners to seek new paradigms and approaches, as neither traditional bureaucracies nor NPM are well suited to the great complexity of this time.   2.1.2 Definitions of Public Sector Innovation Innovation is a term that is broadly, loosely, and liberally used in many settings, and the public sector is no exception. In this section, examples of how PSI is being defined in current research and practice are discussed. The variety of definitions makes evident the inclusion and exclusion of different concepts, as well as different emphasis, embedded values, and explicitly or implicitly defined purpose, all while being called ‘innovation’. Several definitions of PSI are provided here, with an aim to illustrate some areas of convergence, divergence, and emphasis in the academic literature and practitioner work.  Some good examples of general definitions of innovation come from Bason (2010) and The Brookfield Institute (2018). These aim to be clear and accessible to most users and contexts. Bason describes PSI as coming up with new ideas that are successfully implemented to create public value, thus capturing a very broad scope of meaning. The Brookfield Institute defines innovation as “novel structures and processes that enable innovation within government, such as open government and digital government” (p. 6). This is a good example of how many practitioners use their tools and techniques as a proxy for a definition. Innovation is quite commonly defined in this way, and most often human-centred design tools and techniques are used as the proxy definition.   Hartley et al. (2013) define innovation as “…those forms of change that break with the established practices and mindsets of an organization…to create something new, so that innovation is a step change or a disruptive change. Innovation can be either radical or incremental, and it can be based on either the generation of an original invention or the adoption and adaptation of others’ innovations” (p. 822). This definition captures an    18  important area of divergence in the literature, with some researchers including incremental change in their definition of innovation, and others taking a position that innovation must be disruptive, discontinuous, and/or scale shifting in nature (Brown & Osborne, 2013; Geiske et al., 2016; Sørensen & Torfing, 2011). It also describes the purpose of innovation as generating something entirely novel, or something new in a particular context, which is an aspect of defining innovation that has a good degree of convergence in the literature.  Munro (2015) defines innovation as “…changes to services, or products, or ways of working, or organizational arrangement, or democratic approaches that are both new to the council and deliver additional value for its residents, service users and/or local businesses… Focused on leadership actions that helped to attain more major cost-saving innovations” (p. 219). This definition describes more specific public sector activities where innovation might arise, including leadership actions which isn’t often included. It also shows how NPM values of cost-saving and delivering additional value show up in a definition.   There is divergence in the literature about where and when an innovation process ends, and what outcomes, impacts, successes, or other results mark this stage. Some theorists focus on the invention of a novel idea or concept, and others insist on implementation and sometimes scaling in order to be considered a true innovation. Torfing et al. (2016) are in this latter camp, saying that innovation is “...an intended but inherently contingent process that involves the development and realization, and frequently also the spread, of new and creative ideas that challenge conventional wisdom and disrupt the established practices within a specific context” (p. 30).  Walker (2008) affirms this need for implementation and scaling for something to truly be an innovation and goes further to acknowledge innovation as a process and not only an outcome. Innovation is “...a process through which new ideas, objects, and practices are created, developed or reinvented, and which are new for the unit of adoption. Because organizations may innovate in search of legitimacy and not fully adopt an innovation, I also argue that it has to be more than just an idea and implementation has to occur…Innovation is evolutionary rather than radical…By understanding innovation as a dynamic process it is argued that a more comprehensive account of organizational innovativeness will be    19  obtained” (p. 592). Also, of note here is Walker’s inclusion of evolutionary, or incremental change, as well as more disruptive change, further demonstrating this divergence in the literature.   Several theorists embed more nuance into their definitions of PSI. Considine and Lewis (2007) consider innovation to be a particular form of policy and governance, and include three dimensions in their definition: 1) the normative or perceptual frame through which the key players in the system define innovation; 2) individuals’ roles and positions and how this shapes their understanding of how innovation is described and enacted; and 3) the patterns of communication and networking among key actors in the system. They tested this framing through a survey with civil servants and politicians who were asked to describe their normative constructs of innovation, with a focus on the first dimension. The innovation constructs that emerged included: institutional; structural; skeptical; incremental; and adaptive. They then parsed this data out by role within the organization, across different public sector organizations (PSOs), and finally between politicians and bureaucrats to confirm that different PSOs, and different roles within a PSO, perceive innovation differently. A normative approach to definition-making is an interesting choice, as it avoids pinning down a challenging term with one definition and lends itself to more localized adaptation and relevance. However, it also perpetuates a challenge described by Hartley et al. (2013) and Walker (2008) of not being able to compare approaches and results across different contexts as there is not a shared definition of innovation.  As a final lens on defining innovation in the public sector, Lewis et al. (2017) explore four different perspectives on innovation in the public sector drawing on historical and current academic and practitioner literature from a variety of fields. This is useful framing, as it begins to discern different perspectives on how innovation is defined and uncovers some of the assumptions and paradigms that may be implicit within a particular definition. The first perspective they name is innovation as creative destruction. The second perspective is that user engagement in innovation will result in better outcomes. The third perspective is that of an innovation system where connectivity, and the interactions, knowledge exchange, and capacity sharing of networks of actors, will result in innovation. The fourth perspective focuses on the dynamic capabilities of organizations to adapt to change. If each of these    20  perspectives about innovation is carried through into how it might be turned into theory and practice, it is clear that different underlying goals, purpose, values, and paradigms will lead to quite different definitions, practices, and results of innovation.  So why does this matter? Academics and practitioners are defining innovation differently in subtle yet important ways. De Vries et al. (2016) note in their systematic review of 181 PSI articles over the period of 1990 - 2014, that 76% of these studies did not define innovation, and 35% of the studies did not name any goals for their innovation. Hartley et al. (2013) state that innovation in the public sector is under theorized and under researched and is only just beginning to be differentiated from studies of innovation from the private sector. Munro (2015) also notes that without a standard definition of innovation being used in the public sector literature, it’s very difficult to compare findings from different studies and develop the fields of research and practice.    If the field is this unclear about what innovation actually means it becomes very difficult to research, compare, and critique what is happening let alone catalyze much-needed innovations in practice. While the PSI field continues to emerge and develop, it is important that researchers and practitioners are clear about what they mean when they use the term so that more rigour and transparency in defining, studying, applying appropriate processes and methods, and understanding the impacts of various understandings of PSI become possible. Researchers and practitioners also need to get better at describing innovation purpose, making connections to the emerging understandings of contemporary PSI beyond NPM and into different governance paradigms. The definitional tent may remain large, diverse, and inclusive, an approach that is appropriate given the changing governance paradigms and the complexity and variety of challenges that the public sector is facing. Taking a more rigorous approach to defining PSI will help the field to increase credibility, shared learning, and impact.  The next section focuses on types of PSI. It is another aspect of how innovation is being described in the public sector and is important for building a more robust approach to theorizing this work.      21  2.1.3 Types of Public Sector Innovation The PSI definition from Munro (2015) in the previous section provides an example of what the literature calls innovation types. Innovation types describe the domain of public sector activity that an innovation effort focuses on, and to an extent also bounds and nuances innovation purpose in a different way than definitions tend to. Much like definitions, there is variation and ambiguity in the literature about innovation types as well, with many innovation efforts not explicitly naming their area of focus, and/or how that influences their strategy or activities. Here is a list of different innovation types described in the literature, roughly in order from most- to least commonly cited: organizational/administrative; service; policy; process; governance; conceptual, symbolic or rhetorical; ancillary; communication; democracy; engagement; and product (Bason, 2017a; Bekkers & Tummers, 2018; Brookfield, 2018; Sørensen & Torfing, 2011; Tõnurist et al., 2015; Tõnurist et al., 2017; Torugsa & Arundel, 2016; Walker, 2008). Although many view these types as discreet, Torugsa and Arundel (2016) are particularly interested in combinations of multi-dimensional or complex innovations that layer these different types into shared processes and solutions.   Walker (2008) examined innovation types and antecedents, and this study provides some depth to the importance of understanding and articulating type in PSI. The study surveyed 74 local government authorities in the United Kingdom (UK), at both the senior management and service staff levels, over two data collection years (2001&2). They looked for organizational and environmental conditions for innovation based on three innovation types, or foci:  1. Service – three types: total, expansionary, and evolutionary; 2. Organizational process - two types: marketization (increase efficiency or effectiveness in service production and delivery) and organization (structure, strategy, and administrative processes); and  3. Ancillary - organization-environment boundary innovations that require working with others.  Walker then made and tested 14 hypotheses about innovation antecedents related to these three innovation types (e.g. “external communication will have a positive relationship with    22  organization innovations and is likely to have a positive relationship with other innovation types” p. 596). Like Torugsa and Arundel (2016), a key finding from this study was that relationships between different innovation types and conditions are complex. They are sometimes complementary, sometimes negative, and sometimes arise independently of one another. Walker concluded that there is no one-size fits all approach to public sector innovation, and that different approaches need to be used for different contexts and conditions. Walker also reflected that a reductionist approach to measuring innovation through discrete types and variables may not be a useful way to advance knowledge, and that research approaches that use more integrated methods might be better suited to advance the field of research and practice. An innovation concerned with a governance process will be very different than seeking innovation on a particular issue. Different purposes and types of innovation will need to be managed differently, and innovation will likely have different meanings, pointing to the need for a nuanced and broad typology. This is a useful lens through which to look at what academics are and are not paying attention to as contemporary PSI research and practice continues to emerge and develop.  Layering innovation types into the description and understandings of PSI paradigms, history, and definitions offers another useful framework to add rigour and complexity to the variety of ways that innovation can be practiced in the contemporary public sector. The next sections explore what the literature says about enabling conditions and barriers for PSI.  2.1.4 Enabling Conditions for Public Sector Innovation Since the 2008 recession, PSI has been heavily influenced by a downsizing of government budgets in many Western governments, and a dramatic rethinking of how governments can continue to meet expectations and obligations with much more limited resources. At the same time, they are being influenced by rapid digital and technological change. There are increasing expectations for customized citizen experiences, and of putting citizens at the centre of policy and program development, to keep up with the quality of services provided in the private sector. Complex global challenges like climate change and growing inequality are also driving how the public sector understands innovation and its purpose.  Tõnurist et al. (2015) describe this collection of drivers, pressures, and catalysts as: citizen-centred service    23  improvements; fiscal pressures; rapid advancement of information and communication technologies; growth of the entrepreneurial state; and pressures for more charismatic, disruptive, and revolutionary innovation and change.  In their work, Lewis et al. (2017) aim to better understand what supports innovation in the public sector and propose a framework to study innovation capacity with three main elements: structures; networks; and leadership. Sørensen and Torfing (2011) identify two drivers of innovation in the public sector. First, they discuss the increasing expectations that citizens and businesses have of the public sector to be responsive and improve the quality and effectiveness of their work. Second, they describe the growing ambitions of public sector staff and politicians to improve governance and other practices and improve their abilities to solve increasingly complex challenges. Munro (2015) uncovered five enabling conditions to accelerate innovation including: 1) agreement on a clear strategic direction and innovation priorities; 2) leadership for innovation at senior leadership level; 3) foster an organizational culture that encourages innovation, with risk management of particular importance; 4) devote sufficient time and resources; and 5) build effective intra- and inter-organizational collaboration on major innovations.  Drawing from this literature about enabling conditions for public sector innovation, this section is organized into three categories of enabling conditions: structures and governance; networks and collaboration; and leadership.   2.1.4.1 Structures and Governance This literature covers broad terrain, and is generally concerned with the structures, infrastructures, governance processes, and approaches that are important for enabling PSI. Considine and Lewis (2007) explore three types of governance active in most public sector organizations - legislative, managerial, and political governance - and their effects on whether or not they help or hinder innovation. Their results found that enabling innovation depends on several factors in addition to the types of governance, including: the particular government being studied; the role of the person being interviewed for the study; and the specific timing and context that the interview was conducted.     24   Structural and governance related enabling conditions for supporting implementation of complex innovations has been studied by Torugsa and Arundel (2016) as well and they found innovation when: (1) creative, decentralized workplaces encouraged bottom-up innovation; (2) policies and practices supported substantive work on complex innovations rather than quick and simple work on more incremental innovations; (3) capabilities to manage and circumvent barriers were developed and enabled; (4) the right set of a variety of sources for ideas and information were drawn upon; and (5) workplace conditions encouraged individual and team creativity.  Demircioglu and Audretsch (2017) used a 2012 dataset from the Australian Public Service Commission to study the likelihood of innovation in the public sector. They used a framework for enabling conditions adapted from Sahni et al. (2013) and noted that there not many established theoretical and practice-based frameworks to assess enabling conditions for innovation in the public sector. Demircioglu and Audretsch found that structures that enable experimentation, a performance management system that responds to low performers, the presence of feedback loops, and the motivation to make improvements were all important enabling conditions. They also found that budget constraints, the fifth condition studied, did not have a significant effect on enabling innovation, which does not support a commonly held idea from NPM that scarcity and competition for resources can spark innovation.   Boukamel and Emery (2017) describe organizational ambidexterity – the ability to cycle between exploration and exploitation approaches – as an enabling condition for public sector innovation. Through their theoretical study, they identified the tension of working in these two very different modes simultaneously as necessary to innovation. Related to this finding, Eggers and Singh (2009) point to the need for innovation processes, practices and objectives to be mainstreamed within the public sector rather than existing as specialized units in order to build innovation culture and structures within government and achieve real impact. Together these point to structures and governance, and also capabilities, capacities and competencies that exist throughout an organization as important enabling conditions.     25  The enabling conditions for experimental governance are studied by Kronsell and Mukhtar-Landgren (2018). They develop a framework that describes potential municipal government roles in enabling experimentation including: promoter; enabler; partner; and a non-role or active inhibitory role. They tested this framework with 50 urban living labs, a particular PSI form that involves municipal governments and partners working together on sustainability issues. They found that municipalities play all of the roles identified, that these roles can change over time, and that the expression of these roles within a specific municipality is not homogenous as different departments and actors will relate differently to the lab. They identify two enabling conditions of particular importance to municipal governments in their study: to bring funding and legitimacy to the work; and to initiate, facilitate, promote, and/or govern collaborations. This study is also interesting because it names roles that many governments do not consider to be part of their responsibility or jurisdiction, moving beyond their tendency to focus on policy-making, regulatory, and program and service delivery.  2.1.4.2 Networks and Collaboration There are a variety of different ideas in the literature about how networks and collaboration might enable PSI. Sørensen and Torfing (2011) are interested in the capacity of governance networks to contribute to innovation when the right conditions, design, and processes are in place to facilitate sustained and productive collaboration. They identify four important drivers of collaborative innovation: the construction of policy or service problems with a great sense of urgency; the presence of a strong interdependency between empowered and committed actors; agreement on the overall mission and a high level of mutual trust; and the likelihood of significant gains from public administration.  Bekkers and Tummers (2018) take a different line of inquiry in collaborative or networked governance in service of innovation. In their study they theorize four dimensions of collaboration: a high or low degree of willingness to collaborate on the government side; and a high or low degree of willingness to collaborate on the citizen side. This focus on willingness is an interesting and less neutral position to explore that also links directly to the position of power that public sector tends to occupy and how this relates to strength of networks and amount of collaboration that is present in an innovation effort. Considine and    26  Lewis (2007) have a related interest in the effects of external engagement with other governments and organizations relevant to their work, and social network analyses of individual connectedness across the system. They found here that strength and position within networks was more significant than position in the hierarchy for enabling innovation, pointing to a promising research direction to explore - how enabling conditions might be created throughout an organization, not only for those who hold positional authority in a hierarchy.    When the willingness to collaborate is high in both the public sector and with citizens, Blomkamp (2018) offers the potential for co-design to improve policy processes and outcomes. She defines co-design as the democratic involvement of a diverse range of participants in exploring, developing, and testing potential solutions to shared challenges. She also notes that co-design could be a response to the critique of dominant problem-solving and action-oriented practices of design that tend to be Euro-centric, monocultural, and with embedded neoliberal values. She uses academic literature to explore the different claims about the benefits of co-design, and then explores and critiques the potential benefits of this approach to innovation in public policy.  2.1.4.3 Leadership Leadership for innovation in the public sector is a rich enabling condition to explore and is nested within different governance paradigms and how they think about the construct of leadership. Lewis et al. (2017) note that there is a vast literature on charismatic or transformational leadership, and that this is often linked to enabling innovation. In the NPM literature discussed earlier, transformational leadership at senior levels in an organization is often described as the most important leadership style in that paradigm. However, this is informed by research conducted on private firms, and it generally has very little to do with what is known about linking innovation capacity to leadership in the public sector. Hartley et al. (2013) provide a different orientation and describe three leadership roles that can enable PSI including: conveners; mediators; and catalysts. This leadership can come from multiple sources and levels, but leaders must be connected to relevant stakeholders, have legitimacy    27  and access to resources, and have personal competencies like reflexivity, imagination, vision, flexibility, and open-mindedness.  Ricard et al. (2017) conducted a comprehensive study of 365 senior public managers in Copenhagen, Rotterdam, and Barcelona. These respondents were surveyed to understand leadership qualities that public managers regard as important for innovation. They found that leadership style was context dependent and different for each of the three cities studied. Across the three cities, the transformational leadership style which emphasizes vision and inspiration, and the collaborative/interpersonal style that is stimulating, risk taking, and collaborative, were both important for innovation. The other three innovation leadership styles – transactional, entrepreneurial, and network governance – were not as consistently reported by the three cities as being important for innovation but were all acknowledged as important and relevant in at least one of the cities studied.  Together this collection of insights and studies into the conditions that enable innovation in the public sector show the wide-ranging, diverse, sometimes divergent, and under studied nature of this aspect of the field. Different authors choose theoretical or survey- and interview-based studies to explore enabling conditions from their particular fields of interest and expertise, and both quantitative and qualitative approaches are taken. From this literature review, it does not look like any of the theorized frameworks or research designs has been repeated to verify findings beyond the initial study. These studies are also interesting in their diversity. There is clearly a curiousity about the naming, describing, and understanding the relative importance of different enabling conditions for PSI. There is also significant divergence in what is being questioned and revealed through research, and fairly consistent recognition in the literature that enabling conditions are context dependent on many variables. There is not a consistent connection between governance paradigm of focus, definition of innovation, and then a discussion on enabling conditions from within those more nuanced frames - ‘innovation’ is largely treated with one broad brush in these studies. These findings point to how enabling conditions may not be static or consistent, they may vary widely depending on context and conditions, and can change over time as context and conditions change. There is not yet a consensus about strong enabling conditions for PSI in the literature. Perhaps enabling conditions are more alchemy than recipe, or perhaps there    28  are some different ways to think about and theorize this important aspect of PSI research and practice to get to some ideas and frameworks that are more broadly applicable.  2.1.5 Barriers to Public Sector Innovation A discussion about enabling conditions is now balanced out with what the literature has to say about barriers to PSI. Sørensen and Torfing (2011) name five potential barriers to collaborative innovation that sparked the framing for this section: cultural; institutional; inter-organizational; organizational; and identity-related. Each of these is discussed in turn next, with institutional and organizational joined together, and interorganizational and identity-related joined together as well, acknowledging that there are overlaps between the descriptions of these categories. Standard performance management and evaluation methods has been added as an additional barrier to PSI.  2.1.5.1 Cultural Barriers Sørensen and Torfing (2011) describe cultural barriers as the prevalence of a legalistic, zero-error culture and paternalistic professional norms. Torugsa and Arundel (2016) further elucidate this type of barrier as: an unwillingness of managers to take risks; resistance to change by managers; employees believe their ideas will not be seriously considered by managers; political uncertainty; and lack of incentives. In their study they found that employees involved with complex innovations were more aware of a greater number of barriers than employees involved with less complex innovations. They also found through their survey that employees reporting multiple innovation barriers were also highly motivated and creative, qualities that might lead them to find means to overcome these barriers to complex innovations resulting in significant solutions.   De Vries et al. (2018) point to the importance of understanding the perceptions and experiences of innovation adoption to people at different levels in a public sector organization. Their study of governments in the Netherlands implementing a new telework innovation is one of the few studies that looks ahead to what happens into implementation, scaling, and adoption of an innovation in the public sector. It also investigates the different    29  perspectives from senior management and workers on the effects of this top-down innovation, noting that these perspectives can vary in significant ways. Experience and perspective appear to be critical to being able to see and work with cultural barriers that present themselves in innovation efforts.  Brown and Osborne (2013) explore the specific cultural barrier of risk, describing how risk is understood and managed in the public sector. They theorize a framework that is fit-for-purpose in cases where innovation is sought, which they call risk governance. It considers the type of innovation that is being implemented, the type of risk that may result and who/what it might affect, and then gather the necessary information to collaboratively negotiate a response with affected stakeholders. This response considers not only the risk of implementation, but also the potential benefits and the risks associated with non-action.   2.1.5.2 Institutional and Organizational Barriers Institutional barriers result from a strong separation of politics and administration and use of inappropriate designs for dialogue with users (Sørensen & Torfing, 2011). Organizational barriers arise from a lack of focus on innovation, and the absence of procedures for exploration and exploitation (ibid). Although Sørensen and Torfing describe these as two categories, the literature overlaps these two to a significant extent so that has been done here as well. This is the most explored type of barrier to PSI in the literature.  The particular structures of public sector organizations - hierarchy, horizontal and vertical silos, lack of integration, power structures, barriers to knowledge sharing and information diffusion, and traditional decision-making processes and roles - are understood to inhibit innovation by many researchers in this field (Boukamel & Emery, 2017; Carstensen & Bason, 2012; Demircioglu & Audretsch 2017; Timeus & Gascó, 2018). The literature also points to barriers related to evaluation, resilience, and implementation of innovations. Evaluation processes are often inadequate to inform policy decisions in timely, meaningful, and forward-looking ways, and scaling promising innovations and solutions is challenging within government (Carstensen & Bason, 2012).     30  Several researchers describe institutional and organizational barriers as including the lack of organizational strategy for innovation, the lack of formal innovation processes, and goal ambiguity as barriers (Boukamel & Emery, 2017; Carstensen & Bason, 2012; Demircioglu & Audretsch, 2017). Sørensen and Torfing (2011) note that most of the time innovation is episodic and accidental and does not leave the organization with an ongoing capacity to innovate. This may speak indirectly to the amount of human and financial resource dedicated to innovation work, although the literature did not speak to this directly, perhaps because it is difficult to parse innovation budgets and job descriptions out of public sector records.  The tendency of public sector organizations to prioritize operational and financial work, and the preference for stability, predictability, and rule-following are barriers to innovation (Timeus & Gascó, 2018). A lack of organizational ambidexterity (i.e. working in both exploitation and exploration modes), and a tendency for inertia and stagnation in the exploitation mode, are also described as barriers (Boukamel & Emery, 2017; Sørensen & Torfing, 2011). The political economy of public sector organizations, including the performance criteria for governments shaped by different actors with political authority and influence and by political and media scrutiny are noted barriers as well (Demircioglu & Audretsch, 2017; Tõnurist et al., 2015).  Bason (2017a) states that much innovation in the public sector is currently driven by specific individuals more than by established organizational structures, processes, policies, or collaborations. Barriers also include a lack of experience, capacities, competence, and training for civil servants to use innovation frameworks and methods in their work. This would include approaches like taking risks, cooperating across silos, exploring divergence and new ideas, and putting citizen experiences first (Carstensen & Bason, 2012; Demircioglu & Audretsch, 2017). A related barrier is the proliferation of performance measures that typically prevent innovation because they are often focused on known inputs and outputs, and do not leave room for uncertainty or ambiguity (Sørensen & Torfing, 2011). Tõnurist et al. (2015) add that the public sector tends to attract risk-averse individuals and support risk-averse organizational behaviours. Most of the literature about barriers, and enabling conditions as well for that matter, focuses outwardly on organizations and culture. There is not much attention paid to the internal conditions of individuals and the role that this plays in PSI.    31   2.1.5.3 Interorganizational and Identity-related Barriers Interorganizational barriers arise out of the predominance of bureaucratic silos, boundary wars, and groupthink, and identity-related barriers occur when the identities of key stakeholders prevent collaborative innovation (Sørensen & Torfing, 2011). The complex and interrelated nature of public sector services can mean that altering one activity can cause challenges with other activities at a larger scale because they are so deeply networked (ibid). This set of barriers includes those operating at the scale of a larger system, network, or community.  Kronsell and Mukhtar-Landgren (2018) identify a barrier as the role that the public sector tends to play in innovation processes that are concerned with democracy, inclusion, legitimacy, power, and transparency.  They identify the types of policies or domains that are the right fit for a public sector organization to act on based on their responsibility and jurisdiction. They raise the question of what the unique role and responsibility of public sector organizations is in preventing or enabling innovation given their role in society. Whose job is it to innovate on societal challenges? It is important to remember how paradigms of governance and definitions of innovation will significantly affect deliberation on these questions.   Hartley et al. (2013) are concerned with collaborative innovation as a particular paradigm with potential to catalyze innovation in the public sector. They describe some of the things that can create barriers to working in this way. First, it is challenging to bring all of the relevant stakeholders together in a robust and ongoing process because the public sector is generally not practiced in working in this way. Second, actors are generally more tuned to collaboration in transactional, self-interested forms where the benefits to themselves or their organization are clear; however, more relational interactions that create trust, comfort, certainty, and understanding are less common. Third, collaboration may not result in, or foster innovation and/or implementation - there are no guarantees of the results of collaboration. Hartley et al. (2013) also note a relevant higher-level barrier to innovation in    32  the form of a reticence to reformulate traditional roles and relationships of public sector representatives and community actors.   This section explored a wide range of divergent ideas about barriers to PSI, informed both by NPM as well as network and collaborative governance paradigms. In a study that tests different conditions that may contribute to enabling innovation in the public sector, Demircioglu and Audretsch (2017) found that increased barriers to innovation is the most significant predictor of innovation capacity in the public sector. Meaning that tending to barriers can actually become enabling conditions if, they posit, those who understand these barriers could focus on reducing them as an effective innovation strategy in itself. This inspires a reframed view into the barriers described in this section as potential sources for innovation.   2.1.5.4 Performance Management as a Barrier Performance management is the dominant mode of evaluating and measuring impact in public sector organizations (PSOs), and it focuses on data, indicators, reporting, and accountability toward meeting clear and established outcomes (Preskill et al., 2013). The section explores how a performance management approach to evaluating impact may be a barrier to PSI. In a study into evaluation practice in the Canadian government, Segsworth (2005, as quoted by Mayne & Rist, 2006), concluded that evaluations in the Government of Canada from 1977 to 2005 indicate that very little fundamental change to the function being evaluated resulted from the evaluation process. So why might this be, particularly as the nature of the challenges that governments are facing are becoming more acute and complex? What approaches are being used to evaluate and understand impacts of work in the public sector, and are those well suited to some of the complex challenges that governments currently face?  Antadze and Westley (2012) and Mulgan (2009) describe several types of evaluation practices typical in government and arising from conventional accounting practices. These practices include cost-benefit analysis, social return on investment, public value assessments, and government accounting measures. These measures are primarily interested in    33  productivity and value for money, are performance driven, and seek to find relatively simple and direct cause-effect measures.   Patton (2017) notes that evaluation originated in projects, using tools like SMART (specific, measurable, actionable, reasonable/realistic, and timely) goals. Traditional evaluation approaches advocate for clear, specific, and measurable outcomes that can be described and realized through the application of linear logic models. Mayne and Rist (2006) surface a limitation of this project-based evaluation approach typical in the public sector, where specific programs are evaluated without there being a synthesis and integration of the knowledge gained across these evaluations, and a subsequent integration and adaptation of management practices and policies as a result of this discovery and learning. They also note that evaluations are often seen as adversarial rather than viewed as an important activity to facilitate organizational learning and improvement.   Forss et al. (2011) observe that governments need to demonstrate that resources invested in policy initiatives have resulted in good value for money. They also describe three overlapping ethos’ of contemporary management as a potential cause for the increased demand for evaluating complexity including: accountability; results-based management; and evidence-based policy. Evidence-based policy is on the rise as an ‘innovation’ practice within many governments and is an example of a linear logic model for change. Evidence-based policy-making follows a series of steps that typically include (Cairney, 2016; Stoker & Evans, 2016):  Determining a potential policy intervention to affect a desired and specific change (i.e. fewer seniors falling and injuring themselves);  Reviewing existing literature, usually generated using recognized social science research methods and published in journals, to determine what evidence exists about this intervention;   Developing and testing a theory as to why the policy will be effective, what the impacts of the policy will be if it is successful, and comparing this against a counterfactual (i.e. what would have happened without this policy intervention);  Measuring the direct (and sometimes indirect) effects that occur because of the policy, most often using randomized control trials; and  Sharing results in a way that is replicable and testable by others.    34   Approaches like evidence-based policy do not often consider the unintended or whole system impacts of an intervention, they often lack enough contextual nuance to be sure that an intervention that works in one place will have the same effects in another, and they generally underestimate the complexity of public policy and the dynamics of power and politics in public sector systems (Cairney, 2016; Stoker & Evans, 2016).  Puttick and Ludlow (2013) describe five levels in Nesta’s Standards of Evidence required for strong evaluation which further illuminate this linear, performance-oriented, and causation driven approach to measurement that is quite common in the public sector: 1. You can describe what you do and why it matters, logically, coherently, and convincingly. 2. You capture data that shows positive change, but you cannot confirm you caused this. 3. You can demonstrate causality using a control or comparison group. 4. You have one or more independent replication evaluations that confirms these conclusions. 5. You have manuals, systems, and procedures to ensure consistent replication and positive impact.  To close this section, the academic literature on PSI is reasonably robust, comes from several academic lineages, and contains different frameworks, understandings, and perspectives. In summary:  Most public sector innovation efforts do not define what they mean by innovation, and about one third do not set goals for their work;  PSI is categorized into a number of different types, including service, organizational process, and ancillary;  Significant enabling conditions to PSI can take the form of structures, governance, networks, collaboration, and leadership;  Barriers for PSI tend to include cultural, institutional, organizational, interorganizational, and identity-related, and performance management approaches to evaluation.     35   There is a great deal of interesting thinking and research to draw from, although more recent scholars of PSI acknowledge that not long ago much of the literature, practice, and understanding of innovation in the public sector was largely an attempt at direct translation from business to PSO contexts. We now turn to the specific, contemporary construct of public sector innovation labs (PSI labs) as an intervention designed to catalyze PSI.  2.2 Public Sector Innovation Labs There is a rapid proliferation of public sector innovation labs (PSI labs) occurring around the world, with estimates that more than five hundred are now in operation, most of which have started since 2015. Much like defining PSI, the definition of a PSI lab is still contested, although they usually share the common elements of describing their relationship with government, their innovation ambition, and the methods or techniques that they use. A good example of this is from Gryszkiewicz et al., (2016) who describe a PSI lab as “..a semi-autonomous organization that engaged diverse participants - on a long-term basis - in open collaboration for the purpose of creating, elaborating, and prototyping radical solutions to open-ended systemic challenges” (p. 84). Although there are some attempts at convergence around certain definitions, descriptions, purpose, and function of PSI labs in literature and practice, this is still very much a field in development, and marked by a great deal of divergence.   This section discusses aspects of PSI labs explored in academic and practitioner literature including: typologies; design approaches; social innovation approaches; innovation capacities; and evaluation.  2.2.1 Typologies This section summarizes findings from research into PSI labs that aim to describe and organize PSI labs into typologies and is organized into four frames: generations; purpose; position and methods; and practitioner generated. Researchers that have developed their    36  take on typologies have generally drawn from the first generation of twelve (or so) PSI labs and extrapolated typologies from them, and many of these early labs informed one another. So, the sample size and diversity that is drawn from in this section is quite small and self-referential, and is predominantly European, which should be kept in mind when reading it.  2.2.1.1 Generations of PSI Labs One way that researchers are categorizing PSI labs is concerned with how they change or develop over time, arguably with a progression to more strategic positioning and significant impact. Carstensen and Bason (2012) describe three generations of innovation labs that, although now somewhat dated given the rapidly changing nature of the field, is still useful framing. They describe three generations of labs as:  1. Creative platform - focused on employee-oriented ideation processes that aim to create buy-in to trying new methods;  2. Innovation unit - focused on user-centred value creation and using a wider range of different innovation processes and methods; and  3. Change partner - centering both users- and the organization and working on transformation of core public sector organizational narrative and processes.   A potential fourth generation could be added to Carstensen and Bason’s (2012) typologies to include systems-oriented change or transformation. Zivkovic (2018) offers what she calls a ‘systemic design lab’ as being best suited to work on wicked/complex challenges as compared to other PSI lab typologies. This fourth generation, systemic change/transformation, focuses on working with complexity through systems practice, and also has an outward orientation, with interests in collective action and recognizing government as an enabler of change. It also seeks a set of interconnected innovations in both the parts of the system, and their interactions that make up the whole. Zivkovic’s (2018) paper is unique in the PSI lab literature in that it blends different principles, frameworks, and purposes from a wide range of literature, and also focuses specifically on public sector innovation achieved through labs.     37  2.2.1.2 Purpose of PSI Labs Another strategy to categorize PSI labs is by their purpose. Tõnurist et al. (2017) theorize the purposes for initiating PSI labs through literature review and a survey of 35 PSI labs. They begin their paper by examining a wide range of organizational change theories, many originating from literature about the private sector, with an aim to use these theories to describe the different reasons why PSI labs may be created. Their articulation of six propositions for why PSI labs might be created includes: 1) external/environmental complexity, 2) technology, 3) competition between old and new structures, 4) emulation, 5) consolidation of expertise/legitimacy, and 6) learning. An important contribution from this paper to the field is an acknowledgment that different core assumptions and values underlie these different organizational change theories. Some of these connect to the drivers and enabling conditions for PSI discussed in sections 2.1.4 and 2.1.5.   2.2.1.3 Position and Methods of PSI Labs McGann et al. (2018) studied 20 PSI labs and categorized and compared them to one another using the following criteria:   Government oversight (independently run, government enabled, government led, government controlled);   Government funded (none, partly or wholly).   Methods, tools, and techniques used (design-led [1/2 studied]; open government/data; evidence-based; mixed methods)   Stage in the policy-making process that they focus on (most in generating and testing solutions, very few on implementation and/or scaling or decision-making).  Tõnurist et al. (2015) categorize by: method (i.e. design, behavioural, technology focused); sector or issue; relationship with government; the level of government they work with/in; and their change orientation as incremental or systemic. Tõnurist et al. (2017) take a stronger posture about better ways of positioning PSI labs including: a semi-separate or independent structure from their home organization; the ability to attract external funding and resources; the capability to operate with relatively small budgets and fluid and agile structures; and skills in networking, sales, and persuasion to increase the effectiveness of their work within the    38  public sector. In this study they found that the labs studied used relatively simple methods, looked for quick solutions even when faced with complex systems and political dynamics, and often backed out of working through higher level policy change, longer-term engagement, or political tension when it arose.   Hartley et al. (2013) offer some different ideas about categorization by position and method through their five typologies for collaborative governance that support PSI. Although their work does not focus specifically on labs, this categorization is relevant here with its focus on the network and collaborative governance paradigm discussed earlier and includes: intra-organizational skunkworks; inter-organizational networks; governance networks; public-private partnerships; and crowdsourcing. Gryzkiewicz et al. (2016) use a related frame - openness - from which they explore PSI labs. This frame includes open innovation, open science, and open government. The four labs that they examine from the perspective of openness literature are not wholly in the public sector, though they involve and aim to influence the public sector making this relevant here. Gascó (2017) has a similar interest, that of the form of living labs in the public sector, which also has an open innovation and collaborative purpose and approach.   2.2.1.4 Practitioner Generated Typologies In recent years there have been efforts by practitioners interested in supporting field-building to summarize the state of the PSI lab field, and to provide frameworks for how to think about and work on innovation in the public sector. Several of these studies are used here to frame current practices used by PSI labs, with Wascher et al. (2018) noting that much of the academic literature about PSI labs is practice-oriented as well, and that more rigorous research into the functioning and impacts of labs, and if they contribute to significant change efforts, is just beginning.   In 2017, Papageorgiou from Esade Institute for Social Innovation conducted 30 interviews, reviewed reports, and then convened practitioners and enablers of social innovation labs (not only those in the public sector) to create their description of the state of the field. This data revealed that labs orient toward one of four specific approaches: research method, process,    39  or approach; learning and capacity building tool; convening platform; and/or agent of change. Puttick et al. (2014) were also curious about the different approaches of PSI labs, and in their research found that labs focused on one or more of the following activities: creating and developing solutions to solve specific problems; engaging and enabling citizens, non-profits and businesses to find new ideas; transforming the processes, skills and culture of government; and achieving wider policy and systems change.   The Organisation for Economic Co-operation and Development (OECD) published a paper in 2015 that aimed to make a contemporary case for innovation in the public sector. They take an innovation infrastructure approach, and describe four capacities required to build this infrastructure including: focusing on people (skills, values, culture, leadership); putting knowledge to use; attending to the ways of working (collaboration, structures); and rethinking rules and processes. It is one of the few papers that speaks to developing innovation capacities by paying attention to social, organizational, and cultural infrastructures and includes the individual as an important unit of analysis. As these three practitioner-oriented approaches to categorizing the practices and approaches to PSI labs show, there is divergence in the way that PSI labs are categorized by purpose, theory, framework, or approach.   Nesta, the UK innovation foundation, launched the States of Change program in 2018 which aims to build the capacity and impact of innovation labs in the public sector. They produced their Playbook for Innovation Learning (2018) to provide a set of frameworks that can be used to support PSI lab activity. The Playbook is organized into five thematic areas: learning processes and strategy; competencies and expertise levels; content and communication; design and innovation processes; and team and innovation strategy. Of particular interest for this course of study are two diagrams that help to describe and contain the current frameworks and methods used by PSI labs. The first figure they call Principles of Innovation (Figure 3), which describes the different domains that an innovation process or lab should be working with in order to make sure that it is working at the level of principles and mindsets and not only at the level of tools and tactics. Many PSI labs tend to begin with a specific method or tool, and then apply that tool to all of the work that they determine requires an innovation approach, rather than beginning with the purpose and mindset shift required to    40  work differently on a wide range of challenge types and then selecting appropriate methods and tools from this grounding.   Figure 3: Principles of Innovation (Nesta, 2018)  A second diagram in Nesta’s Playbook (2018) further illustrates this point. Figure 4 shows the many different techniques used in innovation labs and organizes them into four general types. Although there are many different ways to produce something like this, what is useful to emphasize is how many different techniques there are to draw from, and that the real skill in innovation lab work is about discernment of the right technique for the right purpose, and of high-fidelity execution of the technique in the specific context in which it is being used. Ryan (2014) cautions against an over-reliance on particular approaches, especially those techniques that are prescriptive, create a high level of procedural constraint, or are too complex for most people to understand as this will likely undermine a process. Ryan (2014) instead encourages a looser application of technique, staying focused on how they create new insights, encourage emergence, and allow for exploration, iteration, and divergence.     41   Figure 4: Innovation methods and tools (Nesta, 2018).  The Brookfield Institute (2018) provides another view into current innovation lab practice, considering different areas of innovation focus (policy and service), as well as who is involved in the process (participatory and expert). Figure 5 shows how they then position different methods in common use in innovation labs using these criteria to help describe which methods are appropriate in different cases. As described earlier, there is a much richer set of PSI lab typologies beyond policy and service, so this figure is quite limited from that point of view. What this figure does do is capture the generally held understanding of the state of practice and includes design, technology, behaviour, and systems method orientations that labs hold in one relatively simple figure.      42   Figure 5: Innovation lab methods (Brookfield Institute, 2018, p.8)  There are some areas of convergence in the typologies of PSI labs; however, there is still a great deal of variety in academic and practitioner literatures. There tends not to be a strong connection between typology and governance paradigm or innovation definition, and there is a tendency to drift toward describing PSI labs by the techniques, tools, or methods that they use. Because PSI labs are so heterogeneous, this makes them difficult to map, study, analyze, and compare. The two most common approaches used by PSI labs are described in the next sections: design and social innovation.  2.2.2 Design Approaches Although most labs use a mix of methods, recent research and the widespread use of techniques like idea generation, user research, and prototyping indicate that design approaches are likely the most widely and commonly used by PSI labs (McGann et al., 2018; Lewis et al., 2020; Puttick et al., 2014). Some examples of well-known PSI labs that predominantly use design to guide their practice are the former Laboratorio Para la Ciudad in Mexico City, Policy Lab UK, and the former MindLab in Denmark. Design is a term in broad use with different definitions and practices, so some distinction is needed here. There are two generalized design approaches used by PSI labs: the approach characterized by human-centred, product, and service design; and a strategic and systemic design approach.      43   Buchanan (2001) articulates this further in his four orders of design: 1) symbolic and visual communications; 2) artifacts and material objects; 3) activities and organized services; and 4) complex systems and environments. Jones and van Patter (2009) referenced in Jones (2014) describe four distinct design domains or generations, building on Buchanan’s work, and acknowledge that design is now engaged in much more complex problems than in recent history:   1.0: Traditional Design – design as making, artifacts and communications;  2.0 Product/Service Design – design for value creation, as integrating; products and services;  Organizational Transformation Design – change oriented, design for work practices, strategies, organizational structures; and  4.0 Social Transformation Design – design for complex societal situations, social systems, policy-making, community design.  Many PSI labs use the term design quite broadly and loosely, so it is not always easy to tell what is guiding their work without inquiring into their ethos and practices. What follows is a deeper dive into these two common design approaches being used in the the PSI lab field.  2.2.2.1 Human-centred and Service Design  Popularized by consultants and business people in the most recent ~15 years, human-centred and service design theories, frameworks and techniques were developed to create more user- or customer-centred goods and services in the business, and more particularly product development, worlds (Boland & Collopy, 2004; Brown, 2009; IDEO, 2015; Kelley, 2005; Knapp, 2016; Kumar, 2013; Stickdorn & Schneider, 2011; Verganti, 2009). These design processes follow a general arc of inspiration, ideation, and implementation and generally jump quite quickly into tools and techniques to deliver different parts of the process. Creativity, empathy, prototyping, user testing, taking risks and learning from failure, and iteration are popular practices in this approach. The frameworks and methods focus on putting people’s needs and wants at the centre of creative processes and using structured    44  creativity to invent novel concepts that improve lived experiences, mostly through products and services.  This version of human-centred and service design practice has been picked up by the public sector without too much adaptation to the practice. Advocates believe that if governments took the same ethos of putting citizens at the centre of their service delivery, many challenges within government could be solved. Although the theory is not clearly articulated as such because the focus tends to be on techniques, there are elements of a theory of change that can be inferred. This practice of design mythologizes the creative process and assumes that expertise lives predominantly within designers and creatives. It focuses on ideation as the main tool for getting to different types of solutions than would otherwise be possible and offers useful methods for prototyping and user testing. It assumes that users or customers know what is best for them, and that this can drive change if designers get more skilled at asking them what this is, and also better at designing and testing ideas based on their needs. This is obviously problematic in the public sector, where time horizons are long, there are deep societal inequities that need to be considered, where adjusting one area of work can affect others in unpredictable ways, and where it does not work to assume that asking individual people how to best solve problems to improve their experiences will result in system wide changes that benefit most people and the environment for the long-term.   2.2.2.2 Strategic and Systemic Design  This section gathers together researchers and practitioners that come to strategic and systemic design from different fields and lineages, and bring those theories, frameworks, and techniques with them to shape this version of design practice. Although this is not an acknowledged and cohesive design approach as of yet, both approaches share a systemic and strategic approach while remaining rooted in the larger discipline and creative practice of design.   Beausoleil (2016) describes strategic design as explicitly engaging in strategic and reflective thinking alongside creative action. It includes phases of research, analysis and understanding, experimentation, invention, and development. It attends to connections and    45  consequences, and invites and initiates diverse conversations, experiences, and social interactions in a complex communications process. It considers power relations, communications, and trust, and enables transformation to emerge into innovation.   Quayle (2017) explores mindset from a strategic design and leadership orientation, and describes ten principles that articulate a design leadership mindset and practice:  Make values explicit;  Know place and experiment;  Value diversity;  Emphasize edges and boundaries;  Bridge gaps and make connections;  Evaluate for fit, scale, and context;  Learn from natural systems;  Apply the Jane Jacobs test (i.e. interconnected thinking and doing, mixing and meshing methods, building a density of ideas, and applying focus);  Attend to patterns; and  Never finished but always complete (i.e. the quest for perfection continues but does not prevent the delivery of something that is excellent in the current conditions).  Systemic design is also becoming a relatively cohesive approach, with academics theorizing and researching the field, and practitioners learning, teaching, and practicing this approach. Jones (2014) describes systemic design as an orientation, or a next-generation practice developed to advance design practices on systemic problems, rather than as a specific design discipline like graphic or industrial design. Zivkovic (2018) emphasizes that a systemic design approach is particularly appropriate for working on wicked, or complex, challenges and also when place-based approaches are appropriate and essential. Systemic design can enable a transition, or a non-linear approach to catalyzing change. It can also support coherent action by diverse actors, co-creation with users, and networked governance (Zivkovic, 2018).      46  Systemic design purposefully integrates systems thinking and methods into human-centered and service design approaches to work on complex challenges. This approach tends to follow a variation of a design process of ‘ask-try-do’, ‘discover-design-deliver’, or ‘discover/orient-define/conceptualize-optimize/plan-execute/measure’ and cycles between convergent and divergent work with a strong action bias (Ryan, 2014; Quayle, 2017). In Figure 6 Jones illustrates the process of systemic design by layering a set of principles on top of a more standardized human-centred or service design approach. Where this diverges from human-centred or service design practice could perhaps be described as deeper attention to mindsets, context, power relationships, leadership, systems, and structures as well (Bason, 2010; Bason 2017a; Bason, 2017b; Quayle, 2017).     Figure 6: Design principles mapped to design model (Jones, 2014, p. 108)  Jones (2014) notes that a fundamental difference between systemic and human-centred design is that they have very different approaches to framing the problems of design and inquiry, even though they come from similar lineages and use some shared language. In referring to Buchanan’s fourth generation of social transformation design, Jones says that this fourth generation has not rooted itself in robust systems and complexity theory meaning that these influences are superficial, ignored, incompatible, or underused, with “insufficient competency to evolve or reconfigure these rigorous systems methods with new practices” from and with design (p. 123). He also notes that systems thinking has more than fifty years of    47  intellectual and practice-based development, whereas design, although having a long history of practice, lacks scholarly follow-through and intellectual development and rigour.  Ryan’s work (2014) takes this process and mindset focus further, and describes six main activities in a systemic design methodology as: inquiring; framing; formulating; generating; reflecting; and facilitating, all used in a nonlinear and iterative fashion with a logic that connects these activities into a strategic learning system. Ryan (2014) provides a framework for systemic design with three mutually reinforcing levels: mindset; methodology; and method. Ryan (2014) helpfully describes mindset further as including the following characteristics, values, and habits:  Inquiring, learning, curious, observant  Open, growth, defers judgment, seeks difference, willing to change one’s mind  Integrative, accommodating, utilizes tension, avoids black/white binaries  Collaborative, teamwork, builds shared ownership and accountability  Centred, mindful, reflective, self-aware  Beausoleil, Quayle, Ryan, and Jones’ thinking demonstrates how the deeper theoretical foundations that tend to be missing in human-centred and service design approaches are brought into this understanding of design. It also brings richness, diversity and depth through the work, perspectives, and practices of different researchers and practitioners that is lacking from the more creativity focused and tactically oriented human-centred and service design fields.   2.2.3 Social Innovation Approaches Social innovation is another approach that PSI labs take, though it tends to be more common in labs led by non-profits, foundations, educational institutions, and others not directly within government. Some of the labs that work within or to influence public sector and that predominantly use social innovation approaches include the City of Guelph’s Innovation Lab, Winnipeg Boldness Project, Auckland Co-Design Lab, and The Australian Centre for Social Innovation, Wicked Lab (Australia). The decade of work catalyzed by Social Innovation    48  Generation sparked a significant movement of social innovation leadership in thinking, research, and practice in Canada that is evident when Canada’s innovation lab ecosystem is studied and compared to those in Europe and elsewhere (Cahill & Spitz, 2017; Radius, 2018).   Westley and Antadze (2015) define social innovation as “a complex process of introducing new products, processes or programs that profoundly change the basic routines, resource and authority flows, or beliefs of the social system in which the innovation occurs. Such successful social innovations have durability and broad impact” (p. 2). Describing complex types of problems as different from clear, complicated, or chaotic problem types is important when framing social innovation challenges (Figure 7). Wascher et al. (2018) quote Howaldt et al. in their definition of social innovation as “practices in areas of social action, prompted by certain actors or constellations of actors with the goal of better coping with needs and problems than is possible by using existing practices. An innovation is therefore social to the extent that it varies social action, and is socially accepted and diffused in society. Depending on circumstances of social change, interests, policies and power, social ideas as well as successfully implemented social innovations may be transformed and ultimately institutionalized as regular social practice or made routine” (p. 9).     49   Figure 7: Cynefin framework (Snowden, 2020)  Murray et al. (2010) and Leurs and Roberts (2018) describe the stages of a social innovation process in similar ways, with the Leurs and Roberts version shared in Figure 8. This is interesting to compare to the process map for systemic design shared in the previous section (Figure 6).      50   Figure 8: Seven stages in a social innovation process (Leurs and & Roberts, 2018)  Westley et al. (2016) translate their definition of social innovation for use in the specific form of a lab. They intentionally define labs loosely as a process to support multi-stakeholder collaboration on a complex social issue that includes both systems and design thinking methods. The authors suggest that a social innovation lab should have the following characteristics: hold a deliberate intent to transform; take advantage of transitions and thresholds; be focused on innovating not just inventing; pay attention to cross-scale dynamics; and catalyze a range of potential innovations (Westley et al., 2016). Wascher et al. (2018) add that social innovation labs is a term used to characterize a variety of different organizational forms that offer a specific space and set of methods to organize and optimize the process to create socially innovative initiatives. They also say that labs help to convene a diverse group of stakeholders, and that real innovation and impact will only happen after the lab, once the interventions that they develop are implemented and we can determine if they have a useful impact or not. Note the significant difference of this definition when compared to that from Gryszkiewicz et al. (2016) at the beginning of this section on PSI labs.      51  Hassan (2014) describes what he calls social labs as having three characteristics: 1. They are social - bringing together diverse stakeholders from multiple sectors to work on a team that acts collectively; 2. They are experimental – ongoing and sustained iterations and experiments that manage a portfolio of activities, not one-off project-based interventions; and 3. They are systemic – going beyond addressing only parts of a system or symptoms, and instead seeking to intervene on root causes that create whole-system problems in the first place.  Scharmer (2016) has created a robust change framework that builds on the work of many researchers, practitioners, and theorists from a very wide range of fields. Theory U is a social innovation framework, although he does not explicitly label it in this way (Figure 9). It describes a change process that pays attention to transformation at different levels (personal, cultural, spiritual, systems), and describes many of the great divides that exist today between who we are as a society, and what we need to become in order to respond to ecological, social, and economic inequalities and crises. What Scharmer points to that is common amongst social innovation theorists and practitioners is that this type of lab practice has a more considered approach to change and transformation happening not only outwardly on a specific problem, but also at the personal, organizational culture, and systems levels.     52   Figure 9: Theory U (Adapted from Scharmer, 2016).  The later stages of a social innovation process focus on scaling impact to change systems. Etmanski (2015) identifies six patterns for scaling social innovation and impact including: think and act like a movement; create a container for your content; set the table for allies, adversaries, and strangers; mobilize your economic power; advocate with empathy; and ’who’ is as important as ’how’. In this vein of scaling, Wascher et al. (2018) are interested in the creation of social innovation ecosystems, and the potential for social innovation labs to contribute to these ecosystems. They explore this interest through action research into two municipal PSI labs in Germany. Their work is interesting as it responds to a common critique that labs are most concerned with the processes that they use, and that they do not pay enough attention to the impacts that they are having on the challenges that they work on. It is also interesting because they begin to articulate what a social innovation ecosystem might include, not only the individual efforts of specific labs, projects, programs, or other initiatives. They describe the following set of actors and settings as important parts of a social innovation ecosystem: resource providers; competitors; complementary organizations and allies; beneficiaries and customers; opponents; affected bystanders; politics and administrative structures; economics and markets; geography and infrastructure; and culture and social fabric. They also write about the importance of creating soft or social infrastructures that    53  facilitate interactions between the members of a localized innovation ecosystem to create trust, knowledge, exchanges of ideas, and cooperation.  There are connections between strategic and systemic design and social innovation that are apparent based on these definitions and descriptions. The most significant differences result from what researchers and practitioners choose to emphasize, often in response to their positionality, purpose, and primary orientation. Strategic and systemic design tends to bring a designerly mindset, lineage, frame of reference, training and set of experiences. Social innovators tend to bring systems theory and thinking, social sciences, and often movement building for transformative social and ecological change. Social innovators tend to view design as a method or technique within a broader theory of change, rather than a design as a theory of change in its own right. Social innovation methods pull from a very large and wide-ranging toolbox and include many of the tools that design-oriented practitioners use as well, particularly in the creative, user research, and prototyping stages.   2.2.4 Innovation Capacities There are distinctions to be made between competencies, capacities, and capabilities, although they are used somewhat indiscriminately and interchangeably in the PSI lab literature. Competencies describe knowledge, behaviours, abilities, and skills, and are concerned with ‘what to think’, and who knows how to do something, and can also be called horizontal development (Jaradat et al., 2018; Petrie, 2014a). Capacities are concerned with ‘how to think,’ and with mindsets, worldviews, attitudes, and methods and structures of thinking, also known as vertical development (Jaradat et el., 2018; Petrie, 2014a; Petrie 2014b; Petrie, 2014c). Capacities can help to grapple with the complexities of the contemporary world and to ask questions of: do we have enough and how much is needed; what is the level of awareness about how mindset affects meaning- and choice-making; and appropriate application of skills and methods (Yorks & Nicolaides, 2013)? Both competencies and capacities can be held at the individual as well as at the organizational levels, and these are interdependent constructions. Capabilities are dis - or enabling infrastructures, conditions, and processes through which capacities and competencies can be applied or inhibited (Jaradat et al., 2018). Capabilities intersect with the framing of enabling conditions    54  and barriers, discussed earlier. Capabilities consider context, available resources, and the interactions of the whole team to consider ‘how can we get done what we need to do?’ Dynamic capabilities are necessary to generate contextually appropriate responses to emergent, complex challenges.   Diving into the PSI lab literature, Timeus and Gascó (2018) define innovation capacity as “the set of competencies that public sector organizations have or need to utilize resources in order to adopt or develop innovations” (p. 993) and aim to make a clear distinction between innovation performance and innovation capacity, which they say are often not made explicit. They say that this distinction is important in order to help scholars and practitioners to understand, create, and evaluate skills and competencies to innovate when necessary, rather than assuming that more innovation is always a good idea and focusing on counting the number of innovations regardless of what is needed.  Gieske et al. (2016), propose a framework describing the attributes of innovation capacity in the public sector. They describe three levels: individual; organization; and network. They then use a variety of literature from different fields and disciplines to conceptualize innovation capacity, and use these to describe three different capacities at each of these levels including: connective capacity; ambidextrous capacity; and learning capacity. This theoretical approach to understanding PSI capacity creates a matrix that provides some very interesting fuel for thought. How might we understand when these different capacities are in place? How might they best be cultivated in practice through innovation labs? What might the outcomes of focusing on increasing these capacities be, and how might we assess them? Capacities can be grown at any part of the organization and system, and do not rely on existing, formal, or traditional sources and structures of power to enact.  Timeus and Gascó (2018) reveal that there is almost no academic research that focuses on the role that innovation labs play in increasing the innovation capacities of organizations in the public sector. They suggest a framework of four organizational capabilities to understand and analyze the specific contribution of innovation labs to growing capacity. These capabilities are: 1) idea generation; 2) knowledge management systems; 3) human resource strategy focused on innovation; and 4) intensity of technology use in the organization. They    55  then tested this framework with the City of Barcelona’s innovation lab and found that all but the technology capability was supported as important innovation capacities in that case. This framework is interesting because it speaks to standard organizational structures and processes that could be more intentionally used to serve growing innovation capacities. Of particular note, this framework is unique in its mention of a human resources strategy as important to PSI lab work. It also positions technology as an innovation infrastructure rather than the purpose or result of an innovation process, which may be more appropriate positioning in the context of complex, systems change oriented innovation labs. This paper also speaks directly to leadership as it relates to innovation capacity. They say that they “view managerial support as a necessary but not sufficient factor for innovation: it is only effective if employees have the competencies and skills to act on the leadership’s signals” (Timeus &  Gascó, 2018, p. 997).  The results of the Timeus and Gascó (2018) case study research into Barcelona’s lab revealed a finding of particular interest to this course of study. They found that the structure of a lab as a discrete, purpose-built innovation unit meant that the rest of the organization assumed that the lab will take responsibility for solving problems in an innovative way, and that the innovation capacities necessary to work in this way did not spread through the rest of the organization and remained contained within the innovation team. This innovation team structure also led to the team members feeling isolated from the rest of the organization, and vulnerable to being dissolved at any time without real consequence to the rest of the organization. They conclude that innovation labs appear to be a practical way for governments to meet pressures to be more innovative, without having real evidence that they accomplish that purpose and potentially aiding in the avoidance of larger-scale reforms.  Some focused field building work supporting innovation capacities and competencies is beginning from practitioner efforts. There are many toolkits with innovation techniques that PSI labs have published, which aids in codifying and sharing practice and also further demonstrates a focus on technique rather than strong theorization of PSI lab work. Leurs and Roberts (2018) have published a guide for innovation learning, that includes a more robust set of theories to guide innovation learning program development and delivery. This guide forms the foundation for a variety of learning programs and resources offered by the field    56  building organization, States of Change, focused on supporting PSI globally. Nesta reviewed and evaluated a community of practice in the Chilean government called Experimenta, which shares their capacity and competency development program and learning (Leurs et al. 2018). The most timely, topical, and relevant writing about this area is in grey literature, typically blog posts from practitioners, who quickly share their work and learning with the field to informally support shared learning. It is not clear what the reach, uptake, or impact of this sharing is for practice, and a useful research project might be to collect and analyze the capacity-focused grey literature in the PSI lab field to reflect back some higher order principles, approaches, practices, and learning.  2.2.5 Evaluation This section explores current thinking and practice relevant to building the art and craft of evaluation approaches applicable for use by PSI labs. The first section delves into promising evaluation approaches from the literature, and the second section discusses some of the frameworks being developed by PSI lab practitioners.   2.2.5.1 Evaluation Approaches from the Literature Evaluation theorists have been developing approaches for working with complexity, acknowledging that complex challenges have characteristics that require unique evaluative mindsets, approaches, and processes (Patton, 2017; Pawson, 2013; Preskill & Gopal, 2014). Patton (2011) characterizes complex systems as having the following attributes: nonlinear; emergent; adaptive; uncertain; dynamic; and co-evolutionary. Forss et al. (2011) and Cabaj (2018) offer some specific ways to work with this complexity as an evaluator. These include being concrete, inventive, flexible, and specific, and also understanding what type of strategic approach to innovation is being used and then designing an appropriate evaluative strategy from there. Lynn and Preskill (2016) suggest that the idea of rigour needs to be reframed when evaluating complexity, with a focus on evaluating: quality of thinking; credible and legitimate claims with transferable findings; cultural context and responsiveness to stakeholder values; and quality and value of the learning process. Several evaluation purposes (i.e. formative, summative, and developmental) as well as approaches to evaluation    57  inquiry (i.e. principles-focused and realist) that are particularly appropriate for evaluating complexity and innovation are described briefly here.  Formative and Summative Evaluation The purpose for formative evaluation is to form, shape, improve, standardize, and finalize a model program, practice, or other intervention. Formative evaluation is described as supporting revision, improving, working out challenges, filling in gaps, and getting feedback from users to get the intervention, or evaluand, ready for the rigorous summative review.   The purpose of summative evaluation is to test and judge. It is a preferred evaluation practice of government as it aims to establish clear cause-effect relationships, and because there are measurable outcomes easily included in performance management systems. Summative evaluation is designed to inform major decisions about the merit, worth, and significance of an intervention based on whether it produced the desired outcomes, and in order to determine its future. These two frameworks are fairly well-established and important components of evaluation in the public sector.  Developmental Evaluation (DE) DE is a type of utilization focused evaluation, which follows the guiding principle of the evaluation focusing on the intended use, by and with intended users, in every aspect and stage of an evaluation (Patton, 1978). DE is discussed in some detail here at it is a particularly promising framework for evaluating complexity, innovation, and PSI labs.  Preskill and Beer (2012) think that DE is best suited for evaluating social innovation, and that it has five characteristics that make it distinct from other evaluation approaches: 1. A focus on social innovations where there is no accepted model (and may never be) for solving the problem. 2. Continuous learning is intentionally embedded into the DE process. 3. An emergent and adaptive evaluation design ensures that the evaluation has purpose and that it can respond in nimble ways to emerging issues and questions. 4. The role of the developmental evaluator is a strategic learning partner and facilitator, which reflects a different role for most evaluators and their clients.    58  5. The developmental evaluator brings a complex systems orientation to the evaluation.  DE provides timely and appropriate information and feedback to social innovators in order to inform and learn from the adaptive development of interventions in complex and dynamic environments (Patton, 2011). DE practitioners are often part of, or working closely alongside, the teams and initiatives that they are supporting, and their evaluative practice can become part of the innovation process itself. DE is designed to work well in conditions of high innovation, exploration, uncertainty, turbulence, rapid change, and emergence. DE processes involve asking evaluative questions, applying evaluation logic, and gathering and reporting data to support project, program, initiative, product, and/or organizational development and learning.  Some of the evaluative questions asked and answered through DE include:   What is developing or emerging as the innovation takes shape?  What variations in effects are we seeing?  What do the initial results reveal about expected progress?  What seems to be working and not working?  What elements merit more attention or changes?  How is the larger system or environment responding to the innovation?  How should the innovation be adapted in response to changing circumstances?  How can the project adapt to the context in ways that are within the project’s control? (Preskill & Beer, 2011).  Principles Focused Evaluation (PFE) As a response to ongoing work with evaluating complex social innovations, as well as his field- and capacity building work with North American and global evaluation practitioners, another evaluation framework emerged for Patton (2017) which he calls principles focused evaluation (PFE). The distinguishing characteristic of PFE is that it focuses on principles as the object of evaluation. PFE is concerned with three primary questions: 1. “To what extent have meaningful and evaluable principles been articulated?     59  2. If principles have been articulated, to what extent and in what ways are they being adhered to in practice?  3. If adhered to, to what extent and in what ways are principles leading to desired results?” (p. ix).   Patton says that effective principles provide meaningful guidance, are useful and inspiring, are developmentally adaptive, and are evaluable (Tamarack, 2018). PFE puts the principles of the evaluators themselves in a more central position than other evaluation frameworks, and by so doing provokes some interesting thinking about making values and principles explicit in complex adaptive system interventions. By focusing on principles as the evaluand, this framework provides space and structure to surface different and conflicting values, creates wider and more diverse range of cultural perspectives to be present, and opens up different ways of being and knowing. This may lead to shifts in some of Meadows (2008) higher points of leverage, including values and mindsets, and may create impetus for reframing work on complex challenges from different principled groundings. Were et al. (2018) review Patton’s PFE methodology from their points of view as evaluators working in a Maori-led collective. They saw some alignment with many of their existing practices, and did not see PFE as a major new direction as Patton described, but instead as reflective of how Indigenous peoples have been drawing on guiding principles to inform lives and decision-making for millennia.    Realist Evaluation (RE) RE shares a focus on working in the terrain of intervening in complex systems but takes a somewhat different tack, and this section draws from several researchers working in this field (Henry et al., 1998; Pawson, 2013; Wong et al., 2016). Realist evaluation is theory-driven, and addresses the questions of what works, for whom, under what circumstances, and how. It places particular emphasis on understanding causation, and how causal mechanisms are shaped and constrained by social, political, economic, and other factors and contexts. This makes it a suitable evaluation process for certain types of evaluations, particularly those involving complex social systems and dynamics involving human values, decisions, and actions in a particular theoretical domain (e.g. human wellness, or early childhood education). RE considers the quality of reasoning, theory, and thinking behind a program, not only the quality of the data and the results.    60   Pawson (2013) describes seven organizing principles of realist evaluation: theory as the unit of analysis; conceptual abstraction as means to establish common language; reusable conceptual platforms and frameworks across interventions; model building to test and refine component theories; adjudication to determine whether an intervention works and to improve theory and model building; trust as mechanism to choose where to focus inquiry; and organized skepticism.  Of particular interest here is the framing of emergent or critical realist theory in RE, acknowledging that reality is socially constructed and not independent of experience. As such, a key component of a realist evaluation is a values-probe to understand the values that key users or participants bring to their perspectives and experiences with the activity being evaluated. RE surfaces the theories underlying the initiative being evaluated as a part of the evaluation, which is not a key feature in the other evaluation approaches discussed here. In some cases, a theory of change may do this, but RE is more academically rigorous in its approach to building a theory base, drawing more consistently from academic research.   This is an interesting potential evaluation approach in a public sector context where many of the operating theories, values, and principles are implicit in organizational culture but rarely named. These theories, values and principles are also rarely engaged with, and this could be of value when there are differences and tensions between departments with different purposes (i.e. finance and law compared to sustainability or social policy), or when tensions arise between a public sector organization and the different communities that it serves.  These are some of the most relevant and interesting evaluation frameworks when thinking about how to make sense and meaning from the work of PSI labs. Although they are being used in social innovation lab evaluation and learning efforts in other sectors, they are not in obvious use by PSI labs. This will be evident in the PSI lab evaluation frameworks developed by field building organizations discussed next.     61  2.2.5.2 Practitioner Generated Evaluation Frameworks Nesta has developed an impact evaluation framework (Figure 10) that they have shared with the field for testing, feedback, and iteration. This is the only PSI lab evaluation framework generated thus far that takes into account personal, organizational culture, and systems transformation measures (Nesta, 2018).   Figure 10: Culture change impact framework (Nesta, 2018)  The Organization for Economic Cooperation and Development (OECD) Observatory for Public Sector Innovation (OPSI) produced a blog post series in late 2018 that provided some interesting ingredients for another way to think about evaluating public sector innovation (OECD, 2018c). Their framing is based on the premise that innovation, or doing something new to the context, is inherently uncertain. The novelty of innovation means that the outcomes cannot be guaranteed or fully expected, and that unanticipated results will occur. This uncertainty can range from the very high, where you have no idea or relevant precedent that can tell you what might happen (e.g. disruptive technologies), to the relatively small, where you have a better chance of being able to assess and anticipate the potential impacts.    62  OPSI proposed a framework describing four facets of innovation: enhancement-oriented; mission-oriented; anticipatory; and adaptive (Figure 11). Note that these differ from the types of PSI in section 2.1.3 (i.e. service, organizational process, and ancillary); they are focused on ambition, or level of disruption instead of working in a particular governance activity. This is a good example of how the PSI lab field is still quite divergent in its efforts to describe and codify its purpose, approaches, and methods and in this case how that then translates to what is evaluated.     Figure 11: Facets of innovation (Adapted from the OECD OPSI, 2018b)  Each facet has its own unique characteristics and they provide a set of potential evaluative questions to help assess whether or not the innovation facet is appropriate for a particular challenge, and whether or not interventions are having a desired effect (Table 1). These are shared in some detail here, as the evaluative questions for each innovation type offer useful insight into the ‘how’ or process of evaluation and learning rather than only the ‘what’ that the Nesta framework focuses on.        63  Table 1: Public sector innovation types and associated evaluative questions (OECD, 2018) Innovation Types Evaluative Questions Enhancement-oriented  Innovations that focus on upgrading practices to achieve efficiencies and better results by building on existing structures. Example: using behavioural insights to improve compliance rates. Is there a relatively stable operating environment with the necessary time and attention to focus on enhancing the status quo? Is there consistency in key people and relationships? Are there strong structured learning practices in place? Is there a clear sense of intent, or a defined activity, that describes what is trying to be achieved? Are there clear accountabilities in place? Is there effective baseline information available? Are employees engaged and/or incentivized to suggest and implement enhancements? Is it a risk-averse or risk-conscious environment that leaves room for smaller changes but not more radical changes? Mission-oriented  Innovations where there is a clear overarching objective and direction for which innovation is harnessed. Example: going to the moon.  Is a clear, ambitious, and meaningful mission, goal, or outcome being sought? Does this goal have resonance, significance, energy, and activity from those involved? Are there tangible and meaningful indicators or objectives in place to measure and evaluate progress? Is a wider ecosystem of actors being engaged than the existing/status quo players? Is there a right level of political engagement or pressure present to mobilize power, commitment, and action? Are there sufficient investments of time, resources, attention, capacities, and capabilities?  Anticipatory  Innovations that explore new frames of reference and paradigms and are about recognizing and engaging with significant uncertainty about what works, what is appropriate, or what is possible. Example: government trying to understand AI in public sector context. Is there a clear sense of consequence of the risks of being blindsided and caught unprepared? Is there explicit freedom to consider the unimaginable, and to encourage and explore more radical ideas? Are unconventional partnerships and relationships encouraged in order to engage with very different perspectives? Is there a strong connection and line of sight with relevant decision-makers to ensure that the work stays connected to what matters? Are senior leaders involved with anticipatory processes?  Adaptive  Are innovation practices and learnings being socialized and diffused? Is formal and informal    64  Innovations that are about adapting to changing conditions and possibilities. Can be small or large in scope; incremental to radical change; operational, service, and/or policy oriented. Example: government adapting to rise of social media. learning about change in practice, emerging opportunities, and shifting expectation curated and cultivated? Are others being inspired and equipped to innovate in their own contexts? Are practices in place to celebrate and recognize that innovation is welcomed and encouraged even when it is not directed? Is there engagement and feedback loops with citizens to strengthen connections with what’s happening with citizen experience and respond as a result? Is there a permissive space for experimentation within clear boundaries for risk, experimentation, and authority?   To review, this section explored how researchers and practitioners are defining, researching, and practicing public sector innovation in the specific construct of a lab. In summary:  PSI labs are a relatively new and promising phenomenon, and are only beginning to be researched and written about in academic literature;  PSI labs are being codified and categorized in a few different ways, including by their generation/evolution over time, their purpose and principles, their position in relationship to government, and the methods and techniques that they use;  Many PSI labs use a mix of methods, with design and social innovation being most broadly used;  Building competencies, capacities, and capabilities for PSI through labs is an important potential lab activity or role, but is not deeply considered yet in PSI lab research or practice, with a few notable exceptions; and  Evaluation processes and approaches are emergent for PSI labs, and there are some promising approaches to strengthen this thinking and practice.   There are significant areas of divergence and contestation in this field-in-development, making it an interesting and important area to research. Innovation is a highly nuanced and contextual term, with many ways to understand, define, and practice it. After exploring this academic literature, immersing in practitioner work (both my own and that of others), and examining my own positionality and ambition for innovation work in the public sector, it became clear that I am most interested in an ambitious, transformative innovation practice    65  with purpose and values of sustainability, equity, and decolonization. This led to an immersion into literature about transformation and how to understand what it might mean in theory and practice in a PSI lab context. The learning from this review of literature is discussed next.  2.3 Change vs. Transformation Orientation to Innovation In reviewing PSI and PSI lab literature and practice, and in considering my own interests and action research orientation, I became curious about what a more ambitious approach to PSI might look like. Westley et al. (2011) say that “to support [9 billion] people without transgressing critical planetary boundaries, efforts to diffuse and scale the most promising innovations must be accelerated. This requires the transformation of the institutions that shape our cultural, political, and economic transactions – in short, shift our governance processes from those that do not privilege systemic innovation to those that do” (p. 775).   The literature reviewed so far here about network and collaborative governance, strategic and systemic design, and social innovation started to point in this direction; however, much of the literature reviewed here takes an implicit change-orientation to what is meant by innovation and works within dominant paradigms and systems of governance. I felt that it was important to explore literature about change, transformation, and emergence as different innovation orientations, as well as how innovation processes might be enacted in interconnected ways at the personal, organizational culture, and systems levels. This was important in order to stretch beyond the current ways PSI labs were being theorized, conceived, and practiced and to enable new and different thinking and possibilities. The literature review now extends into these questions, building from what I had not yet found and was seeking, as well as what was emerging as an important contribution for me to make through this dissertation.  My interest in researching PSI labs is informed by two decades of work and experience in trying to transition complex systems toward more ecologically, socially, and economically just paths and directions. I am committed to a version of PSI, and of PSI lab practice, that is    66  working toward these aims. I am not interested in the version of innovation that continues to hold up neoliberal, colonial, and patriarchal forms of governance, problematic systems and structures of power, and other practices that are masked by under theorized and loosely defined approaches to PSI. This positioning within the body of PSI, and PSI lab research and practice, led to literature exploration into conceptions of transformation which are explored in this section.  Also shaping my thinking and interests, and not represented in the PSI lab literature in any significant way, is that innovation is not only an organization- or issue-oriented activity. Innovation must also consider the internal or interior conditions of the innovator, and who and how they are in relationship with others and the systems that they are a part of. Innovation must also think and act systemically to ensure that it considers interconnected relationships when intervening in complex challenges. There is a body of literature about personal, organizational culture, and systems transformation considered here.  2.3.1 Personal Transformation Omer et al. (2012) offer a strong definition of personal transformation, noting that it is both an individual and collective process, and that “transformative learning entails shifts that have been characterized as shifts in perspective, perceptual lenses, core beliefs, schemas, mental models and mindsets. Such perceptual shifts enable individuals and systems to inhabit new, more complex and emergent landscapes” (p. 375).  Castro Laszlo (2012) says that systems thinking is a gateway to seeing interconnections, and that once we see and feel a new reality, we cannot go back and ignore it. She asks a series of questions that speak to the transformative potential here, and connect personal and systems transformation processes: “What are the emotions evoked by perceiving for the first time the unity, interconnectedness, and relatedness of a system? What are the feelings evoked by perceiving and experiencing disconnection and isolation? The journey from systems thinking to systems being is a transformative learning process of expansion of consciousness – from awareness to embodiment” (p.98).     67  Very few PSI labs include personal transformation as a part of their intent, process, or desired outcomes, and research into PSI labs rarely considers this aspect to, or purpose for, an innovation process. Current codification of useful PSI lab practices is largely missing the personal. Current practices tend to focus on outward, external, and outcome-oriented approaches, and do not directly attend to how individuals show up in change processes. The practices included here, as well as many others from psychological, sociological, spiritual and other lineages, can bring attention to personal transformation being an important and largely missing part of current design and social innovation-oriented PSI lab theories, frameworks and techniques.   Authors here come from fields associated with transformative learning, integral theory, adult learning and development, and adaptive leadership, among others, and there is a blend of theoretical and applied works. Notably absent are works from PSI lab researchers or practitioners, as a clear or explicit focus on personal transformation in PSI labs does not appear to be a current focus. Anecdotally some PSI labs are integrating personal transformation as a part of their lab activities, most often when there is an innovation learning or professional development focus to their work.  2.3.1.1 Transformative Learning Mezirow (2000) defines learning as occurring “...in one of four ways: by elaborating existing frames of reference, by learning new frames of reference, by transforming points of view, or by transforming habits of mind” (p. 19). O’Sullivan et al. (2002) take a more socially conscious approach to describing transformative learning. They describe it as interdisciplinary, multicultural, deeply grounded in a critical response to the neoliberalism of education, and having a strong social change focus as well as transformative change at the individual level. They also connect transformative learning to our current time, much like Westley et al. (2011) do, writing that we are at a transformative historical moment where humans have the power and responsibility to shape the direction that the planet will take.  Mezirow (2000) provides a description of the stages of a transformative learning process or experience:    68  1. “A disorienting dilemma; or finding of a ‘missing piece.’ 2. Self-examination with feelings of fear, anger, guilt, or shame. 3. A critical assessment of assumptions. 4. Recognition that one’s discontent and the process of transformation are shared. 5. Exploration of options for new roles, relationships, and actions. 6. Planning a course of action. 7. Acquiring knowledge and skills for implementing one’s plans. 8. Provisional trying of new roles. 9. Building competence and self-confidence in new roles and relationships. 10. A reintegration into one’s life on the basis of conditions dictated by one’s new perspective; revisioning of self in the eyes and responses of similar others and/or a beneficial cycle of desire, identification and re-apportion of a stronger subjectivity through relations with those who themselves successfully transcend oppression” (p. 22).   Taylor (2009) builds on this process to describe how the original core elements of a transformative approach to teaching have evolved to include additional elements including: individual experience for each learner; promoting critical reflection and examining deeply held assumptions and beliefs; dialogue with self and others; holistic orientation to teaching that engages other ways of knowing; awareness of context; and authentic relationships. O’Sullivan et al. (2002) add that within a more visionary version of transformative education there are responsibilities of: living within planetary context and constraints; educating for integral development; redefining quality of life for all; inclusion of diversity; and growing civic cultures. O’Sullivan et al.’s (2002) work makes strong links between individual transformative education and social action and responsibility, and says that transformative learning approaches need to include both.  2.3.1.2 Integral Theory Wilber’s (2005) work on the individual interior ‘I’, or ‘the me that you can’t see’ quadrant, is interesting to consider when thinking about personal transformation in a PSI lab context. In his all quadrants, all levels, integral operating system, he describes five elements as essential    69  to unlocking human evolutionary potential, drawing from a very wide variety of different theoretical fields and lineages. These five elements are: quadrants; levels; lines; states; and types. An example of particular relevance to transformative learning is to consider the developmental lines in the individual interior quadrant in Figure 12 below. For each of these lines there are different developmental stages (Figure 13). These stages can be described in different ways, but one that is useful here is related to Kegan and Lahey’s (2001, 2009) description of three adult developmental learning stages: the socialized mind; the self-authoring mind; and the self-transforming mind. An individual could be at different stages for each of the developmental lines, it is not evenly distributed. Wilber’s work weaves together a depth of personal transformation theories and frameworks, although it may be challenging to turn this theory into practicable frameworks and techniques in a PSI lab context.    Figure 12: Example of a psychograph showing different developmental lines in the individual interior, or ‘I’ quadrant (Wilber, 2005, p. 28).     70   Figure 13: AQAL Framework showing an example of developmental stages for the transpersonal developmental line (Wilber, 2005)  Petrie (2014a, 2014b, 2014c) has a related interest in leadership that he came to through his practice, that connects to these more theoretical findings from Wilber, Mezirow, and Kegan. Petrie found that in his work in leadership training and education, he was not keeping up with the changing context and needs of leaders today. Through 30 expert interviews he uncovered four trends: a shift from horizontal (skills focused) to horizontal and vertical (stage focused) development; more responsibility and ownership for development needs to rest with individuals rather than their institutions; a greater focus on collective (rather than individual) leadership is needed to create conditions for leadership to flourish, spread and be democratized; and a greater focus on innovation in leadership development methods is needed. Petrie describes four leadership competencies as: strategic thinking; leading change; working with conflict; and leading across boundaries. He then says that each of these competencies has different developmental stages, and that leadership training and support needs to be designed with this developmental approach in mind. He also connects his leadership training and development to Mezirow’s disorienting dilemma saying that a person needs to feel heat, colliding perspectives, and elevated sense-making to spark their development process.  Transformative learning and integral theory offer interesting theoretical constructs through which to consider what personal transformation might look like in the context of a PSI lab.    71  Transformative learning tends to be applied in formal educational settings, with teacher/student relationships, and when students are intentionally choosing to partake in a learning process. PSI labs are not typically constructed in this way, and there does not seem to be much application of transformative learning in non-formal educational contexts so the translation of this framework into PSI and PSI lab contexts is largely unknown and untested. States of Change references Mezirow’s work demonstrating integration of this theory into PSI lab practice. Integral theory can be quite impervious, challenging to understand and unpack, and quite difficult to turn into actionable frameworks and theories in practice.  2.3.1.3 Adaptive Leadership and Learning Kegan and Lahey (2001, 2009) bring some of this theory into more actionable frameworks and methods through their ‘immunity to change’ approach. They build on the premise that adults can continue to learn and develop into adulthood through particular developmental learning stages that are possible in the post-teenage years. They say that this development may or may not happen, and will happen at different rates, dependent upon the unique experiences and motivations of each individual. They focus on the qualities of a disorienting dilemma particular to adult development as requiring: a persistent frustration or dilemma that creates a feeling of the limits of our current way of knowing in a part of our lives that we care about; and that we cannot be overwhelmed by, nor escape or diffuse, this conflict. Kegan and Lahey’s (2001, 2009) immunity to change framework and methods is intended to help adults understand how they resist and prevent making the personal change that they desire. The method helps them to move this immunity to change from an unconscious to a conscious state so that it can be transformed. Kegan and Lahey (2001, 2009) explore both individual and collective immunities to change, which is an interesting approach to bringing personal and organizational culture change in relationship with one another.   Adaptive leadership and learning have strong links to transformative learning and tend to bring the theory of transformative learning more actively into frameworks and techniques that can be more accessibly put into practice outside of formal educational settings. Heifetz (1994) and Heifetz et al. (2009) offer some insightful fuel for this work. They describe the differences between technical (i.e. complicated) and adaptive (i.e. complex) challenges, the    72  importance of properly diagnosing the nature of the challenge, and how adaptive leadership capacities are required to work on adaptive challenges in particular. Heifetz (1994) is particularly interested in leadership in this context and describes leadership as an activity that can be learned, practiced, and grown as opposed to a set of innate qualities that someone is born with. He also explores leadership approaches in cases of both formal and informal authority and how leadership emerges from all different levels in an organization, and offers insights into leadership with informal authority, uncommon in the literature. Heifetz (1994) describes a set of leadership principles that can be used both with and without formal authority, to different extents:  1. Create and maintain a holding environment 2. Command and direct attention 3. Give access to information 4. Control the flow of information 5. Frame issues 6. Orchestrate conflict and contain disorder 7. Choose the decision-making process  Heifetz (1994) also offers some nuance to what leaders without formal authority can bring to work on adaptive challenges that is particularly beneficial. These benefits include: latitude for creative deviance and to raise questions that disturb; taking an issue focus rather than having the expectation of dealing with multiple issues and stakeholder groups; and potentially having front line information resulting from being closer to direct stakeholder experiences.  Heifetz (1994) shares his zone of disequilibrium framework (Figure 14), which relates to Mezirow’s steps in a transformative learning process described earlier. He layers in a process of adaptive leadership in situations of high disequilibrium including: identify the adaptive challenge; keep the level of distress within a tolerable range; focus attention on ripening of issues and not on stress-reducing distractions; give the work back to the people at a rate they can withstand; and protect voices of leadership without authority. This connects to Cabaj’s (2017) pressure cooker analogy of working in a lab context – the heat and pressure need to be calibrated just right in order to create a strong enough container for the work, while not turning up the heat and pressure so high that the lid blows off.     73    Figure 14: The productive zone of disequilibrium (Adapted from Heifetz, 1994)  Finally, Heifetz (1994) offers advice to leaders in how they can manage the personal pressures associated with leading change processes for complex adaptive challenges. This collection of practices includes: regularly observe and reflect; distinguish the differences between self and role; externalize conflict and focus attention on issues; bring in partners to engage the vision of others; listen, using oneself as data to understand interpretations and distortions; find a sanctuary; and preserve a sense of purpose. Heifetz et al. (2009) make the connections between personal leadership on complex adaptive change in systems, and the broader organizational, societal, environmental, and other contexts within which an individual is working.   There is a great deal of potential for further research, experimentation, integration, learning, and evaluation of how personal transformation practices might be integrated into PSI lab practice to achieve more significant transformation at all levels.  2.3.2 Organizational Culture Transformation This section covers promising practices from fields related to organizational culture change, learning organizations, collaborative leadership, collective impact, and communities of    74  practice. Some of these are currently in use in informing PSI lab practice, although often collaborative leadership and collective impact initiatives are viewed as one type of activity with labs as a separate activity, making for a promising opportunity for a deeper integration of these approaches.  2.3.2.1 Organizational Culture Change Theory Schein (2010) says that “…the culture of a group can be defined as a pattern of shared basic assumptions learned by a group as it solved its problems of external adaptation and internal integration, which has worked well enough to be considered valid and, therefore, to be taught to new members as the correct way to perceive, think, and feel in relation to those problems” (p. 18). He also provides a set of events and forces that shape, describe, and define culture including: group behaviours and norms; espoused values; formal philosophy; rules of the game; climate; embedded skills; habits of thinking and mental models; linguistic paradigms; shared meanings; root metaphors or integrating symbols; and formal rituals and celebrations. Much like the more formal and academic writing on bureaucratic and NPM versions of PSI discussed earlier, Schein’s views include some embedded assumptions about culture that come from more corporate, hierarchical, colonial, and paternalistic worldviews that are not named or tested in his writing.  Laloux (2014) is interested in surfacing and testing some of these embedded assumptions about culture in order to see if there are other possible ways of shaping it. He begins with a stated view that the way that we are currently running organizations is broken. This is aligned with how Geels, Westley, Scharmer, Heifetz and some others included in this chapter describe what ‘broken’ means, namely that our organizations are not up to the complex, adaptive challenges that we are facing to ensure ongoing human and ecological flourishing. Laloux (2014) characterizes five organization types based on five different worldviews, including: 1. The mafia boss – a red/impulsive worldview 2. The army – an amber/conformist worldview 1. 3a. Business school – orange/achievement-oriented worldview 2. 3b. Wall Street – orange/mechanistic worldview    75  3. Cooperatives – green/pluralistic worldview 4. Living systems – teal/evolutionary worldview  He views the living systems, evolutionary type as an emerging organizing framework with promise to respond differently and perhaps more appropriately to complex adaptive contexts and challenges. The attributes of evolutionary organizations are: self-management (as opposed to hierarchical or bureaucratic); wholeness of people without partitioning off a ‘professional self’; and evolutionary purpose, where organizations have a life and sense of purpose that they follow. To position Schein and other more classic organizational change theorists in this framework, they would be primarily within the achievement- and mechanistic worldviews, without seeing much beyond those possibilities and instead trying to work within them.   2.3.2.2 Learning Organizations To dip back into the transformative learning literature explored earlier, two chapters in Mezirow’s (2000) book explore what transformative learning might look like at the scale of groups and organizations. The author provides a set of questions to assess the viability of transformative learning practice at the organizational scale and consider things like:  “Do learners develop a critical engagement with their organizational and social world, increasingly recognizing that the existing state of affairs does not exhaust all possibilities and arriving at alternative courses of action?  Do learners develop an increasingly critical account of the cultural conditions on which their own habits of mind are based?  Do learners develop a commitment to a continuing critical re-examination of their points of view and habits of mind?  Does their critical examination make them more aware of how their history has influenced their existing habits of mind?  Are learners confronted with alternative interpretations of their experience in a way that makes visible both their good and bad points as well as the reasons behind their blind spots and misunderstandings? Is capacity enhanced for incorporating their insights into more inclusive and permeable habits of mind?” (p. 276).    76  What is interesting about these questions is that they are still at the individual scale and concerned with how the individual sees themselves in the context of their group or organization. What then do organizational artifacts, practices, habits, and culture tell us about the role of the individual in transformation? The nature of these questions demonstrates some potential signals about how organizational and personal transformation may be connected.   2.3.2.3 Collaboration Kahane (2007, 2010, 2017) has written widely on collaboration, and shares insights and stories from his work on collaborative processes in high conflict and highly complex contexts. He writes about the importance of practices like: talking openly and speaking up; reflective and empathic listening; openness; balancing the forces of love and power; and encouraging people from across significant divides to learn how to work together to co-create new possible futures. Kahane articulates that there are two types of collaboration - conventional and stretch - and that collaboration is only one option for a way of working together, naming the other options as adapting, forcing, and leaving. This is useful framing alongside Australian Research Alliance for Children and Youth (ARACY, 2013) description of collaboration as it is always an optional way of working together, it is not required, and that true collaboration comes along with real commitments and accountabilities to working differently and intentionally together.     77   Figure 15: Comparing cooperation, coordination, and collaboration (ARACY, 2013)  Archer and Cameron (2013) define collaborative leadership as: working across organizational boundaries; having mechanisms to productively handle the conflict that will necessarily arise and to generate value from these tensions; and having mechanisms to share power and accountability. ARACY (2013) has a series of fact sheets that explore collaborative leadership, and helpfully compare the differences between communication, cooperation, and collaboration (Figure 15). This figure clarifies how loosely the term ‘collaboration’ tends to be used, and when used according to their definition how significant the shifts can be in terms of how collaborators relate to one another and work together.  Building on Archer and Cameron and ARACY, Smith and Becker (2018) are interested in the skills required to build strong cross-sector collaborative teams. These skills are:  Building teams: developing trust, managing power dynamics and conflict, and fostering an innovation culture;  Solving problems: understanding impacts on people, taking a systems approach, defining results, and using data; and  Achieving impact: aligning motivations and values, using leverage points, sharing knowledge and learning.     78  2.3.2.4 Collective Impact Collective impact is a particular form of cross-sector collaboration that has been in place informally for many years in a great deal of community-based work. It was documented and codified, and arguably reduced, into specific patterns and practices by Kania and Kramer (2011, 2013) with many organizations then picking up and running with this model. This uptake itself is interesting, as the simplification, clarity of purpose and practice, and language caught fire in many social change sectors and partnerships as a result. Since then, several studies and papers have reviewed and critiqued the tightly codified version to understand the impacts it is having (ORS Impact, 2018) and also recommending improvements and iteration (Cabaj and Weaver, 2016). The Cabaj and Weaver iteration is promising for PSI labs where multiple stakeholders and partners are involved in the process and implementation (Table 2).   Table 2: Moving to a version of collective impact 3.0 (Cabaj & Weaver, 2016, p.3) From To The Leadership Paradigm Management Movement Building The Five Conditions Common Agenda Community Aspiration Shared Measurement Strategic Learning Mutually Reinforcing Activities High Leverage Activities Continuous Communication Inclusive Community Engagement Backbone Containers for Change          79  2.3.2.5 Communities of Practice Wenger et al. (2002) provide another useful framework and method to consider in approaches to organizational change - communities of practice (CoP, Figure 16). A CoP is an intentional shared learning structure that includes:  A domain in which the CoP operates and works;  A clearly defined community that participates and co-creates; and  A set of practices which the community may be sharing, developing, and/or implementing with high fidelity.  CoPs can share the roles and responsibilities of creating a shared learning environment so that the structure is democratized and brings in the skills and talents of group members (Wenger-Trayner & Wenger-Trayner, 2013).    Figure 16: Elements of a community of practice (Adapted from Wenger et al., 2002)  2.3.3 Systems Transformation  To define systems transformation requires some description as well as direction (i.e. systems transformation toward what?). Westley et al. (2011) describe systems transformation toward    80  sustainability, and that a desired path of innovation for systems transformation as moving away from a “destructive pathway and onto one that leads to long-term social and ecological resilience” (p. 763). The authors note that large scale system transformations are at work all around us all the time, including information technology, new energy systems, nanotechnology, and many others that have significant potential to improve our lives, or not. There are also system transformations that are working against the wellbeing of people and planet. If not intentional about it, systems transformations could continue to follow the unsustainable and inequitable development pathways of many ‘innovations’ to date.  2.3.3.1 Adaptive Cycle Gunderson and Holling’s (2002) adaptive cycle (Figure 17) is a theoretical framework that attempts to sharpen and refine thinking about human and ecological systems transformation toward sustainability. They take inspiration from natural cycles and ecology but note that social systems bring additional and unique human attributes that will affect how this model is applied. These human attributes include: foresight and intentionality; communication; creating hierarchies of abstraction; symbolic construction of meaning; reflexivity; and extending human effects and impacts beyond our size through technology.   Figure 17:The adaptive cycle (Gunderson and & Holling, 2002)     81  It is interesting to see a theory that takes its primary inspiration and direction from ecology, and how this differs from many of the other theories included in this chapter that are shaped by fields like psychology and sociology. The adaptive cycle is cyclical rather than linear and shows iteration rather than more laddered or step-wise change. The adaptive cycle does not attribute more value to certain parts of the cycle (i.e. that one part is better than the others), whereas theories like transformative learning and adaptive leadership have a striving to a desired level, state, or end point that is ‘better’ built into the theory. These qualities of the adaptive cycle offer interesting insights into ways of thinking about systems change in complex adaptive systems that PSI labs work within, and how PSI labs might approach this work more cyclically rather than in linear ways.  2.3.3.2 Systems Thinking Meadows (2008) was a deep systems thinker who contributed a great deal to this field, and who was trained in environmental science so also brings an ecological lens to her work. Meadows defines a system as “an interconnected set of elements that is coherently organized in a way that achieves something” (p. 11) and includes stocks, flows, and feedback loops that can create patterns over time. She describes systems as non-linear, resilient, self-organizing, and often generating surprising events and results. They are difficult to draw boundaries around, we can never have perfect and complete information about them, limiting factors and delays shift and change over time with greater or lesser significance, and they are generally very difficult to map and identify points of intervention to shift how they function.  Meadows describes a series of systems traps, or archetypes of system dynamics that result in problematic outcomes. Some examples of particular relevance to PSI labs include: policy resistance, or fixes that continually fail; the drift to low performance; addressing symptoms rather than underlying causes; and seeking the wrong goal. Meadows describes 12 places to intervene in a system, from lower to higher leverage, with transcending paradigms being the highest leverage point of intervention in her model (Figure 18).      82   Figure 18: Points of leverage in a system (adapted from Meadows, 2008).  2.3.3.3 Systems Transformation Over Time and Scale Another idea to explore in systems transformation is that of scale. Geels (2011) offers a model of innovation and transition that includes niche, regime, and landscape level change and how they interact with one another (Figure 19). Westley et al. (2011) provide some additional nuance to Geels’ framing to look at niches, micro-innovations, and the individual social entrepreneur at the micro scale. They then consider the meso- or regime scale and consider institutional entrepreneurs and networks of actors working in a particular problem domain or regime scale. Finally, they describe the macro- or landscape scale, concerned with political, economic, cultural, and legal institutions and seeking innovation at the social or systemic level. This starts to more fully integrate transformation at different scales, including positioning the individual in this framework.       83    Figure 19: Multi-level sustainability transitions (adapted from Geels, 2002).  Considering how to scale social innovations beyond their early experiments or prototypes in order for them to realize their transformative potential is also of interest (Westley et al., 2011). They write that many innovations work incrementally at the niche and regime levels to provide a continuous supply of novelty. While they may generate innovative solutions at these levels, they are unlikely to catalyze transformation, and in many cases may confirm the resilience of the problem at the landscape level.  They write that disruptive innovation focused on transformation at the landscape level, where the largest scale problem domain and constraints are defined, is required for systems transformation. For true transformation to happen, disruptive innovation and broad shifts in the system are required, not only novelty generated at niche and regime scales. This connects to the OECD OPSI facets of innovation shared earlier (Figure 11), the Two Loops Model, and the Three Horizons Model (Figures 20 & 21). The three horizons model of innovation (Figure 20) describes horizon 1 (H1) is a dominant, common, and declining pattern shaping the issue at hand, and horizon 3 (H3) as an emerging possible future pattern. Horizon 2 (H2) is the pattern of disruptive, transitional activities influencing the issue, making the space for innovation, which can either be    84  appropriated by H1 to maintain the dominant pattern, or enrolled in H3 to help it to emerge, amplify, and grow (Sharpe et al., 2016).   Figure 20: Two Loops Model (Adapted from Wheatley & Frieze, no date)  Figure 21: Three Horizons Model of innovation (Sharpe et al., 2016).     85  2.3.3.4 Indigenous Methodologies Indigenous ways of knowing and being are inherently systemic, although they tend to be described in different language like interconnections, all my relations, and ways of knowing, being, feeling, and thinking that are interwoven (Elliott, 2019; Kimmerer, 2013; Kovach, 2009; Simpson, 2013; Simpson 2017a; Simpson 2017b; Smith, 2012; Tagaq, 2018). The very nature of research into personal, organizational culture, and systems transformation - together - call strongly to be informed by Indigenous methodologies. Indigenous worldviews are deeply formed by being in right relationship with people and place, of drawing wisdom from human and non-human ancestors and from the land and water, and through being good ancestors. PSI labs in both research and practice are not drawing from this wisdom with the exception of a few isolated examples. Multi-stakeholder social innovation labs in Canada, Australia, and New Zealand are leading the way in bringing resurgent Indigenous methodologies to the centre of social innovation processes. There are fruitful explorations, insights, and revelations possible by examining the colonial constructs of government and of innovation with Indigenous ways of knowing and being, and with decolonization in mind.   Inspired by these ideas, a working definition of systems transformation might include the following elements. Systems transformation is complex, non-linear, and ambiguous. It is inherently disruptive as it shifts deep patterns, structures, connections, mindsets, and relationships away from those that are destructive and harmful for people and planet and perpetuate unequal distribution of costs and benefits and ongoing colonization. Systems transformation is the shift of these patterns, structures, connections, and relationships toward those that are sustainable, equitable, and generative, and catalyze system-wide flourishing for people and planet. Systems transformation is informed by the embodied knowledge of Indigenous peoples and methodologies, who have been living in relationship, reciprocity, and respect with all our relations since time immemorial (Simpson, 2017a).  2.3.4 Connecting Transformation Literature and PSI Lab Practice This collection of literature from theory and practice indicates that transformation can be approached from many different theoretical angles, and at the personal, organizational    86  culture, and systems levels either discretely or together. There is an interesting oscillation that happens in each of these transformation domains, that of building theory and then finding ways to ground, apply, experiment with, test, and evaluate it through frameworks and methods in a PSI lab context. This leads to a challenge for practitioners who do not typically have time to dig deeply into different potential theoretical frameworks that they could use to inform PSI lab practice, and more strongly theorize their work. This is also reflected in Schulman’s (2018) finding that the norms and constructs that their innovation team worked within were not tested or reimagined, and they continued to repeat old patterns even when they knew that these patterns were no longer working which often kept the challenge that they were working on stuck. Reading into the organizational change literature leads to questions about how Geels, Westley, and other theorists informed by the niche-regime-landscape transformation framework imagined how niche innovations could contribute to transformation at the regime or landscape scales in practice, and also where/if personal transformation fits into this model.  Practitioners in the PSI lab field regularly speak of organizational and systems change as an intended and hoped for result of their work, but the theories that inform the frameworks and methods that they use are not as robust as they could be. The pressures to deliver shorter term and tangible outcomes to specific policy, service, or other challenges are high for most PSI labs, and even though intentions for working toward more transformative outcomes might be there, the public sector systems within which they work are not often patient enough to set up the enabling conditions for this type of change. Worse yet, many bureaucracies do not actually want this kind of change as it fundamentally threatens who currently has power and influence in the system, and those in power may not want that to change. It is also potentially seen as an admission that the way government works is no longer working to solve our most complex challenges, and this is a hard thing to admit from inside this system, and harder yet to try to change.  Notably absent from the organizational culture or systems transformation literature was a strong personal transformation analysis, with the exception of Meadows, Omer, and Castro Laszlo who weave transformation across these levels. It appears as though many theories concerned with systems or landscape level change do not attend to personal transformation.    87  This also shows up in PSI lab research and practice, most of which does not explicitly focus on professional or personal development as a part of their practice, approach, and purpose.   Using the niche-regime-landscape model of transformation, PSI labs tend to work at the niche level, characterized by provocations, problem and purpose reframing or re-visioning, seed sowing, creating windows into new possibilities, inclusion of diverse actors, and creating different qualities of experience for participants. Work at the niche scale is seen to be crucial for transitions at other scales as well because niches can provide the seeds for systemic change. Niche actors and actions aim to influence the regime and hopefully also the landscape, but this is not easy because these levels have strong path dependencies and fixed structures and systems that are very difficult to change. Niche interventions may lead to these higher order changes if they create broader acceptance and alignment around how complex challenges are reframed, build larger and more powerful networks of actors, gain legitimacy and resources, and also result in a stable configuration of a new idea or structure that can persist over time to influence regime and landscape levels. Three core processes in niche development are important when aiming to influence the other two levels, which is useful framing for PSI labs holding a transformative intent to consider: 1. Shift in vision, expectations, and intentions and through doing this attract attention and funding from more actors; 2. Build more diverse, numerous, and committed social networks and expand the resource base available for niche-innovations; and 3. Learn, model/practice, and articulate different processes for working on these complex challenges (Geels, 2011; Kemp et al., 1998; and Schot and Geels, 2008).  In closing this literature review, academic and practitioner-oriented literature in a wide range of disciplines and fields has been drawn from, including governance paradigms, definitions and types of PSI, enabling conditions and barriers for PSI, PSI labs, and finally personal, organizational culture, and systems transformation. The next section analyzes this literature in terms of what it can tell us about PSI lab research and practice and sets up the potency and possibility of the research questions explored in this dissertation.     88  2.4 Analysis of the Literature So what might the curiousities piqued through this literature review tell us about the transformative potential of PSI labs, and the four research questions that we are tracking? Gryszkiewicz et al. (2016), Timeus and Gascó (2018), and Tõnurist et al. (2015, 2017) remark that there is scarce academic literature on PSI labs, and that much of the literature that exists focuses on describing the characteristics and functions of these labs, and not yet on how they perform and contribute to a more innovative public sector. The specific intervention of PSI labs as an innovation catalyst is under studied and not well theorized, is predominantly concerned with descriptions and categorization rather than on critical or comparative studies, and is dominated by research and perspectives from Europe, North America, Australia, and New Zealand (Bekkers & Tummers, 2018; Lewis et al., 2020; McGann et al., 2018; Timeus & Gascó, 2018; Tõnurist et al., 2017). There is a gap in comparing PSI lab responses and approaches within different contexts – cultures, political regimes, roles of government in society, urgency of different types of issues, financial regimes, community engagement, etc. This makes it difficult to examine how PSI labs change over time as these landscape level contexts and considerations shift.    McGann et al. (2018) aim to understand whether or not PSI labs bring a distinct modality to public sector innovation, and state that the potential effects of these labs is a question for future study. They find that there is little agreement about what PSI labs actually are, and that very different language, definitions, and characteristics are used to describe the same labs by different researchers and practitioners. There is interesting critique and field-building literature reflecting and responding to PSI lab theory, frameworks, and techniques in grey literature, blog posts, websites, reports, case studies, evaluations, and reflections from practitioners. There is no academic literature yet on the impacts of PSI labs on the specific challenges that they are working on, and very little literature from practitioners that goes beyond evaluating surface-level lab outputs (i.e. number of workshops, number of participants, number of prototypes invented). There is very little academic or practitioner literature that evaluates other aspects of lab impacts on personal development, capacity building, collaboration, culture change, and other more systemic changes.       89  Hassan (2015) blogs about the ten most common mistakes in setting up a lab, which helps to frame some of the thinking in this section in a concise way: 1. Using the word ‘lab’ as a brand 2. Treating labs as if they are a technique 3. Not knowing what the challenge is that you want to address 4. Not having the appropriate level of resources for addressing your challenge 5. Not having a comparative measure for your lab (i.e. what the costs and impacts of business as usual are) 6. Failing to meet the criteria of social diversity 7. The lab is owned by one organization 8. Inadvertently designing a planning process instead of an experimental process 9. Mashing up multiple processes (i.e. trying to use all of the processes at once) 10. Treating the lab like a pilot project.  There are some emerging shifts in commonly held ideas from research and practice about PSI labs that I am seeing in relation to my research questions and when considered through the lens of transformation (Table 3). This connects to the movement through the four generations of PSI labs shared toward the beginning of this literature review.  Table 3: Emerging shifts in PSI labs. Shifting from: To include where we’ve been and also… Design methods in toolkit Stronger theorization and clearer definition of innovation, with an expanded toolkit to include multidisciplinary methods properly positioned as techniques and not as theory Expert team in innovation methods Expert team in building shared and democratized capacities for innovation, and embedding knowledge, skills, and tools Innovation understood as creating public value for community being served Social innovation on complex challenges like sustainability, equity, and reconciliation/decolonization Finding inventive prototypes on specific challenges and then moving on or shutting down Implementing long-term, scalable, systemic, embedded, and high impact innovations within an ongoing innovation infrastructure or platform Evaluating creativity and Evaluating scaling and embedding solutions, leadership,    90  invention, novelty, volume of projects and outputs learning, outcomes, transformation, and emergence Belief that creativity and disruption is what’s needed to change the public sector Belief that culture change, collaborative leadership, and building innovation infrastructure is what’s needed to transition the public sector  The next sections critically examine seven themes emerging from academic and practitioner literature: defining innovation purpose; theorization of innovation; ambitious systemic change; innovation infrastructures and capacities; power; durability; and impact of evaluation. It generates some rich analysis, critique, and reflection on the state of the PSI lab field. It closes with an iteration and refinement of my original research question based on the learning and reflection provoked by a deeper look at the literature.  2.4.1 Defining Innovation Purpose Although there is some variety in how PSI is defined, and this variety has real consequence to how PSOs understand and action innovation in practice, there is also some convergence around using a relatively inclusive definition of innovation. The Organization for Economic Cooperation and Development (OECD) Observatory of Public Sector Innovation’s (OPSI) facets of innovation model captures this view (Figure 7), where several definitions of innovation are encouraged and contained within a portfolio of approaches. There may be some tentative convergence around the idea that PSOs need more of all of types of innovation, and that the focus should be on actioning this in all of its forms rather than debating which innovation purpose is better or correct.   Researchers investigating PSI through studying cases often take a normative approach to defining PSI, which to a certain extent might indicate convergence around the definition of innovation as not being all that important, and/or are highly contextual. There is some convergence on aspects of the definition of a PSI lab, particularly about what a lab is and what it is not, when it is appropriate to use one, and when other approaches to a change process or complex challenge may be a better choice. There continues to be divergence as well, in thinking about when a lab starts and ends, its (non)role in implementation and scaling    91  of innovations, and whether it is aiming for incremental or discontinuous change, as examples.  There should be concerns with this open and inclusive approach to defining innovation purpose, however. Westley et al. (2011) describe the double-edged sword of innovation. They argue that current understandings and practices of innovation can be driven by technological innovation priorities and interests, which is indeed showing up in the practice of many PSI labs focused on technical and digital disruption and change. They note that current pressures and incentives from governance regimes, markets, and cultural values that drive innovation may actually “mediate against an appropriate and creative response to complex challenges” (p. 765), and that technical and economic innovations to “global environmental challenges are often inimical to the health of the biosphere” (p. 764). They also offer a caution about the different values underlying the term innovation, and how we need to take “…a complex system perspective that recognizes the dynamic links between the social, ecological, and technological subsystems to understand what we see as the paradox of innovation: innovation is both a contributing cause for our current unsustainable trajectory and our hope for tipping in new more resilient directions” (p. 763). This is a foundational critique and reminder for PSI lab practitioners to regularly question how innovation is being defined and understood, as well as asking questions about innovation for what purpose and for whom.  There are researchers and practitioners that think a stronger, more ambitious, clear, and shared definition of innovation is important so that it is possible to compare, contrast, and evaluate across different cases and ultimately grow the legitimacy and impact of the field. Without a shared definition it is hard to assess whether or not innovation is happening, or if PSI labs are actually making a difference to the complex challenges that they work on. It is also challenging to discern a different researcher or practitioner’s embedded values, mindsets, or ideas about what kind of change they are working toward in their innovation without using clearer definitions. This means that the term is being used to describe very different types of purposes and outcomes.      92  It is concerning that some of the outcomes being produced under the auspices of PSI, and by PSI labs, may actually be perpetuating negative outcomes because innovation is not being defined and considered at more systemic levels. For example, if a PSI lab does not directly consider and address existing structures of power and privilege that perpetuate inequity, racism, ecological destruction, and other systemic challenges then it is likely that the solutions that they are producing are superficial or limited in scope (i.e. customer service improvements) and may actually have negative effects when considered within a larger context.  2.4.2 Theorization There is a wide range of theoretical source material used by different academics in their attempts to understand and explain the PSI lab field. This is likely because of the particular backgrounds and interests that researchers are bringing to this topic, and also the highly interdisciplinary nature of the field. A lab can become a platform for exercising many different theoretical frames and underlying purposes and values, and there is a great deal of divergence, and often an absence, of theorizing a PSI lab effort. This gap could compel more rigour about choice-making that PSI labs might make, for example: is the lab about discerning the right/most appropriate theoretical frame for the challenge? Is it about taking a larger more systemic view of change needed in the public sector and then bringing in theories through a lab that best respond to that? There is work to be done to be more discerning in pulling in the best and most appropriate theoretical frameworks for the distinct purpose(s) of PSI labs. The three publications that Tõnurist has been involved with, and referenced throughout this chapter, are the only ones so far to take a comprehensive look into different organizational change theories as they relate to what PSI labs are doing. There are additional change theories that could be included here from different fields and lineages to build a more robust approach to theorization.  Most PSI labs do not clearly articulate the theory that informs their purpose and practice, and use the ‘and then magic happens’ link between activities and outcomes. This missing theorization piece tends to mean that they are focused on methods, techniques, continuance, and finding quick and novel solutions rather than on purpose or the specific role of a PSI lab    93  in a change initiative. There are many recent labs that have been established because they are a current trend in government, which often means that they are not initially set up with a strategic change intent but rather with an intent to solve a problem in a novel, creative, politically salient/acceptable, and/or user-centred way. This is not necessarily a bad way to begin, however, labs that do not then evolve to have more robust theorization tend to get caught in more typical government rapid response processes rather than working in more systemic or transformative ways.  Some labs are stepping into this theorization gap, and by doing so they are making much stronger hypotheses to test, as well as bringing a stronger and more robust set of change theories or frameworks into the practice. Examples include the Finance Innovation Lab (UK), Energy Futures Lab (Alberta), and MaRS Solutions Lab (Toronto). This theorization gap also points to differing expectations around outcomes. Labs without robust theorization seem to have expectations to rapidly produce creative solutions that are not necessarily implemented. They move quickly and break things and celebrate their uniqueness and difference from the rest of the public sector. Labs that more strongly theorize their approach tend to be working in more systemic, longer-term, relational, inclusive/embedded, capacity-oriented, and collaborative ways.    Although many labs define themselves by their techniques (i.e. design, strategic foresight), movement-building organizations like Nesta’s States of Change program are putting techniques in their appropriate place in the landscape of theorization choices an initiative should make. There is a growing recognition amongst practitioners that there is a very large potential set of techniques that can be used by PSI labs, many of which require specific expertise and discernment to practice purposefully, appropriately, and well even if they have a preference for a particular approach. That said, together these techniques are all considered useful and included in the larger understanding of PSI lab practice, and most of them are new to use in the public sector. Together they can help to clarify what is different about how labs work, and how this is different from other change management processes in government that tend to be more closely associated with new public management.     94  2.4.3 Ambitious Systemic Change Schulman (2013) and Kieboom (2014) both participated in a Lab2 workshop in 2013 with 40 lab practitioners from 15 countries to discuss and critically review their own work and generate provocations for the field. Although this is a rapidly changing field and this group was convened several years ago, many of their reflections are still very accurate today. This is perhaps because labs are proliferating and start-ups continue to work through many of the same challenges as they did in 2013. Schulman writes that while there is a great deal that is shared about techniques and processes in the lab community, the field is still very weak in understanding and reporting on results that point to examples and evidence of systems change. Kieboom identified four oversights in lab practice that were seen to contribute to this lack of evidence of systemic impacts including: the trap of seeking quick solutions; the political blind spot; the dictatorship of scale; and the human ‘post-it celebration’ instead of a focus on the complexity of human behaviours.   Kieboom (2018) shared what she noticed as patterns that need to shift to support greater impact following the first ever gathering of Canadian lab practitioners, called Converge, in June 2018. These included:   Labs, and lab language needs to de-expertize, de-eliticize, and decolonize;  Labs increasingly aspire to create discontinuous change; and  Labs acknowledge their roots in place, time, and value their connection to other social movements.  The OECD OPSI facets of innovation framework (Figure 11) positions more ambitious change against other types. This is a helpful framework to show that not all innovation efforts have the same purpose. In reading between the lines in much of the research and practitioner literature, much of what is called PSI is improvement and efficiency-oriented, and more focus must be placed on the transformative, systemic, disruptive, and mission-oriented types of efforts. There is a great potential for PSI lab research and practice to be informed by fields like systems thinking, complexity science, sustainability transitions, critical race theory, decolonization, transformative learning, and others that have a strong systems or    95  transformation orientation that has not yet been thoroughly explored for its potential to inform transformative PSI lab approaches.  2.4.4 Innovation Infrastructures and Capacities There are some interesting ideas emerging in literature and practice about building innovation infrastructures and capacities, which may exist at individual, organizational, and system-wide scales - discretely and in integrated ways. This literature and practice are concerned with creating the enabling conditions for ongoing research, invention, innovation, and implementation rather than more one-off and linear innovation processes that have clearer beginnings and endings. It is concerned with processes of understanding and working with risk and uncertainty, leadership, culture, learning, and other ongoing activities and roles in the public sector. This infrastructure and capacity orientation to PSI and labs takes an embedded long-view rather than the construct of lab as a disruptive add-on positioned as distinct from the organization that it is part of, and that this difference needs to be protected so as not to be consumed by the bureaucracy.   Some view innovation as specific expertise that exists within a lab team and is distinct and unique from typical public sector expertise. An alternative view is that of innovation as capacities and competencies to be built organization-wide, and that sources of expertise are further democratized into community and society. PSI labs that take this posture then engage or co-create with staff and residents in a different way, appropriate to the nature of the challenge and the solutions being sought, while building capacity, collaboration, and social infrastructure along the way.  Lewis et al. (2017) help to frame this idea further, noting that there remains an unresolved question about innovation as an output, rather than as a capacity. They suggest that investigating innovation activities, ranking their effectiveness, and then relating these to innovation infrastructure and capacities could be a fruitful avenue of inquiry that would further advance understanding of PSI labs. Wascher et al. (2018) have identified a related research need to create a case study framework for PSI labs, and to investigate infrastructural elements of: lab as organization; lab as process; and lab as networks of people involved.     96   Schulman (2018) reflects on work that her firm InWithForward has done to build the research and development (R and D) capacities of people and organizations in the social services sector in Vancouver. After four years of work, with 22 teams and 26 prototypes, only two or three of their prototypes had promise for achieving significant social impact. She questions whether they are building capacity for the right things – are we building capacity to adopt R and D methods or are we building capacity to shift our social safety nets into trampolines? In other words, are they focused too much on process and not enough on the impacts and outcomes of their work? She adds that the more that they push ideas that use logics counterintuitive to prevailing systems and social theory, it become less and less likely that organizations steeped within the existing system are able to develop and implement new ideas that challenge existing logics.   Shulman (2018) reflects on how a few of the choices and areas of emphasis of InWithForward’s work to build innovation infrastructure and capacity as being too limiting and offers some alternate ideas:  A focus on shifting organizations and the services that they provide instead of seeing relationships as the lever for change;  A focus on innovations and the false binary of problem/solution and scaling the wrong element;  A focus on people spending time building capacity, but not the same willingness to spend time on implementing what emerges;  A need to move beyond organizational boundaries, to involve a much wider range of stakeholders, mobilize new constituencies, and create some operational distance from the system or practice that you are trying to change; recognizing that the architects of current systems are unlikely to deconstruct those same systems; and  A focus on increasing organizational capacity.  There is an interesting and provocative idea about creating and growing diverse social innovation ecosystems emerging from practitioner and field-building work in parts of the world where the social innovation and PSI lab efforts are more mature. Kieboom (2014) states that “the real value of labs lies in reports on the mix that is found in and between labs. It is    97  important to foster this diversity, rather than seeking unity or sharpening boundaries of what is or isn’t a lab. If we understand systemic innovation and we aim to do this in interaction with people (hence social innovation), then we argue that stimulating diversity, radicality and disruption could raise the potential to create the discontinuous change that we are seeking” (pg. 16).   Social innovation labs might seek to become embedded in, or support the co-creation of, a local ecosystem of social innovation because working with like-minded allies is more fruitful than struggling for change as an isolated lab. There appears to be resilience in this kind of ecosystem approach, which may lead to more significant impact over time. Different people and sectors can take up challenges and approaches in different ways and play different roles in supporting the ecosystem, and there is diversity and resilience in the theories, frameworks, and methods used.   Attention to supporting innovation infrastructures and capacities is quite different than how many PSI lab researchers and practitioners are thinking about growing and supporting PSI lab entities and raises some interesting questions. What are the potential roles of labs in the public sector in supporting and/or creating this type of localized, and potentially even larger scale innovation ecosystem with a movement-building mindset and objectives? What might happen if individual labs could get up and out of their specific contexts and challenges, at least sometimes, to contribute to building the broader field? Is there more resilience, longevity, and transformation potential if an ecosystem approach is taken? There are some strong possible research questions here that will become more possible to explore as the labs landscape matures.  2.4.5 Power PSI labs understand and engage with power in very different ways, and this can have a significant effect on their approach and impact. Most PSI labs acknowledge and exist within traditional hierarchical and political power structures of government. There is a critique that design-oriented labs do not typically have team members with a skill set that includes understanding and engaging with political power appropriately and effectively to unlock    98  enabling conditions for innovation. Other researchers and practitioners are beginning to focus on understanding the actual sources of innovation in public sector organizations and are studying different staff/power levels to see where innovation arises. They are finding that it is only sometimes top-down, and that innovation leadership can arise from different places and levels within an organization, and also quite often from those outside. Some practitioners are more directly challenging systemic and structural inequities and their relationship to who does and does not have power, and trying to create lab processes and structures that embed reconciliation, inclusion, and equity into the ways that they work. This practice is uniquely strong in Canada, Australia, and New Zealand and built into the definitions and understandings of what innovation means in many public and social innovation labs in these countries.  Bencio (2018) attended the Converge conference, and experienced the racism, colonialism, and ageism still embedded in the social innovation labs community in Canada. In a letter that he wrote to participants following the conference, he offered recommendations that offer useful insights into the current state of the field. These are perspectives that do not yet show up in formal academic and practitioner literature on public sector innovation and labs.   Lead Truth and Reconciliation;  Build a critical understanding of anti-oppression;  Build a critical understanding of white supremacy;  Change the ways we include youth; and  Support Indigenous youth and youth of colour.  Bencio’s insights reflect informal conversation that is happening in the social innovation practitioner communities in Canada, but does not yet appear to be emerging more broadly. Some Indigenous leaders are calling for social innovation and PSI labs to ensure that they are not perpetuating current structures, processes, and practices of colonization and racism, and a few lab practitioners are actively working on what this means in terms of who is shaping the labs conversations, communities, practices, and leadership.     99  2.4.6 Durability A common critique of PSI labs is that they are a fad, where governments (and other organizations) see them in other places and start one without really understanding their art and craft, what Tõnurist et al. (2017) called emulation as a driver of establishing PSI labs. This view may be reinforced by the relatively short life spans of many PSI labs, illustrated by the hype cycle (Figure 22), with many not making it through the ‘trough of disillusionment.’   Figure 22: Gartner’s hype cycle (accessed Dec. 2018)  Several labs have documented their stories of development, and share how they evaluate, learn from, adapt, weather external pressures and expectations, and adapt (e.g. MindLab, Finance Innovation Lab, Edmonton Shift Lab). There is much to be learned from researchers and practitioners telling stories of the iterations of their PSI labs, and how they remain relevant and resilient over time and with changing conditions.  Gascó (2017) identifies barriers that illustrate how this hype cycle plays out for PSI labs including: the sustainability of a lab after initial investments and support runs out; lack of clarity, understanding and expectations about the return on investment; scalability of results; association with particular political leadership; insufficient human resources to support lab activities; lack of visibility; and the challenge with public sector accepting a shift in power from top-down to another form (i.e. citizen-led, collaborative). Christensen and Bason (2012)    100  discuss the learning from the MindLab case which emphasizes Tõnurist et al.’s (2017) point that much of the literature is focused on a more scrappy need to justify the ongoing existence of, and investment into, these labs rather than on how they are creating significant impacts on complex public sector challenges.    There is some interesting divergence about where the work of a PSI lab begins and ends which relates to durability, with particular contention about the end point. Many labs believe that they are a focused, intense, and time limited activity working on a particular issue or topic. In this scenario, labs develop possible solutions, experiment, iterate, and then hand promising potential solutions off to others to implement. Some of these practitioners also believe that PSI labs should be working themselves out of work, and that they should be temporary. As innovation practices, processes, and methods are used in the public sector they will become normalized, and a lab will no longer be necessary. An alternative view is that the end is when an idea has successfully been implemented and scaled. If a lab does not attend to implementation, the work of the lab does not matter as it is not necessarily creating real world impact. Because of the complexity and diversity of issues that PSI labs work on, and because of the speed at which government works, ongoing lab platforms that have different challenges cycling through is seen as a better approach than one-off processes. Labs are likely an ongoing need and will need to adapt and respond to changing conditions to keep challenging public sector organizations innovate.   More research into how PSI labs adapt and change over time, how this affects their choices of theory, frameworks, and techniques, and why they close down would be an interesting course for further study, testing the hype cycle model as applied to a PSI lab development path.  2.4.7 Impact and Evaluation Innovation is ultimately a complex, emergent, and multidimensional activity that cannot be measured directly with clear cause-effect relationships, or with an indicator-driven performance management approach. The field and practice of measuring social innovation outcomes and impacts is underdeveloped, and this is limiting investments into creative    101  problem solving for the world’s most complex challenges (Antadze & Westley, 2012). Pawson (2013, p. 19) says that “every attempt to conduct an evaluation is beset with the impossibility of covering every angle; every attempt to conduct a review is faced with the impracticability of chasing down every single issue. So, too, have the evaluation paradigms floundered in the face of complexity… And so, whether evaluation seeks to judge, describe, inspire, or explain, there is an ever-present predicament in claiming to have achieved closure in covering all eventualities… There is no disguising the fact that the ideas driving an intervention are often multitudinous and compelling, no concealing the reality that the same intervention can trigger change in myriad ways, and no way of camouflaging the truth that the different contexts in which programmes are implemented are as wide as society is wide.”    Standard performance management and linear approaches to evaluation do not work well for complex public sector contexts and innovations.  The results that PSI labs are seeking require deeper understandings of strategy, policy, systems change, learning, and other interventions that may lead to innovation. Many PSOs continue to use evaluation methods appropriate for simple or complicated challenges to evaluate work on complex challenges, and this approach can be detrimental (Antadze & Westley, 2012). Although PSI labs are trying to work differently on complex challenges and have a different purpose and measures of success than other government units, they still try to prove value using a similar and narrow set of measures. This influences how PSI labs think about and approach their work when they know that the measure of their worth and the likelihood of their ongoing existence are tied to their ability to prove value in this way.  So, what might this mean for shaping a more effective approach to evaluating innovation in complex contexts in the public sector?  Careful discernment of the differences between evaluating simple, complicated, and complex problem types, and the very different possible evaluation approaches, questions, methods, skills, and expected findings, is important in this practice (Rogers, 2011). There needs to be an understanding that social impacts may be the result of actions by multiple actors over long periods of time, where direct causation cannot be determined, and where interventions and actions are diverse and complex. Social innovations do not have a set of known steps or a map to change-making, and little is known about what will work, where, with whom, under what conditions, and how, thus suggesting the need for a different    102  evaluation purpose and strategy. The demand for predetermined specificity of outcomes at the beginning of an intervention does not work under conditions of high innovation, exploration, uncertainty, turbulence, and emergence. Treating these complex evaluations like simple, closed-system projects is inappropriate, ineffective, insufficient, and can do harm (Patton, 2017). This harm can take the form of stifling innovation, taking fewer risks, and forcing premature convergence and decision-making.  Mayne and Rist (2006) describe the multiple potential roles of evaluators as: judge; auditor; researcher; consultant; facilitator; team member; collaborator; empowerment facilitator; supporter of a cause; synthesizer; performance measurement; promoters of organizational learning; strategist for coping with the information revolution; trainer in strategic planning and goal setting; information manager and meaning-maker; and facilitator of knowledge mobilization. This is a much wider ranging skill- and method set than is typically seen in public sector evaluations. Many practitioners emphasize the importance of setting aside adequate time and resources to engage in evaluation activities, and also the necessity that these be flexible in order to respond to what is emerging as an evaluation process progresses (Mayne & Rist, 2006).  Better Evaluation (2014) provides a practical framework for undertaking an evaluation, along with a large set of methods and tools that draw from different evaluation approaches and practices. Their framework is in seven stages, each with sub-sections, and offers a useful structure to help break down the different components of what an evaluation might include. When swimming in complexity, non-linearity, and emergence it can be helpful to have an anchor to keep ones’ footing. The seven stages include: 1. Manage an evaluation; 2. Define what is to be evaluated; 3. Frame the boundaries for an evaluation; 4. Describe activities, outcomes, impacts, and context; 5. Understand causes of outcomes and impacts; 6. Synthesize data from one or more evaluations; and 7. Report and support use of findings.     103  The work of Nesta and the OECD OPSI to create shared evaluation frameworks is helpful, but the practice, process, and craft of evaluation are still largely missing from this work. These early frameworks tell practitioners what might be important to measure, but not how they might go about doing that, how they make sense of what they are learning, what they then do with that information, and how they use this information to tell stories of impact. One promising area for future action research would be to add evaluation capacities and competencies on PSI lab teams through skills building, tool creation, communities of practice, and other movement building activities. Academic research could then see if the inclusion of these processes, roles, and skills affected the types, impact, and longevity of innovation resulting from PSI lab activity.  Tõnurist et al. (2017) suggest that future work should analyze the different typologies of PSI labs and the contextual factors that play a role in their diverging forms. Gryzkiewicz et al. (2016) note that there is a need for further study into the variety, level, and types of impact that results from different forms, approaches, and types of PSI labs. These authors suggest that using organizational ethnography, structured observation techniques, and practice-based approaches will be useful in allowing future research to bring the practices of actual labs and their own understandings of how and why they work the way that they do into the academic literature. The quality of work that PSI labs produce should be studied in detail, because quick solution-oriented lab approaches may deliver incomplete or unsuitable solutions in the wider public sector context. Sørensen and Torfing (2011) add that qualitative case studies are required to fully understand complex processes and causalities involved in the production of collaborative innovation. Individual case studies will facilitate in-depth analysis of how and under what conditions collaborative interaction enhances public innovation and comparative case studies will facilitate the formulation and testing of more specific hypotheses and contribute to theory building.  Another PSI lab movement-building academic and practitioner research need is to develop shared principles, and potentially some minimum specifications, across PSI labs. This could be done at a country level, for a specific type of public sector agency or level of government, for particular innovation typologies, or for specific complex challenges. The current interest in innovation declarations might be a signal that there is an interest for an approach like this. A    104  set of shared principles and some minimum specifications might help PSI labs to connect to stronger theorization, and to get clearer and more rigorous about their purpose and values and how they intend to act on them. These principles and minimum specifications could then become part of evaluating an overall movement-building approach, and impacts over the longer-term.  Forss et al. (2011) offer some useful food for thought when considering a standard evaluation framework or set of principles. They say that the nature of complex policy evaluation is such that each assignment must be considered in its own right, and that it is not possible to develop generic frameworks. Inspiration can be drawn from other cases; however, each new evaluation and team must develop their own responses as no one ever solved a complex problem in exactly the same way.  Perhaps changing evaluation practices, questions, and approaches is actually another way to have deeper conversations about what governments might need to be, and need to become, at this time. Using the axiom of ‘what is measured, matters’ may be a useful leverage point for change, in a lab context or another innovation effort. A valuable innovation intervention may be to try to shift what it is that we measure, and how it is being measured, in regular public sector activity outside of a lab environment. Evaluation is a useful skill and tool set for public sector innovators to have. It provides a framework for asking better questions, having different conversations, creating a more expansive understanding of impact, and subtly and dramatically shifting thinking and focus.   2.5 In Summary Exploring the literature and practice of PSI and PSI labs and analyzing the state of the field through the lenses of my research questions offers many interesting views into assembling this cabinet of curiousities, and some promising directions to take. In her study looking at patterns across multiple social innovations, Moore (2017) found that innovations that were created ‘for’ the regime that they are a part of, when the regime itself was the body creating the social innovation, tended to perpetuate the dominant paradigm of the regime rather than    105  challenging, or truly innovating, within it. Alternatively, where the innovation is created alongside the existing regime as a proto-regime, where it is protected from predominant systems and structures, but is not so far away as to become disconnected, then the nature of the social innovations that result were much more likely to be systems changing.   This idea connects many of the themes explored in this section and sparks interesting PSI lab research possibilities and questions. What is the purpose and impact of PSI labs that are fully ‘for’ the regime that they are in – funded internally, decisions made through traditional hierarchies, internally oriented in their problem selection, and only accountable within their public sector agency? How do these compare to PSI labs that have multiple funding sources, decision-makers and decision-making process that happen in a different way than through the traditional hierarchy, user engagement in decision-making, and other mechanisms that make the PSI lab more like a proto-regime? There is much to be learned about the different approaches and impacts of these PSI labs through comparative case studies that could potentially knit together many of the threads in this analysis.   Moore found that there are several possible pathways to take from the proto-regime state. The proto-regime may eventually replace the existing regime, be taken up by and change the existing regime, or dissolve, and all of these can happen over different time scales. Lichtenstein (2014) describes four orders of emergence, from no emergence to strong emergence, that is interesting to consider alongside Moore’s thinking about what durable, systemic, and strongly emergent innovations might be in the public sector and how PSI labs might catalyze these processes. Connecting to the idea of innovation infrastructure, Moore suggests that it would be interesting to study those proto-regimes that aim to create change with broad social benefits rather than those focused on a single problem, as from her research so far these broader initiatives appear to have more durability. There is an interesting analogy here with different lab approaches and the ideas about innovation infrastructure shared earlier – some are platforms that work on a range of different activities and issues, whereas others focus on a specific topic. This also provides an interesting view into longer time cycles of change within government, and what role PSI labs are playing in what is perhaps a social innovation that will take a longer time to manifest.     106  Kieboom (2014) offers an interesting set of provocative questions that point to possible ways to think about PSI labs. What if labs…: 1. Design and scale better processes instead of solutions? 2. Spread ethics and ideas instead of solutions? 3. Connected with like-minded movements? 4. Became more politically aware? 5. Were better networks, especially geographically? 6. Were more financially independent? 7. Were more responsive to human behaviour? 8. Invested more in building innovation capacity within local communities? 9. Developed their own evaluation methodologies to support their practices? 10. Prototyped new organizational models?  It is important to note that much of the academic and practitioner literature about governance, innovation, and transformation explored here comes from Euro-centric and/or white colonial perspectives. This is what is currently informing the PSI lab research and practice, to a significant extent. The centres of research and activity in PSI labs are limited in their source worldviews and disciplines, and there is much to be learned from who is not yet a part of this published research or more widely known and acknowledged in their practice. Some additional perspectives and approaches are starting to appear in practitioner literature from places like Canada, where decolonization, inclusion, and equity are being considered in social innovation lab practice, but this is not yet broadly considered in practice or research in the field.  Overall, through this review and analysis of literature, I tested the relevance and potency of the research questions that I thought may be fruitful and useful to pursue according to the needs of the field at this time. My positioning as both researcher and practitioner gives me a privileged and unique perspective, and the research questions of most interest to practitioners are quite different than what many academics have identified in their publications to date. For the most part, the academic literature is focused on describing the field as it is and beginning to describe contemporary applications of innovation in the public sector. It is also mostly concerned with describing methods, and not as much interested in    107  describing theories that guide purpose and practice. As a practitioner I am most interested in the field-building work that answers research questions from the practitioner community as I find it to be interesting, relevant, grounded, and significant. Questions that emerged from practitioners and that influenced the refinement, nuance, and choice-making in exploring my own research questions included: how do we know what is working? How can we build capacities for innovation? How do we build more diverse and inclusive PSI labs? How can we ensure that PSI lab work does not continue to inadvertently enable, encourage, and hold up the systems and structures that are creating many of our complex problems? How can we codify what works so that the proliferation of PSI labs can start from the best- and next practice in the field, rather than from the beginning each time?   This literature review led to a fuller expression and clearer articulation of my initial research interests in the form of four fractals. Figure 2 shows how this literature review relates to the progression of the action research and theory construction cycles of this research. When taken together these questions are concerned with drawing from theory and practice to increase the potential for PSI labs to catalyze significant and positive impacts. These fractals are:  How might we more strongly theorize the purpose for PSI labs, and why might this be important?  How might public sector innovation labs systemically intervene in complex public sector challenges in ways that create stronger enabling conditions for transformative and emergent innovation?  What if PSI labs took a transformative innovation learning approach to their work in order to enable and unleash innovation within and amongst individual, organizational, and ecosystem-scale actors?  How might we measure, evaluate, and tell stories of transformative change resulting from PSI lab interventions?      108  Chapter 3: Research Methodology  My research questions and curiousities do not fit neatly into a disciplinary box, as evidenced by the broad terrain covered in Chapter 2’s literature review. Patton (2002) provides some practical guiding questions to help with understanding options and making methodological decisions in a research endeavour. What are the purposes of the inquiry? Who are the primary audiences for the findings? What questions will guide the inquiry? What data will answer or illuminate the inquiry questions? What resources are available to support the inquiry? What criteria will be used to judge the quality of the findings? Lincoln and Denzin (2013) describe five phases in the research process: 1) the researcher as subject; 2) theoretical paradigms and perspectives; 3) research strategies; 4) methods of collection and analysis; and 5) art, practices, and politics of interpretation and evaluation.  My dual roles as researcher and practitioner demand a methodology that invites both of these perspectives in an authentic, fruitful, and rigorous way. Figuring out how to methodologically connect these research curiousities with my dual roles has been an interesting learning journey for me. I have read and wandered widely in a methodological sense, and circled in on constructivist grounded theory and participatory action research in dialogue with one another (Figure 23), while incorporating a healthy dose of anti-oppressive, social justice, decolonial, and feminist methodologies.     109   Figure 23: Constructivist Grounded Theory and Participatory Action Research methodology  Gergen and Gergen (2008) describe four potentials in the convergence of social construction, research, and action: 1) research as political action; 2) collaboration, beyond individualism; 3) from mapping to world making, joining with people to create new futures; and 4) from theoretical to practical priority, and to generate change in existing conditions. I find this to be a useful way to describe why I chose to bring constructivist grounded theory (CGT) and participatory action research (PAR) together in this study.  My hopes and intentions are that employing this methodological bricolage has enabled new insights to emerge, aided in more robust theorization of public sector innovation labs (PSI labs), and remained oriented toward values-based action and impact. This chapter focuses on my positionality as researcher, and the construction of my research bricolage. It also describes the action research sites, expert interview approach, and research activities and timeline. Together this methodological bricolage generated the findings, analysis, theory, and insights shared in the rest of this dissertation.      110  3.1 Positionality In this section I explore my own positionality and sensitizing concepts as a researcher, and how this informed the approach I took in my research. Blumer (1969) says that human beings act toward things based on the meanings that these things have for us, and that meaning and interpretation arises out of the interactions that we have with others in making sense of these things. The concepts that emerge from critical ideas in social justice research Blumer calls sensitizing concepts. They provide a general frame of reference that suggests directions to explore, questions to raise, and ideas to pursue.   Sensitizing concepts offer ways of seeing, organizing, and understanding experience and they ignite thinking. They can provide places to start inquiry, ways to deepen perception, and form loose frames and tentative approaches to developing ideas, questions, and processes (Charmaz, 2017a; Charmaz, 2017b). Lincoln and Denzin (2013) write about the researcher as multicultural subject with a history, research traditions, concepts of self and other, ethics, politics, and their own sensitizing concept. I am not a neutral researcher, and I do not claim an objective stance. In this section I explore my own positionality and sensitizing concepts through five different lenses that are particularly important for the directions that I followed, and perspectives that I held/hold, during this research journey: individual positionality; role as a ‘pracademic’; sustainability and equity; decolonization; and transformative innovation.  3.1.1 Individual Positionality I am a white, cis-gendered, able-bodied woman and settler from a middle-class background, and hold a great deal of power and privilege in all parts of my life because of this positionality. Through my research methodology exploration, I became increasingly aware of what Charmaz (2017a) describes as the dominance of Anglo-North American worldviews and how they pervade qualitative inquiry, and also how they continue to maintain that quantitative research is the gold standard in research to which everything else is compared. Charmaz encourages self-reflectivity about unearned privileges from race, gender, social class, and health and how this affects personal research approach, as well as how it shapes dominant research approaches in academia.     111   There is a long and strong lineage of researchers from critical race theory, Indigenous methodologies, feminist theory, queer theory and others who name that the research practices deemed ‘valid’ by academic institutions continue to be dominated by white, male, colonizing, straight, and able-bodied worldviews that reproduce a very narrow way of knowing and being (brown, 2017; Kincheloe et al., 2017; Simpson, 2017a; Smith, 2016; Strega and Brown, 2015). This reflection also holds true in my experience working in the public sector, where the same dominant worldview and associated systems and structures are firmly in place.   My own positionality, and the public sector context in which my research took place, made it clear that part of my research methodology would need to attempt to name and make visible the ‘water that we swim in’, and to suggest additional ontologies and epistemologies that might inform an expanded understanding of what innovation might possibly be - and need to be - in the public sector. Because of my positionality, and because I was working within a social structure of Western government that is largely built from similar - albeit masculine - worldviews, I recognized that there was strong potential for this to limit what I was able to see and understand. In order for me to be aware of a broader set of possibilities and imagined futures, and to bring a fuller understanding of what innovation might mean in the context of government, I needed to practice continuous awareness of, and reflection about, my positionality and how this affected my research. This took multiple forms, including reading and reflecting, being in dialogue with white settler friends, family, and colleagues, and listening to the experiences and stories of Black, Indigenous and racialized colleagues, friends, and public figures to unlearn, unsettle, and stay curious throughout my research.  3.1.2 Role as a ‘Pracademic’ Throughout this research I was both a Ph.D. student as well as a civil servant, working within my primary action research site (the City of Vancouver Solutions Lab, or SLab). I had specific and distinct responsibilities, accountabilities, and requirements for these two roles. My job as the founder and manager of the SLab is described here, to be clear and transparent about the nature of this work and how it related to my research.     112   In my role as the SLab manager and sole staff person for this initiative, I was responsible for a variety of functions. Initially, I conceived of the purpose and activities for the SLab, developed a relationship with the manager that I was accountable to, and built an advisory committee of other City of Vancouver staff to help guide the work. Funding during this stage was from a dedicated innovation fund, and was sufficient for 12-18 months of work. After this first stage, reporting shifted to two (less senior) civil servants in two different City of Vancouver departments, the funding source changed to these two departments, and the advisory committee was dissolved. My role continued to be to design, deliver, evaluate, and provide overall strategic leadership to SLab. Throughout this time, all staff and community partners that were involved with SLab were there as partners and colleagues, and not as people in a reporting relationship to me. The exception to this were three graduate students that supported SLab work, who also had dual accountabilities to the City of Vancouver and to their university. These graduate students were not interviewed for this research, and their contributions to this dissertation are summarized in the preface.  None of the people I interviewed in this research reported to me as the manager of the SLab. Some of those interviewed were people that I was directly or indirectly responsible and accountable to through my work (e.g. I reported to them). The same set of interview questions approved by UBC ethics review were used in all of the interviews, with a set of questions for action co-researchers at all three sites, and a different set of questions for ‘experts’. In order to ensure privacy and protect the identities of those interviewed I have not directly or indirectly attributed any of the interview data shared in this dissertation to organizations, roles, or people.  3.1.3 Sustainability and Equity A powerful orientation that I bring to my life, work, and study is that of sustainability and equity, where just and equitable access to healthy and flourishing lives is available to all generations, of all species, for all time. I see the world through this lens, as both a way to make sense of what is happening in the world as well as a way to describe desired possible futures. I hope that my research and work contribute to sustainable and equitable futures, and I orient my thinking, approach, analysis, and action in that direction. I am largely unable    113  to un-see this lens; it is not one that I can effectively take off to really understand and empathize with worldviews that are not oriented in this way.  3.1.4 Decolonization  Of particular interest and importance to me are the methods and approaches that focus on Indigenous ways of knowing and being, and on decolonization (Elliott, 2019; Kimmerer, 2013; Simpson, 2013; Simpson, 2017a; Simpson, 2017b; Smith, 2012; Tagaq, 2018). The very nature of research into sustainability, equity, and innovation calls to be informed by Indigenous methodologies. Indigenous worldviews are about being in right relationship with people and place, drawing wisdom from human and non-human ancestors and from the land and water, and being good ancestors. As a white settler I am learning my way into how to be informed by Indigenous research methods and worldviews without romanticizing or appropriating them. I am also learning my way into my responsibilities as a settler and uninvited guest on the land which I live and work, and what my role and responsibilities to work toward decolonization might be from the place of power and privilege that I occupy. I think that there are fruitful explorations and possible insights available by examining the colonial constructs of government and innovation with decolonization in mind, particularly as an increasing number of public sector organizations in Canada and other colonized lands are committing to reconciliation.  3.1.5 Transformative Innovation My orientation to the theory and practice of innovation in the public sector has developed alongside this CGT and PAR research journey, and in particular the expert interviews and literature review of practitioner work that I conducted early in this journey. I realized that most PSI labs did not strongly theorize their work, and that they rarely made their working definitions of innovation explicit. I did not find the ambitious approach to public sector innovation for sustainability, equity, and decolonization that I was looking and hoping for, and to be honest that I was expecting to see.      114  To build a strong theorization of PSI labs I believe that the definition of ‘innovation’ in use in a particular case needs to be made explicit, and then linked to practice and evaluation of impact through some form of a theory of change. In my early action research and theory building I realized that this was not common practice. I think that innovation must take a position about what it is innovating for, and who benefits from it. My interest and focus is on innovation that holds a transformational intent (as described in section 2.3), and has a directionality toward sustainability, equity, and decolonization. It is highly likely that an undefined and non-directional approach to practicing PSI, or one that fully and unreflectively operates within dominant paradigms of Western governance, will continue to produce the types of outcomes that are typical of government, and to reproduce stuck and problematic systems, structures, behaviours, and paradigms even under the guise of what is called innovation.   3.2 Research Bricolage The research questions explored in this dissertation required a methodology that enabled seeing outside of the dominant paradigms that shape PSI and lab discourse and invited divergence, openness, and non-closure. This was of particular importance for this study because the research and practices of PSI labs tend to implicitly operate within dominant governance paradigms, thus limiting the different ways that the research questions and findings might be interpreted and what might be revealed through this inquiry. I required a methodology that enabled me to stay entangled in the research process without needing to come to a neat conclusion or resolution and also invited my dual roles as researcher and practitioner.    I read widely into potential qualitative research methodologies and orientations that would fit the purpose and ambitions of my research into PSI and labs. I also conducted five interviews with researchers that actively work on aspects of my research inquiry to explore their insights, perspectives, and advice on potential research methodologies that might be appropriate to my research questions. Through this process, and an exploration of my own positionality and sensitizing concepts, I found my way to participatory action research (PAR) and constructivist    115  grounded theory (CGT). When used together and iteratively, theory building (CGT) and theory testing (PAR) can be in dialogue with one another throughout the research process (as shown in Figure 23).   Kincheloe et al. (2017) write about the merits of a critical, qualitative research bricolage methodology. What they mean by this is that in the context of many research methodology options, “a critical research bricolage attempts to create an equitable research field and disallows a proclamation to correctness, validity, truth and the tacit axis of Western power through traditional research… Without proclaiming a canonical and singular method, the critical bricolage allows the researcher to become a participant and the participant to become a researcher” (p.253-4). Piecing together a research methodology as a bricolage generated a way to walk intentionally and reflectively through this research project.  Both CGT and PAR can handle a researcher with an active role in the research questions; a researcher who holds standpoints and privileges, a social justice orientation, and a desire to produce radical, democratizing transformation, as long as these are transparent, made explicit, and are a part of a reflective process (Charmaz, 2017; Lincoln and Denzin, 2017). These methodologies do not require a neutral, objective observer, and invite the researchers’ impact on data and analysis as relevant to the course of inquiry. The role of researcher in these methodologies is as a passionate participant and facilitator of multiple voices and views. This fits well with my orientation as both practitioner and scholar, and the desire that my work supports developments in both of these domains, together.   3.2.1 Participatory Action Research Reason and Bradbury (2008) define action research as “a participatory process concerned with developing practical knowing in the pursuit of worthwhile human purposes. It seeks to bring together action and reflection, theory and practice, in participation with others, in the pursuit of practical solutions to issues of pressing concern to people, and more generally the flourishing of individual persons and their communities” (p.4). Lincoln and Denzin (2017) add that action researchers help to transform inquiry into praxis or action. In PAR, research subjects become co-participants in the processes of inquiry. PAR is a family of approaches,    116  and also an orientation to inquiry, characterized by an eclectic pluralism of sharing, borrowing, improvising, creativity, and mutual and critical reflective learning and responsibility for good practice. For many it is liberationist, and aims to address power imbalances typical of many Anglo-American research approaches. Swantz (2008) suggests three criteria for a valid PAR process and result including: 1) transparency; 2) compatibility of purpose with methods used; and 3) co-researchers able to claim deep knowledge of the research situation, and that they have honestly and fully revealed the findings that they became aware of during the study.  PAR focuses on pressing and real-world challenges faced by participants, and is practical, reflective, pragmatic, and action-oriented. Researchers and participants actively co-create the research process. This includes generating questions and objectives, sharing knowledge, building research skills, interpreting findings, and implementing and measuring results, including tracking the willingness of co-researchers to act on the results of the PAR. Kemmis (2008) writes about the intersection of critical theory and PAR, and says that “…action research must find a way to work not just on the self-realization of persons or the realization of more rational and coherent organizations, but in the interstices between people and organizations, and across the boundaries between lifeworlds and systems” (p.123). Further, invoking Habermas, he adds that “the organization of enlightenment is best understood as a social process, drawing on the critical capacities of groups, not just as an individual process drawing out new understandings in individuals. Together, people offer one another collective critical capacity to arrive at insights into the nature and consequences of their practices, their understandings, and the situations, settings, circumstances and conditions of practice” (p.127). This interpretation of critical PAR points to working at the personal, organizational culture, and systems transformation levels together discussed in Chapter 2.  3.2.2 Constructivist Grounded Theory Grounded theory (GT) is a research methodology that builds theory from what emerges out of data collected using a variety of techniques appropriate to the phenomenon being studied. “Grounded theory methods consist of a systematic approach to qualitative inquiry for the purposes of theory construction…Grounded theory is an interactive method from the    117  beginning of the research through the last draft of the report” (Charmaz, 2017b, p. 1-2). Its purpose is not to be descriptive or comparative, nor does it test hypotheses or existing theory. Charmaz (2014) describes strategies consistent in grounded theory as follows:  “Conduct data collection and analysis simultaneously in an iterative process;  Analyze actions and processes rather than themes and structure;  Use comparative methods;  Draw on data in service of developing new conceptual categories;  Develop inductive abstract analytic categories through systematic data analysis;  Emphasize theory construction rather than description or application of current theories;  Engage in theoretical sampling;  Search for variation in the studied categories or process; and  Pursue developing a category rather than covering a specific empirical topic” (p.15).  In reading the origin story of GT and how it has evolved in theory and practice over time, I connected most strongly with Kathy Charmaz’s iteration called constructivist grounded theory (CGT) (Birks & Mills; Bryant & Charmaz, 2007; Charmaz, 2014; Charmaz, 2017a; Charmaz, 2017b; Glaser & Strauss, 1967; Strauss & Corbin, 1990). In CGT, Charmaz acknowledges the subjectivity and researcher involvement in the construction and interpretation of collected data, which departs from her contemporaries who treated GT as accurate and objective renderings of what was happening in the situations that they studied. Constructivist GT recognizes how historical, social, and situational conditions affect both the phenomenon being studied as well as the research process itself, and also acknowledges that the researcher is playing an active role in shaping the data and analysis (Charmaz, 2017a). Linking back to my positionality and sensitizing concepts related to sustainability, equity, decolonization, and transformation, Charmaz connects CGT to social justice in that together they: “1) take a critical stance toward societal structures and processes; 2) aim for transformation, and 3) demonstrate a strong ethical concern for the individual” (Charmaz, 2017a, p.423).  Charmaz (2014) notes that theory development is the explicit goal of GT, and that purely descriptive accounts are not the purpose. This is the key reason why I departed from using    118  case study as a research methodology and view my cases as sites for theory building and testing rather than as case studies. Yin (2018) says that a case study “benefits from the prior development of theoretical propositions to guide design, data collection, and analysis” (p. 15). This is fundamentally different than the approach that GT takes, where theory emerges from the data. Yin (2018) also says that the specific niche that case studies have a distinct ability to fill includes when “a how or why question is being asked about a contemporary set of events over which a researcher has little or no control” (p. 13). This is also a significant departure from my research approach where I was actively involved with and influencing the research sites I worked with.  Further to Charmaz’s iteration, I layered in Timmermans and Tavory’s (2012) work. They suggest that replacing the typical inductive approach to analysis in grounded theory (i.e. generalizing theory from observable data) with an abductive approach may lead to stronger theory. Abduction is “a creative inferential process aimed at producing new hypotheses and theories based on surprising research evidence” (p. 167), or to be led away from old theory to new insights. The reason to do this, they argue, is that abductive reasoning is more likely to be open to, and lead to, surprising and innovative discoveries when compared to inductive approaches. They argue that in more than 60 years of grounded theory building GT has not led to any significantly ground breaking or new social science theory, which they attribute in part to the strong commitment of GT to inductive reasoning. The more traditional grounded theory approach, where theory is inductively discovered from the data, does not allow for the richness that can result when a broad, creative, and interdisciplinary theoretical orientation can be abductively applied to data collection and sense-making, and be more likely to result in creative leaps.  3.2.3 CGT and Analysis CGT has a very particular way that data analysis happens, shown in Figure 24. It is a bit like a funnel - the researcher begins with collecting an appropriate amount of data for their situation, with the constructivist version of grounded theory meaning that research can be informed by existing theory in the early stages. As the researcher collects data, they most    119  often code it line-by-line, typically using the first cycle coding methods of initial, process, and In Vivo codes (Charmaz, 2017a).   Figure 24: Constructivist grounded theory process map (Tweed & Charmaz, 2012)  As the researcher works with the data, they constantly compare what they are hearing and seeing with what came previously, going back and forth between data and codes, all while diligently writing memos about thoughts, insights, questions, and emerging categories that arise along the way. As the researcher starts to see some saturation of concepts, they move into second cycle coding which typically uses focused, axial, and theoretical codes in order to move data up to a higher level of abstraction, and into categories. This is how the researcher sorts, synthesizes, integrates, and organizes large amounts of data. The categories are worked and re-worked by organizing the codes, sometimes by coding them again, with the ultimate goal of finding/discovering theory that all of the categories and codes can be held within, or described by. Figure 25 shows this coding path, starting big with the data, and narrowing to theory.    120    Figure 25: From data, to codes, to categories, to themes, to theory (Saldaña, 2016, p. 14)  3.2.4 Modes of Thinking for Qualitative Research There was an aspect of the CGT approach to analysis that was intuitively not working for me and my research methodology and interests. This was the practice of breaking the data up into tiny pieces, and then reassembling it back into larger and more generalized chunks. The data was generated largely through in-depth interviews, observation, and reflection, with co-researchers sharing nuanced and interconnected stories of experiences and perspectives. Breaking it up into coded fragments removed these interconnections, and was not adequately reflecting what co-researchers were saying, writing, and doing. Even though CGT is focused on building theory grounded in data, it is a hierarchical and categorical process, where all of the data is subsumed within boxes, often using existing language, theory, and understanding as descriptors. The ultimate goal of CGT is to generate new theory, with    121  ‘theory’ being defined fairly typically in a social science research context. As I read more examples of CGT in action, I was not feeling interested in or inspired by the kinds of findings and theories that these studies typically produced. My research goals are about generating thinking and resources that are useful to growing the impacts of the field, for both researchers and practitioners. This may be theory, but it is very likely other outcomes as well. I continued to reflect on the critique from Timmermans and Tavory (2012) that although the aspiration of CGT is to build/find/discover/uncover/name new theory to describe phenomena, that this research method had yet to realize this potential from their perspective. This, along with using a method of analysis that is reductionist rather than systemic, meant that I went back to the methodology texts to see if I could find another option for analysis.   Freeman’s Modes of Thinking for Qualitative Data Analysis (2017) was the missing piece that I was looking for. She moves away from describing specific qualitative research methods and instead gets into the ways that researchers might think about analysis, and enact and be in relationship with analysis from different paradigms and points of view. She describes five modes of thinking: categorical; narrative; dialectic; poetic; and diagrammatic. CGT is firmly categorical - it is about fitting everything into neat, clean, nested categories and creating, or perhaps imposing, order on your data. Diagrammatical thinking was more compelling and promising for my purposes, named after a philosophical idea attributed to Deleuze and Guattari about bringing constructs together in experimental assemblages to provoke new or different ways of thinking. Diagrammatical thinking “seeks to disrupt conventional ways of thinking about human and nonhuman interactive spaces or networks. It asks that we look beyond the familiar narrative construction of a story and transverse core aspects of its telling in a way that creates new assemblages of moving and rigid formations, junctures, and concepts. Diagrammatical thinking involves looking at change through intra-acting materializing bodies, rather than through preconceived concepts or forms of classification” (Freeman, p. 9). This concept of intra-acting, as well as of staying entangled in the research process throughout rather than trying to get up and above your data, was a better fit with my methodological approach to iterate CGT and PAR, and keep these methodologies in dialogue with one another throughout.     122  This left me with an exciting question and possible direction: what if I reworked my research methodology, particularly the data analysis piece within CGT, and oriented it to a diagrammatical way of thinking? Coding in a particular way is foundational in CGT analysis, so I explored other ways to think about coding that enabled grounding in the data but with a more diagrammatical mindset. Two texts provided insight into how to methodologically approach this question. Saldaña‘s Coding Manual for Qualitative Researchers (2016) describes first and second cycle coding techniques, the transitions between these cycles, and 60+ different ways to code data. MacLure’s chapter called Classification or Wonder? Coding as an Analytic Practice in Qualitative Research (2013) was also helpful. She critiques coding from many different angles, and then suggests some ideas about how to think differently about coding that aligned well with a more diagrammatic approach.  3.2.5 Approach to Coding and Analysis Based primarily on the writing of Freeman and MacLure, and with support from Saldaña, I reframed the approach to analysis typical for CGT to better match what I hoped to achieve through my research and the specific opportunity that working with a CGT and PAR bricolage might bring. This reframe had five elements.   3.2.5.1 Use Different Coding Types Rather than using the standard first cycle CGT coding types, I modified what Saldaña calls eclectic coding. Eclectic coding is as it sounds, and is essentially a bricolage approach to coding. It “employs a purposeful and compatible combination of two or more first cycle coding methods… appropriate as an initial, exploratory technique...when a variety of processes or phenomena are to be discerned from the data” (p. 293). Saldaña still thinks of eclectic coding categorically, but I made it more open and flexible while remaining grounded in the data and using a diagrammatical mode of thinking. The specific types of codes that I used, and held within eclectic coding, were: values; evaluation; and In Vivo codes as described by Saldaña. I also conceptualized and used two new codes: wonder/emergence, and fractals.      123  Inspired by MacLure (2013), Lichtenstein (2014), and Timmermans and Tavory (2012) the code about wonder and emergence pays attention to what is showing up in between, in the embodied, liminal, and relational spaces. Wonder and emergence disrupt boundaries of power/knowledge, certainty/doubt, knowing/feeling, and animate/inanimate, for example. Recognizing the potential for wonder and emergence in the work of coding might invite moments of indecision on the thresholds of knowing, and from this point something new/else, or something unexpected and surprising might arise abductively. The code about fractals is inspired by systems practice. It considers what small pieces of data, or fractals, might tell us about the larger systems they are operating within. Fractals are patterns that exist and replicate across scales; they are wholes within wholes.   3.2.5.2 Instead of Categories Use Assemblages to Provoke Thinking CGT strives toward moving individual codes into higher order and more generalized categories, whereas diagrammatical thinking suggests the concept of assemblages for grouping and linking. An assemblage brings together diverse elements and vibrant materials of all kinds, without reducing them to one, in order to generate thought. Assemblages are in motion, and they are “sites of potential which open up possibilities for new means of expression, a new territorial/spatial organization, a new institution, a new behaviour, or a new realization” (Freeman, 2017). This idea captures the fluidity, movement, and contextual nature of what is happening with PSI labs much better than trying to fix something within a category. I used the rich description of assemblages from Freeman to guide how I organized my data, and this idea also inspired the title and construction of this dissertation as a cabinet of curiousities. I opted for a tactile and visual approach to coding and generating assemblages rather than a digital approach. I felt it was important to be able to see all of the coded data in order to work through this sensemaking approach, and that digital coding and analysis would not adequately enable that.      124  3.2.5.3 Instead of Second Cycle Coding do PAR  Instead of doing a second (or more) cycle of focused coding as CGT typically does, I went back to co-researchers to test, revise, round out, refine, and discuss what was emerging from the data and generation of assemblages. This process drew from action research, data dialogue, and interactive qualitative analysis processes. It provided additional contextual grounding and lived experience from a diversity of perspectives, and this input resulted in a final revision and iteration of the construction and description of the assemblages found in this dissertation. This approach helped to get clearer about what was and is emerging (in motion) about PSI lab theory and practice that can inform the field at this moment in time and hopefully in future.   3.2.5.4 Do Not Seek Saturation, Seek Potential and Possibility  CGT typically seeks saturation of categories as a way to validate findings. As the researcher constructs categories, they go back to their data, or collect fresh data, until they hear repeating information that confirms the categories that they have chosen. Because assemblages are not meant to be fixed and definitive, and are instead seeking possibilities in motion, this process of seeking saturation did not make sense for my approach. Instead of the way that CGT describes saturation, I used the following approaches to test validity in my research. First, as early theory emerged from the data, it was tested in a subsequent cycle of action research and literature review. This helped to circle in on emergent theory that was relevant to practice, and responded to gaps in the literature. To follow the metaphor of assemblages in motion, this process sought a gravitational pull in the direction that I was moving in the research. It followed threads of what co-researchers were holding as potent, promising, and provocative and also my own impulses as a researcher-practitioner. Second, two action research returns were conducted in October and November of 2020, with all co-researchers and expert interviewees invited and half participating. This enabled testing for resonance, rigour, and applicability for practice, generated feedback for theory refinement and future revision, and provided validation of both research process and results. Finally, the theories generated aim to create a sense of openness, possibility, future inquiry, and experimentation through practice rather than a more definitive ‘finding,’ which are expressed in Chapters 4-6.    125   3.2.5.5 Do Not Seek to Find the One Theory to Rule Them All   The goal of this research was not to ‘discover’ one great theory as is typical in CGT. It also was not about trying to get up and above the data in order to gain the perspective required for higher order abstraction as CGT methodologists recommend. The goal was to stay entangled in the CGT - PAR cycle at all stages and to see what emerged, while remaining committed to generating middle-range theory that was useful for both researchers and practitioners (Merton, 1968). Staying with the feeling of being unsettled and unresolved with the data analysis for as long as possible, and not trying to fit everything neatly together at the end of this writing, enabled an ongoing openness that was helpful in that it prevented a premature closure of inquiry. MacLure (2013) uses the metaphor of building a cabinet of curiousities, which I liked and built into my dissertation title. It provides a structure for the assemblages that is not hierarchical or linear while still putting a shape to some patterns, and enables people to interact with what is being produced in their own unique meaning-making ways. The methodology, and subsequent dissertation, aims to provoke plurality of thinking in a certain domain, and leave room for people to make their own meaning from what is being shared.   3.2.5.6 Description of Coding and Analysis This section shares, in some detail, how this CGT process of moving from detailed real-world PAR data to more abstract middle-range theory worked in practice. In the first research cycle, interview data from experts was manually coded to determine what could be discerned from the data. This early data came together in the following clusters (Figure 26):  Definitions, understanding, and theorization of public sector innovation;  Different constructions, conceptualizations, typologies, and roles of PSI labs;  Maturity of the PSI labs landscape;  What is at the leading and learning edges of expert practitioners;  How do lab practitioners understand, measure, and evaluate the impacts of their work;    126   What are the powerful stories and/or evidence that lab practitioners use to describe what is and is not working;  How are lab practitioners building, developing, and scaling their practices; and  Who is producing research in this field.   Figure 26: Illustrated example of what first cycle coding looked like.  This work, along with early action research data and PAR with Solutions Lab, resulted in early findings that led to reframing the initial research question into the four fractals that structured the dissertation. These early findings were that:  There was a wide range of PSI definitions, applications, and perspectives used by labs;  The landscape of both research and practice was relatively immature, rapidly growing and changing;    127   Network-serving, field-building, and knowledge mobilization work was only just beginning;  There was not much diversity in the European labs, and these labs were shaping the global field of thinking and practice about PSI labs;  There were some interesting positive deviants in the PSI labs that were relentlessly focused on improvement and impact, and on critiquing their own practice and the field in general in order to make it stronger, and that these positive deviants were a source of particularly interesting findings;  Labs were not strongly theorized;  Labs were not strategically investing time and effort into culture change, personal change, or movement building; and  Evaluation methods and practices were not strongly developed or in use by most PSI labs. The four fractals of the reframed question were then articulated as:  How might we more strongly theorize the purpose for PSI labs, and why might this be important?  How might public sector innovation labs systemically intervene in complex public sector challenges in ways that create stronger enabling conditions for transformative and emergent innovation?  What if PSI labs took a transformative innovation learning approach to their work in order to enable and unleash innovation within and amongst individual, organizational, and ecosystem-scale actors?  How might we measure, evaluate, and tell stories of transformative change resulting from PSI lab interventions?  Once the research questions had been reframed, a second cycle of PAR and CGT commenced in June 2018 and continued through to December 2019. As action research interventions and data collection was underway, active and regular memo writing was undertaken to make sense of what was happening, to surface what was being learned about the research questions, to document hunches that could be followed, and to collect reflections and insights. Interview data was then transcribed and coded. The first transcribed    128  interviews were coded in great detail using Trint, ensuring that a sample of interviews from each of the action research sites and experts was included in this collection to capture a diversity of experiences and perspectives. As patterns started to emerge in the data, coding became more focused on looking for wonder and emergence in particular, and ideas and insights that had not yet come up. Once an interview transcript was coded digitally, it was reviewed again and codes were written on sticky notes to enable a more tactile experience with working with the data, and to be able to see the codes being generated across different interviews together. Each sticky note was marked with the initials of the interviewee, so that similarities and differences across different action co-researcher typologies could be tracked and noted.  As small batches of sticky notes were generated by coding several interviews at a time, these were posted on walls, clustered into themes, and moved around as additional notes were added. General themes emerged in this early work including: defining innovation; theorization; leadership; enabling conditions; capacities; competencies; evaluation; and emergence (overarching ideas that did not quite fit elsewhere). All of the data related to all four of the research questions was worked with simultaneously as this was how the interviews were constructed, and this also enabled seeing the data as a larger whole in case different organizing structures and/or reframes of the research questions and theory-in-construction became apparent through this process. This reframing did occur within one large cluster of data. It was initially focused on data that related to ideas about leadership and enabling conditions for PSI and labs, but in working through to higher orders of abstraction, more systemic ideas that described a paradigm of transformative innovation, and patterns to move away and toward this paradigm, emerged. Figure 27 shows this process as it was underway. The sticky notes on the left are being clustered into the patterns that surfaced tensions/paradox/feedback loops that eventually became the theory constructed in Chapter 5. The sticky notes on the right of the image show very early clustering of codes related to competencies, capacities, and capabilities.     129   Figure 27: In process photo of creating assemblages from coded data.   This section provides some insight into the progression of a CGT process shown in Figures 24 and 25 as it was implemented in this research: transcripts, memos, and action research interventions are the data; smaller sticky notes are the codes; the larger sticky notes in Figure 27 are moving into assemblages at a higher level of abstraction; and then into the middle-range theory described in Chapters 4-6.   3.3 Summary of Research Sites, Activities, and Timeline This section shares some details about the primary action research site and two secondary action research sites that were engaged in this research. It then shares information about the experts that were interviewed, and provides a description of the research activities and timeline. In total there were 85 co-researchers in this project including: SLab – 32 people; LIUM – 10 people; Hub – 12 people; and other experts – 31 people. Their level of involvement    130  varied depending on whether or not they were experts, from a secondary action research site, or from the primary action research site. Most of the experts participated in one interview. People from LIUM and the Hub had one to several interviews depending on their role within their team, and teams also had action research interventions that involved sharing and dialoging about early theory building in cycle 2. SLab co-researchers had one to several interviews, and all participated directly in one or more direct interactions with SLab workshops and activities.   3.3.1 Research Sites I began with my primary research site as the City of Vancouver Solutions Lab (SLab). As my early research progressed, it became clear that including additional research sites would be beneficial for my research. This would allow for the application of the CGT and PAR research methodology to be applied in different contexts, and thus hopefully have the potential to lead to broader generalizability of the theory and findings that were built. I sought out the following attributes for the additional research sites:  Located within, and accountable to, a public sector body;  Include different levels of government, likely staying in Canada;  Have some differences to the SLab in terms of position within government, team construction, strategic approach, and area of focus;  Are early in their development but not brand new, so that they have worked out some of their basic ways of working and tried a few things, but have not yet established themselves in a fixed way; and  Is a team that is curious, open-minded, ambitious, big-hearted and have a commitment to movement building outside of their own context.  Using these attributes, I added two secondary research sites including the City of Montreal Laboratoire d’innovation urbain de Montreal (LIUM) and the British Columbia Government Public Service Innovation Hub (Hub).      131  3.3.1.1 City of Vancouver Solutions Lab The City of Vancouver is a municipal government in British Columbia, serving about 675,000 people. The Solutions Lab (SLab) was established in late 2016 and initiated by an action in the Healthy City Strategy, a comprehensive piece of public policy focused on social determinants of health. In that strategy there was a call to establish a ‘staff hub’, where staff could work differently on complex and interconnected public policies focused on health, environment, and economic wellbeing, drawing from social innovation approaches. I was the founder and have been the staff person responsible for the SLab since inception. The SLab was into its third iteration at the time of writing this dissertation, although the research shared here is focused on action research and data collected between 2016 to early 2020 when the preliminary, first, and second iterations were completed.   Although the focus of SLab work has shifted throughout its iterations, it has predominantly focused on transformative innovation in four policy domains: Healthy City Strategy; Greenest City Strategy; City of Reconciliation; and Equity. In the first iteration the work focused on co-creating multi-month social innovation lab processes using Theory U as the general process archetype, with teams of City staff and community partners working together. In the second SLab iteration a stronger theory of change was articulated through this action research, a community of practice was added as a key activity to build capacities, competencies, and innovation infrastructure more broadly. Rich descriptions of this work can be found in Appendices A and B. Developmental, formative, and summative evaluation approaches were used to measure impacts and outcomes and adapt SLab approaches in response to what was being learned. SLab’s reporting structure has changed over time, beginning with the City Manager’s Office and a Steering Committee, then shifting to reporting to the Social Policy and Sustainability Departments, and most recently to only the Sustainability Department. Funding remains insecure, with budget sought each year based on funding availability, planned activities, organizational priorities, and the reported outcomes.     In my position as the founder, manager, and sole full-time staff person of this PSI lab, I was in a strong position to actively shape, deliver, and evaluate this action research project, as well as research my own experience, positionality, and learning as the lead actor in shaping the work. The 32 other action researchers with SLab included:    132   Lab team leads: City of Vancouver staff, typically mid-level (i.e senior planners) who were most responsible for the lab questions that were worked on;  Lab team participants: the staff from multiple departments and multiple levels who participated in a lab;  Members of the community of practice (CoP): these were all City of Vancouver staff, from multiple departments and roles in the organization, who were learning and practicing PSI lab frameworks and techniques together;  Senior leadership: these were senior staff in the City of Vancouver who changed over time as the reporting structures shifted;  SLab team: this was myself, along with supporting staff, graduate students, partners, and consultants who together held responsibility for process design, facilitation, evaluation, strategy, management, and resourcing; and  Community partners were active participants in SLab activities and have informed this research, but were not extensively interviewed for this research given the focus on public sector.  I conducted extensive semi-structured interviews with co-researchers between 2018 - 2020. Each interview was about one hour in length, and was recorded for transcription and coding. I also documented strategy, activities, interventions, and reflections from my own experiences and perspectives. At several key points, including the pivots between iterations one and two, and iteration two and three, reflective workshops with SLab co-researchers were held. SLab co-researchers, including some that emerged in 2020 and were not formally interviewed for this research, participated in the final action research cycle in the fall of 2020.  3.3.1.2 Laboratoire d’Innovation Urbaine de Montreal (LIUM) This section draws directly from text provided by LIUM in order to describe their work in their own words. LIUM sits in the municipal government of the City of Montreal (population 1.78 million), which includes the City government and 19 boroughs, each with their own elected officials and staff. LIUM, in its current form, was established in May 2018 to lead the (successful) Smart Cities Canada funding and partnership bid and subsequent implementation. The LIUM aims to address the major urban challenges of the 21st century.    133  LIUM fosters and supports the creation of innovative solutions emerging from all walks of life. It forms a space in which Montrealers, businesses, municipal employees, and partners are invited to co-create a more people-focused, creative, open, and efficient city. LIUM is:  A spark that sets the innovation process in motion.  A free zone where all can explore and think outside the box.  Where limits are pushed, and the status quo is transcended.  Where all can imagine the outlook of an organization, a city, a future, adapted to today’s challenges.  This process does not belong solely to LIUM, but rather relies greatly on teams, departments, boroughs, and partners who serve as project leaders. LIUMs role and true value lies in taking a disruptive role by looking at issues through different lenses to provide fresh points of view. Through its multidisciplinary expertise, various tools, broad network, and unique positions, LIUM enables the emergence of new ideas that go beyond common, or predictable solutions. With its vast network of specialists and partners, its monitoring of best international practices, its solid rootedness in Montreal’s reality, and its understanding of municipal affairs, LIUM takes on a number of roles to meet a variety of needs (adapted from content provided by LIUM).  There were ten co-researchers with LIUM, including the LIUM team and leadership that supervised or helped to enable their work. Several preliminary in person and virtual interviews were conducted to determine applicability of LIUM as an action research site given my research questions. One in-person multi-day site visit occurred, which included interviews of all team members and a team workshop about theories of change, and systemic interventions for public sector innovation. Several interviews with key team members occurred afterwards to track the thinking, activities, and iteration of LIUM, and LIUM team members participated in the final action research share back in fall 2020.   3.3.1.3 British Columbia Government Public Service Innovation Hub  (Hub) This section draws directly from text provided by Hub in order to describe their work in their own words. The seeds for the Hub were planted in the Where Ideas Work Corporate Plan    134  (Government of British Columbia, 2016). This resulte