Open Collections

UBC Graduate Research

Youth Engagement in Community Capacity Building: A Developmental Evaluation Practice Story 2011

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata


SCARP_2011_gradproject_Gillett.pdf [ 1.71MB ]
JSON: 1.0102522.json
JSON-LD: 1.0102522+ld.json
RDF/XML (Pretty): 1.0102522.xml
RDF/JSON: 1.0102522+rdf.json
Turtle: 1.0102522+rdf-turtle.txt
N-Triples: 1.0102522+rdf-ntriples.txt

Full Text

 Youth Engagement in Community Capacity Building: A Developmental Evaluation Practice Story by SARAH GILLETT B.A., University of British Columbia, 2002  A PROJECT SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF  MASTER OF ARTS in THE FACULTY OF GRADUATE STUDIES School of Community and Regional Planning   We accept this project as conforming to the required standard  ............................................................................  .............................................................................  ............................................................................  THE UNIVERSITY OF BRITISH COLUMBIA April 2011 © Sarah Gillett, 2011 1       Youth Engagement in Community Capacity Building: A Developmental Evaluation Practice Story  Sarah Gillett April 2011 2    Acknowledgements Many thanks to Margo Fryer for presenting me with the opportunity to get involved with this project and for providing me with a great reading list and ideas along the way.  Thank you to Susan Grossman for her ongoing support on the ground and to Leonie Sandercock for her encouragement, inspiration and guidance along the way.  Thank you to the community partners for allowing me to participate in your planning process.  Most of all, thank you to the youth participants for working with me, inspiring me and carrying on this work.  Executive Summary Given the growing interest in planning approaches that deal with complexity, this project tells the practice story of one approach, developmental evaluation, in the context of a youth engagement and community capacity building project.  With theoretical roots in communicative and empowerment planning, this project takes an asset-based approach to working with youth and communities. Community capacity is a term that is gaining legitimacy as a metric in which to talk about the health of communities.  Youth are being recognised as important assets in community capacity building.  A literature review of these two fields sets the stage for a case study of developmental evaluation in practice.  The specific case is my involvement in the Engaging Neighbourhood Initiative in the Victoria/Fraserview community of Vancouver, British Columbia.  My practice story explores several aspects of a developmental evaluator’s role: 1) building a culture of evaluative thinking; 2) facilitating reflection and feedback; 3) facilitating participatory evaluation;  4) facilitating decision making; 5) applying an evaluation framework; 6) data collection; 7) feeding back information;  and 8) mobilizing assets.  It concludes by providing the community with some recommendations and more broadly exploring the limits and possibilities of developmental evaluation as a planning tool.  3   Contents Acknowledgements ....................................................................................................................................... 2 Executive Summary ....................................................................................................................................... 2 Introduction .................................................................................................................................................. 5 Context .......................................................................................................................................................... 6 Resilience and complexity ......................................................................................................................... 6 Theoretical Roots of the Project ............................................................................................................... 7 Community Capacity and Social Capital .................................................................................................... 9 Youth Engagement .................................................................................................................................. 11 Developmental Evaluation ...................................................................................................................... 14 The Project .................................................................................................................................................. 16 Methodology ........................................................................................................................................... 16 The Case: Victoria/Fraserview ................................................................................................................ 17 The Case:  The Engaging Neighbourhood Initiative ................................................................................ 20 Developmental Evaluation in Practice ........................................................................................................ 22 Building a Culture of Evaluative Thinking ............................................................................................... 22 Facilitating Reflection and Feedback Sessions ........................................................................................ 24 June 2010 Dinner Debrief ................................................................................................................... 24 February 2011 Event Debrief .............................................................................................................. 26 Facilitating Participatory Evaluation ....................................................................................................... 28 Facilitating Decision Making ................................................................................................................... 30 Applying an Evaluation Framework ........................................................................................................ 33 Data Collection ........................................................................................................................................ 37 Feeding Back Information ....................................................................................................................... 38 4   Mobilizing Assets .................................................................................................................................... 40 Recommendations for the Engaging Neighbourhood Initiative ................................................................. 43 Further Evaluation Recommendations: .............................................................................................. 43 Future of the Initiative Recommendations: ........................................................................................ 43 Limits of Developmental Evaluation ........................................................................................................... 45 Human Resources ............................................................................................................................... 45 Accountability ..................................................................................................................................... 46 What’s in a name? .............................................................................................................................. 47 The Role of Developmental Evaluation in Planning .................................................................................... 47 Developmental Evaluation as a Resilience Tool...................................................................................... 47 Developmental Evaluation as a Capacity Building Tool .......................................................................... 48 Conclusion ................................................................................................................................................... 50 References: ................................................................................................................................................. 52 Appendix 1: Roger Hart’s Ladder of Young People’s Participation ............................................................. 57 Appendix 2: Workshop Facilitation Template............................................................................................. 58 Appendix 3:  Initiative Framework .............................................................................................................. 59 Appendix 4:  Youth Designed Evaluation Framework ................................................................................. 61 Appendix 5: Youth Blogs ............................................................................................................................. 63 Appendix 6: SWOT Analysis ........................................................................................................................ 65  Figure 1: Difference between Traditional and Developmental Approach over time. ................................ 15 Figure 2: Location of Victoria/Fraserview in Vancouver ( ................................... 17 Figure 3: Dynamical Actual-Ideal Comparative Evaluation Framework (Patton, 2010) ............................. 34 Figure 4: Youth brainstorm: What is Community Capacity?....................................................................... 40 5    Introduction  My planning degree has made one thing crystal clear: my career will have me work in situations of high complexity.  As I work on building the skills necessary to interact effectively with the social, environmental and economic challenges facing our communities, I have had the privilege to apply what I was learning to a project that was just starting in a Vancouver neighbourhood.  A number of themes emerged from this project:  community capacity building, resilience, youth engagement and evaluation in the face of complexity.  I came to focus on a practice tool “developmental evaluation” to guide my work and my final written piece.  This paper tells the story of using developmental evaluation in the context of a community capacity building project.  My hope is that this story will help document and build social learning for the partners involved in the project.  This story will also be of interest to anyone passionate about issues of youth engagement, building community capacity and using tools that respond to complexity. For this project I played the role of a facilitator and developmental evaluator for a group of community members in the Victoria/Fraserview neighbourhood.  This group of service providers in partnership with the UBC Learning Exchange and the United Way are working to increase community capacity through engaging youth.  Specifically, they are looking at ways to strengthen relationships between service providers and community members in order to better support the community’s middle year youth (aged 6-12).  My research paper will build on relevant research on the topics of complexity, community capacity, youth engagement and evaluation.  Through a case study, this project describes how youth engagement and developmental evaluation can be applied in practice.  I will also make recommendations on how this approach can be used more generally to support planning in complex situations. “You are entering here the world of uncertain beginnings, muddled middles, and unpredictable endings that ripple on and on without end” (Patton, 2010 p.9). 6   The overall objectives of my graduating project are:  to facilitate a community capacity building process using a developmental evaluation approach  to build capacity in the community’s youth to carry out community capacity building projects  to provide an example of how developmental evaluation can be applied to complex planning challenges. Context  Resilience and complexity  Our work as planners happens within complex environmental, social and economic systems.  A body of literature on complexity highlights that our traditional managerial approaches, which presume a world of simple rules, are wrong-headed and likely to be dangerous (Kay and Schneider, 1994; Innes and Booher, 2010).  “Science has explored the microcosms and the macrocosms; we have a good sense of the lay of the land.  The great unexplored frontier is complexity” (Pagels, 1988).  When planning within a complex system, there is a high degree of connectivity and interdependence between variables and it is difficult to understand the ramifications of change.  Planners must find ways to effectively engage stakeholders in complex issues. In the face of complexity, theorists talk of the importance of resilience.  Authors such as Magis (2010) examine how communities can develop resilience by actively building and engaging the capacity to thrive in an environment characterized by change: “community resilience is not about controlling all the conditions that affect communities.  Rather, it is about individual and community ability to respond to “A growing body of science based in complexity theory and resilience theory demonstrates the falsity of assumptions of stable, predictable and linear behaviour for socio-ecological systems. We must experiment with new adaptive management approaches to transform our economic, social, cultural and environmental systems at systemic, strategic and tactical levels”  (Sarkissan et al, 2009, p.35) 7   change” (p. 403).  Walker and Salt (2006) speak of the need for adaptive learning to build resilience in our communities in the face of a changing climate: “a resilient world would place an emphasis on learning, experimentation, locally developed rules, and embracing change”. Innes and Booher (2010) describe a shift away from traditional linear planning methods relying primarily on formal expertise.  In its place, there is a rise in interest in nonlinear socially constructed processes engaging both experts and stakeholders. Non linear processes are described as always contingent and evolving, cycling back to goals as new things are learned (p. 5).  Innes and Booher (2010) state that traditional approaches to evaluating public programs do not make sense for planning in the face of complexity. Similarly, Westley, Zimmerman and Patton (2006) talk of the need for reflective practice that involves ongoing data collection and assessment to help us adapt decisions in the face of changing conditions (p. 87).  As an alternative to traditional linear evaluative practices, Westely et al. suggest developmental evaluation.  This approach tracks and attempts to make sense of what emerges under conditions of complexity, documenting and interpreting the dynamics, interactions and interdependencies  (Patton, 2011, p. 7).  This approach defined my role and work with this community project as the partners and youth navigate complexity.  It is an approach that attempts to facilitate the collective decision making and adaptive management required to solve complex social problems. Theoretical Roots of the Project  This section explores the theoretical routes that have informed my approach with this project.  The project has been inspired by a combination of the communicative tradition of planning, empowerment planning and asset-based development. “The traditions of communicative planning seek planning processes that are inclusive, discussion- orientated, consensus building and transformative, in that they engender social learning through a respect for difference and recognition of others’ values” (Bond and Thompson-Fawcett, 2007). 8   The communicative body of planning theory has evolved to address how planning process can disempower, exclude, silence and marginalise certain groups (Forester, 1999; Healey, 1997).  In response to power inequalities, communicative planning focuses on the micro-dynamics of planning practice including skills such as astute listening, (Forester, 1999) facilitation and empowerment to ensure those potentially excluded are heard (Healey, 1997, Innes, 1996).    The project starts from the assumption that youth are powerful change agents who are typically excluded from planning processes. The communicative approach aspires to bridge the divide between youth and more traditional planning actors.  This requires an appreciation of different ways of knowing (Sandercock, 2003) and an acknowledgement of the knowledge youth possess that is relevant to planning. This project also draws heavily from the tradition of empowerment planning, teaching youth that they have the power and skills to be planning actors.  Kenneth Reardon’s (2003) story about a university- community relationship in East St. Louis emphasizes the importance of passing on planning knowledge in order to effectively build capacity in a community.  The ultimate goal as a planner is to empower the community to plan for themselves through inspiration, technical support and knowledge transfer.   This project has from the beginning focused on capacity building and attempted working with the youth to organise themselves.  The project’s approach also draws from Kretzmann and McKnight’s (1993) concept of “asset-based” community development which aims to help communities recognize their assets with the goal of mobilizing them for development purposes. Finally, the style of this report was inspired by the importance the communicative tradition of planning places on practice stories (Healey, 1997; Forester, 1999).  This case study not only describes a case of youth engagement in community capacity building, but describes the practice of working as a planner, facilitator and developmental evaluator in this context.  9   Community Capacity and Social Capital  Community capacity is a concept that is gaining prominence as a legitimate metric in the fields of health (Kwan et al. 2003; Neale et al. 2003), sociology (Atkinson and Willis, 2005) and community development (Chaskin, 2001).  Increasingly, community capacity is used as a way in which to talk about the health and resilience of a community.  Parallel to the rise in interest is a need to define what we mean by community capacity, and how we build it.  The most common definition and the one most similar to the one that the youth used to guide their work is: “the combined influence of a community’s commitment, resources and skills that can be deployed to build on community strengths and address community problems and opportunities” (Aspen Institute, 1996, p.1). Community capacity has also been described as the ability to deal with problems without relying on external resources (Atkinson and Willis, 2005).  Chaskin (2001) defines community capacity as the interaction of human capital, organizational resources, and social capital existing within a given community that can be leveraged to solve collective problems and improve or maintain the well-being of a given community.  Chaskin identifies four fundamental characteristics of community capacity including a sense of community (collectively held values), a level of commitment among community members (responsibility that particular individuals take for what happens in the community), the ability to solve problems and access to resources (economic, human, physical and political).  Capacity can be built at the individual, organizational and network level.  Chaskin (2001) outlines four strategies that community building efforts tend to focus on: leadership development, organisational development, community organizing and fostering collaborative relations among organisations. “People should be meaningfully involved in processes of relationship building, community-based learning, participatory planning and joint action that improve life on earth in some way” Scott Graham (2009) p. 1 10   A key concept of community capacity is that of social capital.  Putman (2000) explores how community health, educational achievement, local economic strength and other measures of community well-being are dependent on the level of social capital that exists in a community.  Social capital is defined as the quality of the relationships and the cohesion that exists among its citizens.   As part of Putman’s research, he studied a number of Italian communities to try to understand why some were more democratic, more economically successful, had better health and experienced better educational achievement.  The relatedness that existed among its citizens was the one thing that distinguished the more successful from the less successful towns.  Social capital can be understood as the glue that holds communities together or the networks that facilitate coordination and cooperation for mutual benefit. Dale and Newman (2010) describe social capital as one of the “biggest growth areas in network research”.  However, they argue that while social capital is a necessary condition for sustainable community development, it is not always sufficient to sustain and develop local community initiatives without infusions of economic and human resources along with government policy alignment. Sarkissian et al. (2009) identify social capital as the most critical component of social sustainability.  As part of the Sustainable Region Initiative (2007), a social issues subcommittee defined social sustainability as a community that has sufficient relationships, networks and norms to make improvements to their quality of life and ensure that such improvements are sustainable in the long term.  They use the term social or community capacity to describe the relationships, networks and norms that facilitate collective action to improve upon quality of life and to ensure that such improvements are sustainable. Inspired by this work, and by authors such as John McKnight, Peter Block and Robert Putnam, my research starts from the assumption that building community capacity at the neighbourhood level is integral to a community’s health and resilience.  Although there are similarities between the various definitions of community capacity referenced above, I embark on this project with an understanding that there is still little consensus as to the best way to measure and quantify the phenomenon.  Perhaps that is because community capacity is a perfect example of a complex phenomenon that involves many interrelated systems.  This project attempts to help a community navigate and work with that complexity. 11   Youth Engagement  This project also stems from my experience working in youth leadership, specifically in a community development context.  This experience coupled with my familiarity with youth engagement literature has led me to believe that engaging youth is crucial to a community’s capacity and that youth are a community asset that is often under-utilized and misunderstood.  Many researchers and authors have explored the topic of youth engagement in planning and community development (Checkoway et al. 1995; Hart, 1997; Driskell. 2002; Gurstein et al. 2003; Frank, 2006) however questions remain as to the factors that lead to successful and effective youth engagement.  Frank (2006) highlights that while youth participation in planning has been promoted in the literature, the practice is still uncommon, suggesting that that there are major barriers to the practice.  For planners working towards diverse and inclusive communities, this question also applies to other groups typically marginalized from planning processes. The 1989 UN Convention on the Rights of the Child establishes youth’s basic right to be actors in their own development; to express their views on all matters affecting their lives; and to seek, receive and impart information and ideas (UNICEF 1989).  Driskell (2002) points out in his manual for youth participation that planners have a responsibility to create ways for youth to be able to participate through avenues that are appropriate to them.  Many studies show that when given the opportunity, youth excel in community development roles, but are limited by lack of trust in their abilities from adult counterparts (Gurstein, Lovato & Ross, 2003; Checkoway, 1998; Frank, 2006). There are many ways one could imagine youth in the face of exclusion from planning.  One can view youth as vulnerable and in need of protection; or as developmentally in a period of early psychosocial growth thus lacking the skills to participate.  From a legal view, youth can be treated as citizens-in- “From the beginning, the tone was set that this project was not just about organizing a dinner, but also about building capacity in youth to be effective community capacity builders”. (Gillett, 2010) 12   training.  Lastly, from a romantic view, one can see youth as having values and capabilities that are distinct from and superior to those of adults (Frank, 2006).  The approach taken in this paper is recognizing youth as an asset for community development.  Frank’s review of numerous youth engagement initiatives concludes that involving youth not only has the potential for positive impacts on youth participants, but also provides new information and recommendations that address both youth- specific and community-wide concerns (Frank, 2006). Checkoway outlines three major barriers to youth’s participation.  First of all, the feeling that adults are trying to manipulate youth by ’educating’ them about community development rather than including them in community development processes.  Secondly, a lack of real power in affecting decision making even if youth are present in community development activities.   This absence of power is due to the fact that planners often over-represent stakeholders with economic interests in land use decisions.  The third barrier is that adults in general perceive youth as something to protect, not to involve. Gurstein, Lovato and Ross (2003) identify the following factors that facilitate youth participation: having an equal voice in an adult structure, skill development, feeling supported by the larger organization, being involved in all aspects of projects, adult/youth partnerships, and feeling trusted and given space to develop new ideas.  Overall, the authors identify co-leadership, in structures where there is trust and acceptance, as key to successful youth participation.  Also processes are needed where youth feel heard and having the power to affect change.  In Wheeler and Roach’s (2005) examples of successful youth involvement, a few themes recur:  plans that are youth designed and directed, education that focuses on critical thinking and creating opportunities for youth to know more about themselves.   Checkoway (1995) suggests creating opportunities far from the token consultation of youth.  He recommends avenues for involvement where youth can directly affect their communities and emphasizes the need for education not only for youth, but for planners and the general public. The Funders Collaborative on Youth Organizing (2000) introduces the idea of a continuum of youth engagement approaches.  This tool is very useful when conceptualizing the nature of a youth engagement project and choosing which approach is most appropriate.  On one side of the continuum there is the youth service approach which provides treatment and supports needed to address problems young people encounter.  Next there is the youth development approach which recognizes that there is a list of desired competencies for youth which includes academic, civic, cognitive, emotional, social, 13   cultural and vocational competencies as well as allowing youth to be competent, connected, caring and committed.  The youth leadership development approach helps young people look beyond their personal needs and interests to see their relationship to a collective group, organization, or community. Youth civic engagement is defined as young people developing the skills and habits needed to actively shape democratic society in collaboration with others.  Finally, on the far end of the spectrum, youth organizing is a youth development and social justice strategy that trains young people in community organizing and advocacy, and assists them in employing these skills to alter power relations and create meaningful institutional change in their communities.  The continuum reminds us that different approaches are needed for different youth depending on their past experience.  It also reminds us that all of these approaches are necessary, complementary and build on each other.  While the goal may be to have all youth involved in youth organizing, it is ineffective to offer programming with that approach to youth whose basic needs are not being met.  Evaluating a youth engagement strategy requires a clear idea of what approach is being used in the program in question. In the spirit of Arnstein’s (1969) ladder of citizen participation, Roger Hart (1997) created a ladder specific to youth participation (appendix 1) where in the bottom rungs youth are manipulated, tokenized or used as decoration.  The top rung best describes the result hoped for in this project, where “young people and adults share decision making”.  This rung echoes the Gurstein et al (2003) suggestion that co-leadership and adult/youth partnerships are key to successful youth engagement.  The goal of this project is to create an opportunity for youth to have meaningful decision making power, and that youth and adults can work collaboratively on community building projects.  This project aims to enhance the conditions necessary for this to be possible and sustainable and this report aims to tell the story of how this can be facilitated in practice.      Figure 1: Youth Organized Community Fair 14   Developmental Evaluation  In preparation for this final project, I conducted a directed studies in which I completed a literature review of major traditions in evaluation.  My quest was to learn about evaluation techniques that could be used in the Engaging Neighbourhood Initiative.  Specifically, given our goal in building community capacity, I was interested in learning about evaluation techniques appropriate for situations of high complexity.  What this exercise reinforced for me is that there is no one evaluation method that is superior to any others; rather, the success of a given evaluation method depends on its ability to respond to the users’ evaluative needs.  Judging a project’s value is highly personal and situational; therefore evaluation must achieve intended use by intended users. Evaluation is an important stage of any planning process and is typically integrated within planning models (Boothroyd, 1991; Trousdale, 2005).   Mark et al. (2000) describe four purposes of evaluation: program improvement; knowledge development; an assessment of merit and worth; and oversight and compliance.  An additional purpose of evaluation uncovered through my directed studies was the ability for evaluation to be used as a community capacity building tool.  The most relevant benefit in this case I believe is the trust that can be built throughout the process of collaboratively evaluating a project. “Evaluation approaches can reinforce trust and establish a firm foundation for future engagement processes” (Sarkissian et al. 2009, p. 184).  The evaluative method most appropriate for this project is one that holds the potential for trust building and learning. “The right kind of evaluation can be a powerful tool to help the social innovator stand still and take stock.” (Westley, Zimmerman and Patton in Getting to Maybe, 2006 p. 83). 15   Traditional evaluation methods are described by Westely, Zimmerman and Patton (2006) as a major barrier to learning and innovation.  Premature demands for accountability can squash innovation and burn up precious resources.  The dominant approach to solving problems is one where there is a natural sequence of steps that moves us from problem to solution.   The critique by Westely et al. is that when dealing with the goal of “changing lives” a linear goals based model is inadequate.  When dealing with complex systems, a highly iterative evaluation system is needed that can respond to a lack of control and still stay in touch with what’s unfolding and provide a strategic response.  Developmental evaluation facilitates assessments of how things are unfolding, helps to discern which directions hold promise and suggests what new experiments should be tried.  It is a highly adaptive evaluation method that seeks to connect the needs of stakeholders with responsive solutions.  Developmental evaluation is a way of being useful in innovative settings where goals are emergent and changing rather than predetermined and fixed, time periods are fluid and forward-looking, rather than artificially imposed by external deadline, and the purposes are innovation, change, and learning rather than external accountability (summative) or getting ready for external accountability (formative)  (vii, Patton, 2011).  Another way to distinguish Developmental Evaluation from traditional evaluation is that it is embedded in the process of planning and acting, as opposed to being limited to the beginning (formative) or the end (summative) of a project (figure 1). As Patton highlights in his work, developmental evaluation is not appropriate for every situation.  It is most appropriate for projects in their early stages which deal with complex systems.  Given the complexity of the phenomenon of community capacity, and the early stages of this initiative, this project is well suited to a developmental evaluation approach.  My goal while working on this project was to use developmental evaluation thinking in guiding my work with the group of youth and the steering committee of this project. Figure 2: Difference between Traditional and Developmental Approach over time. Source:  Gamble, 2009 Developmental Evaluation Primer 16   The Project Methodology My interest in telling a practice story has led me to choose case study methodology.  The case in question is my experience as a facilitator, applying developmental evaluation with my work on the Engaging Neighbourhood Initiative in Victoria/Fraserview.  Case study methodology shares many of the theoretical underpinnings of Developmental Evaluation.  Many researchers highlight the strength of this methodology in understanding a complex, interrelated system.   Gerring (2004) highlights the depth of analysis as one of the virtues of case studies.   Gerring describes depth as “the detail, richness, completeness, wholeness, or degree of variance that is accounted for by an explanation”.   Yin (2003) describes case studies as most appropriate to understand complex social phenomena.  It allows investigators to retain the holistic and meaningful characteristics of real-life events such as organizational and managerial processes or neighbourhood change.  VanWynsberghe and Khan (2007) describe how case studies promote learning through the study of the particularities and complexities of a bounded system. Stake and Yin speak to case studies as an evaluation tool.  Stake (2004) specifically touts the case study as an appropriate way to conduct “responsive evaluation.” Responsive evaluation is evaluation that relies from beginning to end on interpretive thinking and is responsive to the perceptions and voices of the people associated with the program.  This approach values the fact that the meanings of accomplishment and success are situational, reflecting to the locality of the program being evaluated. When studying a human phenomenon as complicated as community capacity building, there is little use for predictive theories; however, a context-specific story of an attempt to build community capacity can enhance our understanding of what community capacity looks like and how in one context it can be approached.  While the experience of one community cannot realistically be replicated exactly in another, the force of example helps inspire other communities to look at what factors could be replicated in their specific context. Given my objective not only to better understand community capacity but also to actively build capacity through my work, I borrow heavily from the tradition of Participatory Action Research (PAR).  PAR is appropriate for this project as it is a methodology where the outcome is not only knowledge but the 17   action itself (Kindon, 2005).  Through cycles of action and reflection (Armstrong, 2003), understanding of how to engage youth in community capacity building emerges for those involved.  As a methodology, action research values community members and youth as experts and co-researchers (Baum et al. 2006).  The projects undertaken by the youth arise directly from the needs and desires expressed by our community research partners.  The results of this report are as important as the process that was undertaken and the capacity that was built in doing so. The Case: Victoria/Fraserview Victoria/Fraserview is an ethnically diverse community in Vancouver’s south-east corner.   This neighbourhood is set on the south slope of the rise that runs north from the Fraser River. Victoria- Fraserview stretches from 41st Avenue to the North Arm of the Fraser River, and from Knight Street to Vivian Street. Traditionally it is part of the territory of the Musqueam people.   Archaeological journals have recorded evidence of Musqueam’s existence at the mouth of the North Arm of the Fraser River in excess of 4,000 years during which the land supported fishing, hunting, trapping and gathering (Musqueam, nd).  Victoria/Fraserview was first inhabited by non-indigenous populations in the 1860s.  The community was sparsely populated until after World War II when shortage of housing for war veterans led to 1100 new homes.  The Canadian Mortgage and Housing Corporation expropriated 182 hectares for this housing scheme.   In the early 1950s, Fraserview became known as "diaper town" because there were so many children living in the new subdivision.  Because of its proximity to the Fraserview Golf Course, all the new streets in the Fraserview subdivision were named after golf courses from around the world.  Today, Victoria/Fraserview encompasses a large area of residential and commercial development.  The neighbourhood is largely residential, with a strip of commercial, recreational and cultural activity along Victoria Street.  Traditionally, Victoria-Fraserview has had a strong industrial presence along the north arm of the Fraser River.  However, since the late 1980s, there has been a decline in industrial uses and a Figure 3: Location of Victoria/Fraserview in Vancouver ( 18   trend towards re-use of industrial lands for residential development.  A new comprehensive residential development, Fraserlands, was built along the river waterfront.  The population of approximately 29 200 (Statistics Canada, 2006 Census) grew by 7.5% in the last census cycle, which is higher than the city average (5.9%).  Geographically, there are a few obstacles to enhancing connectivity and cohesiveness throughout the neighbourhood.  Several north-south arterial roads dissect the neighbourhood such as Knight St, which is a major truck route.  Marine Drive also splits the neighbourhood and creates an isolated pocket south of Marine drive along the Fraser River (Fraserlands). This area, zoned residential, comprises townhouses and low-rise apartments, but contains no commercial or recreational services.  The physical separation of this area from the rest of the neighbourhood is exacerbated by the steepness of the rise north of Marine Drive, and the limited number of pedestrian cross walks across Marine (VMCMI, 2009).  The primary buses that provide service to downtown Vancouver run along Knight and Victoria Street. East/West buses run along 41st, 49th and Marine Drive.  There is no rapid transit in the area, but buses connect to the Millennium Line Sky Train.  62% of the total population in Victoria/Fraserview are immigrants (CitySpaces, 2005) with more than one quarter (26.3%) recent immigrants (VMCMI, 2009).   This is somewhat higher than the proportion of recent immigrants living in Vancouver as a whole (19.5%) (City Spaces, 2005). In 2006, 48% of the neighbourhood’s population identified Chinese as their mother tongue, followed by English (25%) Punjabi (6%), Tagalog (4%) and Vietnamese (3%) (City Spaces, 2005). 11% of the residents have no knowledge of French or English (VMCMI, 2009) and approximately 30% of the population is low-income (City Spaces, 2005) indicating that there are a number of challenges facing families, including language  Victoria Fraserview Vancouver Population 2001 27, 152 545, 671 Population 2006 29, 200 578, 041 Rate of change 2001-2006 7.5% 5.9% % recent immigrants 26.3% 19.5% Chinese- Mother Tongue 48.1% 25.3% English -Mother Tongue 25.7% 49.1% Punjabi- Mother Tongue 5.9% 2.7% Tagalog- Mother Tongue 4.0% 2.8% Vietnamese -Mother Tongue 2.8% 1.8% Table 1: Population Statistics 19   barriers and affordable housing (VMCMI, 2009). Victoria-Fraserview’s population differs from that of Vancouver as a whole in that it has a greater share of children and youth (City Spaces, 2005).  There are 3510, 6 – 12 year olds living in the Victoria / Fraserview neighbourhood (VMCMI, 2009) accounting for 8% of the neighbourhood population.   Middle years children (6-12 year olds) coupled with the children under 5 comprise 14% of the total census tract population. The South Vancouver area (comprising of Victoria / Fraserview, Marpole, Sunset and Champlain Heights) has the 3rd highest number of children under 6 in its boundary in the Province (VMCMI, 2009). In 2009, The Vancouver Middle Childhood Matters Initiative (VMCMI) conducted an asset-mapping project in South Vancouver.  The Victoria/Fraserview neighbourhood was selected for several reasons including, timing with other initiatives starting in the community and a high need for support surrounding the 6-12 year old population.  Needs in the community identified by this report include: no Vancouver Parks and Recreation Community Centre located in the neighbourhood, waitlists for licensed after school care and increasing pressure on the elementary schools in the neighbourhood to address the issue of Indo-Canadian gangs.  A number of assets in the community were also identified, most notably a wide variety of service providers who have programming for middle year youth and share a common vision. The primary objective of this project was to gather information in order to assist the guidance of future middle years initiatives in the Victoria / Fraserview neighbourhood.  One of the principle findings was that cultural perceptions, language and lack of information (about organizations, accessibility etc) were barriers for accessing available programs.  The report recommended outreach to parents and caregivers of 6-12 year olds not engaged in non-school activities.  It also recommended increased communication and coordination among middle years service providers.  A recent self-report survey with youth between 6-12 years of age in Vancouver revealed that Victoria/Fraserview is one of the lowest scoring in terms of the middle years development index (MDI).  This index gathered information about children’s Image 1: Victoria Street 20   relationships and connectedness with parents, school and neighbourhood adults and how they spend their time during the after-school hours (Hertzman et al. 2010). The Case:  The Engaging Neighbourhood Initiative This project arises from my involvement with the Engaging Neighbourhood Initiative which is a joint partnership between the University of British Columbia’s Learning Exchange, the United Way, the Vancouver Board of Education and neighbourhood service organizations in Victoria/Fraserview (box 2). Funding for the project came from the United Way due to their emphasis on supporting middle year youth.  In response to the middle years research (VMCMI, 2009; Hertzman et al., 2010) a group of service providers in the area formed the project’s steering committee along with representatives from UBC, the United Way and the Vancouver Middle Years Matters Project.  The Engaging Neighbourhood Initiative’s approach is to work with community partners to find community driven collaborative solutions.  The underlying philosophy behind the initiative is that community-university engagement should be responsive to the priorities set by communities.  The steering committee meets regularly to discuss and oversee the project.  Early on in my involvement with the project, the committee identified lack of cohesion in the neighbourhood as a barrier to serving the community’s youth.  This was informed by the previous work that had been completed with the Victoria/Fraserview Asset Mapping (VMCMI, 2009) project which identified that lack of information to community members was a barrier in them accessing community services.  The focus on relationships arose because many of the ideas brainstormed in my first meeting with the committee were identified as difficult to implement because of lack of community connection. Box 1: Engaging Neighbourhood Initiative Partners Victoria/Fraserview: United Way of the Lower Mainland UBC Learning Exchange South Vancouver Neighbourhood House David Thompson Community School Team Vancouver Middle Childhood Matters Initiative Fraserview Boys and Girls Club 21   The community partners saw a need for more opportunities for community members to informally share resources, to network and to build relationships.  They also saw an opportunity to mobilize the community’s youth to work on this effort.  As a result, it was decided that a group of undergraduate UBC students who grew up in the Victoria/Fraserview neighbourhood would work with high school aged youth currently living in the neighbourhood to plan a series of community events aimed at creating relationships between members of the community. This past summer, as part of an internship with the Engaging Neighbourhood Initiative, I recruited 7 UBC youth and 7 high school youth to form an organizing team for the first community dinner which was held at the Orchard Park BC Housing complex.  All of the youth chosen grew up in the Victoria/Fraserview neighbourhood, and some of them were chosen because they were current residents of the BC housing complex.  I facilitated the group’s planning work by convening the youth and providing them guidance by structuring their weekly meetings, but left the planning work to them.   The dinner they planned was attended by 60-80 people and was well received by the community.  Subsequently, one of the local undergraduate students involved in the project replaced me in my role as coordinator and convened a second group of youth to work on the next community event which focused more specifically on engaging middle year youth.  This group was composed of 4 UBC youth and 9 high school youth.  6 of the youth were involved in planning the first event, while the rest were new recruits.  This process, much more so than the first, was successfully youth driven, as my role in the process was peripheral.  This event was even more elaborate than the first and was attended by 250-300 people and engaged about 30 additional youth volunteers.  The event’s activities specifically aimed at engaging middle year youth.  Attendees at the event commented on the importance of these kinds of activities to help bring parents and youth together in a community setting.  At this stage of the project I continued to play a mentorship role for the youth committee and continued my role as a member of the initiative’s steering committee.  With two events behind them the steering committee and the youth must decide where to take this initiative next.  Through a developmental evaluation lens, this report will tell the story of what has been accomplished and learned to date.  This report has the Image 2: Youth Designed Logo 22   practical application of recording this group’s experiences and making recommendations for future directions, but theoretically, also gives an example of how developmental evaluation can be applied in practice to complex planning problems such as building community capacity. Developmental Evaluation in Practice In this section I tell the practice story of my role as facilitator and developmental evaluator for the Engaging Neighbourhood Initiative in Victoria/Fraserview.  I have organized the story into categories that correspond to different roles of a development evaluator which include:  1) building a culture of evaluative thinking; 2) facilitating reflection and feedback; 3) facilitating participatory evaluation; 4) facilitating decision making; 5) applying an evaluation framework; 6) data collection; 7) feeding back information; and 8) mobilizing assets. Building a Culture of Evaluative Thinking  Inspired by Patton’s (2011) description of the developmental evaluator’s role, infusing evaluative thinking into the work of the youth and the steering committee has been one of the primary goals of my involvement in this project. Specifically for the youth, when recruiting the initial UBC participants they were asked to individually reflect on factors that led to their success growing up in the Victoria/Fraserview neighbourhood.  “What are the key elements that helped you get to where you are today?  Was there anything that could have further helped you and in what ways can you imagine yourself contributing to making that a reality for the next generation?”  Answers included:  volunteer and recreational opportunities, support from family, opportunities to take on responsibilities, participation in community leadership, opportunities to expand their social networks, feeling safe and supported by community members and an opportunity to “The evaluator’s primary function is to infuse team discussions with evaluative questions, thinking and data and to facilitate systematic data- based reflection and decision making in the developmental process” (Patton, 2011, p. 1) 23   interact with diverse populations.  These questions were meant to set the tone of thinking about the community factors that support youth development.  Throughout their experience organizing the dinner, they were often asked to think back to the factors that lead to a strong, supportive community and relate what they were doing to those factors. Once the youth group was selected and put together, I attempted to build evaluative thinking and a culture of debriefing into the way each meeting was structured and facilitated.  From the beginning my intent was to have every meeting except for the first be facilitated by members of the youth organizing team.  I provided a general model of how meetings could be facilitated through example.  I developed a template (appendix 2) to help them incorporate into each meeting an ice breaker and debrief.   I encouraged the youth to collect information as it emerged, reflect back on initial goals, take careful records and be responsive to feedback.  By having them facilitate the meetings I was not only introducing concepts of evaluative thinking, but having them apply them directly to the work they were doing. In the second event, the youth decided to build reflection more explicitly into their meetings.  One way they did this was by arranging for a mid-project evaluation half way through their planning process. This included an opportunity for youth to write one thing that was working well and one thing they would like to see improved which were discussed at the following meeting.  They also reviewed the list of skills they hoped to develop in this project that they had identified at the beginning of the project. This reflection allowed them to take stock of what was working well and what could be improved and allowed them to address this in future meetings.  Image 3: Youth Facilitated Planning Meeting 24   Facilitating Reflection and Feedback Sessions  While the way the meetings with youth were designed encouraged reflection and feedback during every meeting, we held a number of specific events aimed at taking stock of the project at key moments. June 2010 Dinner Debrief The first was a debrief with the youth group after their first event.  The goal of this meeting was to reflect on what each individual student learned through the process, what they learned collectively as a group, and where they wanted their initiative to go in the future.  It was also an opportunity to revisit the original goals we had set from the outset of the project and evaluate how well we met them. We started the evening with an activity called the Web.  I brought a ball of string, and one by one participants threw the ball of string while holding on to a piece of it.  With every throw they shared something that they had learnt through their involvement in this project.  At the end, once everyone has shared something, a web had been made and served as a nice metaphor to talk about the collective knowledge that exists in the group and the connections between all of them.  To unravel the web, one by one, each youth shared something they would do differently if we were to do the event again.  Out of this activity a number of points came up.  Many youth commented on the skills they gained in communication, especially with people they did not know.  Many of the youth gained experience in facilitating meetings.  For those who never got the chance to facilitate, all became familiar with elements of an effective meeting.  They learned how to reach out to different members of the community, be it schools or businesses.  They gained a better understanding of the resources in their community and were impressed by the support of local businesses.  As for things that they would do differently, they would cook more vegetarian options and rely less on junk food donations or purchases. There was also recognition that they still had things to learn.  They felt that they could improve their communication within committees and manage their time better.  I then turned the discussion to ‘what  “One of the basic frameworks for evaluation, aimed at simplifying what we do, is summarized by asking three questions: What? So what? Now what?” Gamble, (2009) p. 47 25   kind of resources do you need to take the next steps?’  They identified that support from the partners would be key as well as access to meeting space, funds and youth volunteers and participants through schools.  They all agreed that they would like to be alerted of future events and opportunities to get involved and about half of them identified they would be willing to take a leadership role in future projects. We looked back on the original vision that they had developed initially and agreed that the dinner played out much like they had hoped.  Their goals for the community were to promote family bonding, create fun activities, build inter-family relationships and include people of all ages.   For themselves, they had hoped to get to know new people, improve communication skills and problem solving skills and learn about resources in their community.  An additional positive outcome that emerged through the debrief was the interactions they were able to have with younger youth who attended.  Exposing younger youth to an event that had been planned by older youth provided them with positive role models in their community.  Because many of the youth from the first event went on to take a leadership role in the second event, this learning was important in inspiring the next event’s focus on recruiting middle year youth specifically and engaging even more youth leaders to model this positive role. Shortly after, we had a debrief with a few members of the youth team and the project’s partners.  We reviewed the notes from the youth’s debrief and added our own ideas of what worked well, what could be improved and talked about next steps.  We reviewed the number and makeup of the attendees and discussed the relationships that were made through the organizing of this event.  The debrief with the partners revealed that overall people felt that the first event was a success.  Many suggestions for the next event emerged and the group agreed that they would like to see more events, especially ones that focus on middle year youth, with UBC youth and high school youth continuing to work together. Image 4: Our Community Brainstorm at Orchard Park Community Dinner 26   February 2011 Event Debrief After the February event I hosted three separate debrief sessions: one with the UBC students, one with the entire group of students and another with the project’s partners.  The goals of these debrief sessions were to uncover the outcomes for the individuals involved, the youth group and for the community at large.  It was also an opportunity to discuss our vision for the future and identify barriers to that vision and brainstorm solutions.  For the youth debrief we started the evening with a soft shoe shuffle, a deep democracy technique (Lewis, 2009) aimed at surfacing as many opinions as possible and allowing people to show if they agree with each opinion by moving physically around the room.  From this exercise we learned that most people agreed that planning the event was a lot of fun, that attendees were very receptive to the event and that the youth were keen to plan another event.  While most agreed that they learned a lot from being involved in the process, the opinion also existed that there was more to learn.  Similarly, while most agreed that they had addressed a need in the community, they was a recognition that there is still more work to be done. After warming up with the soft shoe shuffle, we did a brainstorm of outcomes of the project focusing first on individual outcomes, group outcomes and finally on outcomes for the community.  From this brainstorm, a number of things came up.  On an individual level, the youth felt they had gained important skills in public speaking, community outreach, event planning, volunteer management and teamwork.  For those who were involved for a second time, there was a feeling that they were able to take more initiative than in the first event.  The youth also felt that they were benefiting by building stronger relationships with other youth, community partners and local businesses.  As a group, the youth felt that they succeeded in working together and supporting each other.  Areas of future growth included communication skills, time management, planning and organizing, group management and budgeting.  Again for those who were involved for the second time, there was a feeling that as a group they were getting better at working together and that they succeeded in having a youth driven process. As a practitioner, my relative absence during this event’s planning process and the youth’s success at facilitating their own process and event was a powerful reminder that when trying to empower a group, doing less is more. As for outcomes for the community, the youth identified that this event provided parents and community members an opportunity to interact with children in a fun and interactive way.  They created 27   an event that would be in the collective memory of community members.  It exposed community members to youth in positive roles in their community, breaking down stereotypes about youth and providing an example to younger youth.  One area for improvement is that the youth felt they could have been more explicit that this was an event planned for youth by youth.  They could have also been more explicit in their goal of connecting people to others and to resources. We finished by talking about their vision for the future, identifying barriers to accomplishing that vision and things that could help them overcome those barriers.  I used the metaphor of a wave to organize our conversation around their vision for this movement (the wave), what will get in their way (a wall) and what will allow them to break through the barriers (wind propelling the wave).  From this the youth identified their vision of a sustained youth movement that involved more events and youth- led programs.  Their vision included having a strong identity with a vision and home base.  Their movement is sustained by committed people and collaborations with organizations.  They envisioned doing outreach to discover what other initiatives were going on in the community that they could align themselves with.  Barriers they identified to their movement were loss of momentum, a lack of funds and clear leadership, burn-out and lack of a home base.  When asked what could help them address these barriers they identified that communication tools such as website, facebook, twitter could help them keep in touch.  Bringing on new partners could help them learn about what else is already going on in their community.  They also identified the importance of having a coordinator to help convene them and keep the project going. These ideas were presented to the partners a few days later and led to a fruitful discussion about directions that the project could take in the future.  One of the emerging ideas is to shift from an event by event initiative to a sustained youth movement.  Rather than have youth take on individual events that they need to organize from scratch, having youth familiarize themselves with the work already being done in the community and focusing Image 5: Vision, Barriers and Resources Brainstorm at Youth Debrief 28   their efforts on supporting existing initiatives.  Another idea is supporting the youth to continue to meet regularly for teambuilding and skills development between events and initiatives.  Funding for the project was discussed, and the partners became aware that there was about a year or two of United Way funding left, so that there was a need to shift to a completely community run project by the end of the year.  Partners discussed how they could jointly apply for grants to maintain this community initiative beyond the United Way funding. Facilitating Participatory Evaluation  Engaging members of the program or experience you are evaluating is key to developmental evaluation. The process of engagement between the primary intended users and the evaluator is as much the method of developmental evaluation as any particular design, methods and data-collection tools. (Patton, 2011, p.335).  As Stake emphasized (2004), evaluators have much power to shape the results of an evaluation.  Participatory Evaluation is concerned with where that power lies and attempts to engage those most involved or affected by the program. The Development Assistance Committee (2006) defines participatory evaluation (PE) as a method in which representatives of agencies and stakeholders (including beneficiaries) work together in designing, carrying out and interpreting an evaluation.  Cousins and Whitmore (1998) distinguish between two types of PE: Practical Participatory Evaluation and Transformative Participatory Evaluation.  In Practical Participatory Evaluation the central function is to encourage evaluation use with the implicit assumption that evaluation is geared towards program decision making.  The core premise is that participation will “The evaluator is often part of a development team whose members collaborate to conceptualize, design and test new approaches in a long term, ongoing process of continuous development, adaptation and experimentation” (Patton, 2011, p.1). “A participatory approach makes a lot of sense in developmental evaluation because of the need for high trust and quick feedback” (Gamble, 2009, p. 24). 29   enhance the relevance of the evaluation, which in turn will affect ownership and thus utilization. Research indicates that intended users are more likely to use evaluations if they understand and feel ownership of the evaluation process and findings (Patton, 2011, p.15).  Transformative Participatory Evaluation invokes participatory principles in order to democratize social change.  It is a movement that is deeply rooted in community and international development, adult education and the women’s movement.  A fundamental issue underlying this approach is the question of who creates and controls the production of knowledge.  Evaluation is used to transform power relations and foster social change. Despite different ideological roots, both approaches lead to an increased feeling of ownership and control for the participants of the evaluation. For this project, PE is a powerful way to implement this project’s goal of building youth capacity.  From the beginning, I wanted to engage the youth in evaluating their event and process.  Because the overall goal of the project was to build community capacity, in our first meeting we started with an exploration of what the term community capacity meant to them.  Elements that emerged through our brainstorm were that a strong community is one that is self-sufficient, whose citizens are engaged and committed and take initiatives to get things done and have a high quality of social relationships and cohesion. These themes followed us throughout the process and at key times, the youth were asked to evaluate how their event or actions related to their overall goals. Half way through the planning process for the second event, I was invited in to facilitate a session on evaluation. Earlier in their planning process they had identified a number of goals related specifically to the second event.  We started by reviewing those goals.  For each of their goals I asked them to think about “how will we know we have achieved that goal?”  They came up with a number of ways including observations, stories, numbers of people, kinds of people who attended and participation in future programming.  Then I asked them to think about given their resources and time, what kinds of information could they collect and document from the event.  We then assigned different evaluation tasks to individuals.  This simple evaluation framework was circulated and revisited with the group (appendix 4).  The youth’s primary evaluation implementation strategy was to have key individuals responsible, but to have everyone participate in looking out for a number of key things identified as indicators.  As for recording observations, the youth planned to blog their observations and reflections (appendix 5).  The blog continues to provide them with an easy place for them to collect information 30   and one that is easy to share with the public and the project partners.  This technique was inspired by Holte-McKenzie, Forde and Theobald’s (2006) case study on developing a participatory evaluation framework with a group of Kenyan youth.   This case study provided an example of youth developing and narrowing down indicators and choosing appropriate methods to evaluate them.  Facilitating Decision Making  Often I took on the role of asking questions, describing options and presenting information to aid in decision making.  One decision we needed to make for each event was what kind of youth we should recruit to participate.  For the first event, this decision was made in discussion with community partners before recruitment of the high school youth had begun.  For our first dinner there was a desire to reach out and engage youth who are not typically involved in community initiatives, specifically those living in the Orchard Park Housing complex where the dinner would be held.  Based on past literature on youth engagement, I was keenly aware that despite existing skills and knowledge, when engaging youth new to youth organizing, failure to provide any support or leadership skills development opportunities runs the risk of leaving youth feeling inadequate and frustrated (Wheeler & Roach, 2005).  As explored in the context section of this report, it is important to recognize that different engagement methods are necessary for different youth depending on their personal needs and past experiences in leadership (Funders Collaborative on Youth Organizing, 2000).  I shared with the partners the different kinds of experiences we might want to be creating depending on the type of youth present and their level of development.  In this case we were trying to start a youth led movement so it was important that the dinner succeed.  As well, it was important that there were youth champions with the skills and experience necessary to take a lead on further youth-led projects once the dinner was over.  We also wanted to provide youth who are not typically involved in community initiatives an opportunity to get “Evaluation processes include asking evaluative questions, applying evaluation logic and gathering real- time data to inform ongoing decision making and adaptations” (Patton, 2011, p.1). 31   involved, especially for those living in the Orchard Park complex.  For this project the desire to reach out and the desire to work with youth champions led to the decision to involve both kinds of youth and adapt the way the project was facilitated accordingly.  Overall this was successful, and for some of the youth who got involved in this kind of work for the first time, it was a transformative experience. However, one of the things that emerged in post-event interviews with UBC youth and in my own reflections is that we wished there had been more time to provide skills training for some of the high school youth.  Given the relative short planning period of two months, there was a lot of “learning by doing”, but less time for targeted skills training. The decision of what kind of youth to recruit was revisited when it came time to pick the youth for the second dinner.  This time, the desire to involve more high school youth led to a decision by the steering committee and the new coordinator to shift the balance of the team to have less UBC students and more high school youth.  Given this, the UBC youth felt that they lacked the time and capacity to facilitate adequate leadership training.  Their preference was to engage mostly youth who had a proven history of youth engagement to help foster and identify youth champions to carry the movement forward.  The options were explored during a partner meeting, where I helped facilitate a discussion that included the returning students’ reflections, my own observations and partner preferences.  In the end, the UBC students’ observations influenced the decision made by the steering committee to recruit most of the new youth based on their past leadership experience.    This approach succeeded in identifying a number of youth champions who are very motivated to carry on with this movement. However, post-event reflections also brought out the feeling that there is a desire to broaden the net of youth who get involved so as not to create a dependency on those youth who commonly jump into leadership roles and are often over-committed.  To sustain the movement, the youth felt they need to adopt a rotating door policy with youth taking turns in leadership roles to allow new youth to gain skills and prevent burn-out in existing leaders.   As the initiative evolves, the question of what kind of youth to recruit and how to best support the youth will continuously need to be revisited.  The steering committee and the returning youth’s collective experience of the previous decision making processes and their post-event reflections will inform how they approach this question in the future. 32   With the youth, I was continuously attempting to help facilitate decision making by trying to pull out as many opinions and views as possible while trying not to influence the results with my own opinions.  My approach was influenced by the tradition of deep democracy and specifically, the four step model of decision making (Lewis, 2010).  For example, during a planning meeting for the second event we had to decide whether or not the youth wanted to pursue creating a video project around the event.  This would involve filming interviews with youth leading up to the event and making a film about the event.  As the discussion developed, I had an opportunity to lead them through the four steps of decision making (box 2).  I took on the role of feeding back: 1) information I had collected from previous meetings 2) research I had done on the role of video in planning and 3) my observations on the project.  I also took the role of trying to elicit as many ideas as possible from the group before they made a decision.  I tried to do this in such a way that the youth could make the decision for themselves.  In the example I outline in box 2, it was difficult not to influence the decision, because I was very keen for them to take on the video aspect of the project.  However, I also knew that it was up to them to pick an option that made sense for their current time and resources.  Being familiar with the logistics of the option they had before them, the current capacity of the group and the hopes and feedback I had collected to date, I complemented their decision making process with information but ultimately left the decision to them.  The four step model of decision making allowed me to systematically mine for more “data” to help inform the decision. This data being the opinions and thoughts of the youth that had yet to be uncovered.  Part of my role in both situations described in this section was to document decisions that were being made and Box 2: Deep Democracy Four Step Decision Model (Lewis, 2010) In step one; the goal is to gain all of the views.   In the early stages of planning the second event, the idea for a video project was suggested by one of the youth members who felt strongly about the idea.  I suggested that they collect as many views as possible on the white board. At first only ideas of why we should pursue the video option came up. In step 2, the goal is to make it safe to say no.  Since no one was talking about any of the reasons why we might not do the video project, I made reference to an earlier meeting we had with the project’s adult partners who were also keen about a film project.  I reminded them of a conversation that followed that meeting where the youth expressed concerns that they couldn’t use all of the ideas given their current time and resources.  I reminded the group that it was up to the youth to choose a project that made sense to them and worked on their timeline.  This was my attempt to make it safe to say no without expressing my own opinion about the project. Perhaps the doubts as to whether this project was feasible would have arisen anyways, but in this case, they did arise quite soon after my reference to this meeting.  One of the youth brought up the issue of ethics approval and permission forms. In step 3, the goal is to spread the no.  The concern about ethics opened the door for me to ask “are there any other concerns about the film option”?  It turns out there were many, ranging from the lack of editing skills in the group, the question as to whether we would be allowed to film in schools, the question as to whether this project was possible within the time frame that we had and whether it would distract them from focusing on the event and event planning process.  The conversation made it clear that there was a lot of information missing in order to make a decision about whether or not the film option was feasible. Finally, in step 4, the goal is to understand what those who are not with the majority need to go along with the group’s decision.  In the end, the youth decided to continue planning their event, but not to build it around making a film.  For the one youth who felt very strongly about the film idea I asked:  what do you need to go along?  Her main interest was that the video remains an option as we learn more about the project.   The youth decided to appoint someone to look into the logistics of the option.  In the end, the youth did not find the resources to film the entire process, but are keen to use this resource in future projects.  33   circulate them through minutes and reporting so that there was always a record to go back to in trying to understand how decisions were made and why.  These become important in creating a collective story of how the program evolved over time and adapted to emerging information. Applying an Evaluation Framework  In telling the story of how the project evolved, another one of my tasks was to identify and communicate outcomes and organize information as it related to our overall goals.   One of the tools introduced by Patton is the Dynamical Actual-Ideal Comparative Evaluation Framework (Figure 3).  This is recommended as a tool to compare actual results to hoped-for ideals with updates as new data and understandings emerge throughout the change process (Patton, 2011, p. 257). A logic model is a similar and widely used evaluation tool.  A logic model breaks down evaluation into steps involving inputs, activities, outputs, outcomes and impacts (Kellogg Foundation, 2004).   The main difference between Patton’s model and traditional logic models is that the act of updating the baseline condition and the ideal goals of the project are built into the framework (figure 3).    This is what Gamble (2009) describes as “revised and emergent modeling” which we expect to change and evolve over time (p.49). “In a developmental mode, we move from a logic model as a static instrument, to one that we expect to change and evolve over time” (Gamble, 2009, p.49) 34     In collecting and organizing information, I told the story of the project’s outcomes first through written reports and eventually through a table that summarized our goals, activities, results and lessons learned (appendix 3).  This table was designed to be a living document meant to change over time based on new information and youth and partner feedback.   The format of a table was adopted for the ease with which it could condense a lot of information in one or two pages which could be emailed or printed off and used as a discussion point during meetings. Figure 4: Dynamical Actual-Ideal Comparative Evaluation Framework (Patton, 2010) Update and adapt the vision and ideals in the face of emergent realities. Where should we be?  Ideals vision and hopes BASELINE: Where did we begin? IDEAL Where are we now? How far have we come? What have we learned along the way? Dynamic and retrospective baseline: Update the baseline as new understandings of the beginning situation emerge. 35   The first column expresses the goals at the onset of each event addressing the question “where should we be?”  This column was populated by the results of youth visioning sessions that happened in the planning stages of each event.  The columns titled Did we achieve our goal? and Lessons Learnt were populated by information gathered during event debriefs with youth and partners and meant to address the questions “how far have we come?” and “what have we learned along the way?”  The How do we know? column was meant to draw attention to how we were learning along the way. The table was sent out to partners at key moments for feedback and became a center piece for discussion at some of our meetings.  Based on feedback through email or during meetings, the table was adapted and then re-circulated for comments.    Other inputs came from the youth planning group.  For example, the goals of the events came from youth visioning sessions.  Similarly, when the youth designed their own evaluation framework (appendix 4), their inputs populated the How do we know? column. The baseline, “where did we begin?” is captured with the underlying assumptions that are stated at the top of the document.  Our baseline was informed by past research (Hertzmann et al., 2010; VMCMI, 2010), recommendations that came out of the VMCMI report and current partners perceptions of the issue at hand.  The baseline was continuously being enhanced by data being collected throughout the process.  For example, one of our baseline assumptions was that lack of information and relationships are barriers for some community members in accessing available programming.  During the first event, I had the opportunity to discuss why the event was important with a community program leader who was responsible for bringing ten adults and ten children to the dinner.  She commented that her clients, many of whom did not speak English, were delighted to be involved.  She mentioned that this dinner was just the kind of event that was needed to get parents talking to other parents and connecting with other parts of the neighbourhood, and this need was particularly acute for immigrant families.  This story was recorded and reported back to the steering committee and youth.  This story played a role in strengthening the initial baseline analysis and confirming that we were on track with the current goals of the initiative.  The broad vision of the initiative is captured by the overarching goal stated at the top of the document. This goal has been revisited and refined throughout my involvement with the project.  One emerging 36   result of the youth’s work was the relationships that are being built between members of the community.  This came up repeatedly during reflections and debriefs and I highlighted this in my reports.  For example, one of the youth took the initiative to connect with the local Member of Parliament who attended the event and gave a speech congratulating the youth on their initiative. Another youth took the initiative to connect with the Victoria Business Improvement Society, which gave him access to a number of local businesses, many of which donated to the event.  One of our community partners was able to advertise their programs and sign up a number of people on the spot. In the second event, the youth identified in their evaluation framework that they wanted to look out for instances of connections being made during their event.  These observations were noted on their blog and focused on during the project debrief.  These results in conjunction with conversations with the partners caused us to revisit the overall goal of the project.  It caused us to specify that in our initial goal of building community capacity, more specifically, this project was targeting one aspect of community capacity, that of building relationships in the community. At a recent meeting, when the issue of future funding of the project came up, I was reminded that one of the overarching goals of the project is that this be a community-driven sustainable initiative.  With this in mind, I proposed that we add to our overarching goal a word to emphasize the need to build a sustainable community-driven movement, one that lasts beyond UBC’s and the United Way’s support.  While the table format succeeded in being easy to populate and disseminate, upon reflection I could have chosen a communication tool that better demonstrated the iterative nature of the process.  Although my participation in this project is coming to a close, my hope is that the initiative framework will continue to be revisited and reshaped as the project evolves through further explorations of where are we now?; how far have we come?; and what have we learned along the way?    Image 6: Serving Food at the Orchard Park Dinner 37     Data Collection  One of the keys to developmental evaluation as identified by Patton (2011) is to provide instant feedback.  Speed matters, as adapting to emergent conditions requires getting information in a timely fashion.  Throughout the process I was keeping a journal that recorded observations and decisions being made and noting anything that was emerging, whether planned for or not.  I also encouraged the youth to create a blog to record their observations and reflections and easily share it with project partners (appendix 5).  In addition, at key moments in the project when I was curious to know more, I went after specific data.  The focus of my data collection surrounded three topics: 1) the outcomes of this project for the youth and the youth group’s capacity; 2) the relationships that were being made in the community; and 3) the impact these events were having on community members.  For the youth group, I often had the opportunity to collect data through regular reflection or at specific debrief events; however I also conducted a number of one-on-one informal interviews with the youth as the project progressed to check-in on what they were learning and how they felt about the project.  As for the last two points, specifically for their second event, armed with the youth’s evaluation framework (appendix 4) I deliberately circulated on the day of the event and through casual conversation with community members asked them to comment on what they thought about this event, what this event meant to them, what the benefits of these kinds of events were and asked if they were making any connections in the community.  Having someone intentionally looking for this information and reporting it back allowed for some instant feedback for the youth and the steering committee as to how members of the community perceived their event.  At this stage of the project, if I had undertaken a full-on academic research study, the community partners would still be waiting to see the data.  Although there is an  “Developmental evaluation tracks and attempts to make sense of what emerges under conditions of complexity, documenting and interpreting the dynamics, interactions and interdependencies”.  (Patton, 2011, p. 7)  38   important role for academic research, and it may play an important role at later stages of this project, at this stage, having instant feedback as to the immediate outcomes of the project allows the community partners to remain responsive to what is emerging and plan the next steps of this evolving project accordingly.  Informal data such as conversations, observations and stories have provided this project with important clues as to the impacts of the project at a micro level.  Over time, the frequency of particular observations has suggested trends which have informed the macro level overall goals of the project.  Feeding Back Information  Stake highlights that all research depends on interpretation and that the ongoing interpretive role of the researcher is central in qualitative case studies.  Stake describes the role of the researcher as interpreter and artist, as an agent of new interpretation, new knowledge and new illusion.  This highlights the important responsibility the developmental evaluator has to be conscious of how their past experiences shape their interpretation of the case at hand.  Key to remaining responsive to the community is to continuously check back for understanding.  I was constantly feeding back information through minutes, debrief summaries and reports, and asking partners for feedback.  One instance where this checking back led to a revisiting of the way I had interpreted the situation was during a meeting in which we revisited the overall goals of the project in crafting the project’s framework (appendix 3). I mentioned in passing, assuming I could take for granted, that the overall goal of this project was to build community capacity.  I was surprised by a strong reaction from some members of the steering committee that this did not accurately describe our overall goal.  Reeling for a moment, I then asked questions to uncover how the partners did see the goal of the project.  What I discovered was a strong “The function of research is not necessarily to map and conquer the world but to sophisticate the beholding of it *with+ ‘thick description’, ‘experiential understanding’ and ‘multiple realities.’”  (Stake, 1995). 39   reaction to the term “community capacity” as ill-defined and overused and too broad to describe our current work.  It is also a term that is difficult to talk about in isolation.  As Dale and Newman (2010) highlight, social capital is a necessary condition for sustainable community development; however, it is not always sufficient to sustain and develop local community initiatives without infusions of economic and human resources.  On the ground, this is felt acutely by the partners and perhaps contributes to a hesitation to describe their work in this way.    Community capacity is also a phenomenon that requires many inputs:  relationships, resources, commitment, ability to solve problems, opportunity and leadership (Chaskin,2001; Dale and Newman, 2010).  In the short time that this project was been running we have not been able to address all of the elements of community capacity.  Rather, we are working towards elements of it, and perhaps in order to better evaluate what it is we are doing we need to take a step back and focus specifically on an aspect of community capacity, such as building relationships. This conversation caused us collectively to revisit the baseline situation and the ideal situation (figure 1) and adjust it to better reflect the reality based on emergent information.  What came out of this conversation is that building relationships and creating opportunities better defined our work than building community capacity. Before making my recommendations in this paper, I organized information collected through debriefs of the second event into a SWOT analysis (appendix 6).  This analysis allowed me to reflect the information I was hearing from the youth and the partners and get reactions.  Those who responded agreed that the strengths of this project laid in the collaboration between community partners and youth and the motivation and skills the youth had to take on projects.  The weaknesses identified include a lack of clear identity and purpose.  Opportunities include more events, new partners, stronger relationships, fundraising and skills training.  Threats include loss of momentum, leadership, human resources and funding.  The SWOT provided me with an informed starting point from which to make recommendations that I felt reflected the participants concerns and hopes for the future of this project.    40   Mobilizing Assets  My approach was to intentionally work with the youth in the community to support their assets and build their capacity to continue community capacity building in the future.  From the beginning, the youth’s role in this project was described as the role of community experts.  Having grown up in Victoria/Fraserview, they are the best equipped to examine the challenges and success factors of how the community supports youth.  As described in a previous section (p. 27), one of the first things we did during the first meeting with the UBC students is develop a common definition of what community capacity meant to them.  They identified that community capacity could be described as self-sufficient, committed, having strong relationships and an engaged public.  We then compared their definition to some more academic definitions of community capacity (Aspen Institute, 1996; Chaskin, 2001) and agreed that there was a lot of overlap. I then facilitated a brainstorming session with the youth as to how we could conduct our meetings to ensure the maximum opportunity for the high school youth to build and use their own leadership capacities.  From the beginning, the tone was set that this project was not just about organizing a dinner, but also about building capacity in youth to be effective community capacity builders.  The youth identified that in working with the high school youth it will be important to provide them with “Each community boasts a unique combination of assets upon which to build its future”  Kretzmann and McKnight, 1993, p. 6. Figure 5: Youth brainstorm: What is Community Capacity? 41   opportunities to take leadership roles, to actively mentor leadership skills, to build on existing strengths and to encourage them to take initiative.  It was decided that the facilitator role would be rotated between the youth and that the high school youth would be encouraged to share other roles within the group whether it was taking minutes, keep the meeting on time, collecting ideas for future discussion or arranging the food for the meeting. At the end of our first session we collectively decided on the goals of the project and the topics and activities for our first meeting with the high school youth.  Subsequently, two of the UBC youth took on the role of designing and facilitating the meeting.  From our earlier brainstorming sessions they came to express the goals of the project in their own words.  At this session, the UBC youth presented the project to the high school youth as a youth-led community-building initiative with the goals of bringing youth together and building community capacity through creating opportunities for networking, resource sharing and trust building.  They started their first meeting with an interactive brainstorm on the question:  “What makes a good community?” Each group presented their ideas back to the larger group.  Ideas that emerged included: sharing, celebration, interconnectedness and helping each other out. The project was described to the high school youth as having the following benefits: building leadership skills, work experience, meeting new people, fighting negative stereotypes about youth, drawing on your assets and creativity and inspiring others to get involved in their community.  They then turned it back to the youth and acknowledged that although there is much to be gained, there are a lot of existing assets in the room.  The group then worked together on a group resume which became a collective expression of all of the skills, knowledge and experience that existed in the room. My approach in encouraging the youth to define their definition of community capacity, the goals of the project and the existing assets was in response to one of the barriers to youth engagement identified by Checkoway (1998).  Specifically, the assumption that adults perceive youth as something to protect, not to involve, and that adults typically try to educate youth about community development rather than including them in community development processes.  One of the factors that I believe led to the project’s success was our ability to break down the assumption that youth don’t already know something about community development.  There was a conscious attempt to avoid delivering programming to teach the youth about community development, and instead build on youth’s existing 42   understanding of the topic.  From the first meetings, activities deliberately drew on the youth’s existing perceptions of what a ’strong’ community looked like and what factors led to a community’s success. Decision making power was shared with the youth in most aspects of the project, which is one of the factors the literature identifies as crucial to successful youth engagement (Gurstein et al. 2003).  When attending meetings with the adult partners, the youth were treated like equals and their input as youth from the target community was highly valued and influenced final decisions.   Often when discussing an important issue at the community partner level, the partners would turn it back to the youth as those best equipped to answer the question.  For example when asked how will we measure the success of our second event, the partners identified that the youth were best equipped, based on their own goals, to pick the indicators they think best describe their goals.  At key moments when decisions needed to be made about next steps, partners ensured that youth were present to craft the direction of the project. For example, at my last meeting with partners where only one youth was present, they recognized the need to delay any decision making until they had a chance to discuss with the larger youth group. At a recent working group meeting planning the next stages of the project, a bi-level goal of this initiative emerged.  For the partners, there is a goal of building the youth’s capacity and leadership skills as a way to increase the community’s capacity.  For the youth, their primary goal is to continue being a part of a group that is working towards a common goal.  The results of their work are events that are bringing community members together.  As the youth continue on to plan their next event, the partners are engaging in reaching out to other community members who could be involved in supporting the youth’s work in the future.  Thus, there are two parallel mutually enforcing processes unfolding, the youth’s work in building their capacity and bringing community members together, and the partner’s work in mobilizing assets that will support the youth’s efforts. 43   Recommendations for the Engaging Neighbourhood Initiative This project will continue far beyond my involvement as a graduate student.  At this stage of the project and given my role over the last 15 months, I propose two sets of recommendations.  The first has to do with evaluation based on my experience in the role of a developmental evaluator.  The second set reflect back many of these recommendations  that have come from the community members themselves through discussions and the SWOT analysis, but I have collected them here as a way of documenting the current status and direction of the project. Further Evaluation Recommendations:  Given the breadth of the project’s goals, there is potential to lose sight of the Initiative’s overall objective.  The Initiative framework (appendix 3) is a tool that should be updated and adapted as baseline and goals shift with emerging information.  The youth have been introduced to evaluative thinking through the process and through the use of participatory evaluation.  I recommend that the youth continue to emphasize evaluation and reflection in their work and encourage them to design and use their own evaluation frameworks with each new initiative.    Further workshops on evaluation and reflection throughout the process could further enhance the work the youth are already doing.  Identify and address barriers to taking on continued evaluation.  Once the committee has decided which direction this initiative is going next, I recommend identifying the committee’s evaluative needs, identify barriers to meeting those needs, and building into the plan time and resources for continued developmental evaluation.  Although I believe that a developmental evaluation approach to this project was appropriate, in the year that I have been involved, the real complexities of how this project is affecting the community have not fully had the opportunity to emerge.  I recommend that the steering committee continue to collect and document information on the outcomes of this project and adapt the project accordingly. Future of the Initiative Recommendations:  Co-leadership is identified by Gurstein et al (2003) as integral to successful youth engagement. The collaboration between the steering committee and the youth is a strength of this initiative. Ensure constant and consistent communication between youth and adult partners through regular meetings.  A planning session that involves the youth from both stages, past applicants, existing partners and potential new partners should occur to collectively map out the next steps in the project. 44    Regardless of the specifics of the future of the project, rather than focusing our energies event by event, an effort should be made to sustain an ongoing network of youth who work with partners to identify future projects they would like to take on.  To ensure a consistent supply of human resources, adopt a rotating door approach to the youth group, rotating leadership positions and continuously opening the door to new members on both the youth team and the steering committee.   To ensure continued youth capacity building, the group of youth should identify areas where they would still like to develop skills and that part of the remaining funding be used to secure facilitators.  Wheeler and Roach (2005) identify problem solving, communication and specific tools to take action as crucial skills to support youth leadership.   As a next step, focus on identifying the assets in the community that could help sustain the project beyond funders involvement which could help guide the use of current resources to support those assets. 45    Limits of Developmental Evaluation Before describing the ways in which developmental evaluation can play an important role in planning, this section explores some of the limits and challenges I have come across in the literature and in practice. Human Resources The first has to do with human resources, not only to carry out evaluation, but to engage in the evaluation process.  Patton (2010) discusses the importance of the personal factor in developmental evaluation which he describes as the leadership, interest, enthusiasm, determination, commitment, assertiveness and caring of specific, individual people (p. 56).  He cites research that states that nothing makes a larger difference in the use of evaluations that the personal factor which includes both the interest of officials in learning from evaluation and the desire of the evaluator to get attention for what they know (p.56).  Patton goes on to say that project leadership and support for doing developmental evaluation is a “sine qua non” (without which there is nothing) (Patton, 2010, p. 75).  Developmental evaluation can be a long-term process that does not have the same concrete start and end points as a more traditional evaluation which has important resource implications for organizations and their funders (Gamble, 2009, p. 56). My approach has been one of cultivating a spirit of evaluation and building skills to have youth and committee members take on this role beyond my involvement in the project.  One limitation of my efforts to do so is that all of our partners are working on this project off the side of their desks, and it is unlikely anyone will have the time to focus exclusively on developmental evaluation the way I was able to for my graduating project.  I definitely noticed that at times I would get very little response when sending information out for comments.  This is not because of a lack of interest in the project, but a lack of time and resources and perhaps a failure on my part to feed information out in bite size packages at opportune moments.  The absence of this feedback loop gives the developmental evaluator too much power.  While “evaluators have traditionally been admonished to remain external, independent and objective”… “complexity-based developmental evaluation recognizes that data collection is a form of action and intervention, that the act of observation changes what is observed and that the observer can 46   never really remain outside of and external to what he observes” (Westley et al. 2007, p. 239). Therefore, not only does this approach require a strong leader willing to take on the challenge, but also other committee members need to have the time and be committed to participating in evaluative conversations. Accountability In reporting out on the progress of a project, how does developmental evaluation stack up to more traditional evaluation methods?  One important point to remember is that developmental evaluation does not rely on a single best method, but rather draws on whatever method is best for the task at hand.  There is no reason to believe that within a developmental evaluation process, rigorous quantitative data could not be produced (Gamble, 2009, p. 22).    Westley et al. (2007) describe the challenge of accountability in the following way.  If accountability is traditionally focused on attaining prescribed results, learning becomes nothing more than learning about how best to attain the desired outcome.  Failure as defined in the traditional evaluation becomes reframed as learning within developmental evaluation.  “The only real failure is the failure to learn” (p.240).  From this perspective accountability shifts from compliance to a prescribed goal, to compliance to learning that informs future action.  “The accountability of developmental evaluation rests in its ability to support development.  If nothing is developed, it has failed” (Gamble, 2009, p. 24).  Paired with a number of more traditional evaluation tools, there is no reason that a developmental evaluation approach could not provide rigorous evidence-based data to interested parties. Another threat to developmental evaluation’s legitimacy is that the evaluator has a much closer relationship to what is being evaluated than traditional evaluators.  While this is strength in being able to create a collaborative understanding of the situation at hand and allow for responsive action, this could be perceived as a weakness if there is a need for summative evaluation for accountability.  Both of these issues are important reminders that developmental evaluation is not appropriate for all situations or stages in a project’s development.  There will be times where it might be more appropriate to use summative evaluation to make judgements about a program’s future (Gamble, 2009, p.15).  Similarly, there may be times when it will be more appropriate to bring in an external evaluator to insure independence and objectivity. 47   What’s in a name? Another barrier to developmental evaluation lies in the name of the process.  The word evaluation often conjures up negativity associated with traditional evaluation methods aimed at accountability rather than learning.  In fact, in this project, I started by naming my role on the committee as a developmental evaluator, but then found it was a title that caused a bit of uneasiness and confusion about my role. Eventually, I found myself better able to describe my role by excluding the word evaluation from my title.  Instead I focused on my role in documenting the process and facilitating reflection.  When trying to support a spirit of collaborative learning, the word evaluation conjures fears that do not fit the intention.  The word evaluation tends to focus attention on an end product, such as an assessment of worth.  While developmental evaluation is focused on getting results, it is better described as a process rather than an end product.  Similar terms that might help overcome the baggage associated with the word evaluation include adaptive management, responsive planning or experiential learning. The Role of Developmental Evaluation in Planning Having dealt with some of the limits of developmental evaluation, this next section focuses on the potential developmental evaluation has in planning as a resilience and capacity building tool. Developmental Evaluation as a Resilience Tool An evaluation process that embraces and adapts to change makes it an important tool in dealing with complexity and building resilience.   The literature on complexity emphasizes the need for adaptive, dynamic and flexible design and collective knowledge and decision making (Brown et al. 2010; Walter and Scott, 2006; Innes and Booher, 2010). Walter and SaIt (2006) identify that embracing change is at the heart of resilience: “resilience thinking presents an approach…that embraces human and natural systems as complex systems continually adapting through cycles of change” (p.10).  Developmental evaluation is an approach designed to change as information emerges.  In this approach methods can be flexible and designs can be dynamic. Patton (2011) identifies that developmental evaluators need to be agile, open interactive, flexible, observant and have a high tolerance for ambiguity.  These are all skills necessary for embracing change and adapting in times of uncertainty.  Specific to the field of planning, Du Plessis (2009) stresses the 48   need for planning to change from a “prescriptive activity to a process of reflection and adaptation” (p.6) thus highlighting the need for tools such as developmental evaluation. Faced with complexity a number of authors have described the need to access different types of knowledge for collective decision making.  Developmental evaluation’s emphasis on collaboration does not favour the expert knowledge of one external evaluator, but rather attempts to unearth as many perspectives as possible from those most directly affected by the project in question.  This is in line with Brown et al.’s (2010) call for researchers to recognise multiple knowledge cultures and accept the inevitability of uncertainty.  It also complements Innes and Booher’s (2010) call for dialogue in the face of a loss of legitimacy of “scientifically” developed knowledge (p. 4).  Du Plessis (2009) states that sustainability science as a new paradigm for planning must emphasize learning, adaptation and reflection and acknowledge multiple participants and epistemologies to co-produce knowledge. Sandercock (2003) highlights the need to acknowledge the many other ways of knowing.   This epistemology of multiplicity calls for six different ways of knowing: knowing through dialogue; from experience; through seeking out local knowledge; through learning to read symbolic non-verbal evidence; through contemplation and through action-planning.  Developmental evaluation relies on dialogue, learning through doing and reflection and thus in its approach attempts to unearth the kind of knowledge that is often left out in more technical and pragmatic evaluation methods.  These kinds of knowledge have been identified as crucial to responding to complexity and building resilience. Developmental Evaluation as a Capacity Building Tool Returning to our original definition of community capacity, it involves a combination of a community’s skills, resources, commitment and sense of community.  Chaskin (2001) outlines four strategies that community-building efforts tend to focus on: leadership development, organisational development, community organizing and fostering collaborative relations among organisations.  Based on these factors the following section describes how developmental evaluation provides a powerful tool to strengthen individual and organizational skills, and in its approach build commitment, sense of community and collaborative relations. In the last section I identified that lack of human resources can be a challenge to using a developmental evaluation approach; however it is important to emphasize that the developmental evaluation approach should be doing more than generating findings, but simultaneously  developing individual’s and an 49   organization’s capacity for evaluative thinking.  One aspect of organisational development is a group’s ability to solve problems.  “I see a community’s capacity as its own ability to take charge of and make decisions about what happens in the life of neighbours and residents in a community” (Chaskin, 2001, p. 297).   Through reflection, planning, and adaptive feedback, communities become better decision makers and problem solvers.  The combination of different perspectives from both youth and adult partners, and from different organizations at the table creates opportunities for new knowledge and new approaches which are crucial to fostering the innovation required to approach complex problems.  The collaborative approach has the potential to build commitment and a sense of community through increased communication and trust building.  Collaboratively making decisions, trying new things, reflecting and adapting creates a shared experience of problem solving that builds trust between individuals that future problems can be tackled.  It also strengthens ties between organisations and between adults and youth that will serve them in addressing future challenges. Dale and Newman (2010) warn that social capital is not a sufficient condition for sustainable community development.  The success of networks, over time, appears to be dependent upon external enabling conditions outside the resources of the community.  While this project has been building social capital, it remains vulnerable to conditions outside the resources of the community partners.  Changes in policies in child care, education, welfare assistance, immigrant services are just a few external factors that will affect the quality of life for youth in the community.    However, while social capital is not a sufficient condition for community development, it is most definitely a crucial one.  As Sarkissian et al (2009) state “we need social capital development because changes of the magnitude necessary for sustainable development require collective mobilization of people in communities worldwide” (p. 170).  Despite the limits to focusing solely on community capacity, community members and planners need tools that help build community capacity if we are to be successful in addressing the complexity of the problems we collectively face.  As such, planners should be equipped to use developmental evaluation approaches to working with communities.  50   Conclusion In the true spirit of developmental evaluation, I finish this paper by revisiting my initial objectives for this project.  In facilitating a community capacity building process I believe my involvement has been helpful in organizing information, helping articulate priorities and concerns and facilitating decision making. Although I have been involved for about fifteen months, the work of building community capacity is beyond the scope of what was accomplished during my involvement.  It is an ongoing process. However, I do believe that during this time important relationships are being built between organizations and between organizations and youth and that the youth as a group are gaining important organizational skills which are both crucial components of community capacity (Chaskin, 2001). I believe I have built capacity in the community’s youth not only to plan and organize events, but to work collectively as a group to solve problems and to evaluate their progress as they go.  The group of youth has reported learning skills in outreach, communication, organizing, facilitation and problem solving.  What started as a process that was structured and run by an outside facilitator has evolved into a process that is youth run.  There is still more work that I believe can be done in this domain and the youth themselves have identified many skills that they would still like to learn, which is why I recommend a structure moving forward with continued leadership development built in. This paper does provide an example of how developmental evaluation can be applied to complex planning challenges.  The field of evaluation could use a number of case studies of developmental evaluation in practice and I believe there is much potential for developmental evaluation to be applied to a number of planning contexts.  Further research judging the appropriateness of developmental evaluation in different cases would be very interesting; especially relevant could be cases that examine the role of developmental evaluation in different cultural contexts. Developmental evaluation has an important role to play in the field of planning and perhaps it is already playing that role although not always named as such.  Its approach is consistent with existing planning models that call for a planning practice that is participatory, iterative, adaptive and responsive to the needs of community members.  Developmental evaluation provides us with a powerful tool in the search for new approaches to solving problems and engaging communities.  Community capacity is one of many elusive terms we’ve grappled with as planning students and that we will continue to grapple 51   with in our careers.  Similarly nebulous terms include sustainability, justice, resilience, liveability, inclusivity, diversity.  These are all examples of complex phenomena that planners seek to understand and work with.  As I dive into a world of complexity, I am very grateful to have experienced working with this approach and I eagerly store it in my planning tool box, anticipating that I will be accessing this tool regularly throughout my practice. 52    References:  Arnstein, S. (1969). A Ladder of Citizen Participation. Journal of the American Institute of Planners, 35(4):216-224. Armstrong, Ruth (2003) Participatory Action Research: A guide for community-based organizations. Vision Management Services. Retrieved February 24, 2010 from: Aspen Institute/Rural Economic Policy Program. (1996) Measuring Community Capacity Building. Version 3/96.  Atkinson, R and Willis, P.  (2005) Community Capacity Building- A Practical Guide. Housing and Community Research Unit, University of Tasmania.  Baum, F., MacDougall, C., Smith, D. (2006) Participatory Action Research. Epidemiol Community Health. 60: 854-857. Block, P. (2008)  Community: The Structure of Belonging.  Berrett-Koehler Publications, Inc.  San Francisco.  Bond, S and Thompson-Fawcett, M (2007) Public Participation and New Urbanism: A Conflicting Agenda?  Planning Theory and Practice, Vol. 8 (4), 449-472.  Boothroyd, P.  (1991).  Developing Community Planning Skills: Applications of a Seven- Step Model.  CHS Research Bulletin, February.  Retrieved from  Brown, V., Harris, J. and Russell, J. (2010) Tackling Wicked Problems: Through the Transdisciplinary Imagination.  Earthscan, London.  Chaskin, Robert (2001) Building Community Capacity: A Definitional Framework and Case Studies from a Comprehensive Community Initiative. Urban Affairs Review 36:291.  Checkoway, B. Pothukuchi, K. and Finn, J. (1995) Youth Participation in Community Planning: What are the Benefits?  Journal of Planning Education and Research 14: 134-139.  City of Vancouver (2006) Community Web Pages: Victoria/Fraserview.  Retrieved April 14th 2010 at  53   City of Vancouver (nd) Community Profiles: Victoria/Fraserview. Retrieved December 21, 2010 at  City Spaces (2005) Victoria-Fraserview/Killarney and West Fraserlands: Review and inventory of community facilities.  Accessed December 21, 2010 at: wReport.pdf.  Cousins, B. and Whitmore, E. (1998) Framing Participatory Evaluation. New Directions for Evaluation, no 80, Winter 1998, Jossey-Bass Publishers. Driskell, D. (2002) Creating Better Cities with Children and Youth: A Manual for Participation. Earthscan Publication, Ltd. And UNESCO Publishing.  Dale, A. and Newman, C. (2005) Sustainable development, education and literacy, Journal of Sustainability in Higher Education, 6 (4), 351-362 as cited in Dale, A. and Newman, C. (2010) Social Capital: a necessary and sufficient condition for sustainable community development?  Community Development Journal 45 (1).  Dale, A. and Newman, C. (2010) Social Capital: a necessary and sufficient condition for sustainable community development?  Community Development Journal 45 (1).  Development Assistance Committee (2006) Glossary of Key Terms in Evaluation and Results Based Management.  DAC Network on Development Evaluation.  Retrieved July 15th 2010 at: Du Plessis, C. (2009). Urban sustainability science as a new paradigm for planning. Smart and Sustainable Built Environments (SASBE 09) Conference, Delft, The Netherlands, 14-19 June 2009, pp 8.  Frank, K. (2006) The Potential of Youth Participation in Planning.  Journal of Planning Literature, 20 (4).  Forester, J. (2001) The Deliberative Practitioner: Encouraging Participatory Planning Processes (London, MIT press).  Funders Collaborative on Youth Organizing (2000) An Emerging Model for Working with Youth: Community Organizing + Youth Development= Youth Organizing.  Occasional Papers Series on Youth Organizing, No 1.  Retrieved August 20th 2010  Flyvbjerg, B. (2006) Five Misunderstandings About Case-Study Research. Qualitative Inquiry 2006; 12; 219 Gamble, J.A.A. (2009) Developmental Evaluation Primer. J.W. McConnell Family Foundation.  Retrieved March 4, 2010 at 54 aluation%20Primer%20-%20EN.pdf  Gerring, J (2004) What Is a Case Study and What Is It Good for?  American Political Science Review, Vol. 98, No. 2 May 2004. Gillett, S. (2010) A Neighbourhood Based Youth Engagement Practice Story. Relational Child and Youth Practice Winter2010, Vol. 23 Issue 4, p53-62. Graham, S. (2009) Basic Chemistry for Social Change: A Conception of Civil Society Capacity. SPARC. Gurstein, P. Lovato, P.C. and Ross, S. (2003). Youth Participation in planning: strategies for social action. Canadian Journal of Urban Research, 12 (2), 249-274.  Hart, R. (1997) Children's Participation: The Theory and Practice of Involving Young Citizens In Community Development And Environmental Care.  Unicef. Healey, P. (1997) Collaborative Planning: Shaping Places in Fragmented Societies.  Basingstoke, Macmillan.  Hertzman, C. Schonert-Reichl, K. Hymel, S. Calbick, J. Marriott, D. Pedrini, L. Guhn, M. Sweiss, L. Gadermann, A. Buote, D. (2010) Our Children’s Voices: The Middle Years Development Instrument. Measuring the Developmental Health and Well-Being of Children in Middle Childhood.  Holte-McKenzie, M.  Forde, S. and Theobald, S. (2006) Development of a participatory monitoring and evaluation strategy.  Evaluation and Program Planning.  29 (2006) 365–376. Innes, J. (1996) Information in communicative planning.  Journal of the American Planning Association, 66, pp. 52-62.  Innes, J. and Booher, D. (2010) Planning with Complexity: An introduction to collaborative rationality for public policy.  Routledge, New York.  Kay, J. and E. Schneider,(1994). Embracing Complexity: The Challenge of the Ecosystem Approach. Alternatives 20:3:32-39.  Kellogg Foundation, (2004) Logic Model Development Guide, Michigan, January 2004. Kindon, Sara. “Participatory Action Research.” Qualitative Research Methods in Human Geography. Ed. Iain Hay. Victoria, Australia: Oxford University Press, 2005. 207- 220. Kretzman, J. and McKnight, J. (1993) Building Communities from the Inside Out: A path toward finding and mobilizing community assets.  Center for Urban Affairs and Policy Research, Northwestern University, Evanston, IL. 55    Kwan, B. Frankish, J. Quantz, D. and Flores, J.  (2003) A Synthesis Paper on the Conceptualization and Measurement of Community Capacity.  Institute of Health Promotion Research, University of British Columbia, September 2003 Lewis, Myrna (2010) Deep Democracy Foundation Course Delegate Manual.  Retrieved October 9th 2010 at:  Magis, K. (2010) Community Resilience: An Indicator of Social Sustainability. Society and Natural Resources, 23 (5) 401-416.  Mark, M. Henry, G and Julnes, G.  (2000) Evaluation: An integrated framework for understanding, guiding and improving public and Non-profit policies and programs.  Jossey-Bass, San Francisco. Musqueum (nd) Culture.  Retrieved February 7, 2011 at Neale, S. Littlejohns, L.B. Roy, D.  (2003) Measuring Community Capacity: State of the Field Review and Recommendations for Future Research.  Health Policy Research Program, Health Canada, May 2003. Pagels, H.R.(1988)  The Dreams of Reason as quoted in Patton, M (2011) Developmental Evaluation, Applying Complexity Concepts to Enhance Innovation and Use. The Guilford Press, New York.  Patton, M (2011) Developmental Evaluation, Applying Complexity Concepts to Enhance Innovation and Use. The Guilford Press, New York.  Putnam, Robert. (2000)  Bowling Alone.  Simon & Schuster, New York.  Reordon Kenneth (2003) “Ceola’s Vision, Our Blessing: The Story of an Evolving Community-University Partnership in East St. Louis” in B. Eckstein & J. Throgmorton (eds) Story and Sustainability MIT Press, Cambridge.  Sandercock, Leonie (2003) “Who Knows? Exploring Planning’s Knowledges” Chapter 3, Cosmopolis II: Mongrel Cities in the 21st Century.  Continuum, London.  Sarkissian, Wendy. Hofer, Nancy. Shore, Yollana. Vajda, Steph. and Wilkinson, Cathy.  (2009) Kitchen Table Sustainability: Practical Recipes for Community Engagement with Sustainability.  Earthscan, London. Stake, R. (1995). The Art of Case Study Research, Sage Publications, Thousand Oaks  Stake, R. (2004).  Standards-Based & Responsive Evaluation, Sage Publications, Thousand Oaks  Sustainable Region Initiative Social Action Team (2007) A Resource Guide on Social Sustainability for Municipalities and their Partners.  56   Trousdale, W.  (2005). The Local Economic Development Series: Promoting Local Economic Development through Strategic Planning.  Nairobi: United Nations Human Settlements Program.  United Nations Children Fund (1989).  Convention on the Rights of the Child.  Retrieved on November 1st, 2007 from  Vancouver Middle Childhood Matters Initiative (2009) Asset Mapping Pilot Project: Victoria/Fraserview Neighbourhood.  VanWynsberghe, R and Khan, S. (2007) Redefining Case Study.  International Journal of Qualitative Methods 6 (2) June 2007 Walker, B and Salt, D. (2006) Resilience Thinking: Sustaining Ecosystems and People in a Changing World. Island Press, London.  Westley, F. Zimmerman, B and Patton M.  (2006) Getting to Maybe: How the World is Changed. Random House Canada.  Wheeler, W. and Roach, C. (2005) Community-Based Youth Leadership: A Pathway to Civic Engagement. Innovation Center for Community and Youth Development. Yin, R. (2003) Case Study Research: Design and Methods 3rd Edition. Sage Publications, Thousand Oaks.    57    Appendix 1: Roger Hart’s Ladder of Young People’s Participation     58    Appendix 2: Workshop Facilitation Template  Workshop Facilitator Template  Objectives of the Meeting:  Icebreaker:  Timeline: 5:30pm:  Set up 6pm:  Icebreaker 6:15pm:  Review agenda and Confirm roles: minute taker, time keeper 6:20pm:______ ______ 7pm:  Break- Food 7:15pm: _____ _____ 7:40pm:  Assigning Roles and Topics for Next Meeting 7:50pm:  Debrief and Feedback 8pm:  END  Roles: Facilitators: Minute Taker: Timekeeper: Food Coordinator: Parking Lot Attendant: Other:  Ideas for Next Meeting: 59    Appendix 3:  Initiative Framework  Victoria/Fraserview Engaging Neighbourhood Initiative Overarching Goal:  To build a sustainable, community driven initiative which seeks to build relationships and connect community members to opportunities and resources in Victoria/Fraserview in order to better serve middle year youth. Underlying Assumptions: -Strong relationships, information and communication are needed to support middle year youth. -Youth are powerful change agents in the community and their participation is integral to building community capacity.  Activity Goals Did we achieve our goal? How do we know? Lessons Learnt Orchard Park Community Dinner June 2010          -to build relationships in the community by creating an opportunity for families and residents of all ages to interact in a fun informal setting and make connections -to build capacity in the community’s high school and university youth to plan community events by meeting new people, improving communication and problem solving skills and learning about resources in their community.  -the dinner was well attended (60-80 people) and appreciated by those who attended -youth gained skills in communication, outreach, meeting facilitation, event planning and group work.  -number of attendees -anecdotes collected by students and partner -youth debrief -youth blog documenting their experience -ongoing involvement of 4 of the UBC students and 2 of the high school students -youth have successfully taken on increased responsibility -relationships built between youth, community organizations and businesses -there is a thirst in the community for these kinds of events. - despite the skills gained, would have liked to do more skills development with the high school youth.  60   Community Event February 2011  -to build relationships in the community by creating an opportunity for community members, businesses and service providers to work together - to host an interactive food and games event, in which we engage the youth and their parents -to  encourage participation in after school programs by creating this fun and interactive event -to build capacity in the community’s high school and university youth to sustain a youth led movement  -the event was very well attended (250- 300 people), by a diversity of people, mostly by families with middle year youth -families were engaged in our activities and we received much positive feedback -people learned about new programs and resources -youth took on increased responsibility and gained skills in event planning, group work, communication, volunteer management and feedback/evalu ation.  -number of attendees -youth debrief and reflections -feedback from community members -follow-up with partners -blog entries and photos taken by youth and volunteers highlighting: *connections being made between organizations, parents, businesses, community members *types of people who attend (age, affiliation, neighbourhood) *interest in getting involved in programming  -there is a thirst in the community for these events -youth have significant leadership skills, this project much more than the first was youth driven -there is still more to learn, more ideas for projects and motivation from the youth to remain involved -there are opportunities to shift from organising the youth event by event and planning for a long term network of youth that initiate projects in collaboration with partners -there is an opportunity to strengthen middle year youth participation and involvement and capture middle year feelings and needs    61   Appendix 4:  Youth Designed Evaluation Framework All of us on the day of the event will keep our eyes open for: -connections being made between organizations, parents, businesses, community members Goal How will we know if we’ve achieved it? How and who will measure? To create an environment in which the community members, businesses and service providers work together -Observation of what kind of people attend -Amount and variety of donations -Number of organizations that help us promote our event -Contributions from community partners  -stories of people working together (all, blog) -event photographer to capture who was there (Ryan, Mania, Sarah plus UBC professional photographer) -newsletter article reporting on our partners and donors (Mania) To have an interactive food and games evening, in which we engage the youth and their parents -laughter and smiles -guest book -observations -participant feedback -attendance -Assign an interviewer the day of the event to collect stories through video, photos, voice recorder or written stories (Sarah) -Blog about what we experienced and stories we heard after the event (all youth to blog including volunteers and partners) -Have guest book at the event asking people for feedback (Simon) Encourage participation in after school programs through creating this fun and interactive event -how many people take brochures -who signs up for programs in the future -follow up with partners to see if there is an increase in participation in any of their programs (Mania) -stories of interest expressed by youth or parents at the brochure table (all, blog) 62   -types of people who attend (age, affiliation, neighbourhood) -what people liked, what people didn’t like -interest in getting involved in programming -anything else that is inspiring, exciting and interesting! All of us will blog what we observed after the event, and ask our volunteers to do the same, so that all observations are collected in one spot. 63   Appendix 5: Youth Blogs;           64    65    Appendix 6: SWOT Analysis Strengths Youth driven initiative Youth motivated to take on future projects Youth gaining skills in leadership, event planning , communication and volunteer management Collaboration between community partners Community members responsive to the events Existing relationships between youth and partner organizations, opportunity for co-leadership. Existing Asset-Mapping project to draw from UBC collaboration throughout both events supporting someone in a coordinator role Participation of UBC and highschool youth who grew up in the neighbourhood and therefore have vested interest. Weaknesses Lack of a clear identity Uncertainty about what’s next Lack of a home base Youth still learning planning and organizing skills Ad hoc approach, event by event  Opportunities Momentum: two successful events, the second building on the first. Youth have many ideas for future projects Stronger relationship with existing partners Bringing on new partners Learning more about existing community initiatives and coordinating efforts with existing initiatives Recruiting new members to expand the steering committee (parents, youth and business owners) Supporting a network of youth that meet for teambuilding and skills training and that initiate project ideas in collaboration with partners Fundraising: youth initiated and partners can apply jointly to support the project in the future  Threats Loss of momentum Lack of leadership Lack of communication between youth (internally) and between youth and partners Lack of effective documentation Volunteer burn-out Lack of human resources to manage and evaluate Lack of funding Lack of a clear vision as to the overall goals of the project. 


Citation Scheme:


Usage Statistics

Country Views Downloads
United States 8 0
China 1 19
City Views Downloads
Washington 8 0
Beijing 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}


Share to:


Related Items