Open Collections

UBC Graduate Research

An evaluation of official community plans in Metro Vancouver Desjarlais, Lecia Rose Apr 30, 2016

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
310-Desjarlais_Lecia_Evaluation_Community_Plans_2016.pdf [ 6.9MB ]
Metadata
JSON: 310-1.0305733.json
JSON-LD: 310-1.0305733-ld.json
RDF/XML (Pretty): 310-1.0305733-rdf.xml
RDF/JSON: 310-1.0305733-rdf.json
Turtle: 310-1.0305733-turtle.txt
N-Triples: 310-1.0305733-rdf-ntriples.txt
Original Record: 310-1.0305733-source.json
Full Text
310-1.0305733-fulltext.txt
Citation
310-1.0305733.ris

Full Text

AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVERLecia DesjarlaisSchool of Community and Regional PlanningThe University of British ColumbiaApril 2016AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVERByLECIA ROSE DESJARLAISBA hon., Kwantlen Polytechnic University, 2010A PROJECT SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OFMASTER OF ARTS (PLANNING)inTHE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIESSchool of Community and Regional PlanningWe accept this project as conformingto the required standard______________________________Maged Senbel (Supervisor)______________________________Mark Stevens (Second Reader)THE UNIVERSITY OF BRITISH COLUMBIAApril 2016© Lecia Rose Desjarlais, 2016 ACKNOWLEDGMENTS I feel incredibly fortunate to live in this time and place. Urban planning is an inherently optimistic discipline and I look forward to making our cities a more equitable and just place to live. I would like to recognize the generous support and financial contributions of Fishing Lake First Nations and the Social Sciences and Humanities Research Council of Canada (SSHRC). My supervisor, Dr. Maged Senbel and second reader, Dr. Mark Stevens have been invaluable in providing support and feedback throughout the process.        I thank them for being models of best practice and excellence in planning theory and research. Finally, thank you to my incredible family and friends.                     Lecia Rose Desjarlais AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVEREXECUTIVE SUMMARY Municipal planners and city councils are guided by official community plans when making decisions about their community. A plan is a vision of the future and is a primary tool to influence the future growth and development of a community (Dalton, 1989). They are an instrument of public policy and attempt to address both large and small scale goals and represents the interests of many different stakeholders.  This project evaluates municipal official community plans in Metro Vancouver in order to identify their strengths and weaknesses. Plan evaluation was uncommon until recently due to a lack of consensus on what constitutes a good plan, evaluation protocols that were difficult to validate and generalize, and difficulty obtaining planning documents. This project evaluates the quality of 20 Metro Vancouver plans using three different protocols, the Internal and External Quality protocol by Berke, Godschalk, and Kaiser (2006) and Communicative and Persuasive Quality protocol by Bunnell and Jepson’s (2011), and is supplemented by the Online Communication Quality protocol that is designed to capture digital attributes of municipal planning websites and documents. Notwithstanding the importance and the demands placed on municipal Official Community Plans (OCPs) in the region, this study identifies significant gaps and areas of oversight in plan quality. This led to a series of recommendations and identification of examples of best practices. Foremost, municipalities should draw on the practices used by their neighbours. Other municipalities are a rich resource for ideas and municipal planners can quickly identify missing elements from their own plan by reviewing other municipal OCPs. The following is a list of recommendations from easiest to more challenging to implement:•  Ensure the municipal website includes the five basic questions to a complete story (Who, What, When, Why, and How) when describing the OCP.•  Optimize the PDF when creating the OCP for digital upload. It does not take extensive technical expertise to add small and useful features to the PDF to increase usability and appeal.•  Be comprehensive in the amount of basic factual information about the municipality (demographic information, present state of services and infrastructure, etc.). Many small facts (sometimes obvious to municipal planners) can add up to a comprehensive picture of the current state of the municipality and be helpful to people not already familiar with the community.•  Organize policies under a hierarchy of goals and objectives to help clearly and logically organize the OCP. Doing so demonstrates how policies are linked to the challenges in the community as well as reflects the community’s values and aspirations.•  Better integrate the goals, objectives, and policies outlined in the municipality’s Regional Context Statement throughout the sections of the OCP document.•  Explain and compare alternate scenarios or forecasts for housing and population growth.• Include specific actions that are quantifiable, measurable, and comparable to an earlier baseline period.•   Analyze land demand and supply and highlight the challenges and opportunities the municipality faces for future growth. This study is a small step towards critical evaluation of municipal plans in a region faced with significant planning issues. Planners must be willing to improve planning practices within their own municipality and foster a culture of mutual learning and self-improvement. This will enable municipalities in Metro Vancouver to respond to increasingly complex planning issues facing the region in the future. AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 5TABLE OF CONTENTS(1) INTRODUCTION (1.1) Theoretical Context  (1.2) Purpose of Project (1.3) Research Question(2) PREVIOUS RESEARCH ON PLAN QUALITY (2.1) An Early Approach to Plan Evaluation Criteria (2.2) Internal and External Quality (2.3) Communicative and Persuasive Quality (2.4) Online Communication(3) METHODOLOGY (3.1) Plan Selection (3.2) Protocols (3.3) Procedure (3.4) Reliability (3.5) Scoring(4) FINDINGS (4.1) Descriptive Statistics (4.2) Internal and External Quality (4.3) Communicative and Persuasive Quality (4.4) Online Communication (4.5) Overall Strengths and Weaknesses 6 6 8111212141617181819212223242426282929313137373839424350525457(5) DISCUSSION (5.1) Summary (5.2) Recommendations (5.3) Limitations and Future Research(6) CONCLUSION(7) REFERENCES(8) APPENDICES Appendix 1: Protocol Guides Appendix 2: Modified Protocol Items Appendix 3: Reliability Data Appendix 4: Protocol Subcategory Frequencies Appendix 5: Protocol Scores by Municipality 6  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER(1) INTRODUCTIONThis section describes the theoretical context behind comprehensive planning, the rise in popularity of plan evaluation, the purpose of the study, and research questions.(1.1) Theoretical Context Comprehensive plans are an instrument of public policy and attempt to address both large and small scale goals and represents the interests of many different stakeholders. They are a vision of the future and is a primary tool to influence the future growth and development of a community (Dalton, 1989). Developing comprehensive plans involves a thorough analysis of community assets, identification of goals for growth, and public consultation. Kaiser and Godschalk (1995) observed, “Not only do [land use] plans help decision makers to manage urban growth and change, they also provide a platform for the formation of community consensus about land use issues, now among the most controversial items on local government agendas” (p. 365). Achieving consensus and buy-in from the community is a challenging endeavour. Many theories and approaches have been developed about the plan making process. These often address issues such as consultation techniques, mediating conflicting stakeholder objectives, and ensuring underrepresented groups are heard during the process. The planning research literature has an established history of examining plan making and the planning process. However, it has not focused as much attention on the product of the plan making process – the planning document. Official Community Plans (OCPs) are one of the more complex and future-oriented documents within planning practice. They are comprehensive documents specific to the local context and reflect the priorities of the community for ensuring desirable growth and development. They integrate land use, economy, environment, transportation, utilities and services into a broad set of strategies and policies. Official Community Plans guide municipal planners and city councils when making decisions about their community. Plans are made available online on municipal websites for members of the public to review. Residents, property owners, community groups, developers, and investors can refer to the OCPs to inform their own decision making. Development proposals and municipal projects must conform with policies in the OCPs. However, Metro Vancouver municipalities are not beholden to enact all policies outlined in their OCPs during the planning period. Their uniqueness, combined with periodic amendments, and limited timeframes (many are updated every five years or more) have shaped how academics and practitioners view plans. One researcher posits that the lack of planning evaluation research may be due to the perception that plans are “in large part works of art – designs that defy rational analysis”. (Berke & Godschalk, 2009, pp. 228). There are several reasons why plan evaluation was uncommon until recently: 1)  There was lack of consensus on what constitutes a good plan2)  Evaluation protocol were difficult to validate and generalize3)  Planning documents are difficult to obtain“Given widespread usage and significance of comprehensive plans, it is surprising that plans are not routinely evaluated against accepted plan quality standards” (Berke & Godschalk, 2009, pp. 227)(1) INTRODUCTION AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 7 The lack of consensus about what constitutes a good plan was summarized by Berke and French (1994): “Plan quality is difficult to define. Planners can often differentiate high quality plans from low quality ones, but they are hard pressed to explicitly define the key characteristics of plan quality. The planning literature is surprisingly narrow when it comes to what constitutes a good plan. The planning profession has generally avoided this normative question and focused instead on the methods and processes of plan making.” (237-8) The methodology behind selecting an award-winning plan varies among planning organizations. However, planning researchers must systematically develop their own evaluation criteria for plan quality. These criteria must be validated and replicable for a wide variety of plans. Achieving this is quite challenging. A variety of criteria can be used to assess plan quality. Researchers may be interested in specific geographic locations (country, state/provincial, or regional levels), particular types planning documents (climate plans or comprehensive plans) or planning domains (sprawl reduction or greenhouse gas reduction). There are also several points during the plan-making process that are suitable for evaluation (Baer, 1997). Evaluation may also be conducted internally, by an outside critic, or by academics. The diversity of approaches has limited the generalizability of research findings in planning evaluation.  Content analysis is the standard methodology to evaluate planning documents. Some studies evaluate plans on a country, state, or provincial scale (sprawl-reduction policies in Florida as in Brody, Carrasco, & Highfield, 2006; or climate change planning in British Columbia as in Baynham & Stevens, 2014). Others examine a particular type of planning document (climate action plans, as in Boswell et al., 2010, or land use plans as in Burby & Dalton, 1994). The investigator also may select a particular domain of planning to focus their investigation (sustainable development as in Berke & Conroy, 2000; or smart growth as in Edwards & Haines 2007). Evaluation criteria may only be applicable to one geographic location, type of planning document, or domain. Interest in plan evaluation appears to be growing, as 30 of the 47 evaluation studies published since 1994 were published in the past ten years (Stevens, Lyles, & Berke 2014). Planning scholars now have a foundation of planning evaluation criteria to use or adopt to their own purposes. Bunnell and Jepson (2011) states that “there will never be one best way to plan” (pp. 338). And similarly, there will be never be one single evaluation criteria to judge the quality of all plans. One set of criteria is not appropriate for all plans. As one author stated, the debate and discussion about what constitutes “quality” will likely will continue (Lyles & Stevens, 2014). Plan evaluation has accelerated in recent years due to the increase in accessibility of planning documents at a municipal and national level. It has become necessary for organizations large and small to have representation on the Internet. Advancements in technology have allowed governments to disseminate information and collect public feedback through interactive websites and social media. Canadian municipalities typically score well in their ability to provide content and larger municipalities scored highly (Dolson & Young, 2012). This ease of access has enabled planning scholars to quickly locate and download plan documents.“If plans are to achieve their full potential, they should reflect the highest quality of thought and practice. Only systematic evaluation enables us to identify their specific strengths and weaknesses, to judge whether their overall quality is good, and to provide a basis for ensuring that they reach a desirable standard.”(Berke & Godschalk, 2009, pp. 228)(1) INTRODUCTION8  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVERTable 1: Metro Vancouver at a GlancePopulation (2011)  2,305,9542021 Population Projection  2,780,0002041 Population Projection  3,400,000Area  2,877 km²Total Dwelling Units (2011)  943,807Regional Growth Strategy Adopted  2011(Statistics Canada, 2011; Metro Vancouver, 2011)(1.2) Purpose of Project Taken together, planning practitioners can now draw upon the growing body of existing research and use easily accessible planning documents as a basis for their research. This project will be an analysis of plan quality in the Metro Vancouver region using plan evaluation protocols. There are several reasons to evaluate the official community plans in this region:1)  The municipalities share a common policy base through a Regional Growth Strategy. 2) The region is known for innovative planning practices with high levels of public engagement and interest in planning issues.3) There is opportunity to further validate plan evaluation protocols and techniques in a Canadian context.  Metro Vancouver consists of 21 municipalities, one treaty First Nation, and one electoral area. Table 1 outlines basic demographic data about the region. It is a regional planning authority that encourages cooperation and coordinated policies and among its member. In 2011, Metro Vancouver adopted the Regional Growth Strategy titled Metro Vancouver 2040: Shaping Our Future with a main purpose to encourage growth within a sustainability framework.  Metro Vancouver was formed under a regional strategy mandate contained in the Local Government Act (c. 323, p. 25). This permits municipalities within a regional district to unite to form a regional growth strategy. Municipalities must coordinate their planning efforts and state how their policies adhere to regional goals in Regional Context Statements, contained in their official community plans. The Local Government Act provides a comprehensive list of priorities that the strategy should address. Metro Vancouver has organized these priorities under 5 Goals and 14 Strategies (See Table 2). Municipalities accept these policies by council resolution it is expected that official community plans adhere to the policies of the Regional Growth Strategy. Subsequently, each municipality can be expected to share a similar degree of comprehensiveness to one another due to a shared mandate.“The purpose of a regional growth strategy is to promote human settlement that is socially, economically and environmentally healthy and that makes efficient use of public facilities and services, land and other resources.” (LGA, Chapter 232, Division 1 849 (1))(1) INTRODUCTION AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 9Table 2: Metro Vancouver Regional Growth Strategy Goals and StrategiesGOAL 1Create a Compact Urban AreaStrategy 1.1: Contain urban development within the Urban Containment BoundaryStrategy 1.2: Focus growth in Urban Centres and Frequent Transit Devel-opment AreasStrategy 1.3: Protect Rural areas from urban developmentGOAL 2Support a Sustainable EconomyStrategy 2.1: Promote land development patterns that support a diverse regional economy and employment close to where people liveStrategy 2.2: Protect the supply of industrial landStrategy 2.3: Protect the supply of agricultural land and promote agricul-tural viability with an emphasis on food productionGOAL 3 Protect the Environment and Respond to Climate Change ImpactsStrategy 3.1: Protect Conservation and Recreation landsStrategy 3.2: Protect and enhance natural features and their connectivityStrategy 3.3: Encourage land use and transportation infrastructure that reduce energy consumption and greenhouse gas emissions, and improve air qualityStrategy 3.4: Encourage land use and transportation infrastructure that improve the ability to withstand climate change impacts and natural hazard risks.GOAL 4Develop Complete CommunitiesStrategy 4.1: Provide diverse and affordable housing choicesStrategy 4.2: Develop healthy and complete communities with access to a range of services and amenitiesGOAL 5 Support Sustainable Transportation ChoicesStrategy 5.1: Coordinate land use and transportation to encourage tran-sit, multiple-occupancy vehicles, cycling and walkingStrategy 5.2: Coordinate land use and transportation to support the safe and efficient movement of vehicles for passengers, goods and services(Metro Vancouver, 2011) Planning excellence in the region has been demonstrated in several ways in the past decade. Metro Vancouver is the first jurisdiction in the world to develop a social sustainability framework (Colantonio, 2008). Metro Vancouver also stands apart from to other regions such as Toronto and Calgary metropolitan areas because it has a long traditional of regional planning and strong regional governance (Taylor, Burchfield, 2010). In 2012, Metro Vancouver was awarded second 2nd place in the United Nations Public Service Award. This award recognized Metro Vancouver for excellence in fostering participation in policy making decisions through innovative mechanisms (Metro Vancouver, 2012). British Columbia has been labelled a leader within North America for its climate action planning (Baynham & Stevens, 2014). This is due to its requirement for all municipal official community plans to include targets, policies, and actions to reduce greenhouse gas emissions.  The limited geographic land base in Metro Vancouver may also have necessitated good planning practice. (Taylor, Burchfield, 2010). Compared to other cities, growth in Metro Vancouver is restricted by policies such as the Agricultural Land Reserve (ALR) and the Green Zone in addition to its proximity to the United States’ Border, Coast Mountains, and the Strait of Georgia. In July 2011, the City of Vancouver adopted the Greenest City Action Plan. This set of policies is aimed to make the city the greenest in the world by 2020. A further target is to have the city derive 100% of its energy from renewable sources 2050. Although the City of Vancouver will not be included in this study (explained in the next section), the region has been held in high esteem for their planning policies. Table 3 lists municipalities in Metro Vancouver (excluding the City of Vancouver) that have won awards in last 10 years. A small number of plan evaluation studies have examined planning documents in British Columbia. A study of twenty sustainability plan by Stevens and Mody (2013) found that provisions for implementation were weak and in need of improvement. They noted these plans were not written in a way that committed municipalities to carrying out action included in the plans. Baynham and Stevens (2014) examined municipal OCP targets and policies for reducing greenhouse gas emissions and found the fact base and implementation sections were weak. Stevens (2013) evaluated forty OCPs in southern British Columbia and concluded the weakest features of the plans were implementation, monitoring, and evaluation section. Two studies have examined municipalities specifically within Metro Vancouver, including a dissertation that examined Regional Context Statements (De Sousa, 2015), and another for natural hazard planning (Stevens & Shoubridge, 2015). All told, there is an opportunity to contribute further to planning excellence in the province and represent it better in planning evaluation literature.(1) INTRODUCTION10  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVERTable 3: Municipal OCPs in Metro Vancouver that have won awards in the past 10 years from CIP and PIBC Year Municipality Plan Name Award CategoryAwarding Organization2015 Township of Langley Township of Langley Age-friendly StrategyAward for Planning Excellence Merit: New and Emerging Planning InitiativesCIP2014 City of New WestminsterCity of New Westminster Secured Market Rental Housing PolicyExcellence in Policy Planning: City & Urban Areas (Silver)PIBC2013 City of New WestminsterCity of New Westminster Child Care Strategy ImplementationExcellence in Planning Practice: City & Urban Areas (Gold)PIBC2012 District of North VancouverIdentity DNV 2030: District of North Vancouver Official Community PlanExcellence in Policy Planning: City & Urban Areas (Honourable Mention)PIBC2012 City of Burnaby Burnaby Social Sustainability StrategyAward for Planning Excellence Merit: Social PlanningCIP2011 City of North VancouverCity of North Vancouver Community Energy & Emissions PlanExcellence in Policy Planning PIBC2010 City of LangleyCity of Langley and MVH Urban Planning & Design Inc. Downtown Master PlanComprehensive Policy and Plans PIBC2009 District of Maple Ridge Maple Ridge Town Centre Area PlanAward for Planning Excellence Merit: City and Regional PlanningCIP2008 City of BurnabyBurnaby Economic Development Strategy (EDS) 2020Award for Planning Excellence Merit: Economic DevelopmentCIPCIP – Canadian Institute of Planners, PIBC – Planning Institute of British Columbia (Source: CIP, PIBC)(1) INTRODUCTION AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 11 The next section delves further into the theoretical foundations of planning evaluation. It will focus on the two protocols adopted for the purpose of this project as well as explains the impetus for evaluating online content of planning information. It goes on discuss methodology, including plan selection, protocols, procedure, reliability, and scoring. Following that is an overview of the findings, including strengths, weaknesses, and areas of improvement for municipal OCPs in Metro Vancouver. Lastly the discussion section will recap major findings, make recommendations, explain limitations, and suggest avenues for further research.(1.3) Research Questions Given the expectation for high quality and a degree of congruence between municipal policies in Metro Vancouver, this project seeks to understand how well these plans score on normative criteria of plan quality. Three planning evaluation protocols are used in this study. The first, developed by Berke, Godschalk and Kaiser (2006), examines the content of the individual plan components, relevance, scope, and coverage of the plan in fitting with the local context. The second protocol, by Bunnell and Jepson (2011), examines the communicative and persuasive qualities of plans. The third protocol is developed by the author and is designed to assess the quality of online communication of planning information. A more elaborate discussion of these measures will be discussed in the next section. These protocols help address several research questions about municipal OCPs in Metro Vancouver: The findings of this study will help identify attributes of municipal plans that have been overlooked, weakly addressed, and are in need of improvement. Since all of these municipalities are under the Local Government Act mandate, it may be possible to borrow ideas from one OCP that performed well and adopt it to a neighbouring OCP. Finally, as these are validated measures, scores can be compared to similar evaluation studies in from different regions. It would be valuable to understand how well Metro Vancouver’s OCPs compare to plans from other regions that may have very different mandates for comprehensive plans.(1) What are the key strengths and weaknesses of municipal OCPs in Metro Vancouver?(2) What key attributes of municipal OCPs in Metro Vancouver are in need of improvement?(3) What can municipalities learn from one another in applying best practices for their policy content and communicating information within their OCPs?(1) INTRODUCTION12  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER One of the questions raised through his examination of plan evaluation criteria was whether postmodern criteria could be operationalized. Baer (1997) concluded that integrating a postmodern perspective into planning evaluation or discourse would inevitably open the door to deceitful intentions by a variety of groups and political posturing. Here, plans would be deconstructed with little support from politicians or the public – and essentially be of no use to anyone. With that in mind, Baer tempered his largely modernist criteria with “some postmodernist cautions” (pp. 341). This set of criteria influenced later work in developing the theory behind planning evaluation protocols. This section describes how plan evaluation criteria was shaped by modern and post-modern thinking by Baer (1997). Two primary evaluation protocols were shaped by this foundation, one examining internal and external plan attributes, and the other examining communicative and persuasive qualities. This section also discusses the need for an assessment protocol suitable for municipal planning websites.(2.1) An Early Approach to Plan Evaluation Criteria One impetus for developing plan evaluation criteria was a growing prevalence of mandates for local state planning in the United States. This sparked interest in understanding how these plans were developed and implemented. For example, how they were submitted and reviewed, whether they were mandatory or voluntary, the extent and limitations of their authority, how disagreements were resolved, how public consultation was conducted, how they are revised and amended, as well as how consistent and compatible they are with other plans (Gale, 1992). This opened up many opportunities for planners to study plans and the planning process. As plans became increasingly influential in shaping future growth and development, planners recognized that they must evaluate plans to justify the responsibility placed on these documents (Baer, 1997).  Early thinkers in plan evaluation literature were conscious of the limitations of ridged rational-comprehensive plans. Baer (1997) drew on past research that addressed both modern and post modern issues and suggested a series of evaluation criteria (See Table 4). His early framework helped form the basis of several different planning evaluation criteria. Plans were emphasized as a “product” not an “incidental output of the planning process” (Baer, 1997, pp. 341) during a time where process-oriented research dominated planning literature and highly structured rational-comprehensive plans became prevalent in the United States. (2) PREVIOUS RESEARCH ON PLAN QUALITY(2) PREVIOUS RESEARCH ON PLAN QUALITY AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 13Table 4: General Plan Evaluation Criteria (Baer, 1997) Adequacy of ContextThis explains the context and setting (what and why) of the document. This includes the political/legal context, administrative authority, role of agency/firm, background information, stating who the plan is for, purpose explained, type of plan and scope, overview/summary provided, source of funding, and amount of time in preparation. “Rational Model” ConsiderationsThis shows basic planning considerations, including clearly communicating the theory used. This includes being clear about the criteria to be used, problems, goals, objectives and clearly stated, whether the plan correspond with the approach (e.g. a comprehensive plan would relate to a larger context and other government organizations), the existing infrastructure and assets described, alternatives listed (and are they significantly different enough to be considered ‘true’ alternatives, and finally, mentioning whether trade-offs are permitted.Procedural ValidityThis informs the reader about what went into making the plan and what happens when it is published (the who and how). This includes the people involved in making the plan, how they were chosen, how they were involved, how data was used in recommending policy or action, how technical matters transformed into policy, inclusion of any advisory groups, and distribution of preliminary drafts for public feedback. Adequacy of ScopeThis shows how the plan is connected to the larger world. This includes whether pertinent issues (physical, social, economic, political, cultural, psychological, design) been considered, consideration of issues of efficiency, equity, and predictability, distribution of costs/benefits, relocation/displacement implications, legal implications, feasible in the larger political context.Guidelines for ImplementationThis considers the instruments (ordinances, regulations, budget, schedules) and agencies and individuals to be included in the carrying out of the plan. This include appropriate implementation provisions, priorities for implementation, costs of implementation (and not implementing), the time span, scheduling and coordination of actions, impact analysis, responsibilities of individuals and agencies identified, and whether implementation is realistic.Approach, Data, and MethodologyThis makes clear the technical basis of the plan, with documented sources of information for others to fact-check the plan. This includes ensuring the plan is based on a wide spectrum of data (where possible), flexibility to permit new data to be added, data sources cited, methodology sources cited, levels of data aggregation appro priate and meaningful to the plan.Quality of CommunicationThis allows all information in the plan to be communicated clearly. This includes identifying the reader or client who will be reading the plan, clearly and convincingly presenting information, effectively presenting rationales, and including proposals, recommendations, and conclusions that are consistent with the objectives, matching tone with the message conveyed, and including criteria by which the plan can be judged.Plan FormatThis means other communicative aspects of the plan that send a message regarding competence and accessibility to the reader. This includes the size and format of the plan, date of publication, including authors, table of contents, numbered pages, graphics used effectively, and an attractive layout.(2) PREVIOUS RESEARCH ON PLAN QUALITY14  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER(2.2) Internal and External Quality The first plan evaluation protocol that was sufficiently quantitative and technically verifiable was developed by Berke, Godschalk, and Kaiser (2006) from a section from their book, Urban Land Use Planning. It was a direct extension of Baer’s (1997) work. The protocol was developed to minimize subjectivity and uses a list of 60 criteria that must be searched for and identified within a plan. Items are scored on a three-point scale (0, 1, 2) then summed to give an overall score of plan quality. To achieve this level of objectivity, items with more subjective elements were avoided. This enables plans scores to be compared against one another. The protocol consists of two major conceptual dimensions, internal plan quality, and external plan quality. Internal plan quality encompasses the content and format of the plan, while0 External plan quality covers the relevance, scope, and coverage of the plan in respect to the local context. Table 5 highlights the subcomponents of these two dimensions. These form the basis for the plan quality protocol. The authors emphasize that these criteria should be free to be adopted to the local context and evaluation circumstances.  Berke et al. (2006) identified several common limitations to creating high-quality plans that their protocol aimed to address. For example, a weak fact base can undermine the rationales for decision making and policy selection. Many plans also do not have substantial monitoring and implementation sections. These sections lack measurable objectives, timelines for implementation, and indicators to track progress. Lastly, plans are often challenging to read and comprehend by laypeople. The are sometimes poorly organized and formatted with few images and diagrams. An absence of these elements can greatly undermine the impact a plan hopes on having shaping future land use and development decisions.(2) PREVIOUS RESEARCH ON PLAN QUALITY AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 15Table 5: Components of Internal and External Plan Quality Dimensions (Berke, Godschalk, & Kaiser, 2006)Internal Plan QualityIssues and Vision StatementThis identifies major assets, trends, and impacts forecasted for the community. It should also reflect the concerns and values of the community and include a vision statement for the future. It should also include a description of major opportunities, threats, problems, and issues.Fact BaseThe fact base elaborates on the major assets and identifies issues and problems. It should include the following features: present and future projections for population, economy, land use, land supply, facilities, and infrastructure. It should also describe the state of the natural environment. These should be supported using maps, tables, reference data, with methods and models.Goals and Policy FrameworkThis identifies goals and policies and also demonstrates how they are linked to community values, problems, and aspirations. “Goals are broad expressions about the desired future of the community” (pp. 71), while policies are “established principles to be followed in guiding public and private decisions to achieve a desired future land use and development pattern” (Berke, Godschalk, & Kaiser, 2006, pp. 71). Plan ProposalThis includes describing the future form of the community and outlining an implementation program. It also describes a program for monitoring and evaluating along with means to update and adjust the plan along the way. It includes aspects of infrastructure, transportation, open spaces, land use, water, sewer and more. Specific “actions” to achieve these are prioritized and included on a timeline along with identifying who is responsible for carrying out the actions.External Plan QualityRecognize and Encourage Opportunities to Use PlansPlans should include certain aspects that enhance the chances they will be used. Some of these include being inspirational (offers com-pelling courses of action), action-oriented (clearly articulated action-oriented agenda), flexible (allows for alterative courses of action), and legally defensible (includes explanations of legislative and administrative authority).Create Clear Views and Understanding of PlansThe plan should be relevant and understandable from the perspective of all governmental units (county-level, district-level) and types of readers (citizens, elected officials). A plan that is readily accessible in this way enhances the potential for creating awareness and support for the public interest and could enhance implementation. Account for Interdependent Actions in Plan ScopeA plan should recognize that actions are not carried out in isolation. It should communicate the necessity for coordinating and communi-cating among different organizational departments/units in order to maximize potential for implementation and influence. There should be recognition of the range of possible independent actions possible to carry out policies.Reveal Participation of Formal and Informal Actors (or Institutions)A plan should explain who was involved in making the plan, how they participated, and how the plan evolved in response their participa-tion. This includes government agencies, the private sector, and individual citizens. It’s important to included disadvantaged members of the community in the process, as they are not as powerful, organized and well-funded as other stakeholders.(2) PREVIOUS RESEARCH ON PLAN QUALITY16  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER These criteria were grouped into two major protocol categories. One is pertaining to the readability, synthesis, quality of presentation, narrative, persuasiveness, and realism of plans. The other is related to attributes of format, style, and appearance of plans. Individual items can also be examined with respect to four dimensions described by Hopkins and Zapata (2007):1)  Standardization/rigidity2)  Acknowledgement of uncertainty3)  Understanding of alternatives conveyed4)  Compelling narrative storyline The Communicative and Persuasive Quality protocol was validated in an exploratory analysis of comprehensive plans in the United States. There was an increase in state laws mandating comprehensive plans in the past few decades leading to a rise in plans that were designed to meet specific legislative requirements. A number of different studies arose that examined the various qualities of state-mandated plans. The general consensus is that having a state mandate is positively associated with plan quality. However, state mandates vary widely in content that is required and this can subsequently affect local plans quality (Berke & French, 1994). Bunnell and Jepson (2011) applied their protocol to 20 plans that were state-mandated and 20 plans that were not. After controlling for city size and geographic characteristics, the plans were scored using two independent coders. The coders discussed any discrepancies that arose in their judgement and came to a consensus which was then included in the final analysis. They found all fell short in their communicative and persuasiveness qualities but state-mandated plans were the weakest in this respect. The authors hope that this study encourages planning practitioner to inject some creativity and imagination into their planning documents in the hopes of creating a more effective, high-quality planning document.(2.3) Communicative and Persuasive Quality Mentioned earlier, one set of evaluation criteria is not sufficient for evaluating all plans. It is necessary to evaluate plans from more than one theoretical perspective. Bunnell and Jepson (2011) developed a plan evaluation protocol that examined planning documents from a communication and persuasive perspective. This was partially in response to Berke et al.’s approach to plan quality evaluation that emphasized obtaining objectively measurable items, leaving other aspects of plan quality elements that Baer (1997) emphasized unaddressed. Plans are not policy documents intended only for bureaucrats, they also must connect with non-professional readers and elicit positive participation. They argue that writing a persuasive plan can be an important objective for municipal planners where it may be necessary to get the evoke the attention of elected officials and the public about important community issues. Similarly, Norton (2008) came to the conclusion that the content of the plan “should be distinguished from the way in which that content is conveyed” (pp. 432). Bunnell and Jepson drew their protocol items from previous research by Baer (1997) and Berke et al. (2006), as described earlier. The protocol was also shaped by authors such as Kent (1964), Black (1968), Hopkins and Zapata (2007), Avin (2007), Isserman, (2007), Klosterman, (2007). They identified a set of criteria as fundamental to effective communication: 1)  Including photographs and illustrations2)  Limiting aims and actions3)  Attractive format and page layout4)  Relevant tables and data5)  Attractive, readable, and informative executive summary(2) PREVIOUS RESEARCH ON PLAN QUALITY AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 17 The six basic questions to a complete story, also known as the six “Ws” could serve instructive to structuring a municipal planning webpage. Journalists (and school children) are typically taught that these basic questions must be answered in order to have a complete story (Spencer-Thomas, 2012). Each W is posed as a question, with the expectation that has a factual answer.Who – did that?What - happened?When – did it take place?Where – did it take place?How – did it happen?Why – did that happen? A municipal official community webpage can be evaluated on the basis of these six questions. For example, the “who” would be municipal planners and council, the “what” would define what an official community plan is, the “when” would be time period the planning process was carried out or enacted, the “where” is self-evident and could be skipped, the “how” would be an explanation of the consultation process, and the “why” would be an explanation of the purpose or mandate of developing an OCP. Together, these would form a somewhat complete picture of the planning process and reasoning behind producing a planning document. Similarly, the attributes of the OCPs portable document format (PDF) could also be optimized for accessibility through auditing the PDF features. Matching page numbering, text-searchability, use of bookmarks, embedded hyperlinks, and more, are small and useful features of this format yet many municipalities may not be making full use of the PDF format. These features make navigating a lengthy OCP document faster and easier.  A clear and informative OCP webpage combined with an optimized PDF format together reduce the barriers the public has to locating, accessing, and navigating a OCP document online. It contributes to the overall communicative quality of a plan.(2.4) Online Communication In the past 20 years, the Internet has become the standard for fast and cheap delivery of large quantities of information. All levels of government have recognized the need deliver information and service to citizens – typically referred to as E-Government (West 2001). One essential component of E-government is the facilitation and encouragement of public engagement (Henriksson et al. 2006). Developing an official community plan involves considerable public consultation and a high-quality website can potentially play a large role in disseminating information and gathering public input. After the consultation process is complete, members of the public should still be able to access basic information about the plan, the planning process, as well as have an opportunity to provide feedback as a permanent feature of the planning website. Few studies have examined the usability of municipal websites but none have examined it within the context of planning evaluation. Dolson and Young (2012), examined E-government features of municipal websites in Canada in three major areas: e-content, e-participation, and social media capacity. A key finding was that most municipalities scored well in providing content but many varied widely in their capacity to provide opportunities for public participation (e.g. posting council meeting dates, facilitating online polls and discussion forums) and social media (e.g., presence on Twitter, Facebook, and YouTube). Municipalities should provide fundamental information about planning, and more specifically, the official community plan. Some people may be encountering or learning about planning for the first time, so basic information about OCPs and why they are created is an essential starting point. Ideally, a webpage should be user-friendly and provide essential information and include a link to download the municipal plan. Municipalities should also explain the participation process and include interactive features wherever possible. (2) PREVIOUS RESEARCH ON PLAN QUALITY18  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER Three local governments were excluded from this study, despite being member municipalities of Metro Vancouver. The City of Vancouver is incorporated under the Vancouver Charter, which grants the city different powers than other municipalities under the Local Government Act. It is not legislatively required to produce an OCP. In the 1970s, the city approved a program of creating Local Area Plans. These consist of 22 neighbourhood plans and numerous major planning project areas. The format and priorities of some of these plans differ significantly from a typical OCP. Including all of Vancouver’s 21 neighbourhood plans would be infeasible for this study, and warrants a separate study. Electoral Area A consists of University of British Columbia (UBC), University Endowment Lands (UEL), Barnston Island, Howe Sound communities (includes Passage and Bowyer Islands), Indian Arm/Pitt Lake communities. There is no unifying OCP for Electoral Area A. The UBC and UEL land use planning are administered through the provincial government. Lastly, the Tsawwassen First Nation community does not have an OCP.This section reviews the sample of plans under examination, the protocols, evaluation procedure, efforts to establish statistical reliability, and scoring methods.(3.1) Plan Selection The plans of interest for this study were selected because they were located in the Greater Vancouver Regional District, also known as Metro Vancouver. Mentioned earlier, these plans share a common policy base through a Regional Growth Strategy. The region is also known for innovative planning practices with high levels of public engagement and interest in planning issues. This region is also the most populous in British Columbia and most municipalities likely have larger planning departments than other communities. The official community plans were collected by visiting the municipal planning websites and downloading a copy in a portable document format (PDF). Locating and downloading a copy of each municipality’s OCP took 40 minutes. The ease of this process is a testament to efficiency of municipalities to disseminate information and supports the conclusions made by Dolson and Young (2012). The plans were retrieved online November 2, 2015.  Evaluating online planning information was achieved by identifying the web addresses where the OCPs can be located to download. These links were provided to the plan evaluation coders. Typically, a municipality organizes all OCP information on a single page. This allows readers to access the information all in one place and minimizes the amount of time “clicking around” trying to find information. This is arguably the best-practice for communicating digital information and therefore coders were instructed to evaluate the content for the webpage provided and not navigate away from the page to collect additional information. A list of municipalities included in the analysis is included in Table 6. Twenty plans were evaluated, with populations ranging from nearly half a million to under one thousand. All but two plans were adopted after the year 2000. Most plans have been updated or amended in the last four years. (3) METHODOLOGY(3) METHODOLOGY AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 19(3.2) Protocols As discussed earlier, plan evaluation research uses content analysis as its primary method to establish plan quality. Evaluation protocols are applied to planning documents using trained researchers. This study administered Berke et al’s (2006) Internal and External Quality protocol and Bunnell and Jepson’s (2011) Communicative and Persuasive Quality protocol to 20 municipal OCPs in Metro Vancouver. A separate Online Communication Protocol was developed to capture attributes of municipal planning websites and the features embedded in the plan PDF. Today, OCPs are primarily retrieved online and this makes the webpage content and features of downloadable documents more essential. Explained earlier, no studies have examined attributes of municipal planning webpages. There is an opportunity to apply a new, straightforward, and short protocol in this study. The items are aligned with Bunnell and Jepson’s (2011) conceptualization of communicative and persuasive quality of plans.  The protocols consist of a list of desirable features expected to be present in a plan. Appendix 1 contains all three protocols. Trained coders searched the planning document to identify whether the items appeared. Items that were present in the plan and met the criteria were scored with a ‘1’ and items that were not present in the plan were scored with a ‘0’. A bimodal scoring system was adopted based on Stevens and Senbel forthcoming research who advocated for this method to maximize reliability and ease of scoring. Berke et al. (2006) and Bunnell and Jepson (2011) originally designed their protocol items as scorable on a three-point scale. Table 7 outlines an example of how a three-point scale was converted to a bimodal scale for the purpose of coding and establishing reliability. When scoring the Communicative and Persuasive Quality protocol, the original multi-point scoring method was used.Table 6: Metro Vancouver MunicipalitiesMunicipality Population(2011)Plan AdoptedLast AmendedCity of Surrey 468,251 2014 2015City of Burnaby 223,218 1998 2014City of Richmond 190,473 2012 -City of Coquitlam 126,456 2007 2014Township of Langley 104,177 1979 2015Corporation of Delta 99,863 2005 2015District North Vancouver 84,412 2011 2014District Maple Ridge 76,052 2014 -City of New Westminster 65,976 2012 2014City of Port Coquitlam 56,342 2005 2013City of North Vancouver 48,196 2015 -District of West Vancouver 42,694 2004 -City of Port Moody 32,975 2014 -City of Langley 25,081 2006 2013City of White Rock 19,339 2006 2013City of Pitt Meadows 17,736 2007 2012Bowen Island 3,402 2010 2014Village of Anmore 2,092 2014 -Village of Lions Bay 1,318 2009 -Village of Belcarra 644 2011 -Total Population 1,688,697Not Included in AnalysisCity of Vancouver 605,502Electoral Area A 13,035Tsawwassen First Nation 720Total Population of Metro Vancouver Region2,307,954(Statistics Canada, 2011)(3) METHODOLOGY20  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER Both protocols were also modified to better suit the needs of this study. Stevens and Senbel revised both Bunnell and Jepson’s (2011) and Berke et al.’s (2006) protocol’s in their study of award-winning plans. Items were modified to add more fine-grained detail. Table 8 outlines an example of an existing item in Berke et al. that was split into two new items. This study used the revised protocols created by Stevens and Senbel. In many cases, modifying established and validated protocols may negatively impact reliability. However, Berke et al. (2006) clearly states that planners should not “become captivated with the methods and expertise that they imply. Planners should use these criteria as a starting point, but in the process of plan making they should adopt them and craft their own criteria to fit their circumstance” (pp. 76). As the author of this study was one of the coders Stevens and Senbel’s forthcoming study, she was already experienced in using the protocols. For this reason, they were further modified to better capture the priorities of Metro Vancouver’s regional policies. Items not applicable to Metro Vancouver municipalities were removed and several new items added. Appendix 2 contains the list items that have been added, removed, or modified in some way from the protocols used by Stevens and Senbel.Table 7: Example of Items Converted to a Bimodal Scoring SystemCommunicative and Persuasive Quality (Bunnell & Jepson, 2011)Original Item Revised10. Is there a vision statement that conveys the essence of what the community wants to be and look like in the future?” Yes  ____ (4 points)Somewhat  ____ (2 points)No   ____ (0 points)32 - Vision statement (YES): Is there a vision statement that conveys the essence of what the community wants to be and look like in the future?Yes  ____ (4 points)No  ____ (0 points)33 - Vision statement (SOMEWHAT): Is there a vision statement that conveys the essence of what the community wants to be and look like in the future?Somewhat  ____ (2 points)No   ____ (0 points)Internal and External Quality (Berke et al., 2006)Original Item Revised2b.6 Are data sources cited?Identified, clear, relevant ____ (2 points)Identified, vague  ____ (1 point)Not identified ____ (0 points)11 - Data Sources (YES) Does the plan generally cite data sources?Give a 1 if: YesGive a 0 if: No12 - Data Sources (GENERALLY) Does the plan cite at least one data source?Give a 1 if: YesGive a 0 if: NoTable 8: Example of Modified Item Added/Removed ItemRemovedIs there a preliminary assessment of major trends and impacts of forecasted change during future planning period?Added Assessment (QUALITATIVE): Does the plan include a preliminary assessment of major trends and impacts of forecasted change during the future planning period?AddedAssessment (QUANTITATIVE): Does the plan include a preliminary assessment of major trends and impacts of forecasted change during the future planning period?See Stevens and Senbel’s forthcoming study for a full list of modified items.(3) METHODOLOGY AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 21was selected because it represented a medium-sized city that is located in the Fraser Valley Regional District which is adjacent to Metro Vancouver (Greater Vancouver Regional District). Fort St. John was selected because it represented a city with a smaller population and possibility represented the efforts of a smaller planning department with limited resources. All three plans were enacted by council in roughly the same time frame (2011-2014) as the plans under study.  After the pretest phase, certain items on both protocols added, removed, or modified. The protocols were updated to include these new items. They are marked by a red asterisk in the protocol guides in Appendix 1. Once satisfied with their agreement on the protocol items during the pretest phase, the students independently coded the plans. This protocol followed Stevens, Lyles, & Berke (2014) recommendations for methodology. Later, the student coders met to discuss agreement on all the test phase plans in order achieve a single score for each of the protocols. To reduce chances of order effects (practice or fatigue affecting how the coder responds to items), protocols were counterbalanced. A coder would start with the Internal and External Quality protocol for one plan, and on the next plan they would start with Communicative and Persuasive Quality protocol. Further counterbalancing was achieved by having one coder progress through the plans alphabetically (Anmore to White Rock) and the other coded the plans in reverse alphabetical order (White Rock to Anmore). The Online Communication protocol was administered relatively quickly over the course of one to two evenings. The municipal website component contained six items and the PDF component was scored in under two hours. The Online Communication protocol was also counterbalanced and scored through consensus.(3.3) Procedure Evaluating the online municipal planning information took place between January 22-25, 2016. Hyperlinks to the planning pages are attached to Table 6. It cannot be expected that these hyperlinks remain up to date. Over time, these OCPs will be updated and the online content will be superseded with newer information. One method of viewing past webpages is through using the Internet Archive Wayback Machine (https://archive.org/web/). This tool saves webpage data and makes it possible to view webpages as they appeared on specific dates. Protocol items were organized into protocol guides that aided coders with their scoring and provided a consistent document to reference whilst reviewing the plans. These guides are contained in Appendix 1 and includes all protocol items used in this study. The plans were coded using an Excel document for data entry supplemented with paper copies of the protocol guides. The Excel document was set up to maximize legibility, consistency, and organization, of the protocols and items. Content from the regional context statement, development permit areas, and any attached neighbourhood plans were excluded from analysis.  Two student coders applied the protocols to each plan independently in order to reduce bias and increase reliability. They were planning students enrolled at the School of Community and Regional Planning at UBC and had prior experience using the both existing protocols. As research assistants in the Planning Evaluation lab, they coded plans contained in Stevens and Senbel’s forthcoming study. During the pretest phase, they met regularly to discuss disagreements and came to consensus regarding the interpretation of items. Together, they coded 22 plans as research assistants prior to the current study, giving them significant pretesting experiencing for this study. To refresh their familiarity and consistency with the two protocols, the two students met to discuss their agreement using three pretest OCPs. Three municipalities in British Columbia were selected as pretests, the City of Victoria, Chilliwack, and Fort St. John. The City of Victoria was selected because it represented a larger municipality and is a member of the Capital Regional District, a regional decision-making body much like Metro Vancouver. Chilliwack (3) METHODOLOGY22  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVERTable 9: Recommendations to Establish Reliability and Reproducibility of Data (Lyles & Stevens, 2014)Recommendation How this study followed the RecommendationSeek to replicate existing items when relevant protocols and items have already been developed and tested and clearly identify the sources of all items, whether from previous studies, the authors’ own expertise, or some other source.Applied established protocols and referenced the study that modified certain items in a forthcoming study by Stevens and Senbel. A list of additional changes to items based on the author’s expertise are included in Appendix 2 complete with rationales.Specifically describe the scoring scheme and its application and when items are aggregated explicitly address assumptions about item weighting.The scoring scheme is identical to the established protocols and described in Section 3.5. Subcategory weighting is also explained.Clarify who coded the plans and what type of training they received.The procedure is described in Section 3.3 and explains the experience and training of the two student coders.Employ independent double coding and describe the reconciliation process used to generate the final data set used in analysis.Independent double coding is employed, with reconciliation conducted after collecting independent scores on items. The dataset to establish reliability is based on independent data collection, while scoring of the protocol items is based on consensus between the two coders. Employ pretesting of coding protocols and procedures and report the reliability of the data generated in the pretest process.A pretesting phase (combined with previous experience) was employed. Reliability data is reported in the Findings Section. Appendix 3 contains Intercoder Agreement and Krippendorff’s alpha scores for each item in the protocols.Continue to identify the region and sampling, but make clearer how plans were obtained so that other researchers can more easily create a replication sample.Section 3.1 clearly delineates the plan selection process, explains how plans were obtained, and suggests methods to retrieve plans even under circumstances where they are superseded by newer plans.Assess the reliability of all coding items using Krippendorff’s alpha in lieu of, or in addition to percentage agreement, and report the individual item reliability statistics, or at least a range for the items included in the publication.Appendix 3 contains Krippendorff’s alpha and intercoder agreement scores for each item in the protocols. Items included in the analysis are highlighted. It also includes the lowest threshold of standards. (0.33 and 80% respectively).(Lyles & Stevens, 2014)(3.4) Reliability Plan quality evaluation literature have employed a relatively similar methodology over the years. One particular concern is the inconsistent reporting of data reliability (Stevens et al., 2014). Reliability is the extent to which a given result can be reproduced using the same measures. Low reliability may indicate questionable data or inconsistent techniques and this undermines confidence in conclusions. Despite a systematic approach, plan quality evaluation remains somewhat subjective – particularly intercoder reliability (the degree of agreement between two coders). This has lead researchers to recommend Krippendorff’s alpha as best practice to for determining reliability (Stevens et al., 2014). This study followed the recommendations outlined in Stevens et al. (2014). By adopting a consistent method of reporting reliability, studies can be meaningfully compared across time, place, and contexts. No planning evaluation studies have used Krippendorff’s alpha thus far so the reliability data in this study has provided an important reference point. As per recommendations in Stevens et al. (2014), this study will present reliability findings for each item in the protocol which is contained in Appendix 3. Lyles and Stevens (2014) examined the research methods used in 45 planning evaluation publications to determine whether they followed recommended practices for content analysis. They conclude that progress in developing theories about plan quality is limited due to the shortcoming and gaps in methodology, particularly due to a lack of thoroughness and transparency on the part of researchers and a lack of repeated testing and refining of existing protocols. Lyles and Stevens suggested seven recommendations to improve the reliability and reproducibility of their data (Table 9). This study endeavours to follow each recommendation and an explanation of how this study has attempted to meet each of these recommendations is also included in the table.(3) METHODOLOGY AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 23 The Internal and External Quality protocol was scored by adding the bimodal scores (1 = present, 0 = absent) by each dimensions or subcategory. The scoring scheme differed from the original approach by Berke et al. (2006), so comparisons to scores from other studies are limited. If more than 50% of the items included in a subcategory were removed for reliability reasons, then the entire subcategory was removed. This resulted in the Fact Base: Spatial Design and the Goals and Policy Framework subcategory being dropped. An additional reason was municipalities generally do not use the categories of “goals” and “actions” consistently enough for meaningful comparison. Scoring the Communicative and Persuasive Quality protocol was achieved by collapsing the paired “YES” and “SOMEWHAT” items into the original score categories (4 = Yes, 2 = Somewhat, 0 = No). For pairs that had one item below the reliability threshold, both items were retained and combined. There would be a negative effect on interpretation if items in these pairs were dropped instead of retained. It is possible to compare scores on this protocol with plans from other studies due to the scoring remaining true to the original method, but doing so is beyond the scope of this study. Subcategory scores for each protocols are included in Appendix 4. The Internal and External Quality protocol is broken down by category and item, with frequency referring to the number of municipalities that reported the presence of the item in their plans. Similarly, the Communicative and Persuasive Quality protocol includes the frequency of municipalities reporting “Yes”, “Somewhat” or “No” for each item (with a few exceptions for certain items). Item scores on the Online Communication protocol are included with frequencies for each municipality. Total scores were calculated for each protocol for each municipality. However, a more nuanced analysis was sought by examining the subcategory scores. Subcategories are commonly used as the basis for comparison and are considered best practice in the literature. Norton (2008) contends that specific analyses such as correlations between subcategories or between different protocols is not as important. What is more important is the total subcategory scores compared against one another. As each subcategory contains a different number of items, each must be adjusted for equal weight in order to achieve an balanced total score. A table of total scores by subcategory and municipality is in Appendix 5.(3.5) Scoring At the thresholds recommended by Stevens et al. (2014), too many items would be excluded due to low reliability. They recommended classifying each protocol item into one of four categories based on how discrete/distributed the item could be located within each plan and how few/many items could be located within each plan. This requires familiarity with applying each item and working knowledge of typical planning documents and arguably introduces a degree of subjectivity to every single protocol item used. In addition, it was recommended that special baseline and leading-edge items were identified that were not to be excluded from analysis. Baseline items are those that are likely to be present on most or all plans. Leading-edge items are features that the researcher believe should be included in a plan, and represent advances in planning practice. Selecting baseline and leading edge items introduced another degree of subjectivity to the study. Ideally, baseline and leading edge items would not vary between protocols, but it opens up opportunity for researchers to debate which items should be considered baseline and leading-edge. Subsequently, this study intended to use a threshold of 0.33 score or greater on Krippendorff’s alpha and forego identifying baseline or leading-edge items as the standards for inclusion. Despite this compromise, it was found too many items would be excluded such that the protocol subcategories would not be sufficiently representative of plan quality dimensions. The criterion that is recommended is relatively high compared to existing standards used in other plan evaluation studies. In addition, items that were deemed reliable according to the criteria (scoring >0.33 using Krippendorff’s alpha) often did not correspond to percent agreement. This incongruency combined with the high threshold for inclusion ultimately mean this study used the traditional standard for reliability, percent agreement (>80%) as the threshold for inclusion. The full list of items with Krippendorff’s alpha and percent agreement scores is included in Appendix 3. Out of 93 items on the Internal and External Quality Protocol, 45 was included in the final analysis. The Communicative and Persuasive Quality protocol has 71 items, 38 of which were included in the final analysis. All items in the Online Communication protocol were included, with 100% agreement. Reliability information was not included for the two Online Communication subcategories, due to complete agreement between coders.(3) METHODOLOGY24  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER(4) FINDINGSThis section reports descriptive statistics and noteworthy findings. It first reviews the descriptive statistics from each protocol, followed be a separate section with noteworthy findings within the subcategories of each protocol. This section ends with a list of strengths and weaknesses of municipal OCPs in Metro Vancouver based on how well they performed on the protocols.(4.1) Descriptive Statistics It would be premature to hold up high-scoring municipalities as examples of best practice in plan making. Figure 1 below depicts Internal and External Quality Scores cumulatively using standardized scores so that each dimension can be compared against one another. This values may range from 0.0 (not present) to 1.00 (perfect representation). These seven values can be added up for a total score out of 7. Notice that each municipality has distinct strengths and weaknesses within each component. The average raw score was 18.75 out of a possible total score of 45 (41.6%). The standardized scores are a little more favourable, with an average total score of 3.53 out of 7 (50.4%). The District of West Vancouver, Anmore, and Coquitlam the highest standardized scores, of 4.6 out of 7 (65.7%). Lions Bay and Langley are tied at the bottom with 1.7 out of 7 (24.2%). This chart also depicts components that municipalities generally scored the highest in, with Issues and Vision Statement represented in most plans and plan proposals noticeably underrepresented. 0.00.51.01.52.02.53.03.54.04.55.0D. of West VancouverAnmoreCoquitlamBowen IslandRichmondPort MoodyD.of North VancouverDeltaWhite RockPitt MeadowsNew WestminsterNorth VancouverPort CoquitlamBurnabySurreyMaple RidgeBelcarraLangley TownshipLions BayLangleyParticipation of Actors Account for Interdependent Actions in Plan ScopeCreate Clear Views and Understanding of Plans Encourage Opportunities to Use PlanPlan Proposals Fact BaseIssues and Vision StatementFigure 1: Internal and External Plan Quality Scores (Cumulative by subscale)(4) FINDINGS AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 25 Figure 2 depicts scores on the Communicative and Persuasive Quality protocol. The two subcategories were standardized with value range from 0.0 (not present) to 1.00 (perfect representation). The maximum possible raw score was 92, and the maximum standardized score was 2. The average score was 0.68 out of 2, (34%). Anmore and North Vancouver had the highest score of 1.07 (53.5%) and 0.96 (48%) respectively. Pitt Meadows had the lowest score of 0.27 (13.5%), followed by Lions Bay with 0.42 (21%). This chart also shows that both subcategories appear about equally represented across OCPs. The online Communication Protocol consists of two subcategories of 7 items for a maximum score of 14, as depicted in Figure 3. Since both subcategories were equal in size, there was no need to standardize the scores. 0.00.20.40.60.81.01.2Format, Style, and AppearanceCriteria Related to Readability, Synthesis and Quality of Presentation of Information, NarrativeQuality, Persuasiveness and Realism of Plans0.00.20.4..1.01.2Format, Style, and AppearanceCriteria Related to Readability, Synthesis and Quality of Presentation of Information, NarrativeQuality, Persuasiveness and Realism of PlansFigure 2: Communicative and Pursuasive Quality Scores (Cumulative by subscale)02468101214Municipal Website PDF AttributesFigure 3: Online Communication Quality Scores (Cumulative by subscale) The average score was 8.2 (58.6%). Coquitlam has the highest score at 12 (85.7%), followed by Burnaby, Maple Ridge and Pitt Meadows tied for 11 (78.6%). Langley Township, Belcarra, and the District of North Vancouver are tied for lowest scores at 5 (35.7%). Both subcategories are roughly equally represented by the municipalities. There is no clear relationship between the top scoring and bottom scoring municipalities between each protocol. Municipality population does not play a factor in plan quality. Anmore and Burnaby scored among the highest on two of the protocols. Pitt Meadows, Belcarra, and Langley Township scored among the lowest on two or more of the protocols. Appendix 4 contains frequencies by item for all three protocols.(4) FINDINGS26  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER(4.2) Internal and External Quality To recap, Berke et al. (2006) divide plan quality into two conceptual dimensions, internal and external quality. Internal Plan Quality refers to the content and format of the plan, and consists of 32 items across three components, Issues Vision Statement, Fact Base, and Plan Proposals. External Plan Quality refers to the relevance, scope, and coverage of the plan, and consists of 13 items across four components, Recognize and Encourage Opportunities to Use Plan, Create Clear Views and Understanding of Plans, Account for Interdependent Actions in Plan Scope, and Participation of Actors. Interestingly, there was not a substantial difference in scores between Internal and External plan quality. The average standardized score for Internal plan quality was 0.515 and External plan quality had a standardized score of 0.490. No municipalities stood out as exceptional examples of external or external plan quality. Table 10 summarizes these findings. The Issues and Vision Statement component identifies major assets, trends, impacts forecasted for the community, a description of major opportunities, threats, problems, and issues, as well as vision statement. Most municipalities scored quite well and this component had the overall highest score, with a standardized score of 0.82. Two of the five items were dropped due to low reliability, and subsequently these findings should be interpreted cautiously. Nearly most municipalities had an adequate assessment of major trends and impacts of forecasted change during the future planning period. More interestingly, five municipalities (District of West Vancouver, Langley Township, Langley, and New Westminster) did not have vision statements. The fact base component identifies present and future projections for population, economy, land use, land supply, facilities, and infrastructure. It also assesses the state of the natural environment with facts supported using maps, tables, reference data, with methods and models. The component had 13 items, and was divided into two subcomponents, Description and Analysis of Key Features of Local Planning Jurisdiction and Techniques Used to Clearly Identify and Explain Facts. Municipalities had an overall average score of 0.50. All municipalities described present and future population sizes, which is expected because it is a required under Strategy 1.1 of Metro Vancouver’s Regional Growth Strategy (2011). Anmore, District of West Vancouver, and Langley scored below average in fact base, with no municipality standing out as an exceptional example. The  Plan Proposal component describes the future form of the community and outlines an implementation program. It describes a program for monitoring and evaluating along with means to update and adjust the plan along the way. It includes aspects of infrastructure, transportation, open spaces, land use, water, sewer and more. Specific “actions” to achieve these should be prioritized and included on a timeline along with identifying who is responsible for carrying out the actions. The Plan Proposals component was the largest dimension with 14 items divided into implementation and monitoring subcategories. The third category, spatial design was removed due to low reliability across most of the items. Municipalities scored very low on this dimension with an average standardized score of 0.16. Several OCPs (Burnaby, Langley, Lions Bay, and Pitt Meadows) were missing an implementation section altogether.Table 10: Internal and External Quality Raw and Standardized ScoresDimensionNumber of ItemsAverage Raw ScoreAverage Standardized ScoreInternal Plan QualityIssues and Vision Statement 5 2.45 0.82Fact Base 13 7.45 0.50Plan Proposals 14 2.20 0.16Total 32 6.05 0.52External Plan QualityRecognize and Encourage Opportunities to Use Plan2 1.05 0.53Create Clear Views and Understanding of Plans6 3.00 0.50Account for Interdependent Actions in Plan Scope2 1.00 0.50Participation of Actors 3 1.60 0.53Total 13 6.65 0.49 Grand Total 4518.75 / 45 (41.6%)3.53 / 7 (50.4%)(4) FINDINGS AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 27interpretation. Metro Vancouver municipalities generally scored well on their inclusion and mention of other local plans and programs (Reference to Metro Vancouver was not counted, as it was captured by its own item). The other item, intergovernmental coordination for protecting natural hazards was very low across municipalities, with only Coquitlam, Port Coquitlam, and Port Moody including any mention of coordination in this regard. The last component, Reveal Participation of Formal and Informal Actors (or Institutions) refers to plans that should explain who was involved in making the plan, how they participated, and how the plan evolved in response their participation. This includes government agencies, the private sector, and individual citizens. This component consists of three items, with another three removed due to low reliability. Municipalities scored an average of 0.53. Disappointingly, municipalities did not consistently include participation sections. Five municipalities scored a zero on this important section, including Langley, Lions Bay, Maple Ridge, Port Coquitlam, and Surrey. This will be discussed in the next section. One component of the external plan quality criteria was dropped entirely. The component Goals and Policy Framework, identifies goals and policies and how they are linked to community values, problems, and aspirations. Reliability across nearly all the items in this component was very low. This was due to difficulty with consistently identifying goals and actions across the OCPs. Most municipalities did not consistently use the same terminology for goal or actions and some did not have any policies that could be labeled as goals or actions. Items were added to this component by Stevens and Senbel’s forthcoming study in order to capture a more nuanced and comprehensive assessment of goals and policies. The discussions section will elaborate on these findings. The component Recognize and Encourage Opportunities to Use Plans includes certain attributes that enhance the chance a plan will be used. Some of these include being inspirational (offers compelling courses of action), action-oriented (clearly articulated action-oriented agenda), flexible (allows for alterative courses of action), and legally defensible (includes explanations of legislative and administrative authority). This component was not well represented and poorly captured in the analysis. The original Berke et al. (2006) protocol included three items to represent this component. Stevens and Senbel’s forthcoming study dropped one item due to low reliability and this study used the same protocol they adopted. Municipalities typically scored high on one item, and low on the other. The average quotient was .53. One item, future scenarios (“Does the plan include at least two explicit alternative future growth scenarios?”) was absent from nearly all plans except Anmore. The other item, administrative authority (Is the administrative authority for planning indicated? Does the plan mention the required elements of an OCP as outlined in the LGA?) was modified for this study, and was present in 19 out of the 20 plans. The Communicative and Persuasive Protocol developed by Bunnell and Jepson (2011) better captures this dimension. The Create Clear Views and Understanding of Plans component refers to the relevance and accessibility of plans. Plans should be understandable from the perspective of all governmental units (county-level, district-level) and types of readers (citizens, elected officials). A plan that is readily accessible in this way enhances the potential for creating awareness and support for the public interest and could enhance implementation. Municipal OCPs in Metro Vancouver scored in the middle range on this component, with an average of .50 across 6 items. Nearly all plans included illustrations but at the same time all plans were missing executive summaries. One new item was added to this component that addressed whether plans had a list of amendments. This item will be discussed in the next section. The component, Account for Interdependent Actions in Plan Scope refers to plans that should communicate the necessity for coordinating and communicating among different organizational departments/units in order to maximize potential for implementation and influence. There should be recognition of the range of possible independent actions possible to carry out policies. Two items were removed from this component, leaving two items with municipalities scoring an average score of 0.50. Subsequently, caution should be taken with (4) FINDINGS28  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER(4.3) Communicative and Persuasive Quality This protocol was grouped into two major categories for analysis. One was pertaining to the readability, synthesis, quality of presentation, narrative, persuasiveness, and realism of plans and the other is related to attributes of format, style, and appearance of plans. Protocol items were also examined with respect to four dimensions: standardization/rigidity, acknowledgement of uncertainty, understanding of alternatives conveyed, and compelling narrative storyline (Hopkins and Zapata, 2007). Table 11 summarizes the major findings by subcategory. Municipalities that scored high in one subcategory tended to score high in others. There was a small difference in average scores (31% and 37%) across the subcategories outlined by Bunnell and Jepson (2011). This difference was significant (p = 0.026, two-tailed) meaning municipalities scored better in format, style, and appearance than readability, synthesis, quality of presentation, narrative, persuasiveness, and realism. These subcategories are not as clearly conceptually defined as the ones described in Berke et al. (2006), and further discussion will instead focus on individual items. One category under the Hopkins and Zapata (2007) protocol was dropped due to the items having low reliability (compelling narrative storyline). These items are arguably capture more innovative qualities and examples of best practices, and plans could be expected to score quite low on these. In general, OCPs in Metro Vancouver scored very low on these components of plan quality. The two items that capture standardization/rigidity seek to identify aspects of plans that break away from a predictable and ridged format. It is argued that such a format would not inspire or stimulate the reader’s imagination (Bunnell & Jepson, 2011). The two items were as follows: “Is the plan imaginative and creative (extent of commitment to preparing a meaningful, effective plan)?” and “Does the plan put forward a compelling vision (through illustrations, photographs, maps, and words) of what the future could be like?”. Both items are more subjective, yet they had high enough agreement to be used in the analysis. All municipalities scored between “somewhat” and “no” for these two items. Acknowledgement of uncertainty had extremely low scores with the exception of Anmore, which had a perfect score. These three items identified plans that included alternate scenarios, compared these scenarios, and explained alternate courses of action. Broadly speaking, municipalities in Metro Vancouver did not incorporate alternate scenarios into their community plans. Further discussion will be included in the next section. The category, understanding of alternatives conveyed consisted of three items. It examined the extent that plans conveyed alternative courses of action. Like the previous category, if a plan was missing one aspect of this, it was very likely missing all three aspects. This concept appears to be closely related to acknowledgement of uncertainty, as Anmore was the only municipality to score satisfactorily on these items (6/12). Further discussion will be included in the next section.Table 11: Communicative and Persuasive Quality Raw and Standardized ScoresDimensionNumber of ItemsAverage Raw ScoreAverage Standardized ScoreBunnell & Jepson (2011)Readability, Synthesis and Quality of Presentation, Narrative Quality, Persuasiveness and Realism18 22.3 0.31Format, Style, and Appearance 5 7.40 0.37Total 2329.7/92 (32.3%)0.68/2 (34%)Hopkins & Zapata (2007)Standardization/rigidity 2 2.00 0.25Acknowledgement of uncertainty 3 0.70 0.06Understanding of alternatives conveyed 2 2.10 0.18Compelling narrative storyline 0 n/a n/aTotal 74.8/28 (17.1%)0.48/3 (16%)(4) FINDINGS AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 29(4.4) Online Communication The Online Communication protocol was developed by the author and is intended to be an extension of Bunnell and Jepson’s (2011) approach to communicative and persuasive qualities of plans. This protocol consisted of 14 items, 7 that examine the municipal website, and 7 that examine the attributes of the plan’s PDF. Together, these subcategories represent the accessibility and ease of use of planning documents downloaded online, which is the primary method of obtaining planning information.  There was no significant difference of scores between the municipal website subcategory and the PDF attributes. The average score across OCPs was 4.00 for municipal website, and 4.20 for PDF attributes. There were no ceiling or floor effects of these protocol items, meaning none of the municipalities scored perfect or a complete zero on either subcategory. All told, this protocol serves as a useful extension to the Bunnell and Jepson (2011) protocol. Table 12 summarizes the scores.Table 12: Online Communication Quality Raw ScoresDimension Number of Items Average Raw ScoreBunnell & Jepson (2011)Municipal Website 7 4.00PDF Attributes 7 4.20Total 7 8.20 / 14 (58.6%)(4) FINDINGS30  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER(4.5) Overall Strengths and Weaknesses The following is a list strengths and weaknesses of municipal OCPs in Metro Vancouver based on overall scores on protocol items and subcategories. It is not recommended to compare a single OCP against these strengths and weaknesses, but it serves as an overall indication of what Metro Vancouver municipalities are focusing on and what areas are being overlooked and are in need of improvement. Strengths of Municipal OCPs in Metro Vancouver:•  Including a vision statement•  Describing major trends and impacts through qualitative and quantitative means•  Including current and future population size•  Describing current and future housing supply•  Describing vegetation and forests in community•  Referencing the administrative authority to plan (mentioning the LGA)•  Making connections and references to Metro Vancouver and other governmental bodies•  Including illustrations, diagrams, and pictures•  Including clear and relevant maps and tables•  Citing data sources•  Providing a clear and direct link to download the plan from the municipal website•  Ensuring the plan PDF is text searchable and downloadable as a single documentWeaknesses of Municipal OCPs in Metro Vancouver:•  Including an executive summary (absent from all plans)•  Describing air and water quality•  Including current and future population demographics•  Including goals/policies that are quantifiable and measurable•  Demonstrating intergovernmental coordination for protecting natural hazards•  Including timelines for updating the plan•  Analyzing land demand and supply•  Describing, comparing, or explaining alternate scenarios or forecasts for growth•  Describing actions, priorities, timelines, and identifying • responsible organizations within the implementation section•  Identifying organizations for monitoring•  Clearly organizing a hierarchy of goals > objectives > policies•  Describing how the plan was developed (e.g. through public consultation) on the municipal website(4) FINDINGS AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 31 In Metro Vancouver, there were no clear examples of OCPs that had an outstanding comprehensive fact base. In well-staffed and funded planning departments, much of this basic information is not difficult to obtain. Municipal planning departments likely have the information available but may overlook including it in the OCP. Despite the limited land base in Metro Vancouver, municipalities also did not generally address how land demand and supply would affect or accommodate future growth. This involves a more in-depth analysis which may be beyond the resources of the planning department. Anmore was the only municipality to analyze different scenarios for future population growth and the number of residential units needed. Figure 4 contains a summary table from Anmore’s OCP. Interestingly, this was the only municipality that included multiple consultant teams in the development of the plan. As this is a small municipality, outside experts were recruited to provide technical analysis, and this may be the reason that growth scenarios were included at all.This section summarizes major findings and identifies key attributes of municipal OCPs in Metro Vancouver that are in need of improvement. Outstanding examples of good practice are described, followed by recommendations, an explanation of limitations, and avenues for future research.(5.1) Summary The purpose of this project was to apply three plan evaluation protocols to municipal Official Community Plans in Metro Vancouver. The Internal and External Quality protocol examined the content of the individual plan components, relevance, scope, and coverage of the plan in fitting with the local context. The second protocol examined the Communicative and Persuasive Qualities of plans. The third assessed the quality of Online Communication of planning information though examining the municipal website and attributes of plan’s PDF format. These municipalities share a common policy base through the Metro Vancouver Regional Growth Strategy. Combined with high levels of public engagement, local interest in planning issues, plus a small geographic area, the region has potential to produce leading edge municipal plans and may share similar strengths and weaknesses. By evaluating OCPs in the region, this project seeks to identify attributes of municipal plans that have been overlooked, weakly addressed, or are in need of improvement. Due to these shared attributes, municipalities may be able to borrow and adopt best practices from one another. In the course of this analysis, six major themes emerged for discussion. These themes are described as fact base, policies, implementation, stakeholder engagement, clarity and accessibility, and persuasive and inspiring.(5.1.1) Fact Base The fact base should describe and analyze key attributes of the plan such as demographics, environmental features, and future demand for facilities and infrastructure. This serves as an important basis for any policies and implementation that follow. It is also important for plans to provide essential descriptive information about the community to the readers, some of whom may not be familiar with the community.(5) DISCUSSIONFigure 4: Excerpt from Anmore’s Growth Management Strategy(5) DISCUSSION32  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER(5.1.2) Policies Organizing policies under a hierarchy of goals and objectives is essential for clearly and logically organizing a policy document. Doing so demonstrates how policies are linked to the challenges in the community as well as reflects the community’s values and aspirations. In Metro Vancouver, municipalities did not universally apply the same terminology for goals and objectives. Typically, there are a handful of goals, dozens of objectives, and hundreds of policies. Many did not include any goals or objectives and described policies without any kind of internal hierarchy or description of how policies were prioritized or organized. The Local Government Act requires OCPs to contain objectives and policies but provides no other instructions of how to organize and prioritize them (Local Government Act, c. 471). Table 13 depicts the number of goals and objectives that were used to organize policies for each municipality. Notice that nine municipalities did not use the term goals and five did not use the term objectives. Municipalities using the term goals were relatively consistent in applying it to the highest-order and most general view of the community’s desires. Bunnell and Jepson (2011) awarded more points on their protocol to municipalities that had fewer than 12 goals. The number of goals considered desirable is open to debate, but it appears appropriate from the perspective of this analysis. Stevens (2013) found goals and objectives one of the stronger attributes of municipal OCPs in Southern British Columbia (which included the same municipalities as this study). Objectives are more specific and concrete and have aims that are subordinate to goals. Most municipalities used objectives in their OCPs, but some applied them more generously while others treated them like goals. Bunnell and Jepson (2011) awarded the most points to plans with fewer than 20 objectives, fewer points from 20-40, and zero points to greater than 40 objectives. From the viewpoint of this analysis, this number may be too low, as goals are likely divided into more than 2-3 objectives. Objectives should not come close to outnumbering policies, and having over 150 (as in the case of three municipalities) is likely too many in the view of this author. There is inconsistency and misuse of the hierarchy that organizes goals, objectives, and policies in Metro Vancouver municipalities. Policies within the Regional Context Statements were often organized under goals, objectives and actions. However, elsewhere in the plan, goals, objectives, and actions were not repeated or applied. Municipalities vary widely in the degree of integration of their regional context statements into their OCP (DeSousa, 2015). It is possible that as municipalities update or draft new OCPs in the future that it will better integrated with Metro Vancouver’s regional policies and use a similar hierarchy of goals and policies.“Goals are broad expressions of the desired future conditions of a community. They can initially be derived in the vision statement, but can be followed up by more in-depth analysis of needs and aspirations. Policies are established principles to be followed in guiding public and private decisions to achieve a desired future land use and development pattern.” (Berke, Godschalk, & Kaiser, 2006, pp. 71).Table 13: Organization of Policies by MunicipalityMunicipality Goals ObjectivesAnmore 0 23Belcarra 0 0Bowen Island 12 194Burnaby 24 0Coquitlam 0 74Delta 6 71D.of North Vancouver 12 45D. of West Vancouver 0 85Langley Township 6 58Langley 0 0Lions Bay 0 0Maple Ridge 0 113New Westminster 81 143North Vancouver 5 195Pitt Meadows 0 181Port Coquitlam 0 9Port Moody 24 0Richmond 4 106Surrey 9 31White Rock 9 67(5) DISCUSSION AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 33(5.1.3) Implementation Implementation and monitoring sections are important elements of policy documents as they describe how the policies in the document are going to be achieved. Strong implementation sections not only describe what is to be implemented, but identifies the groups responsible (departments, other governmental organizations, Public-Private Partnerships etc.), a timeline with short/medium/long -term actions and objectives, measurable objectives and indicators, funding strategies, and a description of how progress will be monitored and tracked. It lends a sense of legitimacy and commitment to the policies laid out in the plan, and allows stakeholders to easily review these priorities and timelines for their own decision making purposes. It is also helpful to readers if the section describes how the OCP is connected to other plans (Metro Vancouver, Neighbourhood Plans, Strategic Action Plans, Housing and Homelessness Plans etc.). Metro Vancouver municipalities scored very low on implementation and several (Burnaby, Langley, Lions Bay, and Pitt Meadows) were missing implementation sections altogether. This is congruent with findings from other studies that examined municipalities in British Columbia (Stevens & Mody, 2013; Stevens, 2013; Stevens & Shoubridge, 2015). Note that some municipalities had implementation sections within their Regional Context Statement, but these sections were often inserted as an appendix or placed as a separate section at the beginning of the OCP, and did not integrate very closely with the rest of the document. Content contained in the Regional Context Statements were not included in this analysis. Two municipalities in Metro Vancouver adequately addressed implementation. Figure 5  is part of a table of District of North Vancouver’s implementation section which contains measurable baseline and target indicators. Figure 6 is part of a table from White Rock’s OCP that contains time frames and organizations identified to lead the implementation of the action item. Nether municipality is a perfect example of an implementation section, but both contain very desirable attributes that together should be replicated in other municipalities.Figure 5: District of North Vancouver OCP Implementation SectionFigure 6: White Rock OCP Implementation Section(5) DISCUSSION34  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER(5.1.4) Stakeholder Engagement All municipalities must provide one or more opportunities for consultation with people, organizations, and authorities that would be affected by the plan (Local Government Act, c. 475). Official Community Plans are intended to represent the values and objectives of the community. When plans demonstrate that they have involved the community in the development of the plan, this adds legitimacy to the planning document. Planning in multicultural Canadian cities requires special attention at consultation and outreach stages of planning (Qadeer, 2007). It is important to included disadvantaged members of the community in the process, as they are not as powerful, organized and well-funded as other stakeholders (Baer, 1997). Figure 7: District of West Vancouver OCP Plan Review Process Section Documenting the planning and consultation process is also important because the public is knowledgeable and active about planning issues in their community. Many citizens are active in the local democratic and political processes and are more than willing to point out areas of oversight made by their local government. Municipalities can likely circumvent future political challenges by thoughtfully documenting the consultation process at all stages. Metro Vancouver municipalities demonstrate substantial room to improve their stakeholder engagement and planning process sections. Five municipalities were missing participation sections altogether, including Langley, Lions Bay, Maple Ridge, Port Coquitlam, and Surrey. However, several municipalities should be held up as examples of best practice in communicating community involvement and describing the planning process. The municipalities of Anmore, District of North Vancouver, District of West Vancouver, and Richmond are examples that other municipalities should emulate. Figure 7 contains a page from the plan review process of the District of West Vancouver’s OCP.(5) DISCUSSION AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 35(5.1.6) Persuasive and Inspiring Plans represent shared goals of the community and are intended to guide future growth and development in a manner that is beneficial for all community members. Municipal policy represents substantial financial commitment and focusing of resources, and if followed, can represent significant positive change for community members. Bunnell and Jepson (2011) contend that preparing persuasive plans are important for local governments. They are led by elected officials who are representatives of the public and serve as conduits of public opinion. Planners must channel the desire of the public and politicians into actionable directions for the future. Doing so requires sophisticated techniques to induce commitment and legitimacy from politicians and stakeholders. Metro Vancouver municipalities scored an average of 34% on the communicative and persuasive qualities protocol. The protocol included a number of items about alternate scenarios. Mentioned earlier, municipalities were missing this element from most of their plans which contributed to an overall lower score on the protocol. Five plans were scored somewhat on “exhort and inspire people to act” while the remaining 15 scored a zero on this item. It is clear that municipalities need to work on these attributes. Five municipalities (District of West Vancouver, Langley Township, Langley, and New Westminster) did not have vision statements. Vision statements are often the most obvious and direct method to use inspirational and motivating language. Municipalities must not overlook this important attribute, no matter how lofty and abstract vision statements appear to sound.(5.1.5) Clarity and Accessibility An important aspect of plan quality is ensuring the information is presented and communicated clearly and that readers can easily access and navigate the planning document. Baer (1997) recognized quality of communication and plan format as significant components of plan quality. Anecdotally speaking, more elaborate graphic design and presentation effort is going into planning documents in recent years. This may be due to several reasons:1)  Increasing technical aptitude of planning staff2) Better internet download speeds thereby enabling OCPs to integrate higher-resolution graphics and longer documents3)  Increasing interest in planning issues by the public may signal to planning staff that better visual communication is needed With a few exceptions, municipalities in Metro Vancouver generally have well organized and easy to navigate municipal OCPs. The most common shortcomings were missing executive summaries and list of amendments. One new item was added to the Internal and External Quality Protocol that addressed whether plans had a list of amendments. This item was added was because it was believed it was something that should be present in all plans. By including a list of amendments, it communicates to readers that the plan is a “living” document and not frozen in time. It is important to signal to readers that planning is an ongoing process and the municipality has kept up to date with documenting the amendments to the plan over time.  The Online Communication protocol was specifically designed to assess clarity and accessibility of the digital content of OCPs. The largest finding of this measure is municipalities have poor representation of planning process on the municipal website. Only four municipalities included an online description of the process and consultation that was involved in developing the OCP. Minor attributes of the plan’s PDFs were considered weak, such as utilizing the embedded bookmark feature to help organize the planning document, embedding shortcut links in the table of contents, and matching the PDF page numbering to the document page numbering. Maple Ridge’s OCP is an example of a well-laid out and organized planning document that made full use of the PDF capabilities (that being said, it is missing a front page and the table of contents does not have links to corresponding planning sections). No plan stood out as an exceptional example of online communication.“Plans will be used more widely if they are inspirational. If plans are imaginative and offer compelling courses of action that inspire people to act for the common good, then they have greater potential to change attitudes and beliefs, and encourage harder work and stronger commitment to mobilize resources” (Berke et al., 2006, pp. 73)(5) DISCUSSION36  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER Persuasive and inspiring attributes of plans are difficult to identify and measure, but several examples are noteworthy. Figure 8  is the second page of the city of Richmond OCP that includes artwork by a high school student followed by an explanation of youth involvement in the development of the OCP.  Language such as “Our Village’s Vision” (contained in Anmore’s OCP) subtly communicate to readers that it is a shared document. Inserting comments from the public (credited or anonymous) about hopes, concerns, and observations about the community could also communicate shared responsibility and inspire readers. Municipalities could organize a photography or art contest for community members for inclusion in the OCP. The OCP would credit the photographer/artists who contributed and communicated to readers that the plan was not just a top-down effort and that friends and neighbours were involved in making the document.  Municipalities should use examples such as these to communicate that “we did this together”. Although not overtly and factually persuasive, these opportunities go a long way for inspiring and fostering buy-in from community members and politicians.Figure 8: Page 2 of the City of Richmond’s OCP(5) DISCUSSION AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 37(5.3) Limitations and Future Research Despite best efforts to adhere to the admirable recommendations of Stevens et al. (2014), this study resorted to using intercoder agreement, a less rigorous method of determining reliability of protocol items. The reliability standards for inclusions using Krippendorff’s alpha were too high to allow a reasonable examination of the plans in this study. Plan evaluation studies have been criticized for a lack of details pertaining to methodology, so this study attempted meet these expectorations as best as possible. Full reliability data is included in the methodology section and the appendices. As Baer (1997) reiterates, a clearly articulated and documented plan may not have any bearing in the real world. It might have a distorted (intentionally or unintentionally) view of how things are and how things should be. A plan that scores highly on evaluation criteria may be mean very little. While difficult to verify or disprove, planning and planning evaluation is conducted with the optimism that the future can be shape in a positive way. Future research could compare OCP in Metro Vancouver to other regional district in British Columbia and the rest of Canada. Examining plans for grade-level readability could also be a new avenue for investigation. Ferguson (2014) found that municipal web pages were written at grade levels higher than the average reading grade level in the United States. A more in-depth analysis of social media integration of planning information could shed light on the extent (and depth) of public engagement. Further, this study excluded the City of Vancouver – one of the most extensively planned municipalities in Metro Vancouver. Future research could adapt the evaluation protocol to the neighbourhood plans in the City of Vancouver. Lastly, interviews with planners could shed light on the priorities municipalities have for inclusion in their OCP as well as reveal why certain decisions were made about aspects of public engagement, plan design, use of language, and integration with regional policies.(5.2) Recommendations Evaluating the quality of municipal plans using three different protocols has revealed many opportunities for improvement. Foremost, municipalities should draw on the practices used by their neighbours. Other municipalities are a rich resource for ideas and municipal planners can quickly identify missing elements from their own plan by reviewing other municipal OCPs. This is particularly true for municipalities in this study as they share the same mandates under the Metro Vancouver Regional Growth Strategy. The following is a list of recommendations ordered from easiest to implement to most difficult.• Ensure the municipal website includes the five basic questions to a complete story (Who, What, When, Why, and How) when describing the OCP.• Optimize the PDF when creating the OCP for digital upload. It does not take extensive technical expertise to add small and useful features to the PDF to increase usability and appeal.•  Be comprehensive in the amount of basic factual information about the municipality (demographic information, present state of services and infrastructure, etc.). Many small facts (sometimes obvious to municipal planners) can add up to a comprehensive picture of the current state of the municipality and be helpful to people not already familiar with the community.• Organize policies under a hierarchy of goals and objectives to help clearly and logically organize the OCP. Doing so demonstrates how policies are linked to the challenges in the community as well as reflects the community’s values and aspirations.• Better integrate the goals, objectives, and policies outlined in the municipality’s Regional Context Statement throughout the sections of the OCP document.•   Explain and compare alternate scenarios or forecasts for housing and population growth.•   Include specific actions that are quantifiable, measurable, and comparable to an earlier baseline period.• Analyze land demand and supply and highlight the challenges and opportunities the municipality faces for future growth.(5) DISCUSSION38  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER(6) CONCLUSIONMunicipalities in the Metro Vancouver region face many planning challenges. Planners must coordinate growth through cooperation with regional mandates, grow within a limited land base, and consult with the public who have become increasingly sophisticated and active about planning issues. This is in addition other issues such as responding to climate change, demographic shifts, housing affordability, and building and maintaining services and infrastructure. Municipal planners and city councils are guided by official community plans when making decisions such as these about their community.The purpose of this project has been to evaluate municipal official community plans in Metro Vancouver in order to identify strengths and weaknesses. Plan evaluation was uncommon until recently due to a lack of consensus on what constitutes a good plan, evaluation protocols that were difficult to validate and generalize, and planning documents that were difficult to obtain. This project evaluated the quality of municipal plans using three different protocols. This study identified significant gaps and areas of oversight. For example, not all municipalities have vision statements and some lack participation and plan development sections. Nearly all municipalities did not forecast different growth scenarios as well as integrate measurable objectives and actions. Implementation sections of most plans were shallow and didn’t provide sufficient guidance for future actions and monitoring. Finally, many municipal websites were missing essential communicative information about their OCPs.A series of recommendations were developed and examples of best practices were identified. Municipalities are encouraged to learn and borrow from one another. There are ample opportunities for municipalities to improve the quality of their official community plans. Planners must be willing to take a critical eye to improve planning practices within their own municipality. A culture of innovation and self-improvement in essential to respond to increasingly complex planning issues facing the region in the future.(6) CONCLUSION AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 39(7) REFERENCESColantonio, A. (2008) Measuring social sustainability: best practice from urban renewal in the EU. Working Paper No. 2008/02, EIBURS (European Investment Bank University Re-search Sponsorship Programme), Oxford Brookes University.Dalton, L. C. (1989). The limits of regulation: Evidence from local plan implementation in Cali-fornia. Journal of the American Planning Association 55,2: 151-68.DeSousa, S. (2015). Integrating regional policy in municipal planning: Evaluating regional con-text statements in Metro Vancouver. (Dissertation) Retrieved from https://circle.ubc.ca/ Dolson, J., & Young, R. (2012). Explaining variation in the E-government features of municipal websites: An analysis of E-content, E-participation, and social media features in Canadi-an municipal websites. Canadian Journal of Urban Research, 21(2), 1-24.Edwards, M. M., & Haines, A. (2007). Evaluating smart growth: Implications for small communities. Journal of Planning Education and Research, 27(1), 49-64. doi:10.1177/0739456X07305792Gale, D. E. (1992). Eight state-sponsored growth management programs: A comparative anal-ysis. Journal of the American Planning Association, 58(4), 425.Ferguson, D. W. (2014). Grade-level readability of municipal websites: Are they creating digital inequalities of opportunities that perpetuate the digital divide? Retrieved from ProQuest Digital Dissertations (UMI 3671087)Hamin, E. M. (2011). Integrating adaptation and mitigation in local climate change planning. In Ingram G. K. and Hong, Y. H. (Eds.) In Climate Change and Land Policies, Cambridge: Lincoln Institute of Land Policy.Henriksson, A., Y. Yi, B. Frost, & Middleton, M. (2006). Evaluation instrument for e-government websites. In Proceedings Internet Research 7.0.  Internet Convergences, Brisbane, Queensland, Australia, 1-16.Hopkins, L. D., & Zapata, M. A. (Eds.). (2007). Engaging the future: Forecasts, scenarios, plans, and projects. Cambridge, MA: Lincoln Institute of Land Policy.Isserman, A. M. (2007). Forecasting to learn how the world can work. In L. D. Hopkins & M. A. Zapata (Eds.), Engaging the future: Forecasts, scenarios, plans, and projects (pp. 175–197). Cambridge, MA: Lincoln Institute of Land Policy.Kaiser, E. J., & D. R. Godschalk (1995). Twentieth century land use planning: A stalwart family tree. Journal of the American Planning Association 61 (3): 365-85.Kent, T. J., Jr. (1964). The urban general plan. San Francisco, CA: Chandler.American Planning Institute (2016). Kudos. Retrieved from https://www.planning.org/awards/Avin, U. (2007). Using scenarios to make urban plans. In L. D. Hopkins & M. A. Zapata (Eds.), Engaging the future: Forecasts, scenarios, plans, and projects, (pp. 103–134). Cam-bridge, MA: Lincoln Institute of Land Policy.Baer, W. C. (1997). General plan evaluation criteria: An approach to making better plans. Journal of the American Planning Association, 63(3), 329-344. doi:10.1080/01944369708975926Baynham, M. & Stevens, M. (2014) Are we planning effectively for climate change? An evalu-ation of official community plans in British Columbia, Journal of Environmental Planning and Management, 57:4, 557-587, DOI: 10.1080/09640568.2012.756805Berke, P. R., & Conroy, M. M. (2000). Are we planning for sustainable development?: An eval-uation of 30 comprehensive plans. Journal of the American Planning Association, 66(1), 21. doi:10.1080/01944360008976081Berke, P. R., & French, S. P. (1994). The influence of state planning mandates on lo-cal plan quality. Journal of Planning Education and Research, 13(4), 237-250. doi:10.1177/0739456X9401300401Berke, P., & Godschalk, D. (2009). Searching for the good plan: A meta-analysis of plan quality studies. Journal of Planning Literature, 23(3), 227-240. doi:10.1177/0885412208327014Berke, P. R., Godschalk, D. R., & Kaiser, E. J., with Rodriguez, D. A. (2006). Urban land use planning, 5th ed. Urbana and Chicago: University of Illinois Press.Black, A. (1968). The comprehensive plan. In W. I. Goodman & E. C. Freund (Eds.), Principles and practice of urban planning (pp. 349–378). Washington DC: International City Man-ager’s Association.Boswell, M. R., Greve A. I., & Seale, T. L. (2010): An Assessment of the Link Between Green-house Gas Emissions Inventories and Climate Action Plans, Journal of the American Planning Association, 76:4, 451-462 http://dx.doi.org/10.1080/01944363.2010.503313Bunnell, G., & Jepson, E.J., Jr. (2011). The effect of mandated planning on plan quality. Journal of the American Planning Association, 77(4), 338-353.Burby, R. J., & Dalton, L. C. (1994). Plans can matter! the role of land use plans and state planning mandates in limiting the development of hazardous areas. Public Administration Review,54(3), 229-238.Brody, S. D., Carrasco, V., & Highfield, W. E. (2006). Measuring the adoption of local sprawl: Reduction planning policies in Florida. Journal of Planning Education and Research, 25(3), 294-310. doi:10.1177/0739456X05280546(7) REFERENCES40  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVERKlosterman, R. E. (2007). Deliberating about the future. In L. D. Hopkins & M. A. Zapata (Eds), Engaging the future: Forecasts, scenarios, plans, and projects (pp. 199–219). Cam-bridge, MA: Lincoln Institute of Land Policy.British Columbia. (1996). Local Government Act. [Online] Available from: http://www.bclaws.ca/Recon/document/ID/freeside/96323_00Lyles, W., & Stevens, M. (2014). Plan quality evaluation 1994–2012: Growth and contributions, limitations, and new directions. Journal of Planning Education and Research, 34(4), 433-450. doi:10.1177/0739456X14549752Metro Vancouver (2012). United Nations Award for Metro Vancouver [Press Release]. http://www.metrovancouver.org/media-room/media-releases/MediaReleases/2012-06-25-United-NationsAwardforMetroVancouver.pdfMetro Vancouver (2007). Metro Vancouver Affordable Housing Strategy. Available from: http://www.metrovancouver.org/services/regional-planning/PlanningPublications/ Adopted-MetroVancAffordHousStrategyNov302007.pdfNorton, R. K. (2008). Using content analysis to evaluate local master plans and zoning codes. Land use Policy, 25(3), 432-454. doi:10.1016/j.landusepol.2007.10.006Spencer-Thomas, Owen. (2012). Press release: getting the facts straight. Retrieved from http://www.owenspencer-thomas.com/journalism/media-tips/writing-a-press-release Statistics Canada. (2011). Population and dwelling counts, for Canada, census metropolitan areas, census agglomerations and census subdivisions (municipalities), 2011 census. Retrieved from: http://www12.statcan.gc.ca/census-recensement/2011/dp-pd/hlt-fst/pd-pl/TableStatistics Canada (2011). Population and dwelling counts, for Canada, provinces and territo-ries, and census subdivisions (municipalities), 2011 and 2006 censuses. Retrieved from the Statistics Canada website http://www12.statcan.gc.ca/census-recensement/2011/dp-pd/hlt-fst/pd-pl/Table-Tableau.cfm?LANG=Eng&T=302&PR=59&S=51&O=A&RPP=25Stevens, M.R. (2013). Evaluating the quality of official community plans in southern british columbia. Journal of Planning Education and Research, 33(4), 471-490.Stevens, M. R., Lyles, W., & Berke, P. R. (2014). Measuring and reporting intercoder reliability in plan quality evaluation research. Journal of Planning Education and Research, 34(1), 77-93. doi:10.1177/0739456X13513614Stevens, M.R., & Mody, A.Z.D. (2013). Sustainability plans in british columbia: Instruments of change or token gestures? Canadian Journal of Urban Research, 22(1, Supplement), 46-71.Stevens, M.R., & Senbel, M. (forthcoming). Rational-comprehensive vs. communicative plan-ning: An evaluation of award-winning Plans. Manuscript in preparation.Taylor, Z. T., Burchfield, M. (2010). Growing cities: Comparing urban growth patterns and regional growth policies in Calgary, Toronto, and Vancouver. Toronto: Neptis Foundation.Planning Institute of British Columbia (2016). PIBC Awards. Retrieved from https://www.pibc.bc.ca/content/pibc-awardsQadeer, M. A. (1997) Pluralistic Planning for Multicultural Cities: The Canadian Practice. Journal of the American Planning Association, 63:4, 481-494, DOI: 10.1080/01944369708975941West, D. (2001). Urban E-Government: An Assessment of City Government Websites. Ac-cessed February 20, 2016 at:  http://www.insidepolitics.org/CityWhitePaper.pdf(7) REFERENCES AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER42Appendix 1: Protocol Guides   Internal and External Quality Protocol Guide Communicative and Persuasive Quality Protocol Guide Online Communication Protocol GuideAppendix 2: Modified Protocol Items  Internal and External Quality Protocol Items Communicative and Persuasive Protocol ItemsAppendix 3: Reliability DataAppendix 4: Protocols Subcategory FrequenciesAppendix 5: Protocols Scores by Municipality(8) APPENDICES(8) APPENDICES AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 43 13 - Economy Present (YES) Does the plan include at least one descriptive statement about the present economy of the community?  Give a 1 if: Yes  Give a 0 if: No  14 - Economy Future (YES) Does the plan include at least one descriptive statement about how the community EXPECTS/PREDICTS the future economy to be (NOT as the community HOPES the future economy to be)?  Give a 1 if: Yes  Give a 0 if: No  15 - Environment Features (TOPOGRAPHY) Does the plan include at least one descriptive statement about the topography in/near the community?  Give a 1 if: Yes  Give a 0 if: No  16 - Environment Features (VEGETATION&FORESTS) Does the plan include at least one descriptive statement about vegetation/forests in/near the community?  Give a 1 if: Yes  Give a 0 if: No  17 - Environment Features (WATER BODIES) Does the plan include at least one descriptive statement about water bodies in/near the community?  Give a 1 if: Yes  Give a 0 if: No  18 - Environment Features (WILDLIFE HABITAT) Does the plan include at least one descriptive statement about wildlife habitat in/near the community?  Give a 1 if: Yes  Give a 0 if: No  19 - Environment State (AIR QUALITY) Does the plan include at least one descriptive statement about air quality in/near the community?  Give a 1 if: Yes  Give a 0 if: No  20 - Environment State (DRINKING WATER QUALITY) Does the plan include at least one descriptive statement about the quality of drinking water and/or water used for other domestic purposes available to the community?  Give a 1 if: Yes  Give a 0 if: No     21 - Environment State (WATER BODY QUALITY) Does the plan include at least one descriptive statement about the quality of water bodies (rivers, streams, lakes, etc.) in/near the community?  Give a 1 if: Yes  Give a 0 if: No  22 - Housing Present (YES) * Does the plan include at least one descriptive statement of existing housing supply, composition, location, or demand?  Give a 1 if: Yes  Give a 0 if: No  23 - Housing Future (YES) * Does the plan include at least one descriptive statement of future housing supply, composition, location, or demand?  Give a 1 if: Yes  Give a 0 if: No   24 - Population Present Composition (YES) Does the plan include a description of the composition (e.g. broken down by age and/or gender and/or race, etc.) of the present population?  Give a 1 if: Yes  Give a 0 if: No  25 - Population Future Composition (YES) Does the plan include a description of the composition (e.g. broken down by age and/or gender and/or race, etc.) of the future population?  Give a 1 if: Yes  Give a 0 if: No  26 - Population Present Size (YES) Does the plan include a description of the size of the present population?  Give a 1 if: Yes  Give a 0 if: No  27 - Population Future Size (YES) Does the plan include a description of the size of the future population?  Give a 1 if: Yes  Give a 0 if: No   Internal and External Quality Protocol Guide (Berke et al., 2006)  1- List of Amendments (YES) * Does the plan include a list of amendments?  Give a 1 if: Yes  Give a 0 if: No  2 - Executive Summary (YES) Does the plan contain a section entitled "Executive Summary" or "Summary" or "Overview" or something similar that acts as an Executive Summary?  Give a 1 if: Yes  Give a 0 if: No  3 - Table of Contents Detailed (YES) Does the plan include a table of contents that lists both the chapters and the subheadings?  Give a 1 if: Yes  Give a 0 if: No  4 - Table of Contents Simple (YES) Does the plan include a table of contents that lists only the chapters?  Give a 1 if: Yes  Give a 0 if: the plan does not include a table of contents  5 -Administrative Authority (YES) * Is the administrative authority for planning indicated? Does the plan mention the required elements of an OCP as outlined in the LGA?  Give a 1 if: the plan acknowledges that the authority to plan comes from the province and/or the Local Government Act, or mentions that the Local Government Act contains the provisions for developing an OCP  Give a 0 if: the plan does not acknowledge the source of its authority to plan or does not mention the Local Government Act  6 -Opportunities (YES) Does the plan identify at least one of the community’s major opportunities and/or strengths for desirable land development?  Give a 1 if: Yes  Give a 0 if: No  7 - Threats (YES) Does the plan identify at least one of the community’s major threats and/or weaknesses and/or constraints for desirable land development?  Give a 1 if: Yes  Give a 0 if: No    8 - Assessment (QUALITATIVE) Does the plan include a preliminary assessment of major trends and impacts of forecasted change during the future planning period?  Give a 1 if: the plan makes some kind of qualitative statement about the types of changes they expect their community to experience in the future. It is OK if the qualitative assessment just discusses the quantitative assessment.  Give a 0 if: the plan does not reference expected changes in qualitative fashion  9 - Assessment (QUANTITATIVE) Does the plan include a preliminary assessment of major trends and impacts of forecasted change during the future planning period? NOTE: The most common example of this would be population projections for the future, that could focus just on numbers but could also focus on composition (e.g. more seniors, fewer families with kids, etc.)  Give a 1 if: the plan contains some kind of quantitative analysis of future population, economic indicators, etc., and discusses the projections in the text  Give a 0 if: the plan does not reference expected changes in quantitative fashion  10 - Vision Statement (YES) Does the plan include a separate, self-contained vision statement that identifies in words an overall image of what the community wants to be and look like?  Give a 1 if: Yes  Give a 0 if: No  11 - Data Sources (YES) Does the plan generally cite data sources?  Give a 1 if: Yes  Give a 0 if: No  12 - Data Sources (GENERALLY) Does the plan cite at least one data source?  Give a 1 if: Yes  Give a 0 if: No   APPENDIX 1: PROTOCOL GUIDESAPPENDIX 1: PROTOCOL GUIDES44  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 41 - Goals Economic Development (YES) Does the plan include at least one economic development goal?  Give a 1 if: Yes  Give a 0 if: No  42 - Actions Economic Development (YES) Does the plan contain at least one specific action related to economic development?  Give a 1 if: Yes  Give a 0 if: No  43 - Goals Parks & Open Space (YES) Does the plan include at least one parks & open space goal?  Give a 1 if: Yes  Give a 0 if: No  44 - Actions Parks & Open Space (YES) Does the plan contain at least one specific action related to parks and open space?  Give a 1 if: Yes  Give a 0 if: No  45 - Goals Transportation (YES) Does the plan include at least one transportation goal?  Give a 1 if: Yes  Give a 0 if: No  46 - Actions Transportation (YES) Does the plan contain at least one specific action related to transportation?  Give a 1 if: Yes  Give a 0 if: No  47 - Goals Facilities & Services (YES) Does the plan include at least one facilities & services goal?  Give a 1 if: Yes  Give a 0 if: No  48 - Actions Facilities & Services (YES) Does the plan contain at least one specific action related to facilities and services?  Give a 1 if: Yes  Give a 0 if: No  49 - Goals Food & Agriculture (YES) Does the plan include at least one food & agriculture goal?  Give a 1 if: Yes  Give a 0 if: No  50 - Actions Food & Agriculture (YES) Does the plan contain at least one specific action related to food and/or agriculture?  Give a 1 if: Yes  Give a 0 if: No 51 - Goals Heritage (YES) Does the plan include at least one heritage goal? NOTE: Heritage refers to preserving buildings, sites, etc. with some kind of historical/culture value.  Give a 1 if: Yes  Give a 0 if: No  52 - Actions Heritage (YES) Does the plan contain at least one specific action related to heritage?  Give a 1 if: Yes  Give a 0 if: No  53 - Policies Mandatory (GENERALLY) Does the plan include policies that are generally mandatory (with words like shall, will, require, must) as opposed to suggestive (with words like consider, should, may)?  Give a 1 if: Yes  Give a 0 if: No  54 - Present Land Use Map (YES) * Does the plan include a present land use map? NOTE: Same notes as Future Land Use Map apply here.  Give a 1 if: Yes  Give a 0 if: No  55 - Future Land Use Map (YES)  Does the plan include a future land use map? NOTE: A land use map should show a bird's eye view of the entire community, divided up into different land uses (e.g. residential, commercial, industrial, etc.) with a different color used for each use. The original protocol did not try to make the distinction between Future and Present maps, but I am going to try to.  Give a 1 if: Yes  Give a 0 if: No  56 - Land Use Present Description (YES) Does the plan include at least one descriptive statement of existing land use in relation to the land use map? NOTE: The land use map should not just be dumped into the plan without being referred to.   Give a 1 if: Yes  Give a 0 if: No  57 - Land Use Future Description (YES) * Does the plan include at least one descriptive statement of land use in relation to the future land use map? NOTE: The land use map should not just be dumped into the plan without being referred to.   Give a 1 if: Yes  Give a 0 if: No    28 - Projections Tied (YES) Does the plan make at least one connection between a projection/anticipated change and a policy? NOTE: In theory, a plan's policies should logically follow (at least in part) from the projections of future population and economy. For example, a city that expects to have a lot of growth should probably have some policies that deal with accommodating/managing that growth  Give a 1 if: Yes  Give a 0 if: No  29 - Roads Present (YES)  Does the plan include at least one descriptive statement of the present state of roads and other transportation infrastructure?  Give a 1 if: Yes  Give a 0 if: No  30 - Roads Future (YES) Does the plan include at least one descriptive statement of future needs for roads and other transportation infrastructure? NOTE: This item addresses NEEDS for future roads/transportation infrastructure, not just a description of future plans.  Give a 1 if: Yes  Give a 0 if: No  31 - Schools Present (YES) Does the plan include at least one descriptive statement of the present state of schools?  Give a 1 if: Yes  Give a 0 if: No  32 - Schools Future (YES) Does the plan include at least one descriptive statement of future needs or plans for schools? NOTE: Future plans for schools were included, since municipalities in Canada don’t really provide for schools.  Give a 1 if: Yes  Give a 0 if: No  33 - Sewer/Water Present (YES) Does the plan include at least one descriptive statement of the present state of sewer/water infrastructure and/or services?  Give a 1 if: Yes  Give a 0 if: No         34 - Sewer/Water Future (YES) Does the plan include at least one descriptive statement of future needs for sewer/water infrastructure and/or services? NOTE: This item addresses NEEDS for future sewer/water infrastructure and/or services, not just a description of future plans.  Give a 1 if: Yes  Give a 0 if: No  35 - Tables Format (YES) If the plan has at least one table, does every table in the plan have a title and a data source listed? NOTE: The data source might not be displayed with the table, but might be mentioned somewhere else. In addition, some tables might not require data sources.  Give a 1 if: Yes  Give a 0 if: No  36 - Tables Used (YES) Does the plan include at least one table? NOTE: We are looking here for tables that help to explain/support the reasoning behind issues and/or policies. This item is NOT talking about maps, illustrations, photos, diagrams, charts, figures, etc. Tables contain rows and columns, with data (quantitative or qualitative) in the cells  Give a 1 if: Yes  Give a 0 if: No  37 - Goals Land Use (YES) Does the plan include at least one land use goal?  Give a 1 if: Yes  Give a 0 if: No  38 - Actions Land Use (YES) Does the plan contain at least one specific action related to land use?  Give a 1 if: Yes  Give a 0 if: No  39 - Goals Environment (YES) Does the plan include at least one environment goal?  Give a 1 if: Yes  Give a 0 if: No  40 - Actions Environment (YES) Does the plan contain at least one specific action related to some aspect of the natural environment?  Give a 1 if: Yes  Give a 0 if: No   APPENDIX 1: PROTOCOL GUIDES AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 45 71 - Organizations Monitoring (YES) Does the plan identify at least one organization with responsibility for monitoring? NOTE: Monitoring refers to ensuring that implementation is happening, evaluating the extent to which progress is being made, etc.  Give a 1 if: Yes  Give a 0 if: No  72 - Organizations Monitoring (GENERALLY) Does the plan generally identify organizations with responsibility for monitoring? NOTE: Monitoring refers to ensuring that implementation is happening, evaluating the extent to which progress is being made, etc.  Give a 1 if: Yes  Give a 0 if: No  73 - Updating Plan (YES) * Is there timetable/timeframe mentioned for updating the plan?  Give a 1 if: Yes  Give a 0 if: No  74 - Timelines Implementation (YES) Does the plan identify at least one timeline for implementation? NOTE: Timelines for implementation refer to particular years/dates that are specified for implementing the plan and/or particular actions in the plan.  Give a 1 if: Yes  Give a 0 if: No  75 - Timelines Implementation (GENERALLY) Are timelines for implementation generally identified? NOTE: Timelines for implementation refer to particular years/dates that are specified for implementing the plan and/or particular actions in the plan.  Give a 1 if: Yes  Give a 0 if: No 76 - Timelines Updating (BASED ON MONITORING) If the plan contains a timeline for updating, is the timeline based at least in part on results of monitoring changing conditions?  Give a 1 if: Yes  Give a 0 if: No  77 - Future Scenarios (YES) Does the plan include at least two explicit alternative future growth scenarios? NOTE: For the entire community, not just for a particular area. E.g. Compact scenario, low-density scenario, etc.  Give a 1 if: Yes  Give a 0 if: No    78 - Prioritized Actions (YES) Does the plan contain at least one example of one action being prioritized over another? NOTE: This item refers to a scenario where limited resources might/would prevent a community from implementing all of their actions (ever or right now), such that they should ideally indicate which actions are the most important and should be given priority for implementation.  Give a 1 if: Yes  Give a 0 if: No  79 - Prioritized Actions (GENERALLY) Are the actions listed in the plan generally prioritized in relation to one another? NOTE: This item refers to a scenario where limited resources might/would prevent a community from implementing all of their actions (ever or right now), such that they should ideally indicate which actions are the most important and should be given priority for implementation.  Give a 1 if: Yes  Give a 0 if: No  80 - Cross-Referencing (YES) Does the plan include at least one example of cross-referencing that alerts readers to other sections of the plan that are relevant to the section being read?  Give a 1 if: Yes  Give a 0 if: No  81 - Cross-Referencing (GENERALLY) Does the plan include a consistent cross-referencing system throughout the plan that alerts readers to other sections of the plan that are relevant to the section being read?  Give a 1 if: Yes  Give a 0 if: No   82 - Glossary (YES) Is a glossary of terms and definitions included?  Give a 1 if: Yes  Give a 0 if: No  83 - Illustrations (YES) Are illustrations used (e.g., diagrams, pictures)? NOTE: This item is NOT talking about maps or tables. This item is talking about other types of illustrations, such as photos, diagrams, charts, figures, etc.  Give a 1 if: Yes  Give a 0 if: No    58 - Land Demand (YES) Does the plan include a systematic analysis of how much land will be needed to accommodate projected future growth?  Give a 1 if: Yes  Give a 0 if: No  59 - Land Supply (YES) Does the plan include a systematic analysis of how much land is available to accommodate projected future growth?  Give a 1 if: Yes  Give a 0 if: No  60 - Land Supply Demand Analysis (YES)  Does the plan compare the amount of land needed to accommodate projected future growth with the amount of land available to accommodate future growth?  Give a 1 if: Yes  Give a 0 if: No  61 - Land Use Landscape (YES) Does the plan include at least one example of a proposed location of land uses being influenced by the suitability of landscape features, or does the plan have a general or specific policy to determine the location of land uses based in part on the suitability of landscape features?  Give a 1 if: Yes  Give a 0 if: No  62 - Land Use Transportation (YES) Does the plan include at least one example of a proposed location of land uses being influenced by one or more transportation proposals or existing transportation services, or does the plan have a general or specific policy to determine the location of land uses based in part on the transportation proposals and/or transportation services?  Give a 1 if: Yes  Give a 0 if: No  63 - Land Use Water (YES) Does the plan include at least one example of a proposed (specific or community-wide) location or type/intensity/density of land uses being influenced by one or more water/sewer proposals or existing water/sewer services, or does the plan have a general or specific policy to determine the location of land uses based in part on the water/sewer proposals and or water/sewer services?  Give a 1 if: Yes  Give a 0 if: No        64 - Measurable Objectives & Indicators (YES) Is at least one goal/policy quantified based on measurable objectives and/or indicators (e.g. 60 percent of all residents within ¼ mile of transit service)? NOTE: This should not be a particular standard that the plan is imposing, but rather an indicator that allows the community to track its progress toward meeting a goal or goals.  Give a 1 if: Yes  Give a 0 if: No  65 - Measurable Objectives & Indicators (GENERALLY) Are goals/policies generally quantified based on measurable objectives and/or indicators (e.g. 60 percent of all residents within ¼ mile of transit service)? NOTES: This should not be a particular standard that the plan is imposing, but rather an indicator that allows the community to track its progress toward meeting a goal or goals. There must be more than just one or two measureable objectives to count as “generally”.  Give a 1 if: Yes  Give a 0 if: No  66 - Implementation Section (YES) Does the plan include a separate section/subsection that addresses what needs to be done to implement the plan?  Give a 1 if: Yes  Give a 0 if: No  67 - Actions for Implementation (YES) Does the plan include actions for implementation?  Give a 1 if: Yes  Give a 0 if: No  68 - Monitoring Indicators (YES) Does the plan contain a section or subsection that specifically addressing monitoring?  Give a 1 if: Yes  Give a 0 if: No  69 - Organizations Implementation (YES) Does the plan identify at least one specific organization (not just "the town") with responsibility for implementation?  Give a 1 if: Yes  Give a 0 if: No  70 - Organizations Implementation (GENERALLY) Does the plan generally identify specific organizations with responsibility for implementation? NOTE: The plan should say more than just "the town", but give them credit if they specify the town department responsible (e.g. town public works department)  Give a 1 if: the plan generally identifies organizations responsible for implementation  Give a 0 if: the plan does not generally identify organizations responsible for implementation  APPENDIX 1: PROTOCOL GUIDES46  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVERCommunicative and Persuasive Quality Protocol Guide (Bunnell & Jepson, 2011)  1 - Developed by a consultant? (1/0) Was the plan developed through the aid of a consultant?  Yes   ____ (4 points)  No   ____ (0 points)  2 - Mention previous plans (YES) Does the plan identify, describe and discuss previous plans?  Yes   ____ (4 points)  No   ____ (0 points)  3 - Mention previous plans (SOMEWHAT) Does the plan identify, describe and discuss previous plans?  Somewhat  ____ (2 points)  No   ____ (0 points)  4 - React to previous plans (YES) Does the plan consciously embrace or reject previous plans or planning strategies?  Yes ____ (4 points) It embraces or rejects previous plans   No ____ (0 points) No mention of previous plans.  5 - React to previous plans (SOMEWHAT) Does the plan consciously embrace or reject previous plans or planning strategies?  Somewhat ____ (2 points) previous plans are listed but not discussed.  No ____ (0 points) It does not mention previous plans  6 - Regional context: Acknowledge (YES) * Does the plan address its relationship with Metro Vancouver by including a summary/excerpt of its regional context statement?  Yes ____ (4 points)  No ____ (0 points)  7 - Regional context: Acknowledge (SOMEWHAT) * Does the plan address its relationship with Metro Vancouver by including a summary/excerpt of its regional context statement?  Somewhat  ____ (2 points)  No   ____ (0 points)  8 - Historical perspective (YES) Does the plan provide historical perspective through extensive narrative of its history and how it has changed over time?  Yes   ____ (4 points)  No   ____ (0 points)    9 - Historical perspective (SOMEWHAT) Does the plan provide historical perspective through extensive narrative of its history and how it has changed over time?  Somewhat  ____ (2 points)  No   ____ (0 points)  10 - Unique identity: History (YES) * Does the plan reinforce the community’s unique identity and sense of place by describing the history and/or evolution of the community?  Yes   ____ (4 points)  No   ____ (0 points)  11 - Unique identity: History (SOMEWHAT) * Does the plan reinforce the community’s unique identity and sense of place by describing the history and/or evolution of the community?  Somewhat  ____ (2 points)  No   ____ (0 points)  12 - Unique identity: Culture (YES) * Does the plan reinforce the community’s unique identity and sense of place by acknowledging past or existing indigenous cultures in the community?  Yes   ____ (4 points)  No   ____ (0 points)  13 - Unique identity: Culture (SOMEWHAT) * Does the plan reinforce the community’s unique identity and sense of place by acknowledging past or existing indigenous cultures in the community?  Somewhat  ____ (2 points)  No   ____ (0 points)  14 - Unique identity: geography, economy, political (YES) * Does the plan reinforce the community’s unique identity and sense of place by describing its unique geography, economy, or political culture?  Yes   ____ (4 points)  No   ____ (0 points)  15 - Unique identity: Geography, economy, political (SOMEWHAT) * Does the plan reinforce the community’s unique identity and sense of place by describing its unique geography, economy, or political culture?  Somewhat  ____ (2 points)  No   ____ (0 points)    84 - Horizontal Connections (YES) Does the plan include at least one horizontal connections with other local plans/programs? NOTE: In order to receive a 1, plan should have relatively explicit reference(s) to other municipal plans/policies/programs that are relevant. (For example, instead of including in the OCP a huge section on economic development, the plan might say "see Town Economic Development plan for more details)  Give a 1 if: Yes  Give a 0 if: No  85 - Intergovernmental Coordination  Infrastructure & Services (YES) Does the plan include at least one example of intergovernmental coordination for providing infrastructure and services? NOTE: Coordination refers to actually working together, not just (for example) following provincial regulations or standards. This can involve either actual coordination or encouraged/desired coordination. Studies that explore potential coordination don't count.  Give a 1 if: Yes  Give a 0 if: No  86 - Intergovernmental Coordination  Natural Hazards (YES) Does the plan include at least one example of intergovernmental coordination for protecting natural hazards? NOTE: Coordination refers to actually working together, not just (for example) following provincial regulations or standards. This can involve either actual coordination or encouraged/desired coordination. Studies that explore potential coordination don't count.  Give a 1 if: Yes  Give a 0 if: No   87 - Intergovernmental Coordination  Natural Systems (YES) Does the plan include at least one example of intergovernmental coordination for managing natural systems? NOTE: Coordination refers to actually working together, not just (for example) following provincial regulations or standards. This can involve either actual coordination or encouraged/desired coordination. Studies that explore potential coordination don't count.  Give a 1 if: Yes  Give a 0 if: No         88 - Organizations & Individuals (YES) Does the plan identify at least one organization/individual that was involved in writing the plan? NOTE: This item addresses the entities that actually wrote and put the plan together, NOT the entities who gave input as to what should go in the plan  Give a 1 if: Yes  Give a 0 if: No  89 - Participation Section (YES) Does the plan include a separate section/subsection that describes the public participation process during the development of the plan?  Give a 1 if: Yes  Give a 0 if: No  90 - Participation Techniques (YES) Does the plan include an explanation of at least one specific participation technique that was used during development of the plan? NOTE: The value/quality of the techniques themselves are not being examined here.  Give a 1 if: Yes  Give a 0 if: No  91 - Plan Evolution (YES) Is the plan’s evolution described? NOTE: The evolution of the plan in question, not an evolution of previous plan. A list of amendments is not sufficient.  Give a 1 if: Yes  Give a 0 if: No  92 - Stakeholders (YES) Does the plan identify at least one non-governmental stakeholder group that gave input?  Give a 1 if: Yes  Give a 0 if: No  93 - Stakeholders (BROAD) Does the plan incorporate input from a broad spectrum of stakeholders? NOTE: Ideally, a "broad spectrum of stakeholders" should not only include many stakeholders, but also stakeholders that represent different viewpoints (e.g. business vs. environment)  Give a 1 if: Yes  Give a 0 if: NoAPPENDIX 1: PROTOCOL GUIDES AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 47 31 - Inclusive and Exhorting (SOMEWHAT) * Does the plan use inclusive language that inspires people to act? Is it written in a way that implies “we can do this together” as opposed “this is what your municipality is doing”  Somewhat  ____ (2 points)  No   ____ (0 points)  32 - Vision statement (YES) Is there a vision statement that conveys the essence of what the community wants to be and look like in the future?  Yes   ____ (4 points)  No   ____ (0 points)  33 - Vision statement (SOMEWHAT) Is there a vision statement that conveys the essence of what the community wants to be and look like in the future?  Somewhat  ____ (2 points)  No   ____ (0 points) 34 - Goals meaningful (YES) Are goals clearly stated, and are they a meaningful guide to action and decision making (i.e. are the goals more than “motherhood and apple pie”?  Yes   ____ (4 points)  No   ____ (0 points)  35 - Goals meaningful (SOMEWHAT) Are goals clearly stated, and are they a meaningful guide to action and decision making (i.e. are the goals more than “motherhood and apple pie”?  Somewhat  ____ (2 points)  No   ____ (0 points)  36 - Less than 10 unifying themes (YES) Does the plan focus attention on a limited number of unifying themes (<10)?   Yes = 4  No = 0  Number of goals:  _____ (Type at bottom of Excel doc)  37 - Number of goals (<12) (YES) Does the plan present a limited number of goals:  <12 (4 points)  13–24 (2 points)  >25 (0 points)  38 - Number of goals (13-24 goals) (YES) Does the plan present a limited number of goals:  <12 (4 points)  13–24 (2 points)  >25 (0 points)    39 - Number of goals (>25) (YES) Does the plan present a limited number of goals:  <12 (4 points)  13–24 (2 points)  >25 (0 points)  Number of objectives:  _____ (Type at bottom of Excel doc)  40 - Number of objectives (<20) (YES) Does the plan present a limited number of objectives:  < 20 (4 points)  20 – 40 (2 points)  40 (0 points)  41 - Number of objectives (20-40) (YES) Does the plan present a limited number of objectives:  < 20 (4 points)  20 – 40 (2 points)  40 (0 points)  42 - Number of objectives (>40) (YES) Does the plan present a limited number of objectives:  < 20 (4 points)  20 – 40 (2 points)  40 (0 points)  43 - Recognize uncertainty (YES) Does the plan present more than one forecast of future population and/or job growth, and in so doing recognize uncertainty?  Yes   ____ (4 points)  No   ____ (0 points)  44 - Recognize uncertainty (SOMEWHAT) Does the plan present more than one forecast of future population and/or job growth, and in so doing recognize uncertainty?  Somewhat  ____ (2 points)  No   ____ (0 points)  45 - Alternate scenarios (YES) Does the plan present alternative scenarios, or at the very least compare the Desired Scenario vs. Trend Scenario?  Yes   ____ (4 points)  No   ____ (0 points)  46 - Alternate scenarios (SOMEWHAT) Does the plan present alternative scenarios, or at the very least compare the Desired Scenario vs. Trend Scenario?  Somewhat  ____ (2 points)  No   ____ (0 points)    16 - Explain process (YES) Does the plan establish its legitimacy by explaining the process followed preparing the plan, and steps taken to obtain wide-ranging participation and input from citizens and stakeholders? “Procedural validity” (Baer, 1997)  Yes   ____ (4 points)  No   ____ (0 points)  17 - Explain process (SOMEWHAT) Does the plan establish its legitimacy by explaining the process followed preparing the plan, and steps taken to obtain wide-ranging participation and input from citizens and stakeholders? “Procedural validity” (Baer, 1997)  Somewhat  ____ (2 points)  No   ____ (0 points)  18 - Imaginative and creative (YES) Is the plan imaginative and creative (extent of commitment to preparing a meaningful, effective plan)?  Yes   ____ (4 points)  No   ____ (0 points)  19 - Imaginative and creative (SOMEWHAT) Is the plan imaginative and creative (extent of commitment to preparing a meaningful, effective plan)?  Somewhat  ____ (2 points)  No   ____ (0 points)  20 - More than a collection of separate plan elements (YES) Is the plan more than a collection of separate plan elements (The “check-box” approach to plan making)?  Yes   ____ (4 points)  No   ____ (0 points)  21 - More than a collection of separate plan elements (SOMEWHAT) Is the plan more than a collection of separate plan elements (what I call the “check-box” approach to plan making)?  Somewhat  ____ (2 points)  No   ____ (0 points)  22 - Unifying narrative (YES) Does it contain a unifying narrative storyline that tells an engaging story?  Yes   ____ (4 points)  No   ____ (0 points)  23 - Unifying narrative (SOMEWHAT) Does it contain a unifying narrative storyline that tells an engaging story?  Somewhat  ____ (2 points)  No   ____ (0 points)   24 - Compelling visuals (YES) * Does the plan put forward a compelling vision (through illustrations, photographs, maps) of what the future could be like?  Yes   ____ (4 points)  No   ____ (0 points)  25 - Compelling visuals (SOMEWHAT) * Does the plan put forward a compelling vision (through illustrations, photographs, maps) of what the future could be like?  Somewhat  ____ (2 points)  No   ____ (0 points)  26 - Compelling language (YES) * Does the plan use language that inspires, provokes, or is passionate? Does it appeals to the human/personal side of planning?  Yes   ____ (4 points)  No   ____ (0 points)  27 - Compelling language (SOMEWHAT) * Does the plan use language that inspires, provokes, or is passionate? Does it appeals to the human/personal side of planning?  Somewhat  ____ (2 points)  No   ____ (0 points)  28 - Rationales effectively presented (YES) * Are the arguments and rationales supporting goals and objectives effectively presented? Does it link policies to larger societal goals or connect it to societal benefits? This can include statistics and qualitative trends.  Yes   ____ (4 points)  No   ____ (0 points)  29 - Rationales effectively presented (SOMEWHAT) * Are the arguments and rationales supporting goals and objectives effectively presented? Does it link policies to larger societal goals or connect it to societal benefits? This can include statistics and qualitative trends.  Somewhat  ____ (2 points)  No   ____ (0 points)  30 - Inclusive and Exhorting (YES) * Does the plan use inclusive language that inspires people to act? Is it written in a way that implies “we can do this together” as opposed “this is what your municipality is doing”  Yes   ____ (4 points)  No   ____ (0 points)   APPENDIX 1: PROTOCOL GUIDES48  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 63 - Tables and data relevant (YES) Are tables and other data presented in the plan relevant to the argument that is being made and plan recommendations?  Yes   ____ (4 points)  No   ____ (0 points)  64 - Tables and data relevant (SOMEWHAT) Are tables and other data presented in the plan relevant to the argument that is being made and plan recommendations?  Somewhat  ____ (2 points)  No   ____ (0 points)  65 - What/Who/When/How (ALL 4) Does the plan explicitly underscore the importance of implementing the recommendations contained in the plan by laying out a clear path to plan implementation that specifically identifies what, who, when (implementation dates or prioritized), and how funded:  ___ All four aspects (4 points)  ___ Three or fewer aspects (2 points)  ___ No implementation program or three aspects (0 points)  66 - What/Who/When/How (ONLY 3) Does the plan explicitly underscore the importance of implementing the recommendations contained in the plan by laying out a clear path to plan implementation that specifically identifies what, who, when (implementation dates or prioritized), and how funded:  ___ All four aspects (4 points)  ___ Three of the four aspects (2 points)  ___ No implementation program or three aspects (0 points)  Number of pages:  _____ (Type at bottom of Excel doc)  67 - Plan not bulky (1-99 PAGES) Is the plan not overly bulky; that is, is it compact and easy to carry?  Number of pages (not including appendices):  ___ 1–99 (4 points)  ___ 100–166 (2 points)  ___>167 (0 points)  68 - Plan not bulky (100-66 PAGES) Is the plan not overly bulky; that is, is it compact and easy to carry?  Number of pages (not including appendices):  ___ 1–99 (4 points)  ___ 100–166 (2 points)  ___>167 (0 points)   69 - Plan not bulky (>167 PAGES) Is the plan not overly bulky; that is, is it compact and easy to carry?  Number of pages (not including appendices): ___ 1–99 (4 points) ___ 100–166 (2 points) ___>167 (0 points)  70 - Attractive executive summary (YES) Does the plan include an attractive, highly readable, and informative executive summary?  Yes   ____ (4 points)  No   ____ (0 points)  71 - Attractive executive summary (SOMEWHAT) Does the plan include an attractive, highly readable, and informative executive summary?  Somewhat  ____ (2 points)  No   ____ (0 points)                   47 - Explain alternate scenarios (YES) “Does the plan provide clear explanations of alternative courses of action that enhance community flexibility and adaptation in dealing with complex situations?”  Yes   ____ (4 points)  No   ____ (0 points)  48 - Explain alternate scenarios (SOMEWHAT) “Does the plan provide clear explanations of alternative courses of action that enhance community flexibility and adaptation in dealing with complex situations?”  Somewhat  ____ (2 points)  No   ____ (0 points)  49 - How future will be shaped (YES) Does the plan communicate how future outcomes are likely to be shaped by different policies and courses of action?  Yes   ____ (4 points)  No   ____ (0 points)  50 - How future will be shaped (SOMEWHAT) Does the plan communicate how future outcomes are likely to be shaped by different policies and courses of action?  Somewhat  ____ (2 points)  No   ____ (0 points)  51 - Understanding of consequences (YES) Does the plan convey an understanding of the consequences of different courses of action?  Yes   ____ (4 points)  No   ____ (0 points)  52 - Understanding of consequences (SOMEWHAT) Does the plan convey an understanding of the consequences of different courses of action?  Somewhat  ____ (2 points)  No   ____ (0 points)  53 - Maps clear, relevant, and comprehensible (YES) Are maps included in the plan clear, relevant and comprehensible? (Berke et al., 2006)  Yes   ____ (4 points)  No   ____ (0 points)  54 - Maps clear, relevant, and comprehensible (SOMEWHAT) Are maps included in the plan clear, relevant and comprehensible? (Berke et al., 2006)  Somewhat  ____ (2 points)  No   ____ (0 points)      55 - Well written with minimal jargon (YES) Well written, clear and concise, with a minimum of technical jargon, so that citizens will want to read it?  Yes   ____ (4 points)  No   ____ (0 points)  56 - Well written with minimal jargon (SOMEWHAT) Well written, clear and concise, with a minimum of technical jargon, so that citizens will want to read it?  Somewhat  ____ (2 points)  No   ____ (0 points)  57 - Recognizable branding (YES) Is there a distinctive and recognizable branding element, such as a logo, trademark or title?  Yes   ____ (4 points)  No   ____ (0 points)  58 - Recognizable branding (SOMEWHAT) Is there a distinctive and recognizable branding element, such as a logo, trademark or title?  Somewhat  ____ (2 points)  No   ____ (0 points)  59 - Photographs and illustrations (YES) Does the plan contain photographs and illustrations that support the text and add visual interest?  Yes   ____ (4 points)  No   ____ (0 points)  60 - Photographs and illustrations (SOMEWHAT) Does the plan contain photographs and illustrations that support the text and add visual interest?  Somewhat  ____ (2 points)  No   ____ (0 points)  61 - Visually attractive layout (YES) Does the plan have a visually attractive format and page layout (e.g., columns not too wide; adequate line spacing, so that it is easy to read)?  Yes   ____ (4 points)  No   ____ (0 points)  62 - Visually attractive layout (SOMEWHAT) Does the plan have a visually attractive format and page layout (e.g., columns not too wide; adequate line spacing, so that it is easy to read)?  Somewhat  ____ (2 points)  No   ____ (0 points)   APPENDIX 1: PROTOCOL GUIDES AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 49APPENDIX 1: PROTOCOL GUIDES Online Communication Protocol Guide  Part 2: PDF Attributes  # Municipality 1 2 3 4 5 6 7 1 Anmore        2 Belcarra        3 Bowen Island        4 Burnaby        5 Coquitlam        6 Delta        7 Langley (Township)        8 Langley        9 Lions Bay        10 Maple Ridge        11 New Westminster        13 North Vancouver (District)        12 North Vancouver        14 Pitt Meadows        15 Port Coquitlam        16 Port Moody        17 Richmond        18 Surrey        19 West Vancouver (District)        20 White Rock         Instructions  Identify by putting a 1 (present) or a 0 (not present) for each of the items below:  1) Is the plan downloadable as a single PDF? 2) Is the first page a cover page? (typically with a title, logo, and image) 3) Is the table of contents linked with shortcuts to the corresponding sections in the plan? 4) Is the bookmark feature used to organize the document? 5) Does the PDF page numbering match the document page numbering? 6) Is the document text-searchable? (text can be searched and copied) 7) Are the photographs used generally high-quality? (lacking pixelation, jpg artifacts)   Online Communication Protocol Guide  Part 1: Municipal Website # Municipality OCP Website Download Contact Who What When Why How 1 Anmore http://www.anmore.com/content/official-community-plan        2 Belcarra http://www.belcarra.ca/index-bylaws.htm        3 Bowen Island http://www.bimbc.ca/content/planning-department-ocp-land-use-bylaws        4 Burnaby https://www.burnaby.ca/City-Services/Policies--Projects---Initiatives/Community-Development/Policies.html        5 Coquitlam http://www.coquitlam.ca/planning-and-development/resources/Property-Developer-and-Builder-Resources/Citywide-Official-Community-Plan.aspx        6 Delta http://www.delta.ca/home-property-development/official-community-plan        7 Langley (Township) http://www.tol.ca/Land-Use-and-Development/Community-Plans        8 Langley http://www.city.langley.bc.ca/business-development/official-community-plan        9 Lions Bay http://www.lionsbay.ca/OCP_Amendment.html        10 Maple Ridge https://www.mapleridge.ca/316/Official-Community-Plan        11 New Westminster http://www.newwestcity.ca/business/planning_development/official-community-plan/articles/3266.php        13 North Vancouver (District) https://www.dnv.org/property-and-development/our-official-community-plan-ocp        12 North Vancouver http://www.cnv.org/Your-Government/Official-Community-Plan        14 Pitt Meadows http://www.pittmeadows.bc.ca/EN/main/business/1156/1181.html        15 Port Coquitlam http://www.portcoquitlam.ca/City_Government/City_Departments/Development_Services/Planning_Division/Official_Community_Plan__OCP_.htm        16 Port Moody http://www.portmoody.ca/index.aspx?page=313        17 Richmond http://www.richmond.ca/plandev/planning2/ocp.htm        18 Surrey http://www.surrey.ca/city-services/1318.aspx        19 West Vancouver (District) http://westvancouver.ca/government/bylaws-strategies-reports/strategies-plans/official-community-plan        20 White Rock http://www.whiterockcity.ca/EN/main/city/planning/long-range-planning.html         Instructions  1) Follow the link to the OCP webpage. This is the primary location to access information about the OCP. Do not "click around" or leave the page. Searching for information elsewhere on the website is not necessary. 2) Identify by putting a 1 (present) or a 0 (not present) for each of the items below: a. Download: If a link to download the OCP is located on the page. b. Contact: If contact information for the planning department (phone or email) is provided. (not including a general city-wide inquiry line) c. Who: Does it identify who initiated or enacted the plan? ("The City" or “Council” is sufficient) d. What: Does it describe what a municipal plan is? (ideally a definition) e. When: Does it give a year when the plan was developed? (or last updated/amended)? f. Why: Does it explain the purpose or use of a municipal plan? - this may be described in the same place as the description of "what" is a municipal plan. g. How: Does it describe how it was developed? (through consultation with the public, for example) 3) Any format of media (point form, tables, sketches, or illustrations) that communicate these five W's should be considered.   50  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER Bunnell & Jepson Protocol Items Added/Removed/ Modified Item Rationale Removed State funding received toward the cost of the comprehensive plan? The province does not provide funding for municipalities to draft an OCP. Removed Was the plan developed in-house? It is assumed that all plans in this study were developed in house. The province does not have a regional planning commission (RPC) or regional body that would develop plans on behalf of the municipality. Removed Was the plan developed by a Regional Planning Commission?  Removed Does the plan squarely place the community in its regional context, including analysis of extra-territorial and other driving forces (opportunities and threats)? This item was replaced a different item that better captures the regional context. Modified 6 - Regional context: Acknowledge: Does the plan address its relationship with Metro Vancouver by including a summary/excerpt of its regional context statement? This item was modified because it was very likely to be present in all plans as originally defined. Since all municipalities in Metro Vancouver are required to produce a regional context statement, this item focused on whether it included a summary or excerpt of the context statement. Removed Does the plan reinforce the community’s unique identity and sense of place by conveying an understanding of its unique geography, history, economy, political culture, etc.? This item was broken into three separate dimensions. The intention was to capture a more fine-grained assessment of the community’s unique identity.  Added 10 - Unique identity: History: Does the plan reinforce the community’s unique identity and sense of place by describing the history and/or evolution of the community? Added 12 - Unique identity: Culture: Does the plan reinforce the community’s unique identity and sense of place by acknowledging past or existing indigenous cultures in the community? Added 14 - Unique identity: Geography, economy, political: Does the plan reinforce the community’s unique identity and sense of place by describing its unique geography, economy, or political culture? Modified Does the plan put forward a compelling vision (through illustrations, photographs, maps, and words) of what the future could be like?           The title of this item was changed from “Compelling vision” to “Compelling visuals”. Coders tended to focus on the “vision” aspect of this item and examined the vision statement in order to code this item. However, this item does not require examination of the vision statement alone.    Berke et al. Protocol Items Added/Modified Item Rationale Added 1- List of Amendments (YES): Does the plan include a list of amendments? This is a new item that is expected to be present in most plans. By including a list of amendments, it communicates to readers that the plan is a “living” document and not frozen in time. It is important to signal to readers that planning is an ongoing process and the municipality has documented the changes to the plan over time. Modified 5 -Administrative Authority (YES): Is the administrative authority for planning indicated? Does the plan mention the required elements of an OCP as outlined in the LGA? The text in orange was added to the original protocol item to reflect the local context of planning mandates in British Columbia. Added 22 - Housing Present (YES): Does the plan include at least one descriptive statement of existing housing supply, composition, location, or demand? The Local Government Act c. 473 (1) requires municipalities to include assessments of residential housing needs in the future (including location, amount, and type of density). This follows the structure of Stevens and Senbel.  Added 23 - Housing Future (YES): Does the plan include at least one descriptive statement of future housing supply, composition, location, or demand? Added 54 - Present Land Use Map (YES): Does the plan include a present land use map? Stevens and Senbel include an item pertaining to a future land use map. This item was added with the intention of capturing finer grained details for land use maps. In the experience of the author, plans poorly distinguish between present and future land-use maps. It would be valuable for readers to visually compare the differences between current land-uses and desired land-uses and thus this item was added. Added 57 - Land Use Future Description (YES): Does the plan include at least one descriptive statement of land use in relation to the future land use map? Stevens and Senbel include an item pertaining to the description of existing land-use. This item does not distinguish between present and future land-use maps, even though another item is intended for future land-use maps. This item was added to clearly distinguish between current/future land use maps and descriptions for both types of maps. Modified 73 - Updating Plan (YES): Is there timetable/timeframe mentioned for updating the plan? In the experience of the author, the word “timetable” was less clear than “timeframe” and likely mean the same thing. As a result, the word “timeframe” was added to the item description (in orange).    APPENDIX 2: MODIFIED PROTOCOL ITEMSAPPENDIX 1: MODIFIED PROTOCOL ITEMS AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 51APPENDIX 2: MODIFIED PROTOCOL ITEMS Removed Does the plan exhort and inspire people to act? This item was removed and replaced by two new items. These items attempt to capture more nuance in the plan’s language. One example of compelling language could be an example of how policies could personally affect citizens in their lives “traffic congestion takes time away from spending time with your family”. One example of how a plan could be considered inclusive is through adding quotes from citizen participants during the consultation phase. Another example is through using language like “our”, “we”, “us” and “together”. Added 27 - Compelling language: Does the plan use language that inspires, provokes, or is passionate? Does it appeal to the human/personal side of planning? Added 30 - Inclusive and Exhorting: Does the plan use inclusive language that inspires people to act? Is it written in a way that implies “we can do this together” as opposed “this is what your municipality is doing” Removed Does the plan present compelling arguments for the recommended course of action? In the experience of the author, this item was ambiguous. It followed a series of items that assessed how well a plan described different scenarios for future growth. However, since so few plans actually include alternate scenarios, coders tended to apply this item to any action described in the plan. To prevent future confusion, this item was omitted.  Modified 28 - Rationales effectively presented: Are the arguments and rationales supporting goals and objectives effectively presented? Does it link policies to larger societal goals or connect it to societal benefits? This can include statistics and qualitative trends. In prior coding experience, “effectively” was considered too broad of a word that made it difficult for coder to know the intentions for the item. The text in orange was added to the original protocol item to add more specificity.  52  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVERAPPENDIX 3: RELIABILITY DATAAPPENDIX 3: RELIABILITY DATAInternal and External Quality (Berke et al., 2006)# Item α % Agree Excluded* # Item α % Agree Excluded* # Item α % Agree Excluded*1 List of Amendments 0.707 85% 41 Goals Economic Development (YES) 0.444 75% X 81 Cross-Referencing (GENERALLY) 0.212 60% X2 Executive Summary (YES) 1 100% 42 Actions Economic Development (YES) 0.357 70% X 82 Glossary (YES) 0.803 90%3 Table of Contents Detailed (YES) 1 100% 43 Goals Parks & Open Space (YES) 0.48 75% X 83 Illustrations (YES) 0.494 85%4 Table of Contents Simple (YES) 1 100% 44 Actions Parks & Open Space (YES) 0.086 70% X 84 Horizontal Connections (YES) 0.695 90%5 Administrative Authority (YES) 0.458 90% 45 Goals Transportation (YES) 0.272 65% X 85 Intergov. Coord. Infrastructure & Services (YES) 0.144 65% X6 Opportunities (YES) 0.304 70% X 46 Actions Transportation (YES) 0.086 70% X 86 Intergov. Coord. Natural Hazards (YES) -0.054 85%7 Threats (YES) -0.016 50% X 47 Goals Facilities & Services (YES) 0.022 75% X 87 Intergov. Coord. Natural Systems (YES) 0.086 70% X8 Assessment (QUALITATIVE) -0.026 90% 48 Actions Facilities & Services (YES) 0.536 65% X 88 Organizations & Individuals (YES) 0.606 80%9 Assessment (QUANTITATIVE) 0.391 80% 49 Goals Food & Agriculture (YES) 0.536 80% X 89 Participation Section (YES) 0.606 80%10 Vision Statement (YES) 0.633 85% 50 Actions Food & Agriculture (YES) 0.415 70% X 90 Participation Techniques (YES) 0.409 70% X11 Data Sources (YES) 0.48 80% 51 Goals Heritage (YES) 0.409 70% X 91 Plan Evolution (YES) 0.188 60% X12 Data Sources (GENERALLY) 0.48 75% X 52 Actions Heritage (YES) 0.667 85% X 92 Stakeholders (YES) 0.701 85%13 Economy Present (YES) 0.22 70% X 53 Policies Mandatory (GENERALLY) -0.393 40% X 93 Stakeholders (BROAD) Undefined 75% X14 Economy Future (YES) 0.536 75% X 54 Present Land Use Map (YES) 0.22 70% X Orange Text = new/altered item15 Environment Features (TOPOGRAPHY) 0.064 55% X 55 Future Land Use Map (YES) -0.147 70% X16 Environment Features (VEGETATION&FORESTS) 0.594 80% 56 Land Use Present Description (YES) -0.352 35% X Number of Items Total 9317 Environment Features (WATER BODIES) -0.147 70% X 57 Land Use Future Description (YES) -0.114 75% X Number of items Included 4618 Environment Features (WILDLIFE HABITAT) 0.188 60% X 58 Land Demand (YES) -0.026 90% Number of items Excluded 4719 Environment State (AIR QUALITY) 0.494 85% 59 Land Supply (YES) 0.086 70% X20 Environment State (DRINKING WATER QUALITY) 0.301 75% X 60 Land Supply Demand Analysis (YES) Undefined 95% α % Agree21 Environment State (WATER BODY QUALITY) 0.831 95% 61 Land Use Landscape (YES) 0.071 60% X Number of items meeting reliability thresholds (hi    ≥ 0.33  ≥ 80%22 Housing Present (YES) 0.331 85% 62 Land Use Transportation (YES) 0.272 65% X 50 4823 Housing Future (YES) -0.219 60% X 63 Land Use Water (YES) 0.331 85%24 Population Present Composition (YES) 0.803 90% 64 Measurable Objectives & Indicators (YES) 0.649 95%25 Population Future Composition (YES) 0.695 90% 65 Measurable Objectives & Indicators (GENERALLY) 1 100%26 Population Present Size (YES) -0.026 90% 66 Implementation Section (YES) 0.633 85%27 Population Future Size (YES) -0.054 85% 67 Actions for Implementation (YES) 0.536 80%28 Projections Tied (YES) 0.212 60% X 68 Monitoring Indicators (YES) 1 100%29 Roads Present (YES) 0.316 60% X 69 Organizations Implementation (YES) 0.777 95%30 Roads Future (YES) 0.304 70% X 70 Organizations Implementation (GENERALLY) 1 100%31 Schools Present (YES) 0.594 80% 71 Organizations Monitoring (YES) Undefined 100%32 Schools Future (YES) 0.633 85% 72 Organizations Monitoring (GENERALLY) Undefined 100%33 Sewer/Water Present (YES) 0.28 30% X 73 Updating Plan (YES) 0.501 75% X34 Sewer/Water Future (YES) 0.68 75% X 74 Timelines Implementation (YES) 1 100%35 Tables Format (YES) 0.536 80% 75 Timelines Implementation (GENERALLY) 1 100%36 Tables Used (YES) 0.48 80% 76 Timelines Updating (BASED ON MONITORING) -0.026 90%37 Goals Land Use (YES) 0.389 75% X 77 Future Scenarios (YES) -0.026 90%38 Actions Land Use (YES) 0.389 65% X 78 Prioritized Actions (YES) Undefined 100%39 Goals Environment (YES) 0.398 75% X 79 Prioritized Actions (GENERALLY) Undefined 100%40 Actions Environment (YES) 0.156 75% X 80 Cross-Referencing (YES) 0.015 50% X* Items with reliability lower than 80% agreement were excluded from the analysis. This is indicated by a red X. Also note that items 37 to 52 under the "Goals and Policy Framework" subscale are excluded. For an explanation, see the Scoring section under Methodology. Undefined: Krippendorff's alpha is undefined when the item is coded present from all plans or absent from all plans by both independent coders  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 53APPENDIX 3: RELIABILITY DATACommunicative and Persuasive Quality (Bunnell & Jepson, 2011)# Item α % Agree Excluded* # Item α % Agree Excluded*1 Developed by a consultant? (1/0) 0.4580 90% 40 Number of objectives (<20) (YES) -0.083 80%2 Mention previous plans (YES) 0.086 70% X 41 Number of objectives (20-40) (YES) 0.649 95%3 Mention previous plans (SOMEWHAT) -0.144 45% X 42 Number of objectives (>40) (YES) 0.188 60% *4 React to previous plans (YES) Undefined 100% 43 Recognize uncertainty (YES) Undefined 95%5 React to previous plans (SOMEWHAT) 0.391 80% 44 Recognize uncertainty (SOMEWHAT) Undefined 95%6 Regional context: Acknowledge (YES) 1 95% 45 Alternate scenarios (YES) 1 100%7 Regional context: Acknowledge (SOMEWHAT) -0.026 90% 46 Alternate scenarios (SOMEWHAT) 1 100%8 Historical perspective (YES) -0.04 60% X 47 Explain alternate scenarios (YES) 1 100%9 Historical perspective (SOMEWHAT) -0.182 65% X 48 Explain alternate scenarios (SOMEWHAT) 1 100%10 Unique identity: History (YES) 0.302 65% X 49 How future will be shaped (YES) 1 100%11 Unique identity: History (SOMEWHAT) -0.161 50% X 50 How future will be shaped (SOMEWHAT) -0.054 85%12 Unique identity: Culture (YES) 0.016 50% X 51 Understanding of consequences (YES) Undefined 95%13 Unique identity: Culture (SOMEWHAT) -0.147 70% X 52 Understanding of consequences (SOMEWHAT) Undefined 95%14 Unique identity: Geography, economy, political (YES) -0.182 40% X 53 Maps clear, relevant, and comprehensible (YES) 0.878 95%15 Unique identity: Geography, economy, political (SOMEWHAT) -0.345 45% X 54 Maps clear, relevant, and comprehensible (SOMEWHAT) 0.618 90%16 Explain process (YES) 0.48 80% 55 Well written with minimal jargon (YES) 0.188 60% X17 Explain process (SOMEWHAT) 0.022 80% 56 Well written with minimal jargon (SOMEWHAT) 0.222 65% X18 Imaginative and creative (YES) Undefined 95% 57 Recognizable branding (YES) 0.391 70% X19 Imaginative and creative (SOMEWHAT) -0.219 81% 58 Recognizable branding (SOMEWHAT) 0.144 65% X20 More than a collection of separate plan elements (YES) Undefined 75% X 59 Photographs and illustrations (YES) 0.594 80%21 More than a collection of separate plan elements (SOMEWHAT) 0.22 65% X 60 Photographs and illustrations (SOMEWHAT) 0.389 75% *22 Unifying narrative (YES) -0.114 75% X 61 Visually attractive layout (YES) -0.182 65% X23 Unifying narrative (SOMEWHAT) 0.022 65% X 62 Visually attractive layout (SOMEWHAT) 0.144 65% X24 Compelling visuals (YES) 0.235 80% 63 Tables and data relevant (YES) 0.768 90%25 Compelling visuals (SOMEWHAT) 0.071 60% X 64 Tables and data relevant (SOMEWHAT) 0.331 85%26 Compelling language (YES) -0.219 60% X 65 What/Who/When/How (ALL 4) 1 100%27 Compelling language (SOMEWHAT) -0.222 45% X 66 What/Who/When/How (ONLY 3) 1 100%28 Rationales effectively presented (YES) -0.016 50% X 67 Plan not bulky (1-99 PAGES) 1 100%29 Rationales effectively presented (SOMEWHAT) -0.393 40% X 68 Plan not bulky (100-166 PAGES) 0.649 95%30 Inclusive and Exhorting (YES) Undefined 95% 69 Plan not bulky (>167 PAGES) 0.797 90%31 Inclusive and Exhorting (SOMEWHAT) 0.086 75% * 70 Attractive executive summary (YES) 1 100%32 Vision statement (YES) 0.316 65% X 71 Attractive executive summary (SOMEWHAT) 1 100%33 Vision statement (SOMEWHAT) 0.144 65% X34 Goals meaningful (YES) 0.188 60% * Number of Items Total 7135 Goals meaningful (SOMEWHAT) 0.331 85% Number of items Included 3836 Less than 10 unifying themes (YES) 0.064 55% X Number of items Excluded 3337 Number of goals (<12) (YES) 0.272 65% *38 Number of goals (13-24 goals) (YES) 0.458 90% α % Agree39 Number of goals (>25) (YES) 0.458 90% ≥ 0.33  ≥ 80%Orange Text = new/altered item 27 33Undefined: Krippendorff's alpha is undefined when the item is coded present from all plans or absent from all plans by both independent coders Number of items meeting reliabilitys threshold (highlighted by green box)* Items with reliability lower than 80% agreement were excluded from the analysis. This is indicated by a red X. "YES" and "SOMEWHAT" items were paired together for analysis, even if one of them may have lower than 80% reliability. For an explanation, see the Scoring section under Methodology. 54  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER Plan Proposals: Spatial Design Item Name Description Frequency Land Use Water Does the plan include at least one example of a proposed (specific or community-wide) location or type/intensity/density of land uses being influenced by one or more water/sewer proposals or existing water/sewer services, or does the plan have a general or specific policy to determine the location of land uses based in part on the water/sewer proposals and or water/sewer services? 4 Plan Proposals: Implementation Item Name Description Frequency Implementation Section Does the plan include a separate section/subsection that addresses what needs to be done to implement the plan? 16 Actions for Implementation Does the plan include actions for implementation? 8 Prioritized Actions  Does the plan contain at least one example of one action being prioritized over another?  0 Prioritized Actions – GENERALLY Are the actions listed in the plan generally prioritized in relation to one another?  0 Timelines Implementation – GENERALLY Are timelines for implementation generally identified?  1 Timelines Implementation – GENERALLY Does the plan identify at least one timeline for implementation?  1 Organizations Implementation Does the plan identify at least one specific organization (not just "the town") with responsibility for implementation? 3 Organizations Implementation – GENERALLY Does the plan generally identify specific organizations with responsibility for implementation?  1 Plan Proposals: Monitoring Item Name Description Frequency Monitoring Indicators Does the plan contain a section or subsection that specifically addressing monitoring? 9 Measurable Objectives & Indicators Is at least one goal/policy quantified based on measurable objectives and/or indicators (e.g. 60 percent of all residents within ¼ mile of transit service)? 2 Measurable Objectives & Indicators – GENERALLY Are goals/policies generally quantified based on measurable objectives and/or indicators (e.g. 60 percent of all residents within ¼ mile of transit service)?  1 Organizations Monitoring Does the plan identify at least one organization with responsibility for monitoring?  0 Organizations Monitoring – GENERALLY Does the plan generally identify organizations with responsibility for monitoring?  0 Timelines Updating - BASED ON MONITORING If the plan contains a timeline for updating, is the timeline based at least in part on results of monitoring changing conditions? 2 Encourage Opportunities to Use Plan Item Name Description Frequency Future Scenarios Does the plan include at least two explicit alternative future growth scenarios?  2 Administrative Authority Is the administrative authority for planning indicated? Does the plan mention the required elements of an OCP as outlined in the LGA? 19    Internal and External Quality (Berke et al., 2006) Issues and Vision Statement Item Name Description Frequency Assessment – QUALITATIVE Does the plan include a preliminary assessment of major trends and impacts of forecasted change during the future planning period? 19 Assessment – QUANTITATIVE Does the plan include a preliminary assessment of major trends and impacts of forecasted change during the future planning period? 15 Vision Statement Does the plan include a separate, self-contained vision statement that identifies in words an overall image of what the community wants to be and look like? 15 Fact Base: Description and Analysis of Key Features of Local Planning Jurisdiction Item Name Description Frequency Population Present Composition Does the plan include a description of the composition (e.g. broken down by age and/or gender and/or race, etc.) of the present population? 9 Population Future Composition Does the plan include a description of the composition (e.g. broken down by age and/or gender and/or race, etc.) of the future population? 5 Population Present Size  Does the plan include a description of the size of the present population? 20 Population Future Size Does the plan include a description of the size of the future population? 20 Housing Present Does the plan include at least one descriptive statement of existing housing supply, composition, location, or demand? 18 Land Demand Does the plan include a systematic analysis of how much land will be needed to accommodate projected future growth? 2 Land Supply Demand Analysis Does the plan compare the amount of land needed to accommodate projected future growth with the amount of land available to accommodate future growth? 1 Schools Present Does the plan include at least one descriptive statement of the present state of schools? 14 Schools Future Does the plan include at least one descriptive statement of future needs or plans for schools? 7 Environment Features - VEGETATION & FORESTS Does the plan include at least one descriptive statement about vegetation/forests in/near the community? 15 Environment State - AIR QUALITY Does the plan include at least one descriptive statement about air quality in/near the community? 5 Environment State - DRINKING WATER QUALITY Does the plan include at least one descriptive statement about the quality of drinking water and/or water used for other domestic purposes available to the community? 7 Fact Base: Techniques Used to Clearly Identify and Explain Facts Item Name Description Frequency Tables Format If the plan has at least one table, does every table in the plan have a title and a data source listed? 16 Tables Used Does the plan include at least one table?  17 Data Sources Does the plan cite at least one data source? 10 APPENDIX 4: PROTOCOL SUBCATEGORY FREQUENCIESAPPENDIX 4: PROTOCOL SUBCATEGORY FREQUENCIES AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 55 Communicative and Persuasive Quality (Bunnell & Jepson, 2011) Criteria Related to Readability, Synthesis and Quality of Presentation of Information, Narrative Quality, Persuasiveness and Realism of Plans  Item Name Description Yes Somewhat No React to previous plans  Does the plan consciously embrace or reject previous plans or planning strategies? 12 6 2 Regional context: Acknowledge Does the plan address its relationship with Metro Vancouver by including a summary/excerpt of its regional context statement? 19 1 0  Yes Somewhat No Explain process Does the plan establish its legitimacy by explaining the process followed preparing the plan, and steps taken to obtain wide-ranging participation and input from citizens and stakeholders? 3 8 9 Imaginative and creative Is the plan imaginative and creative (extent of commitment to preparing a meaningful, effective plan)?  0 8 12 Inclusive and Exhorting Does the plan use inclusive language that inspires people to act? Is it written in a way that implies “we can do this together” as opposed “this is what your municipality is doing” 1 2 15 Compelling visuals  Does the plan put forward a compelling vision (through illustrations, photographs, maps, and words) of what the future could be like?  0 12 8 Goals meaningful  Are goals clearly stated, and are they a meaningful guide to action and decision making (i.e. are the goals more than “motherhood and apple pie”? 4 11 5  Goals  ≤12 13–24  ≥25 Number of goals  Does the plan present a limited number of goals? 11 3 6  <20 20–40  >40 Number of objectives  Does the plan present a limited number of objectives? 3 2 15  Yes Somewhat No Recognize uncertainty Does the plan present more than one forecast of future population and/or job growth, and in so doing recognize uncertainty?  1 1 18 Alternate scenarios  Does the plan present alternative scenarios, or at the very least compare the Desired Scenario vs. Trend Scenario? 1 0 19 Explain alternate scenarios  Does the plan provide clear explanations of alternative courses of action that enhance community flexibility and adaptation in dealing with complex situations? 1 0 19 How future will be shaped  Does the plan communicate how future outcomes are likely to be shaped by different policies and courses of action?  0 1 19 Understanding of consequences  Does the plan convey an understanding of the consequences of different courses of action?  0 2 18 Rationales effectively presented  Are rationales for the recommended course of action effectively presented?  3 12 5 Inclusive and Exhorting  Does the plan exhort and inspire people to act?  0 5 15 Maps clear, relevant, and comprehensible  Are maps included in the plan clear, relevant and comprehensible?  14 4 2  Create Clear Views and Understanding of Plans Item Name Description Frequency List of Amendments Does the plan include a list of amendments? 12 Executive Summary Does the plan contain a section entitled "Executive Summary" or "Summary" or "Overview" or something similar that acts as an Executive Summary? 0 Table of Contents Detailed Does the plan include a table of contents that lists both the chapters and the subheadings? 12 Table of Contents Simple Does the plan include a table of contents that lists only the chapters? 8 Glossary Is a glossary of terms and definitions included? 10 Illustrations Are illustrations used (e.g., diagrams, pictures)?  18 Account for Interdependent Actions in Plan Scope Item Name Description Frequency Horizontal Connections Does the plan include at least one horizontal connections with other local plans/programs?  17 Intergov. Coord. Natural Hazards Does the plan include at least one example of intergovernmental coordination for protecting natural hazards?  3 Participation of Actors Item Name Description Frequency Organizations & Individuals Does the plan identify at least one organization/individual that was involved in writing the plan 11 Participation Section Does the plan include a separate section/subsection that describes the public participation process during the development of the plan? 11 Stakeholders Does the plan identify at least one non-governmental stakeholder group that gave input? 10                       APPENDIX 4: PROTOCOL SUBCATEGORY FREQUENCIES56  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER Online Communication Municipal Website Description Frequency Download: If a link to download the OCP is located on the page. 18 Contact: If contact information for the planning department (phone or email) is provided. (not including a general city-wide inquiry line) 11 Who: Does it identify who initiated or enacted the plan? ("The City" or “Council” is sufficient) 7 What: Does it describe what a municipal plan is? (ideally a definition) 15 When: Does it give a year when the plan was developed? (or last updated/amended)? 11 Why: Does it explain the purpose or use of a municipal plan? - this may be described in the same place as the description of "what" is a municipal plan. 14 How: Does it describe how it was developed? (through consultation with the public, for example) 4 PDF Attributes Description Frequency Is the plan downloadable as a single PDF? 17 Is the first page a cover page? (typically with a title, logo, and image) 16 Is the table of contents linked with shortcuts to the corresponding sections in the plan? 4 Is the bookmark feature used to organize the document? 8 Does the PDF page numbering match the document page numbering? 4 Is the document text-searchable? (text can be searched and copied) 18 Are the photographs used generally high-quality (lacking pixellation, jpg artifacts)? 17   Format, Style, and Appearance  Item Name Description Yes Somewhat No Photographs and illustrations  Does the plan contain photographs and illustrations that support the text and add visual interest? 10 8 2 Tables and data relevant  Are tables and other data presented in the plan relevant to the argument that is being made and plan recommendations? 13 5 2  All 4 aspects 3 aspects < 3 or none What/Who/When/How  Does the plan explicitly underscore the importance of implementing the recommendations contained in the plan by laying out a clear path to plan implementation that specifically identifies what, who, when (implementation dates or prioritized), and how funded? 0 1 19  Number of Pages  1-99  100-166 >167 Plan not bulky  Is the plan not overly bulky; that is, is it compact and easy to carry? 6 2 12  Yes Somewhat No Attractive executive summary  Does the plan include an attractive, highly readable, and informative executive summary? 0 0 20     APPENDIX 4: PROTOCOL SUBCATEGORY FREQUENCIES AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER  AN EVALUATION OF OFFICIAL COMMUNITY PLANS IN METRO VANCOUVER 57APPENDIX 5: PROTOCOL SCORES BY SUBCATEGORYInternal and External Quality (Berke et al., 2006)Raw Scores Standardized Scores Raw Scores Standardized ScoresIssues and Vision StatementFact BasePlan ProposalsEncourage Opportunities to Use PlanCreate Clear Views and UnderstandingAccount for Interdependent Actions in Participation of ActorsTotalIssues and Vision StatementFact BasePlan ProposalsEncourage Opportunities to Use PlanCreate Clear Views and UnderstandingAccount for Interdependent ActionsParticipation of ActorsT otal (Standardized)Readability, Synthesis, Presentation, Narrative Quality, Persuasiveness and RealismFormat, Style,                                and Appearance TotalReadability, Synthesis, Presentation, Narrative Quality, Persuasiveness and RealismFormat, Style,                                and Appearance Total (Standardized)*Municipal WebsitePDF AttributesT otalAnmore 3 7 2 2 3 1 3 21 1.0 0.5 0.1 1.0 0.5 0.5 1.0 4.61 34 12 46 0.5 0.6 1.07 2 5 7Belcarra 2 3 2 1 1 0 2 11 0.7 0.2 0.1 0.5 0.2 0.0 0.7 2.34 16 6 22 0.2 0.3 0.52 3 2 5Bowen Island 3 9 2 1 4 1 3 23 1.0 0.6 0.1 0.5 0.7 0.5 1.0 4.41 26 4 30 0.4 0.2 0.56 5 4 9Burnaby 3 8 0 1 3 1 1 17 1.0 0.5 0.0 0.5 0.5 0.5 0.3 3.37 30 8 38 0.4 0.4 0.82 6 5 11Coquitlam 2 8 3 1 4 2 3 23 0.7 0.5 0.2 0.5 0.7 1.0 1.0 4.58 20 8 28 0.3 0.4 0.68 4 4 8Delta 3 10 2 2 4 1 2 24 1.0 0.7 0.1 1.0 0.7 0.5 0.7 4.64 20 8 28 0.3 0.4 0.68 4 5 9D.of North Vancouver 3 6 6 1 4 0 3 23 1.0 0.4 0.4 0.5 0.7 0.0 1.0 4.00 30 8 38 0.4 0.4 0.82 1 4 5D. of West Vancouver 2 10 1 1 3 1 3 21 0.7 0.7 0.1 0.5 0.5 0.5 1.0 3.90 22 8 30 0.3 0.4 0.71 3 3 6Langley Township 1 4 2 0 3 1 0 11 0.3 0.3 0.1 0.0 0.5 0.5 0.0 1.74 12 8 20 0.2 0.4 0.57 3 2 5Langley 1 7 0 1 3 1 0 13 0.3 0.5 0.0 0.5 0.5 0.5 0.0 2.30 18 8 26 0.3 0.4 0.65 4 4 8Lions Bay 2 4 2 1 1 0 0 10 0.7 0.3 0.1 0.5 0.2 0.0 0.0 1.74 16 4 20 0.2 0.2 0.42 5 5 10Maple Ridge 2 10 1 1 3 1 0 18 0.7 0.7 0.1 0.5 0.5 0.5 0.0 2.90 16 8 24 0.2 0.4 0.62 4 7 11New Westminster 2 9 3 1 3 1 2 21 0.7 0.6 0.2 0.5 0.5 0.5 0.7 3.65 24 6 30 0.3 0.3 0.63 4 5 9North Vancouver 3 6 2 1 4 1 1 18 1.0 0.4 0.1 0.5 0.7 0.5 0.3 3.54 26 12 38 0.4 0.6 0.96 2 4 6Pitt Meadows 2 8 0 1 3 1 3 18 0.7 0.5 0.0 0.5 0.5 0.5 1.0 3.70 12 2 14 0.2 0.1 0.27 6 5 11Port Coquitlam 3 9 1 1 2 2 0 18 1.0 0.6 0.1 0.5 0.3 1.0 0.0 3.50 20 6 26 0.3 0.3 0.58 7 5 12Port Moody 3 7 3 1 2 2 2 20 1.0 0.5 0.2 0.5 0.3 1.0 0.7 4.18 28 6 34 0.4 0.3 0.69 6 4 10Richmond 3 10 3 1 3 1 3 24 1.0 0.7 0.2 0.5 0.5 0.5 1.0 4.38 30 8 38 0.4 0.4 0.82 4 3 7Surrey 3 7 2 1 4 1 0 18 1.0 0.5 0.1 0.5 0.7 0.5 0.0 3.28 24 8 32 0.3 0.4 0.73 4 3 7White Rock 3 7 7 1 3 1 1 23 1.0 0.5 0.5 0.5 0.5 0.5 0.3 3.80 22 10 32 0.3 0.5 0.81 3 5 8Max. Possible Score 3 15 14 2 6 2 3 45 1 1 1 1 1 1 1 7 72 20 92 1 1 2 7 7 14Average 2.45 7.45 2.20 1.05 3.00 1.00 1.60 18.75 0.82 0.50 0.16 0.53 0.50 0.50 0.53 3.53 22.30 7.40 29.7 0.31 0.37 0.68 4.00 4.20 8.2* Scores on each subscale was standardized so that categories could be compared against one another. If more than 50% of the items included in a subscale were removed for reliability reasons, then the entire subscale was removed. As a result, the Goal and Policy Framework subscale and Plan Proposals: Spatial Design subscale was removed from the Internal and External Quality Protocol .Communicative and Persuasive Quality      (Bunnell & Jepson, 2011)Online CommunicationAPPENDIX 5: PROTOCOL SCORES BY SUBCATEGORY

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.310.1-0305733/manifest

Comment

Related Items