Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

British Columbia's cumulative effects framework : implementation and contribution to decision-making Vlasschaert, Gillian 2016

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


24-ubc_2016_september_vlasschaert_gillian.pdf [ 1.21MB ]
JSON: 24-1.0305135.json
JSON-LD: 24-1.0305135-ld.json
RDF/XML (Pretty): 24-1.0305135-rdf.xml
RDF/JSON: 24-1.0305135-rdf.json
Turtle: 24-1.0305135-turtle.txt
N-Triples: 24-1.0305135-rdf-ntriples.txt
Original Record: 24-1.0305135-source.json
Full Text

Full Text

BRITISH COLUMBIA’S CUMULATIVE EFFECTS FRAMEWORK: IMPLEMENTATION AND CONTRIBUTION TO DECISION-MAKING by  Gillian Vlasschaert  B.A., Wilfrid Laurier University, 2014  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  MASTER OF ARTS in The College of Graduate Studies  (Interdisciplinary Studies) THE UNIVERSITY OF BRITISH COLUMBIA (Okanagan Campus)  June 2016  © Gillian Vlasschaert, 2016   ii  The undersigned certify that they have read, and recommend to the College of Graduate Studies for acceptance, a thesis entitled:    British Columbia’s Cumulative Effects Framework: Implementation and Contribution to Decision-Making  Submitted by             Gillian Vlasschaert                           in partial fulfillment of the requirements of   The degree of             Master of Arts                                                             Kevin Hanna, Irving K. Barber School of Arts & Sciences, UBC Okanagan Supervisor, Professor (please print name and faculty/school above the line)  Donna Senese, Irving K. Barber School of Arts & Sciences, UBC Okanagan Supervisory Committee Member, Professor (please print name and faculty/school in the line above)  Jon Corbett, Irving K. Barber School of Arts & Sciences, UBC Okanagan Supervisory Committee Member, Professor (please print name and faculty/school in the line above)  Leah Malkinson, Ministry of Forests, Land and Natural Resources Operations University Examiner, Professor (please print name and faculty/school in the line above)  Ross Hickey, Irving K. Barber School of Arts & Sciences, UBC Okanagan External Examiner, Professor (please print name and university in the line above)   June 24, 2016 (Date submitted to Grad Studies)  iii  Abstract Cumulative effects assessment is the systematic evaluation of impacts to the economic, social and environmental condition in combination with other past, present and foreseeable activities on the environment. The province of British Columbia has recently begun the implementation of the Cumulative Effects Framework, a policy that will enable cumulative effects assessments at a broad, strategic scale and provide cumulative effects information to natural resource decision-makers. The objective of this research project is to understand how cumulative effects assessments affect resource management decision-making. This research uses three approaches: 1) analysis of the content of two cumulative effects assessments conducted under the BC Framework; 2) a multiple case study evaluation of ‘pilot projects’ where cumulative effects information has been used in decision-making; and 3) interviews with those who had a working relationship with the Framework and the associated assessments. The findings show that based on early pilot application, the Cumulative Effects Framework has the capacity to meet its identified objectives. Issues of effectiveness were identified in the early stage of implementation resulting from out-of-date datasets and incomplete value assessments. Implementation insights from the analysis include suggestions for maintenance and monitoring practices that can support and improve the ongoing implementation phases, to enhance the functionality and decision-making benefits provided by the Framework. The Framework and resulting cumulative effects information will ultimately only be as useful as natural resource ministries choose to make it, though there are opportunities to improve access and utility.   iv  Preface This research opportunity was made available through collaboration between the University of British Columbia’s Centre for Environmental Assessment Research and British Columbia’s Ministry of Forests, Lands and Natural Resource Operations. The research design was a collaborative undertaking between FLNRO personnel and UBC researchers. While the research question and design were collaborative, the questionnaire was designed for the purpose of the ministry. Dr. Hanna and I provided feedback and suggestions on the questions, but there were limitations in the types of questions that could be addressed, characteristic of any jointly conducted research. The analysis and preparation of this thesis were completed independently, with the valued guidance of my committee. Dr. Hanna provided immense guidance on the entire project, from the research design, data collection through to the final thesis.  FLNRO staff conducted the interviews with CEF information users as part of their evaluation of the CEF pilot projects. The interview data were provided to the Centre for Environmental Assessment Research as secondary data. Within the data collected, there was no identifying information for any of the participants. The research was conducted between September and December of 2015. The University of British Columbia Behavioural Ethics Board approved the use of the secondary interview data provided by FLNRO (certificate number: H16-00775). The Ministry of Forests, Lands and Natural Resources Operations provided Figures 4.1 and 4.2.     v  Table of Contents Examination Committee...............................................................................................................ii Abstract ......................................................................................................................................... iii Preface ........................................................................................................................................... iv Table of Contents ...........................................................................................................................v List of Tables .............................................................................................................................. viii List of Figures ............................................................................................................................... ix List of Abbreviations .....................................................................................................................x Acknowledgements ...................................................................................................................... xi Dedication .................................................................................................................................... xii Chapter 1: Introduction ................................................................................................................1 1.1 Objectives & Research Questions ................................................................................. 12 1.1.1 Conceptual Framework ............................................................................................. 13 1.2 Research Design & Methods ........................................................................................ 15 1.3 Thesis Structure ............................................................................................................ 18 Chapter 2: Policy Evaluations – Definitions, Theory & Practice ............................................20 2.1 Background & Objectives ............................................................................................. 20 2.2 Theoretical Foundations & Approaches ....................................................................... 22 2.3 Procedural Frameworks & Processes............................................................................ 26 2.4 A Brief History of Cumulative Effects Assessment ..................................................... 30 2.4.1 The Current State of Cumulative Effects Assessment .............................................. 31 2.4.2 Cumulative Effects Assessment in Canada ............................................................... 33 Chapter 3: Methodology..............................................................................................................36 vi  3.1 Background & Objectives ............................................................................................. 37 3.2 Case Study Methodology .............................................................................................. 38 3.2.1 Case Study Analysis ................................................................................................. 42 3.3 Interview Methodology ................................................................................................. 42 3.3.1 Interview Analysis – Transcription Process ............................................................. 46 3.4 Rigour & Reflexivity .................................................................................................... 47 Chapter 4: A Multiple Case Study Analysis of CEA Information in the Decision-Making .49 4.1 Background & Objectives ............................................................................................. 49 4.2 Cariboo-Chilcotin Cumulative Effects Assessment...................................................... 51 4.2.1 100 Mile House District ............................................................................................ 55 4.2.2 Tactical Integrated Resource Plan Development in South Chilcotin ........................ 56 4.3 Cumulative Effects Assessment for the Merritt Operational Trial ............................... 58 4.3.1 Merritt Innovative Forest Practices Agreement ........................................................ 61 4.3.2 Merritt Monitoring Program ..................................................................................... 62 4.4 Summary ....................................................................................................................... 63 Chapter 5: The Practice of Decision-Making Utilizing Cumulative Effects ..........................65 5.1 Program Objectives ....................................................................................................... 65 5.2 Implementation Insights................................................................................................ 73 5.3 Lessons for CEA Practice ............................................................................................. 75 Chapter 6: Discussion & Conclusions ........................................................................................77 6.1 Limitations .................................................................................................................... 81 6.2 Recommendations for Policy, Procedure & Decision-Making..................................... 82 References .....................................................................................................................................87 vii  Appendix A: Evaluation Template .............................................................................................99 viii   List of Tables Table 1.1 Auditor General’s Recommendations for the Management of Cumulative Effects ....... 9 Table 2.1 Bardach’s Eightfold Path for Policy Analysis .............................................................. 28 Table 3.1 Decision Case Studies in the Cariboo Region .............................................................. 39 Table 3.2 Decision Case Studies in the Merritt Sub-Region ........................................................ 40 Table 5.1 Interview Rankings for CEA’s Relationship to the Decision-Making Process ............ 69   ix  List of Figures Figure 1.1 Interrelation of the Research Objective, Questions and Chapters. .............................. 19 Figure 4.1 Landscape Units in the Cariboo-Chilotin Region (Dawson, et al., 2014) ................... 54 Figure 4.2 Area Considered within the Merritt CEA (Valdal & Lewis, 2015). ........................... 59 Figure 5.1 Utility of Values Presented in the CEA Information................................................... 67   x  List of Abbreviations AAC    Annual Allowable Uplift BC EAA   British Columbia Environmental Assessment Act BC EAO    British Columbia Environmental Assessment Office CCME    Canadian Council of Ministers of the Environment CEA    Cumulative Effects Assessment CEAA    Canadian Environmental Assessment Act CEF    Cumulative Effects Framework EIA/EA   Environmental Impact Assessment FLNRO   Ministry of Forest Lands and Natural Resource Operations  FPB    Forest Practices Board MOE    Ministry of the Environment  NRPP    Natural Resource Permitting Project OGMA   Old Growth Management Area VEC    Valued Ecosystem Component  xi  Acknowledgements This thesis would not be possible without the assistance and encouragement of my professors, friends and family. The unwavering support of my supervisor, Dr. Kevin Hanna, was the foundation I relied upon throughout graduate school. I am also thankful to the inspiring faculty members of UBCO that I had the pleasure of working with – Dr. Jon Corbett, Dr. Donna Senese, Dr. Rachelle Hole and Dr. John Wagner. During my research, I relied heavily on the assistance and partnership with FLNRO. Special thanks are owed to Leah Malkinson. Without Leah, this research would not have been possible and riddled with missteps.  I owe an immense debt of gratitude to my fellow classmates and university friends, Ailsa Beischer, Sara Vieira, Brent DeWolff and Joanne Taylor, who were not only helpful in academia but in providing laughter and reassurance. To my swan squad – Sarah Medd, Beth Lydon, Brianne Brothers, Amanda Sadler, Cayleigh Dewar, Allanna Casey, Maggie Boothroyd, Niki Kermani and Maggie Goldstein, I thank them for their intelligence, compassion, for offering a couch to crash on and an ear to listen to.  Lastly, to my wonderful family. I will be always in debt to my parents for providing solace, encouragement and horses when most needed. Jordan and Megs, thank you for keeping me grounded, in good humour and well written. I would like to thank the best teacher, Claire Turnbull who showed me what dedication and hard work can lead to, but also when to drop everything and chase them with a pitchfork.   xii  Dedication “The haters are gonna hate, but, well, shake it off.”  1  Chapter 1: Introduction It would be an understatement to say that British Columbia possesses a wealth of natural resources. With competing land use objectives and multiple pressures on the environment and decision-makers1, significant conflicts can occur. Various forms of provincial land use plans cover approximately 90% of the land base, with 37% of this base included in some form of conservation designation (Auditor General of British Columbia, 2015). Development applications for land use require various types of environmental impact assessments (EIA) to support decision-making, falling under federal or provincial jurisdiction. Ideally, EIA promotes sustainable environmental planning through the identification and mitigation of the potential impacts of development and other activities on the environment (Hanna, 2016). In practice, however, the relationship between EIA and decision-making is uncertain. Measuring the impacts, consistency of application, and the appropriateness of assessment and review regulations is a key evaluation challenge in environmental management, and EIA in particular (Rudd et al., 2011).  The lack of substantial research on the operational or decision-making effectiveness and influence of environmental assessment (EA) affects the perceived value and efficacy for such approaches (Baker & McLelland, 2003). Governments not uncommonly choose suboptimal policy instruments based on criteria that can be too broad to be useful, or for ambiguous socio-political reasons (Jaccard & Rivers, 2006). In this vein, as the root causes of environmental problems become increasing complex, the                                                  1 Decision-maker: individual in a position of decision-making authority, typically at a high level within an organization.  2  role of natural resource management within deliberative democratic settings may become less about scientific rationality and more about responding to social demands (Parkins & Mitchell, 2007).  The increasing complexity may affect the role of EIA in managing the impacts of development, and its role in supporting decision-making, in part because it is often confined to specific projects and can be disconnected from larger spatial and temporal impacts. Cumulative effects assessment (CEA) may help fill this void. In the past, CEAs were considered primarily as an additional component to EA. Individual projects had designated effects on the environment, but the combined outcomes of each unrelated task was no more notable that the sum of those distinct effects (MacDonald, 2000). However, ideas have begun to change, as cumulative effects on the environment are now considered to exist in a synergetic manner. Individual projects do have their own unique impacts on the environment, but the combine outcomes of multiple projects can result in an impact greater than the sum of their individual impacts (Crain, Kroeker & Halpern, 2008; Noble, 2010). This approach to assessment allows both proponents2 and decision-makers to better analyze information to understand the current environmental conditions and trends (Heggman & Yarraton, 2011).  The importance and inclusion of cumulative effects assessment is growing in recognition across governments, evident by its incorporation into policy and legislation (Duinker & Greig, 2006; Pope et al., 2013). In Canada, there has also been an increased interest in CEA, albeit quite variably across the country.                                                  2 Proponent: the party proposing a development and as such required to prepare the appropriate environmental assessment (Hanna, 2016) 3  British Columbia is in the midst of adopting a substantial CEA model, though it is still in early implementation stages. Despite the vast interest in CEA, there is considerable disagreement amongst practitioners on numerous aspects of its structure, including definitions, appropriate outcomes, and sufficient methods for determining effects from other projects (Bond & Pope, 2012; Duinker & Greig, 2006; Harriman & Noble, 2008; Noble 2010). There are many challenges in carrying out an effective CEA, with some contending that CEA has failed to deliver on the once promised potential (e.g. Noble, 2010; Parkins, 2011). Challenges limiting the value of current CEAs include: poor quality of practice, institutional inadequacies, poor consideration of alternatives and inadequate cumulative effects management (Pope et al., 2013). Morgan (2012) linked the reason for the poor development and utilization of CEA to a lack of a clear methodology. A key issue in CEA methodology is CEA’s application to individual project based evaluations, which does not allow the assessment to adequately understand and consider the impacts of development across the landscape (Harriman & Noble, 2009). Some have argued that the relationship between CEA and individual projects underlines many of its perceived shortcomings.  There has been a call for a structural framework of strategic-level CEA approaches to ensure cumulative effects are considered within the decision-making process, while still ensuring methodological quality (Therivel & Ross, 2007).  Not only is CEA better suited methodologically to a strategic and broad scale, but is also particularly relevant to long-term planning frames preferred by some government processes (Parkins, 2011). In that long-term context, there may be less temptation to choose alternatives that 4  favour short-term results, a potential outcome of project-based assessment (Therivel & Ross, 2007). While CEA has been promoted and utilized as a tool for decision-makers, there is a need to improve institutional and governance arrangements, to better emphasize the long-term advantages to taking a CEA approach (Parkins, 2011; Therivel & Ross, 2007).  In British Columbia, CEA has been employed on an ad-hoc basis with little policy foundation. Similar to their federal counterparts, the BC Environmental Assessment Office (BC EAO), the agency that administers the province’s Environmental Assessment Act (EAA), published a user guide for environmental assessment with a brief section on cumulative impacts (EAO, 2011). The EAA guides the proponent led assessments for major projects suspected to have adverse environmental impacts (Rutherford, 2016). The EAA mentions cumulative impacts as a single component that may be required as a portion of a standard environmental assessment at the discretion of the director (EAA, s.11(2), 2002). Prior to 2011 no mechanism existed to consider the cumulative impacts of many smaller activities that were exempt from conducting a CEA through the EAA.  British Columbia has recognized the need to move to more integrated approaches of resource management from a regional perspective as opposed to a sector-by-sector or individual project basis (FLNRO, 2014). However, despite the acknowledged importance of cumulative effects, few supporting policies or guidelines exist and very few reviews have been conducted on these assessments (Auditor General of British Columbia, 2015; Duinker & Greig, 2006; Pope et al., 2013). A 2011 report published by the Forest Practices Board (FPB) examined the need to better assess cumulative effects in British Columbia (FPB, 2011). FPB concluded that the current means for CEA are largely 5  inadequate and subsequently leading to the mismanagement of natural resources (FPB, 2011). The report proposed recommendations so that appropriate methods could be applied throughout a land management framework.  British Columbia has taken a number of steps to improve environmental management and decision-making. The Ministry of Forests, Lands and Natural Resource Operations (FLNRO) was formed in 2012, bringing together several areas of resource management that had previously been under the mandate of difference ministries. The broad responsibilities of the new agency include creating policies and conditions for the province’s forests, lands and natural resources while also serving as the legislative authority for permitting, licensing and inter-agency coordination (FLNRO, 2012). Together with the Ministry of the Environment, the two ministries are co-leading development and implementation of a province-wide Cumulative Effects Framework (henceforth referred to as the Framework or CEF). The Framework is the provincial government’s effort to address the short comings of past evaluation practices by promoting policies and tools which foster the periodic assessment of cumulative effects at a broad, strategic scale (FLNRO, 2014). The CEF is meant to provide benefits in: improving the ability to manage for the desired outcomes for values; supporting the assessment of potential impacts to First Nations rights and interests; supporting streamlined decision-making; and supporting transparent and durable decisions (FLNRO, 2014).  One of the key challenges conducting a CEA is the multiple geographical scales that might or should be considered within an assessment. The geographical scale chosen within British Columbia reflects the values defined through stakeholder consultation and 6  the nature of industrial development in the area (FLNRO, 2014). Values chosen for the CEF were selected to be representative of the landscape and as broad as possible (FLNRO, 2014). In BC this has led to the suggestion of regional and sub-regional scale assessments as providing the most effective and financially efficient option (FLNRO, 2014). Existing natural resource districts or strategic land use planning areas were considered the most effective administrative units (FLNRO, 2014). The CEF is focused on using data to support more informed decision-making. The Framework builds on the foundation of land use planning through which multi-stakeholder engagement has defined key valued ecosystem components (VECs) and objectives for their management (Auditor General of British Columbia, 2015). The Framework is based on identified VECs that are evaluated in regional and sub-regional CEAs relative to existing, government approved objectives for the condition of values. The information used to assess the condition of the VECs is collected from existing inventories and provincial monitoring programs and integrated into a database to serve as baseline information. The values were chosen to bring consistency across the province. Standard assessment protocols are being developed for each core value, with allowance for regional variation in the assessment protocol where warranted (FLNRO, 2014). To select CEF values the provincial government uses three criteria: 1. Values that have legal or policy objectives already in legislation, land use plans or other forms of management direction; 2. Values identified in strategic agreements with First Nations, or otherwise identified as supporting an Aboriginal or treaty right; and  3. Values that can be mapped and have robust existing data (FLNRO, 2014).  7  It is also an objective to make VECs and other cumulative effects information more accessible, understandable and useable to decision-makers. This in itself is a novel and potential important contribution of the CEF to EA practice. FLNRO has made a number of initial implementation commitments: completing and evaluating current CEF operational trials and supporting implementation in decision-making in those areas; completing CE assessments currently underway or proposed in other priority areas across the province; developing the values foundation for a core set of values across the province; etc. The Framework is an integral component of the Natural Resource Permitting Project (NRPP). The NRPP was developed by the province in 2014 to bring together the multiple agencies invested in natural resource management. A key objective is efficiency. This emphasizes a streamlined approach to management and a more integrated authorization process to improve and shorten the decision-making process. The NRPP seeks to consolidate the natural resource sector (Government of British Columbia, 2016). The NRPP is being implemented gradually throughout the province, beginning with three aspects:  1. Verified map data and information will be accessible to the public and proponents; 2. Natural resource projects will be monitored throughout the project lifecycle including regulatory approval, construction, operation, compliance and enforcement of post-project obligations; 3. Stewardship values and tools will be incorporated in authorization decision making and project management; and  8  4. Lastly, government will be better equipped to estimate project application approval times (Government of British Columbia, 2016).  The NRPP initiative will integrate business plans, legislation and processes from the multiple agencies affected and included within its mandate (Government of British Columbia, 2016).  Similar to the Framework, the NRPP is intended to support and streamline existing process, not replace them (FLNRO, 2014). The CEF is meant to provide the organizational structure to encourage interagency collaboration of management actions and open access to cumulative effects information for clients, the public and decision-makers (FLNRO, 2014). Therefore, the CEF provides the base upon which the NRPP has been developed. In 2015, The BC Auditor General undertook an audit to measure effectiveness evaluations for FLNRO’s capabilities of cumulative effects management. The Auditor General released a report on May 26th 2015, entitled Managing the Cumulative Effects of Natural Resource Development in B.C. Nine recommendations surfaced and were reported to the provincial government. The recommendations can be categorized in two ways: 1) actions to be undertaken by the Government of British Columbia and 2) actions for FLNRO (Auditor General of British Columbia, 2015).     9  Table 1.1 Auditor General’s Recommendations for the Management of Cumulative Effects Actions Required By Recommendation        Government of British Columbia 1. Assign the province’s natural resource ministries and agencies clear roles and responsibilities on managing cumulative effects. 2. Introduce tools, such as legislation and policy, which will enable the province’s entire natural resource sector to coordinate CE management.  5. Establish and/or update, values that are important for the province to sustain and the acceptable conditions for those values. 7. Establish how CEAs will be used to inform and support natural resource development decisions by ministries in all resource sectors.   10  Actions Required By Recommendation       Ministry of Forests, Lands and Natural Resource Operations 3. Provide decision-makers with instruction and training on how to assess and manage CE when authorizing natural resource use. 4. Consider CE when authorizing natural resource development and document the rationale for its decisions. 6. Monitor the condition of values and make that information available to decision-makers. 8. Annually report to the Legislative Assembly on implementation of the CEF and how risks are being managed. 9. Assess options for accelerating full implementation of the CEF and submit a proposal to Cabinet for decision.    The report noted the lack of legislative and government directives dictating the consideration or management of cumulative effects when authorizing natural resource use (Auditor General of British Columbia, 2015). While it did see the CEF as promising, the report was critical of the lengthy timeframe for CEF implementation (Auditor General of British Columbia, 2015). The audit recognizes the importance of understanding how the government will use the information produced in the CEF in decision-making for natural resource development, a key component of this research. In many respects, the Auditor’s report was premature, focusing on a new program still in the initial stages of design, 11  refinement and gradual implementation.  Regardless of prematurity, the critique of the relatively long rollout period is notable. The two CEAs completed as part of the ‘pilot phase’ of the CEF development have been finished and an analysis of them will allow for identification of potential improvements to the policy and associated practices, permitting more effective widespread implementation. Within the Cariboo region, the CEA was conducted for the entire region. The second assessment was conducted for the Merritt sub-region within the larger Thompson-Okanagan region. These two broad scale assessments utilized existing baseline data within the appropriate natural resource district to determine the current condition of values (including moose and watershed assessments) in relation to existing legal or policy objectives for each value and risk benchmark (FLNRO, 2014). The Framework is intended to allow decision-makers the opportunity to consider the effects from all activities and natural processes in the environment in an effort to avoid adverse environmental, economic and social consequences (FLNRO, 2014).  Both assessments identified emerging trends based on foreseeable future activities on the environment. The knowledge of current conditions was meant to provide decision-makers with a measure of risk associated with each value (FLNRO, 2014). Within the Cariboo region and Merritt sub-region, four decisions were identified that had considered CEA data in the decision-making process. There were two decisions for each area respectively.  The CEA for the Cariboo region was applied to two separate and distinct decision-making contexts. The first case study is for the 100 Mile House District where CEA information was applied to a decision surrounding the issue of increasing cutting permits. The second case study purposefully selected for this region details the 12  experience of CEA information used in the development of a tactical integrated resource plan and the subsequent consultation and engagement with First Nations. Within the Merritt sub-region, CEA was relied upon for an Innovative Forest Practices Agreement and to inform priorities for field-based monitoring goals. The monitoring program conducted in the Merritt sub-region utilized the CEA information to allocate the resources available for the district and to determine the highest priorities for management.  FLNRO plans to have the CEF fully implemented and completely operational throughout the province by 2021 (FLNRO, 2014). There is a need to address the adequacy of the CEAs completed under the Framework and determine how the data presented in the CEAs was considered within decision-making. Examination of the implementation procedure will identify opportunities for modifications that will help the implementation of the CEF. Ideally this supports natural resources management and decision-making in British Columbia, and may help serve as a model for other Canadian jurisdictions.  1.1 Objectives & Research Questions The overarching objective is to understand how cumulative effects assessment information affects and/or informs decision-making. Information from this study can be used to advance the subsequent implementation phase of the Framework, thus building a connection between research and applied use. With the objective in mind, there are three main research questions: 1. Have the CEF program objectives been met? Or to what extent? 13  2. How can the BC experience of the CEF implementation inform the development and improvement of policy, procedures and standards for cumulative effects assessment? 3. What lessons does the BC CEF experience hold for EA and CEA practice and implementation broadly throughout Canada?  1.1.1 Conceptual Framework This research employs a qualitative approach to data collection and analysis. As it is critical to understand the human relationship between the decision-making process and the CEF, qualitative inquiry was considered the appropriate choice. Qualitative inquiry encompasses various tools and a wide variety of methodologies, linking it back to an overarching paradigm (Levy, 2007).  The paradigmatic positioning that guides this study is action research with a pragmatist view. Action research has emerged from complex beginnings; its origins cannot be linked to a single discipline, but instead tied to a number of fields and practitioners (Brydon-Miller et al., 2003; Huang, 2010; McNiff, 2013). Action research allows the researcher to renounce their own expert knowledge by recognizing the skill, experience and comprehension of others (Martin, 2012; Stringer 2007). The researcher is able to “provide information that enables those responsible for making informed judgments about their activities, thus increasing the possibility that their policies, programs and services might be more appropriate and effective for the people they serve” (Stringer 2007, p.171). 14  Action research does resemble some qualitative research methods. However, qualitative research focuses on practice compared to the emphasis on practitioners in action research. This pivotal distinction regularly leaves the qualitative research 'inactionable', or not something that experts can make functional (Huang, 2010). Action researchers acknowledge that knowledge can be created and separately knowledge can be used (Robertson, 2000). It is believed that only through action is real understanding conceivable. As Huang stated, “Theory without action is not theory but speculation” (2010, p. 93).  Pragmatism views qualitative research through a different, but complimentary, lens. The fundamental ontology of pragmatism is focused on actions and changes within a world constantly shifting and evolving (Blumer, 1969). Action is considered as performing a connecting role as a way to create meaningful existence (Goldkuhl, 2012). In order to bring change in a desired way, the required actions must be informed by purpose and knowledge (Goldkuhl, 2012). Dewey also strongly influenced pragmatism. Dewey’s (1938) concept of inquiry is critical in the application of pragmatic positioning towards research and should be viewed as a systematization of humans’ efforts to improve their situation such that knowledge may guide a controlled change. Pragmatism does not dismiss the concept of a whole truth, but rather believes that these so called truths correspond to small samples of the entire reality (Goldkuhl, 2012). Pragmatism is therefore largely concerned with not only explaining and theorizing current situations, but also generating knowledge to guide action in order to realize the possibility of what ‘might be’ (Goldkuhl, 2012).  15  The results of applied research – such as this study – are to be reflected upon by decision-makers, in the hopes of creating positive change (i.e. effective program implementation and maximizing the likelihood of policy success). The combination of these paradigms appropriately reflects the complex undertaking that is policy evaluation. Action research highlights the importance of a transformative orientation to knowledge creation, while pragmatism guides the research with the overarching objective of producing useful knowledge to guide meaningful change in environmental policy and its relationship with decision-making (Brydon-Miller et al., 2003; Dewey, 1938; Goldkuhl, 2012).  1.2 Research Design & Methods In order to answer the research questions – program objectives, implementation insights and general CEA implementation insights – the research requires a multi method approach and therefore has three major components. Each are detailed in subsequent chapters: 1. A literature review with definition of status of knowledge performed as a content assessment of policy evaluation practices,  2. A document analysis evaluation of CEF implementation areas throughout British Columbia, and  3. Interviews with those who have technical and regulatory knowledge of the CEF and associated CEAs.   16  1: A Summary of Previous Literature & Status of Knowledge   Through a preliminary examination of relevant peer reviewed literature and past policy practices, it became evident that there is a lack of information in assessment research illustrating the role of information and different data forms. This extends from CEA processes in multi-sector communication, to knowledge development for participation purposes, and to decision-making. Heggman & Yarraton (2011) suggested a need to improve understanding of the institutional framing and contextual influences of the decision-making process.  Morgan (2012) describes CEAs and procedures as underdeveloped and ineffective. By extracting prominent themes from the literature, this research was able to construct a conceptual and theoretical foundation to support the case study and interview analysis. This component corresponds to the findings detailed in Chapter 2.   2: Multiple Case Study Analysis of CEA Information in the Pilot Regions The primary purpose of case studies is to increase the theoretical understanding of the subject area in question (Thomas, 2005). Many qualitative researchers now view case study analysis as a central component of the research portfolio (Levy, 2007; Yin, 2009). As noted earlier, the case studies used here are in British Columbia and focus on the province’s relatively new Cumulative Effects Framework.  Two CEAs were conducted as trial assessments in their natural resource districts, the Cariboo region and the Merritt sub-region of the Thompson-Okanagan region for the purpose of testing and demonstrating the proposed CE Framework. The two CEAs and associated information were then applied to decision-making contexts. Two decisions for 17  the Cariboo region and the Merritt sub-region CEA respectively were used as trials for the larger policy vision and considered in this research. Each trial offers a different approach to implementation and developing assessments that support the decision-making process. The resulting case studies and document review also served to provide the background information that formed the foundation for reviewing the evaluation results that came from the interviews (conducted by FLNRO). The documents reviewed for the Cariboo region and Merritt sub-region can be found in Table 3.1 and 3.2 respectively. Further information on the analytical process for the case studies can be found in Chapter 3 with the results following in Chapter 4.   3: The Utilization of the Cumulative Effects Framework in British Columbia FLNRO have used interviews as a data collection technique to help in their program evaluation over the past year. The agency provided the results of four evaluation interviews to help in this research. The semi-structured interview approach used by the agency allowed for these questions to be based on previous knowledge and developed in advance, but prevented the opportunity for unplanned questions resulting from respondents’ answers (Richards & Morse, 2013). The questions covered aspects of natural resource management decision-making and program implementation. The questions provided insight into the extent that CEA information is sufficient to support decision-making and the degree to which resource decisions are executed differently as a result. This is a key area of interest to cumulative effects assessment practice and to environmental assessment research more generally. 18  Examining the decision at hand and the values that were assessed in the CEA and considered in the respective decision established key context during the interview process. Detailed information can be found in Chapter 3 on all components and analytical procedures. The results follow in Chapter 5.   1.3 Thesis Structure Figure 1.1 provides a summary for the structure and interrelations found within this thesis. Chapter 2 offers a review of the relevant policy and peer reviewed literature that was informative for background and context. Chapter 3 describes the methodological underpinnings that focused the research. In Chapter 4, the two pilot CEA reports that were conducted for the Cariboo region and Merritt sub-region are reviewed, followed by an examination of documents from the four decision contexts to which they were applied (two for the Cariboo region and two for the Merritt sub-region). In Chapter 5, the Cumulative Effects Framework is characterized from the perspective of individuals with technical and practical knowledge of the policy instrument. This is applied to the findings and methodological approach from Chapters 2 and 3 by evaluating the decision-making process in the four included case studies. In Chapter 6, the results from the previous chapters are integrated with the current context of the Cumulative Effects Framework and recommendations are given for the improvement of cumulative effects assessment throughout the province.    19                    Figure 1.1 Interrelation of the Research Objective, Questions and Chapters.  Objective One:  Understand how cumulative effects assessment information affects decision-making. Information from this study will be used to advance and aid in the subsequent implementation phase of the Framework.  Question One: Have the CEF program objectives been met? If so, to what extent?  Question Two: Can the experience of the initial CEF implementation inform the development and improvement of policy, procedures and standards for cumulative effects assessment?  Question Three: What lessons does the BC CEF experience hold for the CEA practice and implementation that will improve CEA application broadly?  Chapter Two:  Literature Review Chapter Three: Methodology  Chapter Four: Case Studies   Chapter Six: Conclusions and Directions for Future Research  Chapter Five: Evaluation Interviews 20  Chapter 2: Policy Evaluations – Definitions, Theory & Practice  This chapter outlines the literature on policy evaluation and relates this evaluation to decision-making and cumulative effects assessment3. Emphasis was placed on recent works, from 2005 onward; references for important works conducted before 2005 were also identified.  The emphasis on recent works was critical for this research as the Framework is a relatively new endeavor and is still in the process of implementation. It is important implementation is guided and informed by the most recent thinking on policy implementation practices to highlight opportunities for adjustments that will serve to strengthen the CEF. This particular literature review was conducted for the purposes of the CEF in British Columbia; findings within it may assist in other environmental policy implementation practices and cumulative effects research in other jurisdictions.   2.1 Background & Objectives  Policies are born from complex organizations and institutions, involving many influences and actors from various sectors of society, complicating the evaluation process (Maas et al., 2011). Different levels of hierarchical organizations are involved at variable times throughout the policy-making process (Trochim, 2009). The specific nature of these policies and their effectiveness has been described as rational and goal-oriented (Day et al., 2002; Maas et al., 2011). Governments will use the policy agenda to set goals                                                  3 A systematic literature review was undertaken using search engines such as JSTOR, Compendex, GEOBASE, GeoRef and Science Direct. Keywords included policy evaluations, policy implementation, politics, environmental politics, and cumulative effects, cumulative effects assessment, cumulative effects framework and decision-making. 21  for their selected term and associated guiding policies in order to fulfill these goals (Jarrell et al., 2013; Maas et al., 2011). DeGroff & Cargo summarize several important works concerning policy development (Brewer, 1974; Jenkins, 1978; Laswell, 1956) into a six-step process in as follows: 1. Agenda setting 2. Issue definition 3. Policy formulation 4. Policy decision 5. Policy implementation 6. Evaluation and maintenance, succession or termination of the policy (p. 48-49, 2009).  Individual decision-makers directly influence the general effectiveness of management measures and their response to the presented CEA results (Therivel & Ross, 2007). There is a growing recognition within the political environmental of the need to differentiate policy measures that are effective and promise future gains in their respective sectors from those that do not (Maas et al., 2011). Future policy measures could better encourage the ability of policy to serve as a communication channel between government and society, therefore promoting transparent decision-making processes (Sanderson, 2009). Further evaluations and understanding of decision-making may lead to a higher degree of transparency in the democratic setting while also improving public approval (Trochim, 2009). Additionally, as policy divisions are becoming increasingly blurred, governments are beginning to view complex issues, such as environmental decisions, in interdisciplinary terms (Bruyninckx, 2009).  22  A policy’s primary commitment is to achieve the stated objective (Day et al., 2002) and it is the responsibility of those within the political system to ensure that these objectives are met (Heggman & Yarraton, 2011). This task, however, has a high degree of associated complexity (Howlett, 2007; Wholey, 2004). The lifecycle of a policy, its implementation and ramifications are a dynamic and evolving process, influenced by the organizational structure and the sociopolitical context (DeGroff & Cargo, 2009). It is important that researchers contemplating policy evaluations have an understanding of the complexities and challenges associated with assessment at any stage (DeGroff & Cargo, 2009). Policy evaluation is used to help predict the overall benefits that would be received following the implementation of a policy (Daan & Neumann, 2014). Policy evaluations are untaken for numerous reasons including learning, development of policy and improving the accountability of decision-makers (Mickwitz, 2006). As Bardach (2000) notes, “Policy analysis is a social and political activity.” The literature can be broadly grouped into theoretical foundations and approaches, and procedural frameworks and processes.   2.2 Theoretical Foundations & Approaches Policy evaluations are meant to gain insight into specific effects and to promote future success within similar situations (Daan & Neumann, 2014; Maas et al., 2011). Traditionally, policy evaluations have been focused on the defined objective, but there is no solitarily accepted form within the theoretical approaches (Day et al., 2002; Trochim, 2009).  23  Policy implementation has been appropriately recognized to have unique features, as outlined by Parsons (1995, p. 461), “A study of implementation is a study of change: how change occurs, possibility how it may be induced.” The standard policy implementation was previously a centralized top-down model derived largely from a rational management perspective (DeGroff & Cargo, 2009). Characteristics associated with this approach include a controlled, strong management and compliance to guarantee conformity with the principle objectives of the policy (DeGroff & Cargo, 2009). Bresser et al. (2013) voiced concerns that examining a policy based primarily on the objective have led to limited evaluations that lack depth. In contrast to the historical approach, there has been movement amongst scholars within the policy field to shift towards a more democratic undertaking of public policy, including the implementation phase (DeGroff & Cargo, 2009).  When long-term objectives are the principle purpose of a policy implementation, it is difficult for evaluations to be able to attribute the distinct contributions of specific programs from the credit due to higher-level policies (DeGroff & Cargo, 2009). As the anticipated outcomes of a proposed policy have a higher degree of uncertainty than those which result from an existing policy (Maas et al., 2011). Despite these challenges, it is critical that the implementation of any policy is followed by an evaluation of any resulting changes to the human behaviour to determine if the policy objective is truly being reached (Bailey et al., 2011; Mickwitz & Birnbaum, 2009; Wholey, 2004). Mickwitz (2006) proposes intermediate evaluations to combat these challenges. Such an analysis focuses on tracking the development of a policy throughout its lifecycle, identifying its impacts and illustrating emerging outcomes (Mickwitz, 2006). Mickwitz’s 24  intermediate policy evaluations are similar in structure to the study of policy implementation proposed by DeGroff & Cargo (2009). Additionally, Hilden’s (2009) recent work on time horizons outlines the differences between the evaluations based on their time horizons. Hilden (2009) describes the characteristic of midterm evaluations as tracking the development of the policy, specifying the outcomes, identifying the results and noting the weaknesses that materialize.  The inclusion of temporal aspects in policy evaluation stems from the recognition of the complex interaction and complexity between various policies and contributions of other decisions (Bresser et al., 2013). The temporal aspects of policy evaluations are extremely important, as there is a direct relationship between the temporal qualities and the success or failure of the policy (Bresser et al., 2013). There are various dimensions of time that are common throughout the literature including timescales, course of time (uncertainties and complexities) and time framing (time in the hands of policy makers) (Bresser et al., 2013). The temporal scale and timeframe in the evaluation must be clearly defined, based on a thorough knowledge of the processes that the policy may affect (Hilden, 2009). The mismatch of ecological and social scales provides an additional layer of complexity to policy evaluation. This occurs when the scale of the environmental problem does not correspond with the social scale and or policy level (Bruyninckx, 2009). It is important that at the beginning of a policy evaluation the scale of policy coincides with the problem being addressed. Bresser et al. (2013) suggest a temporal dimension of ten years or more, especially in complex implementation scenarios involving multiple government layers.  25  An example of a large temporal consideration with policy and environmental planning is the Thames Estuary Flood Management Strategy, spanning until the year 2100 (Therivel & Ross, 2007). However, even such large temporal scales still present challenges for policy evaluators. That study highlights future developments on the Thames are unknown past 2025 and the longest evaluation available currently considers only until 2075. There is considerable uncertainty and potentially imprecise assumptions at play within these future scenarios. To address this, it is often suggested that multiple negative and positive predictions of future scenarios are provide in any future oriented policy evaluation (Therivel & Ross, 2007).  Time has the ability to reshape policies and political change; decisions originally intended to resolve short-term issues can have long-term consequences (Bresser et al., 2013). The theoretical foundations of policy evaluations need to strive to develop tools and instruments that serve as communication channels between government and society, promoting transparent decision-making processes (Sanderson, 2009). Additionally, as policy divisions between sectors and agencies are increasing blurred, governments are appropriately beginning to view complex issues, such as environmental management decisions, in interdisciplinary terms (Bruyninckx, 2009). This certainly reflects the growing acknowledgement that the environmental problems are inherently complex, require interdisciplinary knowledge and are cumulative in their synergies and outcomes.  Further evaluation and understanding of the decision-making process and its implications may lead not only to a higher degree of transparency in the democratic setting, but also to improved public approval for an individual, corporation, agency or government (Trochim, 2009). The interdisciplinary nature of environmental problems, 26  and the resulting policies, suggests that practitioners and evaluators will require a comparable range of approaches and tools to guide evaluations and to ultimately gain insight into the effectiveness of policy.   2.3 Procedural Frameworks & Processes  Undertaking a policy implementation evaluation is a dynamic and evolving process. The analysis of a policy, throughout its implementation process allows the evaluation to provide insight into the complexities and interactions influencing the overall effectiveness (Flamos & Spyridaki, 2013; Wholey, 2004). A policy is rarely dictated by individual issues, but will encompass political interests, societal awareness and behaviours (Cabugueria, 2000). A multitude of procedural frameworks and processes can be found throughout the policy evaluation literature. These methods can assist in producing relevant knowledge to inform future actions (Goldkuhl, 2012; Mickwitz & Birnbaum, 2009).  The traditional approach taken by bureaucracies in environmental policy has been through command-and-control regulations and legislation (Svenfelt et al., 2010). Recent approaches, by contrast, consider environmental policy through modification and adaption. Decision-makers, researchers and experts are all pivotal players in the development of relevant and robust environmental policy and management (Spruijt et al., 2014; Svenfelt et al., 2010). Howlett (2007) outlined several components of successful policy creation in a recent study. Firstly, government actors must hold the main responsibility for agenda setting initiatives and allow the process to progress logically towards the final decision (Howlett, 2007). Secondly, these same actors must reinvent 27  themselves at different stages of the decision-making process in order to serve in distinctive roles and functions (Howlett, 2007). Lastly, the features of the respective government agency have a significant influence on the particular agency’s ability to mold the final decision according to their preference (Howlett, 2007). The agency shaping the policy must be aware of their influence and create a balanced and effective outcome (Howlett, 2007).  The Government of Canada outlines several principles through which policy development may be improved through utilizing scientific advice (Rudd et al., 2011). The communication between decision-makers and researchers must be open and transparent (Rudd et al., 2011). Rigorous review of the process taken by scientists should be on a continuous basis through measures such as assessing, communicating and identifying uncertainty and risk (Rudd et al., 2011). Lastly, review of important prior decisions may highlight whether advances in knowledge resulted from similar policies or that particular policy in the past (Rudd et al., 2011).  There are certain classifications and characteristics of policy that must be defined in order to understand evaluation procedural frameworks and processes. The following definitions, suggested by Flamos & Spyridaki (2013), form the basic performance criteria of policies to inform evaluations. The first is effectiveness, serving to demonstrate the extent to which the overall objectives of the policy were met (Flamos & Spyridaki, 2013). Efficiency is defined as the ability of the policy to anticipate both the estimated and real costs (Flamos & Spyridaki, 2013).  The authors also make a distinction between two types of efficiency – static and dynamic. The former refers to specific policy instruments and their ability to obtain the desired outcomes under a situation of constraint resulting 28  from a lack of communication (Flamos & Spyridaki, 2013). Meanwhile, dynamic efficiency is concerned with the instrument’s capacity to promote cost reductions through technological advances (Flamos & Spyridaki, 2013).  Bardach (2000) suggested that while policy evaluations might require the guidance of formulas and structures they must also strive to remain flexible to allow tailoring to specific circumstances that are characteristic of qualitative inquiry. The author suggested an eightfold path for policy analysis.   Table 2.1 Bardach’s Eightfold Path for Policy Analysis Steps  Description   1. Define the problem Provides the reason for the evaluation, in addition to assisting in the structure of the final step.  2. Assemble some alternatives Data provides an understanding and a baseline for the analysis. Alternatives can be conceptualized as policy opinions to alleviate whatever the problem is that policy is designed to address.  3. Construct the alternatives   29  Steps Description    4. Select the criteria Determining if the policy in question will result in a positive impact for the community, allowing for the influx of values and perspective into the policy analysis.   5. Project the outcomes Considering all associated outcomes for each alternative that may be applicable to the specific needs of the evaluation.   6. Confront the trade offs It is characteristic of any policy decision to involve trade offs and compromises, the most common of these is when a trade off involves financial resources; therefore this step often also requires a trade off analysis.  7. Decide The researcher recommends what they consider to be the best direction for the respective evaluation.  8. Tell your story The assessment is ready to be presented utilizing a presentation method that best fits the targeted audience.  (Bardach, 2000).   30  The steps are not necessarily to be taken in order, nor is every step relatable to each policy evaluation; rather, they offer some guidance to the process by placing critical concepts, such as defining the problem and the consideration of alternatives, at the beginning (Bardach, 2000).  2.4 A Brief History of Cumulative Effects Assessment Previously, cumulative effects were considered principally as additional information to project level EAs. As each new project contributed impacts on nature, the combined outcome of each single unrelated task could be no more significant than the sum of their parts (MacDonald, 2000). Contemporary thought now considers cumulative impacts on the environment to exist in a synergetic manner as the widespread effect of all environmental interactions can reflect more than the sum of individual processes or projects' impacts (Crain, Kroeker, & Halpern, 2008; Noble, 2010). This highlights the importance of considering all cumulative impacts of individual projects, as well as continued industry development in a regional context (Noble, 2008).  Individual developments are often focused on achieving expeditious project approval. In previous CEA experience, proponents have been eager to complete the assessment to the minimum requirements stipulated. Often, this results in assessments becoming an afterthought, rather than considering them as a central and strategic decision-making tool. A proponent can be left to manage their own effects, but cannot be expected to mitigate effects resulting other developers (Therivel & Ross, 2007). So accounting for such synergies may have little practical value at the level of project-based assessment. Thus, historical project-based CEA contributed little strategic advice 31  regarding temporal impacts and guidance to future developments (Duinker & Greig, 2006). The focus within CEAs on valued ecosystem components (VECs), originated during the 1980s to provide focus to the assessments (Beanlands & Duinker, 1984; Canter & Atkinson, 2011; Harriman & Noble, 2011). VECs are defined as the resources, ecosystems, or societies that stakeholder and other interested parties deem critical and thus are theoretically assessed within the CEA (Canter & Atkinson, 2011). Promoting the sustainability of VECs within human development underlines the central premise of CEA (Duinker & Greig, 2006; Harriman & Noble, 2008). A holistic CEA would not only address the impacts expected to VECs, but would also assess alternative future scenarios and the resulting influence on environmental sustainability each would have (Senner, 2011).   Individual projects often represent a small stressor to VECs, but the sum of their impacts can be substantial (Beanlands et al., 1986; Duinker & Greig, 2006; Therivel & Ross, 2007). CEA at the project level has resulted in shallow and reactive mitigation with no corresponding relationship to strategic or regional environmental decisions (Harriman & Noble, 2008). A change in scope in CEAs is necessary to ensure that the sums of effects of individual projects are appropriately captured.   2.4.1 The Current State of Cumulative Effects Assessment Studies of environmental decision-making and cumulative effects policy present themes that suggest that the inclusion of some aspects of cumulative measures have the potential to significantly shift the process of natural resource decision-making (Berube, 32  2012; Buschke & Vanschoenwinkel, 2013). In their 2006 study, Duinker and Greig outlined future directions for CEAs. The analysis of future scenarios must be securely grounded in the present, utilizing available data to determine the cumulative impact of development as well as those of all potential alternatives (Duinker & Greig, 2006). Carter and Atkinson (2011) also offer insight on the analysis of future scenarios within CEA, stating that qualitative descriptions can be utilized with emphasis on up-or-down shifts of predetermined variables. It is important to note that players within the political system are not neutral; they bring individual and institutional perspectives and value systems to a policy (Adger et al., 2003). The institutions that house these players also exhibit control over individual choices (Adger et al., 2003). The chosen action is largely a reflection of the society in which the assessment exists (Adger et al., 2003). The current institutional arrangements, as argued by Parkins (2006), are temporary with inadequate resources and therefore have limited capacity to consider cumulative planning strategies. CEA has been promoted as a tool for decision-makers, but there is a need evident in the literature to improve the understanding of the institutional framing and contextual influences of decisions that rely on cumulative effects information (Adger et al., 2003; Parkins, 2011; Therivel & Ross, 2007). The support within institutions and governments must be strong, if CEA adoption is to progress in a beneficial way to all practitioners, policy-makers and stakeholders.  In their work Sanchez and Morrison-Saunders (2011) emphasize that the accumulation and utilization of knowledge derived from the assessment is often managed ineffectively, significantly limiting CEA’s potential to change practices. Therefore, it is suggested that the institution governing CEA be grounded in a learning institution, 33  exhibiting qualities of adaptive management (Sanchez & Morrison-Saunders, 2011). In order to promote and utilize these and other qualities key to CEA, it is common that the governance structure reflect a centralized impact assessment agency (Sanchez & Morrison-Saunders, 2011; Morrison-Saunders et al., 2014).  Moving forward, centralized assessment institutions must continuously shift; evolving congruently with the natural resources and environment they manage (Tengo & Von Heland, 2012). Shifting towards a longer-term institutional vision will result in strategic reinforcement of CEA (Parkins, 2011), allowing consideration of technology, data gaps, data sharing and monitoring systems, and interdepartmental cooperation among environment assessment agencies, therefore producing a more comprehensive CEA. This may offer further insights into the process that translates environmental decisions into governance outcomes and policy development (Adger et al., 2003).  2.4.2 Cumulative Effects Assessment in Canada The Canadian experience with CEA began with a focus on stressor-based impact prediction, as well as attempting to predict all impacts associated with individual project appraisals (Noble, 2010). Beanlands and Duinker published one of the first landmark studies of cumulative effects, titled “An Ecological Framework for Environmental Assessment in Canada” (1984). They were the first to utilize and publish the concept of VECs (Beanlands & Duinker, 1984; Connelly, 2011). While CEA garnered some attention through these studies, cumulative effects were not considered widely in Canada until the 1990s (Connelly, 2011; Noble, 2010).  34  During this early period of employing cumulative effects, the Canadian Environmental Assessment and Research Council developed initiatives to provide guidance to practitioners in order to address concerns surfacing from CEA utilization (Harriman & Noble, 2008). In the later part of the 1990s, the Canadian Environmental Assessment Agency (CEAA) published a methodological guide in order to provide practitioners with assistance and a general framework to position their assessments (Berube, 2012). CEA was included in the recent update to the Canadian Environmental Assessment Act 2012, where they were defined as, “Environmental effects of the designated project…any cumulative environmental effects that are likely to result from the designated project in combination with other physical activities that have been or will be carried out” (CEAA, subsection 19.1, 2012).   Within the updated CEAA, it is now mandatory that all projects designated in the Act must include an assessment for cumulative effects (CEAA, 2012). Prior to the inclusion of CEA within the updated CEAA 2012, the Canadian Council of Ministers of the Environment (CCME) published methodological and conceptual frameworks as a response to the lack of best-practice guidelines felt by proponents and practitioners (Harriman & Noble, 2009). The CCME has organized the formation of a cumulative effects working group with an interest of promoting cumulative effects through assessment, monitoring and management within the environmental policy portfolio (CCME, 2009). The CCME is supporting the development of working definitions of CEA terms and principles to address a fundamental issue surrounding CEA utilization to assist those undertaking cumulative effects management (CCME, 2009). 35  While interest exists, CEA has yet to be broadly applied in practice (Noble, 2010). Through the prevalent themes in the literature, it is apparent that natural resource decision-making is a highly complex field and the application of policies and tools, such as the Framework, is an effort to promote improved knowledge of these intricacies.  This literature review makes it evident that there is a pressing need to advance the understanding of the institutional framings and relevant impacts of environmental policies and their associated processes in future research agendas.   36  Chapter 3: Methodology The Framework will facilitate the assessment and management of cumulative effects through policy, procedures and decision support tools to improve natural resource decision-making throughout British Columbia. CEF implementation will allow CE assessments to be conducted throughout the province’s natural resource districts. The appropriate CEA is to be considered when making the respective natural resource decisions. There is hope that furthering decision-makers’ understanding of the current environmental condition in their district will result in more durable and transparent decision-making (FLNRO, 2014).  The CEF does not intend to replace existing workflows, but rather to support and streamline processes (FLNRO, 2014). The CEF is currently in the initial stages of implementation, with the first phase of implementation concluding April 2016. The initial phase of implementation saw two CEA reports completed for the natural resource districts of Cariboo and a sub-region in Thompson-Okanagan, Merritt. The CEAs were each applied to two decision-making contexts in their respective regions. There is a need to address the effectiveness of the CEAs completed under the Framework’s mandate and understand how the information presented in the CEAs was considered in the decision-making process. Analysis of the implementation process will identify areas where modifications may be required within the policies and procedures in order to optimally support the next phase of the Framework’s implementation. The next phase of implementation will finish in 2021, when the CEF is expected to be fully functional throughout British Columbia.  37  3.1 Background & Objectives There is a lack of evaluation procedures for cumulative effects assessments and an absence of fundamental understanding of how these assessments are being considered in decision-making. Most of the material available at this point, including on the CEF in British Columbia, are descriptive or operational. There are cumulative effects assessments and overview documents available, but the policy and procedures are currently under development and not yet available. The process of evaluation by FLNRO, used in this research, will inform the development of the CEF. The analysis utilized in this research employed qualitative inquiry using methods that builds on the policy evaluation outlined by Bardach (2000), and the ‘evaluation of implementation approach’ put forward by Mickwitz (2006) and Hilden (2009). The emphasis in these approaches is on tracking the development of the policy, identifying the impacts and weaknesses and illustrating the emerging themes (Bardach, 2000; Hilden, 2009; Mickwitz, 2006). For both the case studies and examining the evaluation results provided by FLNRO, a narrative analysis process was employed (Adger et al., 2003; Hampton, 2009; Richards & Morse, 2013; Sandelowski, 1991).  Following a review of the literature, two data sources are used to achieve the research objectives: decision case studies and interviews. It is important to note that the methods chosen by various researchers are largely dependent on whether the ontological aim is to describe outcomes, explain, lead to predictions, or simply produce a policy description (Levy, 2007). The ontological aim of this study is to understand and describe the outcomes of the CEF, while also producing descriptions and insight for both the provincial government and CEA more generally. The subsequent sections of this chapter 38  first detail the methodological process for case study analysis and then in 3.3 we turn our attention to the interview methodology. This chapter concludes with a discussion of the rigor and reflexivity evident throughout this research.   3.2 Case Study Methodology Complementing more traditional analytical methods, case studies are now seen as a central apparatus in the development of theories (Levy, 2007; Yin, 2009) with the aim of increasing theoretical understanding of the particular issue (Thomas, 2005). In the Cariboo region, CEA was implemented for the entire region, differing slightly than the assessment in the Thompson-Okanagan, where the CEA was conducted for the Merritt sub-region. Together, the Cariboo region and the Merritt sub-region are the focus for initial implementation evaluation. Within these two areas, four decisions were identified that had used the CEA information, two decisions for each area respectively. The four decision contexts are meant to offer, as Richards & Morse stated (2013) a “powerful representation of the situation”.  Table 3.1 and 3.2 outline the decisions that will be utilized as case studies in this research for the Cariboo region and Merritt sub-region in addition to the documents reviewed for each. The decision case studies allowed comparisons to be drawn between the four, ranging from First Nations consultation, regulatory frameworks and the extent to which CEA information was considered in each context. The CEF implementation and associated CEA evaluations were carried out as trials for the larger policy vision and therefore differ in their assessment of effects and supporting the various decision contexts (FLNRO, 2014). These particular decisions were selected as case studies to provide a 39  holistic understanding of the situation surrounding the implementation of the CEA results to inform decision-making.   Table 3.1 Decision Case Studies in the Cariboo Region  Case Study Documents Analyzed Development of a Tactical Integrated Resource Plan  - CEA supported First Nations consultation and engagement, and development of tactical plan that enabled Cutting Permit approval   Licensee/BCTS Commitments for the South Chilcotin Planning Area      Cutting Permit Authorization in 100 Mile House District  CEA Summary for CP15H, 06H and 35H  Decision Rational for Consultation and Accommodation Tolko CP 06H Moffat Lake and Decision Rational for Consulation and Accomdation Interfor CP35H and RPR16977 Coffee Lake Decision Maker Rationale First Nations Consultation with Canim Lake Indian Band    40   Table 3.2 Decision Case Studies in the Merritt Sub-Region  Case Study Documents Analyzed Merritt Innovative Forest Practices Agreement (IFPA) – Allocation of Timber Volume - IFPA enables licensees to test new and innovative forestry practices intended to improve forest productivity   Merritt TSA Rationale for Increase in Allowable Annual Cut (AAC) Innovative Forestry Practices Agreement Establishing FLNRO Priorities for Field Based Resource Monitoring  No doucments were available for analytical purposes as this was an internal tactical decision-making process.   In qualitative research within political science, the number of case studies is typically not a concern, but rather the case selection based on substantive knowledge (Thomas, 2005). As Morse (2000) outlined, determining sample size is dependent on a number of variables, including the quality of data, the amount of useful information obtained and the use of secondary data. Resulting from the research partnership with FLNRO, the data that are provided for this research were highly detailed and the researcher was allowed access to decision-making documents that would otherwise be difficult to access. As the information was highly detailed, the documents contained valuable information on the CEF and the context to which CEA information was 41  considered in each decision. Morse (2013) also discussed the importance of secondary data, which are defined as reporting on others’ experiences. While Morse was discussing the significance of this concept in an interview setting, the principle can be applied to the decision case studies and associated document review. The documents for each decision-making context outlined the consultation process with proponents and First Nations communities.  Throughout the methodological process some case studies will become more decisive than others when drawing inferences (Levy, 2007). There is some apprehension regarding this approach to case study research, as qualitative researchers must also be cautious of selection bias where there is little difference between the outcomes of the dependent variable (Levy, 2007). However, the issue of selection bias does not concern this study, as all available decision-making contexts were analyzed as case studies. As a result of the narrow implementation of the Framework to date, the limited and purposeful sampling resulted in no case study selection bias from the research team or staff members in FNLRO.  In this study, the focus is on the audience groups identified by FLNRO as evaluation respondents. These are individuals who have a direct working relationship with the CEF. These include: decision-makers, decision support staff and regional CEA management teams (FLNRO, 2014). The information gathered from the case studies provides knowledge for the remaining implementation of the phased CEF. These four case studies were systematically collected and analyzed to develop a foundational understanding of the decision context that the CEA information was applied within.    42  3.2.1 Case Study Analysis Following Yin’s (2009) writing on dimensions of case study analysis, each decision was examined for explanation building and cross-case comparison. The four decision case studies are bound by the limitations outlined by the CEF and are therefore studied as a whole (Richards & Morse, 2013). The case studies were employed in order to understand the entirety of the situation involving the CEF and the decision contexts CEA information was applied to. As FLNRO provided the data from the CEF implementation, the data were detailed, allowing thorough examination within the pre-established boundaries.  Analysis of the documents associated with each decision, outlined in Table 3.1 and 3.2 respectively, allowed for differences to be highlighted, including First Nations consultation, regulatory framework and the extent to which CEA information was considered, while gaining an understanding of the different decisions made and the reasons for them. The decision relevant case studies emphasize the development of connections between CEA information and decision-making. The document analysis was carried out utilizing NVivo (QSR International Pty Ltd, 2014) to code for similarities and content (Yin, 2009). Following identification of themes and explanation building, the cases are comparatively analyzed (Yin, 2009).   3.3 Interview Methodology Semi-structured interviews were conducted by FLNRO as part of their review of the initial CEF implementation. The evaluation was to be used by the Ministry to aid in the ongoing implementation of the CEF program, allowing questions to be based on 43  previous knowledge while also permitting the opportunity for unprepared probes. The focus within the evaluation interviews was on understanding how respondents considered and applied the CEA information provided by the CEF. One of the more challenging components of an evaluation of the Framework is the limited implementation of the policy to date and associated procedures throughout the province. This resulted in a limited number of decision case studies available and therefore, a small number of possible participants who had used the CEF and associated CEAs. It was difficult to predict the exact groups that were exposed to the new policy and therefore limited the ability to predict the number of possible respondents. Response rates on elite questionnaire4 research are non-random and are difficult to predict in an ideal situation (Rockman, 2011). This was especially relevant to this research, as the policy implementation is ongoing. Since the sample population is already an isolated group, there was not an issue of over-extending the parameters of the study. FLNRO consulted with the UBC Centre for Environmental Assessment Research to help design the survey and identify approaches to evaluation suitable for examining the implementation projects, understanding implementation issues and developing an image of the use and impact of the CEF on decision-making. The interviews were conducted in-person by a FLNRO staff member or conference telephone call. The interview recordings were utilized in this research through secondary analysis. Secondary analysis is commonly used when there are variations in the research question or to impose a                                                  4 Elite questionnaire: survey research focused on those who hold leadership positions in institutions and organizations, distinguished by their regular participation in decision-making (Rockman, 2011).   44  different perspective on the data (Fielding & Fielding, 2008). Broadly, there are three main analytic purposes of secondary analysis: further in-depth analysis, additional sub-set analysis and new perspectives on the original data (Fielding & Fielding, 2008). There is some apprehension surrounding the reuse of data, namely that the intent of the original data collection will not align with the secondary analysis (Fielding & Fielding, 2008; Ritchie & Lewis, 2003). This issue was addressed in part through consultation with FLNRO by UBC researchers during the design of the original survey, which meant that although the survey was intended to meet an agency evaluation need, the researchers were aware of the questions and how the results could be used to address CEA related research questions. The interview data were provided by FLNRO to the researchers at UBC and was approved by the University of British Columbia Behavioural Ethics Board (certificate number: H16-00775). Respondents were sent the questionnaire by email from FLNRO prior to the interview. The questionnaire was written to not lead the respondents towards a particular answer. The questionnaire, titled Evaluation Template, can be found in Appendix A. The questions were based on the initial implementation of the Framework, the CEA information presented and VECs considered in the decision. It was important when drafting the questionnaire that the wording of the questions did not influence the participants’ answers, but were specific enough in order to gather information relevant to the research question and guide future action. The interview process allowed for questions to be based on previous knowledge of the CEF and developed in advance withholding the opportunity for unplanned questions resulting from respondents’ answers (Richards & Morse, 2013). The questions addressed a variety of issues and concerns 45  regarding the Framework including the participants’ relationship to the CEF and the decision-making processes, through both predetermined questions and unprepared probes.  Interviews were conducted for each decision in a group setting, with participants representative of each decision case study interviewed together. This allowed for discussion and various perspectives on the same CEA information that had been utilized in the decision-making contexts. The questions were asked by a FLNRO staff member to the group, allowing a discussion if necessary before an aggregate consensus was provided. Four interviews were undertaken, corresponding to the four decision case studies outlined in Table 3.1 and 3.2 respectively.  There is concern that by conducting the interviews in a group setting, there is a possibility that respondents were influenced by each other’s answers. However, this was the evaluation method chosen by FLNRO as it was deemed the most effective for time and resources. While group interviews have been cited as lacking the depth of their individual counterparts (Ritchie & Lewis, 2003), they have multiple benefits that the research team felt outweighed this consequence. Conducting the evaluation interviews in a group setting allowed for the process to be synergistic and evolving as the group worked together to reach a consensus (Stewart & Shamdasi, 1990). Furthermore, the group interactions were able to generate data that would not have been created in an individual interview (Morgan, 1997). The group interviews were recorded using a handheld recording device and then transferred into NVivo (QSR International Pty Ltd, 2014) for transcription and analysis.   46  3.3.1 Interview Analysis – Transcription Process Transcription is both an interpretative and a situated act and is a key component of the analytic process (Bird, 2005; Lapadat & Lindsay, 1999). The paradigmatic positioning of any researcher is likely to influence what is considered significant and the choices made during the transcription process (Bird, 2005). In this study, the data were transcribed as an act of presenting the original narrative in written form (Bird, 2005) to ensure a holistic understanding of the Framework and the political environment it creates. Therefore, during the transcription phase strong emphasis was placed on the everyday interactions between decision-makers and the Framework by applying reflexivity to the transcription process and approaching it from an outside perspective. The transcribed data were summarized and a report prepared for each of the interviewees. The summarized reports also added reflexivity to the study. Through the summarization process decisions were made about which responses were more critical than others. These choices were documented in my research journal. FLNRO approved each report to ensure that the transcription process adequately captured the context. The information gathered from the evaluation was coded in various ways, including descriptively and topic coding, lending itself to the narrative analysis process. This included coding for factual information such as the position held by the respondent, site at which they worked with the Framework and other factual information (Richards & Morse, 2013). The descriptive codes were incorporated into the data management process with each respondent having a section allocated to providing the information relevant to this area of inquiry.  47  Following descriptive coding, the data were examined to determine topics. Topic coding is defined by Richards & Morse (2013) as  “…creating a category or recognizing one from earlier, determining where it belongs among your growing ideas and reflecting on the data you are referring to and how they fit with the other data coded there. Categories were created in order to house all the data relevant to identified topics for organization, description, categorization or analytic purposes” (Richards & Morse, 2013).   This approach to coding is utilized to classify material on a particular topic for later description, categorization or reflection (Richards & Morse, 2013). Topic coding was key to the narrative analysis of the responses, allowing for a greater understanding of the influences guiding CEF-based decision-making. As stated by Sandelowski (1991), the primary objective of narrative exploration is to provide the researcher with an intelligible and inclusive understanding within a specific context. The narrative analysis brought perspective to human interactions with the CEF policy and decision-making concerns that result from the process. The themes identified ran throughout the data and provided insight into the strengths and weaknesses of the Framework.  3.4 Rigour & Reflexivity The rigour and degrees of reflexivity of the qualitative analyses carried out is supported in a number of ways, including purposeful sampling and the analytic techniques. Decisions made throughout the duration of the study, including major concepts, theories and possible sources of bias, were documented in reports of the project so that others may evaluate the research. These reports were shared with research partners at FLNRO and the thesis committee throughout the research process.  48  Reflexivity has previously been defined in simple terms as self-awareness (Robertson, 2000). Within the paradigm of action research, researchers are constantly reflecting on themselves, becoming more aware and the processes they are utilizing (Robertson, 2000). A criticism of action research in the past is the failure to illustrate the human influence in the selection, interpretation and analysis of the data (Hall, 2003). Through the partnership with FLRNO the researcher may be seen as actively involved in data creation, but the role was only observational.   The multiple sources of data utilized (case studies and the FLNRO evaluation) and multiple forms of analysis, including case studies and narrative analysis of the interview responses, allowed for a holistic understanding of the implications of the CEF. The results of the study were shared with participants following their interview. Interpretations within the final product may not mirror each individual’s experience as it is derived from a reduction of the data and is meant to be representative. The data collected during the decision case studies were used to direct further analysis, for example through the semi-structured interview questions. This research aimed to ground the findings conceptually. Describing relationships between themes present in the data highlight variations in application, strengths and weaknesses of the CEF and finally comparing my findings to other research to establish a link more broadly to other policies and practices.    49  Chapter 4: A Multiple Case Study Analysis of CEA Information in the Decision-Making As the CEF is a broad policy, implementation was designed in stages so that regional and sub-regional pilot areas may serve to refine the policy vision. The pilot areas were purposefully selected as case studies by FLNRO in consultation with Dr. Hanna and myself to evaluate recent decisions that have considered the initial CEAs directed by the CEF to gain insight into implementation progress and effectiveness. Insights derived from the decision case studies were used to inform the analysis of the evaluation interviews and the subsequent implementation phase of the Framework by supporting continuous improvement within natural resource decision-making.  Section 4.1 reviews the critical decisions within the government of British Columbia that led to the acknowledgement of the importance of cumulative effects in addition to the formation and underlying assumptions of the CEF. This chapter then examines the CEA conducted for the Cariboo region and Merritt sub-region in Section 4.2 and 4.3. Within Section 4.2 and 4.3 are each decision case study where the CEA information was considered, presented briefly, keeping with the multiple case study analysis outlined by Yin (2009).   4.1 Background & Objectives The BC Environmental Assessment Office (BC EAO) has considered cumulative effects as a component of large project EIAs. However, as the demand for natural resource development grows, the government began to recognize the increasing 50  complexity of environmental management (FLNRO, 2014). The province has introduced a number of measures to improve the consistency and efficiency of natural resource management and decision-making procedures:  The re-alignment of the majority of natural resource agencies to improve the consideration of land management authority (i.e. FLNRO); and  The Cumulative Effects Framework to integrate CE assessments and management into decision-making (FLNRO, 2014).  One of the formidable aspects of conducting a CEA is the multiple geographical scales that can be considered within the assessment’s parameters. The geographical scale chosen within British Columbia was meant to be reflective of the predetermined values through stakeholder consultation and the nature of industrial development in the area (FLNRO, 2014). Additionally, values chosen for the CEF were selected to be representative of the landscape and as broad as possible (FLNRO, 2014). In British Columbia this led to the suggestion of regional and sub-regional scale assessments as providing the most effective and financially efficient option (FLNRO, 2014). Existing natural resource districts or strategic land use planning areas were applied as they were considered the most effectual administrative units (FLNRO, 2014). The CEF is very much about using data, more effectively it can be argued, to support better decision-making, or perhaps better-informed decision-making. The broad-scale CEAs mandated by the Framework utilized existing data for the predetermined natural resource districts to examine the current condition of the values in relation to policy objectives (FLNRO, 2014). The assessments were able to identify emerging trends based on foreseeable future activities on the environment (FLNRO, 51  2014). The evaluation of current conditions provided decision support staff and decision-makers a measure of risk associated with each value (FLNRO, 2014). Moreover, an assessment of foreseeable future conditions was concerned with proposed development on the land (FLNRO, 2014). FLNRO has stated that the analysis of foreseeable future events is critical, as practitioners are able to identify emerging issues or risk and these may result in proactive mitigation measures to counter adverse environmental impacts (Berube, 2012; Canter & Atkinson, 2011; Cooper & Sheate, 2002; Therivel & Ross, 2007).   The two cumulative effects assessments that were examined as a component of this research were for the Cariboo region, Cariboo-Chilcotin Cumulative Effects Assessment and for the Merritt sub-region, Cumulative Effects Assessment for the Merritt TSA Operational Trial. It is the objective of the provincial government to gain insight from the initial assessments and regions to further inform the approach for the remaining period of the CEF’s phased implementation.   4.2 Cariboo-Chilcotin Cumulative Effects Assessment The Cariboo-Chilcotin Cumulative Effects Assessment was broadly meant to provide an initial assessment regarding impacts to First Nations rights and mitigation strategies. The area was selected because of its relevance to First Nations as the traditional territory of the Tsilhqot’in and its high biodiversity (Dawson et al., 2014). Figure 4.1 was created for use in the Cariboo CEA and details the twenty-six “LU” or Land Unit Groups found in the Cariboo-Chilcotin.  52  The selected values for the Cariboo-Chilotin assessment were biodiversity, hydrological stability and four wildlife species including moose, mule deer, marten and grizzly bear (Dawson et al., 2014). The wildlife species were chosen as those traditionally used by First Nation communities, while representing a wide breadth of habitat requirements (Dawson et al., 2014). The values for moose, martin and grizzly bear assessment were all conducted at the landscape scale and a smaller corresponding subunit to provide area specific evaluations (Dawson et al., 2014).  The landscape units in this area have been used since the 1990s for biodiversity assessments (Dawson et al., 2014). They are ecologically similar areas ranging from approximately 30 000 to 70 000 hectares in area (Dawson et al., 2014). The justification for utilizing these areas was to allow the assessment to capture projects of a majority of sizes while simultaneously addressing local impacts specific to First Nation communities. Furthermore, size of landscape units also allowed for the natural range of wildlife species (Dawson et al., 2014).  Other VECs applied different assessment units, with an emphasis placed on the results being meaningful to the definition or management of the specific value. Forest biodiversity assessments were conducted at the biogeoclimatic subzone level within each landscape unit, while mule deer assessments were conducted utilizing the boundaries for their legally designated winter ranges (Dawson et al., 2014).  The report underlines the benefits to First Nation communities and other users through more consistent and comprehensive information available from the government (Dawson et al., 2014). It is hoped that increased access to information will result in greater efficiency by providing a common base to all stakeholders in development 53  proposals while allowing the most relevant issues to be stressed (Government of British Columbia, 2016).  54   Figure 4.1 Landscape Units in the Cariboo-Chilotin Region (Dawson, et al., 2014)55   The CEA for the Cariboo region was applied to unique decision-making contexts throughout the region. The first case study is for the 100 Mile House District where CEA information was applied to gain a better understanding of the environmental condition while issuing cutting permits. The second case study purposefully selected in this region details the use of CEA information in the development of a tactical integrated resource plan and the subsequent consultation and engagement with First Nations.  4.2.1 100 Mile House District The 100 Mile House District used the Cariboo-Chilcotin CEA as a way to assess the cumulative effects associated with cutting permits issued under the Forest Act. These applications were made to facilitate the salvage of timber killed by the mountain pine beetle in 100 Mile House District. Cumulative effects tools were utilized to determine the potentially adverse impacts to watershed condition, moose and marten values. Habitat disturbance to moose by road access was deemed as the largest risk in the 100 Mile House District.  The 100 Mile House District was the first to use any CEA information, and as such were forced to rely on some incomplete value assessments. It was found that within the Cariboo-Chilcotin CEA the watershed values did not cover the entire area relevant to this decision. To deal with this challenge, supplementary watershed assessment were considered including Beaudry’s (2008) hydrologic assessments of Murphy Lake and Coffee Lake Watersheds, Polar Geoscience Limited evaluation of Murphy and Coffee Lake in 2010 and the Watersmith Research Incorporated analysis of 2014. Beaudry recommended a low risk rating for the watershed, differing from the Canim Lake Indian 56  Band who considered the hazard rating within Beaudry’s report too modest. While there were no specific recommendations provided, the report did suggest winter harvesting and protection of live over-story and understory for salvage harvesting. Watersmith Research Incorporated (2014) did make recommendations, the majority of which required action by the proponent.  Two First Nations, the Williams Lake Indian Band and the Canim Lake Indian Band have asserted rights and title over various portions of the area. Consultation was initiated in March and July 2013, with Canim Lake Indian Band appointed as representative of themselves and Williams Lake Indian Band. Following engagement with proponents and First Nation consultation, enhanced measures were identified and agreed to mitigate potential hydrologic concerns. After the decision-maker considered all the information provided by the CEA and other reports, all cutting permits were issued. Despite arguably the same decision being reached with the CEF, the CEA information provided guidance on values and environmental conditions that would not have otherwise been available to the decision-maker.   4.2.2 Tactical Integrated Resource Plan Development in South Chilcotin  The CEA information in this region was used to support First Nations consultation and the development of a tactical plan that enabled the approval of cutting permits. A large portion of South Chilcotin has been relatively undeveloped by economic activity. This is changing, as the region has been identified as a priority area for timber harvesting since the designated areas were damaged by mountain pine beetle epidemic. 57  There is a plan to harvest the area extensively for the next ten years until salvage opportunities are depleted.  All the CEA values were used to inform the decision-making process and to conceptualize a plan for mitigation measures with proponents and First Nations. Moose populations were identified as decreasing in the Chilcotin Region where extensive salvage harvesting was occurring. First Nations have recognized the significance of moose as a food source, an indicator of environmental health and a vital component of aboriginal rights and cultures. The proponents acknowledged the relationship between increased harvesting, concurrent road access and the effects of mountain pine beetle on moose population and agreed to establish a goal of no net loss to moose habitat as a result of their proposal.   The proponents worked collaboratively in order to determine a coordinated approach to harvesting and environmental management, forecasting into the foreseeable future. The Tactical Integrated Resource Plan was developed with the understanding of a relationship between habitat change as a result of timber harvesting, mountain pine beetle morality and moose populations. The guidance provided in the tactical plan is largely a mitigation strategy in an effort to minimize the effects on moose resulting from continuing timber harvest and road development. The collective mitigation measures and management practices allowed the proponents to proceed with salvage harvesting under the condition of no net loss to moose habitat. The appropriate First Nations were consulted with the coordinated practices proposed by the proponents. The statutory decision-maker was thus able to approve the cutting permits based on the landscape level plan.  58  4.3 Cumulative Effects Assessment for the Merritt Operational Trial The operational trial areas also extended to the Thompson-Okanagan region, with specific focus for CEA on the Merritt sub-region. Figure 4.2 corresponds to the area considered in this particular CEA report. The purpose of the CEA for the Merritt Timber Supply Areas (TSA) was to assess risk to resource values and consider the potential implications for decision-making, actions, mitigation and policy evaluation. Extending beyond the CEA report, the information gathered was meant to inform identification of adverse impacts resulting from new natural resource development proposals, First Nation consultation and tactical planning. The CEA examined and analyzed six values:  Fish stream habitat  Moose population  Mule deer population  Grizzly bear population  Visual quality  Old growth management areas (OGMA).            59   Figure 4.2 Area Considered within the Merritt CEA (Valdal & Lewis, 2015).  Any factor that had the potential to affect other values was analyzed employing a risk assessment evaluation, such as watershed condition. Results from risk assessments on mule deer and moose reflected a stable population with some risk identified around limited winter habitat and road density leading to a greater exposure to hunting (Valdal & Lewis, 2015).  60  The fish habitat assessment that was reported in the CEA was fairly limited, only considering the range of salmon and trout at risk focusing on one watershed within the sub-region. Some evaluations were provided the benefit of a larger scale, with hazards being assessed in relation to riparian clearing, sediment input and peak flow conditions throughout the sub-region. The results indicate that as a consequence of cumulative effects through natural resource development and natural disturbance, hazards have increased steadily throughout the past decade (Valdal & Lewis, 2015).  At certain points, the report indicated corresponding legal or policy objectives to specific values. These were assessed as a measure of expectations being met (Valdal & Lewis, 2015). This allowed the CEA to determine which areas were meeting pre-established aims and those that were not. The conclusion from the Merritt CEA illustrates the need for policy guidance throughout the natural resource sectors in an effort to ensure that expectations for values are maintained (Valdal & Lewis, 2015). There were several management recommendations from this CEA case study that can be grouped into three responses: 1. The need for more information including monitoring, more thorough assessment procedures and research in cumulative effects assessment 2. Improved resource management practices  3. Policy evaluation, incorporating evaluation, improvement and development.  The case studies were purposefully selected for each CEA report to illustrate the application of cumulative effects information in a variety of decision-making contexts. The Merritt sub-regional CEA was relied upon for an Innovative Forest Practices Agreement. Secondly, the Regional Monitoring Team applied the information in order to 61  better determine regional monitoring priorities in collaboration with the values that the Framework has considered important.   4.3.1 Merritt Innovative Forest Practices Agreement The Innovative Forest Practices Agreement (IFPA) is one initiative outlined in the Jobs and Timber Accord of British Columbia’s Forest Act (Forest Act, Part 4, Section 59.1). IFPA’s enabled licensees to propose and implement innovative forest practices that can support increased forest productivity. The objectives of IFPA are as follows:  To test new and innovative forestry practices intended to improve forest productivity   Encourage designated Licensees to carry out the forest practices by offering them the opportunity to apply for an increase to their allocated harvest levels to enhance and maintain employment in the forest industry (FLNRO, 2014).  The decision-maker is able to allocate an annual allowable cut (AAC) if they believe there is evidence of a potential increase in timber supply. The Merritt Innovative Forest Practices Agreement was the first of its kind to consider cumulative effects information. Of particular interest is Stuwix, one of the licensees eligible for AAC increase, as it is a First Nations forest company and is owned and operated by eight First Nations Bands in the Nlaka’namux and Syilx territories.  There was extensive First Nation consultation undertaken for the IFPA in this sub-region. However CEA information was not shared as a component of consultation as it was still a draft at the time. There was also engagement with First Nations in 2011 62  around cumulative effects concepts and topics, but it did not expand to outcomes or products associated with the Framework.  FLNRO research partners viewed the decision rationale, but as the information was confidential it could not be detailed for the purposes of this analysis. The decision stated that the consideration of cumulative effects was notable in the identification of the need to regularly revisit the decision and as such, setting a term for the decision. Future analysis was recommended to accommodate additional information about the state and expected state of non-timber values.   4.3.2 Merritt Monitoring Program In this particular case study, CEA information was employed to assist in the process of establishing FLNRO priorities for field based resource monitoring in addition to determining annual priorities for the monitoring program. This regional monitoring program was unique to the Thompson-Okanagan region, where in order to appropriately and effectively allocate resources, monitoring and field based observations were required. The monitoring that occurred paralleled the cumulative effects work that was being done at the time with respect to measuring values.  As this specific use of cumulative effects information is distinctive, the process that followed is methodologically very different from the other decision-making contexts. This particular case is best described as an internal tactical decision as to how best to focus field based monitoring resources. The priorities are defined in part through CEA information, to focus monitoring on the areas assessed at moderate to high risk. For this initiative, First Nations consultation was not required, but the monitoring program did 63  partner with communities for fieldwork purposes, relying heavily on output maps and reports resulting from the CEA information.  The CEA allowed the monitoring program to focus their study area and define monitoring targets through the CEF’s corresponding work regarding value assessment. There was specific attention given to the watershed value and the relationship to fish habitat, as it had been identified as possessing the highest environmental risk for the area. As the region is quite large, the monitoring program is already searching for ways to prioritize the limited resources available to them, making the information provided through the Merritt CEA critical.   4.4 Summary The early phases of policy implementation come at a complex time for the province and for those who work to apply the CEF. This research undertook multiple case study analysis of the Cumulative Effects Framework in British Columbia in an effort to understand the decision-making context that CEA information was being applied to. The four decision case studies were among the first in British Columbia to test the government’s new commitment to considering cumulative effects assessment in decision-making.  The most striking, and frankly unanticipated, finding of the case studies was the clear variation in approach, context and regulatory frameworks in the decision-making process.  There is a wide array of decisions that government makes to allocate and manage lands and resources (development proposal, conservation efforts, natural resource extraction, etc.), with different regulatory frameworks supporting decision-64  making. The CEF is intended to provide consistent information to support this wide array of decision types. The decision types represented different types of decisions that considered CEA information.   The differences between the case studies does not mean that this research was without any insights into the Framework, instead the differences served to emphasize commonalities and strengths of the CEF. For example the 100 Mile House District utilized the cumulative effects information to inform a decision concerning an AAC increase under the Forest Act in comparison to the Merritt CEA which was used to develop priorities for field based resource monitoring. These vast variations in the case studies emphasized the Framework’s applicability to multiple decision-making situations. The CEF is a significant undertaking for the British Columbia government. Therefore, the resulting CEAs and their application to a wide variety of decisions and natural resource management functions is an example of the wider utility of the CEF.  The decision case studies for each CEA also allowed the context for each decision to be analyzed, developing an understanding that serves to anchor the findings from the evaluation interviews. By comparing and contrasting the differences in the case studies application of the CEA information, the research team was able to utilize this knowledge to anticipate probes appropriate for the respondents of each evaluation interview.   65  Chapter 5: The Practice of Decision-Making Utilizing Cumulative Effects Currently, many environmental decisions do not grasp that the issues they are meant to address are characterized by a highly complex and interrelated web of science and value relations (Stahl, 2014). These decisions attempt to achieve environmental goals, while also allowing economic and societal needs to flourish (Stahl, 2014). It is important to understand not only the regulatory and policy foundations (as was achieved in the case studies), but also to engage with those directly affected by policy implementation. This research employed semi-structured interviews to understand the human interactions and relationship to the CEF and associated CEAs.  The evaluation interview questions emphasized particular areas of the initial implementation of the Framework to understand how the information was applied in the decision-making process. The Ministry was provided the opportunity to edit the transcript and provide additional feedback if they chose. In the following sections, the research questions are addressed – program objectives, implementation insights and lastly, lessons learned for cumulative effects practice and implementation generally.   5.1 Program Objectives One of the driving research questions for this thesis was to determine the extent to which CEF program objectives were met. FLNRO promoted the CEF along set benefit streams including: improving the ability to manage for the desired outcomes for values; supporting the assessment of potential impacts to First Nations rights and interests; 66  supporting streamlined decision-making; and supporting transparent and durable decisions (FLNRO, 2014). Overall, feedback on cumulative effects ability to support the achievement of approved government objectives for the condition of values was favourable. The values foundation was identified as a key component of the initial implementation of the Framework, detailed previously in Chapter 1. The values were chosen in order to bring consistency across the province, but remained flexible where regional differences warrant variation (FLNRO, 2014). During the interview process the respondents were asked to collectively determine the utility of the values they considered given the specific decision-context where CEA information was applied, including provincial and regional values. Not every value was considered in the case studies outlined in the previous chapter, as a result some of the participants could only speak to certain values and others could not provide any rankings (note the difference between the rankings received from the 100 Mile House evaluation interview to either of the Merritt interviews). A scale was used in the evaluation interviews, where 1 = not relevant and 5 = extremely relevant. Figure 5.1 shows the aggregate rankings agreed upon by respondents.   67    Scale of 1 to 5 (where 1 = not at all; 2 = slightly; 3 = somewhat; 4 = quite; 5 = very). Figure 5.1 Utility of Values Presented in the CEA Information  For the majority of respondents, the values included in the CEA information were considered beneficial to decision-makers. The natural resource districts are extremely large in British Columbia; cumulative effects information was praised for providing insight into areas that would not have been possible without the Framework due to resourcing and staff limitations. This also speaks to the values that were chosen by the government as being reflective of current environmental conditions and providing additional knowledge to decision-makers.  Watershed and moose values were considered in almost every decision due to their importance to environmental sustainability and the significance of moose to First 100 Mile South-Chilcotin Merritt IFPAMerrittMonitoringWatershed 2 5 0 5Moose 4 5 0 0Marten 5 0 0 00123456Utility of Values 68  Nations’ traditional and cultural use. The watershed value was inconsistent in some regions, likely due to the preliminary implementation of the Framework resulting in a lack of data. Respondents reported that they were forced to consult other documents, often at differing geographical scales than the CEA information, creating challenges for understanding the current environmental trend. It was stated in one interview that due to the incomplete value assessments, more government resourcing was necessary to understand the current condition and trend of watersheds.  The participants were also asked to collectively evaluate how clear and understandable the information was. Despite some of the issues that arose surrounding incomplete assessments, the feedback on the clarity of the information was largely positive. The information was seen as being “clear and usable”, as complex regional ideas were translated into simple yet comprehensive reports. As one respondent stated, “You can reach out and understand each of the ratings and conditions the landscape units is in. It's very tangible”. The one low ranking received can be linked back to the inadequate watershed assessment.  Presenting information in a clear and concise manner supports one of the Framework’s initiatives around streamlining the decision-making process. Respondents in each interview were asked to rank not only if the information was clear and understandable but also the relevancy of CEA information provided. The intent of the Framework is to present the cumulative effects information in a clear, concise manner and provide relevant information to the decision at hand. Table 5.1 details the rankings provided in each interview surrounding questions related to the act of decision-making. It is worth noting that the Merritt Monitoring interview is not present in Table 5.2. This is a 69  result of the CEA information being applied to a unique context in that specific case. Therefore, the interviewees could not speak to the value of the information in a decision-making context. Each question – was it relevant to the decision, whether it supported the decision and if the information was effective in the CEA’s efforts to support the process – was given a value out of 5 by the interviewees, where 1 = not relevant and 5 = extremely relevant. The interviewees were asked to jointly agree on the value given.   Table 5.1 Interview Rankings for CEA’s Relationship to the Decision-Making Process    Interview Question Evaluation Interview Relevant to the Decision Supported the Decision Process Effective Information for Supporting the Decision Process 100 Mile 5 3.5 3 South-Chilcotin  5  4  3 Merritt IFPA 4 4 3  Scale of 1 to 5 (where 1 = not at all; 2 = slightly; 3 = somewhat; 4 = quite; 5 = very).  Despite some areas with incomplete assessments, the CEA information that was provided was found to be of great relevance to the decisions examined. The information 70  was described as being informative to not only decision-makers but also proponents, encouraging engagement with First Nations and stakeholders. One participant noted, “It's information that sparks an informed conversation of how you're doing something.” The criticism that emerged was that more robust data would only serve to increase the information’s relevancy. In some cases the information was either incomplete or not completed at an appropriate scale, resulting in the need to consult other documentation. If a decision-maker is forced to consider information that is not applicable to the respective decision, it will slow the authorization process and increase governmental staff resourcing. Interviewees noted an improvement of the CEF over previous decision-making support tools. Descriptors such as “comprehensive”, “regional wide”, and the ability to highlight important areas for a value assessment were evident in the transcriptions. The lower values were largely connected to the maintenance of the tool, both in monitoring the environmental trends and staff training.  Some FLNRO staff noted that in communication with First Nations communities some felt that the Framework did not adequately represent Aboriginal interests to decision-makers. One respondent interpreted First Nations’ concerns surrounding the effectiveness of values within the Framework, explaining that the choices made using cumulative effects information did not go far enough. For example, in the 100 Mile House District, one interview participant stated that one First Nations band believed the area should not be harvested any further, disagreeing with the low risk rating determined by the CEA and Beaudry’s (2008) hydrologic assessment. Other bands in the same nation did not share the concern. The respondent expanded on the issues, stating that the option to leave the areas unharvested was not possible, resulting in the permits ultimately being 71  issued with recommendations for the proponents. There is reasonable doubt as to whether this and other similar disagreements can be considered as a failing of the Framework, associated policy, tools and procedures. There will continue to be differences of opinions in natural resource decision-making regardless of the tools. It was stated by another participant that although the information and the consultation process is not perfect, it is the best thus far.  Respondents reported further difficulties in attempting to engage First Nations to identify specific values that were of additional concern to them. A key component of any natural resource decision is First Nation consultation and engagement. First Nations were consulted in the three operational decision (100 Mile, South Chilcotin and Merritt IFPA) and were not consulted in the priorities for monitoring (Merritt Monitoring) as it was an internal government tactical decision on how to deploy resources and does not have the potential to infringe First Nations rights. First Nations were however invited to engage in collaborative monitoring and are continuing to do so.  When appropriate, First Nations communities were involved in the authorization process and decision-making contexts analyzed in this research. Likely resulting from a history of suspicion, there may be lingering feelings of mistrust between First Nations, government and natural resource developers. One respondent characterized it as an impact relationship:  “If First Nations feel that there is a huge infringement and they have to wait to deal with that once we engage in the consultation process, you've lost that opportunity for that relationship building between proponents and First Nations. Along with that would also be the benefit of First Nations having this opportunity to better understand and to learn to use this tool so that everybody is kind of speaking the same language when we're looking at these reports.” 72   Participants of the interview stated that additional data with more indicators such as population would be beneficial for First Nations’ interests and engagement moving forward with subsequent phases of implementation. It was echoed in responses that the relationship between First Nations, proponents and government would likely advance from increasing the involvement of First Nations in the processes and tools associated with the Framework. A similar level of engagement has already been utilized in the Merritt monitoring program, thus far successful and resulting in positive relationship building. Building on the knowledge base developed through the case study document review and from narrative analysis imposed on the evaluation interview transcripts, this research gained valuable insight into the human interactions with the CEF, resulting CEAs and their application to decisions.  Two areas of implementation and the CEF program objectives more broadly, which may warrant additional consideration, are establishing organizational roles and responsibilities and First Nations and stakeholder engagement for the evaluation of existing trials. There was concern that there is risk of the CEF being viewed as a FLNRO initiative. In the subsequent phases of implementation, the provincial government may benefit from addressing these issue and developing a greater awareness for other ministries’ organizational roles and responsibilities associated with the NRPP and the CEF.  If the Framework is to meet long-term objectives, extending beyond those outlined in this research question, there must be a commitment echoed not only by decision-makers using the information, but also in government’s commitment to the tool.  73  Furthermore, this research did not focus on First Nation or stakeholder engagement. The context for the decisions was gained through case study document review and evaluation interviews. While First Nations consultation was addressed in the evaluation interviews, benefits to the CEF and the provincial government would likely be received from greater scrutiny regarding this specific objective. This research focused primarily on the first program objective, for supporting the achievement of approved government objectives. Government objectives for values refer to the legal and policy objectives that are currently established in land use plans and legislation for the condition of each value.   5.2 Implementation Insights When the CEF is fully implemented, it will require periodic or ongoing assessments of cumulative effects at a broad, strategic scale (FLNRO, 2014). Phased implementation was meant to effectively reform resourcing and develop the organizational capacity, policy and analytical tools necessary (FLNRO, 2014). The initial implementation was the focus of this research, emphasizing cumulative effects information considered in the decision-making process.  Implementation of such a complex policy is not without challenges. Policy implementation has been perceived as having remarkable components and as requiring an assessment to any subsequent changes to determine whether the policy objective has been achieved (Bailey et al., 2011; Mickwitz & Birnbaum, 2009; Wholey, 2004). Insights can be gained from Phase 1 implementation critical for the development and continuous improvement of the Framework and embedded standards for cumulative 74  effects assessment. Respondents were asked to address any shortcomings they felt warranted greater scrutiny in the subsequent phases of implementation. Most participants reiterated their issues with a lack of data, asking that the tool become more robust. Also echoed was the need for monitoring and general maintenance of the tool. For the CEF to remain an effective decision-support tool, effort is required to ensure the values continue to represent the current situation of the environment.  As is the premise of cumulative effects assessment, impacts have the ability to combine into a sum greater than their individual components. Often it is the unintended or unpredictable impacts that can cause the greatest challenge to environmental management. This realization did not escape interview respondents when noting their experience working with cumulative effects information. One participant described their concerns as follow:  “I just didn't have a lot of confidence on how quick the model was being updated. Cumulative effects for watershed values have to be updated quite regularly…If it's not updated regularly, it's not useful at all…In two years, it’s out of date.”    Additionally, there was a concern raised about individuals choosing to use the information. The following is an excerpt from one of the interview participants discussing this topic, “It's only as effective as somebody deciding to do something about the information…it could help us manage our objectives if we chose to let it”.  These statements stress the importance of the provincial government’s commitment to the CEF, in monitoring and staff training. The government has stated that the CEF will rely on provincial monitoring programs already in place. Consideration may need to be given for extending these programs if the Framework is to remain an effective 75  measure of environmental trends and conditions. General training may also be required for the Framework to become increasing effective so that all the natural resource ministries are educated and able to utilize the CEF and the CEAs.  The successive phases of implementation are tasked with determining the best practices to present the CEA information to decision-makers. Individuals responsible for natural resource allocations are often presented with an excessive amount of information, leaving them at times unable to adequately address those that may have the highest priority to environmental sustainability, societal values or First Nation communities. Consistent standards are needed for decision-makers presented with the cumulative effects information. As one participant stated, “As clear and concise as possible is probably the best for a lot of decision makers who are trying to access information in the most efficient way.”   5.3 Lessons for CEA Practice There is considerable discussion of the importance for conceptual and procedural frameworks guiding the collection and interpretation of cumulative effects assessment (Bardach, 2000; Heggman & Yarraton, 2011; Mickwitz & Birnbaum, 2009). Despite the recognized importance of cumulative effects for environmental management and sustainability there has been little improvement over the past thirty years in Canada (Connelly, 2011). The Cumulative Effects Framework is a key component of the realignment of agency responsibilities and the creation for a more integrated resource management agency, promoting the consideration of cumulative effects information in natural resource decisions throughout the province.  76  While the recent experience in British Columbia showcased that a shift in how governments view environmental management is possible, CEA is not only an assessment, rather, it must be a commitment felt throughout the sector, agencies and follow-up practices. Duinker & Greig (2006) called for the operationalization of regional CEAs. British Columbia is the first province to attempt this and can be viewed as an example of Tengo & Von Heland’s (2012) transformation to a new environmental management paradigm. The provincial government has successfully shifted the focus of CEA from project assessment to a regional assessment context (Duinker & Greig, 2006).  Implementing CEA legislation is a positive change in the management and assessment of environmental impacts. However, evident in the literature, case studies and evaluation interviews, implementation alone is not enough. Monitoring emerged as requiring the greatest attention throughout the evaluation interview process conducted as part of this research. Monitoring is a key component of CEA providing information on current environmental trends and identifying possible mitigation strategies (Canter & Ross, 2010). CEA requires not only a commitment felt at during the assessment procedures, but as Parkins (2011) stated, in long-term investment. As Duinker & Greig (2006) specified, even the most vigorous CEA will be useless if sufficient monitoring and mitigation do not follow it.   77  Chapter 6: Discussion & Conclusions The pressures on the natural environment today are unprecedented, with multiple demands for the same land base. It has been recommended in the literature that there is a need to advance our knowledge of the institutional framing and contextual influences of environmental decision-making processes (Heggman & Yarraton, 2011). Effects caused from one development have the ability to accumulate with impacts from a separate project, resulting in unintended effects to economic, environmental and societal values (FLRNO, 2014). This stresses the need for cumulative effect assessment, simply defined as “the practice of systematically analyzing cumulative environmental change” (Harriman & Noble, 2011, p. 154).  The BC government recently committed to a shift in EA perspective, dedicating significant resources to the development of the Cumulative Effects Framework (FLNRO, 2014). The initial phase of implementation is nearing completion with pilot regions applying the cumulative effects information in decision-making contexts. In recognition of the gap in the literature coupled with British Columbia’s new Framework, the aim of this research was to first understand how cumulative effects assessments affected the decisions in the pilot regions, followed then by application of the knowledge derived from analysis in order to inform change for the remaining CEF implementation phases.  To address this aim, three research questions were asked:  1. Have the CEF program objectives been met? Or to what extent? 2. How can the BC experience of the CEF implementation inform the development and improvement of policy, procedures and standards for cumulative effects assessment? 78  3. What lessons does the BC CEF experience hold for EA and CEA practice and implementation broadly throughout Canada?  Two CEA reports, resulting from the CEF, were considered in four decisions that were then analyzed as case studies to provide context for each decision through document review examining the cumulative effects information considered and the resulting decision. Interview participants were asked to provide descriptions of how the CEA information was used, how effective the CEA information was and finally how or if it influenced the decision outcome.  This research shows that the Framework has or will complete the majority of the intended goals. Thus far, there is positive feedback from those who have directly worked within its parameters. The CEF is meant to provide benefits in: improving the ability to manage for the desired outcomes for values; supporting the assessment of potential impacts to First Nations rights and interests; supporting streamlined decision-making; and supporting transparent and durable decisions (FLNRO, 2014). This research focused on evaluating the effectiveness of trial CEAs and the ability of the information to improve the decision-making process. There may be further investigation warranted with First Nations communities to determine their perspective on the Framework. Additionally, there were suggestions of expanding certain values to incorporate First Nation interests, namely population data for species.  Engagement with First Nations for the evaluation of existing trials and the implementation of the Framework is an area that may warrant reflection by the provincial government. This research did not focus on First Nations communities, and instead discussed the consultation and engagement that occurred in each decision with the 79  interview participants. British Columbia may benefit from an in-depth analysis of this objective to assess if the Framework is benefiting government’s relationship with First Nation communities.  Furthermore, this research sought to determine implementation insights gained throughout the pilot regions for the improvement of policy, procedures and standards. The phased implementation was intended to slowly reform and develop the organizational capacity, policy and analytical tools required to ensure the Framework’s overall success. Emphasis from the pilot areas centered on a lack of data slowing streamlined decision-making and a need to continue monitoring efforts. The environment is a dynamic system, constantly shifting and evolving. The watershed assessment was brought up numerous times throughout the evaluation interview process as a critical value for decision-makers, but also illustrates the necessity of regular monitoring and data renewal. If the values are not monitored and the data does not remain constant, the Framework will be unable to present the current trends and conditions of the values.  Conveniently for the provincial government, these implementation insights are two sides of the same coin. With continued support for monitoring, the data utilized in the Framework will become increasingly enhanced over the long-term. Disregarding the volume of data, the information must still be presented to decision-makers as clearly and concisely as possible. The value of the Framework is its ability to present complex information in eloquent yet simple terms. The provincial government should consider crafting guidelines that decision support staff may use to effectively organize information housed within cumulative effects assessment for decision-makers. 80  One of the original considerations of this research was the possibility of identifying influence criteria for decision-making purposes. The development of such criteria stemmed from a pragmatic perspective of utilizing information to enact meaningful change on future actions (Goldkuhl, 2012). The development of influence criteria was meant to serve in a prescriptive knowledge role, by providing guidance on environmental influences affecting decisions (Goldkuhl, 2012).   Throughout the analysis of the case studies and interviews, it became evident that the development of influence criteria was not a possible conclusion from this research. The decisions relied on differing regulatory frameworks, making the creation of influence criteria an impossible endeavor in this early phase of implementation. Furthermore, as the Framework is in the initial stages of implementation the research conducted was limited in sample size to only two CEA and four interviews. Each application of CEA information to the decision-making process was highly individualized to suit the particular needs of the decision in question.   Future influence criteria may be beneficial in illustrating the current values that prove beneficial to decision-making and those values which are not present but whose absence accounts for lack of effectiveness. Finally, possibilities for the cumulative effects consideration in decision-making could be a guiding principle of the criteria created, providing prospective knowledge to those who adhere to the influence criteria (Goldkuhl, 2012). This may be a consideration for further research by the provincial government as a component of later policy evaluations for the Framework. Validation of the findings will be demonstrated in the degree to which the research resonates with natural resource decision-makers. This has the potential to lead to 81  more durable and transparent decision-making, while simultaneously providing greater clarity for industries (FLNRO, 2014). The pragmatic value of this research will be shared through presentations at academic conferences, offering to present at government organizations addressing environmental decisions and management concerns and lastly in writing through submission of at least one paper for publication.   6.1 Limitations A limitation facing this research is quantity. While the relationship between the researchers and FLNRO provided excellent quality of information and access to decision support staff and decision-makers that otherwise would not have been possible, the number of examples was limited due to early stages of implementation leading to a small number of decision-cases thus far and the small number of evaluation participants. Two CEA reports resulting from the Framework have been applied in decision-making contexts. The number of interviews that were conducted and participants’ ability to recall the specifics of individual values within the Framework was a further limit on the research.  While the reliance on FLNRO evaluations provided access that would otherwise not have been possible, it also meant that certain areas of the evaluations could not be completed due to confidentiality (such as in the Merritt IFPA case study). Furthermore, while FLNRO consulted with the researchers on the research design, a secondary party ultimately collected the data that was provided for this thesis. Secondary data is often regarded as a poor substitute to primary data, as the researcher does have a more limited relationship with the data creation (Throne, 1994). However, Throne (1994) also states 82  the secondary data is an interesting option for those who wish to apply a different lens on the database. For my research, I was able to be privy to the data collection process, allowing for some relationship with the data creation process. Furthermore, I approached the data as an outsider from the government agency that collected the information and was therefore able to apply a different perspective to the findings than would have occurred if analysis had been carried out by FLNRO.  Despite the small sample, the research adds value and substance to the on-going evaluation of an innovative policy for managing cumulative effects. The small sample size does not undermine the accuracy of the findings; it does reduce the insights gained. This research may not serve to inform environmental policy evaluations in multiple jurisdictions; however valuable insights and feedback were gained informing the provincial government on challenges met in the early phases of policy implementation.   6.2 Recommendations for Policy, Procedure & Decision-Making Environmental assessments have been a cornerstone of natural resource decision-making for the past three decades. Management recommendations are the main purpose of EA, in order to make better-informed natural resource decisions. Therefore, it is not inconvincible that a country such as Canada and member provincial governments rich with natural resources would be world leaders in their assessment practices. However, EA and CEA specifically have failed to accomplish what was hoped in their infancy (Noble, 2010).  One of the foremost problems in CEAs of the past, were their application to individual project level assessments (Harriman & Noble, 2009). British Columbia was no 83  exception to this experience, as large-scale project level assessments may be required to assess for cumulative impacts (EAA, 2002). British Columbia is attempting to challenge the status quo, adopting a fresh perspective on cumulative impacts into natural resource decision-making. The adoption of a regional perspective on environmental management is a positive change for how natural resources are viewed in the province. The initial phase of implementation is nearing completion in the province. Thus far the policy has been well received by decision-makers, citing that it has provided beneficial information that would not have been available prior to the CEF. As a final conclusion, this research offers three broad future directions for the subsequent phase of the CEF in hopes of ensuring its continued success and additional areas of inquiry.  1. Increase the monitoring and follow-up resources available.  2. Ensure all government ministries and decision-makers are aware and informed of the paradigmatic shift towards cumulative effects in environmental management in British Columbia.   3. Honour the formally embedded period policy evaluations within the CEF to allow for issue identification, areas for improvement and to determine if objectives are continuously being met.  An aspect of CEA and EA in general that has been an on-going challenge is monitoring and follow-up. For CEA, the key is to manage properly the impacts predicted in light of the proposals and associated impacts. The support for monitoring and maintenance of the Framework is a weakness of the CEF. Issues were identified in the early stages from dated datasets and incomplete value assessment during the evaluation interviews of the initial implementation. The government has mandated periodic strategic 84  level CEAs in an effort to keep the Framework updated. However, if insufficient datasets and other issues from a lack of monitoring programs continue, the Framework is in danger of becoming a large regulatory burden that is an additional exercise for proponents and decision-makers to navigate through. Procedural guidelines should be considered for monitoring in addition to expanding current monitoring programs’ resources.  The long-term sustainability of the CEF program will require stable funding. Although the province is not considering this now, and the survey did not address long term funding, this researcher recommends an analysis of cost recovery options to help support the CEF going forward. Additionally there may be value in replicating aspects of the Merritt monitoring example, as they have been working with the appropriate First Nations communities during the fieldwork.  Another recommendation is to ensure that the NRPP is successful in efforts to streamline environmental management. It is extremely crucial that the natural resource sectors and ministries adhere to the paradigmatic shift in environmental management through the consideration and emphasis on cumulative effects. The Framework is in some danger of being perceived by the other ministries as a FLNRO initiative. This concern was brought forward during the evaluation interviews, with respondents stressing the need for validation and utilization outside of FLNRO. Other government agencies must begin to see the value in the CEF, instead of viewing it primarily as a FLNRO objective. As the respondent stated, “This will only be a success if other ministries take accountability”.  Furthermore, if other ministries have a vested interest in the success of 85  the CEF more individuals are likely to prioritize the resources required to maintain the tool.  As the paradigm through which environmental decision-making is viewed, numerous policies will change and new ones will be implemented, such was the case with the CEF. Following implementation, the literature recommendations an evaluation to determine if the policy is meeting its predefined objectives (Bailey et al., 2011; Mickwitz & Birnbaum, 2009; Wholey, 2004). As was suggested by Maas et al. (2011), it is beneficial to embed policy evaluations formally into the policy lifecycle. The emphasis in implementation evaluations is on tracking the development of the policy, identifying outcomes and emerging themes (DeGroff & Cargo, 2009; Hilden, 2009; Mickwitz, 2006).  This study was ultimately a study of policy implementation, showcasing the benefit of early evaluation especially when a large policy framework, such as the CEF, is the subject of analysis. The implementation evaluation lead to challenges being brought forward through multiple data sources that would have otherwise gone unreported. While the majority of the findings in this research can be addressed by the provincial government with some ease, the early policy evaluation has resulted in the potential for strengthen the CEF over the long-term. Additional analysis of the decision-making process and the implications from early decisions may result in a greater level of transparency in the democratic setting and perhaps increased public approval for those involved in the process (Trochim, 2009).  Nevertheless, it is important to remember that these results only reflect the primary phase of implementation and the successive phases, from April 2016 onwards, 86  will see the Framework implemented in all regions throughout the province. The provincial government has stated that it will periodically evaluate the CEF and associated procedures. An evaluation of the second phase of implementation would provide beneficial insights to researchers and government, to determine if any changes resulting from this research occurred and any further issues that may be present during the final implementation.  Beyond the boundaries of the Framework, if CEA is to be a success not only throughout British Columbia, but perhaps nationally, there must be strong institutional and governmental support for the practice in order to ensure benefits for practitioners, policy-makers and stakeholders. Our current institutional arrangements have been argued as temporary, housing inadequate resources with limited capabilities to consider regional planning strategies (Parkins, 2011). CEA has the potential to become a key component of natural resource decision-making and management if the appropriate regulatory foundations are supportive.  CEA and the institutions and governments that support them must continuously evolve; changing with the environment they are meant to assess (Tengo & Von Heland, 2012). A longer-term vision and environmental management paradigm will only serve to strengthen CEA and EA’s ability to promote sustainable environmental decision-making. British Columbia has the opportunity to forge the path for a new paradigm of environmental assessment, shifting away from environmental governance of the past towards a more sustainable solution and become a leader of environmental assessments in the process.  87  References Adger, N. W., Brown, K., Fairbrass, J., Jordan, A., Paavola, J. Rosendo, S & Seyfang, G. (2003). Governance for sustainability: towards a ‘thick’ analysis of environmental decisionmaking. Environment and Planning, 35: 1095-1110.  Auditor General of British Columbia. (2015). Managing the Cumulative Effects of Natural Resource Development in British Columbia. Retrieved from  Bailey, S. A., Matthew, D., Laurent, J., Wiley, C., Leung, B., & MacIssac, H. (2011). Evaluating Efficacy of an Environmental Policy to Prevent Biological Invasions. Environmental Science & Technology, 45: 2554-2561.  Baker, D. & McLelland, J. N. (2003). Evaluating the effectiveness of British Columbia’s environmental assessment process for first nations’ participation in mining development. Environmental Impact Assessment Review, 23: 581-603.  Bardach, E. (2000). A practical guide for policy analysis the eightfold path to more effective problem solving. New York, N.Y.: Chatham House , Seven Bridges Press. Beanlands, G. E., & Duinker, P. N. (1984). An ecological framework for environmental impact assessment. Journal of Environmental Management, 18(3): 267–277.  Beanlands, G., Erckmann, W., Orians, G., O'Riordan, J., Policansky, D., Sadar., M, Sadler, B. (Eds.) (1986.) Cumulative environmental effects: a bi-national perspective. Ottawa, Canada: The Canadian Environmental Assessment and Research Council and the United States National Research Council, Minister of Supply and Services. 88  Berube, M. (2007). Cumulative effects assessment at Hydro-Quebec: what have we learned?  Impact Assessment and Project Appraisal, 25(2): 101-109. Bird, C. M. (2005). How I stopped dreading and learned to love transcription. Qualitative Inquiry, 5(1): 226-248. Blumer, H. (1969). Symbolic Interactionism: Perspective and Method. Berkeley: University of California Press. Bond, A. & Pope, J. (2012). The state of art of impact assessment in 2012. Impact Assessment and Project Appraisal, 30(1): 1-4.  Bressers, N., Van Twist, M. & Heuvelhof, E. (2013). Exploring the temporal dimensions in policy evaluation. Policy Sci, 46: 23-37.  Brewer, G. D. (1974). The policy sciences emerge: To nurture and structure a discipline.   Policy Sciences, 5(3), 239–244. Bruyninckx, H. (2009). Environmental evaluations practices and the issue of scale In M. Birnbaum & P. Mickmitz (Eds.), Environmental program and policy evaluation: Addressing methodological challenges. New Directions for Evaluation, 122: 31-39.  Brydon-Miller, M., Greenwood, D., & Maguire, P. (2003). Why action research?. Action research, 1(1): 9-28.  Buschke, F. T. & Vanschoenwinkel, B. (2013). Mechanisms for the inclusion of cumulative impacts in conservation decision-making are sensitive to vulnerability and irreplaceability in a stochastically simulated landscape. Journal for Nature Conservation, 22: 265-271.  89  Cabugueira, M. (2000). Voluntary Agreements as an environmental policy instrument – evaluation criteria. Journal of Cleaner Production, 9: 121-133.  Canadian Council of Ministers of the Environment (CCME). (2009). Regional Strategic Environmental Assessment in Canada Principles and Guidance. Canadian Council of Ministers of the Environment, Winnipeg, MB. Retrieved from  Canter, L. W. & Atkinson, S. F. (2011). Multiple uses of indicators and indices in cumulative effects assessment and management. Environmental Impact Assessment Review, 31: 491-501.  Canter, L., & Ross, B. (2010). State of practice of cumulative effects assessment and management: the good, the bad and the ugly. Impact Assessment and Project Appraisal, 28(4), 261-268. Canadian Environmental Assessment Act (CEAA). (2012). Retrieved from Cooper, L. M. & Sheate, W. R. (2002). Cumulative effects assessment: A review of UK environmental impact statements. Environmental Impact Assessment Review, 22: 415-429.  Connelly, R. (2011). Canadian and international EIA frameworks as they apply to cumulative effects. Environmental Impact Assessment Review, 31: 453-456.  Crain, C., Kroeker, K., & Halpern, B. (2008). Interactive and cumulative effects of multiple human stressors in marine systems. Ecology Letters, 11: 1304–1315.  Daan, C. & Nuemann, G. (2014). Policy Evaluation with Temporal Differences: A Survey and Comparison. Journal of Machine Learning Research, 15: 809-883.  90  Dawson, R., Hoffos, R. & McGirr, M. (2014). A Broad Scale Cumulative Impact Assessment Framework for the Cariboo-Chilcotin. Ministry of Forests, Lands and Natural Resource Operations. Day, J., Hocking, M. & Jones, G. (2002). Measuring effectiveness in Marine Protected Areas – principles and practice. World Congress on Aquatic Protected Areas: what works best and how do we know 42: 1-16.  DeGroff, A. & Cargo, M. (2009). Policy implementation: Implications for evaluation. In J. M. Ottoson & P. Hawe (Eds.). Knowledge utilization, diffusions, implementation, transfer, and translation: Implications for evaluation. New Direction for Evaluation, 124: 47-60. Dewey, J. (1938). Logic: The Theory of Inquiry. New York: Henry Holt Duinker, P. & Greig, L. A. (2006). The Impotence of Cumulative Effects Assessment in Canada: Ailments and Ideas for Redeployment. Environmental Management, 37(2): 153–161. Environmental Assessment Act (EAA). (2002). British Columbia Environmental Assessment Act. Retrieved from Fielding, N. G., & Fielding, J. L. (2008). Resistance and adaptation to criminal identity: Using secondary analysis to evaluate classic studies of crime and deviance. Historical Social Research/Historische Sozialforschung, 33(3): 75-93.  Flamos, A. & Spyridaki, N. A. (2013). A paper trial of evaluation approaches to energy and climate policy interactions. Renewable and Sustainable Energy Reviews, 40: 1090-1107.  91  Forest Act. (1996). Retrieved from[RSBC%201996]%20c.%20157/00_Act/96157_01.xml Forests, Lands and Natural Resource Operations (FLNRO). (2014). Addressing Cumulative Effects in Natural Resource Decision Making: CEF Overview Report.  Forests, Lands and Natural Resource Operations (FLNRO). (2012). Ministry of Forest, Lands and Natural Resource Operations Compliance and Enforcement Program: Annual report for year ended March 31, 2012. Retrieved from!publish/web/reports/C_and_E_annual_report_11- 12.pdf. Forest Practices Board (FPB). (2011). Cumulative Effects: From assessment towards management. Retrieved from  Goldkuhl, G. (2012). Pragmatism vs. interpretivism in qualitative information systems research. European Journal of Information Systems, 21: 135-146.  Government of British Columbia. (2016). Natural Resource Permitting Project Factsheet. Retrieved from  Hall, S. (2003). Reflexivity in Emancipatory Action Research: Illustrating the Researcher’s Constitutiveness. In Zuber-Skerritt, O. (Ed.) New Directions in Action Research. Gunpowder Square, London: The Falmer Press.  92  Hanna, K. S. (2016). Environmental Impact Assessment: Process, Setting, and Efficacy. In Hanna, K. (Ed.) Environmental Impact Assessment Practice and Participation (2nd ed.). Don Mills, Ontario: Oxford University Press.  Harriman, J. & Noble, B. (2008). Characterizing project and strategic approaches to regional cumulative effects assessment in Canada. Journal of Environmental Assessment Policy and Management, 10(1): 25–50. Harriman, J., & Noble, B. (2009). Integrating Cumulative Effects in Regional Strategic Environmental Assessment Frameworks: Lesson from Practice. Journal of Environmental Assessment Policy and Management, 11(3): 267-290. Harriman, J. & Noble, B. (2011). Conceptual and methodological challenges to integrating SEA and cumulative effects assessment. Environmental Impact Assessment Review, 31: 154 – 160.  Hampton, G. 2009. Narrative Policy Analysis and the Integration of Public Involvement in Decision Making. Policy Sciences, 42(3): 227-242.  Huang, H. B. (2010). What is good action research. Action Research, 8(1): 93-109. Hegmann, G. & Yarranton, G. A. (2011). Alchemy to reason: Effective use of Cumulative Effects Assessment in resource management. Environmental Impact Assessment Review, 31: 484-490. Hilden, M. (2009). Time horizons in evaluating environmental policies. In M. Birnbaum & P. Mickmitz (Eds.), Environmental program and policy evaluation: Addressing methodological challenges. New Directions for Evaluation, 122: 9-18.  93  Howlett, M. (2007). Analyzing Multi-Actor, Multi-Round Public Policy Decision-Making Processes in Government: Findings from Five Canadian Cases. Canadian Journal of Political Science, 40(3): 659-684.  Jaccard, M. & Rivers, N. (2006). Choice of environmental policy in the presence of learning by doing. Energy Economics, 28: 223-242.  Jarrell, M. L., Ozymy, J. & McGurrin, D. (2013). How to encourage conflict in the environmental decision-making process: imparting lessons from civic environmentalism to local policy-makers. Local Environment, 18(2): 184-200.  Jenkins, W. I. (1978). Policy analysis. London: Martin Robertson. Lapadat, J. C., & Lindsay, A. C. (1999). Transcription in research and practice: From standardization of technique to interpretive positioning. Qualitative Inquiry, 5(1): 64. Laswell, H. D. (1956). The decision process: Seven categories of functional analysis. College Park: University of Maryland. Levy, J. S. (2007). Qualitative Methods and Cross-Method Dialogue in Political Science. Comparative Political Studies, 40(2): 196-214. Maas, R., Kruitwagen, S. & Van Gerwen, O. (2011). Environmental policy evaluation: Experiences in the Netherlands. Environmental Development, 1: 67-78.  MacDonald, L. (2000). Evaluating and Managing Cumulative Effects: Process and Constraints. Environmental Management, 26(3): 299–315.  Martin, M. (2012). From Sensors to Citizens: Exploring the Tensions of using Web 2.0 technologies to support online volunteerism. (Master’s Thesis). The College of 94  Graduate Studies, The University of British Columbia.  McNiff, J. (2013). Action research: Principles and practice (3rd ed.). New York, New York: Routledge. Mickwitz P. (2006). Environmental Policy Evaluation: Concepts and Practice, Commentationes. Scientiarum Socialium 66. Finnish Society of Sciences and Letters: Vaajakoski.  Mickwitz, P. & Birnbaum, M. (2009). Key insights for the design of environmental evaluations. In M. Birnbaum & P. Mickwitz (Eds.), Environmental program and policy evaluation: Addressing methodological challenges. New Directions for Evaluations, 122: 105-112.  Morgan, D. L. (1997). Focus Groups as Qualitative Research (2nd ed.). Thousand Oaks: Sage Publications.  Morgan, R. (2012). Environmental impact assessment: the state of art. Impact Assessment and Project Appraisal, 30(1): 5 – 14.  Morrison-Saunders, A., Pope, J., Bond, A. & Retief, F. (2014). Towards sustainability assessment follow-up. Environmental Impact Assessment Review, 45: 38 – 45.  Morse, J. M. (2000). Determining Sample Size. Qualitative Health Research, 10(1): 3-5. Noble, B. (2008). Strategic approaches to regional cumulative effects assessment: A case study of the Great Sand Hills, Canada. Impact Assessment and Project Appraisal, 26(2): 78–90.  Noble, B. F. (2010). Cumulative environmental effects and the tyranny of small decisions: Towards meaningful cumulative effects assessment and 95  management. Natural Resources and Environmental Studies Institute Occasional Paper, (7). University of Northern British Columbia, Prince George, BC.  Parkins, J. R. (2006). De-centering environmental governance: a short history and analysis of democratic processes in the forest sector of Alberta, Canada. Policy Sciences, 39(2): 183-202. Parkins, J. R. (2011). Deliberative Democracy, Institution Building, and the Pragmatics of Cumulative Effects Assessment. Ecology and Society, 16(3): 20. Parkins, J. R. & Mitchell R. E. (2007). Public Participation as Public Debate. Deliberative Turn in Natural Resource Management. Society & Natural Resources: An International Journal, 18(6): 529-540.  Parsons, W. (1995). Public policy: An introduction to the theory and practice of policy analysis. Northampton, MA: Edward Elgar. Pope, J., Bond, A., Morrison-Saunders, A. & Retief, F. (2013). Advancing the theory and practice of impact assessment: setting the research agenda. Environmental Impact Assessment Review, 41: 1-9.  QSR International Pty Ltd. (2015). NVivo Qualitative Data Analysis Software, Version 11.  Richards, L & Morse, J. M. (2013). Readme First For a User's Guide to Qualitative Methods (Third Ed.). Thousand Oaks: Sage Publications. Ritchie, J., Lewis, J., Nicholls, C. M., & Ormston, R. (Eds.). (2003). Qualitative research practice: A guide for social science students and researchers. Thousand Oaks: Sage Publications 96  Robertson, J. (2000). The three Rs of action research methodology: reciprocity, reflexivity and reflection-on-reality. Educational Action Research, 8(2): 307-326. Rockman, B. (2011). Interviews, elite. In B. Badie, D. Berg-Schlosser & L. Morlino (Eds.) International encyclopedia of political science. (p. 1341-1344). Thousand Oaks: Sage.  Rudd, M. A, Beazley, K. F., Cooke, S. J., Fleishman, E., Lane, D. E., Maschia, M. B., Roth, R., Tabor, G., Bakker, J. A., Bellefontaine, T., et al. (2011). Generation of Priority Research Questions to Inform Conservation Policy and Management at a National Level. Conservation Biology, 25(3): 476-484. Rutherford, M. B. (2016). Impact Assessment in British Columbia. In Hanna, K. (Ed.) Environmental Impact Assessment Practice and Participation (2nd ed.). Don Mills, Ontario: Oxford University Press.  Sánchez, L. E. & Morrison-Saunders, A. (2011). Learning about knowledge management for improving environmental impact assessment in a government agency: The Western Australian experience. Journal of Environmental Management, 92(9): 2260 – 2271.  Sanderson, I. (2009). Intelligent Policy Making for a Complex World: Pragmatism, Evidence and Learning. Political Studies, 57: 699-719.  Sandelowski, M. (1991). Telling stories: narrative approaches in qualitative research. Image: Journal of Nursing and Health, 20: 161-166. Senner, R. (2011). Appraising the sustainability of project alternatives: An increasing role for cumulative effects assessment. Environmental Impact Assessment Review, 31: 97  502-505.  Spruijt, P., Knol, A. B., Vasileiadou, E. Devilee, J., Lebret, E. & Peterson, A. C. (2014). Roles of scientists as policy advisers on complex issues: A literature review. Environmental Science & Policy, 40: 16-25.  Stahl, C. H. (2014). Out of the Land of Oz: the importance of tackling wicked environmental problems without taming them. Environmental System Decisions, 34: 473-477. Stewart, D. W. & Shamdasi, P. M. (1990). Focus Groups: Theory and Practice, Newbury Park: Sage Publications.  Stringer, E. (2007). Action Research (3rd ed.). Thousand Park, CA: Sage Publications. Svenfelt, A., Engstrom, R. & Hojer, M. (2010). Use of explorative scenarios in environmental policy-making: Evaluation of policy instruments for management of land, water, and the built environment. Futures, 42: 1166-1175. Tengo, M. & Von Heland, J. (2012). Adaptive capacity of local indigenous institutions: the case of the taboo forests of southern Madagascar. In Boyd, E. & Folkes, C. (Eds.) Adapting Institutions: Governance, Complexity and Social-Ecological Resilience. Cambridge University Press. Therivel, R. & Ross, B. (2007). Cumulative effects assessment: Does scale matter? Environmental Impact Assessment Review, 27: 365 – 385.  98  Thomas, G. (2005). The Qualitative Foundations of Political Science Methodology.  Perspectives on Politics, 3(4): 855-866.  Throne, S. (1994). Secondary analysis in qualitative research: Issues and implications. In Morse, J. M. (Ed.) Critical issues in qualitative research methods. Thousand Oaks: Sage Publications.  Trochim, W. M. K. (2009). Evaluation policy and evaluation practice. In W.M.K. Trochim, M. M. Mark, & L. J. Cooksy (Eds.), Evaluation policy and evaluation practice. New Directions for Evaluation, 123: 13–32.  Valdal, E. S. & Lewis D. W. (2015). Cumulative Effects Assessment for the Merritt Operational Trial: Draft 3.0. Ministry of Forests, Lands and Natural Resource Operations. Wholey, J. S. (2004). Evaluability assessment. In J. S. Wholey, H. P. Hatry, & K. E. Newcomer (Eds.), Handbook of practical program evaluation (2nd Ed.). San Francisco: Jossey-Bass.  Yin, R. K. (2009). Case study research: Design and methods. Thousand Oaks, Calif.: Sage Publications.  99  Appendix A: Evaluation Template Context: The province is currently developing policy, procedures and tools to support effective and consistent implementation of the Cumulative Effects Framework. At the same time, early implementation is proceeding in selected areas where pilot CEA’s have been complete to date. Case studies have been selected to evaluate recent decisions that have considered these initial CEA’s, for the purpose of gaining insight into implementation progress and effectiveness. Informal interviews are being undertaken with those directly involved in the decision-making process. Insights derived from these case studies will inform the development of provincial policy and procedures and support continuous improvement.   1. Description of Activity Proposed / Decision Context: Decision Context  Respondents  Location  Consultation      100  2. Description of CE / Values information considered CEA Description    Other information considered  Referrals   3. Description of Decision Made and Documentation Decision Outcome      101  Part B: Evaluating the Effectiveness of CEA Information in Supporting Decision-Making B1. To what extent are CEA products sufficient to inform and support decision-making? 1. Please indicate the extent to which you believe the CEA information met each of the following criteria on a scale of 1 to 5 (where 1 = not at all; 2 = slightly; 3 = somewhat’; 4 = quite; 5 = very), and provide any comments you would like to describe the reasons for your evaluation.  1 2 3 4 5 Comments   Clear and understandable        Relevant to the decision        Helpful in supporting the decision process          102   2. Please indicate the extent to which you believe the CEA information was useful in helping to inform your understanding of the condition and trend of the selected values, on a scale of 1 to 5 (where 1 = not at all; 2 = slightly; 3 = somewhat’; 4 = quite; 5 = very). Please indicate the extent to which you believe the CEA information? Values Considered 1 2 3 4 5 Comments                                                    103  3. Was there any CEA information that you were not able to effectively consider in this decision, and if so, why? For example, there were regulatory barriers to your consideration of the information, or was there information not relevant to the decision being made, or not at the right scale?      First Nations Consultation and/or assessment of impacts to rights 4. Did First Nations indicate CE related concerns with this proposed activity/ decision?  5. Was the CEA information shared directly with First Nations? If yes, please describe how it was shared and/or discussed with First Nations.  6. Was the CEA information used to support an assessment of potential impacts to Aboriginal or Treaty rights?    104  B2. To what extent are resource decisions supported and executed differently as a result of CEA information? 7. How did the CEA information influence the decision at hand:  Please describe  If affected the outcome of the decision  It affected the mitigation strategies identified/approved for values  It affected the requirements set out as conditions of the authorization  If affected First Nations consultation & accommodation (e.g. support from First Nation; depth of consultation & accommodation)  If affected the timelines for completing the authorizations process. (e.g. increased or decreased process timelines)    If affected the staff resourcing required to support decision process (e.g. requirement for staff referral avoided, reduced or increased?)   It affected the legal risk associated with this decision  (e.g. risk of appeal? Durability of decision)  Others?   105   8. Overall, how you rate the effectiveness of the CEA information in supporting the decision at hand, on a scale of 1 to 5 (where 1 = not at all effective; 2 = slightly; 3 = somewhat; 4 = quite; 5 = very effective). Please elaborate on your reasons for this rating. 1 2 3 4 5 Comments         9. Overall, how would you rate the potential effectiveness of the CEA information in ensuring government objectives for these values are/continue to be  achieved, on a scale of 1 to 5 (where 1 = not at all effective ; 2 = slightly; 3 = somewhat; 4 = quite; 5 = very effective). Please  elaborate on your reasons for this rating.   1 2 3 4 5 Comments          Opportunities for Improvement 10. Do you have any recommendations for improving the quality or utility of the CEA information that was provided?  106  11. What recommendations do you have for policy, regulation or business processes that would help improve the consideration of cumulative effects in natural resource decision-making broadly, or for specific types of decisions (please define).     


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items