@prefix vivo: . @prefix edm: . @prefix dcterms: . @prefix skos: . @prefix ns0: . vivo:departmentOrSchool "Applied Science, Faculty of"@en, "Community and Regional Planning (SCARP), School of"@en ; edm:dataProvider "DSpace"@en ; dcterms:creator "Procyk, Andrea"@en ; dcterms:issued "2016-02-01T13:08:53"@en, "2010-08"@en ; dcterms:description "Efforts to make urban areas more resilient to disasters are becoming increasingly important. One key facet of resilient urban areas is functioning infrastructure systems. This study represents an evaluation of a methodology used to characterize infrastructure vulnerabilities to disaster events. The intended clients for this project are those interested in the implementation of approaches for understanding what makes urban regions more resilient, such as municipal planners, infrastructure managers, researchers and consultants."@en ; edm:aggregatedCHO "https://circle.library.ubc.ca/rest/handle/2429/32605?expand=metadata"@en ; skos:note " EVALUATING A METHODOLOGY FOR CHARACTERIZING RESILIENCE OF INFRASTRUCTURE SYSTEMS by Andrea Procyk B.A., Simon Fraser University, 2002 A PROJECT SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS (PLANNING) In THE FACULTY OF GRADUATE STUDIES School of Community and Regional Planning We accept this project as conforming to the required standard ………………………………….. ………………………………….. …………………………………. THE UNIVERSITY OF BRITISH COLUMBIA August 2010 © Andrea Procyk, 2010 2 Executive Summary Efforts to make urban areas more resilient to disasters are becoming increasingly important. One key facet of resilient urban areas is functioning infrastructure systems. This study represents an evaluation of a methodology used to characterize infrastructure vulnerabilities to disaster events. The intended clients for this project are those interested in the implementation of approaches for understanding what makes urban regions more resilient, such as municipal planners, infrastructure managers, researchers and consultants. The Analyzing Infrastructures for Disaster Resilient Communities (AIDRC) project developed an approach consisting of four linked, sequential phases: (1) building hazard scenarios and compiling background information; (2) conducting expert interviews; (3) data synthesis; and (4) carrying out a workshop event for information-sharing among infrastructure system owners and operators. By applying this method the AIDRC researchers attempted to address some of the current challenges to fostering infrastructure resilience: conflict between organizational and regional interests, incomplete information regarding vulnerabilities and consequences of infrastructure failure interdependencies (IFIs), and lack of direct and collective learning opportunities. The objective of this study is to determine whether the approach developed and applied by the AIDRC project is “effective”. This study reviews the relevant literature on evaluation, public participation and disaster resilience, specifically to construct a set of criteria with which to evaluate the AIDRC approach. This study defines an “effective” approach as one that: 1) achieves its own objectives; 2) fosters capacity for future decision making; 3) can be replicated in other geographic areas/contexts; and 4) satisfies participants. Data for the evaluation is derived from workshop exit surveys, workshop workbooks, as well as notes taken during interviews and workshop events. Findings indicate that the AIDRC approach fulfilled the evaluation criteria and can be considered a useful means for characterizing regional vulnerabilities and supporting efforts to increase the disaster resilience of communities. 3 Table of Contents 1. INTRODUCTION ........................................................................................................................ 6 1.1 - Client & Terms of Reference ......................................................................................................... 6 1.2 – Project Overview ........................................................................................................................... 6 1.3 - Research Goal ................................................................................................................................. 7 1.4 - Research Questions ....................................................................................................................... 7 1.5 - Research Objectives ....................................................................................................................... 7 2. CONTEXT ................................................................................................................................... 8 2.1 - Monitoring and Evaluation .......................................................................................................... 8 2.2 - Evaluation in Expert Deliberation & Public Participation Literature .................................... 9 2.3 - Disasters and Resilience Literature .......................................................................................... 12 2.4 – Analyzing Infrastructures for Disaster Resilient Communities Approach ........................ 16 2.4.1 - Characteristics & Objectives .................................................................................................... 16 2.4.2 - Description ............................................................................................................................... 17 3. METHODS ................................................................................................................................ 23 3.1 - Steps of Evaluation ...................................................................................................................... 23 3.2 - Evaluation Criteria ...................................................................................................................... 23 3.3 - Data Sources ................................................................................................................................ 24 3.3.1 - Exit Surveys .............................................................................................................................. 25 3.3.2 - Workbook Exercises ................................................................................................................. 25 3.3.3 – Interview and Workshop Notes ............................................................................................... 25 4. RESULTS .................................................................................................................................. 26 4.1 - Participant Information .............................................................................................................. 26 4.2 - Exit Survey Results – 2009 Workshop .................................................................................... 27 4.3 - Exit Survey Results – 2007 Workshop ..................................................................................... 36 4.4 - Workbook Exercises -2007 & 2009 .......................................................................................... 39 5. ANALYSIS & DISCUSSION ...................................................................................................... 48 5.1 - Performance on Evaluation Criteria.......................................................................................... 48 5.1.1 - Does the Analyzing Infrastructures for Disaster Resilient Communities approach achieve its own objectives? ................................................................................................................................... 48 4 5.1.2 - Does the Analyzing Infrastructures for Disaster Resilient Communities approach foster capacity for future decision making? .................................................................................................. 50 5.1.3 - Could the Analyzing Infrastructures for Disaster Resilient Communities approach be replicated in other geographic areas or contexts? ............................................................................. 51 5.1.4 - Are participants generally satisfied with the Analyzing Infrastructures for Disaster Resilient Communities approach? ..................................................................................................................... 52 5.2 - Challenges and Limitations of the Analyzing Infrastructures for Disaster Resilient Communities Approach ....................................................................................................................... 52 5.3 - Challenges and Limitations of the Evaluation......................................................................... 53 6. CONCLUSION .......................................................................................................................... 55 7. REFERENCES ........................................................................................................................... 56 APPENDIX A ................................................................................................................................ 60 2009 - Workbook Exit Survey ................................................................................................... 60 2007 – Workbook Exit Survey .................................................................................................. 63 List of Tables Table 1 - National Academy of Sciences, 2008 Criteria ............................................................................. 11 Table 2 - Critical Infrastructures Represented at the 2007 and 2009 .......................................................... 26 List of Figures Figure 1 - Infrastructure Failure Interdependency Example ....................................................................... 15 Figure 2 - Overview of AIDRC Research Approach .................................................................................. 18 Figure 3 - Service Disruption Diagram ....................................................................................................... 20 Figure 4 - Service Disruption Scale ............................................................................................................ 20 Figure 5 - Interdependency Diagram .......................................................................................................... 21 Figure 6 - Fundamental Objectives Network for Evaluation of AIDRC Approach .................................... 24 Figure 7 - Impacts of Flood on Your System .............................................................................................. 27 Figure 8 - Impacts of Flood on Other Systems ........................................................................................... 28 Figure 9 - Consequences of IFIs ................................................................................................................. 28 Figure 10 - Regional Vulnerability to Disasters ......................................................................................... 29 Figure 11 - Organizational Priority Setting ................................................................................................. 30 Figure 12 - Confidence in Knowledge of IFIs ............................................................................................ 31 Figure 13 - Confidence in Knowledge of Regional Vulnerabilities to Disaster ......................................... 31 Figure 14 - Time Well Spent....................................................................................................................... 32 Figure 15 - Balanced Presentation Materials .............................................................................................. 33 5 Figure 16 - Accurate Presentation Materials ............................................................................................... 33 Figure 17 - Opportunities for Sharing Ideas ............................................................................................... 34 Figure 18 - Opportunities for Interaction .................................................................................................... 34 Figure 19 - Collective Efforts to Manage IFIs ............................................................................................ 35 Figure 20 - Applicability of the Research Approach .................................................................................. 36 Figure 21 - Time Well Spent (2007) ........................................................................................................... 37 Figure 22 - Meet Expectations (2007) ........................................................................................................ 37 Figure 23 - Learning for Organizational Priority Setting (2007) ................................................................ 38 Figure 24 - Infrastructure Interdependencies Diagram - 2009 Flood - Original Pre Workshop ................. 40 Figure 25 - Infrastructure Interdependencies Diagram - 2009 Flood - Revised Post Workshop ................ 41 Figure 26 - Infrastructure Service Disruption Diagram - 2009 Flood - Original Pre Workshop ................ 42 Figure 27 - Infrastructure Service Disruption Diagram - 2009 Flood - Revised Post Workshop ............... 43 Figure 28 - Infrastructure Interdependencies Diagram - 2007 Earthquake - Original Pre Workshop ........ 44 Figure 29 - Infrastructure Interdependencies Diagram - 2007 Earthquake - Revised Post Workshop ....... 45 Figure 30 - Infrastructure Service Disruption Diagram - 2007 Earthquake - Original Pre Workshop ....... 46 Figure 31 - Infrastructure Service Disruption Diagram - 2007 Earthquake - Revised Post Workshop ...... 47 6 1. INTRODUCTION As concerns about the potential impacts of climate change intensify, efforts to make urban areas more resilient to disasters are becoming increasingly important. A key component of resilience for urban areas is functioning infrastructure systems. Because infrastructure systems are interconnected and interdependent, a regional perspective of infrastructure vulnerabilities in disasters is essential. The Analyzing Infrastructures for Disaster Resilient Communities (AIDRC) project developed an approach to characterize infrastructure vulnerability in a real- world context. In creating the approach, researchers attempted to address the current challenges to fostering infrastructure resilience: conflict between organizational and regional interests, incomplete information regarding vulnerabilities and consequences of infrastructure failure interdependencies (IFIs), and lack of direct and collective learning opportunities. The approach consists of four linked, sequential phases: (1) building hazard scenarios and compiling background information; (2) conducting expert interviews; (3) data synthesis; and (4) carrying out a workshop event for information-sharing among infrastructure system owners and operators. This approach is intended to encourage a shared understanding of regional vulnerabilities to infrastructure failures in disasters to provide a foundation for the implementation of mitigation efforts, particularly those that rely on inter-sectoral cooperation. 1.1 - Client & Terms of Reference The intended clients for this project are those interested in the implementation of approaches for understanding what makes urban regions more resilient. These potential clients include: municipal and regional planners, infrastructure managers, emergency planners and managers, researchers, and consultants. The purpose of this project is to evaluate an approach for characterizing regional vulnerability to disasters. The project focuses on providing information that may assist this audience with how to better understand the regional impacts of disasters, particularly with regard to infrastructure failures. This evaluation uses an assumed set of objectives and performance criteria to assess the effectiveness of the AIDRC approach. 1.2 – Project Overview This paper first explores the research context through discussions of the literature pertaining to evaluation, knowledge translation and public participation. This is followed by an overview of the concept of disaster resilience in urban areas and a detailed description of the AIDRC 7 approach to characterizing vulnerabilities to disaster. The evaluation criteria and methods are introduced and research results are presented. Lastly, a discussion section focuses on the performance of the AIDRC approach relative to the evaluation criteria and overarching research questions. 1.3 - Research Goal The purpose of this study is to conduct an evaluation of the AIDRC project‟s approach to characterizing regional vulnerabilities to infrastructure failures in disasters. 1.4 - Research Questions The project is motivated by the following research questions: Is the approach used by the AIDRC project effective? Is there potential for this approach to be used in other cities or regions for developing a regional perspective on the vulnerabilities of infrastructures in disasters? Could this process be applied to other interconnected problems facing regions? 1.5 - Research Objectives The project focuses on the following objectives: Develop an appropriate set of evaluation criteria Apply the criteria to one or more cases Document the effectiveness of the AIDRC approach 8 2. CONTEXT This project was informed by research in two key areas. The literature on monitoring and evaluation, public policy and public participation was used to develop the evaluation approach and evaluation criteria. In addition, the research on disaster resilient communities and infrastructure interdependencies provides the context for the AIDRC methodology that is being assessed. 2.1 - Monitoring and Evaluation The practice of monitoring and evaluation is well developed in the fields of public administration and education. It is also of increasing interest to the field of planning because planners often experience uncertainty in assessing the effectiveness or impact of their interventions (Seasons, 2003). The term „evaluation‟ may be defined as “the systematic assessment of the operation and/or outcomes of a program or policy, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of the program or policy” (Weiss, 1998, 4). Or more simply put, “evaluation is about determining merit or worth” (Chelimsky & Shadish, 1997, p. xii). Monitoring and evaluation serves several purposes. Patton and Sawicki suggest that one such purpose is to help ensure that policies have been implemented as intended (1993). Evaluation is also used to determine whether the correct alternative (i.e. course of action) was selected and if it is having the desired affect (Patton and Sawicki, 1993). In addition, the results from evaluations may provide valuable information on what does and does not work for future programs, planning and policy (Patton and Sawicki, 1993). Evaluations may be conducted either ex-ante or ex-post. Ex-ante evaluations take place prior to implementation of a program or policy whereas ex-post analyses examine programs in operation (Patton and Sawicki, 1993). An ex-post evaluation involves using data to determine whether the policy or program is achieving its objectives, or desired outcomes. Evaluations may employ a variety of approaches and research methods. In “Evaluating with Validity”, House presents a taxonomy of evaluation approaches (1980). Relevant for this case is the decision-making category which represents approaches directed towards decision makers and administrators and uses methods such as surveys, questionnaires and interviews to determine the effectiveness of a program (House, 1980). 9 The following general principles, described by Patton and Sawicki, constitute some of the central components of an evaluation: Determine the focus of the evaluation Determine what change is being measured Define objectives and evaluation criteria Use multiple methods of measurement This project represents an ex-post evaluation of a planning approach. 2.2 - Evaluation in Expert Deliberation & Public Participation Literature In recent years, efforts have been made to evaluate the success of public participation programs, particularly in relation to risk analysis and communication (e.g. Chess, 2000; Rowe and Frewer, 2000; Rowe et al., 2004). In addition, there is ongoing research to determine whether value focused thinking, a defining characteristic of structured decision making, results in “better” decisions (e.g. Arvai et al., 2001; McDaniels et al., 2003). These research endeavours offer insights for developing the evaluation process criteria for this project. National policy in Canada, the United States, and Europe has emphasized public involvement in the decision making process in risk analysis (Renn, 1999). Federal Departments in Canada, such as Health Canada and Fisheries and Oceans Canada (DFO), have developed department policies and made commitments to ensure meaningful public involvement (Health Canada, 2000; DFO, 2004). In addition, the National Academy of Sciences has issued multiple reports encouraging public involvement in risk management, with a 2008 document recommending that “public participation should be fully incorporated into environmental assessment and decision-making processes, and it should be recognized by government agencies and other organizers of the processes as a requisite of effective action, not merely a formal procedural requirement” (Stern and Fineberg, 1996; Dietz and Stern, 2008, p. 226). However, it has also been argued that knowledge about what works in public participation and deliberation is limited (Stern and Fineberg, 1996). Evaluation is therefore an important task to determine the characteristics of public participation that result in a successful process and outcomes. Given that public participation encompasses a wide range of different exercises, such as referendums, public hearings, citizen/public advisory committees, and focus groups, developing methods of evaluation is a complex undertaking. 10 Evaluations of public participation activities tend to explore and assess the process (e.g. Longstaff and Burgess, 2010) or the outcomes. Process evaluations focus on how the participation activities take place whereas outcome evaluations focus on the results of the public participation process (Chess, 2000). Several challenges in conducting evaluations have been identified in the literature, such as a lack of agreed-upon evaluation methods and measurement tools (Rowe et al., 2004). Rowe and Frewer (2000) argue that the lack of experimental results, comparing different participation methods to determine which is more effective, is a reflection of the difficulties associated with conducting controlled experiments in this field (Rowe and Frewer, 2000). One of the most frequently mentioned problems is the lack of widely held criteria for judging the success of an exercise (Rowe et al., 2004), though several researchers have attempted to develop definitions and criteria for success or effectiveness. Rose and Frewer indicate that most of the criteria found in the literature relate to characteristics that make an effective process as opposed the measurement of effective outcomes, such as the quality of the decision (2000). McDaniels et al. (2003) suggest that part of the reason why decision quality is measured by process rather than outcome is that there is no right answer for the types of problems these processes often address. Despite these challenges, definitions of “effectiveness” have emerged and they typically include multiple criteria. Rowe et al. suggests the multi-criteria definitions reflect “the complex and multidimensional nature of success” (2000, p. 4). The National Academy of Sciences 2008 report presents a number of evaluation criteria derived from a thorough review of the literature. These criteria fall into three main categories: quality, legitimacy and capacity. Table 1 provides an overview of each category with its associated indicators. It is recognized by the National Academy of Sciences that the suggested criteria may not be appropriate for all processes or evaluations, but that the broad categories “cover the most important kinds of results and can be made concrete enough to help discriminate between different degrees of performance quality” (Dietz and Stern, 2008, p.73) . Given that the AIDRC project involves stakeholder participation and judgments, it is appropriate to consider these criteria in developing this evaluation. 11 Table 1 - National Academy of Sciences, 2008 Criteria Types of Results Indicators Quality of Assessments or Decisions Concerns expressed by publics were addressed in analysis Information was added; more information was considered in the process Technical analyses were improved Outputs reflected a broad view of the situation that addressed all issues considered important by participants Conclusions were based on and consistent with the best available evidence Innovative ideas were generated for solving problems Legitimacy of Process and Decisions Preexisting conflict was reduced or dissent clearly acknowledged and dealt with, Mistrust among participants, including government agencies, was reduced Participants accepted the assessment or decision process as having conformed to standards of sound analysis and decision making, even if they did not agree with the final assessment or recommendation for action The assessment or decision was widely accepted, even among nonparticipants Participants went outside the process to overturn its results, for example, with legal challenges or attempts to influence legislation (a negative indicator) Capacity for Future Decisions Public participants became better informed about relevant environmental, scientific, social, and other issues Participants and public officials gained a better understanding of each other Public officials gained skill in organizing decision processes Participants gained skill in participatory decision making Scientists gained understanding of public concerns Scientists developed, or committed to develop, new data or methods *Adapted from page 71 of Public Participation in Environmental Assessment and Decision Making 12 2.3 - Disasters and Resilience Literature Disasters, both natural and human-induced, can have far reaching societal impacts. Natural disasters are occurring more often and are having dramatic impacts in terms of human and economic costs (World Bank, 2004). In its most recent report, the Red Cross estimates that there were 235,736 people killed and 213 million people impacted by natural disasters in 2008 (2009, Annex 1, p. 155). Furthermore, the cost of 2008‟s natural disasters is estimated at US$181 billion, which is the second highest of the decade (Red Cross, 2009). Even a single disaster event can have staggering economic costs. For example, California‟s Northridge earthquake in 1994 resulted in more than US$30 billion in losses (Burby, 1999). It is expected that the frequency and cost of natural disasters will continue to increase globally as the result of environmental degradation, climate change, and population growth, especially in cities (World Bank, 2004). Global climate change is not expected to occur as a slow shift in average conditions. Rather, it is expected to present itself as more extreme weather events, such as floods, droughts and heatwaves (van Aalst, 2006). It is anticipated that the extreme weather events, due to increased energy within the climate system, will result in more weather related disasters (Helmer and Hilhorst, 2006). Given the future potential for disasters, efforts to better understand vulnerability to disaster and to reduce or eliminate long term risks from hazards to people and property (i.e. mitigate) are becoming increasingly important (Godschalk, 2003). Godschalk suggests that most of the human and economic losses resulting from disasters have occurred where urban settlements have been developed near known hazard areas, such as earthquake fault zones, floodplains, and shorelines susceptible to hurricanes (2003). Furthermore, the world‟s population is becoming increasingly urbanized, with 2008 representing the year where over 50 percent of the global population was classified as living in cities (UNFPA). Therefore, it is important to address the issue of disasters in relation to urban regions. Cities have been characterized as “the most complex of human creations” and are considered to be highly vulnerable to natural hazards and terrorism (Moor, 2001; Godschalk, 2003, p. 137). According to Godschalk, this vulnerability stems from population density, architectural structures, places of assembly and interconnected infrastructure systems (2003). In light of this vulnerability, the need for disaster planning is evident. The literature indicates that the goal of planning in this context should be to create urban areas that are resilient (Burby, 1999; Godschalk, 2003). The resilience of a complex system, such as a city, can be defined as its capacity to absorb shocks while maintaining function (Chang et al., 13 2010). Similarly, O‟Rourke describes community resilience as an overarching attribute that reflects the degree of community preparedness and the ability to respond to and recover from a disaster (2007). Godschalk argues that it is becoming increasingly important for cities everywhere to develop the ability to withstand a major shock without long-term, debilitating physical, social, or economic damage (2003). Therefore, for a city to be resilient to disaster its physical systems, such as roads and utilities, must be able to withstand extreme stresses (Godschalk, 2003). Godschalk (2003) further expands on this concept by presenting two reasons that resilience is an important goal. First, there is uncertainty in predicting the vulnerability of technological and social systems to hazards. Second, a resilient city should respond better in a disaster, resulting in fewer human and economic losses. A key aspect of resilience in urban environments pertains to infrastructure. Canadians rely on such systems as transportation, water, energy, telecommunications, banking, government services and agriculture for health, safety, and economic well being. Chang et al. suggest that nearly all vital economic and social function depends on the secure and reliable operation of critical infrastructures (2007). Critical infrastructures, or lifeline systems, have been defined as „„physical and information technology facilities, networks, services and assets, which if disrupted or destroyed would have a serious impact on the health, safety, security or economic well-being of Canadians or the effective functioning of governments in Canada‟‟ (GOC, 2004, p. 5). Loss of these critical infrastructure systems can have substantial impacts including economic losses to business, displacement of people from their homes and disrupted emergency response services. Furthermore, disruptions to critical infrastructure systems may prolong and exacerbate the impacts of natural disasters (Chang, 2003). Due to the fact that these systems are so closely linked to societal well-being, the strength and rapid recovery of these systems in an extreme event are clearly linked to concepts of resilience (O‟Rourke, 2007). Given the importance of critical infrastructures, it is crucial for planners and decision makers to understand the vulnerabilities of these systems to disasters. It is recognized that critical infrastructure systems are highly interdependent and interconnected. O‟Rourke clearly describes this characteristic of critical infrastructure systems in the following: “Electric power networks, for example, provide energy for pumping stations, storage facilities, and equipment control for transmission and distribution systems for oil and natural gas. Oil provides fuel and lubricants for generators, and natural gas provides energy for generating stations, compressors, 14 and storage, all of which are necessary for the operation of electric power networks. This reciprocity can be found among all lifeline systems” (2007, p. 23). Rinaldi et al. provide definitions of four types of infrastructure interdependence: physical, cyber, geographical, and logical (2001). Physical interdependence occurs when the output of one infrastructure is required by another infrastructure as an input for operation. For example, a coal-fired power plant may rely on transportation infrastructures (trains) to provide a source of fuel to generate power, but electricity is a required input for the signaling and other operations of the rail infrastructure. Cyber interdependence occurs when the operation of an infrastructure depends on information transmitted through information infrastructure. For example, Supervisory Control and Data Acquisition (SCADA) systems are used to distribute electricity relying on communication systems to rely information for operation, but electricity is also required to operate communications systems. Geographic interdependence occurs when a local environment has the potential to affect service of multiple infrastructures. For example, transportation, electrical and telecommunications infrastructures are often co-located. Lastly, logical interdependence occurs when a relationship exists between two infrastructures that cannot be classified as one of the aforementioned interdependencies. Rinaldi et al. argue that the relationships among infrastructures have been altered by technological, economic and regulatory change and that advances in information technology have resulted in more interconnections, complexity, and centralization of control in infrastructure systems (2001). Research shows that critical infrastructures are highly interconnected, particularly through physical proximity and operational interaction, and systems in crowded urban areas are vulnerable to increased risk from proximity (O‟Rourke, 2007). Peerenboom et al. indicate that interconnected infrastructures are increasingly fragile and vulnerable to disruptions (2002). Furthermore, Chang et al. suggests that infrastructure systems are becoming increasingly vulnerable to failures as a result of more congestion (2007). Due to the influence that critical infrastructure systems have on one another, researchers have been exploring the implications of infrastructure failures. Rinaldi et al. aptly state that “interdependencies increase the risk of failures or disruptions in multiple infrastructures” (2001, p. 22). The term infrastructure failure interdependencies (IFI) is used to describe “failures in interdependent infrastructure systems that are due to an initial infrastructure failure stemming from an extreme event” (McDaniels et al., 2007, p. 175). The feedback loops resulting 15 from interdependencies can initiate and propagate system disturbances in ways that are difficult to anticipate (Rinaldi et al., 2001). Rinaldi et al. have classified interdependence-related disruptions as cascading, escalating or common cause (2001). When a service disruption in one infrastructure causes a failure in a downstream, or dependent, infrastructure that results in a disruption of service in the second infrastructure it is referred to as a cascading failure. An escalating failure is defined as a situation when disruption in one infrastructure intensifies an independent disruption in a second infrastructure by increasing the severity of service disruption or delaying recovery or restoration of services for that infrastructure. Lastly, a situation when two or more infrastructure systems are disrupted at the same time as a result of a common cause is referred to a common cause failure. Figure 1 shows an example of an IFI that could occur as the result of disruption to electrical services. Figure 1 - Infrastructure Failure Interdependency Example Recent disasters have provided many examples of IFIs. The AIDRC project created a database that contains examples of a variety of unique IFIs which occurred in a number of disaster events in recent years, such as the 1998 Ice Storm in eastern Canada. There were 107 IFIs indentified for the Ice Storm. For example, major employers were shut down for two weeks as a result of power outages that affected a large number of people and losses in economic output (Chang et al., 2007). The Kobe earthquake of 1995 resulted in power losses that affected 2.6 million households and resulted in cascading failures in other infrastructure sectors. Transportation was severely affected by traffic signal outages that in turn disrupted the ability of emergency 16 responders to provide services (Chang et al., 2007). These types of disruptions have been experienced as a result of floods, hurricanes and other natural or human-induced disasters. Made evident by the aforementioned examples, the consequences of IFIs can have far reaching societal implications. In addition to the direct impacts on society, a disaster may trigger a series of IFIs that disrupt a number of infrastructures and result in additional health, safety and/or economic impacts. In the instance of a power outage, interdependencies among infrastructure systems could prolong and exacerbate the consequences of the initial outage (McDaniels et al., 2007). Therefore, understanding and addressing the relationships among infrastructures and the potential for IFIs is imperative in order for planners and decision makers to respond effectively to infrastructure disruptions and to foster disaster resilience in urban regions (Peerenboom, 2002). There is a growing body of literature on critical infrastructures and infrastructure failure interdependencies, yet identifying and analyzing interdependencies remains challenging (Rinaldi et al., 2001). While some research has focused on the development of engineering and simulation models (e.g. Chou, 2010), information on the nature, severity and impacts of IFIs in actual disasters is lacking (Chang et al., 2009). Information about the types of IFIs that most often occur, the IFIs that result in the most serious losses and the types of infrastructure that are most vulnerable could help decision makers identify priorities for how to mitigate IFIs (Chang et al., 2009). Chang et al. identify a need for methods to characterize infrastructure vulnerability and resilience in applied, real-world contexts (Chang et al., 2010). 2.4 – Analyzing Infrastructures for Disaster Resilient Communities Approach 2.4.1 - Characteristics & Objectives The AIDRC approach serves as a means of assessing regional vulnerabilities to disasters. It is suggested that it can be applied by planners in order to reduce risks associated with IFIs (Chang et al., 2010).The approach uses structured data gathering and a workshop with information sharing to address three challenges or limitations identified in the existing literature on resilience in the context of urban regions. Chang et al. (2010) identifies these challenges as: 1) Incomplete incentives on the part of infrastructure owners and operators that do not completely address societal interests regarding IFIs 2) Partial or asymmetric information regarding vulnerabilities and consequences of IFIs 17 3) Few opportunities for collective learning from direct experience As described by Chang et al. (2010), the AIDRC approach shares some commonalities with other infrastructure interdependency assessment approaches, but there are several characteristics by which it can be differentiated. The focus of the approach is in support of pre-disaster mitigation and preparedness activities so a broader temporal and decision-making perspective is utilized. This contrasts with methods that aim to support emergency response activities in the immediate post-disaster time frame. As a result, the approach is primarily directed towards the decision- making activities of infrastructure managers and planners and may be somewhat less applicable to emergency managers. The AIDRC method employs a single event approach rather than assess the risks from all hazards. Another defining characteristic of the approach is a systems- orientation. As opposed to identifying critical assets and vulnerability of specific infrastructure elements individually, this approach focuses on the entire network of infrastructures in a given region. Expert judgments serve as a primary source of data on impacts and dependencies between infrastructures whereas other approaches rely more exclusively on engineering data regarding the relationships between infrastructure systems (e.g. Min et al, 2007). Lastly, there is an emphasis on the implications of physical infrastructure damage impacts for society. 2.4.2 - Description The AIDRC approach consists of a series of linked, sequential phases: 1) Database of extreme events 2) Hazard scenario and background information 3) Expert interviews 4) Data synthesis 5) Workshop event for information-sharing among infrastructure system owners and operators These steps were worked through between 2006 and 2010 when the approach was applied using two different case studies set in the Metro Vancouver region. 18 Figure 2 - Overview of AIDRC Research Approach 1) Database of Extreme Events The first phase of the methodological approach is the collection of background information on IFIs that have occurred in various types of disaster events, including: blackouts, ice storms, floods, and earthquakes. A database was created based on newspaper reports from the region of impact for each of the extreme events and verified against other types of data sources, such as government reports. The database may be viewed at: http://www.chs.ubc.ca/dprc_koa/ 2) Hazard Scenario and Background Information The second phase of the approach is the creation of a basic hazard scenario. The hypothetical scenario serves as a realistic base case around which to frame discussion with participants. For the purposes of AIDRC project, two hypothetical scenarios were developed for the Metro Vancouver region (pop. 2,200,000): 1) an earthquake, and 2) a flood. The first scenario characterized an M7.3 earthquake with an epicentre location under the Georgia Strait, 18km southeast of Gibsons, resulting in strong to severe shaking in most of the study region. The second scenario characterized a flood on the lower Fraser River, breaching dykes and causing extensive flooding in Chilliwack, a community 100 km east of Vancouver with a concentration of 19 linear infrastructure assets (electrical transmission lines, rail, highway etc.). The scenarios can be viewed at: http://www.chs.ubc.ca/dprc_koa/practitioner_reports.html. Although they were not developed to be forecasts, the scenarios were intended to represent a realistic base case. Both scenarios were developed with and vetted by experts in the field. A one page description of each disaster event, accompanied by figures and maps (e.g. ground shaking intensity, flood extent), was developed. Additional information pertaining to IFIs experienced in earthquakes and floods in other places with similar development patterns and infrastructure to Canada was compiled. The scenario description along with the summary of other disasters serves as the starting point for interviews with infrastructure representatives. 3) Expert Interviews The third phase of the approach involves in-person interviews with experts representing a variety of infrastructure sectors: utilities (electric power, water, wastewater, natural gas); transportation (bridges and highways, public transit, airports, seaports); telecommunications; health care (regional health authorities, hospitals, private practitioners); and provincial, regional, and local governments. The objective of the interview phase is to collect information about the ability of infrastructures to withstand and recover from extreme events. The AIDRC project involved professionals working in infrastructure engineering, planning and emergency management. One set of interviews used the earthquake scenario and the process was repeated two years later using the flood scenario. Participants were presented with a hypothetical scenario as one possible future situation and asked to characterize infrastructure vulnerabilities and intersectoral interdependencies in the region. Interview questions focused on the infrastructures that each participant‟s organization relied on the most and how they may be affected by the disaster scenario, as well as how their own sector may perform in response to the event. Interviews provided an opportunity to learn about ways to reduce regional vulnerability to disasters due to interdependencies among infrastructures. 4) Data Synthesis The data synthesis step of the process involves analyzing the data from the interviews and developing a succinct means of communicating the findings. The AIDRC project created service disruption diagrams to visually represent the severity of expected service disruption for major infrastructure sectors in a region over time (0 hours, 72 hours, 2 weeks). The colour coding of the diagram is based on the estimated severity of disruption, using a scale developed in previous work by the researchers (Chang et al, 2005). Figure 3 depicts a sample of a service disruption 20 diagram, based on the impacts of the Red River Flood of 1997. Figure 4 illustrates the service disruption scale used to rate the levels of disruption. Figure 3 - Service Disruption Diagram Figure 4 - Service Disruption Scale Two dimensions are rated from low to high in order to determine the overall level of service disruption: spatial extent and severity of impact. The spatial area of the disruption constitutes the extent dimension of the scale. The impact represents both the severity of the consequences and duration of the disruption. For example, a disruption resulting in a single death would be classified as having a high impact but to a low extent, situating it in the moderate disruption category. Interdependency diagrams were also created to represent the relationships between various infrastructure sectors. The diagram can help stimulate thinking around how disruptions in one * Infrastructure disruptions of high impact and high extent are identified as severe disruptions * Disruptions of high impact/low extent or high extent/low impact are identified as moderate disruptions *Disruptions of low extent/low impact are identified as slight disruptions 21 sector may have implications on the infrastructures that rely on it. Figure 5 provides an example of an interdependency diagram using the earthquake data and illustrates the complex linkages that exist between various types of infrastructures. The arrows directed towards the transportation sector denote the sectors that are upstream from transportation (i.e. the sectors on which transportation is dependent). The arrows directed away from the transportation sector towards other infrastructures represent downstream dependencies (i.e. the sectors that depend on transportation). Figure 5 - Interdependency Diagram 5) Workshop Event The final phase of the process is a workshop event wherein the synthesized data from step 4 is used to facilitate discussion among infrastructure representatives (i.e. ideally the interview participants from step 3). The purpose of the workshop phase is to present the interview and 22 database findings to participants and attempt to develop a shared understanding of potential infrastructure failure interdependencies in the disaster scenario. Two workshops were convened during the course of the project: one to discuss possible earthquake related impacts and one focusing on the hypothetical flood. The grey box in Figure 2 outlines some of the key areas of discussion for the workshop events, including cross-sectoral expectations and regional concerns. Findings were presented by the research team, which was followed by facilitated discussion. Workbook exercises enabled participants to revise and augment the information presented in the service disruption and interdependency diagrams. The last phase of the approach enabled infrastructure representatives to learn from another and modify expectations where appropriate, thus forming a foundation for further discussion on potential mitigation efforts. 23 3. METHODS 3.1 - Steps of Evaluation As outlined by Rowe et al. (2008), the first step of an evaluation is to define success and the second is to development instruments with which to measure it. The following steps were used to conduct this evaluation. First, a solid understanding of the objectives of the research approach was ascertained. These objectives, in addition to select criteria from the literature, form the basis of the criteria for the evaluation. Second, once the evaluation criteria were established, performance measures were developed. Third, data was collected to measure the performance of the approach according to the criteria. 3.2 - Evaluation Criteria In order to answer the research question - Is the approach used by the AIDRC project effective? The concept of the term “effective” must first be defined. Evaluation criteria provide the means of defining success. As discussed in the description of the AIDRC approach (section 2.4), there are three primary concerns in the realm of disaster resilience research that the approach attempts to address: 1) incomplete incentives on the part of infrastructure owners and operators that do not completely address societal interests regarding IFIs; 2) partial or asymmetric information regarding vulnerabilities and consequences of IFIs 3) few opportunities for collective learning from direct experience Because this research approach aims to tackle these underlying issues, these three factors comprise appropriate criteria by which to evaluate the approach. In other words, does the approach achieve its desired objectives? Using the aims of the researchers to form part of the evaluation is consistent with Rowe et al.‟s recent study on developing a normative framework for evaluating public engagement exercises (2008). Evaluation criteria were also derived from the National Academy of Sciences (NAS), as previously discussed in the context section (section 2.2). Specifically, the criteria “capacity for future decision making” was selected for this evaluation as an indicator of a quality process (see Table 1). Figure 6 displays a means-ends network of the objectives and some suggested performance measures for this evaluation. The objectives have been separated into means objectives and 24 fundamental objectives. Means objectives are those which help us to achieve our other objectives, whereas fundamental objectives are important in themselves, reflecting what we ultimately want to achieve (Clemen and Reilly, 2001). Figure 6 - Fundamental Objectives Network for Evaluation of AIDRC Approach Therefore, for the purposes of this evaluation an “effective” approach is defined as one that1: 1) Achieves its own objectives 2) Fosters capacity for future decision making 3) Can be replicated in other geographic areas/contexts 4) Satisfies participants 3.3 - Data Sources Data for the evaluation were derived from three main sources: 1 As discussed in section 2.2, evaluation criteria were derived from Chang et al. (2010), Rowe (2008), and Dietz and Stern (2008). 25 1) Participant exit surveys from both the 2007 (earthquake) and the 2009 (flood) workshops 2) Participant workbook exercise responses from the 2007 and 2009 workshops 3) Notes and observations from the 2009 flood scenario expert interviews and workshop event 3.3.1 - Exit Surveys Exit surveys were completed by workshop participants at the 2007 earthquake scenario workshop and the 2009 flood scenario workshop. These brief surveys were intended to capture each participant‟s feedback on the value of the workshop, learning that may have occurred, and possible influence on organizational/institutional priorities for disaster preparedness and mitigation. The surveys used both open and closed ended questions. The closed ended questions were either categorical or Likert-scale. They were used to gauge such things as the degree to which participants felt better informed about the impacts of disasters infrastructure. The open- ended questions gave participants the opportunity to provide more specific feedback or elaborate on the closed ended questions. Appendix A contains the 2007 and 2009 exit survey questions. 3.3.2 - Workbook Exercises During the workshop, participants were presented with the results of the first three components of the research approach. Participants saw visual representations of the results and engaged in discussion about various aspects of the data, such as inconsistencies in expectations. Workbooks containing the visual representations of the results were given to participants and opportunities were given to revise or adjust these diagrams in accordance with new learning or perspectives. The workbook revisions provided a source of data for evaluating the approach. Workbook data was examined for trends towards greater consensus and the number of people who changed their answers relative to their interview responses. 3.3.3 – Interview and Workshop Notes Detailed notes from the interview process, and in particular the workshop, constituted an additional source of data for the evaluation process. During the workshop research assistants took detailed notes to document participant comments and group dialogue. These notes offer additional information regarding whether a shared understanding of the regional vulnerabilities to disasters was emerging as a result of the workshop. 26 4. RESULTS 4.1 - Participant Information There were 13 participants in attendance at the invitational workshop held in November of 2007, including a majority of those that had been interviewed in the earlier phase of the project. These participants represented a substantial proportion of major infrastructure organizations in the region. At the November 2009 workshop, there were 10 participants, many of which had been interviewed, as well as some additional participants who were not able to participate in the earlier phase. Relative to the 2007 workshop the 2009 workshop included representatives from fewer infrastructure sectors. This was intentional as the researchers selected the infrastructures they surmised may be most affected by the specific disaster scenario (i.e. the flood). Table 2 provides an overview of the range of infrastructure sectors and organizations represented at the two workshops. Table 2 - Critical Infrastructures Represented at the 2007 and 2009 2007 Workshop 2009 Workshop Infrastructure Sector Organization Organization Health Children‟s & Women‟s Hospital Provincial Health Services Authority Fraser Health Vancouver Coastal Health Transportation Ministry of Transportation --------------------- Vancouver Port Authority Telecommunications Telus --------------------- Power BC Hydro BC Hydro BC Transmission Corporation Municipal Government City of Richmond --------------------- North Shore Emergency Management Office Other Government Metro Vancouver – Water and Wastewater Metro Vancouver – Water and Wastewater Provincial Emergency Program (2 participants) Provincial Emergency Program Public Safety Canada Emergency Management BC Ministry of Community and Rural Development Natural Gas Terasen Gas Terasen Gas Non-Government Organizations -------------------- Fraser Basin Council 27 Note: (1) 14 participants were scheduled to attend the 2009 workshop, but 4 were unable to attend at the last minute for various reasons, including a potential flood event. They were representatives from the transportation, local government, 0ther government, and health sectors. 4.2 - Exit Survey Results – 2009 Workshop The first set of questions from the exit survey pertained to the degree to which participants became better informed about: a) the impacts of floods on your [the participant‟s] system, b) the impacts of floods on other systems, c) the consequences of infrastructure failure interdependencies, and d) regional vulnerabilities to disasters. Participants could select one of three answers: no better informed, somewhat better informed, or significantly better informed. This series of questions directly relates to the means objective of “participants feeling better informed” as a means of ultimately fostering capacity for future decisions (see Figure 6). The four histograms below display the responses of participants.2 Figure 7 - Impacts of Flood on Your System Slightly over half of participants reported that they were “somewhat better informed” about the impacts of floods on their system. Given that the representatives at the workshop are experts on their own system, it is not surprising that 4 participants indicated that they were “no better informed” about the impacts of floods on their system following the workshop. In contrast, all 2 Tests for statistical significance were not conducted in this study because of the limited number of participants (i.e. small n). 0 1 2 3 4 5 6 significantly better somewhat better no better N u m b e r o f P ar ti ci p an ts ( 9 ) Response Feel Better Informed of the Impacts of Floods on Your System? 28 participants agreed that they became “somewhat better informed” about the impacts of floods on other infrastructure systems (Figure 8 below). Figure 8 - Impacts of Flood on Other Systems Figure 9 shows that participants were almost unanimous with 9 out of 10 indicating that they were “somewhat better informed” about the consequences of IFIs following the workshop. Figure 9 - Consequences of IFIs The final question in this series focused on vulnerability to disaster from a regional perspective as opposed to a sector specific one. Again, nearly all participants responded that they became 0 2 4 6 8 10 12 significantly better somewhat better no better N u m b e r o f P ar ti ci p an ts ( 1 0 ) Response Feel Better Informed of the Impacts of Floods on Other Systems? 0 2 4 6 8 10 significantly better somewhat better no better N u m b e r o f P ar ti ci p an ts ( n =1 0 ) Response Feel Better Informed About the Consequences of IFI's? 29 “somewhat better informed” about regional vulnerabilities to disasters (Figure 10). Overall, it is evident that the majority of participants left the half-day workshop better informed about a number of issues relevant to the relationships between infrastructures and the vulnerability of the metro Vancouver region to disasters. Figure 10 - Regional Vulnerability to Disasters Participants were asked if they learned anything through the workshop that they felt would be helpful in setting their organization‟s priorities for disaster mitigation and preparedness. This question builds on those previously discussed in that it attempts to gather information to help determine whether the workshop fosters capacity for future decision making. 0 2 4 6 8 10 significantly better somewhat better no better N u m b e r o f P ar ti ci p an ts ( n =1 0 ) Response Feel Better Informed About Regional Vulnerabilities to Disasters? 30 Figure 11 - Organizational Priority Setting Almost all participants responded that they “learned a little” that would be helpful for them in setting organizational priorities, with one participant indicating that they learned a lot. Zero participants answered that they learned nothing through the workshop experience. This is consistent with the responses to the other survey questions. There were two questions intended to assess the confidence of participants. It is hypothesized that if participants feel better informed about these issues that they will also feel more confident in their knowledge and ultimately their ability to make decisions. The interview questions from the earlier stage of the study also asked the expert participants to rate their confidence in their responses. Participants were specifically asked about their confidence in their knowledge of IFIs and regional vulnerabilities to disaster. It is expected that participants will be confident in their knowledge of their individual infrastructure sector, but these questions attempt to focus on the regional perspective and the interactions among the various infrastructures. Figure 12 shows that just over half of participants self-reported that they were more confident in their knowledge of IFIs, with the rest remaining neutral. Similarly, half of participants agreed and half were neutral with the statement about confidence in their knowledge of regional vulnerabilities to disasters (Figure 13). 0 1 2 3 4 5 6 7 8 9 10 learned a lot learned a little learned nothing N u m b e r o f P ar ti ci p an ts ( 1 0 ) Response Learning for Organizational Priority Setting? 31 Figure 12 - Confidence in Knowledge of IFIs Figure 13 - Confidence in Knowledge of Regional Vulnerabilities to Disaster Workshop participants were asked if they felt that the workshop was a good use of their time. Although the workshop was an integral component of the research approach and data collection process, the intent of the process is to benefit planners and decision makers in this field. Therefore, it is valuable to know if participants felt that they benefitted from the experience. In an open-ended question, participants were asked to describe the most valuable aspects of the workshop. 0 1 2 3 4 5 6 7 strongly agree agree neutral disagree strongly disagree N u m b e r o f P ar ti ci p an ts ( 1 0 ) Response Feel More Confident in Knowledge of IFI's 0 1 2 3 4 5 6 strongly agree agree neutral disagree strongly disagree N u m b e r o f P ar ti ci p an ts ( 1 0 ) Response Feel More Confident in Knowledge of Regional Vulnerabilities to Disaster 32 Figure 14 - Time Well Spent All participants indicated that the workshop was time well spent. The majority of participants responded that the sharing of information along with the interaction with colleagues was the most valuable aspect of the workshop. One participant reported that it was a valuable opportunity to clarify assumptions and educate others about the vulnerabilities of their sector. Another participant enjoyed learning about the research conducted in this field. Lastly, the workshop catalyzed an interest in one participant to become better informed about the consequences of infrastructure loss in certain areas. Participants were asked four additional questions about the overall workshop experience. Two questions pertained to the workshop materials and information presented. The other two questions addressed opportunities for sharing and interacting with the group. These questions were answered on a 5-point Likert scale ranging from “strongly agree” to “strongly disagree.” 0 2 4 6 8 10 12 yes no N u m b e r o f P ar ti ci p an ts ( 1 0 ) Response Was this Workshop Time Well Spent? 33 Figure 15 - Balanced Presentation Materials Figure 16 - Accurate Presentation Materials Over three-quarters of the participants agreed that the workshop materials were balanced. Fewer participants indicated that they felt the workshop materials were accurate. Although no participants disagreed with the statements about workshop materials, there were some that remained neutral on the subject. In facilitating a process like that used by the AIDRC project, it is important that the facilitators are seen as independent and unbiased. Presenting accurate and balanced information fosters trust with participants. 0 1 2 3 4 5 6 7 8 9 strongly agree agree neutral disagree strongly disagree N u m b e r o f P ar ti ci p an ts ( 1 0 ) Response Materials Presented Were Balanced 0 1 2 3 4 5 6 7 strongly agree agree neutral disagree strongly disagree don't know N u m b e r o f P ar ti ci p an ts ( 1 0 ) Response Materials Presented Were Accurate 34 Figure 17 shows that participants either “agreed” or “strongly agreed” that they had adequate opportunities to share their ideas. Similarly, Figure 18 illustrates that all participants stated that they had adequate opportunities for interaction with other participants. These results speak to the means objective of fostering opportunities to strengthen cross-sectoral relationships. Figure 17 - Opportunities for Sharing Ideas Figure 18 - Opportunities for Interaction Participants were asked their opinion on how useful it would be to have greater collective efforts to try to manage infrastructure interdependencies, irrespective of whether efforts are initiated by the public or private sector. This question speaks to one of the main challenges to fostering infrastructure resilience because responding to IFIs requires sharing information about 0 1 2 3 4 5 6 strongly agree agree neutral disagree strongly disagree N u m b e r o f P ar ti ci p an ts ( 1 0 ) Response Adequate Opportunities to Share My Ideas 0 1 2 3 4 5 6 strongly agree agree neutral disagree strongly disagree N u m b e r o f P ar ti ci p an ts ( 1 0 ) Response Adequate Opportunities for Interaction with Participants 35 vulnerabilities and applying a broader perspective to disaster mitigation. All those who responded to this question indicated that greater collective efforts would be very useful in the management of infrastructure interdependencies. Figure 19 - Collective Efforts to Manage IFIs One of the final survey questions asked participants their opinion on how useful it would be to generalize and apply this approach to other communities or regions for the purposes of characterizing vulnerabilities to disasters. This question addresses one of the evaluation criteria, which suggests that an effective methodology could be applied in other geographical contexts or other contexts with complex interdependencies. 0 2 4 6 8 10 very useful somewhat useful not very useful N u m b e r o f P ar ti ci p an ts ( 9 ) Response Usefulness of Greater Collective Efforts to Try to Manage Infrastructure Interdependencies? 36 Figure 20 - Applicability of the Research Approach Figure 20 shows that all participants responded positively to this question. Half of participants indicated that they thought the approach would be very useful and the other half responded that the approach would be somewhat useful if applied to other communities or regions. One participant commented that “if you have a specific hazard, then you can only really accurately focus on a specific community at a time. If you try to be too generic, or too broad, you dilute your results due to the averaging process.” 4.3 - Exit Survey Results – 2007 Workshop The 2007 Exit Survey was structured differently than that for the 2009 workshop, although some questions were asked in both surveys. The 2007 survey included fewer questions overall and involved more open-ended questions requesting participants to elaborate or explain their answers. Participants were asked their opinion on whether the workshop was time well spent (see figure 21). Participants unanimously agreed that the workshop was time well spent. These responses are consistent with those on the equivalent question in the 2009 survey. Additional comments from participants indicated that it was a good opportunity to exchange ideas and build relationships with a diverse group. 0 1 2 3 4 5 6 very useful somewhat useful not very useful N u m b e r o f P ar ti ci p an ts ( 1 0 ) Response Applicability of Approach to Other Communities or Regions? 37 Figure 21 - Time Well Spent (2007) Participants were then asked if the workshop met their expectations for an event on this topic. In addition to the question on whether the workshop was time well spent, this question serves as an indicator of participant satisfaction. Figure 22 shows that all participants responded positively with 75% answering yes and the remaining indicating that their expectations were somewhat met. Some participants commented that the workshop exceeded their expectations, while others wrote that they had hoped for more of an outcome or a plan to translate the ideas into action. Figure 22 - Meet Expectations (2007) 0 2 4 6 8 10 12 14 yes no N u m b e r o f P ar ti ci p an ts ( 1 2 ) Response Time Well Spent? 0 1 2 3 4 5 6 7 8 9 10 yes somewhat no N u m b e r o f P ar ti ci p an ts ( 1 2 ) Response Meet Expectations 38 Similar to the 2009 survey, participants were asked if they learned anything at the workshop that should influence their organization‟s priorities for disaster mitigation and preparedness. Ten out of twelve participants responded that they learned things that should influence their priorities for mitigation. Although the two remaining participants responded somewhat in the negative, their comments revealed that the workshop reinforced or confirmed their existing priorities. Another participant commented that they gained a new perspective on options to be considered (see figure 23). Figure 23 - Learning for Organizational Priority Setting (2007) The final question of the exit survey asked participants to comment on the aspects of the workshop that they found most valuable and why. Almost all responses touched upon the quality of interactions and dialogue with other sector experts. The following comments are a representation of the feedback that was received: “Great to interact in a facilitated thing-tank environment with other sector experts.” “Discussions on fuel and water supplies. I find it interesting to hear what is important to each sector represented.” “The group selected was very diverse. The agenda was appropriate, good timescale for each component and adequate discussion time.” “Increasing dialogue on interdependencies is also valuable the exchange of stating my dependencies and hearing others expectations and dependencies on me is important in planning, mitigation, and response activities.” 0 2 4 6 8 10 12 yes yes + no no N u m b e r o f P ar ti ci p an ts ( 1 2 ) Response Learn Anything that Should Influence Organizational Priorities? 39 “The interagency discussion resulting in multiple perspectives-especially in a small group of „people in the know‟” “The continuation of dialog and updates [is most valuable.] [T]his is a topic [in] which we need to make more efforts and preparation.” “Discussion of a small group has been very frank and have a better understanding of issues confronting CI partners. This was an excellent forum. Thanks for taking me out of my usual \"practicality\" driven environment to allow free-thinking. Could do similar sessions with other hazards.” 4.4 - Workbook Exercises -2007 & 2009 An additional source of data for the evaluations is the participant responses to the workbook exercises. Following presentations and discussions about infrastructure interdependencies, expectations of service disruption as a result of the disaster as well as highlighting of discrepancies in expectations, participants had the opportunity to review the interdependency and service disruptions diagrams in their workbooks and revise components of them as they saw appropriate. Adjustments to the diagrams are viewed as one indicator of collaborative learning and the development of a shared understanding of regional vulnerabilities. Over half (n=7) of the workshop participants made revisions to the diagrams in their workbooks. Figures 24 and 25 display the overall changes made to the infrastructure interdependencies diagram, based on participant responses. The most significant changes made to this diagram related to the expected level of service disruption. The changes in colour coding show that the expectations for wastewater and water were downgraded to “no loss” from their original “moderate” and “slight” estimates. The original estimates for these sectors were based on an assumption of power failure. When it was revealed through the workshop discussions that the power sector did not anticipate service disruption to the lower mainland as a result of the hypothetical scenario, the estimates were revised accordingly. Another modification was made regarding the relationship between health care, specifically the Provincial Health Services Authority, on the natural gas sector. The arrow indicating that health care has a moderate dependency on natural gas was removed. 40 Figure 24 - Infrastructure Interdependencies Diagram - 2009 Flood - Original Pre Workshop 41 Figure 25 - Infrastructure Interdependencies Diagram - 2009 Flood - Revised Post Workshop Participants also made revisions to the diagram showing infrastructure service disruptions following the hypothetical flood event. Figures 26 and 27 represent the original and the post- workshop revised versions of the service disruption diagrams. Following the discussion and clarification of discrepancies in expectations, participants tended to downgrade their expectations for disruption (e.g. water and wastewater). By the end of the workshop, the only sector expecting a disruption in service was health care, which reported moderate disruption in this scenario. 42 Figure 26 - Infrastructure Service Disruption Diagram - 2009 Flood - Original Pre Workshop No Loss No service disruptions Slight Disruption Low spatial extent and low impact disruptions Moderate Disruption Low spatial extent & high impact or high spatial extent & low impact disruptions Severe Disruption High spatial extent & high impact disruptions Sector Time After Event 0 Hours 72 Hours 2 weeks Power Transportation Water Wastewater Natural Gas Healthcare Solid Waste This table is a visual representation of expected service disruptions by sector at specific time points following the flood scenario. The designations are an aggregation of data from a number of sources, including interviews with sector experts, past flood events, and research findings. The designations are not a forecast of actual service function following a flood; they represent a reasonable expectation of a possible outcome for this particular flood. 43 Figure 27 - Infrastructure Service Disruption Diagram - 2009 Flood - Revised Post Workshop Nearly all of the participants (n=10) in the 2007 earthquake workshop chose to make modifications to at least one of the diagrams in the workbook. Figures 28 and 29 represent the original and post-workshop diagram of infrastructure interdependencies. The discussion concerning interdependencies resulted in some changes to participants‟ perceptions of the relationships between infrastructures. When comparing the two diagrams, it appears that participants tended to upgrade the level of dependency between some sectors (lines 5, 14, 19, 20, 21, 24) from slight to moderate or moderate to severe. In addition, two additional dependencies were identified (lines 27 & 28) that were not part of the original diagram. Participants acknowledged that many dependencies exist and due to this high degree of connectivity, all sectors would likely be adversely affected by service disruptions to any other infrastructure sector. No Loss No service disruptions Slight Disruption Low spatial extent and low impact disruptions Moderate Disruption Low spatial extent & high impact or high spatial extent & low impact disruptions Severe Disruption High spatial extent & high impact disruptions Sector Time After Event 0 Hours 72 Hours 2 weeks Power Transportation Water Wastewater Natural Gas Healthcare Solid Waste 44 Figure 28 - Infrastructure Interdependencies Diagram - 2007 Earthquake - Original Pre Workshop 45 Figure 29 - Infrastructure Interdependencies Diagram - 2007 Earthquake - Revised Post Workshop Adjustments were also made to the expected service disruption diagrams. When comparing Figures 30 and 31, there seems to be a trend to moving towards higher expectations of service disruption. Following group discussions, most sectors reported that they would experience significant service disruption in the immediate aftermath of an earthquake event of this magnitude, which would exacerbate existing dependencies. 46 Figure 30 - Infrastructure Service Disruption Diagram - 2007 Earthquake - Original Pre Workshop Sector Time After Event 0 Hours 72 Hours 2 weeks Power Communication Water Transportation (Intraregional) Transportation (Interregional) Healthcare Government Natural Gas Wastewater No Loss No service disruptions Slight Disruption Low spatial extent and low impact disruptions Moderate Disruption Low spatial extent & high impact or high spatial extent & low impact disruptions Severe Disruption High spatial extent & high impact disruptions This table is a visual representation of expected service disruptions by sector at specific time points following the scenario earthquake. The designations are an aggregation of data from a number of sources, including interviews with sector experts, past earthquake events, and research findings. The designations are not a forecast of actual service function following an earthquake; they represent a reasonable expectation of a possible outcome for this particular earthquake. The high number of severe disruptions reflects the fact that many infrastructure systems may be overwhelmed by volume, such as telecom, or may have to shut down to due to external factors such as debris, such as road systems. 47 Figure 31 - Infrastructure Service Disruption Diagram - 2007 Earthquake - Revised Post Workshop The workbook exercises for both workshops revealed that the majority of participants altered their expectations for disruption in the hypothetical scenario. Furthermore, there is evidence to suggest that participants developed a more comprehensive understanding of the interdependencies among the various infrastructure sectors in the region. Sector Time After Event 0 Hours 72 Hours 2 weeks Power Communication Water Transportation (Intraregional) Transportation (Interregional) Healthcare Government Natural Gas Wastewater No Loss No service disruptions Slight Disruption Low spatial extent and low impact disruptions Moderate Disruption Low spatial extent & high impact or high spatial extent & low impact disruptions Severe Disruption High spatial extent & high impact disruptions 48 5. ANALYSIS & DISCUSSION The exit survey and workbook results presented in the previous section provide data for assessing the performance of the AIDRC approach relative to the evaluation criteria specified in section 3.2. In addition to a discussion on the performance of the methodology, this section will also describe some of the complexities of executing this type of approach and the challenges of evaluating a planning process. 5.1 - Performance on Evaluation Criteria 5.1.1 - Does the Analyzing Infrastructures for Disaster Resilient Communities approach achieve its own objectives? In developing the approach for the AIDRC project, the researchers attempted to address three main challenges to fostering infrastructure resilience: 1) incomplete incentives on the part of infrastructure owners and operators that do not completely address societal interests regarding IFIs; 2) partial or asymmetric information regarding vulnerabilities and consequences of IFIs 3) few opportunities for collective learning from direct experience The first challenge appears to be quite difficult to overcome in that infrastructure managers‟ main objectives in a disaster event are to manage damage to their own system, minimize revenue loss, and maintain the organization‟s reputation (Chang et al., 2010). Incentives do not exist to encourage infrastructure providers to also consider in planning or decision making that disruption to their system may cascade through other dependent infrastructures. It is difficult to identify a governance system that can encourage providers to address broader societal interests because management of infrastructure systems occurs at different scales. For example water and wastewater are managed at the regional scale and electric power is managed on a provincial scale. Furthermore, while some infrastructures are managed by the public sector, others are managed through Crown corporations at arm‟s length from the government and others still are provided through the private sector. The AIDRC methodology for characterizing regional vulnerabilities to disaster does not change the incentive structure for infrastructure managers. However, the approach enables infrastructure representatives to develop a better understanding of the connectivity between the region‟s infrastructure systems and the importance of managing interdependencies among them. Participants realized through the discussion of the earthquake scenario that because of 49 the complexity of the relationships among the various sectors, all of the region‟s infrastructures could be adversely affected if there are service disruptions to any other sector. Results from the exit surveys also revealed that the majority of participants learned things through the workshop that would be helpful for setting organizational priorities. These findings suggest that the AIDRC approach may be useful in addressing this first obstacle to regional infrastructure resilience. The second obstacle the AIDRC approach aims to address is incomplete information regarding the vulnerabilities and consequences of IFIs. The results of the exit survey and workbook exercises along with observations from the workshop confirm that this approach is effective in overcoming this obstacle. First, most of the participants responded that they feel better informed about the consequences of IFIs as well as about regional vulnerabilities to disasters. Furthermore, half of the participants indicated that they felt more confident in their knowledge of the consequences of IFIs and regional vulnerabilities to disasters. In addition to participant self-assessment, discussion at the workshop and modifications to the interdependency and service disruption diagrams indicate that information about IFIs and vulnerabilities in the context of the disaster scenarios became more complete. Participants were presented with slides revealing the discrepancies in service disruption expectations from the interview data. This opened up the opportunity for participants to clarify assumptions and uncertainties as a group. For example, during the flood workshop it was revealed that the power sector did not expect to have any service loss as a result of this event. The representative from the water and wastewater sector mentioned that the service disruption estimates given in that sector‟s interview were based on an expectation that there would be some level of power disruption. Once this misconception was clarified, the water and wastewater sector disruption estimates were downgraded accordingly. Also at the flood workshop, discussion concentrated on some uncertainty around possible disruption to land transportation. Because the transportation representative was unable to attend the workshop, a range of possibilities based on different assumptions and potential outcomes were discussed. This dialogue enabled participants to explore concepts of thresholds and tipping points that could turn what was identified as a minor disaster to one of larger consequence. This type of group deliberation enabled participants to expand their understanding of the vulnerabilities in the region and the consequences of IFIs.3 3 After the workshop, the research team followed up with key contacts to further explore the remaining transportation questions. These findings were reported back to participants in a workshop summary report. 50 The final obstacle to infrastructure resilience identified by the AIDRC research project is that there are few opportunities for infrastructure managers to collectively learn about these issues. Through the use of a hypothetical disaster scenario, vetted by experts to be one possible future event, the AIDRC project effectively created an opportunity for collective learning. The hypothetical scenario gave participants from a range of the region‟s infrastructure sectors a common starting place to share their knowledge and experience. This observation is supported by the exit survey responses and workbook exercise results as well. As previously mentioned, participants answered that they felt better informed about the impacts of flood on other infrastructures as well as the consequences of IFIs. The fact that participants made modifications to the diagrams following the group discussion further indicates that there was an opportunity to learn collectively. 5.1.2 - Does the Analyzing Infrastructures for Disaster Resilient Communities approach foster capacity for future decision making? The data collected for this evaluation show that the AIDRC approach fosters capacity for future decision making. Based on the means ends network presented in Figure 6, there are four identified means for fostering future decision making capacity: foster opportunities for collaborative learning, foster availability of information on vulnerabilities to disaster and consequences of IFIs, foster opportunities to strengthen cross-sectoral relationships, and foster participant‟s perception of feeling better informed. The research approach performs well on the objective of collaborative learning. As discussed in the previous section, the workshop provided an opportunity for infrastructure representatives to meet face to face and dialogue about their assumptions, expectations, vulnerabilities and priorities. One of the final questions in the interview script asked participants if they would like to revise any of their answers. None of the participants made revisions to their original answers in the interview. However, the workbook exercises at the end of the workshop gave participants the opportunity to make modifications to the diagrams and the vast majority made revisions (earthquake workshop n=10 of 12, flood workshop n=7 of 10). These findings suggest that through the facilitated group discussion participants were exposed to new ideas that altered their understanding of the issues. Hence, collective learning occurred. In addition, the survey results showed that the majority of participants felt they learned things at the workshop that could help with organizational priority setting. 51 It is hypothesized that fostering the availability of information on vulnerabilities to disaster and the consequences of IFIs will improve the capacity of infrastructure managers to make decisions. The AIDRC approach generates information directly relevant for infrastructure managers through a number of means: the presentation of data on IFIs in other disaster events, such as the Red River flood and the Kobe earthquake; the facilitated discussion around discrepancies in the service disruption expectations of infrastructure managers; and the dialogue on interdependencies among infrastructures. Furthermore, this information is made available through reports and a searchable database publicly available on the project website. As mentioned in the previous section, almost all participants responded that they felt better informed about the consequence of IFIs as well as about regional vulnerabilities to disaster. These results support the argument that the AIDRC approach fosters the availability of information on these issues. Given that the issue of infrastructure resilience involves many stakeholders and a great deal of complexity, strong cross-sectoral relationships are thought to be valuable in supporting regionally beneficial decisions. The findings from the workshop exit surveys strongly support the notion that the AIDRC approach helps to strengthen relationships among infrastructure representatives. Every participant responded positively to the survey questions about opportunities to share ideas with the group and opportunities to interact with the group. Furthermore, additional comments provided by participants emphasized that the opportunity to interact in the “think-tank” environment was one of the most valuable aspects of the exercise. The final means identified for supporting future decision making capacity is fostering participants perception of feeling better informed. The results presented suggest that the workshop component of the approach resulted in most participants feeling at least somewhat better informed about the impacts of floods on other infrastructures, the consequences of IFIs, and regional vulnerabilities to disasters. It is important to note that participants may have varying levels of experience and knowledge on these issues, so some people may benefit more from the information presented while others may find more benefit in other facets of the exercise. 5.1.3 - Could the Analyzing Infrastructures for Disaster Resilient Communities approach be replicated in other geographic areas or contexts? While the AIDRC approach was applied with two different scenarios to the metro Vancouver region, an effective methodology should have the capacity to be applied in other contexts. There 52 was only one question in the survey that addressed this facet of the evaluation. However, all of the participants at the flood workshop responded that it would be either “somewhat useful” or “very useful” to generalize and apply this approach to other communities or regions for the purposes of characterizing vulnerabilities to disasters. The approach is comprised of a logical sequence of steps with clear objectives that should translate well to other municipalities or urban regions. It would be beneficial to apply this approach in another setting as a case study in order to fully assess its transferability. 5.1.4 - Are participants generally satisfied with the Analyzing Infrastructures for Disaster Resilient Communities approach? The final evaluation criterion pertains to participant satisfaction with the approach. Given that the approach is designed to overcome the obstacles to infrastructure resilience experienced by infrastructure managers and planners, the approach should be viewed positively by the participants. The survey data indicate the AIDRC project experience was positive. Workshop participants unanimously agreed that the workshop was time well spent. In addition, the majority of participants at the earthquake workshop responded that the workshop met their expectations. As presented in the results section, many participants provided very positive feedback on their experience at the workshop. In fact, many of the participants at the flood workshop in 2009 were also involved in the 2007 earthquake workshop. This indicates that they derived enough benefit from the first workshop to attend the second event. Overall, this analysis demonstrates that the AIDRC approach performed well on each of the evaluation criteria. Therefore, it can be concluded that the AIDRC approach is an effective means to characterize regional vulnerabilities to disasters. 5.2 - Challenges and Limitations of the Analyzing Infrastructures for Disaster Resilient Communities Approach Although the AIDRC approach met the evaluation criteria there are additional challenges and limitations to consider for the future. First, the approach uses a single hypothetical scenario from which to gather data and engage with workshop participants. While this is helpful to focus the attention of participants, it is difficult to know if a different scenario, either a more extreme event or a different hazard, would elicit different perspectives on the nature of the interdependencies between infrastructure sectors. When comparing the flood and earthquake processes, the earthquake scenario appeared to highlight the connectivity and the ramifications 53 of IFIs more than the flood scenario. However, the types of dependencies discussed in each workshop were similar (e.g. reliance on power). Second, only one representative from each organization was interviewed. While efforts were made to interview the most knowledgeable representative of the organization, it is difficult to determine whether a second representative would provide similar data. When possible, there was an attempt to have the interviewed representative participate in the workshop as well. However, if that individual was unavailable, another representative would attend. The knowledge and experience that participants bring to the workshop is important given the stated value of the dialogue among infrastructure experts at the workshop. One of the important practical considerations for future application of this approach is the question of who initiates and coordinates such a process. The coordinating body should be impartial and trustworthy. Academic researchers are a logical fit, but resources are required to coordinate this process. Participants expressed interest in the continued involvement of academia in these types of practical issues. Regional planning committees that work on issues of disaster mitigation may be suitable for the role of coordination. Based on the responses and positive feedback of participants, the need for an organization to connect infrastructure managers and share information would be of value in fostering regional resilience. 5.3 - Challenges and Limitations of the Evaluation The evaluation of planning processes involves some challenges. There is not a standard set of evaluation criteria and performance measures to apply. As a result, evaluations can be quite subjective. This evaluation attempted to objectively define what constitutes an “effective” approach in this circumstance and use these criteria as a framework for assessment. The evaluation is relatively small-scale and involved a limited number of participants, offering limited possibilities for statistical testing of results. In defining the evaluation criteria it was important to create criteria that were measurable. Due to the difficulty in measuring some types of outcomes the evaluation was therefore limited. For instance, it would have been valuable to know if any organization‟s priorities for disaster mitigation actually changed following the workshop. A more comprehensive evaluation could have included a follow-up interview or survey of participants six months after the workshop. Designing the survey instrument required balancing several objectives. It was desirable to 54 maximize the amount of information gathered to inform the evaluation. However, it was also important to ensure the survey could be completed quickly and painlessly. The participants had already volunteered their time to be interviewed and attend the workshop so acknowledging their contribution and respecting their time was essential. Thus, while it would have been desirable to have apply a before and after approach or have additional follow-up, the evaluation approach ultimately employed was determined in part by practical constraints. 55 6. CONCLUSION Given that a central component of disaster resilient communities is resilient infrastructure systems, the AIDRC approach has important implications for planning practice. The AIDRC methodology was developed to overcome some of the key challenges to fostering infrastructure resilience and this evaluation has provided data to support its usefulness in this regard. In particular, this project has demonstrated the importance and effectiveness of creating collaborative learning opportunities with infrastructure managers. In summary, the AIDRC project attempted to develop a practical approach for characterizing infrastructure vulnerabilities and resilience. The intent of the approach is to aid infrastructure managers and planners in efforts to foster resilience in urban regions. The purpose of this evaluation was to assess the overall effectiveness of the AIDRC approach. The findings from this project indicate that the AIDRC approach fulfilled the objectives selected for the evaluation: achieve its own objectives, fosters capacity for future decision making, replicable in other geographic areas/context, and satisfies participants. Therefore, it is concluded that the AIDRC methodology is a useful means for characterizing regional vulnerabilities and supporting efforts to increase the disaster resilience of communities. 56 7. REFERENCES Analyzing Infrastructures for Disaster-Resilient Communities. (2010). Glossary. Retrieved from http://www.chs.ubc.ca/dprc_koa/glossary.html#r Arvai, J.L., Gregory, R., and McDaniels, T.L. (2001). Testing a structured decision approach: Thinking for deliberative risk communication. Risk Analysis, 21(6), 1065-1076. Burby, R.J., Beatley, T., Berke, P.R., Deyle, R.E., French, S.P., Godschalk, D.R., Kaiser, E.J., Kartez, J.D., May, P.J., Olshansky, R., Paterson, R.G. and Platt, R.H. (1999). Unleashing the Power of Planning to Create Disaster-Resilient Communities. Journal of the American Planning Association, 65(3), 247-258. Chang, S.E., McDaniels, T.L., Fox, J., Dhariwhal, R. and Longstaff, H. (2010). Towards Disaster- Resilient Cities: Characterizing Resilience of Infrastructure Systems with Expert Judgments. (working paper). Chang, S.E., McDaniels, T.L., Beaubien, C. (2009). Societal impacts of infrastructure failure interdependencies: Building an empirical knowledge base. ASCE Conf. Proc., 357(66). DOI:10.1061/41050(357)66 Chang, S.E., McDaniels, T.L., Mikawoz, J., and Peterson, K. (2007). Infrastructure failure interdependencies in extreme events: Power outage consequences in the 1998 Ice Storm. Natural Hazards, 41, 337-358. Chang, S., McDaniels T., Reed, D., and Peterson, K. (2005). Mitigation of extreme event risks: Electric power and infrastructure failure interactions. In Richardson, H., Gordon, P. and Moore, J. (Eds.), The economic impacts of terrorist attacks (pp. 70-90). Northampton, MA: Edward Elgar Publishing. Chang, S.E. (2003). Evaluating disaster mitigations: Methodology for urban infrastructure systems. Natural Hazards Review 4(4), 186-196. 57 Chelimsky, E., Shadish, W. (Eds.). (1997). Evaluation for the 21st Century. Thousand Oaks, CA: Sage. Chess, C. (2000). Evaluation environmental public participation: Methodological questions. Journal of Environmental Planning and Management, 43(6), 769-784. Chou, C. and Tseng, S. (2010). Collection and analysis of critical infrastructures interdependency relationships. Journal of Computing in Civil Engineering. doi:10.1061/(ASCE)CP.1943-5487.0000059 Clemen, R.T., Reilly, T. (2001). Making hard decisions with decision tools. Belmont, CA: Duxbury Press. Dietz, T., and Stern, P.C. (Eds.). (2008). Public participation in environmental assessment and decision making: Panel on public participation in environmental assessment and decision making. National Research Council of the National Academies. Washington, D.C.: National Academies Press. Fisheries and Oceans Canada. (2004). Consultation framework for Fisheries and Oceans Canada. Retrieved from http://www.cen- rce.org/eng/caucuses/water/docs/DFO%20Consultation%20Publications/Consultation%20Fra mework%20for%20Fisheries%20&%20Oceans%20Canada%20March%202004.pdf Godschalk, D. R. (2003). Urban hazard mitigation: Creating resilient cities. Natural Hazards Review, 4(3), 136-143. Government of Canada. (2004). Government of Canada position paper on a national strategy for critical infrastructure protection. Retrieved from http://www.acpa- ports.net/advocacy/pdfs/nscip_e.pdf Health Canada, Corporate Consultation Secretariat, Health Policy and Communications Branch. (2000). Health Canada Policy Toolkit for Public Involvement in Decision Making. No.: H39- 549/2000E. Retrieved from http://www.hc-sc.gc.ca/ahc-asc/pubs/_public- consult/2000decision/ 58 Helmer, M. and Hilhorst, D. (2006). Natural disasters and climate change. Disasters, 30(1), 1-4. House, E. (1980). Evaluating with Validity. Sage Publications. International Federation of Red Cross Red Crescent Societies. (2009). World disaster report 2009. Retrieved from http://www.redcross.ca/article.asp?id=31985&tid=001 Longstaff, H., and Burgess, M. (2010). Recruiting for representation in public deliberation on the ethics of biobanks. Public Understanding of Science, 19(2), 212-224. McDaniels, T.L. Chang, S., Peterson, K., Mikawoz, J., and Reed, D. (2007). An Empirical Framework for Characterizing Infrastructure Failure Interdependencies. Journal of Infrastructure Systems, 13(3), 175-184. McDaniels, T.L., Gregory, R., Arvai, J., and Chuenpagdee, R. (2003). Decision structuring to alleviate embedding in environmental valuation. Ecological Economics, 46, 33-46. Min, H.-S. J., W. Beyeler, T. Brown, Y.J. Son, and A.T. Jones. 2007. \"Toward modeling and simulation of critical national infrastructure interdependencies,\" IIE Transactions 39: 57-71. Moor, J. (2001). Cities at risk. Habitat Debate, 7(4), 1–6. O‟Rourke, T.D. (2007). Critical infrastructures, interdependencies, and resilience. The Bridge, 37(1), 22-29. Patton, C. and Sawicki, D. (1993). Basic methods of policy analysis and planning (2nd ed.). Upper Saddle River, NJ: Prentice Hall. Peerenboom, J.P., Fisher, R.E., Rinaldi, S.M., and Kelly, T.K. (2002, January-February). Studying the chain reaction. Electric Perspectives, 1-9. Renn, O. (1999). A model for an analytic-deliberative process in risk management. Environ. Sci. Technol., 33(18), 3049-3055. 59 Rinaldi, S. M., Peerenboom, J. P. and Kelly, T. K. (2001). Identifying, understanding, and analyzing critical infrastructure interdependencies. IEEE Control Systems Magazine, 2001, 11- 25. Rowe, G., Horlick-Jones, T., Walls, J., Poortinga, W., and Pidgeon, N.F. (2008). Analysis of a normative framework for evaluating public engagement exercises: Reliability, validity and limitations. Public Understanding of Science, 17, 419-441. Rowe, G., Marsh, R., and Frewer, L.J. (2004). Evaluation of a deliberative conference. Science, Technology & Human Values, 29(1), 88-121. Rowe, G., and Frewer, L.J. (2000). A framework for evaluation. Science, Technology & Human Values, 25(1), 3-29. Seasons, M. 2003. Monitoring and evaluation in municipal planning. Journal of the American Planning Association 69(4):430-40. Stern, P.C., and Fineberg, V. (1996). Understanding risk: Informing decisions in a democratic society. National Research Council, Committee on Risk Characterization. Washington, D.C.: National Academy Press. United Nations Population Fund (cited as UNFPA). Linking population, poverty and development. Retrieved from http://www.unfpa.org/pds/urbanization.htm van Aalst, M.K. (2006). The impacts of climate change on the risk of natural disasters. Disasters, 30(1), 5-18. Weiss, C. (1998). Evaluation: Methods for studying programs and policies (2nd ed.). Upper Saddle River, NJ: Prentice Hall. World Bank. (2004). Natural disasters: Counting the cost. Retrieved from http://web.worldbank.org/WBSITE/EXTERNAL/NEWS/0,,contentMDK:20169861~menuPK:3 4458~pagePK:64003015~piPK:64003012~theSitePK:4607,00.html 60 APPENDIX A 2009 - Workbook Exit Survey 1) Please use the table below to answer the series of questions. Following this workshop, to what degree do you feel better informed about: 2) In your opinion, was this workshop time well spent? YES NO If yes, what aspects of the workshop were most valuable? _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ 1 2 3 No better informed Somewhat better informed Significantly better informed a) the impacts of floods on your system (e.g. government, health care)? b) the impacts of floods on other systems? c) the consequences of infrastructure failure interdependencies? d) regional vulnerabilities to disasters? Workbook Exercise #4 - Exit survey Instructions: We would greatly appreciate it if you could take a few minutes to complete this survey about today’s workshop. Your feedback will help us in the development of future research and workshops. 61 3) Please use the table below to respond to the following statements. 1 2 3 4 5 Strongly disagree Disagree Neutral Agree Strongly agree a) The materials presented by the team were balanced in terms of content. b) The materials presented by the team were accurate. c) There were adequate opportunities to share my ideas with the group. d) There were adequate opportunities for me to interact with other participants. e) Following this workshop, I feel more confident in my knowledge of infrastructure failure interdependencies. f) Following this workshop, I feel more confident in my knowledge of regional vulnerabilities to disasters. 4) 1 2 3 Learned nothing Learned a little Learned a lot How much did you learn through this workshop that would be helpful in setting your organization’s priorities for disaster mitigation and preparedness? Please describe the most important insights you gained in relation to your priorities for disaster mitigation and preparedness. _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ 62 _____________________________________________________________________________________ __________________________________________________ 5) This workshop is one component of an approach developed to characterize regional vulnerabilities to infrastructure failures in disasters. The approach consists of four linked phases: (1) hazard scenario and background information; (2) expert interviews; (3) data synthesis; and (4) a workshop event for information- sharing among infrastructure system owners and operators. 6) Is there any information that you provided today that you would NOT like us to share in forthcoming papers, reports, or presentations? YES NO Please use the space below to explain your response. _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ __________________________________________________ Please return your workbook with signed consent form to one of the student note takers before you leave. Thank you very much for your participation! 1 2 Somewhat useful 3 Not very useful Very useful In your opinion, how useful would it be to generalize and apply this approach to other communities or regions for the purposes of characterizing vulnerabilities to disasters? In your opinion, how useful would it be to have greater collective efforts to try to manage infrastructure interdependencies? (irrespective of whether efforts are initiated by the public or private sector) 63 2007 – Workbook Exit Survey 1) In your opinion, was this workshop time well spent? YES NO Please use the space below to explain your response. _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ __________________________________________________ 2) Did we achieve our stated objectives? YES NO Please use the space below to explain your response. _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ __________________________________________________ 3) After participating in this workshop, are you satisfied with your organization's priorities for disaster mitigation and preparedness? YES NO Please use the space below to explain your response. _____________________________________________________________________________________ _____________________________________________________________________________________ Workbook exercise #5. Exit survey Instructions: We would appreciate it if you could take a few moments to answer the following questions before you leave for the day. 64 _____________________________________________________________________________________ _____________________________________________________________________________________ __________________________________________________ 4) As a result of this workshop, do you think your organization should reconsider its disaster mitigation and preparedness priorities? YES NO Please use the space below to explain your response. _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ __________________________________________________ 5) What aspects of this workshop did you find most valuable, and why Please use the space below to explain your response. _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ __________________________________________________ 6) Is there any information that you provided today that you would NOT like us to share in forthcoming papers, reports, or presentations? YES NO Please use the space below to explain your response. _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ __________________________________________________ "@en ; edm:hasType "Graduating Project"@en ; edm:isShownAt "10.14288/1.0102513"@en ; dcterms:language "eng"@en ; ns0:peerReviewStatus "Unreviewed"@en ; edm:provider "Vancouver : University of British Columbia Library"@en ; dcterms:rights "Attribution-NonCommercial-NoDerivatives 4.0 International"@en ; ns0:rightsURI "http://creativecommons.org/licenses/by-nc-nd/4.0/"@en ; ns0:scholarLevel "Graduate"@en ; dcterms:isPartOf "University of British Columbia. SCARP Graduating Projects"@en ; dcterms:subject "Disaster resilience"@en ; dcterms:title "Evaluating a methodology for characterizing resilience of infrastructure systems"@en ; dcterms:type "Text"@en ; ns0:identifierURI "http://hdl.handle.net/2429/32605"@en .