Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Drinking water management and governance in small drinking water systems : integrating continuous performance… Bereskie, Ty Anthony 2017

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2017_may_Bereskie_Ty.pdf [ 6.01MB ]
Metadata
JSON: 24-1.0347281.json
JSON-LD: 24-1.0347281-ld.json
RDF/XML (Pretty): 24-1.0347281-rdf.xml
RDF/JSON: 24-1.0347281-rdf.json
Turtle: 24-1.0347281-turtle.txt
N-Triples: 24-1.0347281-rdf-ntriples.txt
Original Record: 24-1.0347281-source.json
Full Text
24-1.0347281-fulltext.txt
Citation
24-1.0347281.ris

Full Text

DRINKING WATER MANAGEMENT AND GOVERNANCE IN SMALL DRINKING WATER SYSTEMS: INTEGRATING CONTINUOUS PERFORMANCE IMPROVEMENT AND RISK-BASED BENCHMARKING   by Ty Anthony Bereskie M.S., Colorado School of Mines, 2013  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  DOCTOR OF PHILOSOPHY in The College of Graduate Studies (Civil Engineering)  THE UNIVERSITY OF BRITISH COLUMBIA (Okanagan)  April 2017  © Ty Anthony Bereskie, 2017   ii  Examination Committee The undersigned certify that they have read, and recommend to the College of Graduate Studies for acceptance, a thesis entitled:  Drinking Water Management and Governance in Small Drinking Water Systems: Integrating Continuous Performance Improvement and Risk-based Benchmarking   Submitted by  Ty Anthony Bereskie  in partial fulfillment of the requirements of  the degree of  Doctor of Philosophy .   Dr. Rehan Sadiq, School of Engineering / University of British Columbia Okanagan   Supervisor, Professor and Associate Dean    Dr. Manuel J. Rodriguez, ESAD / Université Laval  Co-Supervisor and Committee Member, Professor    Dr. Kasun Hewage, School of Engineering / University of British Columbia Okanagan  Supervisory Committee Member, Associate Professor    Dr. Thomas Johnson, School of Engineering / University of British Columbia Okanagan  University Examiner, Professor    Dr. Jianbing Li, Environmental Science and Engineering Program, University of Northern British Columbia  External Examiner, Professor and Co-Director – UNBC/UBC Environmental Engineering Program    April 18, 2017  (Date Submitted to Grad Studies)   Additional Committee Members include:  N/A   N/A    iii  Abstract Drinking water suppliers face challenges associated with changing populations, evolving economies, aging infrastructure, and shifting consumer demands. In small drinking water systems (SDWSs), these challenges are amplified by the pressure created from financial shortfalls and limited human resources. SDWSs are prone to higher rates of drinking water quality failure, are more vulnerable to spatiotemporal variability in water quality, and may be more susceptible to waterborne disease outbreaks than larger systems. Despite these challenges, SDWSs are overlooked in traditional academic and industrial studies, which often place a focus on larger, more complex drinking water supply systems (DWSSs) and the exploration, development, and implementation of new treatment technologies.  Given the current state of SDWSs, the main objectives identified for this research were to incentivize continuous performance, improve data resolution and water quality assessment practices for decision-making, and propose an improved drinking water quality management approach for SDWSs. This was accomplished in four distinct steps. The first step was to review the current state of practice of quality management systems and drinking water management systems and approaches in different parts of the world and within Canada to identify management gaps and potential areas for improvement. The second step was to explore the concept of continuous performance improvement and incentivize implementation through functional performance benchmarking. The third step was to improve on current drinking water quality assessment and benchmarking practices by implementing risk through quantifying degrees of compliance/non-compliance and spatial (i.e. location in the distribution system) and temporal (i.e. seasonal) variability through fuzzy rule-based modeling. The fourth and final step   iv  was to propose an improved drinking water management framework that fits within the bounds of Canada’s current decentralized governance system. The results of this research have the potential to be used by drinking water utility managers, operators, and planners to improve drinking water quality management in SDWSs at the federal, provincial/territorial, and municipal levels and improve on the current drinking water quality assessment and decision-making processes in place.    v  Preface I, Ty Anthony Bereskie, have developed all contents in this thesis under the supervision of Dr. Rehan Sadiq (University of British Columbia) and Dr. Manuel J. Rodriguez (Laval University). Dr. Husnain Haider (Qassim University) and Dr. Ianis Delpla (Laval University) have provided critical feedback and review of the publications listed below. Most of the contents of this thesis are published or under review for publication in scientific journals.  A version of Chapter 2 has been submitted for publication as a review article in Water Policy, an IWA Publishing journal, with the title “Drinking Water Management in Canadian Provinces and Territories: A Review and Comparison of Management Approaches for Ensuring a Safe Drinking Water Supply” (Bereskie et al., 2017b).  A version of Chapter 3 has been published in Elsevier journal, Science of the Total Environment, with the title “Framework for continuous performance improvement in small drinking water systems” (Bereskie et al., 2017c).  A version of Chapter 4 has been submitted for publication in Environmental Monitoring and Assessment, a Springer Journal, with the title “Small Drinking Water Systems under Spatiotemporal Water Quality Variability: A Risk-based Performance Benchmarking Framework” (Bereskie et al., 2017d).  A version of Chapter 5 has been accepted for publication in Environmental Management, a Springer journal, with the title “Drinking water management and governance in Canada: An innovative plan-do-check-act (PDCA) framework for a safe drinking water supply” (Bereskie et al., 2017a).   vi  Table of Contents  Examination Committee ............................................................................................................... ii Abstract ......................................................................................................................................... iii Preface .............................................................................................................................................v Table of Contents ......................................................................................................................... vi List of Tables .............................................................................................................................. xiv List of Figures ............................................................................................................................ xvii List of Symbols .............................................................................................................................xx List of Abbreviations ................................................................................................................ xxii Acknowledgements ................................................................................................................. xxvii Dedication ................................................................................................................................. xxix Chapter 1: Introduction ................................................................................................................1 1.1 Background ..................................................................................................................... 1 1.2 Research Motivation and Knowledge Gaps .................................................................... 2 1.3 Research Objectives ........................................................................................................ 3 1.4 Thesis Structure and Organization .................................................................................. 4 1.5 Proposed Framework ...................................................................................................... 6 1.6 Meta Language................................................................................................................ 8 Chapter 2: Literature Review .......................................................................................................9 2.1 Background ..................................................................................................................... 9 2.1.1 Drinking Water in Canada ........................................................................................ 10 2.1.1.1 Classification of Drinking Water Supply Systems ........................................... 10   vii  2.1.2 Small Drinking Water Systems (SDWSs) ................................................................ 14 2.1.2.1 Differences between Small Drinking Water Systems and Larger Systems ...... 15 2.1.2.2 Spatiotemporal Variability ................................................................................ 17 2.1.3 Performance in a Drinking Water Context ............................................................... 18 2.1.3.1 Drinking Water Advisories ............................................................................... 19 2.2 Drinking Water Management Systems ......................................................................... 22 2.2.1 Quality Management Systems .................................................................................. 24 2.2.1.1 Hazard Analysis and Critical Control Points .................................................... 24 2.2.1.2 ISO 9001:2015 Quality Management System Requirements ........................... 26 2.2.2 Drinking Water Management Systems ..................................................................... 27 2.2.2.1 Australia ............................................................................................................ 27 2.2.2.2 Canada............................................................................................................... 29 2.2.2.3 New Zealand ..................................................................................................... 31 2.2.2.4 World Health Organization Water Safety Plan ................................................. 33 2.2.3 Canadian Provincial/Territorial Drinking Water Management Approaches ............ 35 2.2.3.1 Alberta............................................................................................................... 36 2.2.3.2 British Columbia ............................................................................................... 38 2.2.3.3 Manitoba ........................................................................................................... 39 2.2.3.4 New Brunswick ................................................................................................. 40 2.2.3.5 Newfoundland and Labrador ............................................................................ 41 2.2.3.6 Northwest Territories ........................................................................................ 42 2.2.3.7 Nova Scotia ....................................................................................................... 43 2.2.3.8 Nunavut ............................................................................................................. 44   viii  2.2.3.9 Ontario .............................................................................................................. 45 2.2.3.10 Prince Edward Island .................................................................................... 47 2.2.3.11 Quebec .......................................................................................................... 47 2.2.3.12 Saskatchewan ................................................................................................ 50 2.2.3.13 Yukon ............................................................................................................ 51 2.3 Comparison of Quality Management Systems and Drinking Water Quality Management Systems ............................................................................................................... 52 2.3.1 Catchment to Consumer Recommended Quality Management Elements ................ 58 2.4 Summary ....................................................................................................................... 59 Chapter 3: Continuous Performance Improvement in Small Drinking Water Systems ......61 3.1 Background ................................................................................................................... 62 3.1.1 Continuous Performance Improvement .................................................................... 62 3.1.2 Plan-Do-Check-Act Cycle ........................................................................................ 63 3.1.3 Water Quality Indices ............................................................................................... 64 3.1.3.1 Canadian Council of Ministers of the Environment Water Quality Index ....... 65 3.1.3.2 British Columbia Water Quality Index ............................................................. 68 3.1.3.3 Centre St. Laurent Water Quality Index ........................................................... 68 3.1.3.4 Quebec Index .................................................................................................... 69 3.1.4 Benchmarking ........................................................................................................... 70 3.1.4.1 Benchmarking Process ...................................................................................... 71 3.1.4.2 Benchmarking and Drinking Water Management ............................................ 73 3.2 Approach and Methodology ......................................................................................... 74 3.2.1 Data Collection ......................................................................................................... 75   ix  3.2.2 Calculate Water Quality Index.................................................................................. 75 3.2.3 Functional Performance Benchmarking ................................................................... 76 3.2.4 Identification and Prioritization of Improvement Actions ........................................ 77 3.2.5 Implementation of Performance Improvement Actions ........................................... 77 3.2.6 Continuous Performance Improvement .................................................................... 78 3.3 Demonstration using Small Drinking Water Systems in Newfoundland and Labrador (Canada) .................................................................................................................................... 79 3.3.1 Data Preparation........................................................................................................ 80 3.3.1.1 Performance indicators ..................................................................................... 81 3.3.2 Calculating Water Quality Index .............................................................................. 84 3.3.3 Functional Performance Benchmarking ................................................................... 85 3.3.4 Identification and Prioritization of Improvement Actions ........................................ 85 3.3.5 Implementation of Improvement Actions ................................................................. 88 3.4 Demonstration Results for Overall WQI Performance ................................................. 88 3.5 Demonstration Results for an Individual Performance Indicator (THM4) ................... 90 3.6 Summary ....................................................................................................................... 92 Chapter 4: Risk-based Water Quality Performance Benchmarking for Small Drinking Water Systems ..............................................................................................................................93 4.1 Background ................................................................................................................... 93 4.1.1 Risk Management ..................................................................................................... 94 4.1.1.1 Risk-based Drinking Water Management ......................................................... 96 4.1.2 Fuzzy Sets ................................................................................................................. 97 4.1.3 Fuzzy Logic .............................................................................................................. 98   x  4.1.3.1 Fuzzification ..................................................................................................... 99 4.1.3.2 Defuzzification ................................................................................................ 100 4.1.3.2.1 Centroid Defuzzification ........................................................................... 101 4.1.4 Fuzzy Rules ............................................................................................................. 101 4.1.5 Fuzzy Inference Systems ........................................................................................ 102 4.1.5.1 Mamdani-type FIS .......................................................................................... 102 4.1.5.2 Sugeno-type FIS.............................................................................................. 103 4.2 Approach and Methodology ....................................................................................... 104 4.2.1 Establish Performance Indicators ........................................................................... 105 4.2.2 Select and Categorize Performance Indicators ....................................................... 106 4.2.3 Acquire Water Quality Data ................................................................................... 107 4.2.4 Establish Universe of Discourse and Hierarchical Mapping of Selected Performance Indicators............................................................................................................................. 107 4.2.5 Calculate RWQI Using Fuzzy Rule-Based Modeling ............................................... 109 4.2.6 Transform Risk into RWQI Performance and Perform Risk-based Functional Performance Benchmarking................................................................................................ 110 4.3 Demonstration Using Small Drinking Water Systems in Quebec and Newfoundland and Labrador, Canada ............................................................................................................. 111 4.3.1 Selected Small Drinking Water Systems ................................................................ 112 4.3.2 Temporal Risk ......................................................................................................... 113 4.3.3 Spatial Risk ............................................................................................................. 115 4.3.4 Fuzzy Rule-based Modeling ................................................................................... 116 4.4 Results ......................................................................................................................... 118   xi  4.5 Sensitivity Analysis .................................................................................................... 123 4.6 Summary ..................................................................................................................... 126 Chapter 5: An Innovative Plan-Do-Check-Act (PDCA) Approach to Drinking Water Management and Governance in Canada ...............................................................................128 5.1 Background ................................................................................................................. 129 5.2 Approach and Methodology ....................................................................................... 129 5.2.1 Federal Governance ................................................................................................ 130 5.2.2 Provincial/Territorial Governance .......................................................................... 133 5.2.3 Municipal Governance ............................................................................................ 139 5.2.4 Challenges ............................................................................................................... 140 5.3 Proposed PDCA-WSP Framework ............................................................................. 143 5.3.1 Plan ......................................................................................................................... 146 5.3.2 Do ............................................................................................................................ 147 5.3.3 Check ...................................................................................................................... 148 5.3.4 Act ........................................................................................................................... 149 5.3.5 Strategic Action Plan for Implementation of PDCA-WSP ..................................... 150 5.3.5.1 Phase 1 ............................................................................................................ 151 5.3.5.2 Phase 2 ............................................................................................................ 152 5.3.5.3 Phase 3 ............................................................................................................ 153 5.3.5.4 Small Drinking Water Systems ....................................................................... 153 5.4 Summary ..................................................................................................................... 154 Chapter 6: Conclusions and Recommendations .....................................................................156 6.1 Conclusions ................................................................................................................. 156   xii  6.2 Originality and Contribution ....................................................................................... 160 6.3 Limitations and Recommendations............................................................................. 162 Bibliography ...............................................................................................................................166 Appendices ..................................................................................................................................187 Appendix A: CCME WQI Inputs/Outputs for CPI Framework Demonstration ..................... 188 Appendix A-1 : CPI Framework CCME WQI Input Data (NL) ...................................... 188 Appendix A-2 : Established CCME WQI Guidelines for CPI Framework Demonstration…………………………………………………………………………..... 191 Appendix A-3 : Seasonal CCME WQI Outputs by SDWS for CPI Framework ............. 192 Appendix A-4 : Seasonal CCME WQI Outputs by Parameter for CPI Framework Demonstration …………………………………………………………………………...193 Appendix B: Fuzzy rules for Risk-based Performance Benchmarking Framework (RWQI) ... 196 Appendix B-1 : Matrix Defining Fuzzy Rules for ‘Generation 5’ ................................... 196 Appendix B-2 : Matrix Defining Fuzzy Rules for ‘Generation 4’ ................................... 200 Appendix B-3 : Matrix Defining Fuzzy Rules for ‘Generation 3’ ................................... 217 Appendix B-4 : Matrix Defining Fuzzy Rules for ‘Generation 2’ ................................... 229 Appendix C: CCME WQI and RWQI Inputs and Outputs ....................................................... 233 Appendix C-1 : CCME WQI and RWQI Outputs for Generation 1 and 2 (NL and QC) ... 233 Appendix C-2 : CCME WQI Outputs for Generation 3 (NL and QC) ............................ 234 Appendix C-3 : RWQI Outputs for Generation 3 (NL and QC) ......................................... 235 Appendix C-4 : RWQI Outputs for Generation 4 (NL) ...................................................... 236 Appendix C-5 : RWQI Outputs for Generation 4 (QC) ...................................................... 237 Appendix C-6 : RWQI Inputs and Outputs for Generation 5 (NL) .................................... 238   xiii  Appendix C-7 : RWQI Inputs and Outputs for Generation 5 (QC) .................................... 243   xiv  List of Tables Table 2-1. Sized-based classification of drinking water supply systems...................................... 13 Table 2-2. Provincial small drinking water system terminology and defined size (Pons et al., 2015) ............................................................................................................................................. 14 Table 2-3. Boil water advisories by province/territory in Canada as of February 29, 2008 (Eggertson, 2008) .......................................................................................................................... 20 Table 2-4. Modern HACCP principles (Ropkins and Beck, 2000; Sperber, 2005; Jayaratne, 2008)....................................................................................................................................................... 25 Table 2-5. Australian Framework for Management of Drinking Water Quality general areas (NHMRC, 2011) ........................................................................................................................... 28 Table 2-6. Process for Australian Framework for Management of Drinking Water Quality (NHMRC, 2011) ........................................................................................................................... 29 Table 2-7. Canadian Council of Ministers of the Environment multi-barrier approach elements and description (CCME, 2002) ..................................................................................................... 31 Table 2-8. New Zealand Ministry of Health suggested steps for water safety plan development (NZMOH, 2015) ........................................................................................................................... 33 Table 2-9. World Health Organization Water Safety Plan elements and description (Davison et al., 2006; WHO, 2011) .................................................................................................................. 34 Table 2-10. Principle processes for Alberta Drinking Water Safety Plan (Government of Alberta, 2012) ............................................................................................................................................. 37 Table 2-11. British Columbia CS2TA modules (BCMHLS, 2010) ............................................. 39 Table 2-12. Levels of the MBSAP in Newfoundland and Labrador (Government of Newfoundland and Labrador, 2014) ............................................................................................. 41   xv  Table 2-13. Northwest Territories Safe Drinking Water Framework and Strategy categories and barriers (GNWT, 2014) ................................................................................................................. 43 Table 2-14. Ontario Drinking Water Quality Management Standard plan/do elements (Ontario MOE, 2007) .................................................................................................................................. 46 Table 2-15. Quebec Water Policy orientations and key actions (Quebec MoE, 2002) ................ 47 Table 2-16. Quebec Integrated Water Resources Management implementation steps (Gangbazo, 2004; Baril et al., 2006) ................................................................................................................ 49 Table 2-17. Saskatchewan Safe Drinking Water Strategy goals and objectives (Government of Saskatchewan, 2003)..................................................................................................................... 51 Table 2-18. Comparison of selected quality management systems and drinking water management systems/recommendations ....................................................................................... 55 Table 2-19. Comparison of selected Canadian drinking water management approaches ............ 56 Table 2-20. Frequency of elements found in selected drinking water quality management systems....................................................................................................................................................... 57 Table 2-21. Catchment to consumer recommended drinking water management elements ........ 59 Table 3-1. CCME WQI categorization (modified after CCME, 2001 for drinking water utilities)....................................................................................................................................................... 68 Table 3-2. System information from selected small drinking water systems in Newfoundland and Labrador, Canada .......................................................................................................................... 80 Table 3-3. Summary table of seasonal Canadian Council of Ministers of the Environment Water Quality Index for the selected water utilities ................................................................................ 84 Table 3-4. Generalized improvement matrix for performance improvement ............................... 85 Table 3-5. Ranking mechanism for cumulative performance improvement ................................ 87   xvi  Table 3-6. Improvements chosen for Newfoundland and Labrador demonstration ..................... 88 Table 4-1. Example of potential microbiological, chemical, and physical drinking water performance indicators (WHO, 2006; Hurley et al., 2012; Islam et al., 2013a; Scheili et al., 2014)…………………………………………………………………………………………… 106 Table 4-2. Linguistic description, universe of discourse, and associated water quality, spatial, and temporal criteria .......................................................................................................................... 108 Table 4-3. System information for selected small drinking water systems in Newfoundland and Labrador and Quebec, Canada .................................................................................................... 113 Table 4-4. Selected performance indicators and established universe of discourse for demonstration .............................................................................................................................. 117 Table 4-5. Description of RWQI and CCME WQI values/rankings (CCME, 2001) .................... 117 Table 5-1. Description of applicable federal legislation related to drinking water (Health Canada, 2013) ........................................................................................................................................... 131 Table 5-2. Applicable drinking water and source water protection legislation/policy, associated quality management frameworks, and other requirements (Hill et al., 2008; Dunn et al., 2014b)………………………………………………………………………………………….. 136    xvii  List of Figures Figure 1-1. Thesis structure and organization................................................................................. 5 Figure 1-2. Proposed research framework for improved drinking water quality management in small drinking water systems .......................................................................................................... 6 Figure 2-1. Drinking water plants in Canada, by population served (Statistics Canada, 2010) ... 11 Figure 2-2. Example diagram of a small drinking water system .................................................. 15 Figure 2-3. Example diagram of a medium to large-sized drinking water system ....................... 16 Figure 2-4. Boil water advisories in Canada by community size from 2010-2015 (Environment and Climate Change Canada, 2016).............................................................................................. 20 Figure 2-5. Water quality related reasons for boil water advisories in Canada from 2010-2015 (Environment and Climate Change Canada, 2016) ...................................................................... 21 Figure 2-6. Potable water volumes processed by drinking water plants in Canada by source water type in 2011 from each province/territory (Statistics Canada, 2011) ........................................... 36 Figure 3-1. Illustration and description of steps in the PDCA cycle (Reid et al., 1999; Taylor et al., 2013) ....................................................................................................................................... 64 Figure 3-2. Flowchart of the performance benchmarking process (adapted from Elmuti and Kathawala, 1997) .......................................................................................................................... 72 Figure 3-3. Proposed continuous performance improvement framework .................................... 74 Figure 3-4. Relative performance of SDWSs in terms of performance gaps between calculated WQI and established benchmark (modified after Haider, 2016a) ................................................ 77 Figure 3-5. Proposed continuous performance improvement framework conceptual diagram .... 78 Figure 3-6. Continuous performance improvement framework application for overall WQI score of three selected small drinking water system in Newfoundland and Labrador demonstration ... 89   xviii  Figure 3-7. Continuous performance improvement framework application for functional performance benchmarking of THM4-R1 of three selected small drinking water systems in Newfoundland and Labrador demonstration ................................................................................ 91 Figure 4-1. Generic framework for risk management .................................................................. 95 Figure 4-2. Differences between crisp sets and fuzzy sets (adapted from Sadiq et al., 2015) ...... 98 Figure 4-3. Differences between traditional logic and fuzzy logic ............................................... 99 Figure 4-4. Standard trapezoidal membership function .............................................................. 100 Figure 4-5. Proposed framework for risk-based performance benchmarking of small drinking water systems .............................................................................................................................. 105 Figure 4-6. Proposed RWQI hierarchy, coding schematic, and description/example for each generation in the proposed framework ....................................................................................... 109 Figure 4-7. Conceptual diagram for proposed risk-based functional performance benchmarking (modified from Haider, 2016a) ................................................................................................... 111 Figure 4-8. Survey results (2007 and 2011) of potable water use volumes in Canada and assigned seasonal risk (Statistics Canada 2013) ........................................................................................ 114 Figure 4-9. Drinking water supply system spatial risk diagram ................................................. 116 Figure 4-10. Comparison of CCME WQI scores and RWQI for all 16 SDWSs at Generation 1: a) NL small drinking water systems, b) QC small drinking water systems .................................... 119 Figure 4-11. Comparison of CCME WQI and RWQI for all 16 SDWSs at Generation 2: a) NL SDWSs, b) QC SDWSs .............................................................................................................. 121 Figure 4-12. Comparison of CCME WQI and RWQI for QC SDWS-6 for each performance indicator in Generation 3 ............................................................................................................ 122   xix  Figure 4-13. Percent contribution of the nine selected PIs (Generation 3) to RWQI performance (Generation 1) ............................................................................................................................. 124 Figure 4-14. Percent contribution of seasonal variation (Generation 4) to chemical PI (FRC, HAA5, THM4) performance (Generation 3) .............................................................................. 125 Figure 4-15. Percent contribution of distribution system location (Generation 5) to seasonal HAA5 performance (Generation 4) ............................................................................................ 126 Figure 5-1. Current framework for drinking water quality governance in Canada (CCME, 2004; Kayser et al., 2015) ..................................................................................................................... 130 Figure 5-2. SWOT analysis of the current Canadian federal, provincial/territorial, and municipal governance and management system .......................................................................................... 142 Figure 5-3. Proposed PDCA-WSP framework for drinking water management in Canada ...... 144 Figure 5-4. Strategic action plan for prioritization of specific improvement actions for the implementation of the proposed PDCA-WSP framework .......................................................... 151    xx  List of Symbols µA Membership function of set A A Set of numbers A̰ Value for input ‘x’ A̰1k Fuzzy value for input ‘x1’ representing the kth antecedent pair A̰2k Fuzzy value for input ‘x2’ representing the kth antecedent pair Ai Mean level of exceedance for variable ‘i’ for guideline ‘i’ Ai Fuzzy subset corresponding to a linguistic constant b1, b2, b3, b4  Defining ranges for standard fuzzy trapezoidal membership function B̰ Value for input ‘y’ Bj Fuzzy subset corresponding to a linguistic constant B̰k Fuzzy subset representing the kth consequent CCME WQI Canadian Council of Ministers of the Environment Water Quality Index excursioni Excursion value for objective ‘i’  f(x) Function of ‘x’ f(x,y) Function of ‘x’ and ‘y’ F1 (BC WQI) Percentage of measurements in which one or more guidelines are exceeded F1 (CCME WQI) Scope, the extent of water guideline non-compliance of a time period F2 (BC WQI) Percentage of measurements in which one or more guidelines are exceeded F2 (CCME WQI) Frequency, the percentage of individual tests that do not meet their objectives F3 (BC WQI) Maximum (normalized to 100) by which any guidelines are exceeded F3 (CCME WQI) Amplitude, the amount by which failed test values do not meet their objectives FailedTestValuei Value of failed test ‘i’ Fi Frequency of values that exceed a guideline ‘i’ for a variable i Location within the drinking water system Isubi Subindex score for the subindex ‘i’ j Type of improvement k Number of IF-THEN proposition   xxi  min Minimum operator  n Total number of variables nse Normalized sum of exclusions Objectivej Objective guideline value ‘j’ r Linguistic IF-THEN propositions RWQI Overall risk-based water quality value  t time x Input ‘x’ for Sugeno (1985) FIS x Water quality indicator x1 Input ‘x1’ (antecedent) for Mamdani (1977) FIS x2 Input ‘x2’ (antecedent) for Mamdani (1977) FIS X Universe of discourse X Overall WQI value X Input fuzzy variable Xmax Maximum possible WQI value Xmin Minimum possible WQI value y Input ‘y’ for Sugeno (1985) FIS y Single output ‘y’ for Mamdani (1977) FIS Y Output fuzzy variable z Output variable for Sugeno (1985) rule z Output variable for centroid defuzzification z = f(x,y) Crisp function z* Crisp output of centroid defuzzification Z Improvement action Z1 Source water protection improvement actions Z2 Drinking water treatment improvement actions Z3 Distribution system management improvement actions   xxii  List of Abbreviations AB Alberta AS/NZS Australia/New Zealand Standard AWWA American Water Works Association BC British Columbia BC WQI British Columbia Water Quality Index BCMHLS British Columbia Ministry of Healthy Living and Sport BCMOH British Columbia Ministry of Health BOD Biological oxygen demand BWA Boil water advisory BWO Boil water order C Chemical CCME Canadian Council of Ministers of the Environment CCP Critical control point CAD Canadian dollar CDW Federal-Provincial-Territorial Committee on Drinking Water COA Center of area COD Chemical oxygen demand COG Center of gravity COM Center of maximum CPI Continuous performance improvement CS2TA Comprehensive drinking water source-to-tap assessment CSL Centre St. Laurent DBP Disinfection by-product DNCA Do not consume advisory DNCA Do not consume order DNDA Do not drink advisory DNDO Do not drink order   xxiii  DNUA Do not use advisory DNUO Do not use order DWA Drinking water advisory DWAP Drinking Water Assistance Programme DWP Drinking water plan DWMS Drinking water management system DWSNZ Drinking water standards for New Zealand DWSP Drinking water safety plan DWSS Drinking water supply system E. coli Escherichia coli ESRD Environment and Sustainable Resource Development ETA Event tree analysis EU European Union FC Fecal coliforms FIS Fuzzy inference system FMEA Failure mode effect analysis FMECA Failure mode effects and criticality analysis  FRBM Fuzzy rule-based modeling FRC Free residual chlorine FTA Fault tree analysis GCDWQ Guidelines for Canadian Drinking Water Quality GNWT Government of the Northwest Territories GUDI Groundwater under the direct influence of surface water H High HAA Haloacetic acid HACCP Hazard Analysis and Critical Control Points HAN Haloacetonitriles HAZOP Hazard and operability HK Haloketones   xxiv  HRA Human reliability analysis ISO International Organization for Standardization  IWRM Integrated water resources management L Low LDWS Large drinking water system LOM Left of maximum MED Medium M Microbiological MAC Maximum allowable concentration MB Manitoba MBA Multi-barrier approach MBSAP Multi-barrier strategic action plan MDWS Medium-sized drinking water system MISO Multiple input, single output MoE Ministry of the Environment NB New Brunswick NCCEH National Collaborating Centre for Environmental Health NHMRC Australian National Health and Medical Research Council NL Newfoundland and Labrador NS Nova Scotia NTU Nephelometric turbidity unit NU Nunavut  NWT Northwest Territories NZMOH New Zealand Ministry of Health ODWMS Ontario Drinking Water Quality Management Standard OECD Organization for Economic Cooperation and Development OMOHLTC Ontario Ministry of Health and Long-Term Care ON Ontario P Physical   xxv  PDCA Plan-do-check-act PDCA-WSP Plan-do-check-act water safety plan PE Prince Edward Island Ph.D. Doctor of Philosophy  PHRMP Public health risk management plan PI Performance indicator QC Quebec QMS Quality management system QWP Quebec Water Policy R1 Beginning of the distribution system R2 Point in the distribution system equidistant from the beginning and extremity R3 Extremity of the distribution system RBD Reliability block diagram ROM Right of maximum RS Risk score RWQI Risk-based performance benchmarking framework SDWS Small drinking water system SK Saskatchewan SSF Slow sand filtration SWOT Strengths-Opportunities-Weaknesses-Threats TC Total coliforms TDS Total dissolved solids TFN Triangular fuzzy number THM Trihalomethane TOC Total organic carbon TSS Total suspended solids UOD Universe of discourse US United States USEPA United States Environmental Protection Agency   xxvi  USGS United States Geological Survey UV Ultraviolet  VH Very high VL Very low WHO World Health Organization WQ Water quality WQI Water quality index WSA Water Security Agency WSP Water safety plan WSSD World Summit on Sustainable Development YT Yukon  ZFN Trapezoidal fuzzy number   xxvii  Acknowledgements I would like to thank my supervisors, Dr. Rehan Sadiq and Dr. Manuel J. Rodriguez for their patience and guidance throughout my research. Dr. Sadiq helped me to fully engage in the academic process and to think outside the box while providing nothing but positivity and encouragement from my first day as a Ph.D. student. He spent many hours helping me to fully develop ideas and concepts, but knew exactly when to give me the space necessary for critical thinking and personal growth. I would also like to thank Dr. Husnain Haider, my third unofficial advisor, and friend, who has served as a mentor to me throughout my entire tenure at UBCO. Even after moving to Saudi Arabia (i.e. an 8 hour time difference), he still provided me with valuable insight, constructive criticism, and encouragement and always made himself available to listen.  I must also provide thanks to my committee member Dr. Kasun Hewage, fellow graduate students and friends, Dr. Nilufar Islam, Tim Hurley, Oleg Shabarchin, Josephine Morgenroth, Hassan Iqbal, and Roberta Dyck, and my colleagues at Laval University, Dr. Anna Scheili, Dr. Ianis Delpla, Dr. Christelle Legay, and Louis Coulombe for their assistance, guidance, and friendship. Special gratitude must also be given to Francine Baril at Université Laval and Shannon Hohl and Angela Perry at UBCO for assisting with the financial and administrative aspects of my graduate research. This work would also not have been possible without the financial support from the Natural Sciences and Engineering Research Council (NSERC) and the RES’EAU WaterNET.     xxviii  I also extend my love and sincerest thanks to my mother, father, brother, niece, aunts, uncles, cousins, and friends who listened to my every frustration over the past few years. They were always there for me when I needed it most, even from over 3,000 miles away.  Lastly, I thank my girlfriend Kelsey, who joined me moving to another country and sacrificed much to be with me during this stressful, long process. Not only did she make the best of a difficult situation, she excelled! Her optimism, joy, and love provided me with a constant source of inspiration to push forward with this research.   xxix  Dedication To those who bring me joy, inspire my creativity, and make me a better person.   1  Chapter 1: Introduction  1.1 Background The World Health Organization (WHO) (2011) declared access to safe drinking water as an essential component of health and hygiene and a basic human right. While major drinking water issues in developed countries are rare and often isolated, their occurrence can be catastrophic, resulting in significant public health, social, and economic repercussions (Hrudey et al., 2003; Rizak and Hrudey, 2008; Jayaratne, 2008). In Canada, the management of a safe drinking water supply has been under scrutiny since two major waterborne disease outbreaks in the early 2000s changed the drinking water landscape throughout the country. In 2000, an outbreak of Escherichia coli (E. coli) 0157:H7 and Camylobacter jejuni (C. jejuni) in Walkerton, Ontario (ON), a community of 4,800 people, resulted in over 2,300 cases of illness and seven deaths (O’Conner, 2002a; Hrudey et al., 2003). Less than one year later, an outbreak of Cryptosporidium parvum in North Battleford, Saskatchewan (SK), a community of 15,000 people, lead to between 5,800 – 7,100 cases of illness.  Today, in the United States (US) and Canada, public drinking water is generally considered very safe and poses a negligible risk to consumers (Moffatt and Struck, 2011). However, for residents of small, rural, and First Nations communities, safe drinking water at the tap is not as assured (Hrudey, 2011). Small drinking water systems (SDWSs) account for the vast majority of systems in operation throughout Canada and serve millions of people (Statistics Canada, 2011). They are often simple in nature, lacking the financial backing and human resources required for more complex treatment technologies (Moffat and Struck, 2011; Scheili et al., 2014). Despite this, they   2  are expected to meet the same regulatory requirements and customer expectations of larger, more robust drinking water supply systems (DWSSs) while falling outside of traditional academic, political, and industrial interests (Hrudey, 2011; Moffat and Struck, 2011; Scheili et al., 2014).  1.2 Research Motivation and Knowledge Gaps Throughout the world, waterborne disease outbreaks and drinking water supply issues have drawn attention to the importance of properly managing and governing drinking water and public health (Dunn et al., 2014a). Like all other infrastructure systems, DWSSs face internal challenges associated with rapid growth or shrinking populations, aging infrastructure and asset management, shifting economies, and evolving consumer demands (Danilenko et al., 2014; Haider et al., 2014). They also face external threats from a changing climate, raising concerns about water quality and quantity and increased spatiotemporal variability into the future, along with the omnipresent and ever-changing socioeconomic landscape (Danilenko et al., 2014; Haider et al., 2014).   Beyond these challenges, literature has shown that SDWSs are constrained by smaller tax-bases, less friendly economies of scale (i.e. the proportionate savings in costs associated with serving more people), limited human resources, and underqualified and/or undertrained staff (Moffatt and Struck, 2011; Scheili et al., 2014; Dyck et al., 2014; Haider et al., 2015a). SDWSs are prone to higher rates of drinking water quality failure (Scheili et al., 2014), are more vulnerable to spatiotemporal variability of water quality (Symanski et al., 2004; Lee et al., 2013; Dyck et al., 2014; Scheili et al., 2014; Charisiadis et al., 2015), and may be more vulnerable to waterborne disease outbreaks (Moffatt and Struck, 2011) than their larger counterparts. However, despite the   3  documented issues and prevalence of SDWSs throughout the world, research into enhancing drinking water management through incorporating improved analytics, better decision-making practices, and industrial quality management techniques has been largely overlooked in favor of developing new processes and technologies (Islam et al., 2011; Haider et al., 2015a).  1.3 Research Objectives The main goal identified for this research was to gauge the current state of drinking water management and governance, specifically in SDWSs, throughout Canada, provide a framework that incentivizes continuous improvement in DWSSs, improve data resolution and water quality assessment practices for decision-making, and propose an improved drinking water management approach for Canada. To accomplish this goal, the specific objectives defined for this research were:  Objective 1 – Conduct a critical review of existing drinking water management systems (DWMSs) and identify applicable research gaps for SDWSs.  Objective 2 – Develop a drinking water quality improvement framework based on the principles of continuous performance improvement (CPI).  Objective 3 – Develop a risk-based benchmarking approach for use by SDWSs as an alternative to traditional performance assessment techniques.  Objective 4 – Demonstrate the application of CPI and risk-based benchmarking as part of an improved drinking water management approach within the current decentralized governance structure in Canada.     4  It is important to note that the concepts presented as part of Objective 2 and Objective 3 are not designed as alternatives to health-based regulatory guidelines and standards, but represent the development of performance metrics to be used for comparison (i.e. performance benchmarking) and ease of understanding for the general public.  1.4 Thesis Structure and Organization This thesis contains six chapters. Figure 1-1 illustrates the structure and organization related to each objective defined in Section 1.3. Chapter 1 covers the introduction to the subject matter, the research motivation, and highlights the proposed research framework. Chapter 2 covers the review of applicable literature in the area of study. Chapters 3-5 are based on the research gaps and concepts discussed in Chapter 2. Objectives 1, 2, 3, and 4 have been achieved and discussed in Chapters 2, 3, 4, and 5 respectively. Finally, the conclusions and future research recommendations are provided in Chapter 6.   5   Figure 1-1. Thesis structure and organization  6  1.5 Proposed Framework To achieve the objectives highlighted in Section 1.3, a research framework is presented in Figure 1-2 highlighting the applicable research elements and components. This figure also illustrates the applicable deliverables (i.e. Paper 1, Paper 2, Paper 3, and Paper 4) published or submitted for publication as part of this research.   Figure 1-2. Proposed research framework for improved drinking water quality management in small drinking water systems  Areas such as water quality monitoring and performance assessment have been studied in detail over the past decade. However, very little research has been conducted into specifically improving drinking water management. The first step to improving drinking water management is to understand the current systems and approaches in place.    7  In Chapter 2, a review of quality management systems (QMSs) and DWMSs was conducted to compare management elements in Canada (i.e. federal and provincial/territorial) and throughout the world. This information was then used to highlight potential management gaps for the development of new DWMSs or improving existing DWMSs throughout Canada.  In Chapter 3, the concepts of CPI and functional performance benchmarking were reviewed. While these concepts are usually applied to improve performance in private sector organizations, in this section, they were applied as part of a drinking water quality framework to provide an incentive for continuous improvement in SDWSs. This concept was then applied in a demonstration using the Canadian Council of Ministers of the Environment (CCME) Water Quality Index (WQI) methodology for aggregating water quality data for SDWSs in Newfoundland and Labrador (NL), Canada.  In Chapter 4, the incorporation of risk and fuzzy rule-based modeling (FRBM) was applied to water quality assessment and benchmarking in SDWSs. This approach was tailored to provide an alternative to the traditional, binary systems of water quality assessment based solely on compliance and/or noncompliance while incorporating spatiotemporal variability. This concept was then applied in a demonstration using data from SDWSs in Newfoundland and Labrador and Quebec (QC), Canada and the results were compared to the traditional CCME WQI technique used in Chapter 3.  In Chapter 5, drinking water management and governance in Canada was reviewed. Using the frameworks and techniques established in Chapter 2, Chapter 3, and Chapter 4, a framework   8  based on the principles of plan-do-check-act (PDCA) and the WHO water safety plan (WSP) recommendations was proposed (i.e. the PDCA-WSP) to fit within Canada’s current decentralized drinking water governance system.   1.6 Meta Language  The majority of the research in this thesis included a specific, technical vocabulary with well-defined usage. However, portions of this research also included broad management principles and policies (often incorporating generic terminology) that are not typically blended into science and engineering applications. As such, this section was included to ensure consistency and understanding within this thesis.  For the purpose of this research, the term ‘framework’ was used to represent holistic methods (e.g. CPI framework). The term ‘approach’ and/or ‘strategy’ was used to represent less formal and/or less defined versions of ‘frameworks’. For example, in Chapter 2, Prince Edward Island (PE) does not have a formal DWMS, but features a broad approach to drinking water management. The term ‘model’ was used for specific components of a framework, when detailed modeling is used to fill research gaps for performance assessment or management of SDWSs (e.g. MATLAB Simulink model for calculating RWQI). The terms ‘technique’, ‘method’, and ‘methodology’ were used interchangeably for applied mathematical and statistical procedures (e.g. centroid defuzzification technique). The terms ‘benchmark’ and ‘benchmarking’ in the context of this research represent the comparison of performance externally (i.e. functional (industry) benchmarking) against similar water utilities [Section 3.1.4].    9  Chapter 2: Literature Review A part of this chapter has been submitted for publication in Water Policy, an IWA Publishing journal, as a review article titled “Drinking Water Management in Canadian Provinces and Territories: A Review and Comparison of Management Approaches for Ensuring a Safe Drinking Water Supply” (Bereskie et al., 2017b).   This chapter contains three main sections. The first section contains a literature review of background information related to the current state of drinking water management in Canada, specifically in SDWSs. With regards to Objective 1 as defined in Chapter 1, the second and third sections include a critical literature review and comparison of DWMSs and approaches used to ensure safe drinking water throughout the world and within Canada. Although the effectiveness and impact of management practices can be difficult to quantify, by comparing the Canadian state of practice and included management elements at the national and provincial/territorial levels against the WHO, world leaders, and well established QMSs, context can be provided to gauge the comprehensiveness or lack thereof in regards to provincial and territorial DWMSs. This information is then used to highlight potential management gaps for the development of new DWMSs or improving existing DWMSs throughout Canada.  2.1 Background Throughout the world, an estimated 748 million people lack access to an improved source of drinking water and hundreds of millions more cannot rely on sufficient quantities of safe drinking water (WHO, 2014). Over the past several years, there has been an increase in reports of violent conflicts over drinking water and this trend is likely to continue in the wake of rapid   10  population growth and a changing climate (Gleick and Heberger, 2014). Canada, despite being known for a relative abundance of high-quality freshwater supplies, is far from immune to drinking water quality and quantity issues (Exall et al., 2006; PaiMazumder et al., 2013; Yusa et al., 2015).   2.1.1 Drinking Water in Canada In much of Canada, fresh water sources are widely available for use. However, in some areas, there are moderate to high threats associated with water availability where demand exceeds supply (Environment Canada, 2009; Exall et al., 2006). Exall et al. (2006) reported that in 2001, approximately 25% of municipalities with public DWSSs reported water quantity shortages. Recent assessments on the impacts of climate change in Canada indicate further freshwater supply shortages in some areas, especially within the Prairie Provinces (i.e. Alberta (AB), Saskatchewan, and Manitoba (MB)) (Exall et al., 2006; PaiMuzumder et al., 2013; Yusa et al., 2015). There are also significant drinking water quality concerns. A study by Murphy et al. (2015) reported that municipal DWSSs (serving more than 1,000 people) in Canada may be responsible for over 300,000 cases of acute gastrointestinal illness per year, of which over a third can be attributed to problems within the distribution system. The annual economic burden of these waterborne disease outbreaks was estimated in a study by Vinson (2012) at approximately 2.7 billion dollars (CAD).  2.1.1.1 Classification of Drinking Water Supply Systems  As of 2011, the average Canadian is estimated to use approximately 250 liters of treated water per day for residential water use (Environment Canada, 2014). While this is a dramatic reduction   11  from the 342 liters of treated water used per day used in 1991, it still represents one of the highest per capita water use volumes in the world (Environment Canada, 2014). A 2011 survey of over 2,000 individual Canadian DWSSs (serving 300 people or more), producing a combined 5,103 million cubic meters of treated water, found that 84% of the population (i.e. approximately 29 million people) relies on drinking water from municipal DWSSs (Statistics Canada, 2011; Murphy et al., 2015). Of the surveyed systems, almost 80% serve populations less than 5,000 people (Figure 2-1) (Statistics Canada, 2011).    Figure 2-1. Drinking water plants in Canada, by population served (Statistics Canada, 2010)  Drinking water supply systems in Canada are generally categorized based on their type (i.e. ownership and type of population served) and/or size (i.e. population served, number of connections, or quantity of water delivered) (Moffatt and Struck, 2011; Pons et al., 2015). Schuster et al. (2005) categorized DWSSs in Canada as either private (i.e. privately owned, supplying drinking water to owners and guests), semi-public (i.e. privately owned, supplying drinking water to the general public - often on a seasonal basis), or public (i.e. publicly owned   12  municipal DWSSs) (Moffatt and Struck, 2011). However, Moffatt and Struck (2011) noted that this classification fails to incorporate privately operated, public owned systems such as nursing homes, hospitals, and mobile home parks. A 2011 survey of over 2,000 individual Canadian DWSSs (serving 300 people or more), found that 84% of the population (i.e. approximately 29 million people) relied on drinking water from public, municipal DWSSs and Pérard (2009) noted that private sector participation levels in regards to drinking water were ‘low or inexistent’ (i.e. less than 10% of the population is supplied drinking water privately) (Statistics Canada, 2010; Statistics Canada, 2011; Murphy et al., 2015; Pérard, 2009).   For the purpose of this paper, the classification system adopted by Health Canada (2013) was used, which identifies ‘large systems’ as DWSSs serving more than 5,000 people, ‘small systems’ as serving between 501 and 5,000 people, ‘very small systems’ as serving between 26 and 500 people, and ‘micro-systems’ as serving up to and including 25 people. DWSSs are generally categorized based on their size (i.e. population served or number of connections), but exact values and thresholds used can vary widely (Table 2-1) (Ford et al., 2005; Corton and Berg, 2009; WHO, 2011). Along with differences between countries, provinces, territories, and states, size-based classification of DWSSs can also vary by organizations within the same country. For example, Ford et al. (2005) highlighted the difference in defining SDWSs between the United States Environmental Protection Agency (USEPA) (i.e. servicing a population less than 3,300 people) and the United States Geological Survey (USGS) (i.e. servicing a population less than 10,000 people).     13  Table 2-1. Sized-based classification of drinking water supply systems Size Classification Health Canada (2013)b (population) New Zealand (2010)c (connections) USEPA( 2011)d (population) USGS (2000)a (population) Large > 5,000 > 10,000 > 100,000 > 10,000 Medium 2,500 - 10,000 3,300 - 100,000 Small 501- 5,000 < 2,500 < 3,300 < 10,000   a Focazio, 2000 b Health Canada, 2013 c Lambert and Taylor, 2010 d USEPA, 2011 Health Canada (2013) established four size-based categories for DWSSs owned or leased by the Federal Government or First Nations communities.   Large systems - serving more than 5,000 people  Small systems - serving between 501 and 5,000 people  Very small systems - serving between 26 and 500 people  Micro-systems - serving up to and including 25 people  Provincially and territorially, the terminology and definitions referring to SDWSs vary as well (Table 2-2). For example in British Columbia, a ‘small water system’ refers to a DWSS serving between 0-500 people, but in Quebec, a ‘small system’ refers to a DWSS serving between 201-1000 people (i.e. with a ‘very small system’ serving between 21-200 people). Although most provinces classify DWSSs by population served, provinces such as Manitoba and Saskatchewan base their classification of DWSSs on service connections and liters of drinking water flow per day, respectively. There are also areas such as New Brunswick with no specifically identified terminology for small systems and other jurisdictions, like the Northwest Territories, where the definition is especially broad, considering a ‘small system’ as the simplest DWSS classified according to complexity and capacity (Pons et al., 2015).    14  Table 2-2. Provincial small drinking water system terminology and defined size (Pons et al., 2015) Province/Territory Terminology Defined size Alberta (AB) Small water system 0-500 people British Columbia (BC) Small water system 0-500 people Manitoba (MB) Semi-public water system 0-15 service connections or a public facility (e.g. school, hospital) with own water supply New Brunswick (NB) Not specifically identified N/A Newfoundland and Labrador (NL) Small system 501-1500 people (very small systems are defined as serving 0-500 people) Northwest Territories (NT) Small system Classified according to complexity and capacity of treatment system with small systems represent the simplest systems. Nova Scotia (NS) Not specifically identified N/A Nunavut (NU) N/A N/A Ontario (ON) Small drinking water system Classified as business or premise that makes drinking water available to the public but does not get drinking water from municipal DWSS Prince Edward Island (PE) Small drinking water system 0-100 people Quebec (QC) Small system 201-1000 people (very small systems are defined as serving 21-200 people) Saskatchewan (SK) Semi-private waterworks < 18,000 liters per day of flow Yukon (YT) Small drinking water system A system other than a large DWSS, that provides drinking water that: a) may have a water source or obtain drinking water from a large DWSS, b) has infrastructure that collects, produces, treats or stores drinking water, and c) has a distribution system with 0-14 service connections or up to 4 delivery sites  For the purpose of this research and throughout this thesis, a SDWS refers to any DWSS serving less than 5,000 people, a medium-sized drinking water system (MDWS) refers to any DWSS serving between 5,000 and 50,000 people, and a large drinking water system (LDWS) refers to any DWSS serving more than 50,000 people.  2.1.2 Small Drinking Water Systems (SDWSs) Throughout much of Canada, DWSSs are generally simple. They intake raw source water from surface sources (i.e. ponds, lakes, rivers, streams) or groundwater and subsequently add a disinfecting agent - usually chlorine, in the form of chlorine gas or hypochlorite, directly to the raw source water to inactivate microorganisms (i.e. primary disinfection) (Health Canada, 2008). In some cases, where provincial regulations make it mandatory, SDWSs supplied by surface water also include treatment processes before primary disinfection (e.g. filtration). Secondary disinfection (i.e. the process of creating residual disinfectant in the distribution system) then   15  occurs to prevent microbial growth within the distribution system (Health Canada, 2008). In some systems, primary and secondary disinfection are combined to both inactivate microorganisms present in the source water and prevent microbial growth within the distribution system. An example diagram illustrating a typical SDWS can be found in Figure 2-2.    Figure 2-2. Example diagram of a small drinking water system  This figure illustrates a typical SDWS utilizing either a groundwater source or surface water source (in provinces/territories with no additional requirements), however, some provinces such as Quebec require additional treatment prior to disinfection of surface source water.  2.1.2.1 Differences between Small Drinking Water Systems and Larger Systems In larger DWSSs, the economies of scale allow for more comprehensive and advanced DWSSs (Scheli et al., 2014). These systems often incorporate multiple levels of treatment technology and administrative controls (Figure 2-3). This can include alternative disinfection methods such as ozonation and ultraviolet (UV) for primary disinfection, and chloramination for secondary   16  disinfection with other engineered steps such as screening, coagulation/flocculation, sedimentation, and/or slow sand filtration (SSF) for ensuring safe drinking water. They can also employ additional steps as part of the distribution network, such as chlorine boosters to ensure proper levels of residual disinfectant (i.e. enough to stymie microbial growth, but not enough to encourage high levels of disinfection by-products (DBPs)) throughout the entire system (Islam et al., 2013a). An example showing the potential complexity of a medium to large-sized DWSS can be found in Figure 2-3.   Figure 2-3. Example diagram of a medium to large-sized drinking water system  Differences between SDWSs and MDWSs/LDWSs are not just technological. Larger DWSSs also have more financial (i.e. larger tax base) and human resource backing (i.e. more, better trained operators) and are less prone to the impacts of spatiotemporal variability [Section 2.1.2.2] (Haider et al., 2015a; Scheili et al., 2014).     17  2.1.2.2 Spatiotemporal Variability Spatiotemporal variability of drinking water management in the context of this research refers to water quality variation caused by spatial factors (i.e. different locations in a DWSS) and temporal factors (i.e. seasonal differences). These problems have been highlighted in literature for LDWSs and MDWSs and are often addressed with improved management strategies or engineered technologies (Scheili et al., 2014). However, the lack of resources for mitigation options in SDWSs can amplify the problems created by spatiotemporal variability.   Geographically, Canada faces some of the most seasonally variable weather throughout the world (Dyck et al., 2014). Almost 90% of treated drinking water by volume in Canada is processed from surface sources (Statistics Canada, 2011). This represents a water supply exposed to the environment and therefore extremely vulnerable to temporal changes resulting from precipitation, runoff/erosion, temperature, and land use (Dyck et al., 2014; Scheili et al., 2014). From a temporal standpoint, previous research by Rodriguez and Sérodes (2001), Coulibaly and Rodriguez (2003), Symanski et al. (2004), Mercier Shanks et al. (2013), Lee et al. (2013), Guilherme and Rodriguez (2014), Scheili et al. (2014), and Charisiadis et al. (2016) found that seasonal variability had a significant impact on drinking water quality.   Spatially, maintaining consistent drinking water quality throughout an entire DWSS is a difficult proposition. Previous studies by Rodriguez and Sérodes (2001), Hinckley et al. (2005), Coulibaly and Rodriguez (2003), Mercier Shanks et al. (2013), Lee et al. (2013), and Scheili et al. (2014) have identified significant spatial variation within DWSSs related to drinking water quality. In SDWSs, there are fewer consumers, resulting in less water demand (often spread further apart   18  than in LDWSs or MDWSs) and thus, there could be higher residency times associated with the drinking water in the distribution system. These factors, along with less sophisticated treatment technologies (including a lack of chlorination boosters), can result in significant water quality variation within a singular DWSS (Symanski et al., 2004; Islam et al., 2013b; Lee et al., 2013; Charisiadis et al., 2016).  2.1.3 Performance in a Drinking Water Context Drinking water utilities are responsible for providing access to safe and reliable drinking water from source to tap (CCME, 2004). Safe drinking water represents a consistent water supply that does not pose a significant risk to public health over a lifetime of consumption (WHO, 2011). To meet these expectations, regulatory drinking water quality guidelines/standards (e.g. Guidelines for Canadian Drinking Water Quality (GCDWQ), US National Primary Drinking Water Regulations, US National Secondary Drinking Water Regulations) have been developed and implemented throughout the world. This approach to ensuring safe drinking water measures the performance of a drinking water utility by comparing individual water quality performance indicators (PIs) (e.g. turbidity, E. coli counts, etc.) from treated drinking water against a designated value, usually developed from human health targets. Based on this comparison, drinking water quality is determined either as acceptable (i.e. below the defined guideline/standard or within the acceptable range) or as an exceedance (i.e. above the defined guideline/standard or outside of the acceptable range). When water quality fails to meet targets, drinking water advisories (DWAs) can be enacted to protect consumers.    19  2.1.3.1 Drinking Water Advisories Health Canada (2016) defines DWAs as “…preventative measures put in place to protect public health from drinking water that could be contaminated.” They list four different types of DWAs:  Boil water advisories/orders (BWAs/BWOs) for microbial contamination  Do not consume advisories/orders (DNCAs/DNCOs), also known as do not drink advisories/orders (DNDAs/DNDOs), when water is not safe for consumption and boiling the water will not remove the contaminant (i.e. chemical contamination)  Do not use advisories/orders (DNUAs/DNUOs) , when water is not safe for any use and boiling the water will not remove the contaminant  Water avoidance advisories, which typically follow natural disasters (or other catastrophic events) and drinking water or its source may be contaminated  DWAs are put in place for various reasons including equipment failures, maintenance issues, and water quality exceedances, but can also be introduced as precautionary measures. Eggertson (2008) found that there were a total of 1766 BWAs across Canada and 93 BWAs in place in First Nations communities. Table 2-3 shows a breakdown of the active BWAs by province/territory as of February 29, 2008.   While more recent, specific data is unavailable on active BWAs by province, Environment and Climate Change Canada (2016) reported that in 2015, more than 95% of all BWAs occurred in SDWSs, with 79% occurring in DWSSs serving 500 people or less. Figure 2-4 shows the BWAs in place by community size from 2010-2015.     20  Table 2-3. Boil water advisories by province/territory in Canada as of February 29, 2008 (Eggertson, 2008) Province/Territory Active boil water advisories  Alberta (AB) 13 British Columbia (BC) 530 Manitoba (MB) 59 New Brunswick (NB) 2 Newfoundland and Labrador (NL) 228 Northwest Territories (NT) 1 Nova Scotia (NS) 67 Nunavut (NU) 0 Ontario (ON) 679 Prince Edward Island (PE) 0 Quebec (QC) 61 Saskatchewan (SK) 126 Yukon (YT) 0   Figure 2-4. Boil water advisories in Canada by community size from 2010-2015 (Environment and Climate Change Canada, 2016)  As is evident in Figure 2-4, the number of BWAs remained consistent from 2010 to 2015. Water quality specific reasons account for less than half of all BWAs (with the rest occurring from equipment failure, water quantity issues, or unreported problems). Figure 2-5 shows the percentage of water quality-related BWAs from E.coli, total coliforms (TC), turbidity, source water quality, or other water quality (WQ) exceedances.   21   Figure 2-5. Water quality related reasons for boil water advisories in Canada from 2010-2015 (Environment and Climate Change Canada, 2016)  Based on the information from Figure 2-4 and Figure 2-5, it is clear that SDWSs are in drastic need of improvement and overall, drinking water performance in Canada is stagnating. While technological improvement of individual DWSSs are important in ensuring safe, consistent drinking water and for adapting to consumer demands, it represents corrective (i.e. reactive) measures. Alternatively, improving drinking water management as a whole provides a preventative (i.e. proactive) approach that can cost-effectively improve drinking water performance in individual DWSSs and across Canada by improving and/or supplementing the performance assessment and decision-making mechanisms currently in place.    22  2.2 Drinking Water Management Systems The WHO (2011) states, “Preventative management is the preferred approach to ensuring drinking water safety and should take account of the characteristics of the drinking water supply from catchment and source to its use by consumers.” DWMSs are defined as systems of policies, procedures, and administrative/behavioral controls designed to ensure safe drinking water from source to tap. They are designed not only for achieving regulatory compliance, but also to meet the need for preventative management of DWSSs and to incentivize continuous performance improvement. A study by Baum et al., (2015) stated, “Over 35 countries worldwide have multiple water systems that have well-documented cases of either voluntarily or mandatorily implemented WSPs, or their equivalent under other names, that served as a preventative risk management approach in an effort to ensure the safety of drinking water quality.” While there may be some countries with less-documented approaches to drinking water management implemented, that still leaves dozens of countries and hundreds of millions of people depending on under-protected and potentially vulnerable DWSSs.   DWMSs provide the foundation and framework for ensuring safe drinking water and have historically taken many different forms, with elements incorporated from safe food handling practices to industrial quality management applications (Sinclair and Rizak, 2004; Yokoi; 2006; Jayaratne, 2008; Gunnarsdóttir and Gissurarson, 2008). This varied approach is especially evident in Canada, a country with a decentralized governance structure where provincial and territorial governments are mostly autonomous in the context of drinking water management and governance [Chapter 5].     23  To fully understand drinking water management and drinking water management in Canada, a review of widely applied QMSs (i.e. formal systems of procedures and policies designed to ensure quality objectives are met, documented, and improved upon) and emulated national DWMSs was conducted and compared to the existing drinking water management approaches being used throughout Canada (ISO, 2016). While the effectiveness and impact of management practices and principles can be difficult to quantify, Hrudey (2011) noted, “…much of Canada remains out of step with the international leaders in adopting management systems for assuring safe drinking.” By comparing the Canadian state of practice and included drinking water management elements against WHO WSP recommendations, international leaders, and widely applied QMSs, context can be provided to gauge the comprehensiveness or lack thereof in Canadian national, provincial, and territorial DWMSs.  It is important to note that jurisdictional and/or organizational definitions and interpretations of what constitutes a DWMS vary widely, much like the approaches themselves. In order to properly review and fully understand the existing approaches to drinking water management and quality management across provinces/territories, countries, and differing applications, jurisdictional/organizational definitions and interpretations were used in Section 2.2.1, 2.2.2, and 2.2.3. However, when comparing the selected DWMSs and QMSs in Section 2.3, common elements with more general terminology were selected for standardization and ease of comparison.    24  2.2.1 Quality Management Systems Two generalized QMSs were selected as part of this review to highlight the differences between more traditional quality management approaches and more specific DWMSs. The Hazard Analysis and Critical Control Points (HACCP) framework and the International Organization for Standardization (ISO) 9001:2015 Quality Management System Requirements were selected for comparison as they have diverse applications, are often associated with food and drink, and a review of applicable literature highlighted these approaches as foundations for many existing DWMSs (Martinez-Costa et al., 2009; Sinclair and Rizak, 2004; Kafetzopoulos et al., 2013).  2.2.1.1 Hazard Analysis and Critical Control Points The HACCP QMS is a risk-based framework for ensuring quality management. It was originally developed for use in food production, manufacture, and distribution, and was designed to encourage safe practices and incorporate transparency (Kafetzopoulos et al., 2013). The HACCP system is generally defined as a systematic framework for the identification, assessment, and control of hazards and was developed as a proactive alternative to end-point testing (Ropkins and Beck, 2000). It is widely recognized as the best framework for assuring product safety and controlling safety hazards (Kafetzopoulos et al., 2013). Historically, the basis for the first HACCP system was first developed in the late 1960s by The Pillsbury Company to ensure food safety for space flights (Ropkins and Beck, 2000). In 1972, it was applied to their commercial division for the manufacture of consumer food products and consisted of three primary principles (Ropkins and Beck, 2000; Sperber, 2005). 1. Conduct hazard analysis to identify and assess hazards associated with the final product   25  2. Determine critical control points (CCPs) – the steps or stages within production at which hazards may be controlled, reduced or eliminated 3. Monitoring – observation and monitoring of established CCPs  In 1997, the modern HACCP system was redeveloped to meet the specific needs of food producers, manufacturers, and distributors (Ropkins and Beck, 2000; Sperber, 2005). It was based on twelve principles and included an emphasis placed on documentation (Table 2-4).  Table 2-4. Modern HACCP principles (Ropkins and Beck, 2000; Sperber, 2005; Jayaratne, 2008) No. Step Description  1 Assemble HACCP team Assemble a team to develop, implement, and verify the HACCP system. 2 Describe product Develop a detailed description of the product, including water quality standards and/or health-based targets. 3 Identify intended use Identify use (i.e. human consumption). 4 Construct flow diagram Identification (and development of a flow diagram) of all elements in a system from beginning to end. 5 Confirm flow diagram Confirmation of flow diagram developed in Step 4. 6 Conduct a hazard analysis Conduct hazard analysis to identify and assess hazards associated with the final product. 7 Determine CCPs Determine CCPs - the steps or stages within production at which hazards may be controlled, reduced, or eliminated. 8 Establish critical limits Establish critical limits for controlling each CCP. 9 Establish a system to monitor control of the CCPs Establish monitoring procedures to determine if limits have been exceeded. 10 Establish corrective actions Establish corrective actions to be taken if control is lost and define procedures for maintaining control. 11 Validation and verification of the HACCP plan Establish verification procedures for assessing the effectiveness of the HACCP system. 12 Establish documentation and record keeping Establish documentation and record keeping procedures to provide proof of compliance.  An HACCP-based DWMS has been used in Iceland since 1995 when legislation classified water as a food and required management strategies to prevent contamination (Gunnarsdóttir, 2012; Hulsmann and Smeets, 2011). Since its inception, the HACCP system has resulted in water quality performance increases throughout the country and has led to the development of a mini-HACCP process tailored to SDWSs for water utilities between 500-5000 people (Jayaratne, 2008; Gunnarsdóttir and Gissurarson, 2008; Gunnarsdóttir, 2012). This QMS has also been   26  implemented in Slovenia where, much like in Iceland, drinking water is covered under food legislation (Hulsmann and Smeets, 2011).  2.2.1.2 ISO 9001:2015 Quality Management System Requirements The ISO 9001:2015 standard is a process-based approach, based on the PDCA cycle, designed for use by an organization that needs to demonstrate its ability to consistently provide products and/or services that meet customer and regulatory requirements (ISO, 2015). ISO (2015) states, “All of the requirements of this International Standard are generic and are intended to be applicable to any organization, regardless of its type or size, or the products and services it provides,” and since 1987 the ISO 9001 standard has been used by many organizations in many industries throughout the world (Cianfrani and West, 2014). The ISO 9001:2015 standard is broken down into seven quality management principles.  Customer focus  Leadership  Engagement of people  Process approach  Improvement  Evidence-based decision making  Relationship management  Although ISO 9001 was originally intended for manufacturing companies, it has since been used as a foundation (and/or complimentary piece) of DWMSs, such as the Australian Framework for Management of Drinking Water Quality (Martinez-Costa et al., 2009; Sinclair and Rizak, 2004).   27  This can be attributed to the fact that the ISO 9001 standard offers a reasonable pathway for implementing quality management and requires a high level of documentation and auditing, leading to increased transparency and quality verification (Martinez-Costa et al., 2009; Terziovski et al., 2003).  2.2.2 Drinking Water Management Systems Four overarching DWMSs were chosen from Australia, Canada, New Zealand, and the WHO for this investigation and represent some of the more studied and replicated DWMSs found throughout the world (Hrudey et al., 2006). It is important to note that the US was not included as part of this review as DWMSs are generally handled at the state level (e.g. the California Safe Drinking Water Plan), with the United States Environmental Protection Agency (USEPA) having federal oversight over issues related to the Safe Drinking Water Act, specifically water quality regulations including the Lead and Copper Rule, the Surface Water Treatment Rule, the Total Coliform Rule, and the Disinfectants/Disinfection By-Products Rule (NRC, 2006). While the USEPA does not have a recommended DWMS, they have promoted the WHO WSP approach for international partner countries to improve drinking water quality since 1998 (USEPA, 2016). DWMS in the EU were also not incorporated due to difficulty in finding the necessary specific information to gauge comprehensiveness across the selected management elements.  2.2.2.1 Australia The Australian Framework for Management of Drinking Water Quality was developed by the Australian National Health and Medical Research Council (NHMRC) in collaboration with the   28  Co-operative Research Centre for Water Quality and Treatment to incorporate preventative risk management in a drinking water supply context to support consistent and comprehensive implementation by suppliers (NHMRC, 2011; Sinclair and Rizak, 2004). This framework incorporates elements of HACCP, ISO 9001 (Quality Management), ISO 14001 (Environment Management), and Australia/New Zealand Standard (AS/NZS) 4360:2004 (Risk Management) for the management of drinking water quality from source to tap (NHMRC, 2011; Sinclair and Rizak, 2004). The framework addresses four general areas (Table 2-5) and includes 12 elements considered good practice for system management of drinking water supplies (Table 2-6).  Table 2-5. Australian Framework for Management of Drinking Water Quality general areas (NHMRC, 2011) No. General Area Description 1 Commitment to drinking water quality management The development of a commitment (active participation) to drinking water quality management within an organization.  2 System analysis and management The understanding of an entire water supply system, the hazards and events that can comprise drinking water quality, and the preventative measures and operational control necessary for assuring safe and reliable drinking water. 3 Supporting requirements Requirements include basic elements of good practice such as employee training, community involvement, research and development, validation of process efficacy, and systems for documentation and reporting. 4 Review Evaluation and audit processes and their review by senior executive to ensure the management system is functioning satisfactorily. These components provide a basis for review and continual improvement.             29  Table 2-6. Process for Australian Framework for Management of Drinking Water Quality (NHMRC, 2011) No. Step Description 1 Commitment to drinking water quality management Organizational support and long-term commitment, in the form of drinking water quality policy, regulatory and formal requirements, and engaging stakeholders, serves as a foundation in implementing an effective DWMS. 2 Assessment of the drinking water supply system Assessment of the drinking water supply system includes a full water supply system analysis, assessment of water quality data, and hazard identification and risk assessment. 3 Preventative measures for drinking water quality management Preventative measures for drinking water quality management includes the use of multiple barriers and CCPs. These measures are used to prevent hazards from occurring or reducing them to acceptable levels. 4 Operational procedures and process control Operational procedures and process control measures are designed to achieve a high-quality water supply and effectively control processes and activities that govern drinking water quality.  5 Verification of drinking water quality Verification of drinking water quality and consumer satisfaction provides an assessment of overall performance of the system. It also provides a useful indication of problems within the water supply system and the necessity for any immediate corrective actions or incident and emergency response. 6 Management of incidents and emergencies Management of incidents and emergencies is essential for protecting public health and maintaining consumer confidence. Whenever possible, emergency scenarios should be identified and incident and emergency protocols should be planned and documented. 7 Employee awareness and training Employee awareness and training is essential to enable and motivate employees to make effective decisions. 8 Community involvement and awareness Community involvement and awareness can beneficially impact public confidence in the water supply by providing transparency and education. 9 Research and development Research and development includes research monitoring, validation of processes, and design of equipment and helps to ensure continual improvement. 10 Documentation and reporting Proper documentation and reporting provides the foundation for the establishment and maintenance of effective DWMSs. 11 Evaluation and audit Long-term evaluation of drinking water quality results and auditing of a DWMS is required to determine efficiency of preventative strategies. 12 Review and continual improvement Review by senior executives can lead to continual improvement of the DWMS.   As part of the Australian Framework, the NHMRC (2011) developed a guidance approach specifically for SDWSs serving populations less than 1,000 people. This modified methodology places an emphasis on a preventative approach to managing water quality with less of a focus on water quality testing and monitoring programs. It consists of four main components.  Assessment of the drinking water supply  Preventative measures for drinking water quality management  Implementation of operational procedures and process control  Verification of drinking water quality  2.2.2.2 Canada Drinking water management in Canada has dramatically shifted since the two major waterborne disease outbreaks in the early 2000s (O’Connor, 2002b; Hrudey et al., 2003). Subsequent   30  investigations resulted in the establishment of the Canadian Council of Ministers of the Environment (CCME) Multi-barrier Approach (MBA) for managing drinking water throughout Canada (O’Connor, 2002b; CCME, 2004). In this approach, barriers, in the form of physical or administrative/behavioral improvements, are implemented to improve the overall quality and management of drinking water (CCME, 2004; Hrudey et al., 2006; Alberta Environment; 2009). In the event one barrier fails, back-up systems and processes are in place to protect the safety of drinking water (GNWT, 2005).   Using the CCME MBA, all potential control barriers are identified along with potential limitations (CCME, 2002). The CCME MBA is categorized into three main components: the source, drinking water treatment, and drinking water distribution (CCME, 2002).   Water source (i.e. source water protection) – the coordinated approach to develop plans (short-term and long-term) to protect and potentially enhance drinking water source quality. This component can be broken down further into three categories; delineating source water protection areas, identifying contaminants of concern, and assessing and ranking risk vulnerability.  Treatment system (i.e. water treatment process selection) – the selection of drinking water treatment based on source water quality and quantity, finished water quality, reliability of equipment, regulatory requirements, and human resources/financial demands.  Distribution system (i.e. distribution system development and operation)– the designing and operating of a drinking water distribution system to sustain minimum operating pressure at the maximum hourly flows and meet or exceed pipe performance standards.   31  These elements are then addressed by using a system of procedures and tools (Table 2-7).  Table 2-7. Canadian Council of Ministers of the Environment multi-barrier approach elements and description (CCME, 2002) No. Element Description 1 Legislative and policy frameworks Legislative and policy frameworks highlight responsibilities for each aspect of the drinking water system and should be reviewed and revised as necessary. 2 Public involvement and awareness Public involvement and awareness include appropriate levels of partnership and communication among stakeholders to increase transparency and availability of public health information. 3 Guidelines, standards, and objectives Regulations provide utility managers and system owners with water quality targets to meet and can be used as part of the decision-making process. 4 Research, science, and technology Research, disease surveillance, and other scientific and technological advancement/development allow for more integrated water quality monitoring and the potential for improving operations. 5 Management  Drinking water supply management requires the cooperation of stakeholders in different fields (e.g. health, environment, industry) and requires qualified personnel to ensure treatment facility and distribution system are operating at optimum levels. 6 Monitoring Water quality monitoring includes the sampling of water quality at the source, after treatment, and within the distribution network. This allows operators to modify treatment if water quality fluctuates to ensure regulatory compliance and safe drinking water. 7 Source water protection and management Protection of source water based on watershed management involving a coordinated approach among stakeholders to develop short and long-term plans to prevent, minimize, or control potential sources of pollution or enhance water quality. 8 Drinking water treatment Drinking water treatment is key to eliminating pathogens and chemical substances found in source waters. They should be regularly reviewed and upgrade as necessary. 9 Drinking water distribution systems Distribution systems are the final physical barrier in the multi-barrier approach. After water is treated, its quality must be maintained throughout the distribution system.  The CCME (2004) states, “The benefits associated with implementing a multi-barrier approach could include better public health protection, a reduction in healthcare costs, better management of water treatment costs, and indirectly, increased environmental protection.” Other benefits listed by the CCME (2004) include better and more effective communication with stakeholders and the public in regards to drinking water, better protected source water, ongoing education and training of DWSS operators, better maintained and funded DWSSs, and the capacity to better handle emergencies (CCME, 2004)  2.2.2.3 New Zealand The New Zealand Ministry of Health (NZMOH) is responsible for the regulation of public health under the Health Act 1956 and subsequent amendments (NZMOH, 2015). The Health (Drinking   32  Water) Amendment Act 2007 to the Health Act 1956 mandated that all water suppliers have a duty to ensure safe drinking water (NZMOH, 2008). This amended act also required the development and implementation of a WSP (originally known as a Public Health Risk Management Plan (PHRMP)) for all drinking water suppliers serving over 500 people and is complemented by the Drinking-water Standards for New Zealand (DWSNZ) (NZMOH, 2008; Hubbert, 2013; Hrudey; 2011).  The NZMOH (2015) integrated DWMS is based on the WHO (2004) drinking water quality guidelines and quality assurance principles (Table 2-8) (Taylor, 2002; Hubbert, 2013). It is designed to promote interaction and support throughout the entire drinking water supply system from the drinking-water supplier and the public health officers to other stakeholders and the public (Taylor, 2002).             33  Table 2-8. New Zealand Ministry of Health suggested steps for water safety plan development (NZMOH, 2015) No. Step  Description 1 Produce overview of supply Identification (and development of a flow diagram) of all elements of a water supply system from the catchment to the consumer’s property. 2 Identify barriers to contamination Inventory of protective barriers in place to contribute to the safety of the entire water supply. These fundamental barriers must achieve the following: 1. Prevention of contaminants entering the raw water of the supply. 2. Removal of particles from the water. 3. Inactivation of microorganisms in the water. 4. Maintenance of the quality of the water during distribution. 3 Identify events that may introduce hazards Identification of potential events that may introduce hazards into the drinking water supply using NZMOH Water Safety Plan Guides.  4 Identify possible causes of each event, preventative measures, and corrective actions Identification of causes, preventative measures, and corrective actions (using NZMOH Water Safety Plan Guides) associated with the identified hazards from Step 3. 5 Decide where improvements should be made This is the first of three steps for preparing an improvement schedule, which is designed to list any of the four fundamental barriers, preventative measures, checks, or corrective actions missing from a supply. 6 Decide on order of improvements Prioritization of improvements based on the improvements identified in Step 5. Factors such as public health impact (using NZMOH Water Safety Plan Guides), availability of resources, and ease with which improvements can be implemented must be considered. 7 Draw up timetable Development of an improvement schedule to assign completion dates and responsibility for each improvement. 8 Identify links with other quality assurance systems Identification of other quality assurance systems in place (such as ISO 9000/14000) and implementation into WSP. 9 Develop contingency plan Preparation of contingency plans (suggested contingency plans are provided in each NZMOH Water Safety Plan Guides) to ensure there is a protocol for situations that may pose a threat to the drinking water quality. 10 Performance assessment of plans Development of a procedure for the review and updating of the WSP. Reasons to update a plan may include: a change in the circumstances of a water supply, the identification of possible new events and their causes, the discovery that one or more preventative measures or corrective actions are unsatisfactory, and/or a contingency plan has failed when implemented. 11 Development of communication policy Creation of a communication policy should identify and record the people to whom reports concerning the management of risk to the supply should be made, what information these reports should contain, and how often they should be made. 12 Review and improvement Review and improvement of WSP process.  In small communities serving between 25-5000 people, the NZMOH has developed the Drinking Water Assistance Programme (DWAP) designed to meet the technical and financial needs of small suppliers and has also developed a risk management kit for small drinking water suppliers (NZMOH, 2015).  2.2.2.4 World Health Organization Water Safety Plan WSPs represent an alternative framework to many current DWMSs. While conventional DWMSs often rely on mitigating risks already present in a water supply system, a WSP approach focuses on preventing risks from entering and therefore reducing the likelihood of a negative impact on human health (Bartram et al., 2009). Based on WHO water quality guidelines, Bartram   34  et al. (2001) developed one of the first water quality management frameworks to incorporate public health concerns, risk assessment, the establishment of health-based targets, and risk management. This approach to managing drinking water quality provided a foundation for many current DWMSs and since 2004, a WSP approach has been recommended by the WHO for preventative management of water supplies regardless of size or level of sophistication (WHO, 2004; Yokoi, 2006).  In the Fourth Edition of the Guidelines for Drinking-water Quality, the WHO (2011) promotes a 10-step WSP, which build on a foundation of multiple barriers, the HACCP QMS system, and other systematic management frameworks/approaches. The WHO (2009), also published the Water Safety Plan Manual (based on Chapter 4 of the 2004 Third Edition of the Guidelines for Drinking-water Quality), which provides a step-by-step guide for drinking water suppliers to develop and implement a WSP. The WHO WSP approach consists of five key elements (Table 2-9) and three key components (Table 2-9, No. 2-4) (Davison et al., 2006).  Table 2-9. World Health Organization Water Safety Plan elements and description (Davison et al., 2006; WHO, 2011) No. Element Description  1 Setting health-based targets Health-based targets are based on an evaluation of health concerns and can be coordinated with other regulatory guidelines and standards. 2 System assessment A system assessment is used to determine whether or not the drinking water supply chain (source to tap) can deliver water quality that meets identified targets. This also includes assessment of design criteria of new systems. 3 Effective operational monitoring Effective operational monitoring refers to the identification of control measures in a drinking water system that will control identified risks and ensure that health-based targets are met. 4 Management and communication Management and communication refer to actions to be conducted during normal operation or incident conditions and documenting the system assessment, including upgrade and improvement planning, monitoring and communication plans, and supporting programs. 5 Surveillance Independent monitoring verifies the above components are operating properly and effectively.  Based on these elements, the WHO (2011) defined six primary objectives of a WSP in ensuring safe drinking water.   35  1. Development of an understanding of specific systems and its capability to supply water that meets water quality targets 2. Identification of potential sources of contamination and how they can be controlled 3. Validation of control measures employed to control hazards 4. Implementation of a system for operation monitoring of the control measures within the water system 5. Timely corrective actions to ensure that safe water is consistently supplied 6. Verification of drinking water quality to ensure that the WSP is being implemented correctly and is achieving the performance required to meet relevant national, regional, and local water quality standards or objectives  2.2.3 Canadian Provincial/Territorial Drinking Water Management Approaches In Canada, the CCME recommends a MBA for providing safe drinking water but allows significant autonomy and flexibility for individual provinces and territories resulting in vastly different approaches to drinking water management (Chapter 5). These differences can largely be attributed to vastly different populations, water usage, and source types across provinces/territories (Figure 2-6).     36   Figure 2-6. Potable water volumes processed by drinking water plants in Canada by source water type in 2011 from each province/territory (Statistics Canada, 2011)  Figure 2-6 highlights the differences between potable water volumes processed and source water type across all Canadian provinces and territories. This information is critical in understanding the approach to drinking water management in each province/territory. For example, in provinces with higher volumes of groundwater and groundwater under the direct influence of surface water (GUDI) (e.g. Quebec, Ontario), it could be expected that more information on the management of these sources would be available.   2.2.3.1 Alberta  The province of Alberta became the first jurisdiction to regulate WSPs in North America, with the implementation of the Drinking Water Safety Plan (DWSP) in 2011, moving away from the source to tap CCME MBA (Perrier et al., 2014; Reid et al., 2014; Alberta Environment, 2009).   37  The development of the DWSP was designed to overcome limitations associated with the MBA (Reid et al., 2014). The Government of Alberta (2012) established four principle processes for an effective DWSP (Table 2-10).  Table 2-10. Principle processes for Alberta Drinking Water Safety Plan (Government of Alberta, 2012) No. Principle Processes Description 1 Collection of best available information about the water supply system Collection of information about a water supply system will vary by complexity, but typical sources of information can include water quality records, public health incidents, plant records, and watershed activities.  2 Identification of present risks and circumstances that will threaten public health Identification of risks and circumstances that will threaten safe drinking water can be conducted using risk sheets (e.g. source, treatment, network, and consumer) as part of the Government of Alberta DWSP template. Other site-specific risks and causes may need to be added. 3 Assessment of improvement actions to reduce risk to an acceptable level Assessment of improvement actions can be addressed using the Government of Alberta Action Summary Sheet and Key Risks Sheet. 4 Inventory of available resources, prioritization (and auditing) of improvement actions, and timetable for implementation of improvements Inventory of available resources, prioritization of improvement actions, and creation of a timetable for implementation of improvements can be addressed using the Government of Alberta Action Summary Sheet to address the identified improvement actions from Principle 3.  The Government of Alberta (2012), also lists three other important considerations for DWSPs. 1. A DWSP cannot work in isolation, so you must communicate and discuss findings with stakeholders and other relevant parties. 2. For the DWSP to work, the identified necessary actions to mitigate risks must be implemented. 3. The DWSP is a ‘living document’ that should be reviewed regularly and updated when necessary.  Based on these principles and considerations, the Alberta Environment and Sustainable Resource Development (ESRD) developed a Microsoft Excel template, which provides generic risk for four key risk areas (i.e. source, treatment, distribution network, consumer), designed to act as a single source for all relevant information about a water supply system (Reid et al., 2014; Perrier et al., 2015). While the long-term success of this program is still uncertain, research has shown it   38  presents a practical option for effective water management and has potential to be applied throughout Canada (Gagnon, 2014). The Alberta DWSP does not provide a specific guidance document or procedures for small utilities.  2.2.3.2 British Columbia The British Columbia Comprehensive Drinking Water Source-to-Tap Assessment (BC CS2TA) is an assessment program designed to integrate the various components of the water supply system by determining strengths and weaknesses and identifying existing and potential threats to safe drinking water to assist in the decision-making process (BCMHLS, 2010). It was developed based on ten guiding principles and consists of eight modules (Table 2-11): 1. Drinking water protection is a public health issue, hence drinking water assessments should focus on threats to public health. 2. Drinking water assessments should be a tool to assist in the protection of drinking water. 3. Drinking water assessments should be conducted in an integrated manner, with consideration for both source and system components. 4. Drinking water assessments should embody the MBA. 5. Drinking water assessments should be an opportunity for education and communication among stakeholders. 6. Drinking water assessments should be focused on preventing problems. 7. Drinking water assessments should be science based. 8. Drinking water source assessments should be flexible and tailored to the size and type of the water system and the level of risk to its users.   39  9. Drinking water assessments should result in the development and implementation of specific actions and/or recommendations. 10. Drinking water assessments should foster and promote the highest water quality possible through stewardship and involvement of the broader community.  Table 2-11. British Columbia CS2TA modules (BCMHLS, 2010) No. Module Description 1 Delineate and characterize drinking water source(s) The delineation and characterization of the drinking water source area allow for the evaluation of the integrity and location of surface water intakes and ground water wells. 2 Conduct contaminant source inventory A contaminant source inventory is to be used to identify and describe land uses, human activities, and other potential contaminant sources that could potentially affect source water quality. 3 Assess water supply elements The assessment of water supply elements includes the identification of factors, such as source water type, water quality and quantity, size of the population served, and age of the system, and is designed to be used to identify problems or concerns. 4 Evaluate water system management, operation, and maintenance Water system management, operation, and maintenance provide a systematic approach for investigating the human aspect of how the water system is operated to provide safe drinking water. 5 Audit water quality and availability Auditing of water quality and quantity provides an evaluation of the success of the water system in meeting the goal of providing safe, palatable drinking water to all users and encompasses a review of water quality monitoring and customer feedback. 6 Review financial capacity and governance of water systems Reviewing of the financial capacity and governance of a water system includes a review of the financial management of the water system, available funding mechanisms, governance and accountability, and the response to development pressures. 7 Characterize risks from source to tap The characterization of risks from source to tap is the focal point of the CS2TA program. It includes a structured approach for identifying the areas of greatest risk and is designed to foster an understanding of the strengths and weaknesses throughout a water supply system. 8 Recommend actions to improve drinking water protection The recommendation of actions to effectively manage the risks identified throughout the assessment process is important to enhance the safety and sustainability of the drinking water supply.   The BC CS2TA was developed not as a prescriptive assessment methodology or a set of detailed instructions, but a flexible methodology for use in identifying and evaluating drinking water risks from source to tap (BCMHLS, 2010). While there are no specific approaches for smaller systems, the BCMHLS (2010), was designed to be adaptable for water supply systems of all types and sizes.  2.2.3.3 Manitoba  Drinking water in Manitoba is governed by the Manitoba Water Stewardship’s Office of Drinking Water, which enforces The Drinking Water Safety Act and the Manitoba Water Quality   40  Standards, Objectives, and Guidelines (2011) (Manitoba Water Stewardship, 2011a; Manitoba Water Stewardship, 2011b). The province also helps to protect drinking water at the source using The Water Protection Act (Legislative Assembly of Manitoba, 2005; Manitoba Water Stewardship, 2011a). Combined, the legislative structure provides a source-to-tap framework for the protection of provincial drinking water system (Manitoba Water Stewardship, 2011a).   In 2003, Manitoba Water Stewardship released The Manitoba Water Strategy, which documented the importance of the development of an integrated water planning and management system. Eight years later, the Legislative Assembly of Manitoba introduced the Save Lake Winnipeg Act, a bill requiring planning authorities in the Capital Region to prepare a DWP as part of a development plan review or major amendment (Legislative Assembly of Manitoba, 2011). While this act is not province-wide and places an emphasis on water supply over water quality, it provides the closest thing to a DWMS currently operational in Manitoba (Manitoba Provincial Planning Regulation, n.d.). Manitoba does not have a specific DWMS approach for smaller utilities, however, it is important to note that the Manitoba Water Stewardship (2007), provides a document, Best Practices Manual for Small Drinking Water Systems, to assist SDWSs in achieving regulatory compliance and meeting management and operational challenges.   2.2.3.4 New Brunswick The population of New Brunswick is generally rural, with about 40% of the population obtaining drinking water from domestic groundwater wells (Government of New Brunswick, 2009). The remaining 60% of the population relies on municipal DWSSs (Government of New Brunswick, 2009). The New Brunswick Department of Health and Department of the Environment and Local   41  Government cooperatively lead the regulation of public drinking water supplies with the Department of Health holding responsibility for assessing public health risk and the Department of the Environment and Local Government responsible for regulating and protecting drinking water systems. Despite having a population of 756,800 people and strong drinking water legislation in place, New Brunswick is one of the only provinces without an established approach to drinking water management (Conservation Council of New Brunswick, 2016; Statistics Canada, 2016).  2.2.3.5 Newfoundland and Labrador  The Multi-barrier Strategic Action Plan (MBSAP) is used in Newfoundland and Labrador for managing drinking water quality (Government of Newfoundland and Labrador, 2014). The Government of Newfoundland and Labrador (2014) states, “The MBSAP is considered to be the most effective method of managing drinking water system and has been implemented by other jurisdictions throughout Canada.” The Government of Newfoundland and Labrador (2014) has broken down the MBSAP into three levels of components, with Level 1 focused on operations, Level 2 focused on management, and Level 3 focused on legislation and supporting programs (Table 2-12).  Table 2-12. Levels of the MBSAP in Newfoundland and Labrador (Government of Newfoundland and Labrador, 2014) Level 1 Level 2 Level 3  Drinking water distribution  Drinking water treatment  Source water protection    Corrective measures  Data management and reporting  Inspection and enforcement  Monitoring  Operator education and certification  Guidelines, standards, and objectives  Legislative and policy framework  Public involvement and awareness  Research and development    42  The implementation of the MBSAP involves collaboration between four provincial government departments; Environment and Conservation, Government Services, Health and Community Services, and Municipal Affairs (Government of Newfoundland and Labrador, 2010). Despite over 90% of all DWSS serving 1,500 people or less, Newfoundland and Labrador do not have a modified DWMS for small systems.  2.2.3.6 Northwest Territories  Drinking water management in the Northwest Territories has progressed further than in both Nunavut (NU) and the Yukon (YT). Despite the fact there are only 34 public water systems in the Northwest Territories, the waterborne disease outbreaks in Walkerton, Ontario and North Battleford, Saskatchewan pushed the territory to begin development of a DWMS in 2001 (Government of the Northwest Territories, 2005). The Northwest Territories Safe Drinking Water Framework and Strategy is based on a three-layered MBA designed to incorporate all levels of governments (Government of the Northwest Territories, 2014).  The barriers are broken down into the following categories (Table 2-13). 1. Keeping the Northwest Territories water clean 2. Making drinking water safe 3. Proving drinking water is safe      43  Table 2-13. Northwest Territories Safe Drinking Water Framework and Strategy categories and barriers (GNWT, 2014) Keeping NT Water Clean Making Drinking Water Safe Proving Drinking Water is Safe  Community water licensing.  Testing of source water quality.  Coordinated watershed decision making.  Effective legislation.  Public education.  Standards and guidelines.  Trained and certified water treatment plant operators.  Water treatment and distribution systems.  Effective legislation.  Public education.  Testing and monitoring of treated water quality.  Public access to water quality data.  Incident tracking and reporting.  Public reporting on NT drinking water quality.  Assessment of water treatment infrastructure and operations.  Public education.  In 2011, the Government of the Northwest Territories published the NWT Water Stewardship Strategy to improve cooperation of water management between stakeholders and to address gaps and weaknesses in water stewardship within the Territory (Government of the Northwest Territories, 2011). The Government of the Northwest Territories (2015) also recently introduced a new website (www.nwtdrinkingwater.ca) to increase transparency for the Northwest Territories drinking water quality, treatment processes, and roles and responsibilities of the communities and governmental departments. The Northwest Territories has not developed a specified drinking water management approach for smaller utilities.  2.2.3.7 Nova Scotia Drinking water in Nova Scotia (NS) has been at the forefront of environmental policy and management since the early 1990s, when the Government of Nova Scotia published the Clean Water Task Force report in 1991 and the Sustainable Development Strategy for Nova Scotia in 1992, which resulted in the 1995 Environment Act (Government of Nova Scotia, 2002). This piece of legislation strengthened drinking water management across the Province and was supplemented by the 2000 Water and Wastewater Facility Regulations and adoption of the GCDWQ (Government of Nova Scotia, 2002). The Government of Nova Scotia (2002)   44  developed the Nova Scotia Drinking Water Strategy to protect and manage drinking water and identified three key management elements; clear roles and responsibilities, multi-barrier management, and inter-departmental drinking water management. This document also contained an action plan for better drinking water management and in 2005, The Nova Scotia Minister of Environment and Labour published a report highlighting that all action items had been completed (Nova Scotia Environment and Labour, 2005). The specific drinking water management elements included as part of the Nova Scotia Drinking Water Strategy are broad and can be found in Table 2-19.  It is also important to note that a separate document, authored by Nova Scotia Environment and Labour (n.d.), titled Safe Drinking Water for Public Water Systems: A Diligent Approach categorizes multiple barrier management of drinking water in Nova Scotia into three steps; keeping clean water clean (source protection), making it safe (water treatment and system operation), and proving it’s safe (monitoring and testing).  2.2.3.8 Nunavut  Nunavut has a population of about 11,000 residents living in 14 coastal communities (Martin et al., 2007). Drinking water treatment in Nunavut is generally simple, rarely incorporating any chemical or physical treatments aside from chlorination (Daley et al., 2014). Due to the cold climate and small tax base in Nunavut, traditional distribution systems are rare and most communities rely on trucked water delivery from a central municipal treatment facility (Martin et al., 2007; Daley et al., 2014). From a governance standpoint, all water resources (and water quality and quantity monitoring) in Nunavut are managed by the Aboriginal Affairs and   45  Northern Development Canada (AANDC), which gives the department provincial-styled responsibilities, while the Nunavut Water Board is responsible for water permitting. It is also important to note that that the Canadian Federal Government has ownership of the water in Nunavut (NCCEH, 2014). In Nunavut, the Department of Health and its environmental health officers are responsible for enforcing public water supply regulations, however there is no established approach to drinking water management for the Territory.  2.2.3.9 Ontario  After the waterborne disease outbreak in Walkerton, Ontario, The Walkerton Inquiry resulted in two reports associated with the causes of the outbreak (Part 1) and a proposed strategy for improvement (Part 2) (O’Connor, 2000b; O’Connor, 2000a; Hrudey, 2011). In Part 2, 93 recommendations were made to implement an MBA that resulted in the Ontario Drinking Water Quality Management Standard (ODWMS) and tighter drinking water regulations. The ODWMS was developed to compliment the updated legislative and regulatory guidelines in place in the province and was designed for both small (with some slight modifications) and large systems. This DWMS is mandated throughout the province for managers and operators of municipal residential drinking water systems (Ontario MoE, 2007).  The Ontario Ministry of the Environment (MoE) (2007) emphasized four areas in developing the ODWMS. 1. Proactive/preventative rather than strictly reactive management strategies to identify and manage risks to public health 2. The establishment and documentation of management procedures   46  3. Meeting these procedures 4. Continuous improvement of the management system  Implementation of the ODWMS is divided into three steps (Plan/Do, Check, and Improve) (Ontario MoE, 2007). The Plan/Do section is subsequently broken down into 17 elements (Table 2-14).   Table 2-14. Ontario Drinking Water Quality Management Standard plan/do elements (Ontario MOE, 2007) No. Element Description  1 Quality management system The development and documentation of a QMS (and maintenance of the system) that meets the requirements of the operational plan and ODWMS.  2 Quality management system policy The development of a QMS policy with three main commitments; maintenance and improvement of the QMS, legislative compliance, and the providing of safe drinking water to the consumer. 3 Commitment and endorsement The creation of an operation plan that is endorsed by management and the ability to prove its commitment to the QMS. 4 Quality management system representative The description of specific requirements for selecting a QMS representative (appointment by management) and defines responsibilities and authorities for that role. 5 Document and records control The establishment and use of document control and management procedures as part of the QMS. 6 Drinking-water system The development of a process description and flow chart characterizing the entire drinking water system from source to consumer. 7 Risk assessment Element 7 and 8 require the completion (and regular updating) of a risk assessment (using a risk assessment table) for the drinking water system along with the implementation and documentation of risk management strategies. These modules include the identification of hazards and potential hazardous events, an assessment of the risk associated with hazardous events, ranking of hazardous events according to associated risk, the identification of control measures, the identification of CCPs, the establishment of critical control limits for each CCP, and documentation of the risk assessment process and outcomes. 8 Risk assessment outcomes 9 Organizational structure, roles, responsibilities, and authorities Description (and communication) of the organizational structure of the operating authority that consists of a description of roles, responsibilities, and authorities. 10 Competencies The requirement that conditions in the operational plan are met and includes a documentation of the knowledge, skills, and abilities of all personnel (whose jobs affect drinking water quality) and identification of necessary activities to ensure competency requirements are met.  11 Personnel coverage The development of procedures to ensure adequate coverage of duties affecting drinking water quality by competent personnel  12 Communications The requirement of documented procedures (communication plan) describing how the QMS and QMS procedures are communicated between owners, operators, suppliers, and the public. 13 Essential supplies and services The management and documentation of essential supplies and services that could potentially impact drinking water quality. 14 Review and provision of infrastructure The requirement of documented procedures (and reporting of results) for the annual review of drinking water system infrastructure. 15 Infrastructure maintenance, rehabilitation, and renewal The requirement of a summary (and communication of summary) detailing infrastructure maintenance, rehabilitation, and renewal programs for a drinking water system. It also includes monitoring of the effectiveness of the maintenance program. 16 Sampling, testing, and, monitoring Element 16 and 17 require the establishment and implementation of procedures describing water sampling (and sampling plans) and testing and monitoring for process control and finished water quality. These procedures include surface water monitoring activities and details about the calibration and maintenance of the measurement and recording equipment. 17 Measurement and recording, equipment calibration, and maintenance 18 Emergency management The requirement that operational plans include (and conform to) emergency procedures and contact information, which includes information about communication, response, and recovery procedures (and testing of procedures), emergency training, responsibilities of personnel and management, municipal emergency planning measures, and an emergency contact list.     47  2.2.3.10 Prince Edward Island Prince Edward Island is the smallest province in Canada, with a population of 148,600 (Statistics Canada, 2016). In regards to drinking water management, the Government of Prince Edward Island (2001) published a document titled, “Clear from the Ground to the Glass: 10 Points to Purity.” While the Government of Prince Edward Island (2001) states, “Our ten point strategy uses an MBA to protect drinking water from the ground to the glass, including source protection, system design and operation, and monitoring reporting,” the ten points do not highlight any specific drinking water management approaches or recommended management elements.  2.2.3.11 Quebec The Quebec Water Policy (QWP) was implemented in 2002 to ensure the sustainable management of water and protection of public health and the environment (Quebec MoE, 2002). According to the Quebec MoE (2002), the QWP revolves around five key orientations with specified key actions (Table 2-15). These orientations are then implemented through 16 specific key actions and 57 governmental agreements (Baril et al., 2006).   Table 2-15. Quebec Water Policy orientations and key actions (Quebec MoE, 2002) No. Orientation Key Actions 1 Water governance reform 1. Revision of the legal framework pertaining to water 2. Implementation of watershed-based management 3. Acquisition of knowledge and information about water 4. Introduction of economic instruments for governance 5. Strengthening of Quebec’s partnerships and relationships 2 Integrated management of the St. Lawrence River 1. Grant the St. Lawrence special status 2. Integrated management of the St. Lawrence River  3 Protection of water quality and aquatic ecosystems 1. Ensuring safe, quality drinking water 2. Protecting aquatic ecosystems 4 Continued clean-up and improved management of water services 1. Intensifying agricultural clean-up efforts 2. Broadening industrial clean-up efforts 3. Supplementing municipal clean-up efforts 4. Ensuring the sustainability of municipal infrastructures while improving the management of water services 5 Promotion of water-related recreotourism activities 1. Expanding access to water and promoting sportfishing 2. Promoting water safety and the quality of life on lakes and watercourses 3. Promoting nautical tourism    48  The QWP was designed to comprehensively incorporate both integrated water resources management (IWRM) and stakeholder involvement and is defined as a non-regulatory, voluntary approach to watershed management (Quebec MoE, 2002; Roy et al., 2009; Rizvi and Adamowski, 2013; Hill et al., 2008; Baril et al., 2006). IWRM refers to the coordinated optimization and management of both upstream and downstream resources (e.g. water, land, habits, etc.) to maximize economic and social benefits without compromising the sustainability of the environment (WSSD, 2002; Rahaman and Varis, 2005). Together, these elements define the DWMS approach in the Province.  With the 2002 implementation of the QWP, 33 priority watersheds were initially chosen by the Government of Quebec to develop an IWRM plan. This voluntary plan was designed to serve as a planning tool for determining and prioritizing interventions within a watershed to reach objectives defined by the water stakeholders (Gangbazo, 2004; Hill et al., 2008). More recently, the Government of Quebec has extended the IWRM plan principles to the entirety of the province across 40 watershed management zones (Gangbazo, 2011). The planning and implementation of the IWRM involves seven different steps (Table 2-16).         49  Table 2-16. Quebec Integrated Water Resources Management implementation steps (Gangbazo, 2004; Baril et al., 2006) No. Steps Description 1 Formation of a technical committee The committee should be comprised of all stakeholders having activities in the watershed and potential project funders (from academia, government, business, and community). 2 Collection of information about the water system Collection of information about threats, action possibilities, and water and ecosystem-related issues. It is composed of two parts: watershed description (e.g. activities, land use, etc.) and diagnosis of water and related ecosystem issues. 3 Setting issues and orientations  Definition of water management major concerns or fundamental challenges which must be addressed by watershed organizations. Actors must define the main actions for resolving issues identified. This step allows for the providing of a strategical overview of the different actions that should be defined and implemented. 4 Setting objectives and choice of indicators  General long-term objectives and specific short-term objectives must be defined at this stage. The objectives could be related to source water state and water uses. The indicators are administrative (performance) and environmental (ecosystem health and recovery). 5 Action plan development  Adoption of solutions (projects or activities) including voluntary or mandatory actions and enforcement programs (i.e. training and awareness programs), which may be conducted to achieve objectives and sustaining gains. The action plan could include: • monitoring programs (administrative and environmental) • partners responsibilities (depending on their commitment) • budget • funding sources • planning 6 Action plan implementation Action plan implementation is the responsibility of water stakeholders. Ideally, all governmental programs on water use, water pollution control and protection, or ecosystems restoration must be involved. Watershed organizations must plan additional measures (e.g. public information, raising funds, and political commitment) to ensure the long-term viability of the project. 7 Follow up and action plan assessment Follow-up and assessment of environmental and socioeconomic impacts of the action plan and communication to the public.  In 2012, a strategy for the protection and conservation of source water intended for human consumption was proposed by the Government of Quebec (Government of Quebec, 2012). The strategy includes five steps. 1. Water sources inventory 2. Sources vulnerability assessment/risk assessment 3. Preparation of protection and conservation measures 4. Establishing implementation mechanisms of conservation and protection measures 5. Establishing monitoring mechanisms  In 2014, a regulation for drinking water source protection (defined as source waters used for human consumption and/or food processing) was implemented (Government of Quebec, 2014). This piece of legislation was aimed at setting standards for source water intakes, regulating water   50  quality sampling, and identifying and regulating activities that could impact source water. It also included the mandatory implementation of vulnerability assessments of surface water sources for specified factors (e.g. microorganisms, turbidity, fertilizers, etc.) (Government of Quebec, 2014). A guidance document for the preparation and implementation of this vulnerability analysis was also developed and corresponds to the first step in the 2012 source water strategy discussed previously. Quebec does not have a specific DWMS approach for small utilities, however the Quebec MoE has developed a design guide for small drinking water treatment plants to assist small utilities (Ellison, 2009).   2.2.3.12 Saskatchewan  In Saskatchewan, a formal water management framework has been in place since 1999 (Government of Saskatchewan, 1999). After the waterborne disease outbreak in North Battleford, Saskatchewan in 2002, the province replaced the original drinking water framework with the Safe Drinking Water Strategy in 2003 in an attempt to “…demonstrate the province’s commitment to making the conservation and protection of our water a top priority” (Government of Saskatchewan, 2003, p. 1). As part of the Safe Drinking Water Strategy, the Government of Saskatchewan (2003) identified five guiding principles. 1. Human health as a primary concern 2. Preventing risks to drinking water is a high priority 3. Realist pricing to acknowledge the value of safe drinking water 4. Accurate and timely information about water problems and solutions is essential. 5. Governmental and stakeholder cooperation is key in developing and implement water management solutions   51  Four overarching goals with specific objectives were also identified (Table 2-17).  Table 2-17. Saskatchewan Safe Drinking Water Strategy goals and objectives (Government of Saskatchewan, 2003) Goals Objectives Waterworks systems provide safe, clean, and sustainable drinking water.  Waterworks staff are capable and well trained.  Infrastructure produces water that meets Canadian Drinking Water Standards.  Waterworks systems and operations are financially sustainable. The drinking water regulatory system is clear and effective.  Regulations are clear and ensure that health and drinking water quality will be protected.  Professional regulatory staff have access to the tools necessary to ensure compliance. Source waters are protected now and into the future.  Risks to source water are known.  Watersheds are protected, natural purification and protection processes are maximized, and the potential for contamination is minimized. Citizens and consumers trust and value their drinking water and the operations that produce it.  Citizens have meaningful access to information about the quality of their water.  Reduced consumption of water.  Consumers value quality water and are willing to pay for it.  Citizens and consumers trust the quality and reliability of their drinking water systems and are confident in the regulatory system.  Today, the Saskatchewan Water Security Agency (WSA) is responsible for the planning, implementation, and reporting associated with drinking water governance and management (as of 2012) (Government of Saskatchewan, 2014). However, the Saskatchewan MoE, Ministry of Government Relations, the Ministry of Health/Health Regions, the Ministry of Agriculture, and SaskWater, also play roles in the management of the drinking water supply (Government of Saskatchewan, 2014). Despite this extensive agency cooperation between different stakeholders, Saskatchewan does not have mandatory DWMS in place throughout the province and does not have a specific drinking water management approach for smaller utilities.   2.2.3.13 Yukon Drinking water management in the Yukon is in its relative infancy. In 2003, the Canadian Federal Government transferred water management responsibilities to the Yukon government (Government of Yukon, 2014b). In 2014, the Government of the Yukon, with input from the   52  Yukon First Nation governments, released the Yukon Water Strategy and Action Plan to address water management within the Territory (Government of Yukon, 2014a). This approach to water management consists of six priority areas. 1. Better understanding and management of groundwater 2. Planning for water needs now and into the future 3. Improving water management programs 4. Maintaining/improving access to safe drinking water 5. Promoting the sustainable use of water 6. Improving the sharing of information about Yukon’s water  In the Yukon, water licenses are issued by the Yukon Water Board and Health and Social Services are responsible for the regulation of drinking water systems (Government of Yukon, 2011). While the Yukon Water Strategy and Action Plan is considered a milestone for water management in the Territory, there is limited information included related specifically to drinking water management.   2.3 Comparison of Quality Management Systems and Drinking Water Quality Management Systems There are many distinct differences and small nuances between the reviewed QMSs and DWMSs. In Table 2-18 elements from the ISO 9001:2015 QMS, the HAACP QMS are compared to the DWMSs implemented/recommended by Australia, Canada, New Zealand, and the WHO across different quality management elements. The included elements were based on a review of literature and regulatory documents and are divided into six categories.   53   Administrative – Regulatory requirements, stakeholder involvement, and organizational controls  Assessment – Analysis of complete system and sub-processes  Mitigation – Review, inventory, and planning of preventative measures and management  Monitoring and verification – Review and validation of performance monitoring practices  Improvement – Identification and prioritization of improvements and development  Documentation and review – Record-keeping, auditing, and review processes  Within Canada, ten provincial and three territorial DWMSs and/or approaches (or lack thereof) were reviewed. These DWMSs were compared based on the same elements and categories found in Table 2-19 and listed above. Table 2-18 shows the differences between the generalized QMSs, the WHO WSP guidelines, and the three selected national DWMSs. It is readily apparent that Australia and New Zealand have developed more comprehensive systems than that of Canada and that the generic QMSs place an emphasis on documentation and review. It is also important to note that the WHO WSP guidelines are lacking in areas such as mitigation and improvement. With these guidelines seeing widespread application and implementation in developing countries, it could be expected that mitigation and continuous improvement are made more of a priority.  The results in Table 2-19 illustrate the differences of DWMSs and approaches that can be found throughout Canada. Given the fact that Canada is one of the most decentralized countries in the world, some variation in included quality management elements was expected (Hill et al., 2008).   54  However, while some provinces like British Columbia and Ontario appear to provide comprehensive approaches to drinking water management and cover the majority of identified quality management elements, provinces and territories with less developed approaches, such as New Brunswick, Nunavut, Prince Edward Island, and the Yukon may be leaving consumers susceptible to drinking water quality issues by including fewer quality management elements. This vulnerability could be amplified even further in small, rural, and First Nations communities where human resource and budgetary constraints already result in significant challenges (Moffatt and Struck, 2011; Scheili et al., 2014).  In Table 2-20, the frequency of each element across all provinces and territories in Canada was reviewed to determine the most commonly implemented quality management elements. The general QMSs and national DWMS approaches/recommendations were excluded to provide a better picture of the provinces/territories where the reviewed DWMSs and approaches excel, and to highlight potential deficiencies within Canada. The information presented in Table 2-20 further highlights the lack of consistency amongst drinking water management and their included quality management elements.   55  Table 2-18. Comparison of selected quality management systems and drinking water management systems/recommendations Category Elements HAACP2,4,6,7 ISO 9001 (2015)3 Australia4 Canada1 New Zealand5 WHO WSP,8 Administrative Assemble quality management team(s) X - X - -   X Commitment to drinking water quality management - - X - - - Communication outreach plan  - X X - - X Drinking water quality policy - - X X X - Management plan - - - - - - Modified method for smaller systems X - X - X - Public involvement and awareness - X X X - - Regulatory and formal requirements (legally binding) X X X X X - Health-based targets - - - - - X Assessment Critical control points  X - X - - - Flow diagram X - X - X - Hazard identification/risk assessment X - X - X X Hazard (risk) prioritization - - X - - - Identification of barriers and/or control measures - - - - X X System analysis X - X - X X Mitigation Contingency planning/ incident response protocols  - - X - X - Employee involvement (awareness, competency, training) X X X X - X Equipment capability and maintenance verification - X X - - - Established critical limits X - - - - - General preventative measures (non-specific) - - - - X - Multiple barriers X - X X - - Watershed (source) management - - X X - - Monitoring and verification Operational monitoring X X X - - X Control measure monitoring - X X - - X Customer satisfaction monitoring - X X - - - General monitoring (non-specific) X - - - - - Independent monitoring - - - - - X Process validation X X - - - - Water quality monitoring X - - X - - Improvement Corrective actions X X X - X X Inventory of available resources for improvement - - - - - - Prioritization of improvements - - - - X - Research and development - - X X - - Timetable for improvements - - - - X - Documentation and review Audit/review of QMS X X X - X X Continuous improvement - X X X X - Documentation, record-keeping, and reporting (general) X X X - - X QMS Improvement Plan - X - - - - Identification of links to other quality assurance systems - - - - X X Management of documentation and record-keeping X X - - - - Review by senior executive X X X - - - 1. CCME, 2002 2. Davison et al., 2006 3. ISO, 2015 4. NHMRC, 2011 5. NZMOH, 2015 6. Ropkins and Beck, 2000 7. Sperber, 2003 8. WHO, 2011    56  Table 2-19. Comparison of selected Canadian drinking water management approaches Category Elements  AB6 BC1,2,3 MB14 NB4 NL7,8 NS9 NT12 NU4 ON15 PE4,10 QC5 SK10 YT4 Administrative Assemble quality management team(s)  - X X - - X - - X - X - - Commitment to drinking water quality management  - - - - X X - - X - - - - Communication outreach plan   - - - - - X X - X - X - - Drinking water quality policy  X X X - X X - - X X - - - Management plan  X - - - - - - - X - X - - Modified method for smaller systems  - X - - - - - - X - - - - Public involvement and awareness  X - - - X - X - X - X X - Regulatory and formal requirements (legally binding)  X - X - - X X - X - - X X Health-based targets  - - - - - - - - - - - - - Assessment Critical control points   - - - - - - - - X - - - - Flow diagram  - X - - - - - - X - X - - Hazard identification/risk assessment  X X - - - X - - X - X - - Hazard (risk) prioritization  - X - - - - - - X - - - - Identification of barriers and/or control measures  - X - - - - X - X - X - - System analysis  X X X - - X X - X - X - - Mitigation Contingency planning/ incident response protocols   - X - - - - X - X - - - - Employee involvement (awareness, competency, training)  X X X X - X X X X X X X X Equipment capability and maintenance verification  - X - - - - X - X - - - - Established critical limits  - - - - - - - - X - X - - General preventative measures (non-specific)  - - - - - - - - - - - - - Multiple barriers  - X - - X X X - X - - X - Watershed (source) management  - X X - X X X - X - X X - Monitoring and verification Operational monitoring  - X - - X X X - X - - - - Control measure monitoring  - - - - X X - - X - X - - Customer satisfaction monitoring  - X - - X - X - X - - - - General monitoring (non-specific)  - - - - - - - - - - - - - Independent monitoring  - - - - - - - - - - X - - Process validation  - X - - - X - - X - - - - Water quality monitoring  - X - - X X X - X - X X - Improvement Corrective actions  X X - - X X - - X - X X - Inventory of available resources for improvement  X X X - - X - - X - X - - Prioritization of improvements  X X X - - - - - X - X - - Research and development  - - - - X - - - X - - - - Timetable for improvements  X X X - - - - - X - X - - Documentation and review Audit/review of DWMS  X X - - - X - - X - X - - Continuous improvement  - - - - - X - - X - - - - Documentation, record-keeping, and reporting (general)  - X - - X X X - X - X - - DWMS improvement plan  - X - - - - - - X - X - - Identification of links to other quality assurance systems  - X - - - - - - - - - - - Management of documentation and record-keeping  - X - - - - - - X - X - - Review by senior executive  - - - - - X - - X - - - - 1. BCMHLS, 2010 2. BCMOH, 2013 3. BCOPHO, 2015 4. Dunn et al., (2014) 5. Gangbazo, 2011 6. Government of Alberta, 2012 7. Government of Newfoundland and Labrador, 2001 8. Government of Newfoundland and Labrador, 2014 9. Government of Nova Scotia, 2002 10. Government of Prince Edward Island, 2001 11. Government of Saskatchewan, 2003 12. Government of the Northwest Territories, 2005 13. ISO, 2015 14. Manitoba Provincial Planning Regulation, n.d. 15. Ontario MoE, 2007   57  Table 2-20. Frequency of elements found in selected drinking water quality management systems Category Elements Frequency of Element in all Reviewed DWMS* Administrative Assemble quality management team(s) 38.5% Commitment to drinking water quality management 23.1% Communication outreach plan 30.8% Drinking water quality policy 53.8% Management plan 23.1% Modified method for smaller systems 15.4% Public involvement and awareness 46.2% Regulatory and formal requirements (legally binding) 53.8% Health-based targets 0.0% Assessment Critical control points 7.7% Flow diagram 23.1% Hazard identification/risk assessment 38.5% Hazard (risk) prioritization 15.4% Identification of barriers and/or control measures 30.8% System analysis 53.8% Mitigation Contingency planning/ incident response protocols 23.1% Employee involvement (awareness, competency, training) 92.3% Equipment capability and maintenance verification 23.1% Established critical limits 15.4% General preventative measures (non-specific) 0.0% Multiple barriers 46.2% Watershed (source) management 61.5% Monitoring and verification Operational monitoring 38.5% Control measure monitoring 30.8% Customer satisfaction monitoring 30.8% General monitoring (non-specific) 0.0% Independent monitoring 7.7% Process validation 23.1% Water quality monitoring 53.8% Improvement Corrective actions 53.8% Inventory of available resources for improvement 46.2% Prioritization of improvements 38.5% Research and development 15.4% Timetable for improvements 38.5% Documentation and review Audit/review of DWMS 38.5% Continuous improvement 15.4% Documentation, record-keeping, and reporting (general) 46.2% DWMS improvement plan 23.1% Identification of links to other quality assurance systems 7.7% Management of documentation and record-keeping 23.1% Review by senior executive 15.4%  While there are a few elements found consistently throughout the majority of the studied DWMSs (e.g. regulatory and formal requirements, public involvement and awareness, system analysis), most of the elements in the categories of monitoring and verification and documentation and review appear to be overlooked.    58  2.3.1 Catchment to Consumer Recommended Quality Management Elements After reviewing the selected QMSs and DWMSs and approaches, it is evident that there are significant difference between the included QMS elements in different parts of the world and across Canada. While these difference can most likely be attributed to different reasons (e.g. legislation, priorities) and some elements may be considered more important than others, the lack of consistency across DWMSs is noteworthy given the same goal of providing safe, clean drinking water. Based on the comparison of the different QMSs and DWMSs and approaches, a list of recommended quality management elements from catchment to consumer was developed (Table 2-21Table 2-21).   This basic framework was created to highlight specific drinking water management elements than can be applied to improve existing drinking water management approaches across Canada. While gauging the performance and individual importance of specific quality management elements is a difficult proposition, this generalized framework was developed to highlight important specific drinking water management elements and more general quality management elements to better address monitoring, verification, documentation, and review.          59  Table 2-21. Catchment to consumer recommended drinking water management elements Category Recommended Elements Administrative  Drinking water quality policy  Public involvedness and awareness  Regulatory and formal requirements Assessment  Flow diagram  Hazard identification / risk assessment  Identification of barriers and control measures Mitigation  Contingency planning/incident response protocols  Employee involvement  Equipment capability and maintenance verification  Multiple barriers  Watershed (source) management Monitoring and verification  Customer satisfaction monitoring  Operational monitoring  Process validation  Water quality monitoring Improvement  Corrective actions  Inventory of available resources for improvement  Prioritization of improvements Documentation and review  Audit/review of DWMS  Continuous improvement  Documentation, record-keeping, and reporting  Management of documentation and record-keeping  Although more elements and endless combinations of elements can be included in developing a robust DWMSs, the selected elements in Table 2-21 represent building blocks for a comprehensive drinking water management approach that can be used to improve existing DWMSs and develop new DWMSs.  2.4 Summary Drinking water management systems encompass a wide variety of differing policies, procedures, and administrative/behavioral controls for ensuring safe drinking water throughout the world. Canada is no exception to this variability, with vastly different approaches to drinking water management across the country. While quantifying the effectiveness and impact of management practices is difficult given their nature, by comparing the Canadian state of practice and included management elements at the national and provincial/territorial level against the WHO, world leaders in regards to DWMSs, and well established QMSs, context can be provided to   60  characterize the comprehensiveness, or lack thereof, of the existing Canadian provincial and territorial DWMSs.  61  Chapter 3: Continuous Performance Improvement in Small Drinking Water Systems A part of this chapter has been published in Science of the Total Environment, an Elsevier journal, as an article titled “Framework for continuous performance improvement in small drinking water systems” (Bereskie et al., 2017c).  With regards to Objective 2 as defined in Chapter 1, a CPI framework was developed in Chapter 3 to provide SDWS managers and operators an approach to gauge their current performance against similar systems and to track performance improvement from the implementation of the new technologies or innovations into the future. The proposed CPI framework incorporates the use of a WQI and functional performance benchmarking to evaluate and compare drinking water quality performance of an individual water utility against that of a representative benchmark. The results can then be used to identify and prioritize the most vulnerable water quality indicators and subsequently identify and prioritize performance improvement strategies.   The CPI framework was demonstrated using data collected from SDWSs in Newfoundland and Labrador, Canada, incorporating the CCME WQI method. The content and methodologies presented in Chapter 3 were designed to address some of the identified drinking water management gaps and SDWS challenges identified in Chapter 2.    62  3.1 Background Mandated drinking water standards are set to ensure consistent safe drinking water and represent a static value not to be exceeded. Focusing on regulatory compliance alone can result in a stagnant improvement strategy that lacks the flexibility to adapt with future regulatory or technological advancements. Through the use of a CPI framework based on assessing performance with a WQI and using functional performance benchmarking, SDWSs can have a better, more resilient option for gauging current success and tracking projected water quality improvement into the future.  3.1.1 Continuous Performance Improvement The central theme of CPI revolves around continuous, measurable advancement and innovation that can be the result of incremental progression or radical changes brought forth by innovation or technology (Bessant et al., 1994; Berger, 1997; Zangwill and Kantor, 1998; Bhuiyan and Baghel, 2005). While the concept dates back to the 1800s, the first notable use of CPI as a tool to gain a competitive advantage was by Toyota in the 1950s (Zangwill and Kantor, 1998). Today, many different CPI techniques have been developed and utilized across a wide variety of industries (e.g. six sigma, lean manufacturing).   Public utilities do not experience the same pressures for CPI and competitive advantage as in the private sector. They are provided with static performance standards and guidelines set by regulators and can have little reason to focus on incremental improvement. However, while competitive advantage is often cited as the main driver for implementing CPI, internal improvements can also provide other benefits and create new opportunities (Zangwill and   63  Kantor, 1998). CPI is appealing, especially in situations where the use of a process-oriented approach (i.e. improvement based on the performance of individual processes and steps in between processes) is more desirable than a traditional, result-oriented structure (i.e. improvement based entirely on final results) (Berger, 1997). One of the most well-known and widely applied CPI concept is the PDCA cycle.  3.1.2 Plan-Do-Check-Act Cycle The PDCA cycle, also known as the Deming Cycle, was designed to coordinate continuous improvement plans by categorizing improvement actions into a dynamic cycle of four steps - plan, do, check, and act (Figure 3-1) (Langley et al., 1994; Moen and Norman, 2006; Moen, 2009; Lodgaard and Aasland, 2011). The concept was originally developed by Walter Shewhart and W. Edwards Deming in the 1950s and was designed to encourage the use of small-scale improvements for continuous performance improvement while allowing for rapid assessment of improvement actions (Moen and Norman, 2006; Moen, 2009; Taylor et al., 2013). It has since seen widespread study and application in academia and industry ranging from use in managing environmentally responsible process improvements (Reid et al., 1999) to product development (Lodgaard and Aasland, 2011) and healthcare (Taylor et al., 2013). It is also featured as a component of ISO 9001:2015, a leading international standard for the development of quality management systems (ISO, 2015).    64   Figure 3-1. Illustration and description of steps in the PDCA cycle (Reid et al., 1999; Taylor et al., 2013)  In relation to water and the environment, the PDCA cycle has been successfully implemented for environmental management in the Santa Clara Valley Water District in California (Ruiz, 2008), as part of the Vancouver Airport Authority Environmental Management Plan (Vancouver Airport Authority, 2014), and is recommended by the International Petroleum Industry Environmental Conservation Association (IPIECA) (2013) for water management of onshore oil and gas activities.   3.1.3 Water Quality Indices A WQI is a unitless number (i.e. usually between 0-100) representing a water quality value obtained by incorporating various statistical and mathematical techniques to aggregate applicable PIs to quantify overall water quality (Pesce and Wunderlin, 2000; Swamee and Tyagi, 2000; CCME, 2001; Hurley et al., 2012). WQI-based approaches have been extensively applied for measuring source water quality, but recent studies by Hurley et al. (2012), Islam et al. (2013a), and Scheili et al. (2015) have adapted their use specifically for drinking water quality. While a WQI-based approach is not designed to replace detailed water quality testing and comparison, its application provides a metric for normalizing and evaluating overall water quality both internally   65  and externally. Depending on the use and data available for comparison, any number of PIs can be integrated. However, it is important to note that a higher number of indicators provides a more representative overview of water quality. A generic equation for a WQI can be found in Equation 3-1.  𝑿 = 𝒇(𝒙𝟏, 𝒙𝟐, … 𝒙𝒏)  𝑿 ∈ [𝑿𝒎𝒊𝒏, 𝑿𝒎𝒂𝒙]                                                                (Equation 3-1)    In Equation 3-1, ‘X’ represents the overall WQI value (usually a score between 0-100) and ‘f’ represents a function designed to incorporate and transform multiple selected water quality PIs with differing units (i.e. x1, x2,…xn) into one, unitless value (Hurley et al., 2012). A few examples of WQIs can be found in the sections below.  3.1.3.1 Canadian Council of Ministers of the Environment Water Quality Index The CCME WQI has been used for diverse applications throughout Canada ranging from source water quality evaluation to distribution system water quality (CCME, 2001; Khan et al., 2004; Lumb et al., 2006; Hurley et al., 2012; Islam et al., 2013a). The CCME WQI consists of three factors: scope (F1), frequency (F2), and amplitude (F3) (CCME, 2001). ‘F1’ represents the extent of water quality guideline non-compliance over a given time period and is calculated using Equation 3-2.   𝑭𝟏 = (𝑵𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒇𝒂𝒊𝒍𝒆𝒅 𝒗𝒂𝒓𝒊𝒂𝒃𝒍𝒆𝒔𝑻𝒐𝒕𝒂𝒍 𝒏𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒗𝒂𝒓𝒊𝒂𝒃𝒍𝒆𝒔) × 𝟏𝟎𝟎                                                                        (Equation 3-2)    66  ‘F2’ represents the percentage of individual tests that do not meet objectives, referred to as ‘failed tests’ and is calculated using Equation 3-3.  𝑭𝟐 = (𝑵𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒇𝒂𝒊𝒍𝒆𝒅 𝒕𝒆𝒔𝒕𝒔𝑻𝒐𝒕𝒂𝒍 𝒏𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒕𝒆𝒔𝒕𝒔) × 𝟏𝟎𝟎                                                                                                         (Equation 3-3)  ‘F3’ represents the amount by which failed test values do not meet their objectives. ‘F3’ is calculated in three steps. The first step represents the number of times in which an individual concentration is greater or less than the objective. This is referred to as an ‘excursion’ and when the test value must not exceed the objective it is calculated using Equation 3-4.  𝒆𝒙𝒄𝒖𝒓𝒔𝒊𝒐𝒏𝒊 = (𝑭𝒂𝒊𝒍𝒆𝒅𝑻𝒆𝒔𝒕𝑽𝒂𝒍𝒖𝒆𝒊𝑶𝒃𝒋𝒆𝒄𝒕𝒊𝒗𝒆𝒋) − 𝟏            (Equation 3-4)  For the cases in which the test value must not fall below the objective, Equation 3-5 is used.   𝒆𝒙𝒄𝒖𝒓𝒔𝒊𝒐𝒏𝒊 = (𝑶𝒃𝒋𝒆𝒄𝒕𝒊𝒗𝒆𝒋𝑭𝒂𝒊𝒍𝒆𝒅𝑻𝒆𝒔𝒕𝑽𝒂𝒍𝒖𝒆𝒊) − 𝟏            (Equation 3-5)  The collective amount by which individual tests are above or below compliance is referred to as the ‘normalized sum of exclusions’ (nse) and is calculated using Equation 3-6.  𝒏𝒔𝒆 = (∑ 𝒆𝒙𝒄𝒖𝒓𝒔𝒊𝒐𝒏𝒊𝒏𝒊=𝟏 ) ÷ (𝒏𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒕𝒆𝒔𝒕𝒔)          (Equation 3-6)    67  ‘F3’ is then calculated using Equation 3-7, which scales the ‘nse’ from the objectives to yield a range between 0 and 100.   𝑭𝟑 = (𝒏𝒔𝒆𝟎.𝟎𝟏𝒏𝒔𝒆+𝟎.𝟎𝟏)                         (Equation 3-7)  After calculating values for ‘F1’, ‘F2’, and ‘F3’, the CCME WQI is calculated using Equation 3-8.  𝑪𝑪𝑴𝑬 𝑾𝑸𝑰 = 𝟏𝟎𝟎 − (√𝑭𝟏𝟐+𝑭𝟐𝟐+𝑭𝟑𝟐𝟏.𝟕𝟑𝟐)            (Equation 3-8)  CCME (2001), selected the use of the factor of 1.732 as each of the three individual index factors can reach values as high as 100. This means the vector length can reach a maximum of 173.2 and division by 1.732 results in a maximum vector length of 100 (CCME, 2001). The CCME WQI was developed to compare measured source water quality values against regulatory guidelines to produce a score ranging from 0-100, with 0 representing the lowest quality water and 100 representing the highest quality water (Hurley et al., 2012). Table 3-1 shows the CCME WQI ranking modified for drinking water assessment.        68  Table 3-1. CCME WQI categorization (modified after CCME, 2001 for drinking water utilities) Ranking CCME WQI Value Description Excellent 95-100 Water quality regulatory objectives are met virtually all of the time and DWAs virtually never occur. Good 80-94 Water quality rarely departs from regulatory objectives and DWAs rarely occur. Fair 65-79 Water quality sometimes departs from regulatory objectives and DWAs sometime occur. Marginal 45-64 Water quality often departs from regulatory objectives and DWAs often occur. Poor 0-44 Water quality usually departs from regulatory objectives and DWAs are common.  3.1.3.2 British Columbia Water Quality Index The British Columbia WQI was the predecessor to the CCME WQI and was designed to improve communication with the public while providing a general means of comparing and ranking water bodies throughout British Columbia (Equation 3-9) (BCMoE, 2001).  𝑿 = (𝑭𝟏𝟐 + 𝑭𝟐𝟐 + 𝑭𝟑𝟐)𝟏𝟐              (Equation 3-9)  In Equation 3-9, ‘F1’ represents the percentage of water quality guidelines exceeded, ‘F2’ represents the percentage of measurements in which one or more of the guidelines are exceeded, and ‘F3’ represents the maximum (normalized to 100) by which any of the guidelines are exceeded (BCMoE, 2001; CCME WQI, 2001)  3.1.3.3 Centre St. Laurent Water Quality Index  The Centre St. Laurent WQI was developed by Environment Canada for reporting on the St. Lawrence River (Equation 3-10) (CCME, 2001). This methodology was designed to calculate different WQIs, depending on water use and considerations (CCME, 2001).  𝑿 = [∑(𝑨𝒊 × 𝑭𝒊)] ÷ 𝒏           (Equation 3-10)   69   In Equation 3-10, ‘Ai’ represents the mean level of exceedance for variable ‘i’ for guideline ‘i’. The term ‘n’ represents the total number of variables. When a variable value exceeds a guideline for that variable, the ratio of exceeding value/guideline value is calculated. These ratios are then summed and divided by the number of times they occur. ‘Fi’ represents the frequency of values that exceed a guideline for a given variable (relative to the total number of values obtained for that variable) (Equation 3-11).  𝑭𝒊 = 𝑭𝒆𝒙𝒄𝒆𝒆𝒅/𝑭𝒕𝒐𝒕𝒂𝒍                        (Equation 3-11)  3.1.3.4 Quebec Index The Quebec Index was based on a WQI system developed by Smith (1990) for use in New Zealand and incorporated the use of the Delphi method (i.e. a technique based on a series of questionnaires and the expert group response) for the calculation of sub-indices (Equation 3-12) (Hébert, 1997; CCME, 2001). The Quebec Index was designed to represent the worst case scenario for any of the measured variables (CCME, 2001).  𝑿 = 𝒎𝒊𝒏(𝑰𝒔𝒖𝒃𝟏, 𝑰𝒔𝒖𝒃𝟐, … 𝑰𝒔𝒖𝒃𝒏)                             (Equation 3-12)  In Equation 3-12, ‘min’ represents the minimum operator (referring to the use of the lowest subindex rating to produce the final rating score) and ‘Isubi’ represents the subindex score for the subindex ‘i’. Smith (1990) recommended four water uses for which index scores can be derived;   70  general (i.e. water with no principal use, but subject to competing uses), bathing, water supply, and fish spawning.  3.1.4 Benchmarking Public utilities are provided with static performance standards set by governmental regulators and can have little reason to focus on incremental improvement. Performance benchmarking can provide further incentive for improvements by comparing performance locally, regionally, or against industry leaders. Performance benchmarking refers to the process of measuring performance against other entities, regardless of whether or not they are direct competitors (Talluri and Sarkis, 2001; Malec, 1994; Camp, 1989). Elmuti and Kathawala (1997) defined four different types of performance benchmarking, consisting of internal benchmarking, competitive benchmarking, functional (industry) benchmarking, and process (generic) benchmarking. A description of each can be found below.  Internal – Benchmarking conducted internally within an organization. This technique is usually used to identify the best internal procedures and transferring them to other portions of an organization. This type of benchmarking is best used as a baseline for external benchmarking.  Competitive (external) - Competitive benchmarking, also known as external benchmarking) is conducted externally with direct competitors having competing products, services, or work processes. This type of benchmarking can be beneficial, but obtaining competitor information can be difficult.   71   Functional (industry) - Functional benchmarking, also known as industry benchmarking, is conducted externally against industry leaders or the best functioning operations of certain organizations. Partners in this process usually share some common technological and market characteristics, but do not have direct competition.  Process – Process benchmarking, also known as generic benchmarking, focuses on the best work processes, procedures, and functions of an organization. This type of benchmarking is used across dissimilar organizations and while extremely effective, can be difficult to implement.   3.1.4.1 Benchmarking Process While performance benchmarking has been used in corporate settings for decades to improve competitive advantage, its use  in municipal settings is still being explored (Talluri and Sarkis, 2001; Adebanjo et al., 2010). The use of performance benchmarking can allow operators and planners to gauge performance against other similar municipalities and create increased transparency for consumers. It can also be a useful tool in identifying operational and strategic gaps and subsequently in choosing best management practices to remedy problems (Yasin, 2002). A flowchart of the performance benchmarking process can be found below in Figure 3-2.    72   Figure 3-2. Flowchart of the performance benchmarking process (adapted from Elmuti and Kathawala, 1997)  The performance benchmarking process begins with the identification of a function to benchmark. In terms of drinking water quality, this can include individual parameters such as DBPs or free residual chlorine (FRC) or incorporate multiple factors through the use of a WQI. After collecting applicable data for the selected function, performance can then be measured against that of a similar utility or the average of a group of selected utilities. By measuring the performance gap, mitigation actions and technologies can be prioritized and implemented to meet and exceed the selected benchmark.    73  3.1.4.2 Benchmarking and Drinking Water Management While benchmarking has seen widespread implementation since the 1970s, it has seen limited applications in regards to drinking water suppliers. Corton (2003) described a water utility benchmarking scheme implemented by Peru’s regulatory sector focused on PIs in three categories: quality of service, management efficiency, and efficiency in managing financial issues. Corton and Berg (2009) benchmarked water utilities in Costa Rica, El Salvador, Guatemala, Honduras, Nicaragua, and Panama for three core indicators including operational performance, cost, and quality. Marques et al., (2014) performed nonparametric benchmarking of Japanese water utilities for efficiency based on both institutional and operational factors. Haider et al., (2016a) proposed a model for benchmarking small-to-medium-sized water utilities based on metric benchmarking of individual PIs grouped into indices to assess the performance of seven functional components (i.e. water resource and environmental sustainability, personnel adequacy, physical assets efficacy index, operational integrity, and water quality and public health safety, quality of service, and economic viability) of the utilities.   These approaches to benchmarking all provided a unique insight into gauging the overall performance of drinking water suppliers, but lacked detailed insight specifically on using benchmarking to compare and improve drinking water quality and drinking water utility performance    74  3.2 Approach and Methodology The proposed CPI framework involves obtaining information from SDWSs and analyzing performance against comparable SDWSs using functional performance benchmarking and a WQI-based approach (Figure 3-3).   Figure 3-3. Proposed continuous performance improvement framework  The overall goal of this framework is to improve quality and consistency of drinking water in both individual water utilities and throughout an entire representative benchmark. The five-step process is described in detail below.   75  3.2.1 Data Collection  The CPI framework initiates with the collection of relevant water quality data (i.e. water quality indicators ‘x’) from SDWSs [Step 1]. The targeted data includes information from a SDWS-of-concern along with data from other similar SDWSs. Operators and/or support staff can easily accomplish this step by implementing proper data inventory management within the utility. The data should include, at a minimum, testing at the source water, after treatment, and at multiple points within the distribution network. The similar SDWSs will optimally consist of small water utilities serving similar populations with the same source type (e.g. lake or river) and treatment type (e.g. chlorination only). This framework can be applicable for groundwater systems, but during the benchmarking process should only be compared with other groundwater systems. Water utilities with a range of infrastructure ages and operator experience can also provide a more representative standing of regional performance. While there is no defined minimum or maximum number of participants in developing the benchmark, the utilities selected should provide data representative of the study area for best results.  3.2.2 Calculate Water Quality Index  After data collection, a WQI (e.g. CCME WQI), defined as ‘X’, integrating chemical, microbiological, and physical water quality indicators is used to aggregate the selected indicators into a performance range [Step 2] (see Equation 3-8). A list of outputs using the CCME WQI can be found in Appendix A.    76  3.2.3 Functional Performance Benchmarking  Performance (measured in terms of a WQI or by individual drinking water quality indicators) of a SDWS-of-concern is compared to that of a representative benchmark comprised of data from other comparable SDWSs (when ‘t = 0’, i.e. the beginning of the benchmarking process) [Step 3]. If the SDWS-of-concern performs below the benchmark, improvements are required immediately. Depending on the performance gap found, major and/or minor improvements will be necessary (Figure 3-4). If the SDWS-of-concern is performing above the benchmark, and the selected WQI value is less than the maximum (i.e. 100), investigation to determine minor or major improvements is necessary along with careful observation to maintain established drinking water quality.   This step must be the result of cooperation between regional utility operators and planners. As there is often regular communication (including training, workshops, and meetings) in regards to water treatment, further interactions such as data sharing can be implemented without much additional effort. It is also important to investigate individual drinking water quality indicators at this point to determine where to focus performance improvement. With adequate data availability, operators and/or support staff can easily calculate WQI using a Microsoft Excel spreadsheet or online WQI calculator.    77   Figure 3-4. Relative performance of SDWSs in terms of performance gaps between calculated WQI and established benchmark (modified after Haider, 2016a)  3.2.4 Identification and Prioritization of Improvement Actions  After gauging the severity of needed improvements, improvement actions, defined as ’Z’ depending on location within the drinking water system ’i’ and type of improvement ’j’ (i.e. ‘Zij’), can be identified and prioritized [Step 4]. This evaluation allows SDWS operators and managers to most efficiently improve water quality by appropriating resources to the most vulnerable areas first. After identifying and prioritizing the water quality indicators, appropriate improvement actions can be identified and ranked according to their projected efficiency  3.2.5 Implementation of Performance Improvement Actions  The implementation of improvement actions (when ‘t=1’, i.e. the end of the benchmarking process) could take the form of an operational modification, technological upgrade, policy change, or additional operator training(s) and could be implemented immediately or over a defined time period [Step 5].    78  3.2.6 Continuous Performance Improvement After improvement actions are implemented and optimized within the system, updated water quality data for the SDWS-of-concern and the other similar SDWSs needs to be collected [Step 1] to recalculate WQI [Step 2] and conduct the functional performance benchmarking [Step 3] to determine the next step. This cycle is repeated until the selected WQI reaches the maximum value (i.e. 100). After the desired water quality level is achieved, the CPI focus can shift to higher objectives such as maintaining of service reliability and focusing on customer satisfaction and aesthetic water quality.  Figure 3-5 shows a conceptual diagram of the proposed CPI framework. In this figure, solid lines represent overall SDWS performance in terms of a selected WQI or individual drinking water indicator with the period from 2010-2015 representing the existing benchmark and current state of technology (i.e. a bare minimum approach based on current regulatory schemes).    Figure 3-5. Proposed continuous performance improvement framework conceptual diagram   79  The dots represent the performance values for individual water utilities. Future benchmarks are projected based on the improvement of performance from interventions, technological advances, changing regulations, and/or customer expectations. Keeping in mind the existing gap in the benchmarking processes of smaller systems and the resulting data limitation issues, a five-year interval has been hypothetically proposed to emphasize the improvement process. The performance increase during a given five-year interval can be implemented at any point in the improvement cycle, but as the benchmarking process only occurs at the end of a given cycle, the performance increase is only evident every five years. It should be noted that this interval can be increased or reduced depending on the actual outcomes of the benchmarking process and the capabilities of the participating utilities. The goal of CPI is not only to improve upon the original or previous benchmark for a factor or group of factors, but also to cluster individual utilities closer to the benchmark value. This clustering effect represents more consistent, higher quality drinking water.  Steps 3 through 5 and the actual implementation of the framework into a SDWS are the responsibility of planners, regulators, and decision-makers and must incorporate input from operators and other support staff.  3.3 Demonstration using Small Drinking Water Systems in Newfoundland and Labrador (Canada) SDWSs in Newfoundland and Labrador provide a good representation of the overall current difficulties and state of SDWSs in Canada. The communities are aging, leading to significant financial and human resource hurdles (Government of Newfoundland and Labrador, 2009).   80  Drinking water treatment in many of these communities consists only of the addition of chlorine to raw surface source water and results in aesthetic, biological, and chemical water quality issues (Scheili et al., 2014). These small water utilities have an aging infrastructure, lack qualified dedicated operators, and are prone to DWAs (Khan et al., 2004).  3.3.1 Data Preparation  A functional performance benchmark for this demonstration was developed using data collected from seven municipalities in Newfoundland and Labrador. These water utilities serve populations between 400 and 2500 residents and have treatment facilities and distribution networks ranging from new (i.e. less than 10 years old) to old (i.e. greater than 50 years old). A wide range of population size and infrastructure age was used to establish benchmarks which will be applicable to a wide range of local SDWSs. System information from the chosen SDWSs can be found in Table 3-2. The facilities selected only use the addition of chlorine directly to raw surface water for disinfection and have no other additional forms of treatment.   Table 3-2. System information from selected small drinking water systems in Newfoundland and Labrador, Canada Name  Pop. serveda Source type Treatment system age Dist. system age SDWS 1 737 Pond 5-10 years 20-30 years SDWS 2 1924 Lake 6 years 30-50 years SDWS 3 2122 Pond 44 years 44 years SDWS 4 1031 Pond >50 years 71 years SDWS 5 1681 Brook 20 years 20 years SDWS 6 452 River <5 years 1 year SDWS 7 998 River <5 years 34 years a Statistics Canada, 2011    81  3.3.1.1 Performance indicators Five drinking water quality PIs from seven SDWSs were identified for use to demonstrate the CPI framework. The data was collected by Scheili et al. (2014) from a 2010-2011 sampling campaign of 25 SDWSs in Newfoundland and Labrador and Quebec. Samples were collected at multiple points within the distribution network and included water quality PIs designed to highlight the spatial and temporal variations of the physical, chemical, and microbiological quality of both source and drinking water (Scheili et al, 2014). Analysis of these samples was conducted using the Standard Methods for the Examination of Water and Wastewater, 19th edition (AWWA, 1999).  All of the selected systems used surface water sources and the selected PIs consisted of trihalomethanes (THMs), haloacetic acids (HAAs), FRC, turbidity, and total coliforms (TC). PIs are weighted evenly in the CCME WQI, however, a modified method could be developed to quantify risk or prioritize certain indicators. When using the CCME WQI, more or different indicators could be incorporated to get a better overall picture of drinking water quality, as can be found in the WQI proposed by Scheili et al. (2016) , but the CCME WQI framework is designed to work with as few or as many variables as the researcher sees fit (Hurley et al., 2012).   THMs and HAAs are the most studied and well-understood DBPs. These compounds are prevalent in SDWSs supplied by surface waters, as they rely heavily on chlorine for disinfection and source water containing relatively high levels of natural organic matter (NOM). In this study, THM4 (chloroform, bromodichloromethane (BDCM) dibromochloromethane (DBCM), and bromoform) and HAA5 (monochloroacetic acid, dichloroacetic acid, trichloroacetic acid,   82  monobromoacetic acid, and dibromoacetic acid) were used. To accommodate for spatial variation, values for both DBPs were measured at the beginning (R1), middle (R2), and extremity (R3) of the distribution network. The beginning and the extremity refers to the first and last accessible public point or residency where sampling was possible. The middle was determined as a point equidistant from the beginning and the extremity (Scheili et al., 2014). THMs and HAAs are regulated by the USEPA (2009) at 80 µg/L and 60 µg/L for THM4 and HAA5 respectively. Health Canada (2014) has guidelines established at 100 µg/L for THM4 and 80 µg/L for HAA5.  FRC and turbidity were chosen due to their importance in aesthetic water quality and in the formation of DBPs. FRC, like the selected DBPs, was measured in three locations in the distribution network, while turbidity was only measured at the source, before chlorination. As none of the systems from Newfoundland and Labrador selected for additional study use treatment other than chlorination directly applied to the raw source water, the turbidity values at the source are considered to be representative of what is likely found in the distribution system. The USEPA (2009) regulates turbidity based on the type of filtration treatment used. Systems using conventional or direct filtration (i.e. coagulation, flocculation, and filtration) cannot exceed 1 NTU (and samples for turbidity must be less than or equal to 0.3 NTUs in at least 95% of samples in any month). Systems using filtration other than conventional or direct filtration must follow state limits, which must include samples never exceeding 5 NTUs.  Health Canada (2014) also establishes treatment limits based on filtration type used. There is an established guideline of less than or equal to 0.1 NTU at all times (in at least 99% of   83  measurements per operational filter period or per month) for surface water sources with membrane filtration. Water utilities using conventional and direct filtration for surface waters have a guideline of less than or equal to 0.3 NTUs, which must be met at least 95% of the time per operational filter period or per month, never to exceed 1 NTU. In the case of slow sand filtration (SSF) and diatomaceous earth filtration for surface waters, turbidity samples must be less than or equal to 1 NTU (in at least 99% of measurements per operational filter period or per month), never to exceed 3 NTU. For systems that use groundwater, turbidity should generally be below 1 NTU (Health Canada, 2014). It is also worth noting that the WHO (2011), recommends a maximum guideline of 5 NTUs, with a target below 1 NTU if possible, for SDWSs with limited resources, although this is not applicable in Canada or the US.   FRC standards are more complex, as a healthy range that minimizes DBP production while maximizing microbial disinfection is desirable. The USEPA has an FRC maximum of 4.0 mg/L under the Stage 1 Disinfectant/Disinfection By-products Rule and also recommends at least 0.2 mg/L FRC in the distribution network (USEPA, 2006). The WHO (2011) recommends an FRC concentration range between 0.4-0.6 mg/L depending on pH. Due to the difficulty of maintaining FRC in SDWSs, a range between 0.2-0.8 mg/L could be considered as optimal (Islam et al., 2013a).  Presence of TC was chosen as a PI to represent the microbial quality of the drinking water. While TC is not commonly used as an indicator for potential health impacts from microorganisms, it provides insight into the disinfection performance of the treatment system and distribution network-related water quality failure (contaminant intrusion, presence of   84  biofilm). Health Canada (2014) has established a MAC of none detectable/100 mL in water leaving the treatment plant. In this demonstration, TC was measured only in the middle of the distribution network. As there are no booster chlorination stations present in the selected SDWSs, the TC values provided an adequate snapshot of microbial growth and contamination.  3.3.2 Calculating Water Quality Index  The CCME WQI was selected as the WQI methodology for use in this demonstration. WQI values were calculated for each water utility selected in Step 1 and were based on Health Canada (2014) guidelines for THM4, HAA5, turbidity, and TC and the optimal range presented by Islam et al. (2013a) for FRC. The WQI values were calculated using data from all three points in the distribution network. Scope, frequency, amplitude and total CCME WQI were calculated seasonally throughout the year and are shown in Table 3-3 as ‘F1’, F2’, and ‘F3’ respectively.  Table 3-3. Summary table of seasonal Canadian Council of Ministers of the Environment Water Quality Index for the selected water utilities SDWSs Season F1 F2 F3 WQI SDWSs Season F1 F2 F3 WQI SDWS 1  Autumn Winter Spring Summer Average 81.8 81.8 72.7 81.8  61.9 71.9 51.5 54.5  26.5 27.2 15.3 22.5  38.8 35.2 47.8 41.8 40.9 SDWS 5 Autumn Winter Spring Summer Average 63.6 72.7 27.3 54.5  54.5 38.6 30.0 27.3  48.1 20.9 15.0 100  44.2 50.9 75.0 32.4 50.9 SDWS 2 Autumn Winter Spring Summer Average 90.9 100 45.5 72.7  35.7 57.6 28.0 41.4  73.5 24.5 11.7 14.2  29.4 31.9 68.0 51.0 45.2 SDWS 6 Autumn Winter Spring Summer Average 90.9 81.8 100 100  65.9 81.8 81.8 81.8  100 68.9 50.8 80.2  13.2 22.3 19.8 12.2 16.9 SDWS 3 Autumn Winter Spring Summer Average 81.8 45.5 72.7 81.8  57.5 39.4 56.3 60.6  31.7 26.6 28.2 50.1  39.4 62.0 44.5 34.5 45.1 SDWS 7 Autumn Winter Spring Summer Average 90.9 81.8 81.8 72.7  72.5 58.6 63.6 63.6  100 33.0 28.1 37.4  11.5 38.8 38.0 40.2 32.0 SDWS 4 Autumn Winter Spring Summer Average 100 90.9 90.9 90.9  81.0 81.4 77.3 81.8  57.1 57.3 48.3 63.6  18.7 22.2 25.7 20.4 21.7 Total Avg. Autumn Winter Spring Summer Overall 81.0 78.5 72.2 80.2 78.0 59.4 58.3 56.3 59.6 58.4 58.1 41.0 32.6 49.2 45.2 31.4 37.6 43.4 34.0 36.6    85  3.3.3 Functional Performance Benchmarking  Observing water quality through the lens of a WQI provides a measurable and comparable value for comparison across different SDWSs and creates a pathway to determine which indicators are most impacting the drinking water quality. Using the WQI data, the performance of each individual SDWS can then be compared to an established benchmark (Total Avg./Overall in Table 3-3).   3.3.4 Identification and Prioritization of Improvement Actions  A generalized improvement matrix consisting of source water protection strategies, drinking water treatment improvements, and distribution system management upgrades was developed for SDWS operators and managers through a review of improvement techniques (Table 3-4).  Table 3-4. Generalized improvement matrix for performance improvement Type  Improvement actions  Source water protection  (Z1) Z11 Intake depth modifications  Z12 Extended (dry) detention ponds Z13 Grass swales Z14 Wetlands Z15 Wet detention ponds Drinking water treatment  (Z2) Z21 Basic treatment: chlorination (primary and secondary disinfection) and pH adjustment Z22a Conventional treatment: coagulation, flocculation, and slow sand filtration (chlorination as primary and secondary disinfection) Z23a Advanced primary disinfection: ozonation as a replacement for primary chlorination Z24a Advanced secondary disinfection: chloramination as a replacement for secondary chlorination Distribution system management (Z3) Z31 Chlorine booster stations in distribution network Z32 Water storage tank aeration a Represents drinking water treatment improvements in addition to Z22  These actions can be implemented individually or as a planned improvement chain. This table does not represent an exhaustive list of applicable actions for a given water utility, but provides a   86  general template for developing performance improvement chains for individual SDWSs at the source, treatment, and within the distribution system.  When creating a performance improvement chain for drinking water quality improvement, actions identified at the source are often the most cost-efficient (Clark and Boutin, 2001). Management at the source is more affordable than comparable upgrades to a water treatment facility or distribution network and can provide a more preventative option as opposed to the more traditional corrective retrofits (Islam et al., 2011). For DBPs, there are two primary approaches used to reduce the formation of DBPs during treatment in SDWSs. The first consists of the reduction of DBP precursors in the raw source water prior to chlorination and the second involves implementing alternative disinfectants for primary and/or secondary disinfection (Bond et al., 2011). While the latter is less expensive and should be given priority, the former should eventually be implemented at a later stage of the CPI framework.  To project performance improvement, a qualitative cumulative ranking mechanism was used (Table 3-5). A qualitative approach was chosen due to the variation and uncertainty in projecting performance improvement from case studies and pilot projects found in the literature. Although this method does not allow for exact projections of future water quality, it can provide a valuable resource for drinking water utility planners.        87  Table 3-5. Ranking mechanism for cumulative performance improvement Ranking Estimated Cumulative Performance Improvement Very low (VL) 0-20% Low (L) 21-40% Medium (MED) 41-60% High (H) 61-80% Very high (VH) >80%  For the purpose of demonstrating the CPI framework, a performance improvement plan was developed using performance improvement results from the Government of Newfoundland and Labrador (2009) and from a pilot study conducted by Guay et al. (2005) in Quebec (Table 3-6). Cumulative performance improvement was then estimated quantitatively and applied to the selected improvement actions. Although performance improvement data was available for THM4, HAA5, turbidity, and TC, no performance improvement information was available for FRC. It is important to note that the performance improvement results obtained from these studies are site-specific and may not reflect actual performance improvement at a given SDWS. However, information from the literature can be used to estimate and project potential improvements as part of the CPI framework. For more comprehensive and accurate projections of drinking water quality and performance improvement advanced modeling techniques, such as those explored by Rodriguez et al., (2000) for THM formation, can be explored.  Table 3-6 shows the cumulative improvement from the selected improvement chain. In this demonstration, the improvement from a given action is dependent on the implementation of prior improvement items from the chain.     88  Table 3-6. Improvements chosen for Newfoundland and Labrador demonstration  Action Year of Implementation THM4 HAA5 Turbidity TC Z11 Intake depth modifications 2017 La La - - Z22 Conventional: coagulation, flocculation, and slow sand filtration (chlorination as primary and secondary disinfection) 2020 Lb Lb Hb MEDb Z23 Advanced primary disinfection: ozonation as a replacement for chlorination 2025 Hb Hb - - Z24 Advanced secondary disinfection: chloramination as a replacement for chlorination 2030 VHb VHb - - a Government of Newfoundland and Labrador, 2009 b Guay et al., 2005  3.3.5 Implementation of Improvement Actions  After calculating the WQI and choosing appropriate performance improvement actions, performance improvement was projected for the existing benchmark [see Section 3.4 below].   3.4 Demonstration Results for Overall WQI Performance Figure 3-6 shows the results of the CPI framework for the SDWSs in Newfoundland and Labrador. In Figure 3-6, the black markers represent the seasonal SDWS data collected from the seven Newfoundland and Labrador sites and the solid black line represents the established functional performance benchmark. The benchmark was calculated as the average of the seasonal data points from the seven SDWSs (36.6). Based on data availability and water quality testing frequency, the benchmark can also be calculated seasonally or monthly using either an average value or minimum value (depending on the quality of data). The gray shaded area represents the range of performance (and projected range of performance) between the highest and lowest performing water utilities. In the figure, Number 1 represents the implementation of intake depth modifications (Table 3-6 – Z11), Number 2 represents the implementation of coagulation, flocculation, and slow sand filtration (with chlorination being used for both primary   89  and secondary disinfection) (Table 3-6 – Z22), Number 3 and 4 represent the replacement of chlorination with ozonation for primary disinfection (with chlorination still used for secondary disinfection) (Table 3-6 – Z23) and the replacement of chlorination with chloramination for secondary disinfection (Table 3-6 – Z24) respectively.    Figure 3-6. Continuous performance improvement framework application for overall WQI score of three selected small drinking water system in Newfoundland and Labrador demonstration  As Figure 3-6 shows, lower performance water utilities will improve rapidly with the implementation of improvement strategies, while higher-performing utilities will have a relatively gradual improvement in overall water quality. As the benchmark increases, individual water utilities also cluster, leading to higher quality, more consistent drinking water (i.e. tighter clustering around the benchmark). After creating the existing benchmark from the seven water utilities, three specific SDWSs from the benchmark were individually investigated (Figure 3-6) for the selected improvement actions in Table 3-6. SDWS 5, 6, and 7 were chosen to represent   90  performance improvement in a high performing, average, and low performing water utility. It is evident in Figure 3-6 that each of the selected SDWSs has a different performance response to each of the chosen improvement actions. For example, in 2030 with the replacement of chlorination with chloramination for secondary disinfection (Table 3-6– Z24) as part of the selected improvement plan, SDWS 6 is projected to have a massive water quality improvement, while SDWS 5 is projected to remain stagnant. This is due to the variation in baseline water quality and differing water quality indicators of concern for each site. In this instance, SDWS 5 could be better suited to selecting a more appropriate action for improvement, such as booster chlorination in the distribution network which would mitigate some of the FRC issues found. It is also of note that SDWS 6 is projected to perform lower than the projected range of performance for improvement actions 3 and 4 (Figure 3-6). In this instance, other improvement actions should be investigated (that better mitigate the water quality concerns facing SDWS 6) to provide a better water quality projection and better use of already limited resources.  3.5 Demonstration Results for an Individual Performance Indicator (THM4) The CPI framework was also plotted for an individual PI, THM4, to show the performance improvement in relation to the current drinking water standards. In Figure 3-7, the CPI framework for THM4 at sampling point R1 in the distribution network is shown.     91   Figure 3-7. Continuous performance improvement framework application for functional performance benchmarking of THM4-R1 of three selected small drinking water systems in Newfoundland and Labrador demonstration  As is evident in Figure 3-7, the existing benchmark consisting of the seven Newfoundland and Labrador SDWSs, is calculated higher than both the Health Canada guidelines and USEPA standards. As the improvement actions are implemented, the future projected benchmark is projected to first exceed the Health Canada (2014) guideline and later exceed the USEPA (2009) standard. The CPI concept illustrates improvement beyond regulatory standards. This provides resiliency in the event of changing legislation or customer concern and, as time progresses, the clustering of the water utilities also becomes apparent. This signifies more consistent water quality in terms of THM4 formation.    92  After creating the existing THM4-R1 benchmark, the three specific SDWSs were again compared to the benchmark (Figure 3-7). SDWS 5, 6, and 7 were chosen to represent performance improvement in a high performing, average performing, and low performing water utility.   While CCME WQI varies by projected improvement strategy given the multiple drinking water indicators incorporated, THM4 at R1 has a consistent rate of improvement across all SDWSs. However, much like the WQI performance improvement, THM4 improvement occurs more quickly with lower performing utilities and more gradually with better performing utilities. After reviewing both Figure 3-6 and Figure 3-7, it is apparent that SDWS 5 would benefit much more targeting a drinking water indicator other than THM4 as the resources needed for this improvement are not practical given the other water quality concerns  3.6 Summary The CPI framework provides a tool for gauging current performance against similar water utilities and projecting water quality improvements into the future, through the lens of both overall water quality and specific drinking water indicators. It incorporates a WQI-based approach to aggregated PIs and functional performance benchmarking to incentivize performance improvement. This CPI framework not only provides comparisons and future projections but creates an opportunity for SDWS operators and planners to carefully manage limited resources and implement improvement strategies that will result in higher quality, more consistent drinking water.    93  Chapter 4: Risk-based Water Quality Performance Benchmarking for Small Drinking Water Systems A part of this chapter has been submitted for publication in Environmental Monitoring and Assessment, a Springer journal, as an article titled “Small Drinking Water Systems under Spatiotemporal Water Quality Variability: A Risk-based Performance Benchmarking Framework” (Bereskie et al., 2017d).  With regards to Objective 3 as defined in Chapter 1, a hierarchical risk-based water quality performance benchmarking framework integrating FRBM for SDWSs (RWQI) was proposed in Chapter 4 to incorporate spatiotemporal variability and degrees of compliance/non-compliance into drinking water quality assessment. This approach was then implemented and compared to the CCME WQI using drinking water quality data from 16 SDWSs in Newfoundland and Labrador and Quebec, Canada. Finally, Monte Carlo simulations were performed to evaluate the sensitivity of the proposed framework. Chapter 4 builds on the continuous performance improvement and performance benchmarking foundation established in Chapter 3 and addresses some of the drinking water challenges and SDWS challenges identified in Chapter 2.  4.1 Background Traditionally, the overall performance of a drinking water supplier is quantified based on either 1) the exceedances of individual PIs or 2) by using a WQI. The first, individual PI exceedances, is determined by calculating the total number of exceedances (or percent of exceedances) over a given time period. The results from all available or selected PIs are then evaluated (e.g. summed   94  or averaged annually) to gauge the overall water quality performance of an individual drinking water supplier (Chang et al., 1999; Hurley et al., 2012). The second approach to measuring the overall performance of a drinking water supplier involves the use of a WQI [Section 3.1.3]. Both of these traditional approaches to drinking water quality fail to incorporate the consequence of spatiotemporal variability (i.e. water quality failure dependent on location in the drinking water supply system and time of the year). This leads to performance assessment data lacking resolution, applicability, and usefulness, especially in SDWSs that can be prone to significant spatiotemporal variability (Scheili et al., 2014).  In Chapter 3, a water quality performance benchmarking framework designed to incorporate the CCME WQI as a means for continuous performance improvement was proposed. While this method incorporates multiple PIs to generate an overall water quality score for easy implementation into the performance benchmarking process, it can still be considered binary, as the foundation of the CCME WQI is based on either compliance or non-compliance of a user-defined guideline or range for each PI. In order to move beyond this paradigm, risk, in terms of water quality, can be incorporated.  4.1.1 Risk Management There are different definitions for risk throughout literature, but it is commonly understood as the combination of the probability or likelihood of the occurrence of an event and the severity of consequence of the event which must be determined separately (Equation 4-1) (Mitchell, 1999; Ale, 2002; Aven and Jinnem, 2007; USFDA, 2009).     95  Risk score (RS) = Probability of occurrence × Consequence                      (Equation 4-1)   It has been used as a metric for performance evaluation in a wide variety of fields since gaining prominence after World War II (Dionne, 2013). Risk management refers to both the avoidance of hazards and the reduction of their potential harm and often involves decision-making in situations with high risks and significant uncertainties (Aven and Jinnem, 2007; Häring, 2015). The risk management process (Figure 4-1) is generally considered to have three distinct processes; risk analysis, risk evaluation, and risk control/mitigation (Blackhurst et al., 2008; Häring, 2015).   Figure 4-1. Generic framework for risk management  Risk analysis is the determination of risks in a given context including the identification of hazards and threats, cause analyses, consequence analyses, and risk description (Aven and Jimmen, 2007; Häring, 2015). For example, some risks associated with drinking water in SDWSs include inadequate treatment technologies and undertrained operators. Risk evaluation refers to the ranking and prioritization of identified risks along with the determination of   96  acceptable levels of risk (Häring, 2015). For example, the risk presented from an increased turbidity levels is dwarfed by the risk presented from an increased presence of E.coli. Finally, risk control/mitigation refers to the process of developing and implementing measures designed to reduce, modify, or transfer risk (Aven and Jinnem, 2007). For example to address risk of high turbidity in a DWSS, a utility may consider adding filtration treatment capabilities. This step also includes risk monitoring, which is often overlooked in literature, but is important in ensuring CPI (Blackhurst et al., 2008).  4.1.1.1 Risk-based Drinking Water Management Some commonly used risk assessment techniques are Bayesian networks, event tree analysis (ETA), failure mode effect analysis (FMEA), failure mode effects and criticality analysis (FMECA), fault tree analysis (FTA), hazard and operability (HAZOP), human reliability analysis (HRA), Markov analysis, and reliability block diagram (RBD) (Hokstad et al., 2009). There have also been previous applications for risk-based methodologies in drinking water, such as presented by Sadiq et al., (2004) to aggregate water quality failure in distribution networks, the Ontario Ministry of Health and Long-Term Care (MOHLTC) (2008) for drinking water quality testing frequency, Haider et al., (2016c) for customer satisfaction management in small and medium-sized water utilities, and Turner et al. (2016) to account for uncertainty due to climate change. However, despite this widespread application in the field of drinking water, the incorporation of risk into drinking water quality assessment and evaluation has yet to be explored in depth.    97  4.1.2 Fuzzy Sets All risk-based approaches can face uncertainties associated with data limitations, sampling error, and vagueness in expert opinions (Haider et al., 2016c; Goodwin et al., 2015; Sadiq et al., 2004). This is particularly true when quantifying the risk of drinking water quality failure in SDWSs, which often rely only on one part-time staff member (often with limited training and experience) for operations, monitoring, and troubleshooting (Coulibaly and Rodriguez, 2003; Haider et al., 2016c). While subjectivity and judgment are not typically associated with measuring drinking water quality against defined guidelines/standards, fuzzy set theory can be used to address the imprecision associated with measuring performance based on singular water quality guidelines/standards and the uncertainties present in SDWSs operations and monitoring (Mujumdar and Sasikumar, 2002).   The theory of fuzzy sets refers to the concept in which everything is considered based on degrees of certainty (Zimmerman, 2001). This is in contrast to traditional crisp sets, which is based on conventional binary logic in which statements can be either ‘true’ or ‘false’, with nothing in between (Zimmerman, 2001). The concept of fuzzy sets was introduced by Zadeh (1965) and describes a fuzzy set ‘A’ as the relationship between an uncertain quantity ‘x’ and a membership function ‘µA’, which has a range between 0 and 1, with 0 representing complete uncertainty (i.e. non-membership) and 1 representing complete certainty (i.e. full membership) (Zadeh, 1965; Zimmermann, 2001; Ross, 2010). Fuzzy sets are an extension of traditional (crisp) set theory (where ‘x’ has either full membership in ‘A’ or not at all) (Sadiq et al., 2004). Figure 4-2 illustrates the differences between a crisp set and a fuzzy set.    98   Figure 4-2. Differences between crisp sets and fuzzy sets (adapted from Sadiq, 2015)  4.1.3 Fuzzy Logic Fuzzy logic is the practical application of fuzzy set theory (Klir et al., 1997). The term fuzzy logic was first introduced by Zadeh (1965) with his proposal of fuzzy set theory and expanded on with subsequent research on fuzzy systems (Zadeh, 1971) and linguistic variables and truth operators for soft computing (Zadeh, 1975b; Zadeh, 1997; Zimmermann, 2001). While traditional logic like crisp numbers represents either ‘true’ or ‘false’ propositions, fuzzy logic incorporates degrees of truth for reasoning (Klir et al., 1997; Klir and Yuan, 2000). Lee (1990) described the differences between fuzzy logic and traditional logical systems stating that fuzzy logic systems “…provide an effective means of capturing the approximate, inexact nature of the real world,” and Zadeh (1975a) described fuzzy logic as a logic of approximate reasoning using   99  linguistic variables (e.g. ‘true’, ‘rather true’, ‘very true’, ‘false’), which refer to linguistic terms represented by values of crisp numbers within a specific range (Klir and Yuan, 2000).  Figure 4-3 highlights the differences between traditional logic and fuzzy logic using concentration of turbidity (NTUs) and linguistic risk values (i.e. ‘Low’, ‘Medium’, and ‘High’) as an example. While traditional logic has defined thresholds between linguistic terms (i.e. 0.75 NTUs is the threshold between ‘Low’ and ‘Medium’ risk), fuzzy logic has degrees of truth (i.e. at 2.25 NTUs, risk has partial (0.5) membership in both ‘Medium’ and ‘High’).   Figure 4-3. Differences between traditional logic and fuzzy logic  4.1.3.1 Fuzzification  Fuzzification refers to the conversion of a precise, crisp value to a fuzzy value (Ross, 2010). While any shape of a fuzzy number is possible, the selected shape should be justified by available information and resolution of the outcome. Generally, triangular fuzzy numbers (TFN)   100  (Figure 4-2) or trapezoidal fuzzy numbers (ZFN) are most popular for representing linguistic values (Sadiq et al., 2004). Figure 4-4 shows the standard trapezoidal membership function used in this research with ‘b1’, ‘b2’, ‘b3’, and ‘b4’ defining the ranges of a ZFN for ‘Medium’.   Figure 4-4. Standard trapezoidal membership function  4.1.3.2 Defuzzification  Defuzzification refers to the process of transforming a fuzzy number or fuzzy set into a crisp (or point estimate) value (Ross et al., 2010; Sadiq et al., 2004). Defuzzification methods vary from simplistic (e.g. based on minimum or maximum membership) to more complex (e.g. Chen’s 1985 ranking method) (Zimmermann, 2001; Chen, 1985). The most common types of defuzzification are based on extreme values or centroid approaches. Extreme value approaches to defuzzification are designed to use extreme values (usually the maximum value) based on a calculated core to define a crisp output. Common extreme defuzzification techniques include left of maximum (LOM), right of maximum (ROM), or center of maximum (COM) (Zimmermann, 2001).     101  Centroid approaches, also known as area methods, were developed to consider more information than extreme value approaches (Zimmermann, 2001). The best-known centroid approaches are center of area (COA) and center of gravity (COG). The COA method is based on the center of the area with membership greater than zero while the COG method is based on the center of the largest area (Zimmermann, 2001; Ross, 2010). Other popular defuzzification methods include the max membership principle, the weighted average method, the mean-max membership (also known as the middle-of-maxima), and the center of sums method (Ross, 2010).   4.1.3.2.1 Centroid Defuzzification For the purpose of this research, centroid defuzzification returning the center of the area under the curve was selected for use as it represents one of the most prevalent and appealing defuzzification option (Ross, 2010). The centroid technique was developed by Sugeno (1985) and is calculated using Equation 4-2.  𝒛∗ =  ∫ 𝛍𝑨(𝒛)∗𝒅𝒛∫ 𝛍𝑨(𝒛)𝒅𝒛                                        (Equation 4-2)  In Equation 4-2, ‘z*’ represents the crisp output, ‘UA’, is the aggregated membership function, and ‘z’ is the output variable.  4.1.4 Fuzzy Rules Adriaenssens et al. (2004) describes fuzzy rules, also known as fuzzy IF-THEN rules, as consisting of two parts: an antecedent part stating the conditions of input variables (e.g. IF Input 1 is ‘high’), and a second part describing the corresponding values of the output variables (e.g.   102  THEN Output 1 is ‘high’). Since most systems involve more than one rule, an aggregation strategy must be implemented (Ross, 2010). There are two main aggregation strategies - conjunctive (i.e. rules that must be jointly satisfied) where the logical operator ‘AND’ is used (e.g. Input 1 is ‘low’ AND Input 2 is ‘low’ THEN Output 1 is ‘low’) and disjunctive (i.e. system where at least one rule must be satisfied) where the logical operator ‘OR’ is used (e.g. Input 1 is ‘high’ OR Input 2 is ‘high’, THEN Output 1 is ‘high’) (Ross, 2010).  4.1.5 Fuzzy Inference Systems Fuzzy inference systems (FISs), also known as fuzzy rule-based modeling (FRBM) systems, are designed to incorporate estimates from one or more inputs and generate a set of outputs using the fuzzy set theory [Section 4.1.2] and fuzzy IF-THEN rules [Section 4.1.4] (Ross, 2010; Islam et al., 2013b). While there are many documented advantages with using an FIS system, arguably the most important is that it allows for the incorporation of both qualitative (i.e. expert opinion) and quantitative (i.e. water quality data) information (Jang, 1993). The two most common types of FIS are the Mamdani-type FIS (Mamdani, 1977) and the Sugeno-type FIS (Takagi and Sugeno, 1985) (Kaur and Kaur, 2012). Descriptions and differences of the two types of FISs can be found below in Sections 4.1.5.1 and 4.1.5.2.   4.1.5.1 Mamdani-type FIS  The Mamdani-type FIS (Mamdani 1977) is the most popular FIS in use today because of the intuitive manner it uses to describe expert knowledge (Ross, 2010, Kaur and Kaur, 2012). It is also the most popular type of FIS used in environmental management settings (Adriaenssens et al., 2004). Despite its widespread applications, the Mamdani-type FIS is more computationally   103  demanding than other methods that quantify outputs as crisp values (Kaur and Kaur, 2012). The Mamdani-type FIS can be broken into four basic steps. Step 1: Fuzzification of inputs [Section 4.1.3.1] Step 2: Determination of fuzzy IF-THEN rules for all inputs and outputs. For example, IF THM4 concentration is ‘low’, THEN chemical water quality is ‘high’ Step 3: Use of fuzzy operators (e.g. AND, OR, NOT) to infer relationships between inputs and outputs. For example, IF THM4 concentration is ‘low’ AND HAA5 concentration is ‘high’, THEN chemical water quality is ‘medium’ Step 4: Defuzzification of outputs [Section 4.1.3.2]  In Equation 4-3, Ross (2010) describes a typical rule in a Mamdani-type FIS (with two non-interactive inputs ‘x1’ and ‘x2’ (antecedents) and a single output ‘y’ described by a collection of ‘r’ linguistic IF-THEN propositions) as:  IF x1 is A̰1k and x2 is A̰2k THEN yk is B̰k, for k = 1, 2,…,4                                                 (Equation 4-3)  Where ‘A̰1k’ and ‘A̰2k’ are fuzzy sets representing the kth antecedent pairs and ‘B̰k’’ is the fuzzy set representing the kth consequent.  4.1.5.2 Sugeno-type FIS  The Sugeno-type FIS (Takagi and Sugeno, 1985) also known as the Takagi-Sugeno-Kang FIS, was developed in 1985 as an alternative to the Mamdani-type FIS (Takagi and Sugeno, 1985). This technique has been lauded as computationally efficient and is popular for uses with   104  optimization and adaptive techniques (Kaur and Kaur, 2012). The Sugeno-type FIS is exactly the same as the Mamdani-type FIS in regards to fuzzifying inputs and applying fuzzy operators, but differs in generating outputs. While the Mamdani-type FIS uses the defuzzification of a fuzzy output to generate a crisp value, the Sugeno-type FIS uses a weighted average to compute a crisp output (i.e. there is no output membership function and no defuzzification) (Knapp, 2007; Kaur and Kaur, 2012). In Equation 4-4, Ross (2010) describes a typical rule in a Sugeno model (with 2 inputs ‘x’ and ‘y’ and output ‘z’) as having the form:  IF x is A̰ AND y is B̰ THEN z is z = f(x,y)                                                   (Equation 4-4)  Where ‘z = f(x,y)’ is a crisp function. In the event that ‘z = f(x,y)’ is a constant, the FIS is called a zero order Sugeno model (Ross, 2010).  4.2 Approach and Methodology For the proposed framework, a risk-based approach incorporating spatiotemporal variability, the Mamdani-type FIS, and functional performance benchmarking approach is presented (Figure 4-5). The overall goal of this framework is to address issues associated with drinking water performance data that is lacking resolution, applicability, and usefulness while facilitating a more efficient decision-making process for SDWS planners and operators. A conceptual diagram for the proposed risk-based water quality benchmarking framework can be found in Figure 4-5 and each step is described in detail below.    105   Figure 4-5. Proposed framework for risk-based performance benchmarking of small drinking water systems  4.2.1 Establish Performance Indicators  This framework initiates with a review of potential water quality PIs using available literature, governmental reports, and available data [Step 1]. These PIs should encompass a complete picture of drinking water quality from catchment to consumer, although the exact PIs chosen may vary depending on available data. Haider et al. (2014) previously reviewed specific PIs for   106  small and medium-sized water supply systems and Swamee and Tyagi (2000), Hurley et al., (2012), Scheili et al. (2015), have proposed specific PIs for inclusion into WQIs tailored for drinking water quality.  4.2.2 Select and Categorize Performance Indicators  Water quality PIs are generally classified into three broad categories; biological, chemical, and physical contaminants, although more categories can be incorporated when considering other factors (e.g. customer satisfaction, financial/economic information, operational efficiency, and quality of service)(Sadiq et al., 2003; Haider et al., 2014). In Step 2, PIs are selected and categorized. An example of potential PIs can be found in Table 4-1.  Table 4-1. Example of potential microbiological, chemical, and physical drinking water performance indicators (WHO, 2006; Hurley et al., 2012; Islam et al., 2013a; Scheili et al., 2014) Microbiological Chemical Physical Cryptosporidium counts E.coli counts Enterovirus counts Fecal coliforms (FC) Giardia counts Heterotrophic plate counts (HPC) Biological oxygen demand (BOD) Chemical oxygen demand (COD) Emerging DBPs Free residual chlorine (FRC) Haloacetic acids (HAAs) Haloacetonitriles (HANs) Haloketones (HKs) Heavy metals Total chlorine Total hardness Total nitrogen Total phosphorous Trihalomethanes (THMs) Alkalinity Conductivity Total dissolved solids (TDS) Total organic carbon (TOC) Total suspended solids (TSS) Turbidity UV254 absorbance  After identifying potential PIs, multi-criteria analysis, like that used by Haider et al. (2015b), can be used for the selection of applicable PIs, however, data availability is often a limiting factor, especially when evaluating SDWSs.     107  4.2.3 Acquire Water Quality Data  The third step of the framework involves the collection of water quality data for the selected PIs. This step includes the incorporation of data highlighting spatial (i.e. location in the supply system) and temporal (i.e. seasonal) variation. While traditional benchmarking typically relies on annual results, this can create issues, specifically with SDWSs that are often more vulnerable to seasonal water quality variation (Rodriguez et al., 2003; Coulibaly and Rodriguez, 2003; Francisque et al., 2009; Scheili et al., 2014; Dyck et al., 2014; Guilherme and Rodriguez, 2014 ).   4.2.4 Establish Universe of Discourse and Hierarchical Mapping of Selected Performance Indicators  Establishing a universe of discourse (UOD) is one of the first steps associated with implementing fuzzy logic as it represents all possible values for an input to a fuzzy system and represents Step 4 of the RWQI framework (Ross, 2010). The established UODs for the PIs, spatial factors, and temporal factors can be developed using literature and regulatory guidelines and standards. For the purpose of this framework, most of the factors are linguistically described as ‘Very Low’, ‘Low’, ‘Medium’, ‘High’, or ‘Very High’ or ‘Low’, ‘Medium’, or ‘High’ in the form of fuzzy trapezoidal numbers. A description of the linguistic risk terms, associated UOD, and applicable water quality, spatial, and temporal criteria can be found in Table 4-2.        108  Table 4-2. Linguistic description, universe of discourse, and associated water quality, spatial, and temporal criteria Linguistic risk description Universe of discourse (UOD) (5 descriptors) Universe of discourse (UOD) (3 descriptors) Water quality criteria Spatial criteria Temporal criteria Very low (VL) 0, 0, 1, 2 - Water quality is well below standards and/or guidelines - - Low (L) 1, 2, 3, 4 0, 0, 2, 4 Water quality is below standards and/or guidelines Water quality issues affect the least amount of people (i.e. beginning of supply system) Water quality issues arise when water use is at its lowest (i.e. Winter) Medium (M) 3, 4, 6, 7 2, 4, 6, 8 Water quality meets or slightly exceeds standards and/or guidelines Water quality issues affect a moderate amount of people (i.e. extremity of supply system) Water quality issues arise when water use is moderate (i.e. Autumn, Spring) High (H) 6, 7, 8, 9 6, 8, 10, 10 Water quality exceeds standards and/or guidelines Water quality issues affect the largest amount of people (i.e. middle of supply system) Water quality issues arise when water use is at its highest (i.e. Summer) Very High (VH) 8, 9, 10, 10 - Water quality greatly exceeds standards and/or guidelines  - -  After establishing the UOD, a hierarchy can be established. The proposed RWQI hierarchy and coding schematic can be found in Figure 4-6. This hierarchy consists of a bottom-to-top approach consisting of six generations. While a higher generation (e.g. Generation 1) consists of a performance value encompassing more information (i.e. more PIs), it lacks the resolution and utility of the information generated in the lower generations (e.g. Generation 5). A description and example of the coding schematic for each generation can be found as part of Figure 4-6.     109    Generation Description and example Coding example 1 Overall drinking water quality risk X 2 Risk of a performance category (e.g. Category 1) X1 3 Risk of an individual PI in a given category (e.g. Category 1, PI-1) X1,1 4 Seasonal risk of an individual PI (e.g. Category 1, PI-1, Winter) X1,1,2 5 Seasonal performance of an individual indicator at a given point in the distribution network  (e.g. Category 1, PI-1, Winter, Point 2 in Supply System) X1,1,2,2 6 Individual sample in a given season of an individual indicator at a given point in the distribution network (e.g. Category 1, PI-1, Winter, Point 2 in Supply System, Sample 1) X1,1,2,2,1 Figure 4-6. Proposed RWQI hierarchy, coding schematic, and description/example for each generation in the proposed framework  4.2.5 Calculate RWQI Using Fuzzy Rule-Based Modeling  For Step 5 of the proposed RWQI framework, the Mamdani-type FIS (Mamdani, 1977) IF-THEN rule system [Section 4.1.5.1] was selected as it as it represents the most common method of deductive inference for fuzzy systems based on linguistic rules (Ross, 2010). For the RWQI framework, a multiple input, single output (MISO) model is used. This approach determines an appropriate output using an input of multiple linguistic variables (Márquez-Vera et al., 2016). For example, a MISO model with three inputs can be written as follows (Equation 4-5):     110  Ri,j,k: If X1 is Ai, X2 is Cj, and X3 is Dk then Y is Bl;  i = 1, 2, …, M k = 1, 2, …,O                                    (Equation 4-5)       j= 1, 2, …, N  l = 1, 2, …, P  Fuzzy operators (e.g. AND, OR) were then established to generate the output using the established membership functions. For example, if ‘X1’ is ‘Low’ AND ‘X2’ is ‘Low’, AND ‘X3’ is ‘Medium’, then ‘Y’, the output, is defined as ‘Low’. To obtain crisp outputs, defuzzification was conducted for Generations 1 – 5 using the Sugeno (1985) centroid method in MATLAB Simulink [Section 4.1.3.2.1]. This allows for a crisp risk score (RS) at each of the generations and provides additional data for analysis and functional performance benchmarking [Step 6].    4.2.6 Transform Risk into RWQI Performance and Perform Risk-based Functional Performance Benchmarking  The final step [Step 6] of the proposed risk-based water quality performance benchmarking framework involves the transformation of RS (i.e. a value between 0 and 100, with 0 representing the lowest risk and 100 representing the highest risk) into RWQI performance (i.e. a value between 0 and 100, with 0 representing the lowest performance and 100 representing the highest performance) (Equation 4-6).  RWQI = (100 - X)                            (Equation 4-6)  The results can then be used for functional performance benchmarking to compare performance against a defined benchmark value and, directly or indirectly, other similar SDWSs. A conceptual diagram for the functional performance benchmarking can be found in Figure 4-7. In   111  this figure, three utilities are shown with varying levels of RWQI performance and RSs in relation to an established risk-based benchmark.    Figure 4-7. Conceptual diagram for proposed risk-based functional performance benchmarking (modified from Haider, 2016a)  For utilities performing well below the benchmark, major improvements are needed to address the large performance gaps. For utilities performing slightly below the benchmark, major or minor improvements are necessary depending on implement technology. For utilities performing better than the benchmark, priority shifts to continuous performance improvement, the concept of continuous measurable advancement and innovation (Bhuiyan and Baghel, 2005).   4.3 Demonstration Using Small Drinking Water Systems in Quebec and Newfoundland and Labrador, Canada The proposed RWQI framework was demonstrated using data collected from 16 SDWSs (eight from each province) in Newfoundland and Labrador and Quebec, Canada, and the results were compared to data aggregated using the CCME WQI. The data available for use in this demonstration was collected by Scheili et al. (2014) from a 2010-2011 sampling campaign of   112  SDWSs in Newfoundland and Labrador and Quebec. Quebec has a significantly different approach to drinking water treatment than Newfoundland and Labrador due to differing provincial regulations on water treatment for DWSSs using surface water sources. In Quebec, water treatment generally integrates a combination of physical and chemical treatments (e.g. chlorination and filtration) to raw source water whereas drinking water treatment in most DWSSs operating in Newfoundland and Labrador is much simpler (i.e. only chlorination) (Scheili et al., 2014).   4.3.1 Selected Small Drinking Water Systems In this demonstration, the selected SDWSs served populations between 300-4000 people. Information summarizing the implemented drinking water treatment systems along with types of raw water sources and population served are summarized in Table 4-3. All of the selected utilities are relying on surface water sources (e.g. ponds, lakes, streams, rivers). While the RWQI framework could apply to utilities using groundwater as a source, based on the fact that almost 90% of processed water volume treated by municipal operators is from surface sources in Canada, this framework is only applied to SDWSs primarily using surface water sources (Statistics Canada, 2011). However, if groundwater-sourced SDWSs were to be included, their drinking water quality performance should only be compared to that of other similar groundwater sourced utilities in the benchmarking process.       113  Table 4-3. System information for selected small drinking water systems in Newfoundland and Labrador and Quebec, Canada Newfoundland and Labrador (NL) SDWSs Pop.a Source  Chlorination Filtration Activated carbon UV NL SDWS-1 737 Pond X - - - NL SDWS-2 1984 Lake X X - - NL SDWS-3 335 Brook X - - - NL SDWS-4 2122 Pond X - - - NL SDWS-5 1031 Pond X - - - NL SDWS-6 1681 Brook X - - - NL SDWS-7 807 Pond X - - - NL SDWS-8 998 Pond X - - - Quebec (QC) SMWUs Pop.a Source  Chlorination Filtration Activated carbon UV QC SDWS-1 3439 River X X - - QC SDWS-2 966 River X X X - QC SDWS-3 1373 Lake X - - X QC SDWS-4 3458 River X X X X QC SDWS-5 1500 River X X - - QC SDWS-6 1596 Lake X X - - QC SDWS-7 1223 Lake X X - - QC SDWS-8 3880 Lake X - - - a Statistics Canada, 2011  Samples were collected monthly (and subsequently averaged seasonally) at multiple points within the distribution network (Scheili et al., 2014). The samples were analyzed using The Standard Methods for the Examination of Water and Wastewater, 19th edition (AWWA, 1999). Nine drinking water quality PIs were identified to demonstrate the framework. These indicators were selected to highlight the spatial and temporal variation of the treated drinking water and were categorized as either microbiological (i.e. E.coli, HPC, TC), chemical (i.e. FRC, HAA5, THM4), or physical (i.e. pH, temperature, turbidity).  4.3.2 Temporal Risk Temporal (i.e. seasonal) risk and the subsequent consequence was incorporated into the RWQI framework through FRBM based on the findings of a Statistics Canada (2013) survey reporting the 2007 and 2011 seasonal potable water use volumes throughout Canada (Figure 4-8). In this demonstration, a higher risk and subsequent consequence is attributed to higher water use/consumption (e.g. bathing, cooking, and drinking) and a lower risk and subsequent   114  consequence is attributed to lower water use/consumption. By incorporating temporal risk, drinking water owners, operators, and planners can quantify the consequence between operational disruptions that may occur in summer (i.e. when water demand is at its highest) and winter (i.e. when water demand is at its lowest). Based on the survey, winter was designated as ‘Low risk’, autumn and spring were designated as ‘Medium risk’, and summer was designated as ‘High risk’ for the framework’s implementation.    Figure 4-8. Survey results (2007 and 2011) of potable water use volumes in Canada and assigned seasonal risk (Statistics Canada 2013)  Although this represents a somewhat simplistic approach to integrating the concept of temporal risk and consequence, it provides a methodology for increasing data resolution and usefulness for utility owners, planners, and operators without changing existing drinking water quality sampling procedures/requirements (i.e. limited or no additional financial and human resource burden). This is especially important in SDWSs, where human resources are limited and skilled   115  labor is at a premium. While these risk designations will most likely be similar in most of the north hemisphere (i.e. the highest risk associated with summer when water demand is highest), the inputs can be tailored by operators and planners for site-specific conditions for a given drinking water utility (i.e. changing or altering risk designations based on specific drinking water demand) and even potentially expanded given additional resources.  4.3.3 Spatial Risk Spatial risk was incorporated into the RWQI framework through FRBM using the DWSS spatial risk diagram found in Figure 4-9. In this figure, the beginning of the supply system (i.e. R1, the area closest to the treatment facility) is classified as ‘Low risk’, being less susceptible to cross-contamination through broken or leaking water distribution mains and serving the smallest population. The middle of the supply system (i.e. R2, the area serving the highest portion of a given population) is classified as ‘High risk’. The extremity of the supply system (i.e. R3, the area furthest from the treatment facility) is classified as ‘Medium risk’ due to the possibility of encountering the maximum number of system failures and serving the second highest population. These classifications were developed based on likely population served and distance from the water treatment facility.     116   Figure 4-9. Drinking water supply system spatial risk diagram  Although this diagram is not necessarily applicable in all situations or for all PIs, it provides a general risk diagram that can help to quantify spatial risk as is, or can be used as a template for SDWS planners and operators to use when tailoring a more specific risk diagram for a given drinking water utility.  4.3.4 Fuzzy Rule-based Modeling The UOD for each of the selected PIs was determined using the Health Canada (2014) GCDWQ, the USEPA (2009) National Primary Drinking Water Regulations, and the Islam et al. (2013b) study highlighting an optimal FRC range in SDWSs (Table 4-4). Table 4-4 shows the defined UOD for each PI along with highlighting the values used for the CCME WQI calculations as part of the comparison in Section 4.4. The UOD was developed to incorporate degrees of compliance and non-compliance. For example, with THM4, the Health Canada (2014) guideline is 0.1 mg/L. If a sample has a concentration of 0.11 mg/L, while above the guideline, is only defined as   117  ‘Medium’ risk. However, a THM4 sample concentration of 0.25 mg/L, also above the guideline, would be considered as a ‘Very High’ risk.  Table 4-4. Selected performance indicators and established universe of discourse for demonstration Code PIs Units CCME WQI Guidelines Universe of Discourse (UOD) Very Low Low Medium High Very High  Microbiological PIs (X1) X1,1 E.coli No. / 100mL 0a,c 0 - 1 0.5 - 2 1.5 - 3 2.5 - 6 5.5 - 10 X1,2 HPC No. / 100mL 0 0 - 1 0.5 - 2 1.5 - 3 2.5 - 6 5.5 - 10 X1,3 TC No. / 100mL 0a,c 0 - 1 0.5 - 2 1.5 - 3 2.5 - 6 5.5 - 10  Chemical PIs (X2) X2,1 FRC mg/L 0.2-0.8b - 0.2-,0.9 0.05-0.25, 0.8-1.6 0-0.1, 1.5-4 - X2,2 THM4 mg/L 0.1a 0 - 0.05 0.025 - 0.1 0.075 - 0.15 0.125 - 0.2 0.175 - 0.3 X2,3 HAA5 mg/L 0.1a 0 - 0.05 0.025 - 0.1 0.075 - 0.15 0.125 - 0.2 0.175 - 0.3  Physical PIs (X3) X3,1 pH - 7-10.5a - 7 - 10.5 6 – 7.5, 10-11.5 3 - 6.5, 11-13 - X3,2 Temperature °C 15a 0 - 5 2.5 - 10.5 8 - 16 13.5 - 23.5 21.5 - 30 X3,3 Turbidity NTU 1a,b 0 - 0.5 0.25 - 1.25 1 - 2 1.75 - 3.25 3 - 5 a Health Canada, 2014 b Islam et al., 2013a c USEPA, 2009  Risk scores for Generations 1-5 were then calculated using Fuzzy Logic Toolbox in MATLAB Simulink. A description of RWQI and CCME WQI values, rankings, and description can be found in Table 4-5.  Table 4-5. Description of RWQI and CCME WQI values/rankings (CCME, 2001) RWQI Fuzzy Values RWQI Performance RWQI Risk CCME WQI Values CCME WQI Rankings CCME WQI Description 0-20 Very low  Very High  0-44 Poor Water quality virtually never exceeds standards and/or guidelines, DWAs virtually never occur. 10-40 Low  High 45-64 Marginal Water quality rarely exceeds standards and/or guidelines, DWAs rarely occur. 30-70 Medium  Medium 65-79 Fair Water quality issues sometimes exceed standards and/or guidelines, DWAs sometimes occur. 60-90 High  Low 80-94 Good Water quality often exceeds standards and/or guidelines, DWAs often occur. 80-100 Very High  Very Low 95-100 Excellent Water quality virtually always exceeds standards and/or guidelines, DWAs are common.    118  The number of rules for each generation was based on the number of inputs. 15 fuzzy rules were established for each input in Generation 6; 125 fuzzy rules were established for each input in Generation 5; 625 fuzzy rules were established for each input in Generation 4; 125 fuzzy rules were established for each input in Generation 3; and 125 fuzzy rules were established for each input in Generation 2. The full matrix of fuzzy rules established for the framework can be found in Appendix B.  4.4 Results Risk scores were calculated for each output (at each generation) for each of the 16 SDWSs in Newfoundland and Labrador and Quebec. After calculating the RS for each of the SDWSs, the results were transformed into RWQI. The performance results for all 16 SDWSs for RWQI and CCME WQI results for Generation 1 (i.e. overall drinking water quality) can be found in Figure 4-10 and Appendix C. The most obvious characteristic in this graph is the difference between drinking water quality between Newfoundland and Labrador and Quebec. This can be attributed to the differences in implemented drinking water treatment technologies, with Quebec consistently having more advanced systems in place than can be found in Newfoundland and Labrador (see Table 4-3).   119   Figure 4-10. Comparison of CCME WQI scores and RWQI for all 16 SDWSs at Generation 1: a) NL small drinking water systems, b) QC small drinking water systems  In Figure 4-10, some utilities show little difference between RWQI and CCME WQI (e.g. NL SDWS-6), while the variation of others, such as QC SDWS-6, are readily apparent. As the same water quality data is used for both approaches, similar results were expected despite the   120  incorporation of additional information (i.e. spatiotemporal variation and degrees of compliance/non-compliance). However, the utilities showing large differences between the two approaches highlight how the traditional water quality assessment results can be misleading. When benchmarking small groups of similar drinking water utilities (like eight in both Newfoundland and Labrador and Quebec), misleading data from just one SDWS can significantly alter the entire benchmarking process. To investigate this further, the CCME WQI and RWQI Generation 2 performance (i.e. for each PI category) for all 16 SDWSs was calculated and plotted in Figure 4-11 to help further highlight the differences between the approaches.   121   Figure 4-11. Comparison of CCME WQI and RWQI for all 16 SDWSs at Generation 2: a) NL SDWSs, b) QC SDWSs  In Figure 4-11, the differences between CCME WQI and RWQI are illustrated for the microbiological (M), chemical (C), and physical (P) categories defined in Table 4-4. While the results in Figure 4-10 showed very similar results for many SDWSs, Figure 4-11 highlights the differences between PI categories for each SDWS. In this figure, the large variation between   122  CCME WQI and RWQI in QC SDWS-6 is shown with higher resolution. While the CCME WQI score is above 80 for each PI category, the RWQI results show differences for both the chemical and physical PI categories (‘Medium’ risk).   To further illustrate the difference between the two approaches, the CCME WQI and RWQI Generation 3 performance (i.e. for each of the nine PIs) for QC SDWS-6 was calculated and plotted in Figure 4-12.   Figure 4-12. Comparison of CCME WQI and RWQI for QC SDWS-6 for each performance indicator in Generation 3  Figure 4-12 illustrates the differences between the performances of each of the nine PIs (i.e. Generation 3) between CCME WQI and RWQI. The CCME WQI, based almost entirely on either the compliance or non-compliance of the PIs, overestimates performance in areas such as   123  temperature and pH, while the RWQI approach incorporates degrees of compliance or non-compliance along with spatiotemporal factors to provide a more comprehensive picture of overall drinking water quality performance. Figure 4-10, Figure 4-11, and Figure 4-12 highlight how traditional water quality assessment metrics can be misleading when used in the benchmarking process and show how the integration of spatiotemporal variation and degrees of compliance/non-compliance can provide more comprehensive results for decision makers.   4.5 Sensitivity Analysis Monte Carlo simulations, consisting of 5,000 iterations of uniformly distributed random inputs, were performed using Matlab Simulink to measure sensitivity (i.e. the variability of the framework outputs with changes to the inputs). Uniform distribution (i.e. distribution where the probability of all outcomes is equal) was chosen over other distribution types due to the lack of available data (i.e. the frequency of sample concentrations for PIs) except for the minimum and maximum thresholds associated with the established UODs. The generated data was then analyzed with Oracle Crystal Ball (i.e., a Microsoft Excel-based add-on), which uses Spearman’s rank correlation coefficient to calculate correlation coefficients and percentage contribution of each input. In Figure 4-13 the percent contribution of the nine PIs (Generation 3) on RWQI (Generation 1) is illustrated. This sensitivity analysis shows that the physical and chemical PIs had a higher impact on RWQI than the microbiological PIs.    124   Figure 4-13. Percent contribution of the nine selected PIs (Generation 3) to RWQI performance (Generation 1)  In Figure 4-14, the percent contribution for each season (Generation 4) of the chemical PIs (Generation 3) was plotted. The results of this sensitivity analysis are consistent with the temporal risk factors presented in the demonstration, with summer representing the highest percent contribution and winter the lowest.      125   Figure 4-14. Percent contribution of seasonal variation (Generation 4) to chemical PI (FRC, HAA5, THM4) performance (Generation 3)  Finally, in Figure 4-15  the percent contribution for each of the three locations in the supply system (i.e. R1, R2, R3) (Generation 5) on the seasonal performance of FRC (Generation 4) was plotted. The lesser percentage contribution of FRC than the other water quality parameters can be attributed to more critical parameters such as THMs and HAAS which are primarily by-products of higher chlorine concentrations. The results shown in this figure are also generally consistent with the spatial risk factors presented in the demonstration, with R2 representing the highest risk and highest contribution and R1 representing the least risk and lowest contribution.    126   Figure 4-15. Percent contribution of distribution system location (Generation 5) to seasonal HAA5 performance (Generation 4)  4.6 Summary The proposed RWQI framework addresses several shortfalls of more traditional drinking water quality assessment approaches by incorporating severity (i.e. degrees of compliance and/or non-compliance for PIs) and spatiotemporal variability. For example, in conventional water quality assessment, a sample showing a THM4 concentration 0.099 mg/L is evaluated as compliant when compared to the Health Canada (2014) guideline of 0.1 mg/L. While this value does not exceed the regulatory guideline, there is still an inherent risk associated with 0.099 mg/L THM4 that is not incorporated into conventional drinking water quality performance assessment, but is included as part of the RWQI framework in the form of fuzzy sets. The use of fuzzy sets allows for the incorporation of degrees of compliance and non-compliance while addressing the uncertainty associated with SDWSs performance assessment and the imprecision associated with measuring performance against defined water quality guidelines and/or standards. In addition, by incorporating risk factors based on season and location in the drinking water supply system, the   127  RWQI framework provides data more representative of overall drinking water quality (with higher data resolution) to be used as part of the functional performance benchmarking process.  In the demonstration of 16 SDWSs in Newfoundland and Labrador and Quebec, the RWQI framework was compared to the more traditional CCME WQI approach. The differences in these methodologies can be seen in Figure 4-10, Figure 4-11, and Figure 4-12 for overall drinking water quality, PI category, and for the nine selected PIs for a specific SDWS. While the CCME WQI shows utilities such as QC SDWS-6 providing ‘Good’ drinking water quality, RWQI shows that this same utility as having ‘Medium’ risk and performance. These results were expected as the inclusion of more risk factors (i.e. spatial and temporal risk) and degrees of compliance/non-compliance provide a more representative performance assessment than the more binary CCME WQI approach.    128  Chapter 5: An Innovative Plan-Do-Check-Act (PDCA) Approach to Drinking Water Management and Governance in Canada A part of this chapter has been accepted for publication in Environmental Management, a Springer journal, as an article titled “Drinking water management and governance in Canada: An innovative plan-do-check-act (PDCA) framework for a safe drinking water supply” (Bereskie et al., 2017a).  With regards to Objective 4 as defined in Chapter 1, a Strengths-Weaknesses-Opportunities-Threats (SWOT) analysis of drinking water management and governance in Canada at the federal, provincial/territorial, and municipal levels was conducted in Chapter 5. Based on this analysis, a modified WSP (defined as the PDCA-WSP framework), was proposed for the general population (i.e. not including the indigineous population and not applying to DWSSs on First Nations reserves) based on WHO WSP recommendations and the principles of PDCA for CPI and a strategic action plan exploring implementation is presented. This proposed framework is presented as an alternative to the current drinking water management approach in Canada and is designed to fit within and incorporate the existing governance structure. Chapter 5 builds on the literature review from Chapter 2 (i.e. the state of practice of DWMSs throughout Canada) and incorporates concepts and methodologies from Chapter 3 (i.e. continuous performance improvement and performance benchmarking) and Chapter 4 (i.e. risk-based drinking water assessment/benchmarking).     129  5.1 Background In Canada, drinking water governance is highly decentralized (Bakker and Cook, 2011) with federal, provincial/territorial, and municipal governance responsibilities creating fragmentation, governance gaps, overlap, and significant challenges to managing drinking water (Hill et al., 2008; Bakker and Cook, 2011). Hrudey (2011) states, “…the current system is structured largely to download responsibility for safe drinking water to the lowest level of public authority, municipal government.” This approach is in direct contrast to recommendations by the WHO and practices in the European Union (EU) and the US. While many of the solutions needed to address drinking water problems in Canada are technological (especially in SDWSs), since threats to drinking water quality and quantity are strongly related to human activities, the solutions to many water problems can be based on changing human behavior and water governance (Simms and de Loë, 2010).   5.2 Approach and Methodology In Canada, there is a three-tiered top-to-bottom approach for the drinking water quality management consisting of federal, provincial/territorial, and municipal governance. In theory, each of these tiers is intertwined to provide safe drinking water quality throughout the country from source to tap (Figure 5-1). However, in practice, there is a considerable lack of coordination and information sharing between levels of governance, hindering effective drinking water management (Saunders and Wenig, 2007; Hill et al., 2008; Bakker and Cook, 2011). Based on the current governance structure, the federal government provides oversight through leadership, research and development, and recommendations for safe drinking water practices, while the provincial and territorial governments dictate water quality standards, manage monitoring and   130  enforcement, and provide provincial/territorial DWMSs and approaches (CCME, 2002; CCME, 2004). Finally, the municipal (i.e. local) level is responsible for administration, performance monitoring, management, source water protection, and maintenance of operations.    Figure 5-1. Current framework for drinking water quality governance in Canada (CCME, 2004; Kayser et al., 2015)  5.2.1 Federal Governance Drinking water governance (and most environmental governance) in Canada is decentralized, with provincial and territorial governments given considerable legislative powder while the Federal Government is responsible for several specific matters (e.g. waters flowing across provincial/territorial boundaries and international boundaries) and filling a general oversight role (Saunders and Wenig, 2007; Simms and de Loë, 2010). Federally, multiple institutions (e.g. Environment and Climate Change Canada, the CCME, Health Canada, and the Federal-Provincial-Territorial Committee on Drinking Water (CDW)) work together on the federal management of drinking water (CCME, 2004). Environment and Climate Change Canada is   131  responsible for administering legislation on water-related activities (e.g. the Canada Water Act, the International River Improvement Act, the Department of the Environment Act). Other relevant federal legislation for drinking water management includes the GCDWQ, the Canadian Labour Code, the Food and Drugs Act, the National Defence Act, and the Corrections and Conditional Release Act. A description of the applicable legislation can be found in Table 5-1.  Table 5-1. Description of applicable federal legislation related to drinking water (Health Canada, 2013) Federal legislation Description Guidelines for Canadian Drinking Water Quality (GCDWQ) Basic parameters are chosen to provide clean, safe drinking water. They are designed to be used in every jurisdiction in Canada as a basis for establishing their own requirements for drinking water quality with the goal of ensuring national consistency. Canadian Environmental Protection Act, 1999 While this act is most well known as the cornerstone of Canada’s environmental legislation, in terms of drinking water, this legislation (along with the Fisheries Act, Antarctic Environmental Protection Act, and Arctic Water Pollution Prevention Act) provides a framework for source water protection from hazardous contaminants. Canada Water Act Specifies management of the water resources in Canada. This includes research, planning, and implementation of programs related to the conservation, development, and utilization of water resources. Canada Labour Code The federal government’s legal obligation to its employees to provide potable waters to employees in accordance with prescribed standards. Food and Drugs Act Bottled water (including all pre-packaged water and ice) is considered as a food under Canadian Law and must be free of poisonous or harmful substances. The GCDWQ provides the basis for establishing levels of safe substances as there are no specified limits in this regulation. National Defence Act Act giving the Chief of the Defense Staff powers of command, responsibilities, and discretion regarding the health of members of the Canadian Forces. Some of the responsibilities are applicable to drinking water. Corrections and Conditional Release Act Regulations to provide safe drinking water for inmates in correctional institutions.  The CCME provides guidance and recommendations for water utilities to provide safe drinking water through the MBA (CCME, 2004). The MBA represents a preventative drinking management strategy designed to incorporate the implementation of multiple administrative, behavioral, and physical barriers to ensure safe drinking water from source to tap (CCME, 2004). In the event one barrier fails, back-up systems and processes are in place to protect the safety of drinking water (GNWT, 2005). While drinking water experts believe that an MBA is necessary to prevent contamination in a DWSS, it represents a broad concept and set of guidelines rather than a specified approach or self-contained system for ensuring safe drinking water through   132  quality management (Boyd, 2006). More details on the CCME MBA can be found in Section 2.3.2.  Federally, Canada and Australia are the only Organization for Economic Cooperation and Development (OECD) countries without legally enforceable federal drinking water standards despite WHO recommendations (Bakker, 2007; Bakker and Cook, 2011; Dunn et al., 2014b). Instead, Health Canada (2014) publishes voluntary health-based guidelines, the GCDWQ. The GCDWQ includes contaminants that could lead to adverse health effects in humans, are frequently detected or could be expected to be found in a large number of DWSS throughout Canada, and the contaminant is detected, or could be expected to be detected, in drinking water at a level that is of possible human health significance (Health Canada, 2014). Boyd (2006) noted, “The GCDWQ were originally called standards, but the name was deliberately changed in the 1970s to make it clear that guidelines do not have a legislative basis and are not legally enforceable as national standards.” While the GCDWQ is intended to be used in every jurisdiction in Canada as a basis for establishing their own requirements for drinking water quality with the goal of ensuring national consistency, as of 2011, only one of the 13 provinces/territories, the Northwest Territories, had adopted all 94 GCDWQ (Dunn et al., 2014b).  Dunn et al. (2014b) found that only 16 of the 94 GCDWQ were applied across all 13 provinces and territories. This approach is in direct contrast to the EU and the US, where enforceable national standards are applied, with the ability for states/provinces to enact stricter, but never more lenient, standards (Boyd, 2006). It is also important to note that the GCDWQ have also   133  been considered outdated (i.e. with a backlog of outdated guidelines due to budget reductions) and more lenient (i.e. weaker maximum allowable concentration (MAC) than other comparable jurisdictions) than standards in the EU and the US (Boyd, 2006; Christensen et al., 2010).   Finally, the Canadian Federal Government has responsibility on federal lands (e.g. national parks), federal facilities (e.g. office buildings, laboratories, military bases, etc.), on board common carriers (e.g. ships, airplanes), on boundary and transboundary water issues, in First Nations communities (i.e. with a duty to ensure safe drinking water for First Nation populations on federal land), and provides scientific and technical expertise to the provinces and territories through the CDW (Morris et al., 2007; Zubrycki et al., 2011; Health Canada, 2013; Simms and de Loë, 2010).  5.2.2 Provincial/Territorial Governance During the past century, the role of provincial/territorial governments have matured into their constitutional responsibilities and as de Loë and Kreutzwiser (2007) describe “…are clearly the primary water managers in Canada today” and have the most direct responsibility in regards to drinking water management (de Loë and Kreutzwiser, 2007). While Health Canada (2014) guidelines and the MBA can be adopted or modified to meet the needs of specific provinces/territories, ultimately the decision rests with the provincial/territorial governments. This has resulted in strong and comprehensive DWMSs in provinces such as Ontario and Quebec, however, less proactive provinces and territories have been slower to adapt. Because of this, Hrudey (2011) declared “…much of Canada remains out of step with the international leaders in adopting management systems for assuring safe drinking water.”   134   Although the MBA provides a broad framework for drinking water management, provinces and territories are ultimately responsible for their own interpretation which results in varying policies, regulatory requirements, and management practices (Dunn et al., 2015). This variation is not only between provinces and territories, but also within provinces (i.e. within the same watershed and across same water providers) (Dunn et al., 2014b). For example, when it comes to microbial risk management, Ontario has introduced a strong framework based on the MBA (including source water protection, a strong legislative framework, and stringent standards), while British Columbia hasn’t formally embraced the MBA and encourages a more dated and voluntary approach to microbial risk management (Dunn et al., 2014a). Examples of the differences that can be found between provinces and territories can be found below in Table 5-2.  Table 5-2 highlights the differences in drinking water legislation/policy, source protection legislation/policy, and quality management approaches. It illustrates the number of different ministries and agencies responsible for aspects of drinking water governance in Canada and also shows additional differences between provinces for water quality monitoring, required treatments, and operator certification. A report by Christensen (2006), found that less than half of all provinces and territories require advanced treatment of surface water for public DWSSs (i.e. treatment beyond disinfection of raw surface source water including filtration for turbidity and parasite removal), which is mandatory in the EU and the US (Christensen et al., 2010; Bakker and Cook, 2011).     135  While some provincial/territorial differences are expected given the provincial/territorial autonomy provided by federal legislation, they can create problems in ensuring consistent drinking water quality nationally and in keeping Canada in sync with leaders in drinking water management around the world. Previous studies have explored the differences between provincial/territorial drinking water management including; Dunn and Bakker (2009), which analyzed current approaches to measuring and assessing water security in Canada based on freshwater-related indicators, Christensen et al., (2010) which compared provincial water quality standards and legal requirements, and Dunn et al., (2015) which reviewed microbial management approaches in British Columbia and Ontario. In summary, these studies documented the extreme variability in approaches to water security, drinking water quality standards/regulations, and approaches to drinking water management across Canada.   136  Table 5-2. Applicable drinking water and source water protection legislation/policy, associated quality management frameworks, and other requirements (Hill et al., 2008; Dunn et al., 2014b) Province/territory and responsible ministries Drinking water legislation/policy Source protection legislation/policy Quality management framework Legally binding drinking water standards Number of parameters used Required treatment Operator certification requirements Alberta (AB)  Alberta Environment  Standards and Guidelines for Municipal Waterworks, Wastewater and Storm Drainage Systems (1997)  Public Utilities Board Act (2000)  Water Act (2000)  Potable Water Regulation (2003)  Water for Life (Government of Alberta, 2003; Government of Alberta, 2008; Government of Alberta, 2009)   Alberta Drinking Water Safety Plan (Government of Alberta, 2012) Yes 72 Disinfection and filtration (for surface water and GUDI supplies) Yes British Columbia (BC)  Ministry of Environment  Ministry of Health Services  Water Act (1996) o Water Regulation (1988) o Ground Water Protection Regulation (2004)  Water Protection Act (1996)  Water Utility Act (1996)  Drinking Water Protection Act (2001)  Drinking Water Protection Regulation (2003)  Water Sustainability Act (2016)  Drinking Water Protection Act (2001)  Environmental Management Act (2003)  Water Sustainability Act (2016)  British Columbia Comprehensive Drinking Water Source-To-Tap Assessment (British Columbia Ministry of Health Living and Sport, 2010) No 94 Disinfection Yes Manitoba (MB)  Manitoba Conservation  Manitoba Water Stewardship  Water Rights Act (1987, 2005)  Ground Water and Water Wells Act (2001, 2003)  Drinking Water Safety Act (2002)  Water and Wastewater Facility Operators Regulation (2003)  Water Supply Commissions Act (2005)  Environment Act (1987)  Water Protection Act (2005) o Nutrient Management Act (2008)   Livestock Manure and Mortalities Management Regulation (2008) Manitoba Drinking Water Plan (Manitoba Water Stewardship, 2003) Yes 94 Disinfection Yes   137  Province/territory and responsible ministries Drinking water legislation/policy Source protection legislation/policy Quality management framework Legally binding drinking water standards Number of parameters used Required treatment Operator certification requirements New Brunswick (NB)  New Brunswick Environment  New Brunswick Natural Resources  Municipalities Act (1973, 1981, 1995)  Public Utilities Act (1973)  Potable Water Regulation (1983)  Health Act (1988, 2005)  Water Act (1989, 1990, 1994, 2000, 2001, 2002)  Clean Environment Act (1982)  Clean Water Act (2000)  Wellfield Protect Area Designation Order (2000)  Watershed Protected Area Designation Order (2001) NA No 94 NA Yes Newfoundland and Labrador (NL)  Department of Environment and Conservation  Public Health Act (1996)  Municipalities Act (1999)  Water Resources Act (2002, 2003, 2004, 2005)   Water Resources Act (2002)  Environmental Protection Act (2002, 2005) Newfoundland and Labrador Multi-barrier Strategic Action Plan (Government of Newfoundland and Labrador, 2014) No 94 Disinfection No Nova Scotia (NS)  Nova Scotia Environment and Labour  Nova Scotia Natural Resources  Municipal Government Act (2998, 2001, 2002, 2004)  Water and Wastewater Facilities and Public Drinking Water Supplies Regulations (2005)  Water Resources Protection Act (2000)  Drinking Water Strategy (Government of Nova Scotia, 2002) A Drinking Water Strategy for Nova Scotia (Government of Nova Scotia, 2002) Yes (microbial, physical, and chemical)  Aesthetic parameters are not enforceable. 96 Disinfection and filtration Yes Ontario (ON)  Ministry of the Environment  Ministry of Natural Resources   Municipal Water and Sewage Transfer Act (1997)  Safe Drinking Water Act (2002)  Sustainable Water and Sewage Systems Act (2002)  Ontario Clean Water Act (2006)  Lakes and Rivers Improvement Act (1990)  Ontario Water Resources Act (2001)  Nutrient Management Act (2002)  Clean Water Act (2006) Ontario Drinking Water Quality Management Standard (Ontario Ministry of the Environment, 2007) Yes 106 Disinfection and filtration Yes Prince Edward Island (PE)  Environment, Energy, and Forestry  Water and Sewerage Act (1988, 2003)  Environmental Protection Act (1988, 2005)  Water Wells Act (1988, 2004)  Water and Wastewater Facility Operating Regulations (2004)  Environmental Protection Act (1988, 2005)  10 Points to Purity (Government of Prince Edward Island, 2001)  Prince Edward Island Watershed Strategy (Government of Prince Edward Island, 2015) 10 Points to Purity (Government of Prince Edward Island, 2001) No 94 NA Yes   138  Province/territory and responsible ministries Drinking water legislation/policy Source protection legislation/policy Quality management framework Legally binding drinking water standards Number of parameters used Required treatment Operator certification requirements Quebec (QC)  Ministère du Développement durable, de l'Environnement et des Parcs   Public Health Act (2001)  Environmental Quality Act (2005) o Regulation respecting the quality of drinking water (2001)  Quebec Water Policy (Quebec Ministry of Environment, 2002)  Strategy of protection and conservation of water sources intended for human consumption (Government of Quebec, 2012)  Regulation for water intake and sources protection (Government of Quebec, 2014) Quebec Water Policy/Integrated Water Resources Management (Quebec Ministry of the Environment, 2002) Yes 83 Disinfection and filtration (above certain turbidity) Yes Saskatchewan (SK)  Saskatchewan Environment  Saskatchewan Water Corporation  Saskatchewan Watershed Authority  Rural Municipalities Act (1989)  Public Health Act (1994)  The Water Regulations (2002)  Health Hazard Regulations (2002)    Conservation and Development Act (1978)  Water Management Framework (Government of Saskatchewan, 1999)  Environmental Management and Protection Act (2002)  Safe Drinking Water Strategy (Government of Saskatchewan, 2003)  Saskatchewan Watershed Authority Act (2005) Saskatchewan Safe Drinking Water Strategy (Government of Saskatchewan, 2003) Yes 65 Disinfection (for groundwater) and filtration (surface, mixed, or GUDI supplies) Yes Northwest Territories (NT)  NT Environment and Natural Resources  NT Public Works and Services, Water, and Sanitation  Public Utilities Act (1988, 1993, 1995, 1998, 1999, 2004)  Public Health Act (1990, 2004) o Public Water Supply Regulations (1990, 2004, 2009)  Arctic Water Pollution Prevention Act (1970)  Environmental Protection Act (1988, 1991, 1998)  Environmental Rights Act (1988, 1999, 2000) Northwest Territories Safe Drinking Water Framework and Strategy (Government of the Northwest Territories, 2005) Yes 94 Disinfection No Nunavut (NU)  Department of Environment  Public Utilities Act (1999)  Nunavut Waters and Nunavut Surface Rights Tribunal Act (2002) NA No 94 Disinfection Yes Yukon (YT)  Environment Yukon  Yukon Health and Social Services  Public Health and Safety Act (2002, 2007)  Public Utilities Act (2002)  Water Regulation, Bulk Delivery of Drinking Water Regulation (2003)  Yukon Waters Act and Regulation (2003)  Environment Act (1991, 2002)  Yukon Waters Act and Regulation (2003) Yukon Water Strategy and Action Plan (Government of Yukon , 2014) Yes 28 Disinfection Yes    139  5.2.3 Municipal Governance Much like in the US, municipal governance in Canada refers to management at a local level (that can include one or more individual DWSS(s)). Municipal governments only hold powers and responsibilities provided by their provincial or territorial governments (Loë and Kreutzwiser, 2007; Simms and de Loë, 2010) After the Walkerton, Ontario outbreak, Canadian provinces started to hold municipal authorities more directly accountable for providing safe drinking water and today, drinking water management is more decentralized than other utility sectors (Bakker and Cook, 2011). However, Hrudey (2011) notes that “…the regulatory system remains generally more reactive than preventative” and “focuses more on monitoring treated water quality rather than primarily on training and process monitoring aimed at ensuring operators know their own system better.”  Generally, municipal responsibilities include all technical and administrative management from source to tap of an individual municipal water utility. This includes maintenance, monitoring, and the implementation of municipal DWMSs tailored to an individual utility that can range from simple (i.e. the addition of a disinfecting agent to raw surface source water) to complex (i.e. multiple types of physical and chemical drinking water treatment technologies) [see Section 2.1.2]. This range depends mainly on the type and quality of source water (surface water or groundwater) and on the provincial/territories regulatory requirements for removing microbiological, chemical, and physical contaminants (CCME, 2004).    140  5.2.4 Challenges Based on the review of literature of challenges associated with the current water governance structure in managing drinking water in Canada several themes arise including:  fragmentation across political boundaries (Hill et al., 2008; Dunn and Bakker, 2009; Simms and de Loë, 2010; Bakker and Cook, 2011),  governance gaps, overlapping responsibilities, duplication of efforts (Dunn and Bakker, 2009; Simms and de Loë, 2010; Bakker and Cook, 2011)  discrepancies among the mandates and administration leading to confusion surrounding leadership responsibilities and inconsistent resource allocation (Simms and de Loë, 2010),  lack of accountability and coordination between tiers of governance (Bakker and Cook, 2011),  inadequate monitoring and enforcement (Dunn and Bakker, 2009; Bakker and Cook, 2011)  resistance to change and barriers to learning (Simms and de Loë, 2010),  failure to integrate activities at spatial and temporal scales (Dunn and Bakker, 2009; Simms and de Loë, 2010),  difficulties in evaluating performance (Simms and de Loë, 2010), and  tension between harmonization (i.e. the selective standardization of laws, rules, and norms) and subsidiarity (i.e. the delegation of decision-making and policy implementation to the lowest-appropriate scale) (Bakker and Cook, 2011).    141  Based on these themes, a modified SWOT diagram was developed (Figure 5-2). This technique is designed to illustrate elements necessary as part of the decision-making process internally (i.e. strengths and weaknesses) and externally (i.e. opportunities and threats) (Dyson, 2004). While SWOT analyses are typically associated with business development and industry, for the context of this research a SWOT analysis was conducted to summarize the strengths and weaknesses(i.e. areas where the current MBA excels) and weaknesses (i.e. areas where the current MBA is lacking) of the current Canadian approach to drinking water governance and management while highlighting the opportunities for improvement and potential external threats(i.e. external areas where the MBA can be supplemented or improved on) and potential threats (i.e. external areas where there is risk associated with the current MBA or in implementing a new system). This analysis was conducted at the federal, provincial/territorial, and municipal levels using a review of applicable literature to highlight the strengths, weaknesses, opportunities, and threats in each area, respectively (Jackson et al., 2003; Dyson, 2004).     142   Figure 5-2. SWOT analysis of the current Canadian federal, provincial/territorial, and municipal governance and management system  Bakker and Cook (2011) note that, “Canada’s constitutional reality, in which federal and provincial governments share authority over water, is unlikely to change” and the decentralized governance structure and its impact on drinking water management has been examined extensively in literature (Saunders et al., 2007; Simms et al., 2010; Bakker and Cook, 2011; von der Porten et al., 2013; Dunn et al., 2015). For the purpose of this paper, the proposed PDCA-WSP framework was designed to fit within the current Canadian governance structure, while integrating the principles of the WHO WSP and PDCA for CPI. The proposed PDCA-WSP framework can be found in Figure 5-3.    143  5.3 Proposed PDCA-WSP Framework Given the unique relationship and differences between federal, provincial/territorial, and municipal drinking water governance and management and the unlikely event of major governance changes, a modified drinking water management approach, the PDCA-WSP framework, is proposed (Figure 5-3). This approach is based on the WHO (2011) WSP recommendations [Section 2.3.4] and the principles of PDCA for CPI [Section 3.1.2].  The black stars in Figure 5-3 represent common elements that can be found within the current CCME MBA. As mentioned earlier [Section 2.2.2.2], the MBA represents a broad concept and provides guidelines rather than a specified approach for ensuring safe drinking water and while some similarities exist, the PDCA-WSP was developed as a self-contained and malleable management system with each element representing a critical component (with some potential for jurisdictional differences/interpretations) (Boyd, 2006).      144   Figure 5-3. Proposed PDCA-WSP framework for drinking water management in Canada  The top box in Figure 5-3 describes the responsibilities of the Canadian Federal Government. Based on the current governance structure and current principles of the MBA, the Federal Government is responsible for areas of federal jurisdiction, developing and maintaining   145  provincial/territorial cooperation, federal funding, development and upkeep of the GCDWQ for establishing health-based water quality guidelines, and research and development. The arrows out of the top box represent the flow of resources and information between the Federal Government, the provincial/territorial governments, and the PDCA-WSP.  The bottom box of Figure 5-3 describes the oversight responsibilities of the provincial/territorial governments based on the current governance structure. The provincial and territorial governments are responsible for allocation of funding, provincial/territorial supporting legislation, the development and upkeep of water quality standards (that should ideally align with the GCDWQ), ensuring municipal cooperation, and the development and monitoring of administrative and training requirements. The arrows out of the top box represent the flow of resources and information between the provincial/territorial governments, the Federal Government, and the PDCA-WSP.  The large middle box at the center of Figure 5-3 represents the bulk of the proposed PDCA-WSP framework. The majority of the PDCA-WSP framework is a municipal responsibility, with the provincial/territorial government playing an enforcement, benchmarking, management review, and continuous improvement role and the Federal Government providing oversight and responsibilities for management review and continuous improvement beyond the MBA. The proposed PDCA-WSP framework is based on the elements recommended by the WHO (2011) and principles of the PDCA cycle along with performance benchmarking and continuous performance improvement concepts proposed in Chapter 3. The 10 steps of the proposed PDCA-WSP can be found below:   146  5.3.1 Plan  Step 1: Assemble the team to prepare the WSP Assembling a diverse quality management team is the first step to most quality management systems as it establishes multiple viewpoints, cooperation, and information sharing. Ideally, the WSP team should be comprehensive, comprised of all stakeholders having activities in the watershed and potential project funders (i.e. academia, government, businesses, and community members). This team is tasked with working together to ensure the subsequent steps are properly developed, managed, and executed. The Ontario Ministry of Environment (2007) has developed a document describing the process for setting up a team for quality management system implementation that can be used as a reference for Step 1.  Step 2: Document and describe the system A full analysis (including the development of a flow diagram) of a drinking water supply system from catchment to consumer is necessary to gauge the current performance of a drinking water system. This assessment should include a full water supply system analysis, assessment of applicable water quality data, public health incidents, plant records, watershed activities, hazard identification and risk characterization, and control measure identification. In cases of SDWSs, this step is often just a collection of best available information and historic data about the DWSS. The British Columbia CS2TA Guideline and the Australia NHMRC (2011) provide easily interpreted resources that can be used for guidance in documenting and describing a DWSS (BCMHLS, 2010).  Step 3: Document and describe compliance and performance monitoring   147  Documenting and describing of compliance and performance monitoring is necessary to fully understand the operational checks and balances in place within a drinking water system from catchment to consumer. This step includes documenting and describing of implemented control measures, establishing procedures to verify the management system is working effectively and meeting targets (i.e. regulatory standards/guidelines) and also includes the establishment of documentation and communication procedures. The WHO (2011) and Ontario Ministry of Environment (2007) both provide information referencing specific actions associated with documenting operational and performance monitoring.  5.3.2 Do Step 4: Development of supporting programs Supporting programs, such as establishing standardized or recommended hygiene practices, operator training, source protection, and standard operating procedures, are all necessary for ensuring high quality drinking water from source to tap and are the responsibility of both the province and municipality. References developed by the Ontario Ministry of the Environment (2007), Quebec Ministry of Environment (2002), and CCME (2004) provide reference documents for operator training, source protection, and watershed/aquifer management, respectively. Other examples of supporting programs, such as the Haider et al. (2016) risk-based framework for improving customer satisfaction, a decision support tool for utility operators and planners to improve customer satisfaction through quantifying and prioritizing customer complaint risk, can also be integrated as more advanced supporting programs designed to improve DWSSs performance.    148  Step 5: Maintenance and monitoring Performance monitoring includes the sampling of water quality at the source, along the treatment process, after treatment, and within the distribution network. This allows operators to modify treatment if water quality fluctuates to ensure regulatory compliance and safe drinking water. Performance monitoring can also incorporate customer satisfaction, operational reliability, and system efficiency. The New Zealand Ministry of Health (2015) and the National Research Council (2006) provide valuable resources for the monitoring of drinking water at the source, during treatment, and in the distribution system. For even more detail in performance monitoring, risk based assessment techniques, such as proposed in Chapter 4 can also be explored.  5.3.3 Check Step 6: Enforcement Currently, provincial/territorial governments are responsible for enforcement of water quality standards. Based on the proposed PDCA-WSP framework, provincial/territorial responsibilities can be expanded to include enforcement of supporting programs such as administrative requirements, operator training requirements, and source water protection strategies, as well as supporting funding for specific initiatives (emerging issues, infrastructure renewal, etc.). Further information about the specifics aspects associated with planning and implementation of monitoring enforcement can be found in the WHO (2011) Guidelines for Drinking Water Quality.  Step 7: Auditing and performance benchmarking   149  The long-term evaluation and auditing of drinking water quality results is required to determine efficiency of preventative strategies and corrective actions. While the WHO WSP (2011) does not include performance benchmarking as a component, Corton (2003), Corton and Berg (2009), and Marques et al. (2014), and Haider et al. (2015)  have previously explored and documented the benefits of benchmarking water utilities. In the proposed PDCA-WSP framework, internal benchmarking is a municipal responsibility while external benchmarking is a provincial/territorial responsibility that can be expanded to include independent evaluation. Vieira (2011), in a case study exploring WSP implementation in Portugal, identified independent auditing as a key issue in drinking water management at all levels of governance and the auditing process should ideally follow the ISO 9001:2015 and ISO 19011:2011 procedures (Tricker, 2016).  5.3.4 Act Step 8: Corrective actions Based on information from Step 6: Enforcement and Step 7: Auditing and performance benchmarking, corrective actions can be taken to eliminate causes of regulatory non-conformances (i.e. exceedances of water quality standards) or non-conformance of mandated supporting programs.   Step 9: Management review Management reviews are necessary to ensure proper implementation and operation of any quality management system. These reviews refer to the performance evaluation of the implemented management systems and also include the identification of opportunities for system improvement   150  (ISO, 2008). In the proposed PDCA-WSP, management review is a necessary component at the federal, provincial/territorial, and municipal levels and specific information on implementing management review can be found as part of ISO 9001:2015 (ISO, 2015).  Step 10: Continuous performance improvement The final step of the proposed PDCA-WSP is the concept of continuous performance improvement. While this step can refer to any upgrades or improvements to increase performance, its inclusion also ensures the goal and foundation of PDCA as an improvement cycle is not overlooked or ignored in the process.  After all 10 steps of the proposed PDCA-WSP, the cycle returns to Step 1 and includes the implementation of all corrective actions, improvements, or upgrades. Municipal use of this approach can occur as often or as seldom as seen fit by stakeholders, but ideally should be performed seasonally (to account for spatiotemporal variation) or annually (in SDWSs where resources are more scarce). At the federal and provincial/territorial levels, reviews of responsibilities and supporting activities should be conducted annually.  5.3.5 Strategic Action Plan for Implementation of PDCA-WSP Based on the Vieira (2011) strategic approach for WSP implementation scaling-up in Portugal, a strategic action plan for the implementation of the PDCA-WSP approach to drinking water management was developed (Figure 5-4) This plan categorizes the implementation of the PDCA-WSP into three distinct phases, with the focus of Phase 1 on developing the federal foundation for the PDCA-WSP, the focus of Phase 2 on implementation of the PDCA-WSP framework and   151  supporting programs at the provincial/territorial level, and with Phase 3 focused on widespread municipal implementation of the PDCA-WSP framework. In between phases, the process can be improved based on feedback from the institutional and practical improvement actions and the development of the applicable supporting mechanisms.    Figure 5-4. Strategic action plan for prioritization of specific improvement actions for the implementation of the proposed PDCA-WSP framework 5.3.5.1 Phase 1 The first phase consists of developing the foundations for implementing the proposed PDCA-WSP. While provinces/territories employ significant autonomy, especially in regards to drinking water management, building the framework at the federal level provides a template and example for the provinces and territories. This can be done by developing and by either supplementing the existing CCME MBA with PDCA-WSP elements or by overhauling the entire national drinking   152  water management strategy to a system more reminiscent of the WHO WSP recommendations. Pilot projects for individual municipalities can also be established to gauge performance of the PDCA-WSP framework and identify areas of improvement. In Phase 1, pilot projects should be structured in each province/territory to account for governance differences and consist of DWSSs of varied documented performance. To establish pilot projects, supporting mechanisms (i.e. the development of tools and organization of training workshops and discussion forums) for the PDCA-WSP must be identified and developed at the federal and provincial/territorial level to quantify and analyze performance. At this stage, a provincial/territorial strategic action plan tailored specifically for each province/territory can also be developed to guide Phase 2 provincial/territorial implementation.  5.3.5.2 Phase 2 The second phase represents the provincial/territorial and legislative actions necessary for implementation of the PDCA-WSP framework. Institutionally, this could include implementation of supporting legislation at the national and provincial/territorial level and the supplementation of the current regulatory initiatives that vary by province/territory. This also includes the implementation of the provincial/territorial strategic action plans, the establishment of more pilot projects, and further developing the protocol necessary for review/auditing (a major provincial/territorial responsibility in Figure 5-3). The supplementary mechanisms in Phase 2 include a review of pilot project performance from Phase 1, the development of a DWSS database for easier comparison/benchmarking, and specific provincial/territorial training workshops and forums.   153  5.3.5.3 Phase 3 Finally, after the federal and provincial/territorial foundation for the PDCA-WSP framework is developed, the widespread municipal implementation (and implementation of all auditing and review/compliance procedures) of the proposed framework can occur in Phase 3. Institutionally, this includes any needed legislative changes identified in Phases 1 and 2 along with any necessary improvements to the PDCA-WSP framework. Supporting mechanisms for Phase 3 include the assessment of performance from Phase 2 pilot projects, the organizing of training workshops and forums for municipal operators, and the implementation of the independent auditing process to ensure a properly functioning drinking water management system.  5.3.5.4 Small Drinking Water Systems Although the idea of a more comprehensive approach to drinking water management is especially daunting for the owners, operators, and customers of SDWSs with limited human and financial resources, the water quantity and quality issues in small and rural communities are well-established [Chapter 1 and Chapter 2]. While SDWSs cannot afford or justify large investments in new technologies like in larger systems, they can place emphasis on behavioral/management improvements and targeted incremental upgrades to improve their drinking water supply. In terms of implementing the proposed PDCA-WSP framework, the federal and provincial/territorial governments can help to address some of the human resources or financial gaps associated with SDWS implementation or a modified and streamlined PDCA-WSP can be developed to ensure more accessible and widespread implementation.    154  5.4 Summary The current approach to drinking water management in Canada leaves room for improvement, especially in terms of clearly defining roles/responsibility, ensuring accountability, and implementing continuous performance improvement principles. Although there have been no major waterborne disease outbreaks in Canada since 2001, the consistent presence of BWAs, especially in SDWSs is concerning. While there are many challenges presented with the idea of a more comprehensive federal, provincial/territorial, and municipal drinking water management framework within Canada’s current governance structure, a more extensive, preventative approach for ensuring safe drinking water is a necessity. This is especially true given the prospect of water quantity and quality problems into the future that will only be accelerated in the face of the discovery of emerging contaminants, climate change, and population growth (Richardson, 2003; Schindler, 2000; Bakker and Cook, 2011).  The proposed PDCA-WSP framework provides a comprehensive alternative to the current drinking water management approach and is designed to fit within and incorporate the existing governance structure. The proposed framework provides an organized and easily understood management approach that can be implemented in small, medium, and large DWSSs and clearly defines the federal, provincial/territorial, and municipal roles and responsibility. Like most management systems, the PDCA-WSP is designed to be malleable and easily adapted for specific needs identified throughout the project. The development of this proposed framework can enable legislators and policy makers at all levels of governance to better apply and understand the process for effective risk assessment and risk management in DWSSs. It also provides a pathway to helping preserve the health of the current population, while providing a   155  means for protecting drinking water into the future and ensuring Canada’s spot as a world leader in regards to drinking water management.   156  Chapter 6: Conclusions and Recommendations Drinking water suppliers face challenges associated with changing populations and economies, aging infrastructure, and evolving consumer demands (Danilenko et al., 2011). In SDWSs, these challenges are amplified by the strain created from financial shortfalls and limited human resources (Moffatt and Struck, 2011). SDWSs are prone to higher rates of drinking water quality failure (Scheili et al., 2014), are more vulnerable to spatiotemporal variability of water quality (Dyck et al., 2014; Scheili et al., 2014), and may be more vulnerable to waterborne disease outbreaks (Moffatt and Struck, 2011) than larger systems. Despite these difficulties, SDWSs are overlooked in traditional academic and industry study, which often place a focus on larger, more complex DWSS and exploring new treatment technologies (Scheili et al., 2014). While developing improved technologies for drinking water treatment is of utmost importance in progressing the field, the financial challenges associated with SDWSs make widespread implementation of new technologies a difficult proposition and push focus towards improvement from a quality management standpoint.   6.1 Conclusions The main objectives identified for this research were to review and compare DWMSs, incentivize CPI, improve data resolution and drinking water quality assessment practices for decision-making and benchmarking, and propose an improved drinking water management approach for Canada. This was accomplished by critically reviewing the current state of practice of DWMSs in Canada and throughout the world by identifying and comparing individual included management elements, exploring the concept of CPI and ways to incentivize implementation through performance benchmarking, improving on current drinking water   157  quality assessment practices by implementing risk (i.e. degrees of compliance/non-compliance) and quantifying spatiotemporal variability, and finally by proposing an improved DWMSs that fits within the bounds of Canada’s current governance system. It is also important to highlight the fact that the concepts presented as part of Chapter 3 and Chapter 4 are not designed as alternatives to health-based regulatory guidelines and standards, but represent the development of performance metrics to be used for comparison (i.e. performance benchmarking) and ease of understanding for the general public. A short summary of Chapter 2, Chapter 3, Chapter 4, and Chapter 5 can be found below.  In Chapter 2, the current state of drinking water and drinking water management in Canada was critically reviewed. A detailed description of challenges and difficulties facing SDWSs were explored and the differences between SDWSs and MDWSs/LDWSs were highlighted. General QMSs (i.e. HACCP and ISO 9001), federal DWMSs (i.e. the Australian Framework for Managing Drinking Water Quality, CCME WQI, NZMOH WSP, and WHO WSP), and provincial/territorial DWMSs were reviewed and compared across 41 management elements divided into six categories (i.e. administrative, assessment, mitigation, monitoring and verification, improvement, documentation and review elements). The resulting comparison highlighted the significant variation in included management elements between DWMSs, especially between the Canadian provincial and territorial approaches. The frequency of management elements was then reviewed to determine the most common management elements in the reviewed provincial/territorial DWMSs and this data was subsequently used to propose recommended foundational management elements for use in developing new DWMSs or improving existing DWMSs    158   In Chapter 3, a CPI improvement framework incorporating the CCME WQI and functional performance benchmarking was proposed as a means for improving drinking water quality and performance in SDWSs. First, a literature review was conducted for CPI, PDCA, WQIs (including the CCME WQI, BC WQI, Centre St. Laurent WQI, and Quebec Index), and for general benchmarking processes. This information was then used to develop the proposed CPI framework for SDWS manager and operators to gauge current performance and track performance improvement into the future. The CPI framework was demonstrated using water quality data from seven SDWSs in Newfoundland and Labrador using the CCME WQI to aggregate the selected PIs (i.e. FRC, HAA5, THM4, TC, turbidity). A generalized improvement matrix and plan was developed for use in the demonstration and applied as a treatment train to project future water quality and performance improvement. Results were calculated for the overall WQI values and for an individual PI (i.e. THM4) to demonstrate the potential impact of the CPI framework on drinking water quality and SDWS performance.  In Chapter 4, a hierarchical risk-based water quality performance benchmarking framework integrating FRBM for SDWSs was proposed. First, a literature review was conducted for risk and risk management, fuzzy sets, fuzzy logic (including fuzzification and defuzzification), fuzzy rules, and FISs (including Mamdani-type FISs, and Sugeno-type FISs). Secondly, this information was used in developing the proposed RWQI framework, an alternative drinking water assessment methodology designed to incorporate degrees of compliance/non-compliance (through FRBM) and spatiotemporal variability (i.e. through highlighting seasonal and spatial risk). Thirdly, the RWQI framework was demonstrated using 16 SDWSs (i.e. eight in   159  Newfoundland and Labrador and eight in Quebec) and nine water quality PIs (i.e. E.coli, FRC, HAA5, HPC, pH, TC, temperature, THM4, turbidity) in MATLAB Simulink and compared to the CCME WQI. The demonstration showed that the RWQI framework provides an in-depth state of water quality and benchmarks SDWSs more rationally based on the frequency of occurrence and consequence of failure events. A Monte Carlo simulation was in this section using Oracle Crystal Ball to measure the sensitivity of the RWQI framework using uniformly distributed random values.  In Chapter 5, current drinking water management and governance practices in Canada were reviewed and a framework incorporating CPI, PDCA, and the WHO WSP designed to fit within Canada’s existing decentralized governance structure was proposed (PDCA-WSP). First, background information related to the Walkerton Report and Canadian drinking water management history and practices were discussed. Secondly, Canadian top-to-bottom drinking water governance structure (i.e. federal, provincial/territorial, and municipal governance) was reviewed and the applicable legislation, roles and, responsibilities at each level were identified and compared to EU, US, and WHO best management practices. Thirdly, a SWOT analysis was performed and the PDCA-WSP framework was proposed integrating the principles of PDCA and WHO WSP recommendations and designed to fit within the current Canadian governance structure. Finally, a strategic action plan was proposed and discussed for the potential implementation of the PDCA-WSP framework in Canada.    160  6.2 Originality and Contribution Sufficient safe public drinking water is not a guarantee into the future. This is especially true in small, rural, and First Nations communities that must rely on improving management strategies (i.e. analytics and the decision-making process) over more expensive and human resource intensive technology upgrades. The research presented in this thesis has developed innovative, original techniques and frameworks that have demonstrated potential for improving drinking water management, governance, and the associated decision-making process for DWSS managers, operators, and planners. The main contributions and originality of this research can be categorized into two distinct, but intertwined categories. 1. Contributions related to management, governance, and policy 2. Practical contributions at the municipal/regional level  The first, contributions related to management, governance, and policy describes the work in Chapter 2 and Chapter 5 and represents significant contributions that can be applied at the federal, provincial, and territorial levels within Canada and potentially adapted for use in other countries throughout the world.  Drinking water management systems represent the primary means for preventative management of DWSSs. While the effectiveness and impact of management practices can be difficult to quantify, by comparing the Canadian state of practice and included management elements against best management practices, context can be provided to gauge the comprehensiveness of provincial and territorial DWMSs and identify areas of improvement. The information presented in Chapter 2 can be used to influence   161  approaches to drinking water management in Canada at the provincial, territorial, and federal level and be used by other countries working to develop or improve DWMSs.  Building off of the DWMS review, the PDCA-WSP and subsequent strategic action plan proposed in Chapter 5 was designed as a specific approach to strengthen drinking water management in Canada within the current decentralized governance structure by integrating DWMS and QMS best management practices and CPI. The PDCA-WSP was developed to be applicable for DWSSs of all sizes and addresses many of the current issues (e.g. fragmentation, governance gaps, overlapping responsibilities, duplication of efforts, etc.) found in Canadian drinking water management.   The second, practical contributions at the municipal regional level, describes the work in Chapter 3 and Chapter 4 and represents significant contributions that can be applied for use in individual or regional groups of municipal DWSSs.  Drinking water utilities have little reason to focus on improvement beyond regulatory compliance. The CPI framework presented in Chapter 3 was proposed as a catalyst for continuous performance improvement of drinking water quality through introducing performance benchmarking. Both the CPI framework, performance benchmarking methodology, and adapted CCME WQI for drinking water quality assessment can be immediately introduced in DWSSs as tools for gauging current performance individually and against similar water utilities and provides a pathway to project water quality improvement into the future.  Current drinking water assessment and performance benchmarking practices fail to incorporate degrees of compliance/non-compliance of water quality and spatiotemporal   162  variability resulting in misleading and/or inaccurate information. The RWQI framework presented in Chapter 4 was proposed as a method to quantify these factors for use in measuring overall drinking water quality for performance benchmarking. This risk-based framework and the associated soft computing methodologies have the potential for implementation into drinking water assessment practices in DWSSs of all sizes and can provide better data resolution for use in decision making and resource allocation.  6.3 Limitations and Recommendations The main limitation associated with the research presented in this thesis involves the limited data availability from SDWSs and their subsequent use in the demonstration/case studies presented in Chapter 3 and Chapter 4. This research incorporated data from 16 SDWSs in Newfoundland and Labrador and Quebec, Canada. This represents a very small microcosm of SDWSs in Eastern Canada that may not be representative of SDWSs in other provinces, territories, states, or countries with differing regulations/standards and management/governance approaches. It is also important to note that the data used for the demonstration/case studies was collected from a one year, monthly sampling campaign that may not be representative of the norm. However, the frameworks and methodologies proposed in this thesis are overarching and designed to accommodate data that is dynamic in nature and provide applicable outputs/information throughout Canada.  Other specific limitations identified can be found below:  In Chapter 2, limitations of reviewing QMSs and DWMSs and identifying foundational DWMS elements were analyzed. While an inventory and comparison of QMS and   163  DWMS elements is important in determining the state of drinking water management practices, comparing and quantifying performance of DWMSs is a difficult proposition given their qualitative nature and the lack of specific performance assessment data or measurement initiatives associated with drinking water management.  In Chapter 3, there are less quantifiable factors that must be considered when measuring performance and projecting future water quality for the CPI framework. The performance benefits from operational trainings, experienced operators, system familiarity, and dedicated staff is still being explored and cannot easily be incorporated into a WQI-based approach. Also, while a WQI-based framework provides a useful metric for measuring water quality, other facts such as aesthetic water quality and customer satisfaction are difficult to measure and incorporate, but are important when assessing overall water utility performance and projecting improvements into the future. Other limitations to the proper implementation of the CPI framework are the selection of adequate PIs to represent water quality, availability of applicable data given human resource constraints, and participation from water utilities that may shy away from bringing attention to water quality problems.  In Chapter 4, potential limitations were identified with implementation of the RWQI framework. The established UOD can vary significantly depending on the user and their motivations. For example, one user may consider the risk of a THM4 value as ‘Very High’, while a second user considers the same value as ‘Medium’, which can significantly alter the results. The results produced can also vary based on differing approaches to establishing fuzzy rules, which can directly/indirectly result in improper   164  weighting of PIs and risks at each generation. Much like in Chapter 3, there are also limitations associated with the selection of adequate PIs to represent water quality.  In Chapter 5, one major potential limitations was identified with the PDCA-WSP framework. While the individual elements of this DWMS approach have been thoroughly researched and demonstrated throughout different parts of the world and for different applications, it is difficult to quantify or project the potential PDCA-WSP application, specifically for drinking water management elements and within Canada’s governance system. This can be attributed to the scale of this approach and the uncertainty associated with legislation, cost, public opinion, and the ever-evolving federal-provincial/territorial relationship in Canada.  Research into drinking water management of SDWSs is still in its infancy. Specific future research recommendations for the research presented in Chapter 2, Chapter 3, Chapter 4, and Chapter 5 can be found below.  In Chapter 2, future studies should include more QMSs and international DWMSs and potentially provide a performance metric to gauge effectiveness. While most Canadian provinces monitor performance from year-to-year, looking at performance and comparing management approaches is outside of the traditional scope of assessment. Impacts and effectiveness of DWMSs on small, rural, and First Nations communities could also provide utility for regulators, decision-makers, and consumers.  In Chapter 3, future studies should include more water quality PIs (i.e. using questionnaires to aggregate expert opinion) to implement into the proposed WQI-based framework. While water quality PIs may vary depending on sampling programs,   165  including more indicators will help provide a better picture of overall drinking water quality for a given water utility. Other factors such as seasonal adapted operations and anticipated land use shifts should also be explored to improve drinking water quality and to prioritize improvement actions more rationally as part of Step 4 and Step 5 of the CPI framework.  In Chapter 4, future studies should include exploration into more detailed spatial and temporal risk factors. Instead of using generalized spatial and temporal risk factors, there is potential to have differing risk factors based on specific PIs. Specific risk-based methodologies can also be incorporated to monitor full utility performance by including source water quality, customer satisfaction, operational reliability, and operator training/experience.   In Chapter 5, future studies should be focused on the implementation of the proposed PDCA-WSP framework or other modified WHO WSP approaches in utilities of all sizes throughout different parts of Canada. Information from these pilot applications could be analyzed to tailor the framework to the unique governance structure and spatiotemporal factors present throughout Canada. A survey of water utility operators and planners, especially in newer utilities, designed to highlight specific needs and concerns in developing and implementing DWMSs could also be of use in improving the overall approach to drinking water management throughout the country. A feasibility study and economic analysis of implementing the PDCA-WSP to determine potential expenditures and cost savings would also be a logical next step before implementation.   166  Bibliography Adebanjo, D., Abbas, A., & Mann, R. (2010). An investigation of the adoption and implementation of benchmarking. International Journal of Operations & Production Management, 30(11), 1140–1169. http://doi.org/10.1108/01443571011087369 Adriaenssens, V., De Baets, B., Goethals, P. L. M., & De Pauw, N. (2004). Fuzzy rule-based models for decision support in ecosystem management. Science of the Total Environment, 319(1–3), 1–12. http://doi.org/10.1016/S0048-9697(03)00433-9 Alberta Environment. (2009). Alberta Environment’s drinking water program: A “Source to Tap, Multi-Barrier” Approach. Edmonton. Retrieved from http://environment.gov.ab.ca/info/library/8156.pdf Ale, B. J. M. (2002). Risk assessment practices in The Netherlands. Safety Science, 40(1–4), 105–126. http://doi.org/10.1016/S0925-7535(01)00044-3 American Water Works Association (AWWA). (1999). Standard Methods for the Examination of Water and Wastewater. Aven, T., & Jinnem, J. E. (2007). Risk Management With Applications from the Offshore Petroleum Industry. Springer Science+Business Media, LLC. http://doi.org/10.1007/978-1-84628-653-7 Bakker, K. (2007). Eau Canada: The Future of Canada’s Water. Vancouver, BC: UBC Press. Bakker, K., & Cook, C. (2011). Water governance in Canada: Innovation and fragmentation. International Journal of Water Resources Development, 27(2), 275–289. http://doi.org/10.1080/07900627.2011.564969 Baril, P., Maranda, Y., & Baudrand, J. (2006). Integrated watershed management in Quebec (Canada): a participatory approach centred on local solidarity. Water Science and Technology, 53(10), 301–307. Bartram, J., Corrales, L., Davison, A., Deere, D., Drury, D., Gordon, B., … Stevens, M. (2009). Water Safety Plan Manual. Geneva. http://doi.org/10.1111/j.1752-1688.1970.tb00528.x Bartram, J., Fewtrell, L., & Stenström, T. (2001). Harmonised assessment of risk and risk management for water-related infectious disease : an overview. Water Quality: Guidelines, Standards and Health. Retrieved from http://www.who.int/water_sanitation_health/dwq/iwachap1.pdf Baum, R., Amjad, U., Luh, J., & Bartram, J. (2015). An examination of the potential added value of water safety plans to the United States national drinking water legislation. International   167  Journal of Hygiene and Environmental Health, 218(8), 677–685. doi:10.1016/j.ijheh.2014.12.004 Bereskie, T., Rodriguez, M. J., & Sadiq, R. (2017a). Drinking water management and governance in Canada, A conceptual plan-do-check-act (PDCA) water safety plan (WSP) framework for safe drinking water. Manuscript accepted for publication in Environmental Management. doi: 10.1007/s00267-017-0873-9 Bereskie, T., Delpla, I., Rodríguez, M. J., & Sadiq, R. (2017b). Drinking Water Management in Canadian Provinces and Territories: A Review and Comparison of Management Approaches for Ensuring a Safe Drinking Water Supply. Manuscript submitted for publication. Bereskie, T., Haider, H., Rodriguez, M. J., & Sadiq, R. (2017c). Framework for continuous performance improvement in small drinking water systems. Science of the Total Environment, 574, 1405–1414. http://doi.org/10.1016/j.scitotenv.2016.08.067 Bereskie, T., Haider, H., Rodriguez, M. J., & Sadiq, R. (2017d). Small Drinking Water Systems under Spatiotemporal Water Quality Variability: A Risk-based Performance Benchmarking Framework. Manuscript submitted for publication. Berger, & Anders. (1997). Continuous improvement and kaizen: standardization and organizational designs. Integrated Manufacturing Systems, 8(2), 110–117. http://doi.org/10.1108/09576069710165792 Bessant, J., Caffyn, S., Gilbert, J., Harding, R., & Webb, S. (1994). Rediscovering continuous improvement. Technovation, 14(1), 17–29. http://doi.org/10.1016/0166-4972(94)90067-1 Bhuiyan, N., & Baghel, A. (2005). An overview of continuous improvement: from the past to the present. Management Decision, 43(5), 761–771. http://doi.org/10.1108/00251740510597761 Blackhurst, J. V, Scheibe, K. P., & Johnson, D. J. (2008). Supplier risk assessment and monitoring for the automotive industry. International Journal of Physical Distribution & Logistics Management, 38(2), 143–165. http://doi.org/10.1108/09600030810861215 Boyd, D. R. (2006). The Water We Drink: An International Comparison of Drinking Water Standards and Guidelines. Vancouver, BC. British Columbia Ministry of Environment (BCMoE). (2001). The British Columbia Water Quality Index. Retrieved from http://www.env.gov.bc.ca/wat/wq/BCguidelines/indexreport.html   168  British Columbia Ministry of Health (BCMOH). (2013). Small Water System Guidebook. Retrieved from http://www2.gov.bc.ca/assets/gov/environment/air-land-water/small-water-system-guidebook.pdf British Columbia Ministry of Health Living and Sport (BCMHLS). (2010). Comprehensive Drinking Water Source-To-Tap Assessment Guideline. Retrieved from http://www2.gov.bc.ca/assets/gov/environment/air-land-water/cs2ta-intro.pdf British Columbia Office of the Provincial Health Officer (BCOPHO). (2015). Progress on the Action Plan for Safe Drinking Water in British Columbia. Victoria. Retrieved from http://www2.gov.bc.ca/assets/gov/environment/air-land-water/pho-drinkingwater2015-web.pdf Camp, R. C. (1989). Benchmarking: The Search for Industry Best Practices that Lead to Superior Performance. Milwaukee, WI: ASQC Quality Press. Canadian Council of Ministers of the Environment (CCME). (2001). Water Quality Index 1.0 Technical Report. Retrieved from http://www.ccme.ca/files/Resources/calculators/WQI Technical Report (en).pdf CCME. (2002). From Source to Tap: The multi-barrier approach to safe drinking water. Retrieved from http://www.hc-sc.gc.ca/ewh-semt/alt_formats/hecs-sesc/pdf/water-eau/tap-source-robinet/tap-source-robinet-eng.pdf CCME. (2004). From Source to Tap: Guidance on the Multi-Barrier Approach to Safe Drinking Water. Winnipeg, Manitoba. Chang, E. E., Chiang, P. C., Chao, S. H., & Chuang, C. L. (1999). Development and implementation of source water quality standards in Taiwan, ROC. Chemosphere, 39(8), 1317–1332. http://doi.org/10.1016/S0045-6535(99)00035-1 Charisiadis, P., Andra, S. S., Makris, K. C., Christophi, C. A., Skarlatos, D., Vamvakousis, V., … Stephanou, E. G. (2015). Spatial and seasonal variability of tap water disinfection by-products within distribution pipe networks. Science of the Total Environment, 506–507, 26–35. http://doi.org/10.1016/j.scitotenv.2014.10.071 Chen, S. (1985). Ranking fuzzy numbers with maximizing set and minimizing set. Fuzzy Sets and Systems, 17(3), 113–129. Christensen, R., Goucher, N., & Phare, M.-A. (2010). Seeking Water Justice: Strengthening Legal Protection for Canada’s Drinking Water. Cianfrani, C. A., & West, J. E. (2014). ISO 9001:2008 Explained and Expanded. Milwaukee, Wisconsin: American Society for Quality, Quality Press.   169  Clark, R. M., & Boutin, B. K. (2001). Controlling Disinfection By-Products and Microbial Contaminants in Drinking Water Controlling Disinfection By-Products and Microbial. Conservation Council of New Brunswick. (2016). Water Policy in New Brunswick. Retrieved November 11, 2016, from https://www.conservationcouncil.ca/our-programs/freshwater-protection/water-policy-in-new-brunswick/ Corton, M. L. (2003). Benchmarking in the Latin American water sector: the case of Peru. Utilities Policy, 11(3), 133–142. Corton, M., & Berg, S. (2009). Benchmarking Central American water utilities. Utilities Policy, 17(3–4), 267–275. Retrieved from http://www.sciencedirect.com/science/article/pii/S0957178708000891 Coulibaly, H. D., & Rodriguez, M. J. (2003). Spatial and temporal variation of drinking water quality in ten small Quebec utilities. Journal of Environmental Engineering and Science, 2(1), 47–61. http://doi.org/10.1139/s03-001 Coulibaly, H. D., & Rodriguez, M. J. (2004). Development of performance indicators for small Quebec drinking water utilities. Journal of Environmental Management, 73(3), 243–55. http://doi.org/10.1016/j.jenvman.2004.07.003 Daley, K., Castleden, H., Jamieson, R., Furgal, C., & Ell, L. (2014). Municipal water quantities and health in Nunavut households: An exploratory case study in coral Harbour, Nunavut, Canada. International Journal of Circumpolar Health, 73, 1–9. http://doi.org/10.3402/ijch.v73.23843 Danilenko, A., van den Berg, C., Macheve, B., & Morffitt, J. (2014). The IBNET Water Supply and Sanitation Blue Book 2014: The International Benchmarking Network For Water and Sanitation Utilities Databook. Washington, D.C.: World Bank. http://doi.org/10.1596/978-1-4648-0276-8 Davison, A., Deere, D., Stevens, M., Howard, G., & Bartram, J. (2006). Water Safety Plan Manual. Retrieved from http://cdrwww.who.int/entity/water_sanitation_health/dwq/manual.pdf de Loë, R., & Kreutzwiser, R. D. (2007). Challenging the Status Quo: The Evolution of Water Governance in Canada. In K. Bakker (Ed.), Eau Canada : The Future of Canada’s Water. Vancouver, BC: UBC Press. Dionne, G. (2013). Risk Management: History, Definition and Critique (Vol. 17). Dunn, G., & Bakker, K. (2009). Canadian Approaches to Assessing Water Security: An Inventory of Indicators. Vancouver, BC. Retrieved from http://www.watergovernance.ca/PDF/IndicatorsReportFINAL2009.pdf   170  Dunn, G., Harris, L., Cook, C., & Prystajecky, N. (2014a). A comparative analysis of current microbial water quality risk assessment and management practices in British Columbia and Ontario, Canada. Science of the Total Environment, 468–469, 544–552. http://doi.org/10.1016/j.scitotenv.2013.08.004 Dunn, G., Bakker, K., & Harris, L. (2014b). Drinking Water Quality Guidelines across Canadian provinces and territories: Jurisdictional variation in the context of decentralized water governance. International Journal of Environmental Research and Public Health, 11(5), 4634–4651. http://doi.org/10.3390/ijerph110504634 Dunn, G., Harris, L., Bakker, K., Dunn, G., Harris, L., & Bakker, K. (2015). Microbial risk governance : challenges and opportunities in fresh water management in Canada. Canadian Water Resources Journal, 40(3), 237–249. http://doi.org/10.1080/07011784.2015.1043648 Dyck, R., Cool, G., Rodriguez, M., & Sadiq, R. (2014). Treatment, residual chlorine and season as factors affecting variability of trihalomethanes in small drinking water systems. Frontiers of Environmental Science & Engineering, 1–9. http://doi.org/10.1007/s11783-014-0750-1 Dyson, R. G. (2004). Strategic development and SWOT analysis at the University of Warwick. European Journal of Operational Research, 152(3), 631–640. http://doi.org/10.1016/S0377-2217(03)00062-6 Eggertson, L. (2008). Investigative report: 1766 boil-water advisories now in place across Canada. Canadian Medical Association Journal (CMAJ) (Vol. 178). Ottawa. http://doi.org/10.1503/cmaj.080525 Ellison, D. (2009). Design guide for small drinking water treatment plants, Québec, Ministère du Développement durable, de l’Environnement et des Parcs. 2009. Elmuti, D., & Kathawala, Y. (1997). An overview of benchmarking process: a tool for continuous improvement and competitive advantage. Benchmarking: An International Journal, 4(4), 229–243. http://doi.org/10.1108/14635779710195087 Environment and Climate Change Canada. (2016). Drinking Water Advisories in Canada. Environmental Indicators. Retrieved from https://www.ec.gc.ca/indicateurs-indicators/default.asp?lang=en&n=2C75C17A-1 Environment Canada. (2009). Threats to water availability in Canada, 2009. Retrieved from https://www.ec.gc.ca/indicateurs-indicators/default.asp?lang=en&n=FCFA75B2-1 Environment Canada. (2014). Canadian Environmental Sustainability Indicators: Data Sources and Methods for the Residential Water Use in Canada Indicator. Retrieved from https://www.ec.gc.ca/indicateurs-indicators/default.asp?lang=en&n=D43360E1-1   171  Exall, K., Marsalek, J., & Schaefer, K. (2006). Water reuse in Canada: opportunities and challenges. In P. Hlavinek, T. Kukharchuyk, J. Marsalek, & I. Mahrikova (Eds.), Integrated Urban Water Resources Management (pp. 253–262). Springer. http://doi.org/10.1007/1-4020-4685-5_26 Focazio, M. J., Welch, A. H., Watkins, S. a, Helsel, D. R., & Horn, M. a. (2000). A retrospective analysis on the occurrence of arsenic in ground-water resources of the United States and limitations in drinking-water-supply characterizations. Reston, VA.  Ford, T., Rupp, G., Butterfield, P., & Camper, A. (2005). Protecting Public Health in Small Water Systems. Bozeman, MT. Retrieved from https://ia800206.us.archive.org/31/items/C02D17F1-7CF3-4673-A Francisque, A., Rodriguez, M. J., Miranda-Moreno, L. F., Sadiq, R., & Proulx, F. (2009). Modeling of heterotrophic bacteria counts in a water distribution system. Water Research, 43(4), 1075–1087. http://doi.org/10.1016/j.watres.2008.11.030 Gagnon, G. (2014). Drinking Water Safety Plans. Retrieved from http://www.cwn-rce.ca/assets/Uploads/Drinking-Water-Safety-Plan-Report-2015.pdf Gangbazo, G. (2004). Élaboration d’un plan directeur de l’eau : guide à l’intention des organismes de bassins versants. Quebec, QC. Gangbazo, G. (2011). Guide pour l’élaboration d’un plan directeur de l’eau : un manuel pour assister les organismes de bassin versant du Québec dans la planification de la gestion intégrée des ressources en eau. Quebec, QC. Gleick, P. and Heberger, M. (2014). Water and conflict: events, trends and anlysis (2011 - 2012). The World’s Water Series Volume 8. Retrieved from http://worldwater.org/wp-content/uploads/sites/22/2013/07/www8-water-conflict-events-trends-analysis.pdf Goodwin, D., Raffin, M., Jeffrey, P., & Smith, H. M. (2015). Apply the water safety plan to water reuse: towards a conceptual risk management framework. Environmental Science: Water Research & Technology, 1, 709–722. http://doi.org/10.1039/C5EW00070J Government of Alberta. (2012). A guidance framework for the production of drinking water safety plans. Retrieved from http://environment.gov.ab.ca/info/posting.asp?assetid=8489&searchtype=asset&txtsearch=drinking water safety Government of New Brunswick. (2009). Know Your H2O: Domestic Water Quality Monitoring. Retrieved from http://www2.gnb.ca/content/dam/gnb/Departments/env/pdf/Water-Eau/KnowYourH2O.pdf   172  Government of Newfoundland and Labrador. (2009). Best Management Practices for the Control of Disinfection by-Products in Drinking Water Systems in Newfoundland and Labrador. St. John’s, NL. Government of Newfoundland and Labrador. (2010). Drinking Water Safety in Newfoundland and Labrador: Annual Report 2010. Retrieved from http://www.env.gov.nl.ca/env/waterres/reports/drinking_water/annual_report_2009-10.pdf Government of Newfoundland and Labrador. (2014). Drinking Water Safety in Newfoundland and Labrador: Annual Report 2015. St. John’s. Retrieved from http://www.env.gov.nl.ca/env/waterres/reports/drinking_water/annual_report_2015.pdf Government of Nova Scotia. (2002). A Drinking Water Strategy for Nova Scotia. Retrieved from https://www.novascotia.ca/nse/water/docs/NSWaterStrategy.pdf Government of Prince Edward Island. (2001). Clear from the ground to the glass: 10 Points to Purity. Retrieved from http://www.gov.pe.ca/photos/original/10points2purity.pdf Government of Quebec. (2012). Strategy of protection and conservation of water sources intended for human consumption (Projet de stratégie de protection et de conservation des sources destinées à l’alimentation en eau potable). Government of Quebec. (2014). Regulation for water intake and sources protection (Règlement sur le prélèvement des eaux et leur protection). Loi sur la qualité de l’environnement, chapitre Q-2, r. 35.2 Gazette Officielle du Québec, N° 31. Government of Saskatchewan. (1999). Water Management Framework, 1–3. http://doi.org/10.1002/ejoc.201200111 Government of Saskatchewan. (2003). Saskatchewan’s Safe Drinking Water Strategy. Retrieved from http://www.saskh2o.ca/pdf/ltsdws_report2003.pdf Government of Saskatchewan. (2014). Annual Report for 2014-2015: State of Drinking Water Quality in Saskatchewan. Retrieved from https://www.wsask.ca/Global/About WSA/Annual Reports and Plans/Drinking Water Annual Report/WSA-Drinking-Water-Report-14-2015.pdf Government of the Northwest Territories. (2005). Managing Drinking Water Quality in the Northwest Territories. Government of the Northwest Territories. (2011). Northern Voices, Northern Waters: NWT Water Stewardship Strategy.   173  Government of the Northwest Territories. (2014). GNWT Report on Drinking Water. Retrieved from http://www.nwtdrinkingwater.ca/wp-content/uploads/MACA-Drinking-Water-Report-2014.pdf Government of the Northwest Territories. (2015). 2015 Water Quality Summary.Guay, C., Rodriguez, M., & Sérodes, J. (2005). Using ozonation and chloramination to reduce the formation of trihalomethanes and haloacetic acids in drinking water. Desalination, 176, 229–240. http://doi.org/10.1016/j.desal.2004.10.015 Guilherme, S., & Rodriguez, M. J. (2014). Occurrence of regulated and non-regulated disinfection by-products in small drinking water systems. Chemosphere, 117C, 425–432. http://doi.org/10.1016/j.chemosphere.2014.08.002 Gunnarsdóttir, M. J. (2012). Safe drinking water : Experience with Water Safety Plans and assessment of risk factors in water supply. University of Iceland. Gunnarsdóttir, M. J., & Gissurarson, L. R. (2008). HACCP and water safety plans in Icelandic water supply: Preliminary evaluation of experience. Journal of Water and Health, 6(3), 377. http://doi.org/10.2166/wh.2008.055 Haider., H., Sadiq, R., & Tesfamariam, S. (2014). Performance indicators for small and medium-sized water supply systems: a review. Environmental Reviews, 22(March), 1–40. Haider, H., Sadiq, R., Tesfamariam, S., & Asce, M. (2015a). Inter-Utility Performance Benchmarking Model for Small-to-Medium-Sized Water Utilities : Aggregated Performance Indices. J. Water Resour. Plann. Manage., 4015039(18). http://doi.org/10.1061/(ASCE)WR.1943-5452.0000552. Haider, H., Sadiq, R., & Tesfamariam, S. (2015b). Selecting performance indicators for small to medium sized water utilities: Multi-criteria analysis using ELECTRE method. Urban Water Journal, 12(4), 305–327. Haider, H., Sadiq, R., & Tesfamariam, S. (2016a). Inter-Utility Performance Benchmarking Model For Small-to-Medium-Sized Water Utilities: Aggregated Performance Indicies. J. Water Resour. Plann. Manage., 142(1). Haider, H., Sadiq, R., & Tesfamariam, S. (2016b). Intra-utility Performance Management Model (In-UPM) for the sustainability of small to medium sized water utilities: Conceptualization to development. J. Clean. Prod., (133), 777–794. Haider, H., Sadiq, R., & Tesfarmariam, S. (2016c). Risk-based Framework for Improving Customer Satisfaction through System Reliability in Small-Sized to Medium-Sized Water Utilities. J. Manage. Eng., 32(5).   174  Häring, I. (2015). Risk analysis and management: engineering resilience. Springer Science+Business Media, LLC. http://doi.org/10.1007/978-981-10-0015-7 Health Canada. (2008). Protecting the Water in your Pipes. Retrieved from http://www.hc-sc.gc.ca/fniah-spnia/pubs/promotion/_environ/pipe-conduite/index-eng.php Health Canada. (2013). Guidance for Providing Safe Drinking Water in Areas of Federal Jurisdiction Version 2. Ottawa, Ontario. Retrieved from http://healthycanadians.gc.ca/publications/healthy-living-vie-saine/water-federal-eau/alt/water-federal-eau-eng.pdf Health Canada. (2014). Guidelines for Canadian Drinking Water Quality Summary Table. Ottawa, Canada. Retrieved from http://www.hc-sc.gc.ca/ewh-semt/pubs/water-eau/sum_guide-res_recom/index-eng.php#e Health Canada. (2016). Drinking Water Advisories in First Nations Communities. Retrieved June 4, 2016, from http://www.hc-sc.gc.ca/fniah-spnia/promotion/public-publique/water-dwa-eau-aqep-eng.php#more Hébert, S. (1997). Développement d’un indice de la qualité bactériologique et physico-chimique de l’eau pour les rivières du Québec. Québec, Québec. http://doi.org/no EN/970102 Hill, C., Furlong, K., Bakker, K., & Cohen, A. (2008). Harmonization Versus Subsidiarity in Water Governance: A Review of Water Governance and Legislation in the Canadian Provinces and Territories. Canadian Water Resources Journal, 33(4), 315–332. http://doi.org/10.4296/cwrj3304315 Hinckley, A. F., Bachand, A. M., Nuckols, J. R., & Reif, J. S. (2005). Identifying public water facilities with low spatial variability of disinfection by-products for epidemiological investigations. Occupational and Environmental Medicine, 62(7), 494–9. http://doi.org/10.1136/oem.2004.017798 Hokstad, P., Røstum, J., Sklet, S., Rosén, L., Pettersson, T. J. R., Linde, A., … Niewersch, C. (2009). Methods for risk analysis of drinking water systems from source to tap - Guidance report on Risk Analysis. Hrudey, S. E. (2011). Safe Drinking Water Policy for Canada - Turning Hindsight into Foresight. C.D. Howe Institute Commentary, (323). Hrudey, S. E., Conant, B., Douglas, I. P., Fawell, J., Gillespie, T., Hill, D., … Sinclair, M. (2011). Managing Uncertainty in the Provision of Safe Drinking Water. Water Science and Technology, 11(6), 675–681.   175  Hrudey, S. E., Hrudey, E. J., & Pollard, S. J. T. (2006). Risk management for assuring safe drinking water. Environment International, 32(8), 948–957. http://doi.org/10.1016/j.envint.2006.06.004 Hrudey, S. E., Payment, P., Huck, P. M., Gillham, R. W., & Hrudey, E. J. (2003). A fatal waterborne disease epidemic in Walkerton, Ontario: comparison with other waterborne outbreaks in the developed world. Water Science and Technology, 47(3), 7–14. Hubbert, D. (2013). Identifying Drinking Water Regulatory Frameworks. University of Victoria. Hurlbert, M., Diaz, H., Corkal, D. R., & Warren, J. (2009). Climate change and water governance in Saskatchewan, Canada. International Journal of Climate Change Strategies and Management, 1(2), 118–132. http://doi.org/10.1108/17568690910955595 Hulsmann, A., & Smeets, P. (2011). Towards a Guidance Document for the implementation of a Risk Assessment for small water supplies in the European Union. Nieuwegein. Hurley, T., Sadiq, R., & Mazumder, A. (2012). Adaptation and evaluation of the Canadian Council of Ministers of the Environment Water Quality Index (CCME WQI) for use as an effective tool to characterize drinking source water quality. Water Research, 46(11), 3544–3552. http://doi.org/10.1016/j.watres.2012.03.061 International Organization for Standardization (ISO). (2015). ISO 9001: Quality management systems - Requirements. Geneva, Switzerland: International Organization for Standardization. International Petroleum Industry Environmental Conservation Association (IPIECA). (2013). The IPIECA Water Mangement Framework for onshore oil and gas activities. London, United Kingdom. Retrieved fro Islam, N., Sadiq, R., Rodriguez, M. J., & Francisque, A. (2011). Reviewing source water protection strategies: A conceptual model for water quality assessment. Environmental Reviews, 19(NA), 68–105. http://doi.org/10.1139/a11-001 Islam, N., Sadiq, R., Rodriguez, M. J., & Francisque, A. (2013a). Evaluation of source water protection strategies: a fuzzy-based model. Journal of Environmental Management, 121, 191–201. http://doi.org/10.1016/j.jenvman.2013.02.022 Islam, N., Sadiq, R., & Rodriguez, M. J. (2013b). Optimizing booster chlorination in water distribution networks: a water quality index approach. Environmental Monitoring and Assessment, 185(10), 8035–50. http://doi.org/10.1007/s10661-013-3153-z Jackson, S. E., Joshi, A., & Erhardt, N. L. (2003). Recent research on team and organizational diversity: SWOT analysis and implications. Journal of Management, 29(6), 801–830. http://doi.org/10.1016/S0149-2063(03)00080-1   176  Jang, J. S. R. (1993). ANFIS: Adaptive-Network-Based Fuzzy Inference System. IEEE Transactions on Systems, Man and Cybernetics, 23(3), 665–685. http://doi.org/10.1109/21.256541 Jayaratne, A. (2008). Application of a risk management system to improve drinking water safety. Journal of Water and Health, 6(4), 547–557. http://doi.org/10.2166/wh.2008.061 Kafetzopoulos, D. P., Psomas, E. L., & Kafetzopoulos, P. D. (2013). Measuring the effectiveness of the HACCP Food Safety Management System. Food Control, 33(2), 505–513. http://doi.org/10.1016/j.foodcont.2013.03.044 Kaur, A., & Kaur, A. (2012). Comparison of Mamdani-Type and Sugeno-Type Fuzzy Inference Systems for Air Conditioning System. International Journal of Soft Computing & Engineering, 2(2), 323–325. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.486.1238&rep=rep1&type=pdf Kayser, G. L., Amjad, U., Dalcanale, F., Bartram, J., & Bentley, M. E. (2015). Drinking water quality governance: A comparative case study of Brazil, Ecuador, and Malawi. Environmental Science and Policy, 48, 186–195. http://doi.org/10.1016/j.envsci.2014.12.019 Khan, A. A., Paterson, R., & Khan, H. (2004). Modification and application of the Canadian Council of Ministers of the Environment Water Quality Index (CCME WQI) for the communication of drinking water quality data in Newfoundland and Labrador. Water Quality Research Journal of Canada, 39(3), 285–293. Klir, G. J., St. Clair, U. H., & Yuan, B. (1997). Fuzzy Set Theory: Foundations and Applications. Upper Saddle River, NJ: Prentice Hall PTR. Klir, G. J., & Yuan, B. (2000). Fuzzy Sets and Fuzzy Logic: Theory and Applications. New Delhi, India: Prentine Hall of India. Knapp, R. B. (2007). Fuzzy Sets and Pattern Recognition. Retrieved November 11, 2016, from http://www.cs.princeton.edu/courses/archive/fall07/cos436/HIDDEN/Knapp/fuzzy.htm Lambert, A., & Taylor, R. (2010). Water Loss Guidelines. Retrieved from https://www.waternz.org.nz/Folder?Action=View File&Folder_id=101&File=100503_waterloss_guidelines.pdf Langley, G. J., Nolan, K. M., & Nolan, T. W. (1994). The Foundation of Improvement. Quality Progress, 27(6), 81–86. Lee, C. (1990). Fuzzy logic in control systems: fuzzy logic controller, Parts I and II. IEEE Trans. Syst. Man Cybern., 20, 404–435.   177  Lee, J., Kim, E.-S., Roh, B.-S., Eom, S.-W., & Zoh, K.-D. (2013). Occurrence of disinfection by-products in tap water distribution systems and their associated health risk. Environmental Monitoring and Assessment, 185, 7675–7691. http://doi.org/10.1007/s10661-013-3127-1 Legislative Assembly of Manitoba. Bill 46: The Save Lake Winnipeg Act (2011). 5th session, 39th legislature. Retrieved from https://web2.gov.mb.ca/bills/39-5/b046e.php Legislative Assembly of Manitoba. The Water Protection Act (2005). Retrieved from https://web2.gov.mb.ca/laws/statutes/ccsm/w065e.php Ling, J., & Husain, T. (2014). Technologies to Remove DBPs in Drinking Water in Newfoundland and Labrador - A Review. St. John’s, NL. Lodgaard, E., & Aasland, K. E. (2011). An Examination of the Application of Plan- Do-Check-Act Cycle in Product Development. In International Conference on Engineering Design, ICED11. Lumb, A., Halliwell, D., & Sharma, T. (2006). Application of CCME water quality index to monitor water quality: A case of the Mackenzie River Basin, Canada. Environmental Monitoring and Assessment, 113, 411–429. http://doi.org/10.1007/s10661-005-9092-6 Malec, H. A. (1994). Benchmarking Barometers for Products and Processes. Quality and Reliability Engineering International, 10, 455–465. Mamdani, E. H. (1977). Application of fuzzy logic to approximate reasoning using linguistic synthesis. IEEE Transactions on Computers, C-26(12), 1182–1191. http://doi.org/10.1109/TC.1977.1674779 Manitoba Provincial Planning Regulation. (n.d.). Guide to Developing a Drinking Water Plan. Manitoba Water Stewardship. (2003). The Manitoba Water Strategy. Retrieved from https://www.gov.mb.ca/conservation/waterstewardship/waterstrategy/pdf/index.html?print&print&print Manitoba Water Stewardship. (2007). Best Practices Manual for Small Drinking Water Systems. Retrieved from http://www.gov.mb.ca/waterstewardship/odw/reg-info/operations-monitor/best_practices_for_small_drinking_water_systems-manual.pdf Manitoba Water Stewardship. (2011a). Manitoba Water Quality Standards, Objectives, and Guidelines. Retrieved from https://www.gov.mb.ca/waterstewardship/water_quality/quality/pdf/mb_water_quality_standard_final.pdf   178  Manitoba Water Stewardship. (2011b). Safe Drinking Water. Retrieved from https://www.gov.mb.ca/waterstewardship/odw/public-info/fact_sheets/pdf/factsheet_drinking_water_program.pdf Marques, R. C., Berg, S., & Yane, S. (2014). Nonparametric benchmarking of Japanese water utilities: Institutional and environmental factors affecting efficiency. Journal of Water Resources Planning and Management, 140(5), 562–571. http://doi.org/10.1061/(ASCE)WR.1943-5452.0000366 Márquez-Vera, M. A., Ramos-Fernández, J. C., Cerecero-Natale, L. F., Lafont, F., Balmat, J.-F., & Esparza-Villanueva, J. I. (2016). Temperature control in a MISO greenhouse by inverting its fuzzy model. Computers and Electronics in Agriculture, 124, 168–174. http://doi.org/10.1016/j.compag.2016.04.005 Martin, D., Bélanger, D., Gosselin, P., Brazeau, J., Furgal, C., & Déry, S. (2007). Drinking water and potential threats to human health in Nunavik: Adaptation strategies under climate change conditions. Arctic, 60(2), 195–202. http://doi.org/10.1016/j.neuroimage.2009.09.064 Martinez-Costa, M., Choi, T. Y., Martinez, J. A., & Martinez-Lorente, A. R. (2009). ISO 9000/1994, ISO 9001/2000 and TQM: The performance debate revisited. Journal of Operations Management, 27(6), 495–511. http://doi.org/10.1016/j.jom.2009.04.002 Mercier Shanks, C., Sérodes, J.-B., & Rodriguez, M. J. (2013). Spatio-temporal variability of non-regulated disinfection by-products within a drinking water distribution network. Water Research, 47, 3231-3243 Mitchell, V. (1999). Consumer perceived risk: conceptualisations and models. European Journal of Marketing, 33(1/2), 163–195. Retrieved from http://dx.doi.org/10.1108/03090569910249229 Moen, R. (2009). Foundation and History of the PDSA Cycle. In Asian Network for Quality Conference. Tokyo, Japan. Retrieved from https://www.deming.org/sites/default/files/pdf/2015/PDSA_History_Ron_Moen.pdf Moen, R., & Norman, C. (2006). Evolution of the PDCA Cycle. Moffatt, H., & Struck, S. (2011). Water-borne Disease Outbreaks in Canadian Small Drinking Water Systems. Morris, T. J., Boyd, D. R., Brandes, O., Bruce, J. P., Hudon, M., Lucas, B., … Phare, M. (2007). Changing the flow: A Blueprint for Federal Action on Freshwater. Retrieved from http://poliswaterproject.org/sites/default/files/ChangingtheFlow_1.pdf Murphy, H. M., Thomas, M. K., Medeiros, D. T., McFadyn, S., & Pintar, K. D. M. (2015). Estimating the number of cases of acute gastrointestinal illness (AGI) associated with   179  Canadian municipal drinking water systems. Epidemiology and Infection, 1371–1385. http://doi.org/10.1017/S0950268815002083 Mujumdar, P. P., & Sasikumar, K. (2002). A fuzzy risk approach for seasonal water quality management of a riversystem. Water Resources Research, 38(1), 5–9. http://doi.org/10.1029/2000WR000126 National Collaborating Centre for Environmental Health (NCCEH). (2014). Small Drinking Water Systems: Who Does What in Nunavut? National Health and Medical Research Council (NHMRC) (Australia). (2011). Australian Drinking Water Guidelines 6. Canberra. http://doi.org/1864965118 National Research Council (NRC). (2006). Drinking Water Distribution Systems: Assessing and Reducing Risks. Washington, D.C.: National Academies Press. Retrieved from http://www.nap.edu/download.php?r New Zealand Ministry of Health (NZMOH). (2008). Drinking-water Standards for New Zealand 2005 (Revised 2008) (Vol. 2005). Wellington, New Zealand. Retrieved from http://www.health.govt.nz/publication/drinking-water-standards-new-zealand-2005-revised-2008-0 Nova Scotia Environment and Labour. (2005). A Drinking Water Strategy for Nova Scotia: Final Report. Halifax, NS. Nova Scotia Environment and Labour. (n.d.). Safe Drinking Water for Public Water Systems: A Diligent Approach. Halifax, NS. Retrieved from https://novascotia.ca/nse/water/docs/DrinkingWaterDueDiligenceFull.pdf NZMOH. (2015). Annual Report on Drinking-water Quality. Wellington, New Zealand. Retrieved from http://www.health.govt.nz/system/files/documents/publications/annual-report-on-drinking-water-quality-2013-2014-v2.pdf NZMOH. (2015). Guidelines for Drinking-water Quality Management for New Zealand. Wellington, New Zealand. Retrieved from http://www.health.govt.nz/system/files/documents/publications/guidelines-drinking-water-quality-management-for-new-zealand-2015-oct15.pdf O’Connor, D. R. (2002a). Report of the Walkerton Inquiry: The events of May 2000 and related issues, part one. Toronto: Queen’s Printer for Ontario. O’Connor, D. R. (2002b). Report of the Walkerton Inquiry: The events of May 2000 and related issues, part two. Toronto: Queen’s Printer for Ontario.   180  Ontario Ministry of Health and Long-Term Care (MOHLTC). (2008). Small Drinking Water Systems (SDWS) Risk Assessment Directives Guideline Document. Ontario Ministry of the Environment (MoE). (2007). Implementing Quality Management: A Guide for Ontario’s Drinking Water Systems. Retrieved from https://dr6j45jk9xcmk.cloudfront.net/documents/1592/4-1a-1a-quality-management-system-part-1-en-pdf.pdf PaiMazumder, D., Sushama, L., Laprise, R., Khaliq, M. N., & Sauchyn, D. (2013). Canadian RCM projected changes to short- and long-term drought characteristics over the Canadian Prairies. International Journal of Climatology, 33(6), 1409–1423. http://doi.org/10.1002/joc.3521 Pérard, E. (2009). Water supply: Public or private? An approach based on cost of funds, transaction costs, efficiency and political costs. Policy and Society. http://doi.org/10.1016/j.polsoc.2008.10.004 Perrier, E., Kot, M., Castleden, H., & Gagnon, G. A. (2014). Drinking water safety plans: Barriers and bridges for small systems in Alberta, Canada. Water Policy, 16(6), 1140–1154. http://doi.org/10.2166/wp.2014.207 Pesce, S. F., & Wunderlin, D. a. (2000). Use of Water Quality Indices to Verify the Impact of Cordoba City (Argentina) on Suquia River. Water Research, 34(11), 2915–2926. Plummer, R., Velaniskis, J., de Grosbois, D., Kreutzwiser, R. D., & de Loe, R. (2010). The development of new environmental policies and processes in response to a crisis: the case of the multiple barrier approach for safe drinking water. Environmental Science and Policy, 13(6), 535–548. http://doi.org/10.1016/j.envsci.2010.05.004 Pons, W., Young, I., Truong, J., Jones-Bitton, A., McEwen, S., Pintar, K., & Papadopoulos, A. (2015). A systematic review of waterborne disease outbreaks associated with small non-community drinking water systems in Canada and the United States. PLoS ONE, 10(10), 1–17. http://doi.org/10.1371/journal.pone.0141646 Quebec Ministry of Environment (MoE). (2002). Quebec Water Policy. Retrieved from http://www.mddelcc.gouv.qc.ca/eau/politique/policy.pdf Rahaman, M. M., & Varis, O. (2005). Integrated water resources management: Evolution, prospects and future challenges. Sustainability: Science, Practice & Policy, 1(1), 15–21. http://doi.org/10.1016/S0011-9164(99)00099-5 Reid, D. C., Abramowski, K., Beier, a., Janzen, a., Lok, D., Mack, H., … Vatcher, R. (2014). Implementation of Alberta’s drinking water safety plans. Water Quality Research Journal of Canada, 49(1), 5. http://doi.org/10.2166/wqrjc.2013.063   181  Reid, R. A., Koljonen, E. L., & Bruce Buell, J. (1999). The Deming Cycle Provides a Framework For Managing Environmentally Responsible Process Improvements. Quality Engineering, 12(2), 199–209. http://doi.org/10.1080/08982119908962577 Richardson, S. (2003). Disinfection by-products and other emerging contaminants in drinking water. Trends in Analytical Chemistry, 22(10), 666–684. http://doi.org/10.1016/S0165-9936(03)01003-3 Rizak, S., & Hrudey, S. E. (2008). Drinking water safety- challenges for community managed systems. Journal of Water and Health, 6(1), 33–41. http://doi.org/10.2166/wh.2008.033 Rizvi, Z., & Adamowski, J. (2013). First Nation capacity in Québec to practise integrated water resources management. International Journal of Water, 7(3), 161–190. http://doi.org/10.1504/IJW.2013.054859 Rodriguez, M. J., Serodes, J. B., & Morin, M. (2000). Estimation of water utility compliance with trihalomethane regulations using a modelling approach. Journal of Water Supply, (Levallois 1997), 57–73.  Rodriguez, M. J., & Sérodes, J. B. (2001). Spatial and temporal evolution of trihalomethanes in three water distribution systems. Water Research, 35(6), 1572–1586. http://doi.org/10.1016/S0043-1354(00)00403-6 Rodriguez, M., Vinette, Y., Serodes, J., & Bouchard, C. (2003). Trihalomethanes in drinking water of greater Quebec region (Canada): Occurrence, variations and modelling. Environmental Monitoring and Assessment, 89(1), 69–93. http://doi.org/10.1023/A:1025811921502 Ropkins, K., & Beck, A. J. (2000). Evaluation of worldwide approaches to the use of HACCP to control food safety. Trends in Food Science and Technology, 11(1), 10–21. http://doi.org/10.1016/S0924-2244(00)00036-4 Ross, T. J. (2010). Fuzzy Logic with Engineering Applications. Wiley (Third Edit). http://doi.org/10.1002/9781119994374 Roy, D., Oborne, B., & Venema, H. D. (2009). Integrated Water Resources Management (IWRM) in Canada: Recommendations for Agricultural Sector Participation. Winnipeg, Manitoba. http://doi.org/10.1016/j.pce.2008.07.001 Ruiz, M. A. (2008). Plan-Do-Check-Act Approach to an Environmental Management System. In Innovative Energy Workshop. Sacramento, California. Sadiq, R. (2004). Performance evaluation of slow sand filters using fuzzy rule-based modelling. Environmental Modelling & Software, 19(5), 507–515. http://doi.org/10.1016/S1364-8152(03)00165-8   182  Sadiq, R. (2015). Advanced Methods in Risk Analysis [Powerpoint slides]. Kelowna, BC. Sadiq, R., Kleiner, Y., & Rajani, B. (2003). Forensics of water quality failure in distribution system: a conceptual framework. Indian Water Works Association, 35(4), 267–278. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.8137&rep=rep1&type=pdf Sadiq, R., & Rodriguez, M. J. (2004). Fuzzy synthetic evaluation of disinfection by-products--a risk-based indexing system. Journal of Environmental Management, 73(1), 1–13. http://doi.org/10.1016/j.jenvman.2004.04.014 Saunders, J. O., & Wenig, M. (2007). Whose Water? Canadian Water Management and the Challenges of Jurisdictional Fragmentation. In Eau Canada : The Future of Canada’s Water (pp. 119–142). Vancouver, BC: University of British Columbia Press. Scheili, A., Rodriguez, M. J., & Sadiq, R. (2014). Seasonal and spatial variations of source and drinking water quality in small municipal systems of two Canadian regions. Science of the Total Environment, 508, 514–524. http://doi.org/10.1016/j.scitotenv.2014.11.069 Scheili, A., Rodriguez, M. J., & Sadiq, R. (2015). Development, application, and sensitivity analysis of a water quality index for drinking water management in small systems. Environmental Monitoring and Assessment, 187(11), 685. http://doi.org/10.1007/s10661-015-4908-5 Schuster, C. J., Ellis, A. G., Robertson, W. J., Charron, D. F., Aramini, J. J., Marshall, B. J., & Medeiros, D. T. (2005). Infectious Disease Outbreaks Related to Drinking Water in Canada, 1974-2001. Canadian Journal of Public Health, 96(4), 254–258. Retrieved from http://www.jstor.org/stable/41995866 Schindler, D. W. (2001). The cumulative effects of climate warming and other human stresses on Canadian freshwaters in the new millennium 1. Can. J. Fish. Aquat. Sci., 58(1), 18–29. Simms, G.; de Loë, R. C. (2010). Challenges for water governance in Canada: A discussion paper. Governance for Source Water Protection in Canada Report No. 2. Waterloo, ON. Retrieved from http://www.governanceforwater.ca/sites/default/files/WPGG_Report_No2_WEB.pdf Sinclair, M., & Rizak, S. (2004). Drinking-Water Quality Management: the Australian Framework. Journal of Toxicology & Environmental Health: Part A, 67(20–22), 1567–1579. http://doi.org/10.1080/15287390490491873 Smith, D. G. (1990). A better water quality indexing system for rivers and streams. Water Research, 24(10), 1237–1244. http://doi.org/10.1109/TAC.1974.1100589   183  Sperber, W. H. (2005). HACCP and transparency. Food Control, 16, 505–509. http://doi.org/10.1016/j.foodcont.2003.10.012 Statistics Canada. (2010). Human Activity and the Environment Freshwater supply and demand in Canada. Retrieved from http://www.statcan.gc.ca/pub/16-201-x/2010000/t236-eng.htm Statistics Canada. (2011). Survey of Drinking Water Plants. Retrieved from www.statcan.gc.ca/pub/16.../16-403-x2013001-eng.pdf Statistics Canada. (2013). Average daily treated water volumes per month for Canada and selected drainage regions, 2005 to 2007. Retrieved from http://www.statcan.gc.ca/pub/16-403-x/2013001/ct002-eng.htm Statistics Canada. (2016). Table 051-0001 - Estimates of population by age group and sex for July 1, Canada, Provinces and territories, CANSIM (database). Retrieved June 4, 2016, from http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/demo02a-eng.htm Sugeno, M. (1985). An introductory survey on fuzzy control. Inf. Sci., 36, 59–83. Swamee, P. K., & Tyagi, A. (2000). Describing Water Quality with Aggregate Index. J. Env. Eng, 126(5), 451–455. Symanski, E., Savitz, D. A., & Singer, P. C. (2004). Assessing spatial fluctuations, temporal variability, and measurement error in estimated levels of disinfection by-products in tap water: implications for exposure assessment. Occup. Environ. Med., 61(1), 65–72. Retrieved from http://oem.bmj.com/cgi/content/long/61/1/65 Takagi, T., & Sugeno, M. (1985). Fuzzy identification of systems and its applications to modeling and control. Systems, Man and Cybernetics, IEEE Transactions on, SMC-15(1), 116–132. http://doi.org/10.1109/TSMC.1985.6313399 Talluri, S., & Sarkis, J. (2001). A computational geometry approach for benchmarking. International Journal of Operations & Production Management, 21(1/2), 210–223. http://doi.org/10.1108/01443570110358549 Taylor, M. (2002). Development of Drinking-water Management in New Zealand since 1992. Wellington, New Zealand. Retrieved from http://www.moh.govt.nz/notebook/nbbooks.nsf/0/6d2e4dfe48b2d0b1cc25783600721d2e/$FILE/nzdw-management.pdf Taylor, M. J., McNicholas, C., Nicolay, C., Darzi, A., Bell, D., & Reed, J. E. (2013). Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf, (September), 1–9. http://doi.org/10.1136/bmjqs-2013-001862   184  Terziovski, M., Power, D., & Sohal, A. S. (2003). The longitudinal effects of the ISO 9000 certification process on business performance. European Journal of Operational Research, 146(3), 580–595. http://doi.org/10.1016/S0377-2217(02)00252-7 Tricker, R. (2016). ISO 9001: 2015 Audit Procedures (Fourth Edi). Routledge. Turner, S. W. D., Blackwell, R. J., Smith, M. A., & Jeffrey, P. J. (2016). Risk-based water resources planning in England and Wales: challenges in execution and implementation. Urban Water Journal, 13(2), 182–197. http://doi.org/10.1080/1573062X.2014.955856 United States Environmental Protection Agency (USEPA). (2006). The Effectiveness of Disinfectant Residuals in the Distribution System. Retrieved from www.epa.gov/ogwdw/disinfection/tcr/pdfs/issuepaper_effectiveness.pdf USEPA. (2009). National Primary Drinking Water Regulations. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/11800007 USEPA. (2011). Drinking Water Infrastructure Needs Survey and Assessment Fifth Report to Congress. Washington, D.C. Retrieved from https://www.epa.gov/sites/production/files/2015-07/documents/epa816r13006.pdf USEPA. (2016). Protecting Drinking Water Quality through Promotion of Water Safety Plans. Retrieved May 23, 2016, from https://www.epa.gov/international-cooperation/protecting-drinking-water-quality-through-promotion-water-safety-plans United States Food and Drug Administration (USFDA). (2006). Guidance for Industry. Q9 Quality Risk Management. Rockville, MD. Retrieved from www.fda.gov/cder/guidance/index.htm Vancouver Airport Authority. (2015). Vancouver airport authority 2015-2019 environmental management plan. Vancouver, BC. Retrieved from http://www.yvr.ca/-/media/yvr/documents/about-yvr/environment/2014-environmental-management-plan.pdf?la=en Vinson, N. G. (2012). Towards estimating the economic burden of waterborne illness in Canada: what do we know, where do we go?, 12. Retrieved from http://nparc.cisti-icist.nrc-cnrc.gc.ca/eng/view/accepted/?id=319c22ff-b36b-402b-a67d-b05d5171f3a7 von der Porten, S., & De Loe, R. C. (2013). Collaborative approaches to governance for water and Indigenous peoples: A case study from British Columbia, Canada. GeoForum, 50, 149–160. http: World Health Organization (WHO). (2004). Guidelines for Drinking-water Quality (Vol. 3). Geneva. Retrieved from http://www.who.int/water_sanitation_health/dwq/gdwqvol32ed.pdf   185  WHO. (2009). Water Safety Plan Manual: Step-by-step risk management for drinking-water suppliers. Geneva, Switzerland. Retrieved from http://apps.who.int/iris/bitstream/106 WHO. (2011). Guidelines for Drinking Water Quality (Vol. 4). Geneva. WHO. (2014). Investing in Water and Sanitation: Increasing Access, Reducing Inequalities (Vol. 1). Geneva, Switzerland. Retrieved from http://www.who.int/water_sanitation_health/publications/glaas_report_2014/en/ World Summit on Sustainable Development (WSSD). (2002). Report of the World Summit on Sustainable Development. In Johannesburg World Summit on Sustainable Development (p. 176). New York, NY. Retrieved from Available from: http://www.unmillenniumproject.org/documents/131302_wssd_report_reissued.pdf Yasin, M. M. (2002). The theory and practice of benchmarking: then and now. Benchmarking: An International Journal, 9(3), 217–243. Retrieved from http://dx.doi.org/10.1108/14635770210428992 Yokoi, H. (2006). Study on the Introduction of Hazard Analysis and Critical Control Point (HACCP) Concept of the Water Quality Management. Water Science and Technology, 53(4/5), 483–492. Yusa, A., Berry, P., Cheng, J. J., Ogden, N., Bonsal, B., Stewart, R., & Waldick, R. (2015). Climate change, drought and human health in Canada. International Journal of Environmental Research and Public Health, 12(7), 8359–8412. http://doi.org/10.3390/ijerph120708359 Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8(3), 338–353. Zadeh, L. A. (1971). Toward a Theory of Fuzzy Systems. Aspects of Network and System Theory. Berkeley, CA. Retrieved from https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19690027655.pdf Zadeh, L. A. (1975a). Fuzzy Logic and Approximate Reasoning. Synthese, 30(i), 407–428. http://doi.org/10.1017/CBO9781107415324.004 Zadeh, L. A. (1975b). The concept of a linguistic variable and its application to approximate reasoning-I. Information Sciences, 8(3), 199–249. http://doi.org/10.1016/0020-0255(75)90036-5 Zadeh, L. a. (1997). Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic. Fuzzy Sets and Systems, 90(2), 111–127. http://doi.org/10.1016/S0165-0114(97)00077-8   186  Zangwill, W. I., & Kantor, P. B. (1998). Toward a Theory of Continuous Improvement and the Learning Curve. Management Science, 44(7), 910–920. http://doi.org/10.1287/mnsc.44.7.910 Zimmermann, H. J. (2001). Fuzzy Set Theory and Its Applications (4th ed.). New York, NY: Springer Science+Business Media, LLC. Zubrycki, K., Roy, D., Venema, H. D., & Brooks, D. (2011). Water Security in Canada: Responsibilities of the federal government. Retrieved from http://www.iisd.org/pdf/2011/water_security_canada.pdf   187  Appendices   188  Appendix A: CCME WQI Inputs/Outputs for CPI Framework Demonstration  Appendix A-1 : CPI Framework CCME WQI Input Data (NL) SDWS Season THM4-R1 THM4-R2 THM4-R3 HAA5-R1 HAA5-R2 HAA5-R3 FRC-R1 FRC-R2 FRC-R3  Turbidity TC SDWS1 AUTUMN 137.07 136.73 134.71 83.30 118.98 162.32 1.54 0.80 0.38  0.36 0.00 SDWS1 AUTUMN 75.25 83.74 125.87 65.40 101.69 118.94 1.49 1.02 0.20  0.49 N/A SDWS1 AUTUMN 83.16 127.91 158.97 84.81 N/A 168.82 2.08 1.56 0.52  0.45 0.00 SDWS1 AUTUMN 91.21 111.71 139.86 82.06 138.29 153.77 1.67 1.27 0.10  0.38 0.00 SDWS1 WINTER 104.16 138.69 101.39 99.68 156.85 157.63 1.73 1.12 0.30  0.32 0.00 SDWS1 WINTER 80.42 120.50 137.78 99.34 118.37 205.37 1.45 1.07 0.80  0.38 N/A SDWS1 WINTER 93.38 113.32 117.09 92.85 154.77 134.06 1.48 1.13 1.14  0.51 0.00 SDWS1 SPRING 87.62 80.97 98.34 64.83 126.54 115.39 1.21 0.86 0.86  0.28 0.00 SDWS1 SPRING 74.15 96.18 95.90 81.19 95.33 100.54 1.21 1.14 1.07  0.27 0.00 SDWS1 SPRING 71.40 117.75 143.30 71.34 104.61 154.05 1.13 1.01 0.80  0.39 0.00 SDWS1 SUMMER 77.50 100.61 125.83 61.75 83.97 132.74 0.86 0.65 0.52  0.72 0.00 SDWS1 SUMMER 112.34 9.86 155.34 100.00 96.36 87.11 0.48 0.21 0.04  0.51 0.00 SDWS1 SUMMER 101.90 141.72 153.94 72.50 117.32 132.57 1.28 1.27 0.28  0.56 0.00 SDWS2 AUTUMN 75.09 87.76 114.74 102.91 126.53 71.28 0.77 0.45 0.19  0.62 0.00 SDWS2 AUTUMN 75.65 72.48 105.86 7.74 63.84 86.27 0.11 0.37 0.21  0.60 0.00 SDWS2 AUTUMN 119.99 324.35 85.49 48.83 133.16 207.24 0.64 0.55 0.34  0.69 0.00 SDWS2 AUTUMN 81.65 91.53 N/A 51.99 129.42 N/A 0.82 0.65 0.41  1.28 109.10 SDWS2 WINTER 67.24 171.71 114.13 100.30 112.40 147.87 0.79 0.59 0.46  1.22 0.00 SDWS2 WINTER 112.87 69.47 132.82 118.36 122.60 171.26 1.06 1.14 1.01  0.65 1.00 SDWS2 WINTER 43.46 53.15 68.60 66.12 74.26 106.36 0.14 0.09 0.08  0.59 0.00 SDWS2 SPRING 61.91 48.87 70.22 88.25 124.58 187.75 0.45 0.48 0.25  1.33 0.00 SDWS2 SPRING 39.34 65.57 93.91 73.82 102.98 129.99 0.28 0.24 0.19  0.37 N/A SDWS2 SPRING 56.54 N/A 83.38 N/A N/A N/A N/A N/A N/A  N/A 0.00 SDWS2 SUMMER N/A N/A N/A 59.41 72.25 129.06 0.52 0.82 0.67  1.07 0.00 SDWS2 SUMMER 65.27 89.71 86.97 96.65 107.43 144.51 0.39 0.56 1.58  1.04 0.00 SDWS2 SUMMER 73.85 226.78 106.55 70.04 95.83 57.50 N/A 0.51 0.21  1.19 0.00 SDWS3 AUTUMN N/A 167.10 169.80 N/A 98.18 4.32 0.54 0.19 0.12  0.52 0.00   189  SDWS Season THM4-R1 THM4-R2 THM4-R3 HAA5-R1 HAA5-R2 HAA5-R3 FRC-R1 FRC-R2 FRC-R3  Turbidity TC SDWS3 AUTUMN 126.51 116.93 114.67 205.15 197.32 65.75 1.07 0.17 0.10  0.57 0.00 SDWS3 AUTUMN 201.11 342.85 140.03 204.57 165.54 101.96 0.65 0.31 0.06  0.53 0.00 SDWS3 AUTUMN N/A 70.45 60.48 N/A 116.57 86.70 0.46 0.28 0.08  0.40 0.00 SDWS3 WINTER 79.81 87.58 76.78 125.44 133.25 57.27 0.42 0.33 0.05  0.49 0.00 SDWS3 WINTER 84.23 85.39 77.28 151.58 137.95 13.29 0.17 0.16 0.08  0.46 0.00 SDWS3 WINTER 75.56 52.35 72.05 113.65 103.98 17.81 0.18 0.06 0.10  N/A 0.00 SDWS3 SPRING 80.53 109.71 85.22 80.92 N/A 40.09 0.34 0.10 0.11  0.25 0.00 SDWS3 SPRING 76.46 95.20 109.58 117.36 180.90 9.19 0.24 0.14 0.09  2.07 0.00 SDWS3 SPRING 97.80 175.20 215.20 98.43 162.15 77.16 0.09 0.14 0.10  1.30 0.00 SDWS3 SUMMER 40.59 126.94 99.94 39.71 73.78 45.07 0.11 0.17 0.16  0.52 16.40 SDWS3 SUMMER 119.93 123.33 125.78 171.38 60.37 122.29 0.14 0.10 0.05  0.49 5.30 SDWS3 SUMMER 125.12 103.54 116.89 76.32 74.17 3.20 0.14 0.09 0.10  0.49 0.00 SDWS4 AUTUMN 175.42 342.48 503.60 254.78 3.65 277.83 1.80 0.26 0.19  2.21 0.00 SDWS4 AUTUMN 137.16 177.78 N/A 173.89 82.63 39.27 1.11 0.03 0.03  0.86 1.00 SDWS4 AUTUMN 243.26 430.35 397.07 241.77 242.28 203.96 1.50 0.12 0.07  0.81 0.00 SDWS4 AUTUMN 161.27 173.65 158.86 175.57 120.40 N/A 1.38 0.09 0.04  1.08 0.00 SDWS4 WINTER 238.30 154.79 119.40 180.72 323.11 174.36 -1.00 0.09 0.03  9.64 0.00 SDWS4 WINTER 95.44 209.30 223.26 239.48 222.61 317.30 1.20 0.21 0.08  1.43 0.00 SDWS4 WINTER 98.49 241.98 174.08 148.41 245.06 166.07 0.92 0.11 0.07  1.35 0.00 SDWS4 WINTER 83.92 212.63 168.73 176.52 273.21 243.80 1.54 0.10 0.06  2.90 0.00 SDWS4 SPRING 76.55 201.71 206.99 180.24 251.54 150.22 0.90 0.14 0.07  1.11 0.00 SDWS4 SPRING 153.92 99.71 211.40 216.79 70.07 106.34 1.60 0.07 0.04  2.10 0.00 SDWS4 SUMMER 153.18 150.47 163.55 249.27 3.70 100.14 1.61 0.05 0.05  1.17 22.20 SDWS4 SUMMER 165.88 194.55 166.09 222.80 77.55 92.32 0.98 0.05 0.06  1.19 1.00 SDWS4 SUMMER 201.47 307.30 295.98 78.63 3.52 18.49 2.14 0.04 0.07  1.66 0.00 SDWS5 AUTUMN 45.47 127.78 176.36 42.36 139.89 77.12 1.53 0.13 0.03  0.33 N/A SDWS5 AUTUMN N/A N/A N/A N/A N/A N/A N/A N/A N/A  N/A 0.00 SDWS5 AUTUMN 30.40 117.15 114.62 51.34 114.84 61.28 1.24 0.02 0.02  0.65 0.00 SDWS5 AUTUMN 50.78 102.54 105.81 50.80 105.55 118.11 1.21 0.33 0.10  0.17 0.00 SDWS5 WINTER 49.40 62.36 73.60 25.60 65.74 64.63 2.01 1.41 1.21  0.11 0.00 SDWS5 WINTER 19.49 37.27 35.90 19.75 48.27 55.38 1.49 1.21 1.13  0.13 0.00 SDWS5 WINTER 10.83 25.19 20.41 8.84 25.54 19.87 1.25 1.39 1.28  0.12 0.00 SDWS5 WINTER 60.48 112.77 109.27 99.88 122.10 126.12 1.18 0.09 0.07  0.65 0.00   190  SDWS Season THM4-R1 THM4-R2 THM4-R3 HAA5-R1 HAA5-R2 HAA5-R3 FRC-R1 FRC-R2 FRC-R3  Turbidity TC SDWS5 SPRING 5.82 5.82 35.37 14.77 32.96 29.47 1.66 1.41 1.38  0.13 0.00 SDWS5 SPRING N/A N/A 78.88 21.84 72.03 77.58 1.35 0.94 0.88  0.40 0.00 SDWS5 SUMMER 23.89 97.11 84.68 27.79 76.57 70.54 0.00 0.69 0.03  0.89 0.00 SDWS5 SUMMER 92.65 55.40 71.31 87.32 24.57 74.20 0.45 0.02 0.05  1.02 0.00 SDWS5 SUMMER 37.08 26.99 28.68 28.06 79.33 97.14 1.48 0.86 0.75  0.50 0.00 SDWS6 AUTUMN 20.65 512.83 248.28 404.74 206.23 6.97 2.20 1.83 1.83  0.60 0.00 SDWS6 AUTUMN 192.10 290.96 60.00 330.17 297.59 26.14 1.87 0.00 0.76  1.35 0.00 SDWS6 AUTUMN 284.58 221.90 376.88 235.02 339.08 199.70 0.73 0.05 0.16  0.79 0.00 SDWS6 AUTUMN 166.71 283.38 324.28 204.38 355.96 344.70 1.94 0.34 0.68  0.98 0.00 SDWS6 WINTER 248.31 125.49 233.37 259.30 144.32 84.58 2.50 0.08 0.04  0.80 0.00 SDWS6 WINTER 109.79 233.45 215.95 262.15 342.80 311.30 3.70 5.00 0.02  0.80 0.00 SDWS6 WINTER 123.65 194.98 190.97 164.43 290.38 326.93 2.50 0.16 0.01  0.73 0.00 SDWS6 SPRING 92.21 198.51 213.30 127.74 247.88 233.47 3.10 0.94 0.12  1.03 0.00 SDWS6 SPRING 107.67 198.09 183.59 158.25 243.06 225.78 2.50 0.94 0.07  0.81 1.00 SDWS6 SPRING 148.66 244.25 219.28 203.52 176.46 270.20 2.90 1.50 0.43  0.94 0.00 SDWS6 SUMMER 181.07 293.55 283.63 172.07 367.03 212.92 2.90 1.67 0.81  0.82 83.10 SDWS6 SUMMER 279.13 284.69 287.18 419.16 252.74 257.78 2.50 0.32 0.17  3.07 0.00 SDWS6 SUMMER 317.83 440.16 312.96 141.52 599.99 8.20 2.50 0.56 0.15  1.33 0.00 SDWS7 AUTUMN 330.01 43.82 N/A 223.99 118.69 230.39 N/A N/A N/A  0.37 0.00 SDWS7 AUTUMN 167.46 231.14 206.38 133.28 181.60 243.91 2.10 0.91 0.57  0.42 0.00 SDWS7 AUTUMN 194.58 226.36 60.97 265.42 181.85 236.19 1.40 0.01 0.01  0.46 0.00 SDWS7 AUTUMN 192.41 210.32 208.15 297.43 192.84 201.35 2.50 0.33 0.00  1.30 0.00 SDWS7 WINTER 79.08 136.34 139.26 156.87 202.61 230.12 -1.00 0.14 0.20  0.42 0.00 SDWS7 WINTER 107.31 180.68 160.94 155.36 159.94 203.72 1.89 0.44 0.20  0.45 N/A SDWS7 WINTER 85.96 94.19 N/A 128.75 100.39 N/A 0.54 0.10 0.13  0.43 0.00 SDWS7 SPRING 81.23 153.59 123.71 108.07 144.82 148.68 0.70 0.46 0.34  0.38 0.00 SDWS7 SPRING 83.02 158.41 118.94 99.84 160.65 165.57 1.44 0.12 0.83  0.39 0.00 SDWS7 SPRING 107.60 187.35 163.42 109.36 166.91 165.19 1.76 0.17 0.69  0.80 0.00 SDWS7 SUMMER 122.85 198.79 166.82 121.49 161.56 193.96 1.39 0.25 0.68  0.88 0.00 SDWS7 SUMMER 139.23 186.89 235.22 162.44 112.15 208.66 1.60 0.21 0.65  0.89 0.00 SDWS7 SUMMER 141.38 274.50 222.41 167.42 12.19 211.24 1.66 0.32 1.02  0.52 0.00    191  Appendix A-2 : Established CCME WQI Guidelines for CPI Framework Demonstration Variable No Water Quality Parameter Non-compliance if: Value1 Value 2 Unit 1 THM4-R1 > 100  µg/L 2 THM4-R2 > 100  µg/L 3 THM4-R3 > 100  µg/L 4 HAA5-R1 > 80  µg/L 5 HAA5-R2 > 80  µg/L 6 HAA5-R3 > 80  µg/L 7 FRC-R1 <> 0.2 0.8 mg/L 8 FRC-R2 <> 0.2 0.8 mg/L 9 FRC-R3 <> 0.2 0.8 mg/L 10 Turbidity > 1  NTU 11 TC > 0   #   192  Appendix A-3 : Seasonal CCME WQI Outputs by SDWS for CPI Framework  Station Index Period F1 F2  F3 CCME WQI SDWS1 AUTUMN 81.8 61.9  26.5 38.8 SDWS1 WINTER 81.8 71.9  27.2 35.2 SDWS1 SPRING 72.7 51.5  15.3 47.8 SDWS1 SUMMER 81.8 54.5  22.5 41.8 SDWS2 AUTUMN 90.9 35.7  73.5 29.4 SDWS2 WINTER 100.0 57.6  24.5 31.9 SDWS2 SPRING 45.5 29.2  12.1 68.0 SDWS2 SUMMER 72.7 41.4  14.2 51.0 SDWS3 AUTUMN 81.8 57.5  31.7 39.4 SDWS3 WINTER 45.5 40.6  27.2 61.5 SDWS3 SPRING 72.7 56.3  28.2 44.5 SDWS3 SUMMER 81.8 60.6  50.1 34.5 SDWS4 AUTUMN 100.0 81.0  57.1 18.7 SDWS4 WINTER 90.9 81.4  57.3 22.2 SDWS4 SPRING 90.9 77.3  48.3 25.7 SDWS4 SUMMER 90.9 81.8  63.6 20.4 SDWS5 AUTUMN 63.6 54.5  48.1 44.2 SDWS5 WINTER 72.7 38.6  20.9 50.9 SDWS5 SPRING 27.3 30.0  15.0 75.0 SDWS5 SUMMER 54.5 27.3  100.0 32.4 SDWS6 AUTUMN 90.9 65.9  100.0 13.2 SDWS6 WINTER 81.8 81.8  68.9 22.3 SDWS6 SPRING 100.0 81.8  50.8 19.8 SDWS6 SUMMER 100.0 81.8  80.2 12.2 SDWS7 AUTUMN 90.9 72.5  100.0 11.5 SDWS7 WINTER 81.8 58.6  33.0 38.8 SDWS7 SPRING 81.8 63.6  28.1 38.0 SDWS7 SUMMER 72.7 63.6  37.4 40.2   193  Appendix A-4 : Seasonal CCME WQI Outputs by Parameter for CPI Framework Demonstration Station Index Period 1 2 3 4 THM4-R1 THM4-R2 THM4-R3 HAA5-R1 Number of Tests Number of Failed Tests Percent Failed (%) Number of Tests Number of Failed Tests Percent Failed (%) Number of Tests Number of Failed Tests Percent Failed (%) Number of Tests Number of Failed Tests Percent Failed (%) SDWS1 AUTUMN 4 1 25.0 4 3 75.0 4 4 100.0 4 3 75.0 SDWS1 WINTER 3 1 33.3 3 3 100.0 3 3 100.0 3 3 100.0 SDWS1 SPRING 3 0  3 1 33.3 3 1 33.3 3 1 33.3 SDWS1 SUMMER 3 2 66.7 3 2 66.7 3 3 100.0 3 1 33.3 SDWS2 AUTUMN 4 1 25.0 4 1 25.0 3 2 66.7 4 1 25.0 SDWS2 WINTER 3 1 33.3 3 1 33.3 3 2 66.7 3 2 66.7 SDWS2 SPRING 3 0  2 0  3 0  2 1 50.0 SDWS2 SUMMER 2 0  2 1 50.0 2 1 50.0 3 1 33.3 SDWS3 AUTUMN 2 2 100.0 4 3 75.0 4 3 75.0 2 2 100.0 SDWS3 WINTER 3 0  3 0  3 0  3 3 100.0 SDWS3 SPRING 3 0  3 2 66.7 3 2 66.7 3 3 100.0 SDWS3 SUMMER 3 2 66.7 3 3 100.0 3 2 66.7 3 1 33.3 SDWS4 AUTUMN 4 4 100.0 4 4 100.0 3 3 100.0 4 4 100.0 SDWS4 WINTER 4 1 25.0 4 4 100.0 4 4 100.0 4 4 100.0 SDWS4 SPRING 2 1 50.0 2 1 50.0 2 2 100.0 2 2 100.0 SDWS4 SUMMER 3 3 100.0 3 3 100.0 3 3 100.0 3 2 66.7 SDWS5 AUTUMN 3 0  3 3 100.0 3 3 100.0 3 0   SDWS5 WINTER 4 0  4 1 25.0 4 1 25.0 4 1 25.0 SDWS5 SPRING 1 0  1 0  2 0  2 0   SDWS5 SUMMER 3 0  3 0  3 0  3 1 33.3 SDWS6 AUTUMN 4 3 75.0 4 4 100.0 4 3 75.0 4 4 100.0 SDWS6 WINTER 3 3 100.0 3 3 100.0 3 3 100.0 3 3 100.0 SDWS6 SPRING 3 2 66.7 3 3 100.0 3 3 100.0 3 3 100.0 SDWS6 SUMMER 3 3 100.0 3 3 100.0 3 3 100.0 3 3 100.0 SDWS7 AUTUMN 4 4 100.0 4 3 75.0 3 2 66.7 4 4 100.0 SDWS7 WINTER 3 1 33.3 3 2 66.7 2 2 100.0 3 3 100.0 SDWS7 SPRING 3 1 33.3 3 3 100.0 3 3 100.0 3 3 100.0 SDWS7 SUMMER 3 3 100.0 3 3 100.0 3 3 100.0 3 3 100.0   194  Station Index Period 5 6 7 8 HAA5-R2 HAA5-R3 FRC-R1 FRC-R2 Number of Tests Number of Failed Tests Percent Failed (%) Number of Tests Number of Failed Tests Percent Failed (%) Number of Tests Number of Failed Tests Percent Failed (%) Number of Tests Number of Failed Tests Percent Failed (%) SDWS1 AUTUMN 3 3 100.0 4 4 100.0 4 4 100.0 4 3 75.0 SDWS1 WINTER 3 3 100.0 3 3 100.0 3 3 100.0 3 3 100.0 SDWS1 SPRING 3 3 100.0 3 3 100.0 3 3 100.0 3 3 100.0 SDWS1 SUMMER 3 3 100.0 3 3 100.0 3 2 66.7 3 1 33.3 SDWS2 AUTUMN 4 3 75.0 3 2 66.7 4 2 50.0 4 0   SDWS2 WINTER 3 2 66.7 3 3 100.0 3 2 66.7 3 2 66.7 SDWS2 SPRING 2 2 100.0 2 2 100.0 2 0  2 0   SDWS2 SUMMER 3 2 66.7 3 2 66.7 2 0  3 1 33.3 SDWS3 AUTUMN 4 4 100.0 4 2 50.0 4 1 25.0 4 2 50.0 SDWS3 WINTER 3 3 100.0 3 0  3 2 66.7 3 2 66.7 SDWS3 SPRING 2 2 100.0 3 0  3 1 33.3 3 3 100.0 SDWS3 SUMMER 3 0  3 1 33.3 3 3 100.0 3 3 100.0 SDWS4 AUTUMN 4 3 75.0 3 2 66.7 4 4 100.0 4 3 75.0 SDWS4 WINTER 4 4 100.0 4 4 100.0 3 3 100.0 4 3 75.0 SDWS4 SPRING 2 1 50.0 2 2 100.0 2 2 100.0 2 2 100.0 SDWS4 SUMMER 3 0  3 2 66.7 3 3 100.0 3 3 100.0 SDWS5 AUTUMN 3 3 100.0 3 1 33.3 3 3 100.0 3 2 66.7 SDWS5 WINTER 4 1 25.0 4 1 25.0 4 4 100.0 4 4 100.0 SDWS5 SPRING 2 0  2 0  2 2 100.0 2 2 100.0 SDWS5 SUMMER 3 0  3 1 33.3 3 2 66.7 3 2 66.7 SDWS6 AUTUMN 4 4 100.0 4 2 50.0 4 3 75.0 4 3 75.0 SDWS6 WINTER 3 3 100.0 3 3 100.0 3 3 100.0 3 3 100.0 SDWS6 SPRING 3 3 100.0 3 3 100.0 3 3 100.0 3 3 100.0 SDWS6 SUMMER 3 3 100.0 3 2 66.7 3 3 100.0 3 1 33.3 SDWS7 AUTUMN 4 4 100.0 4 4 100.0 3 3 100.0 3 2 66.7 SDWS7 WINTER 3 3 100.0 2 2 100.0 2 1 50.0 3 2 66.7 SDWS7 SPRING 3 3 100.0 3 3 100.0 3 2 66.7 3 2 66.7 SDWS7 SUMMER 3 2 66.7 3 3 100.0 3 3 100.0 3 0      195  Station Index Period 9 10 11 FRC-R3 Turbidity TC Number of Tests Number of Failed Tests Percent Failed (%) Number of Tests Number of Failed Tests Percent Failed (%) Number of Tests Number of Failed Tests Percent Failed (%) SDWS1 AUTUMN 4 1 25.0 4 0  3 0   SDWS1 WINTER 3 1 33.3 3 0  2 0   SDWS1 SPRING 3 2 66.7 3 0  3 0   SDWS1 SUMMER 3 1 33.3 3 0  3 0   SDWS2 AUTUMN 4 1 25.0 4 1 25.0 4 1 25.0 SDWS2 WINTER 3 2 66.7 3 1 33.3 3 1 33.3 SDWS2 SPRING 2 1 50.0 2 1 50.0 2 0   SDWS2 SUMMER 3 1 33.3 3 3 100.0 3 0   SDWS3 AUTUMN 4 4 100.0 4 0  4 0   SDWS3 WINTER 3 3 100.0 2 0  3 0   SDWS3 SPRING 3 3 100.0 3 2 66.7 3 0   SDWS3 SUMMER 3 3 100.0 3 0  3 2 66.7 SDWS4 AUTUMN 4 4 100.0 4 2 50.0 4 1 25.0 SDWS4 WINTER 4 4 100.0 4 4 100.0 4 0   SDWS4 SPRING 2 2 100.0 2 2 100.0 2 0   SDWS4 SUMMER 3 3 100.0 3 3 100.0 3 2 66.7 SDWS5 AUTUMN 3 3 100.0 3 0  3 0   SDWS5 WINTER 4 4 100.0 4 0  4 0   SDWS5 SPRING 2 2 100.0 2 0  2 0   SDWS5 SUMMER 3 2 66.7 3 1 33.3 3 0   SDWS6 AUTUMN 4 2 50.0 4 1 25.0 4 0   SDWS6 WINTER 3 3 100.0 3 0  3 0   SDWS6 SPRING 3 2 66.7 3 1 33.3 3 1 33.3 SDWS6 SUMMER 3 3 100.0 3 2 66.7 3 1 33.3 SDWS7 AUTUMN 3 2 66.7 4 1 25.0 4 0   SDWS7 WINTER 3 1 33.3 3 0  2 0   SDWS7 SPRING 3 1 33.3 3 0  3 0   SDWS7 SUMMER 3 1 33.3 3 0   3 0     196  Appendix B: Fuzzy rules for Risk-based Performance Benchmarking Framework (RWQI)   Appendix B-1 : Matrix Defining Fuzzy Rules for ‘Generation 5’  Generation 5 Rules  Autumn No.  Concentration Location Gen 4 Outcome 1 Very Low Low Very Low 2 Very Low Medium Very Low 3 Very Low High Low 4 Low Low Low 5 Low Medium Low 6 Low High Medium 7 Medium Low Low 8 Medium Medium Medium 9 Medium High High 10 High Low Medium 11 High Medium High 12 High High Very High 13 Very High Low High 14 Very High Medium Very High 15 Very High High Very High   197  Winter No.  Concentration Location Gen 4 Outcome 1 Very Low Low Very Low 2 Very Low Medium Very Low 3 Very Low High Low 4 Low Low Low 5 Low Medium Low 6 Low High Medium 7 Medium Low Medium 8 Medium Medium Medium 9 Medium High Medium 10 High Low Medium 11 High Medium High 12 High High High 13 Very High Low High 14 Very High Medium Very High 15 Very High High Very High   198  Spring No.  Concentration Location Gen 4 Outcome 1 Very Low Low Very Low 2 Very Low Medium Very Low 3 Very Low High Low 4 Low Low Low 5 Low Medium Low 6 Low High Medium 7 Medium Low Medium 8 Medium Medium Medium 9 Medium High Medium 10 High Low Medium 11 High Medium High 12 High High High 13 Very High Low High 14 Very High Medium Very High 15 Very High High Very High   199  Summer No.  Concentration Location Gen 4 Outcome 1 Very Low Low Very Low 2 Very Low Medium Very Low 3 Very Low High Low 4 Low Low Low 5 Low Medium Low 6 Low High Medium 7 Medium Low Medium 8 Medium Medium Medium 9 Medium High Medium 10 High Low Medium 11 High Medium High 12 High High High 13 Very High Low High 14 Very High Medium Very High 15 Very High High Very High   200  Appendix B-2 : Matrix Defining Fuzzy Rules for ‘Generation 4’ Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 1 Very High Very High Very High Very High Very High 2 High Very High Very High Very High Very High 3 Medium Very High Very High Very High Very High 4 Low Very High Very High Very High Very High 5 Very Low Very High Very High Very High High 6 Very High Very High High Very High Very High 7 High Very High High Very High Very High 8 Medium Very High High Very High Very High 9 Low Very High High Very High High 10 Very Low Very High High Very High High 11 Very High Very High Medium Very High Very High 12 High Very High Medium Very High Very High 13 Medium Very High Medium Very High High 14 Low Very High Medium Very High High 15 Very Low Very High Medium Very High High 16 Very High Very High Low Very High Very High 17 High Very High Low Very High High 18 Medium Very High Low Very High High 19 Low Very High Low Very High High 20 Very Low Very High Low Very High Medium 21 Very High Very High Very Low Very High High 22 High Very High Very Low Very High High 23 Medium Very High Very Low Very High High 24 Low Very High Very Low Very High Medium 25 Very Low Very High Very Low Very High Medium 26 Very High Very High Very High High Very High 27 High Very High Very High High Very High 28 Medium Very High Very High High Very High 29 Low Very High Very High High High 30 Very Low Very High Very High High High 31 Very High Very High High High Very High 32 High Very High High High Very High 33 Medium Very High High High High 34 Low Very High High High High 35 Very Low Very High High High High 36 Very High Very High Medium High Very High 37 High Very High Medium High High   201  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 38 Medium Very High Medium High High 39 Low Very High Medium High High 40 Very Low Very High Medium High Medium 41 Very High Very High Low High High 42 High Very High Low High High 43 Medium Very High Low High High 44 Low Very High Low High Medium 45 Very Low Very High Low High Medium 46 Very High Very High Very Low High High 47 High Very High Very Low High High 48 Medium Very High Very Low High Medium 49 Low Very High Very Low High Medium 50 Very Low Very High Very Low High Medium 51 Very High Very High Very High Medium Very High 52 High Very High Very High Medium Very High 53 Medium Very High Very High Medium High 54 Low Very High Very High Medium High 55 Very Low Very High Very High Medium High 56 Very High Very High High Medium Very High 57 High Very High High Medium High 58 Medium Very High High Medium High 59 Low Very High High Medium High 60 Very Low Very High High Medium Medium 61 Very High Very High Medium Medium High 62 High Very High Medium Medium High 63 Medium Very High Medium Medium High 64 Low Very High Medium Medium Medium 65 Very Low Very High Medium Medium Medium 66 Very High Very High Low Medium High 67 High Very High Low Medium High 68 Medium Very High Low Medium Medium 69 Low Very High Low Medium Medium 70 Very Low Very High Low Medium Medium 71 Very High Very High Very Low Medium High 72 High Very High Very Low Medium Medium 73 Medium Very High Very Low Medium Medium 74 Low Very High Very Low Medium Medium 75 Very Low Very High Very Low Medium Low 76 Very High Very High Very High Low Very High   202  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 77 High Very High Very High Low High 78 Medium Very High Very High Low High 79 Low Very High Very High Low High 80 Very Low Very High Very High Low Medium 81 Very High Very High High Low High 82 High Very High High Low High 83 Medium Very High High Low High 84 Low Very High High Low Medium 85 Very Low Very High High Low Medium 86 Very High Very High Medium Low High 87 High Very High Medium Low High 88 Medium Very High Medium Low Medium 89 Low Very High Medium Low Medium 90 Very Low Very High Medium Low Medium 91 Very High Very High Low Low High 92 High Very High Low Low Medium 93 Medium Very High Low Low Medium 94 Low Very High Low Low Medium 95 Very Low Very High Low Low Low 96 Very High Very High Very Low Low Medium 97 High Very High Very Low Low Medium 98 Medium Very High Very Low Low Medium 99 Low Very High Very Low Low Low 100 Very Low Very High Very Low Low Low 101 Very High Very High Very High Very Low High 102 High Very High Very High Very Low High 103 Medium Very High Very High Very Low High 104 Low Very High Very High Very Low Medium 105 Very Low Very High Very High Very Low Medium 106 Very High Very High High Very Low High 107 High Very High High Very Low High 108 Medium Very High High Very Low Medium 109 Low Very High High Very Low Medium 110 Very Low Very High High Very Low Medium 111 Very High Very High Medium Very Low High 112 High Very High Medium Very Low Medium 113 Medium Very High Medium Very Low Medium 114 Low Very High Medium Very Low Medium 115 Very Low Very High Medium Very Low Low   203  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 116 Very High Very High Low Very Low Medium 117 High Very High Low Very Low Medium 118 Medium Very High Low Very Low Medium 119 Low Very High Low Very Low Low 120 Very Low Very High Low Very Low Low 121 Very High Very High Very Low Very Low Medium 122 High Very High Very Low Very Low Medium 123 Medium Very High Very Low Very Low Low 124 Low Very High Very Low Very Low Low 125 Very Low Very High Very Low Very Low Low 126 Very High High Very High Very High Very High 127 High High Very High Very High Very High 128 Medium High Very High Very High Very High 129 Low High Very High Very High High 130 Very Low High Very High Very High High 131 Very High High High Very High Very High 132 High High High Very High Very High 133 Medium High High Very High High 134 Low High High Very High High 135 Very Low High High Very High High 136 Very High High Medium Very High Very High 137 High High Medium Very High High 138 Medium High Medium Very High High 139 Low High Medium Very High High 140 Very Low High Medium Very High Medium 141 Very High High Low Very High High 142 High High Low Very High High 143 Medium High Low Very High High 144 Low High Low Very High Medium 145 Very Low High Low Very High Medium 146 Very High High Very Low Very High High 147 High High Very Low Very High High 148 Medium High Very Low Very High Medium 149 Low High Very Low Very High Medium 150 Very Low High Very Low Very High Medium 151 Very High High Very High High Very High 152 High High Very High High Very High 153 Medium High Very High High High 154 Low High Very High High High   204  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 155 Very Low High Very High High High 156 Very High High High High Very High 157 High High High High High 158 Medium High High High High 159 Low High High High High 160 Very Low High High High Medium 161 Very High High Medium High High 162 High High Medium High High 163 Medium High Medium High High 164 Low High Medium High Medium 165 Very Low High Medium High Medium 166 Very High High Low High High 167 High High Low High High 168 Medium High Low High Medium 169 Low High Low High Medium 170 Very Low High Low High Medium 171 Very High High Very Low High High 172 High High Very Low High Medium 173 Medium High Very Low High Medium 174 Low High Very Low High Medium 175 Very Low High Very Low High Low 176 Very High High Very High Medium Very High 177 High High Very High Medium High 178 Medium High Very High Medium High 179 Low High Very High Medium High 180 Very Low High Very High Medium Medium 181 Very High High High Medium High 182 High High High Medium High 183 Medium High High Medium High 184 Low High High Medium Medium 185 Very Low High High Medium Medium 186 Very High High Medium Medium High 187 High High Medium Medium High 188 Medium High Medium Medium Medium 189 Low High Medium Medium Medium 190 Very Low High Medium Medium Medium 191 Very High High Low Medium High 192 High High Low Medium Medium 193 Medium High Low Medium Medium   205  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 194 Low High Low Medium Medium 195 Very Low High Low Medium Low 196 Very High High Very Low Medium Medium 197 High High Very Low Medium Medium 198 Medium High Very Low Medium Medium 199 Low High Very Low Medium Low 200 Very Low High Very Low Medium Low 201 Very High High Very High Low High 202 High High Very High Low High 203 Medium High Very High Low High 204 Low High Very High Low Medium 205 Very Low High Very High Low Medium 206 Very High High High Low High 207 High High High Low High 208 Medium High High Low Medium 209 Low High High Low Medium 210 Very Low High High Low Medium 211 Very High High Medium Low High 212 High High Medium Low Medium 213 Medium High Medium Low Medium 214 Low High Medium Low Medium 215 Very Low High Medium Low Low 216 Very High High Low Low Medium 217 High High Low Low Medium 218 Medium High Low Low Medium 219 Low High Low Low Low 220 Very Low High Low Low Low 221 Very High High Very Low Low Medium 222 High High Very Low Low Medium 223 Medium High Very Low Low Low 224 Low High Very Low Low Low 225 Very Low High Very Low Low Low 226 Very High High Very High Very Low High 227 High High Very High Very Low High 228 Medium High Very High Very Low Medium 229 Low High Very High Very Low Medium 230 Very Low High Very High Very Low Medium 231 Very High High High Very Low High 232 High High High Very Low Medium   206  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 233 Medium High High Very Low Medium 234 Low High High Very Low Medium 235 Very Low High High Very Low Low 236 Very High High Medium Very Low Medium 237 High High Medium Very Low Medium 238 Medium High Medium Very Low Medium 239 Low High Medium Very Low Low 240 Very Low High Medium Very Low Low 241 Very High High Low Very Low Medium 242 High High Low Very Low Medium 243 Medium High Low Very Low Low 244 Low High Low Very Low Low 245 Very Low High Low Very Low Low 246 Very High High Very Low Very Low Medium 247 High High Very Low Very Low Low 248 Medium High Very Low Very Low Low 249 Low High Very Low Very Low Low 250 Very Low High Very Low Very Low Very Low 251 Very High Medium Very High Very High Very High 252 High Medium Very High Very High Very High 253 Medium Medium Very High Very High High 254 Low Medium Very High Very High High 255 Very Low Medium Very High Very High High 256 Very High Medium High Very High Very High 257 High Medium High Very High High 258 Medium Medium High Very High High 259 Low Medium High Very High High 260 Very Low Medium High Very High Medium 261 Very High Medium Medium Very High High 262 High Medium Medium Very High High 263 Medium Medium Medium Very High High 264 Low Medium Medium Very High Medium 265 Very Low Medium Medium Very High Medium 266 Very High Medium Low Very High High 267 High Medium Low Very High High 268 Medium Medium Low Very High Medium 269 Low Medium Low Very High Medium 270 Very Low Medium Low Very High Medium 271 Very High Medium Very Low Very High High   207  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 272 High Medium Very Low Very High Medium 273 Medium Medium Very Low Very High Medium 274 Low Medium Very Low Very High Medium 275 Very Low Medium Very Low Very High Low 276 Very High Medium Very High High Very High 277 High Medium Very High High High 278 Medium Medium Very High High High 279 Low Medium Very High High High 280 Very Low Medium Very High High Medium 281 Very High Medium High High High 282 High Medium High High High 283 Medium Medium High High High 284 Low Medium High High Medium 285 Very Low Medium High High Medium 286 Very High Medium Medium High High 287 High Medium Medium High High 288 Medium Medium Medium High Medium 289 Low Medium Medium High Medium 290 Very Low Medium Medium High Medium 291 Very High Medium Low High High 292 High Medium Low High Medium 293 Medium Medium Low High Medium 294 Low Medium Low High Medium 295 Very Low Medium Low High Low 296 Very High Medium Very Low High Medium 297 High Medium Very Low High Medium 298 Medium Medium Very Low High Medium 299 Low Medium Very Low High Low 300 Very Low Medium Very Low High Low 301 Very High Medium Very High Medium High 302 High Medium Very High Medium High 303 Medium Medium Very High Medium High 304 Low Medium Very High Medium Medium 305 Very Low Medium Very High Medium Medium 306 Very High Medium High Medium High 307 High Medium High Medium High 308 Medium Medium High Medium Medium 309 Low Medium High Medium Medium 310 Very Low Medium High Medium Medium   208  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 311 Very High Medium Medium Medium High 312 High Medium Medium Medium Medium 313 Medium Medium Medium Medium Medium 314 Low Medium Medium Medium Medium 315 Very Low Medium Medium Medium Low 316 Very High Medium Low Medium Medium 317 High Medium Low Medium Medium 318 Medium Medium Low Medium Medium 319 Low Medium Low Medium Low 320 Very Low Medium Low Medium Low 321 Very High Medium Very Low Medium Medium 322 High Medium Very Low Medium Medium 323 Medium Medium Very Low Medium Low 324 Low Medium Very Low Medium Low 325 Very Low Medium Very Low Medium Low 326 Very High Medium Very High Low High 327 High Medium Very High Low High 328 Medium Medium Very High Low Medium 329 Low Medium Very High Low Medium 330 Very Low Medium Very High Low Medium 331 Very High Medium High Low High 332 High Medium High Low Medium 333 Medium Medium High Low Medium 334 Low Medium High Low Medium 335 Very Low Medium High Low Low 336 Very High Medium Medium Low Medium 337 High Medium Medium Low Medium 338 Medium Medium Medium Low Medium 339 Low Medium Medium Low Low 340 Very Low Medium Medium Low Low 341 Very High Medium Low Low Medium 342 High Medium Low Low Medium 343 Medium Medium Low Low Low 344 Low Medium Low Low Low 345 Very Low Medium Low Low Low 346 Very High Medium Very Low Low Medium 347 High Medium Very Low Low Low 348 Medium Medium Very Low Low Low 349 Low Medium Very Low Low Low   209  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 350 Very Low Medium Very Low Low Very Low 351 Very High Medium Very High Very Low High 352 High Medium Very High Very Low Medium 353 Medium Medium Very High Very Low Medium 354 Low Medium Very High Very Low Medium 355 Very Low Medium Very High Very Low Low 356 Very High Medium High Very Low Medium 357 High Medium High Very Low Medium 358 Medium Medium High Very Low Medium 359 Low Medium High Very Low Low 360 Very Low Medium High Very Low Low 361 Very High Medium Medium Very Low Medium 362 High Medium Medium Very Low Medium 363 Medium Medium Medium Very Low Low 364 Low Medium Medium Very Low Low 365 Very Low Medium Medium Very Low Low 366 Very High Medium Low Very Low Medium 367 High Medium Low Very Low Low 368 Medium Medium Low Very Low Low 369 Low Medium Low Very Low Low 370 Very Low Medium Low Very Low Very Low 371 Very High Medium Very Low Very Low Low 372 High Medium Very Low Very Low Low 373 Medium Medium Very Low Very Low Low 374 Low Medium Very Low Very Low Very Low 375 Very Low Medium Very Low Very Low Very Low 376 Very High Low Very High Very High Very High 377 High Low Very High Very High High 378 Medium Low Very High Very High High 379 Low Low Very High Very High High 380 Very Low Low Very High Very High Medium 381 Very High Low High Very High High 382 High Low High Very High High 383 Medium Low High Very High High 384 Low Low High Very High Medium 385 Very Low Low High Very High Medium 386 Very High Low Medium Very High High 387 High Low Medium Very High High 388 Medium Low Medium Very High Medium   210  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 389 Low Low Medium Very High Medium 390 Very Low Low Medium Very High Medium 391 Very High Low Low Very High High 392 High Low Low Very High Medium 393 Medium Low Low Very High Medium 394 Low Low Low Very High Medium 395 Very Low Low Low Very High Low 396 Very High Low Very Low Very High Medium 397 High Low Very Low Very High Medium 398 Medium Low Very Low Very High Medium 399 Low Low Very Low Very High Low 400 Very Low Low Very Low Very High Low 401 Very High Low Very High High High 402 High Low Very High High High 403 Medium Low Very High High High 404 Low Low Very High High Medium 405 Very Low Low Very High High Medium 406 Very High Low High High High 407 High Low High High High 408 Medium Low High High Medium 409 Low Low High High Medium 410 Very Low Low High High Medium 411 Very High Low Medium High High 412 High Low Medium High Medium 413 Medium Low Medium High Medium 414 Low Low Medium High Medium 415 Very Low Low Medium High Low 416 Very High Low Low High Medium 417 High Low Low High Medium 418 Medium Low Low High Medium 419 Low Low Low High Low 420 Very Low Low Low High Low 421 Very High Low Very Low High Medium 422 High Low Very Low High Medium 423 Medium Low Very Low High Low 424 Low Low Very Low High Low 425 Very Low Low Very Low High Low 426 Very High Low Very High Medium High 427 High Low Very High Medium High   211  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 428 Medium Low Very High Medium Medium 429 Low Low Very High Medium Medium 430 Very Low Low Very High Medium Medium 431 Very High Low High Medium High 432 High Low High Medium Medium 433 Medium Low High Medium Medium 434 Low Low High Medium Medium 435 Very Low Low High Medium Low 436 Very High Low Medium Medium Medium 437 High Low Medium Medium Medium 438 Medium Low Medium Medium Medium 439 Low Low Medium Medium Low 440 Very Low Low Medium Medium Low 441 Very High Low Low Medium Medium 442 High Low Low Medium Medium 443 Medium Low Low Medium Low 444 Low Low Low Medium Low 445 Very Low Low Low Medium Low 446 Very High Low Very Low Medium Medium 447 High Low Very Low Medium Low 448 Medium Low Very Low Medium Low 449 Low Low Very Low Medium Low 450 Very Low Low Very Low Medium Very Low 451 Very High Low Very High Low High 452 High Low Very High Low Medium 453 Medium Low Very High Low Medium 454 Low Low Very High Low Medium 455 Very Low Low Very High Low Low 456 Very High Low High Low Medium 457 High Low High Low Medium 458 Medium Low High Low Medium 459 Low Low High Low Low 460 Very Low Low High Low Low 461 Very High Low Medium Low Medium 462 High Low Medium Low Medium 463 Medium Low Medium Low Low 464 Low Low Medium Low Low 465 Very Low Low Medium Low Low 466 Very High Low Low Low Medium   212  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 467 High Low Low Low Low 468 Medium Low Low Low Low 469 Low Low Low Low Low 470 Very Low Low Low Low Very Low 471 Very High Low Very Low Low Low 472 High Low Very Low Low Low 473 Medium Low Very Low Low Low 474 Low Low Very Low Low Very Low 475 Very Low Low Very Low Low Very Low 476 Very High Low Very High Very Low Medium 477 High Low Very High Very Low Medium 478 Medium Low Very High Very Low Medium 479 Low Low Very High Very Low Low 480 Very Low Low Very High Very Low Low 481 Very High Low High Very Low Medium 482 High Low High Very Low Medium 483 Medium Low High Very Low Low 484 Low Low High Very Low Low 485 Very Low Low High Very Low Low 486 Very High Low Medium Very Low Medium 487 High Low Medium Very Low Low 488 Medium Low Medium Very Low Low 489 Low Low Medium Very Low Low 490 Very Low Low Medium Very Low Very Low 491 Very High Low Low Very Low Low 492 High Low Low Very Low Low 493 Medium Low Low Very Low Low 494 Low Low Low Very Low Very Low 495 Very Low Low Low Very Low Very Low 496 Very High Low Very Low Very Low Low 497 High Low Very Low Very Low Low 498 Medium Low Very Low Very Low Very Low 499 Low Low Very Low Very Low Very Low 500 Very Low Low Very Low Very Low Very Low 501 Very High Very Low Very High Very High High 502 High Very Low Very High Very High High 503 Medium Very Low Very High Very High High 504 Low Very Low Very High Very High Medium 505 Very Low Very Low Very High Very High Medium   213  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 506 Very High Very Low High Very High High 507 High Very Low High Very High High 508 Medium Very Low High Very High Medium 509 Low Very Low High Very High Medium 510 Very Low Very Low High Very High Medium 511 Very High Very Low Medium Very High High 512 High Very Low Medium Very High Medium 513 Medium Very Low Medium Very High Medium 514 Low Very Low Medium Very High Medium 515 Very Low Very Low Medium Very High Low 516 Very High Very Low Low Very High Medium 517 High Very Low Low Very High Medium 518 Medium Very Low Low Very High Medium 519 Low Very Low Low Very High Low 520 Very Low Very Low Low Very High Low 521 Very High Very Low Very Low Very High Medium 522 High Very Low Very Low Very High Medium 523 Medium Very Low Very Low Very High Low 524 Low Very Low Very Low Very High Low 525 Very Low Very Low Very Low Very High Low 526 Very High Very Low Very High High High 527 High Very Low Very High High High 528 Medium Very Low Very High High Medium 529 Low Very Low Very High High Medium 530 Very Low Very Low Very High High Medium 531 Very High Very Low High High High 532 High Very Low High High Medium 533 Medium Very Low High High Medium 534 Low Very Low High High Medium 535 Very Low Very Low High High Low 536 Very High Very Low Medium High Medium 537 High Very Low Medium High Medium 538 Medium Very Low Medium High Medium 539 Low Very Low Medium High Low 540 Very Low Very Low Medium High Low 541 Very High Very Low Low High Medium 542 High Very Low Low High Medium 543 Medium Very Low Low High Low 544 Low Very Low Low High Low   214  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 545 Very Low Very Low Low High Low 546 Very High Very Low Very Low High Medium 547 High Very Low Very Low High Low 548 Medium Very Low Very Low High Low 549 Low Very Low Very Low High Low 550 Very Low Very Low Very Low High Very Low 551 Very High Very Low Very High Medium High 552 High Very Low Very High Medium Medium 553 Medium Very Low Very High Medium Medium 554 Low Very Low Very High Medium Medium 555 Very Low Very Low Very High Medium Low 556 Very High Very Low High Medium Medium 557 High Very Low High Medium Medium 558 Medium Very Low High Medium Medium 559 Low Very Low High Medium Low 560 Very Low Very Low High Medium Low 561 Very High Very Low Medium Medium Medium 562 High Very Low Medium Medium Medium 563 Medium Very Low Medium Medium Low 564 Low Very Low Medium Medium Low 565 Very Low Very Low Medium Medium Low 566 Very High Very Low Low Medium Medium 567 High Very Low Low Medium Low 568 Medium Very Low Low Medium Low 569 Low Very Low Low Medium Low 570 Very Low Very Low Low Medium Very Low 571 Very High Very Low Very Low Medium Low 572 High Very Low Very Low Medium Low 573 Medium Very Low Very Low Medium Low 574 Low Very Low Very Low Medium Very Low 575 Very Low Very Low Very Low Medium Very Low 576 Very High Very Low Very High Low Medium 577 High Very Low Very High Low Medium 578 Medium Very Low Very High Low Medium 579 Low Very Low Very High Low Low 580 Very Low Very Low Very High Low Low 581 Very High Very Low High Low Medium 582 High Very Low High Low Medium 583 Medium Very Low High Low Low   215  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 584 Low Very Low High Low Low 585 Very Low Very Low High Low Low 586 Very High Very Low Medium Low Medium 587 High Very Low Medium Low Low 588 Medium Very Low Medium Low Low 589 Low Very Low Medium Low Low 590 Very Low Very Low Medium Low Very Low 591 Very High Very Low Low Low Low 592 High Very Low Low Low Low 593 Medium Very Low Low Low Low 594 Low Very Low Low Low Very Low 595 Very Low Very Low Low Low Very Low 596 Very High Very Low Very Low Low Low 597 High Very Low Very Low Low Low 598 Medium Very Low Very Low Low Very Low 599 Low Very Low Very Low Low Very Low 600 Very Low Very Low Very Low Low Very Low 601 Very High Very Low Very High Very Low Medium 602 High Very Low Very High Very Low Medium 603 Medium Very Low Very High Very Low Low 604 Low Very Low Very High Very Low Low 605 Very Low Very Low Very High Very Low Low 606 Very High Very Low High Very Low Medium 607 High Very Low High Very Low Low 608 Medium Very Low High Very Low Low 609 Low Very Low High Very Low Low 610 Very Low Very Low High Very Low Very Low 611 Very High Very Low Medium Very Low Low 612 High Very Low Medium Very Low Low 613 Medium Very Low Medium Very Low Low 614 Low Very Low Medium Very Low Very Low 615 Very Low Very Low Medium Very Low Very Low 616 Very High Very Low Low Very Low Low 617 High Very Low Low Very Low Low 618 Medium Very Low Low Very Low Very Low 619 Low Very Low Low Very Low Very Low 620 Very Low Very Low Low Very Low Very Low 621 Very High Very Low Very Low Very Low Low 622 High Very Low Very Low Very Low Very Low   216  Generation 4 Rules No. Autumn Winter Spring Summer Gen 3 Outcome 623 Medium Very Low Very Low Very Low Very Low 624 Low Very Low Very Low Very Low Very Low 625 Very Low Very Low Very Low Very Low Very Low   217  Appendix B-3 : Matrix Defining Fuzzy Rules for ‘Generation 3’ Generation 3 Rules Chemical No. FRC THM4 HAA5 Gen 2 Outcome 1 Very Low Very Low Very Low Very Low 2 Low Very Low Very Low Low 3 Medium Very Low Very Low Low 4 High Very Low Very Low Medium 5 Very High Very Low Very Low Medium 6 Very Low Low Very Low Low 7 Low Low Very Low Low 8 Medium Low Very Low Medium 9 High Low Very Low Medium 10 Very High Low Very Low Medium 11 Very Low Medium Very Low Low 12 Low Medium Very Low Medium 13 Medium Medium Very Low Medium 14 High Medium Very Low Medium 15 Very High Medium Very Low High 16 Very Low High Very Low Medium 17 Low High Very Low Medium 18 Medium High Very Low Medium 19 High High Very Low High 20 Very High High Very Low High 21 Very Low Very High Very Low Medium 22 Low Very High Very Low Medium 23 Medium Very High Very Low High 24 High Very High Very Low High 25 Very High Very High Very Low High 26 Very Low Very Low Low Low 27 Low Very Low Low Low 28 Medium Very Low Low Medium 29 High Very Low Low Medium 30 Very High Very Low Low Medium 31 Very Low Low Low Low 32 Low Low Low Medium 33 Medium Low Low Medium 34 High Low Low Medium 35 Very High Low Low High 36 Very Low Medium Low Medium   218  Generation 3 Rules Chemical No. FRC THM4 HAA5 Gen 2 Outcome 37 Low Medium Low Medium 38 Medium Medium Low Medium 39 High Medium Low High 40 Very High Medium Low High 41 Very Low High Low Medium 42 Low High Low Medium 43 Medium High Low High 44 High High Low High 45 Very High High Low High 46 Very Low Very High Low Medium 47 Low Very High Low High 48 Medium Very High Low High 49 High Very High Low High 50 Very High Very High Low Very High 51 Very Low Very Low Medium Low 52 Low Very Low Medium Medium 53 Medium Very Low Medium Medium 54 High Very Low Medium Medium 55 Very High Very Low Medium High 56 Very Low Low Medium Medium 57 Low Low Medium Medium 58 Medium Low Medium Medium 59 High Low Medium High 60 Very High Low Medium High 61 Very Low Medium Medium Medium 62 Low Medium Medium Medium 63 Medium Medium Medium High 64 High Medium Medium High 65 Very High Medium Medium High 66 Very Low High Medium Medium 67 Low High Medium High 68 Medium High Medium High 69 High High Medium High 70 Very High High Medium Very High 71 Very Low Very High Medium High 72 Low Very High Medium High 73 Medium Very High Medium High 74 High Very High Medium Very High   219  Generation 3 Rules Chemical No. FRC THM4 HAA5 Gen 2 Outcome 75 Very High Very High Medium Very High 76 Very Low Very Low High Medium 77 Low Very Low High Medium 78 Medium Very Low High Medium 79 High Very Low High High 80 Very High Very Low High High 81 Very Low Low High Medium 82 Low Low High Medium 83 Medium Low High High 84 High Low High High 85 Very High Low High High 86 Very Low Medium High Medium 87 Low Medium High High 88 Medium Medium High High 89 High Medium High High 90 Very High Medium High Very High 91 Very Low High High High 92 Low High High High 93 Medium High High High 94 High High High Very High 95 Very High High High Very High 96 Very Low Very High High High 97 Low Very High High High 98 Medium Very High High Very High 99 High Very High High Very High 100 Very High Very High High Very High 101 Very Low Very Low Very High Medium 102 Low Very Low Very High Medium 103 Medium Very Low Very High High 104 High Very Low Very High High 105 Very High Very Low Very High High 106 Very Low Low Very High Medium 107 Low Low Very High High 108 Medium Low Very High High 109 High Low Very High High 110 Very High Low Very High Very High 111 Very Low Medium Very High High 112 Low Medium Very High High   220  Generation 3 Rules Chemical No. FRC THM4 HAA5 Gen 2 Outcome 113 Medium Medium Very High High 114 High Medium Very High Very High 115 Very High Medium Very High Very High 116 Very Low High Very High High 117 Low High Very High High 118 Medium High Very High Very High 119 High High Very High Very High 120 Very High High Very High Very High 121 Very Low Very High Very High High 122 Low Very High Very High Very High 123 Medium Very High Very High Very High 124 High Very High Very High Very High 125 Very High Very High Very High Very High   221  Generation 3 Rules Microbial No. E.coli HPC TC Gen 2 Outcome 1 Very Low Very Low Very Low Very Low 2 Low Very Low Very Low Low 3 Medium Very Low Very Low Medium 4 High Very Low Very Low Medium 5 Very High Very Low Very Low High 6 Very Low Low Very Low Low 7 Low Low Very Low Low 8 Medium Low Very Low Medium 9 High Low Very Low Medium 10 Very High Low Very Low High 11 Very Low Medium Very Low Low 12 Low Medium Very Low Medium 13 Medium Medium Very Low Medium 14 High Medium Very Low High 15 Very High Medium Very Low High 16 Very Low High Very Low Low 17 Low High Very Low Medium 18 Medium High Very Low Medium 19 High High Very Low High 20 Very High High Very Low High 21 Very Low Very High Very Low Medium 22 Low Very High Very Low Medium 23 Medium Very High Very Low High 24 High Very High Very Low High 25 Very High Very High Very Low Very High 26 Very Low Very Low Low Very Low 27 Low Very Low Low Low 28 Medium Very Low Low Medium 29 High Very Low Low Medium 30 Very High Very Low Low High 31 Very Low Low Low Low 32 Low Low Low Medium 33 Medium Low Low Medium 34 High Low Low High 35 Very High Low Low High 36 Very Low Medium Low Low 37 Low Medium Low Medium 38 Medium Medium Low Medium   222  Generation 3 Rules Microbial No. E.coli HPC TC Gen 2 Outcome 39 High Medium Low High 40 Very High Medium Low High 41 Very Low High Low Medium 42 Low High Low Medium 43 Medium High Low High 44 High High Low High 45 Very High High Low Very High 46 Very Low Very High Low Medium 47 Low Very High Low Medium 48 Medium Very High Low High 49 High Very High Low High 50 Very High Very High Low Very High 51 Very Low Very Low Medium Low 52 Low Very Low Medium Low 53 Medium Very Low Medium Medium 54 High Very Low Medium Medium 55 Very High Very Low Medium High 56 Very Low Low Medium Low 57 Low Low Medium Medium 58 Medium Low Medium Medium 59 High Low Medium High 60 Very High Low Medium High 61 Very Low Medium Medium Medium 62 Low Medium Medium Medium 63 Medium Medium Medium High 64 High Medium Medium High 65 Very High Medium Medium Very High 66 Very Low High Medium Medium 67 Low High Medium Medium 68 Medium High Medium High 69 High High Medium High 70 Very High High Medium Very High 71 Very Low Very High Medium Medium 72 Low Very High Medium High 73 Medium Very High Medium High 74 High Very High Medium Very High 75 Very High Very High Medium Very High 76 Very Low Very Low High Low   223  Generation 3 Rules Microbial No. E.coli HPC TC Gen 2 Outcome 77 Low Very Low High Medium 78 Medium Very Low High Medium 79 High Very Low High High 80 Very High Very Low High High 81 Very Low Low High Low 82 Low Low High Medium 83 Medium Low High Medium 84 High Low High High 85 Very High Low High High 86 Very Low Medium High Medium 87 Low Medium High Medium 88 Medium Medium High High 89 High Medium High High 90 Very High Medium High Very High 91 Very Low High High Medium 92 Low High High High 93 Medium High High High 94 High High High Very High 95 Very High High High Very High 96 Very Low Very High High Medium 97 Low Very High High High 98 Medium Very High High High 99 High Very High High Very High 100 Very High Very High High Very High 101 Very Low Very Low Very High Low 102 Low Very Low Very High Medium 103 Medium Very Low Very High Medium 104 High Very Low Very High High 105 Very High Very Low Very High High 106 Very Low Low Very High Medium 107 Low Low Very High Medium 108 Medium Low Very High High 109 High Low Very High High 110 Very High Low Very High Very High 111 Very Low Medium Very High Medium 112 Low Medium Very High Medium 113 Medium Medium Very High High 114 High Medium Very High High   224  Generation 3 Rules Microbial No. E.coli HPC TC Gen 2 Outcome 115 Very High Medium Very High Very High 116 Very Low High Very High Medium 117 Low High Very High High 118 Medium High Very High High 119 High High Very High Very High 120 Very High High Very High Very High 121 Very Low Very High Very High High 122 Low Very High Very High High 123 Medium Very High Very High Very High 124 High Very High Very High Very High 125 Very High Very High Very High Very High   225  Generation 3 Rules Physical No. pH Temp. Turbidity Gen 2 Outcome 1 Very Low Very Low Very Low Very Low 2 Low Very Low Very Low Very Low 3 Medium Very Low Very Low Very Low 4 High Very Low Very Low Low 5 Very High Very Low Very Low Low 6 Very Low Low Very Low Low 7 Low Low Very Low Low 8 Medium Low Very Low Low 9 High Low Very Low Low 10 Very High Low Very Low Low 11 Very Low Medium Very Low Low 12 Low Medium Very Low Low 13 Medium Medium Very Low Low 14 High Medium Very Low Low 15 Very High Medium Very Low Medium 16 Very Low High Very Low Low 17 Low High Very Low Medium 18 Medium High Very Low Medium 19 High High Very Low Medium 20 Very High High Very Low Medium 21 Very Low Very High Very Low Medium 22 Low Very High Very Low Medium 23 Medium Very High Very Low Medium 24 High Very High Very Low Medium 25 Very High Very High Very Low Medium 26 Very Low Very Low Low Low 27 Low Very Low Low Low 28 Medium Very Low Low Low 29 High Very Low Low Low 30 Very High Very Low Low Medium 31 Very Low Low Low Low 32 Low Low Low Medium 33 Medium Low Low Medium 34 High Low Low Medium 35 Very High Low Low Medium 36 Very Low Medium Low Medium 37 Low Medium Low Medium 38 Medium Medium Low Medium   226  Generation 3 Rules Physical No. pH Temp. Turbidity Gen 2 Outcome 39 High Medium Low Medium 40 Very High Medium Low Medium 41 Very Low High Low Medium 42 Low High Low Medium 43 Medium High Low Medium 44 High High Low Medium 45 Very High High Low Medium 46 Very Low Very High Low Medium 47 Low Very High Low Medium 48 Medium Very High Low High 49 High Very High Low High 50 Very High Very High Low High 51 Very Low Very Low Medium Medium 52 Low Very Low Medium Medium 53 Medium Very Low Medium Medium 54 High Very Low Medium Medium 55 Very High Very Low Medium Medium 56 Very Low Low Medium Medium 57 Low Low Medium Medium 58 Medium Low Medium Medium 59 High Low Medium Medium 60 Very High Low Medium Medium 61 Very Low Medium Medium Medium 62 Low Medium Medium Medium 63 Medium Medium Medium High 64 High Medium Medium High 65 Very High Medium Medium High 66 Very Low High Medium High 67 Low High Medium High 68 Medium High Medium High 69 High High Medium High 70 Very High High Medium High 71 Very Low Very High Medium High 72 Low Very High Medium High 73 Medium Very High Medium High 74 High Very High Medium High 75 Very High Very High Medium High 76 Very Low Very Low High Medium   227  Generation 3 Rules Physical No. pH Temp. Turbidity Gen 2 Outcome 77 Low Very Low High Medium 78 Medium Very Low High High 79 High Very Low High High 80 Very High Very Low High High 81 Very Low Low High High 82 Low Low High High 83 Medium Low High High 84 High Low High High 85 Very High Low High High 86 Very Low Medium High High 87 Low Medium High High 88 Medium Medium High High 89 High Medium High High 90 Very High Medium High High 91 Very Low High High High 92 Low High High High 93 Medium High High High 94 High High High Very High 95 Very High High High Very High 96 Very Low Very High High Very High 97 Low Very High High Very High 98 Medium Very High High Very High 99 High Very High High Very High 100 Very High Very High High Very High 101 Very Low Very Low Very High High 102 Low Very Low Very High High 103 Medium Very Low Very High High 104 High Very Low Very High High 105 Very High Very Low Very High High 106 Very Low Low Very High High 107 Low Low Very High High 108 Medium Low Very High High 109 High Low Very High Very High 110 Very High Low Very High Very High 111 Very Low Medium Very High Very High 112 Low Medium Very High Very High 113 Medium Medium Very High Very High 114 High Medium Very High Very High   228  Generation 3 Rules Physical No. pH Temp. Turbidity Gen 2 Outcome 115 Very High Medium Very High Very High 116 Very Low High Very High Very High 117 Low High Very High Very High 118 Medium High Very High Very High 119 High High Very High Very High 120 Very High High Very High Very High 121 Very Low Very High Very High Very High 122 Low Very High Very High Very High 123 Medium Very High Very High Very High 124 High Very High Very High Very High 125 Very High Very High Very High Very High   229  Appendix B-4 : Matrix Defining Fuzzy Rules for ‘Generation 2’ Generation 2 Rules No. Chemical Microbiological Physical Gen 1 Outcome 1 Very Low Very Low Very Low Very Low 2 Low Very Low Very Low Low 3 Medium Very Low Very Low Low 4 High Very Low Very Low Medium 5 Very High Very Low Very Low Medium 6 Very Low Low Very Low Low 7 Low Low Very Low Low 8 Medium Low Very Low Medium 9 High Low Very Low Medium 10 Very High Low Very Low Medium 11 Very Low Medium Very Low Low 12 Low Medium Very Low Medium 13 Medium Medium Very Low Medium 14 High Medium Very Low Medium 15 Very High Medium Very Low High 16 Very Low High Very Low Medium 17 Low High Very Low Medium 18 Medium High Very Low Medium 19 High High Very Low High 20 Very High High Very Low High 21 Very Low Very High Very Low Medium 22 Low Very High Very Low Medium 23 Medium Very High Very Low High 24 High Very High Very Low High 25 Very High Very High Very Low High 26 Very Low Very Low Low Low 27 Low Very Low Low Low 28 Medium Very Low Low Medium 29 High Very Low Low Medium 30 Very High Very Low Low Medium 31 Very Low Low Low Low 32 Low Low Low Medium 33 Medium Low Low Medium 34 High Low Low Medium 35 Very High Low Low High 36 Very Low Medium Low Medium 37 Low Medium Low Medium   230  Generation 2 Rules No. Chemical Microbiological Physical Gen 1 Outcome 38 Medium Medium Low Medium 39 High Medium Low High 40 Very High Medium Low High 41 Very Low High Low Medium 42 Low High Low Medium 43 Medium High Low High 44 High High Low High 45 Very High High Low High 46 Very Low Very High Low Medium 47 Low Very High Low High 48 Medium Very High Low High 49 High Very High Low High 50 Very High Very High Low Very High 51 Very Low Very Low Medium Low 52 Low Very Low Medium Medium 53 Medium Very Low Medium Medium 54 High Very Low Medium Medium 55 Very High Very Low Medium High 56 Very Low Low Medium Medium 57 Low Low Medium Medium 58 Medium Low Medium Medium 59 High Low Medium High 60 Very High Low Medium High 61 Very Low Medium Medium Medium 62 Low Medium Medium Medium 63 Medium Medium Medium High 64 High Medium Medium High 65 Very High Medium Medium High 66 Very Low High Medium Medium 67 Low High Medium High 68 Medium High Medium High 69 High High Medium High 70 Very High High Medium Very High 71 Very Low Very High Medium High 72 Low Very High Medium High 73 Medium Very High Medium High 74 High Very High Medium Very High 75 Very High Very High Medium Very High 76 Very Low Very Low High Medium   231  Generation 2 Rules No. Chemical Microbiological Physical Gen 1 Outcome 77 Low Very Low High Medium 78 Medium Very Low High Medium 79 High Very Low High High 80 Very High Very Low High High 81 Very Low Low High Medium 82 Low Low High Medium 83 Medium Low High High 84 High Low High High 85 Very High Low High High 86 Very Low Medium High Medium 87 Low Medium High High 88 Medium Medium High High 89 High Medium High High 90 Very High Medium High Very High 91 Very Low High High High 92 Low High High High 93 Medium High High High 94 High High High Very High 95 Very High High High Very High 96 Very Low Very High High High 97 Low Very High High High 98 Medium Very High High Very High 99 High Very High High Very High 100 Very High Very High High Very High 101 Very Low Very Low Very High Medium 102 Low Very Low Very High Medium 103 Medium Very Low Very High High 104 High Very Low Very High High 105 Very High Very Low Very High High 106 Very Low Low Very High Medium 107 Low Low Very High High 108 Medium Low Very High High 109 High Low Very High High 110 Very High Low Very High Very High 111 Very Low Medium Very High High 112 Low Medium Very High High 113 Medium Medium Very High High 114 High Medium Very High Very High 115 Very High Medium Very High Very High   232  Generation 2 Rules No. Chemical Microbiological Physical Gen 1 Outcome 116 Very Low High Very High High 117 Low High Very High High 118 Medium High Very High Very High 119 High High Very High Very High 120 Very High High Very High Very High 121 Very Low Very High Very High High 122 Low Very High Very High Very High 123 Medium Very High Very High Very High 124 High Very High Very High Very High 125 Very High Very High Very High Very High    233  Appendix C: CCME WQI and RWQI Inputs and Outputs  Appendix C-1 : CCME WQI and RWQI Outputs for Generation 1 and 2 (NL and QC) SDWS CCME WQI RWQI Generation 1 Generation 2 Generation 1 Generation 2 M C P M C P NL SDWS-1  50.00 39.79 51.29 72.77 7.50 5.00 5.00 5.00 NL SDWS-2  41.67 41.44 72.48 72.69 7.50 5.00 5.00 7.50 NL SDWS-3  53.70 59.16 45.04 59.12 7.50 2.50 7.50 5.43 NL SDWS-4  46.30 63.22 52.41 72.73 7.50 2.50 7.25 5.00 NL SDWS-5  67.59 37.28 17.48 36.52 9.25 5.00 9.25 9.25 NL SDWS-6  30.56 44.90 70.97 93.07 5.00 2.50 5.00 5.00 NL SDWS-7  49.07 55.26 46.78 86.39 7.50 5.00 5.00 5.00 NL SDWS-8  52.78 60.79 35.54 79.57 7.50 2.50 9.25 5.00 QC SDWS-1   50.00 39.79 51.29 72.77 7.50 5.00 5.00 5.00 QC SDWS-2  41.67 41.44 72.48 72.69 7.50 5.00 5.00 7.50 QC SDWS-3  53.70 59.16 45.04 59.12 7.50 2.50 7.50 5.43 QC SDWS-4  46.30 63.22 52.41 72.73 7.50 2.50 7.25 5.00 QC SDWS-5  67.59 37.28 17.48 36.52 9.25 5.00 9.25 9.25 QC SDWS-6  30.56 44.90 70.97 93.07 5.00 2.50 5.00 5.00 QC SDWS-7  49.07 55.26 46.78 86.39 7.50 5.00 5.00 5.00 QC SDWS-8   52.78 60.79 35.54 79.57 7.50 2.50 9.25 5.00     234  Appendix C-2 : CCME WQI Outputs for Generation 3 (NL and QC) SDWS Generation 3 -CCME WQI FRC HAA5 THM4 E.coli HPC TC pH Turbidity Temp. NL SDWS-1 43.78 51.48 58.64 100.00 0.80 100.00 38.71 100.00 79.59 NL SDWS-2 93.19 51.87 72.36 100.00 5.21 44.08 38.54 79.54 100.00 NL SDWS-3 55.13 28.34 50.30 100.00 24.56 78.60 18.26 100.00 59.11 NL SDWS-4 41.81 64.31 51.31 100.00 36.27 57.59 59.13 79.39 79.58 NL SDWS-5 11.65 27.89 13.42 100.00 17.13 43.91 17.91 13.38 79.42 NL SDWS-6 35.20 93.13 86.21 100.00 17.44 100.00 100.00 100.00 79.23 NL SDWS-7 51.24 37.57 51.55 67.39 27.39 77.98 79.59 100.00 79.57 NL SDWS-8 56.74 21.55 28.66 100.00 7.80 100.00 59.17 100.00 79.52 QC SDWS-1 61.38 100.00 100.00 100.00 31.32 100.00 38.75 100.00 79.53 QC SDWS-2 38.36 100.00 100.00 100.00 17.68 100.00 79.58 100.00 78.75 QC SDWS-3 74.96 100.00 93.19 100.00 17.68 34.61 38.66 78.91 100.00 QC SDWS-4 64.36 100.00 86.39 100.00 48.14 100.00 18.28 100.00 79.32 QC SDWS-5 44.94 100.00 93.19 100.00 31.82 100.00 17.13 100.00 79.27 QC SDWS-6 66.89 100.00 93.20 100.00 55.32 100.00 79.58 100.00 79.55 QC SDWS-7 100.00 100.00 100.00 100.00 100.00 100.00 79.59 100.00 100.00 QC SDWS-8 28.93 44.60 79.58 100.00 38.97 100.00 17.31 100.00 79.26   235  Appendix C-3 : RWQI Outputs for Generation 3 (NL and QC) SDWS Generation 3 -RWQI FRC HAA5 THM4 E.coli HPC TC pH Turbidity Temp. NL SDWS-1 2.50 5.00 5.00 0.75 9.25 0.75 5.00 2.50 7.50 NL SDWS-2 0.75 5.00 7.50 0.75 9.17 0.75 7.50 5.00 5.00 NL SDWS-3 5.00 7.50 7.50 0.75 5.00 0.75 7.50 2.50 8.22 NL SDWS-4 7.50 5.00 5.00 0.75 0.75 5.00 5.00 2.50 7.96 NL SDWS-5 9.25 5.00 9.25 0.75 7.50 5.00 9.25 7.50 9.25 NL SDWS-6 2.50 4.34 2.50 0.75 5.00 0.75 2.50 2.50 5.00 NL SDWS-7 0.75 5.44 5.91 2.50 5.00 0.75 5.00 2.50 7.50 NL SDWS-8 5.00 9.25 9.25 0.75 2.50 0.75 5.00 2.50 7.50 QC SDWS-1 5.00 0.75 2.50 0.75 5.00 0.75 7.50 0.75 7.50 QC SDWS-2 0.75 2.50 2.50 0.75 2.50 0.75 5.00 0.75 7.50 QC SDWS-3 2.50 2.50 2.50 0.75 7.50 0.75 7.50 2.50 5.00 QC SDWS-4 2.50 2.50 2.50 0.75 2.50 0.75 7.50 0.75 7.50 QC SDWS-5 5.00 2.01 3.92 0.75 7.50 0.75 9.25 0.75 7.50 QC SDWS-6 5.00 1.62 2.50 0.75 0.75 0.75 5.00 0.75 7.50 QC SDWS-7 0.75 0.75 0.75 0.75 0.75 0.75 7.50 2.50 5.00 QC SDWS-8 5.40 5.00 5.00 0.75 5.00 0.75 9.25 2.50 7.50   236  Appendix C-4 : RWQI Outputs for Generation 4 (NL) Performance Indicator NL SDWS-1 NL SDWS-2 NL SDWS-3 NL SDWS-4 NL SDWS-5 NL SDWS-6 NL SDWS-7 NL SDWS-8 Gen4-FRCAutumn 5.00 0.75 2.50 5.00 7.50 7.50 2.50 3.51 Gen4-FRCWinter 2.50 0.75 0.75 5.00 7.50 2.50 2.50 5.00 Gen4-FRCSpring 2.50 2.50 5.00 7.50 7.50 2.50 0.75 5.00 Gen4-FRCSummer 0.75 0.81 5.00 7.50 9.19 0.75 0.75 5.00 Gen4-HAA5Autumn 7.50 5.00 5.00 7.50 7.50 5.00 8.22 9.25 Gen4-HAA5Winter 7.50 5.84 9.25 5.00 9.25 2.50 5.00 9.25 Gen4-HAA5Spring 5.00 7.50 5.00 5.00 7.50 2.50 5.00 7.50 Gen4-HAA5Summer 5.00 5.00 9.22 2.50 2.50 3.66 6.10 7.50 Gen4-THM4Autumn 5.44 6.30 5.00 7.50 9.25 5.00 8.46 9.25 Gen4-THM4Winter 5.00 5.00 4.54 2.50 7.62 2.50 5.00 7.50 Gen4-THM4Spring 5.00 4.16 5.00 5.00 9.25 0.75 5.00 7.50 Gen4-THM4Summer 5.00 7.50 9.25 5.00 9.25 5.00 5.00 9.18 Gen4-EcoliAutumn 0.75 0.75 0.75 0.75 0.75 0.75 9.25 0.75 Gen4-EcoliWinter 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-EcoliSpring 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-EcoliSummer 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-HPCAutumn 9.25 9.25 9.25 9.25 9.25 9.25 9.25 9.25 Gen4-HPCWinter 9.25 9.25 1.88 0.75 2.50 1.46 9.25 0.75 Gen4-HPCSpring 9.25 9.25 7.50 2.02 0.75 2.50 0.75 0.75 Gen4-HPCSummer 9.25 9.25 0.75 2.02 9.25 9.25 2.50 0.75 Gen4-TCAutumn 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-TCWinter 0.75 9.25 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-TCSpring 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-TCSummer 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-pHAutumn 0.75 1.87 7.50 0.75 7.50 0.75 0.75 5.00 Gen4-pHWinter 7.50 7.50 7.50 3.73 7.50 5.58 7.50 7.50 Gen4-pHSpring 7.50 9.25 7.50 7.50 9.25 0.75 5.00 7.50 Gen4-pHSummer 7.50 8.21 7.50 7.50 7.50 5.00 5.58 5.00 Gen4-TurbidityAutumn 2.50 2.50 2.50 2.50 7.50 2.50 2.50 2.50 Gen4-TurbidityWinter 2.50 2.50 2.50 2.50 9.25 0.75 1.00 2.50 Gen4-TurbiditySpring 3.67 5.00 5.00 6.75 7.50 2.50 5.00 5.00 Gen4-TurbiditySummer 2.50 5.00 2.50 2.50 7.50 2.50 2.50 2.50 Gen4-TempAutumn 7.50 7.50 8.54 7.55 7.50 3.35 5.44 7.50 Gen4-TempWinter 3.83 0.75 2.50 5.84 2.50 9.25 7.50 2.50 Gen4-TempSpring 4.91 0.75 3.66 7.06 7.50 2.50 2.50 2.50 Gen4-TempSummer 7.82 7.50 9.25 8.37 9.25 7.50 9.25 9.25   237  Appendix C-5 : RWQI Outputs for Generation 4 (QC) Performance Indicator QC SDWS-1 QC SDWS-2 QC SDWS-3 QC SDWS-4 QC SDWS-5 QC SDWS-6 QC SDWS-7 QC SDWS-8 Gen4-FRCAutumn 5.00 2.50 2.50 2.50 9.25 5.11 5.11 5.00 Gen4-FRCWinter 4.73 2.50 0.75 2.50 5.00 2.50 2.50 5.00 Gen4-FRCSpring 2.50 2.50 2.50 0.81 2.18 2.50 2.50 5.00 Gen4-FRCSummer 5.00 0.78 5.62 3.56 5.00 7.50 7.50 6.21 Gen4-HAA5Autumn 2.45 2.50 2.50 2.50 2.50 2.50 2.50 5.00 Gen4-HAA5Winter 0.75 2.21 4.16 1.80 1.60 0.75 0.75 5.00 Gen4-HAA5Spring 0.75 2.43 0.75 2.50 0.75 1.36 1.36 5.00 Gen4-HAA5Summer 0.75 2.50 2.50 2.50 2.50 2.50 2.50 5.00 Gen4-THM4Autumn 5.00 2.50 5.00 2.50 3.93 5.00 5.00 5.00 Gen4-THM4Winter 0.75 2.21 2.11 1.80 2.50 1.97 1.97 2.50 Gen4-THM4Spring 0.75 2.50 0.75 2.50 1.46 2.50 2.50 2.50 Gen4-THM4Summer 2.41 5.00 3.70 5.00 5.00 2.50 2.50 5.00 Gen4-EcoliAutumn 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-EcoliWinter 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-EcoliSpring 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-EcoliSummer 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-HPCAutumn 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-HPCWinter 0.75 2.02 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-HPCSpring 2.02 0.75 0.75 0.75 0.75 0.75 0.75 2.50 Gen4-HPCSummer 7.50 0.75 9.25 2.02 9.25 2.50 2.50 9.25 Gen4-TCAutumn 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-TCWinter 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-TCSpring 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-TCSummer 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Gen4-pHAutumn 5.00 5.00 5.00 7.50 9.25 1.22 1.22 9.25 Gen4-pHWinter 7.50 7.50 7.50 7.50 9.25 5.58 5.58 9.25 Gen4-pHSpring 7.50 2.50 7.50 7.75 9.25 5.00 5.00 9.25 Gen4-pHSummer 7.50 5.58 7.50 7.50 9.25 7.50 7.50 9.25 Gen4-TurbidityAutumn 0.75 0.75 7.50 0.75 0.75 0.75 0.75 2.50 Gen4-TurbidityWinter 0.75 0.75 0.75 0.75 0.75 0.75 0.75 2.50 Gen4-TurbiditySpring 2.50 5.00 2.50 2.50 2.50 5.00 5.00 5.00 Gen4-TurbiditySummer 2.50 0.75 2.50 2.31 0.75 0.75 0.75 2.50 Gen4-TempAutumn 7.50 7.50 7.50 7.50 7.50 7.50 7.50 7.50 Gen4-TempWinter 0.75 2.50 2.50 0.75 0.75 0.75 0.75 0.75 Gen4-TempSpring 0.75 2.50 2.50 1.63 2.50 0.75 0.75 2.50 Gen4-TempSummer 9.25 9.25 7.50 9.25 9.25 9.25 9.25 9.25   238  Appendix C-6 : RWQI Inputs and Outputs for Generation 5 (NL) Performance indicator NL SDWS-1 NL SDWS-2 NL SDWS-3 NL SDWS-4 NL SDWS-5 NL SDWS-6 NL SDWS-7 NL SDWS-8 In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. Gen5-FRCAutumnR1 1.70 7.50 0.59 0.75 0.48 0.75 0.68 0.75 1.45 2.50 1.33 2.50 1.33 2.50 1.03 2.50 Gen5-FRCAutumnR2 1.16 5.00 0.51 2.50 1.31 5.00 0.24 4.25 0.13 7.50 0.16 7.50 0.93 5.00 0.29 2.50 Gen5-FRCAutumnR3 0.30 0.75 0.29 0.77 0.51 0.75 0.09 7.73 0.08 7.92 0.05 9.25 0.75 0.75 0.22 6.29 Gen5-FRCWinterR1 1.55 5.00 0.66 0.75 0.27 0.81 0.26 0.83 1.17 2.50 1.48 2.50 1.31 2.50 1.53 4.17 Gen5-FRCWinterR2 1.11 5.00 0.61 2.50 0.75 2.50 0.18 7.50 0.13 7.50 1.03 5.00 0.95 5.00 0.18 7.50 Gen5-FRCWinterR3 0.75 0.75 0.52 0.75 0.71 0.75 0.08 7.99 0.06 8.72 0.92 2.50 0.81 1.04 0.73 0.75 Gen5-FRCSpringR1 1.18 2.50 0.37 0.75 0.39 0.75 0.22 4.35 1.25 2.50 1.51 2.85 1.14 2.50 1.63 7.50 Gen5-FRCSpringR2 1.00 5.00 0.36 2.50 0.78 2.50 0.13 7.50 0.11 7.50 1.18 5.00 0.54 2.50 0.21 6.79 Gen5-FRCSpringR3 0.91 2.50 0.22 6.10 0.02 9.25 0.10 7.50 0.06 8.96 1.13 2.50 0.51 0.75 0.75 0.75 Gen5-FRCSummerR1 0.87 2.14 0.46 0.75 1.31 2.50 0.13 5.00 1.58 6.29 0.64 0.75 1.24 2.50 1.88 7.50 Gen5-FRCSummerR2 0.71 2.50 0.63 2.50 2.06 9.25 0.12 7.50 0.05 9.25 0.52 2.50 0.66 2.50 0.16 7.50 Gen5-FRCSummerR3 0.28 0.79 0.82 1.28 1.19 2.50 0.10 7.50 0.06 8.72 0.28 0.80 0.45 0.75 0.40 0.75 Gen5-HAA5AutumnR1 78.89 2.50 52.87 2.50 104.06 2.50 204.86 7.50 211.50 7.50 48.17 2.41 134.35 3.68 230.03 7.50 Gen5-HAA5AutumnR2 119.65 7.50 113.24 7.50 149.81 9.25 144.40 8.63 112.24 7.50 120.09 7.50 143.03 8.55 168.75 9.25 Gen5-HAA5AutumnR3 150.96 7.50 121.60 5.00 2.50 0.75 64.68 2.50 173.69 7.50 85.50 3.87 160.66 7.50 227.96 9.25 Gen5-HAA5WinterR1 97.29 2.50 94.93 2.50 183.70 5.71 130.22 3.24 186.28 5.87 38.52 1.94 118.91 2.50 146.99 4.76 Gen5-HAA5WinterR2 143.33 8.55 103.09 7.50 152.58 9.25 125.06 7.50 266.00 9.25 65.41 5.00 133.92 7.94 154.31 9.25 Gen5-HAA5WinterR3 165.69 7.50 141.83 6.43 174.50 7.50 29.46 1.19 225.38 9.25 66.50 2.50 124.93 5.00 216.92 9.25 Gen5-HAA5SpringR1 72.45 2.50 81.04 2.50 133.81 3.68 98.90 2.50 198.52 7.33 18.31 0.75 81.51 2.50 105.76 2.50 Gen5-HAA5SpringR2 108.83 7.50 113.78 7.50 75.24 5.00 171.53 9.25 160.81 9.25 52.50 5.00 89.11 6.13 157.46 9.25 Gen5-HAA5SpringR3 123.33 5.00 158.87 7.50 117.63 5.00 42.15 2.12 128.28 5.24 53.53 2.50 90.93 4.29 159.81 7.50 Gen5-HAA5SummerR1 78.08 2.50 75.37 2.50 188.85 6.13 95.80 2.50 183.57 5.71 47.72 2.41 120.68 2.50 150.45 5.00 Gen5-HAA5SummerR2 99.22 7.33 91.84 6.43 255.85 9.25 69.44 5.00 28.26 2.97 60.16 5.00 143.27 8.55 95.30 6.80 Gen5-HAA5SummerR3 117.47 5.00 110.36 5.00 304.88 9.25 56.85 2.50 70.32 2.50 80.63 3.36 138.90 6.13 204.62 9.25 Gen5-THM4AutumnR1 96.67 2.50 88.09 2.50 129.80 3.24 163.81 5.00 179.28 5.31 42.22 2.12 133.84 3.68 221.11 7.50   239  Performance indicator NL SDWS-1 NL SDWS-2 NL SDWS-3 NL SDWS-4 NL SDWS-5 NL SDWS-6 NL SDWS-7 NL SDWS-8 In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. Gen5-THM4AutumnR2 115.02 7.50 144.03 8.63 130.42 7.73 174.33 9.25 281.06 9.25 115.82 7.50 163.77 9.25 177.91 9.23 Gen5-THM4AutumnR3 139.85 6.23 102.03 5.00 35.57 1.75 121.24 5.00 353.18 9.25 132.26 5.55 145.20 6.76 158.50 7.50 Gen5-THM4WinterR1 92.65 2.50 74.52 2.50 127.79 2.97 79.87 2.50 129.04 3.11 35.05 1.68 84.28 2.50 90.78 2.50 Gen5-THM4WinterR2 124.17 7.50 98.11 7.17 94.76 6.76 75.11 5.00 204.68 9.25 59.40 5.00 85.54 5.87 137.07 8.12 Gen5-THM4WinterR3 118.75 5.00 105.18 5.00 78.15 2.97 75.37 2.50 171.37 7.50 59.79 2.50 95.83 5.00 150.10 7.50 Gen5-THM4SpringR1 77.72 2.50 52.60 2.50 106.98 2.50 84.93 2.50 115.24 2.50 5.82 0.75 117.73 2.50 90.62 2.50 Gen5-THM4SpringR2 98.30 7.17 57.22 5.00 75.84 5.08 126.70 7.59 150.71 9.25 5.82 2.50 86.83 5.96 166.45 9.25 Gen5-THM4SpringR3 112.51 5.00 82.50 3.57 96.41 5.00 136.67 5.96 209.20 9.25 57.12 2.50 99.15 4.92 135.36 5.79 Gen5-THM4SummerR1 97.25 2.50 69.56 2.50 210.85 7.50 95.22 2.50 173.51 5.00 51.20 2.50 116.88 2.50 134.49 3.68 Gen5-THM4SummerR2 84.06 5.71 158.24 9.25 170.74 9.25 117.94 7.50 217.44 9.25 59.83 5.00 118.94 7.50 220.06 9.25 Gen5-THM4SummerR3 145.04 6.76 96.76 4.69 214.35 9.25 114.20 5.00 208.54 9.25 61.56 7.50 121.89 5.00 208.15 9.25 Gen5-EcoliAutumnR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 3.15 7.50 0.00 0.75 Gen5-EcoliAutumnR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 3.15 9.25 0.00 2.50 Gen5-EcoliAutumnR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 3.15 7.50 0.00 0.75 Gen5-EcoliWinterR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-EcoliWinterR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-EcoliWinterR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-EcoliSpringR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-EcoliSpringR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-EcoliSpringR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-EcoliSummerR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-EcoliSummerR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-EcoliSummerR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-HPCAutumnR1 18.50 7.50 1.90 4.61 10.24 7.50 0.43 0.75 22.70 7.50 0.00 0.75 4.00 7.50 1.98 4.92 Gen5-HPCAutumnR2 18.50 9.25 1.90 6.76 10.24 9.25 0.43 2.50 22.70 9.25 0.00 2.50 4.00 9.25 1.98 7.33 Gen5-HPCAutumnR3 18.50 9.25 1.90 6.29 10.24 9.25 0.43 0.75 22.70 9.25 0.00 0.75 4.00 7.50 1.98 7.22 Gen5-HPCWinterR1 4.00 7.50 7.50 7.50 0.65 1.49 0.00 0.75 1.00 2.50 0.60 1.28 3.00 7.50 7.50 0.75   240  Performance indicator NL SDWS-1 NL SDWS-2 NL SDWS-3 NL SDWS-4 NL SDWS-5 NL SDWS-6 NL SDWS-7 NL SDWS-8 In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. Gen5-HPCWinterR2 4.00 9.25 7.50 9.25 0.65 3.52 0.00 2.50 1.00 5.00 0.60 3.24 3.00 9.25 7.50 2.50 Gen5-HPCWinterR3 4.00 7.50 7.50 9.25 0.65 1.49 0.00 0.75 1.00 2.50 0.60 1.28 3.00 7.50 7.50 0.75 Gen5-HPCSpringR1 22.67 7.50 3.00 7.50 2.00 5.00 0.67 1.57 0.00 0.75 1.00 2.50 0.00 0.75 0.67 0.75 Gen5-HPCSpringR2 22.67 9.25 3.00 9.25 2.00 7.50 0.67 3.63 0.00 2.50 1.00 5.00 0.00 2.50 0.67 2.50 Gen5-HPCSpringR3 22.67 9.25 3.00 7.50 2.00 7.50 0.67 1.57 0.00 0.75 1.00 2.50 0.00 0.75 0.67 0.75 Gen5-HPCSummerR1 115.00 7.50 7.67 7.50 0.00 0.75 0.67 1.57 93.50 7.50 92.00 7.50 1.33 2.50 1.33 0.75 Gen5-HPCSummerR2 115.00 9.25 7.67 9.25 0.00 2.50 0.67 3.63 93.50 9.25 92.00 9.25 1.33 5.00 1.33 2.50 Gen5-HPCSummerR3 115.00 9.25 7.67 9.25 0.00 0.75 0.67 1.57 93.50 9.25 92.00 9.25 1.33 2.50 1.33 0.75 Gen5-TCAutumnR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.25 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCAutumnR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.25 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-TCAutumnR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.25 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCWinterR1 0.00 0.75 36.70 7.50 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCWinterR2 0.00 2.50 36.70 9.25 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-TCWinterR3 0.00 0.75 36.70 9.25 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCSpringR1 0.00 0.75 0.00 0.75 0.50 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCSpringR2 0.00 2.50 0.00 2.50 0.50 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-TCSpringR3 0.00 0.75 0.00 0.75 0.50 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCSummerR1 0.00 0.75 0.00 0.75 0.00 0.75 7.23 7.50 7.73 7.50 0.00 0.75 0.67 1.57 0.00 0.75 Gen5-TCSummerR2 0.00 2.50 0.00 2.50 0.00 2.50 7.23 9.25 7.73 9.25 0.00 2.50 0.67 3.63 0.00 2.50 Gen5-TCSummerR3 0.00 0.75 0.00 0.75 0.00 0.75 7.23 9.25 7.73 9.25 0.00 0.75 0.67 1.57 0.00 0.75 Gen5-pHAutumnR1 7.92 0.75 7.46 1.51 6.84 5.00 7.74 0.75 6.71 5.00 8.29 0.75 7.82 0.75 7.19 3.90 Gen5-pHAutumnR2 7.92 2.50 7.46 3.04 6.84 7.50 7.74 2.50 6.71 7.50 8.29 2.50 7.82 2.50 7.19 5.40 Gen5-pHAutumnR3 7.92 0.75 7.46 1.69 6.84 7.50 7.74 0.75 6.71 7.50 8.29 0.75 7.82 0.75 7.19 5.25 Gen5-pHWinterR1 6.68 5.00 6.53 5.00 6.33 5.67 7.36 2.74 6.55 5.00 7.11 4.46 6.88 5.00 6.88 5.00 Gen5-pHWinterR2 6.68 7.50 6.53 7.50 6.33 7.91 7.36 4.08 6.55 7.50 7.11 6.29 6.88 7.50 6.88 7.50 Gen5-pHWinterR3 6.68 7.50 6.53 7.50 6.33 7.50 7.36 3.38 6.55 7.50 7.11 6.31 6.88 7.50 6.88 7.50 Gen5-pHSpringR1 6.73 5.00 5.93 7.50 6.55 5.00 6.71 5.00 4.73 7.50 7.50 0.75 7.25 3.59 6.90 5.00   241  Performance indicator NL SDWS-1 NL SDWS-2 NL SDWS-3 NL SDWS-4 NL SDWS-5 NL SDWS-6 NL SDWS-7 NL SDWS-8 In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. Gen5-pHSpringR2 6.73 7.50 5.93 9.25 6.55 7.50 6.71 7.50 4.73 9.25 7.50 2.50 7.25 5.00 6.90 7.50 Gen5-pHSpringR3 6.73 7.50 5.93 7.50 6.55 7.50 6.71 7.50 4.73 7.50 7.50 0.75 7.25 4.72 6.90 7.50 Gen5-pHSummerR1 6.41 5.35 6.14 6.53 6.49 5.04 6.40 5.39 6.41 5.39 7.32 3.24 7.11 4.46 7.17 3.90 Gen5-pHSummerR2 6.41 7.70 6.14 8.55 6.49 7.52 6.40 7.73 6.41 7.73 7.32 4.60 7.11 6.29 7.17 5.40 Gen5-pHSummerR3 6.41 7.50 6.14 7.50 6.49 7.50 6.40 7.50 6.41 7.50 7.32 4.15 7.11 6.31 7.17 5.25 Gen5-TurbidityAutumnR1 0.42 5.18 0.80 2.50 0.56 2.50 0.51 2.50 1.24 4.92 0.38 1.90 0.44 2.20 0.64 2.50 Gen5-TurbidityAutumnR2 0.42 7.60 0.80 5.00 0.56 5.00 0.51 5.00 1.24 7.33 0.38 4.06 0.44 4.49 0.64 5.00 Gen5-TurbidityAutumnR3 0.42 5.18 0.80 2.50 0.56 2.50 0.51 2.50 1.24 7.22 0.38 1.90 0.44 2.20 0.64 2.50 Gen5-TurbidityWinterR1 0.40 3.42 0.82 2.50 0.46 2.32 0.48 2.41 3.83 7.50 0.25 0.76 0.28 1.09 0.43 2.17 Gen5-TurbidityWinterR2 0.40 5.52 0.82 5.00 0.46 4.69 0.48 4.84 3.83 9.25 0.25 2.52 0.28 2.97 0.43 4.45 Gen5-TurbidityWinterR3 0.40 3.42 0.82 2.50 0.46 2.32 0.48 2.41 3.83 9.25 0.25 0.76 0.28 1.09 0.43 2.17 Gen5-TurbiditySpringR1 0.31 3.96 0.85 4.69 0.48 4.69 1.21 4.69 1.61 4.69 0.26 4.69 0.44 4.69 0.52 4.69 Gen5-TurbiditySpringR2 0.31 5.96 0.85 5.00 0.48 4.84 1.21 6.84 1.61 7.50 0.26 2.74 0.44 4.53 0.52 5.00 Gen5-TurbiditySpringR3 0.31 3.96 0.85 2.50 0.48 2.41 1.21 6.42 1.61 7.50 0.26 0.92 0.44 2.22 0.52 2.50 Gen5-TurbiditySummerR1 0.60 6.26 1.10 3.77 0.38 1.88 0.50 2.50 1.34 5.00 0.80 2.50 0.63 2.50 0.76 2.50 Gen5-TurbiditySummerR2 0.60 8.34 1.10 5.79 0.38 4.04 0.50 5.00 1.34 7.50 0.80 5.00 0.63 5.00 0.76 5.00 Gen5-TurbiditySummerR3 0.60 6.26 1.10 4.60 0.38 1.88 0.50 2.50 1.34 7.50 0.80 2.50 0.63 2.50 0.76 2.50 Gen5-TemperatureAutumnR1 13.73 2.12 12.63 5.00 15.48 6.76 14.80 6.04 11.17 5.00 8.50 3.24 9.47 4.21 13.55 5.00 Gen5-TemperatureAutumnR2 13.73 4.37 12.63 7.50 15.48 8.72 14.80 8.18 11.17 7.50 8.50 5.39 9.47 6.23 13.55 7.50 Gen5-TemperatureAutumnR3 13.73 2.12 12.63 5.00 15.48 6.76 14.80 6.04 11.17 5.00 8.50 3.24 9.47 4.21 13.55 5.00 Gen5-TemperatureWinterR1 8.66 2.01 2.20 0.75 7.42 2.50 9.67 4.37 4.95 2.48 19.25 7.50 11.05 5.00 5.73 2.50 Gen5-TemperatureWinterR2 8.66 4.21 2.20 2.50 7.42 5.00 9.67 6.43 4.95 4.96 19.25 9.25 11.05 7.50 5.73 5.00 Gen5-TemperatureWinterR3 8.66 2.01 2.20 0.75 7.42 2.50 9.67 4.37 4.95 2.48 19.25 7.50 11.05 5.00 5.73 2.50 Gen5-TemperatureSpringR1 9.20 4.69 0.30 0.75 8.60 3.36 10.10 4.69 10.33 4.84 4.70 2.37 4.45 2.25 7.67 2.50 Gen5-TemperatureSpringR2 9.20 3.36 0.30 2.50 8.60 5.47 10.10 6.89 10.33 7.17 4.70 4.76 4.45 4.57 7.67 5.00 Gen5-TemperatureSpringR3 9.20 1.37 0.30 0.75 8.60 3.36 10.10 4.69 10.33 4.84 4.70 2.37 4.45 2.25 7.67 2.50 Gen5-TemperatureSummerR1 15.03 2.50 14.30 5.63 16.90 7.50 15.40 6.64 17.87 7.50 13.33 5.00 15.80 7.17 16.77 7.50   242  Performance indicator NL SDWS-1 NL SDWS-2 NL SDWS-3 NL SDWS-4 NL SDWS-5 NL SDWS-6 NL SDWS-7 NL SDWS-8 In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. Gen5-TemperatureSummerR2 15.03 5.00 14.30 7.88 16.90 9.25 15.40 8.63 17.87 9.25 13.33 7.50 15.80 9.02 16.77 9.25 Gen5-TemperatureSummerR3 15.03 2.50 14.30 5.63 16.90 7.50 15.40 6.64 17.87 7.50 13.33 5.00 15.80 7.17 16.77 7.50   243  Appendix C-7 : RWQI Inputs and Outputs for Generation 5 (QC) Performance indicator QC SDWS-1 QC SDWS-2 QC SDWS-3 QC SDWS-4 QC SDWS-5 QC SDWS-6 QC SDWS-7 QC SDWS-8 In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. Gen5-FRCAutumnR1 0.35 0.75 1.02 2.50 0.37 0.75 0.78 0.75 0.18 5.00 0.52 0.75 0.56 0.75 1.44 2.50 Gen5-FRCAutumnR2 0.34 2.50 0.95 5.00 0.53 2.50 0.50 2.50 0.05 9.25 0.08 8.06 0.62 2.50 0.73 2.50 Gen5-FRCAutumnR3 0.03 9.25 0.58 0.75 0.21 6.83 0.14 7.50 0.03 9.25 0.23 5.16 0.55 0.75 0.02 9.25 Gen5-FRCWinterR1 0.41 0.75 1.05 2.50 0.30 0.75 1.04 2.50 0.33 0.75 0.66 0.75 0.57 0.75 1.02 2.50 Gen5-FRCWinterR2 0.41 2.50 0.94 5.00 0.26 2.50 0.87 4.35 0.15 7.50 0.14 7.50 0.57 2.50 0.38 2.50 Gen5-FRCWinterR3 0.06 8.86 0.80 0.84 0.46 0.75 0.52 0.75 0.10 7.50 0.38 0.75 0.49 0.75 0.02 9.25 Gen5-FRCSpringR1 0.40 0.75 1.07 2.50 0.49 0.75 0.84 1.68 0.45 0.75 0.67 0.75 0.58 0.75 1.29 2.50 Gen5-FRCSpringR2 0.37 2.50 0.96 5.00 0.36 2.50 0.76 2.50 0.43 2.50 0.22 6.10 0.64 2.50 0.34 2.50 Gen5-FRCSpringR3 0.20 7.30 0.89 2.39 0.08 7.99 0.30 0.75 0.24 3.73 0.49 0.75 0.60 0.75 0.04 9.25 Gen5-FRCSummerR1 0.43 0.75 0.85 1.85 0.35 0.75 0.75 0.75 0.19 5.00 0.45 0.75 0.63 0.75 1.44 2.50 Gen5-FRCSummerR2 0.40 2.50 0.73 2.50 0.07 8.32 0.37 2.50 0.10 9.25 0.04 9.25 0.62 2.50 0.21 6.59 Gen5-FRCSummerR3 0.02 9.25 0.61 0.75 0.12 7.50 0.07 8.32 0.27 0.81 0.16 7.50 0.51 0.75 0.01 9.25 Gen5-HAA5AutumnR1 39.02 1.94 71.52 2.50 54.49 2.50 72.33 2.50 57.67 2.50 47.56 2.41 6.60 0.75 177.96 5.24 Gen5-HAA5AutumnR2 37.05 3.96 76.83 5.16 84.51 5.79 46.62 4.76 59.04 5.00 52.85 4.45 6.78 2.50 136.34 8.06 Gen5-HAA5AutumnR3 18.32 0.75 77.05 2.83 77.87 2.97 45.87 2.32 52.32 2.50 52.88 2.50 7.28 0.75 18.49 0.75 Gen5-HAA5WinterR1 8.04 0.75 36.23 1.75 39.49 1.94 30.66 1.37 20.87 0.75 22.00 0.75 13.45 0.75 114.74 2.50 Gen5-HAA5WinterR2 9.40 2.50 29.63 3.24 47.74 4.84 32.49 3.47 30.64 3.36 25.30 2.50 13.98 2.50 138.09 8.18 Gen5-HAA5WinterR3 6.42 0.75 40.77 2.06 83.45 3.57 37.72 1.88 26.81 0.98 26.08 0.87 19.86 0.75 73.15 2.50 Gen5-HAA5SpringR1 13.85 0.75 32.20 1.45 18.19 0.75 31.62 1.45 15.55 0.75 21.62 0.75 13.82 0.75 122.45 2.50 Gen5-HAA5SpringR2 12.10 2.50 36.70 3.96 22.62 2.50 37.41 4.04 22.96 2.50 29.89 3.24 12.75 2.50 132.59 7.88 Gen5-HAA5SpringR3 14.06 0.75 35.15 1.68 45.07 2.27 40.00 2.01 22.02 0.75 25.33 0.75 13.86 0.75 77.15 2.83 Gen5-HAA5SummerR1 21.82 0.75 62.29 2.50 30.44 1.28 92.44 2.50 37.11 1.82 42.22 2.12 6.48 0.75 121.84 2.50 Gen5-HAA5SummerR2 20.86 2.50 68.54 5.00 40.78 4.29 49.04 4.92 54.70 5.00 42.20 4.37 6.11 2.50 120.71 7.50 Gen5-HAA5SummerR3 6.26 0.75 70.08 2.50 74.09 2.50 41.72 2.12 70.37 2.50 44.73 2.27 6.80 0.75 61.07 2.50 Gen5-THM4AutumnR1 45.47 2.27 59.31 2.50 50.09 2.50 56.37 2.50 57.99 2.50 54.60 2.50 9.62 0.75 92.87 2.50   244  Performance indicator QC SDWS-1 QC SDWS-2 QC SDWS-3 QC SDWS-4 QC SDWS-5 QC SDWS-6 QC SDWS-7 QC SDWS-8 In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. Gen5-THM4AutumnR2 46.72 4.76 61.28 5.00 57.05 5.00 61.44 5.00 75.24 5.00 100.96 7.50 9.86 2.50 102.09 7.50 Gen5-THM4AutumnR3 42.58 5.00 69.66 2.50 102.06 5.00 71.24 2.50 82.17 3.47 67.85 2.50 7.74 0.75 95.04 4.61 Gen5-THM4WinterR1 13.82 0.75 35.83 1.75 30.47 1.28 30.06 1.28 26.40 0.98 22.90 0.75 14.83 0.75 72.87 2.50 Gen5-THM4WinterR2 13.29 2.50 25.42 2.50 34.23 3.68 31.57 3.47 40.24 4.21 33.25 3.57 15.51 2.50 79.92 5.39 Gen5-THM4WinterR3 18.11 0.75 43.83 2.22 71.79 2.50 35.92 1.75 38.23 1.88 26.45 0.87 14.99 0.75 45.28 2.27 Gen5-THM4SpringR1 19.88 0.75 40.13 2.01 20.25 0.75 35.42 1.68 23.77 0.75 22.96 0.75 12.89 0.75 57.51 2.50 Gen5-THM4SpringR2 19.81 2.50 41.21 4.29 22.48 2.50 40.07 4.21 30.11 3.24 37.63 4.04 12.25 2.50 86.34 5.87 Gen5-THM4SpringR3 22.70 0.75 46.30 2.32 45.13 2.27 51.60 2.50 33.78 1.61 27.06 0.98 12.56 0.75 68.51 2.50 Gen5-THM4SummerR1 39.45 1.94 80.98 2.50 30.85 1.37 83.58 2.50 60.71 2.50 42.49 2.12 10.60 0.75 108.39 2.50 Gen5-THM4SummerR2 32.44 3.47 82.52 5.63 41.16 4.29 103.80 7.50 107.45 7.50 72.04 5.00 8.94 2.50 101.43 7.50 Gen5-THM4SummerR3 43.48 2.17 90.16 4.21 83.54 3.68 101.75 5.00 96.95 4.76 63.19 2.50 9.79 0.75 63.76 2.50 Gen5-EcoliAutumnR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-EcoliAutumnR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-EcoliAutumnR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-EcoliWinterR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-EcoliWinterR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-EcoliWinterR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-EcoliSpringR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-EcoliSpringR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-EcoliSpringR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-EcoliSummerR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-EcoliSummerR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-EcoliSummerR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-HPCAutumnR1 2.00 5.00 55.25 7.50 2.05 5.00 4.30 7.50 66.50 7.50 0.50 0.75 0.00 0.75 0.00 0.75 Gen5-HPCAutumnR2 2.00 7.50 55.25 9.25 2.05 7.50 4.30 9.25 66.50 9.25 0.50 2.50 0.00 2.50 0.00 2.50 Gen5-HPCAutumnR3 2.00 7.50 55.25 9.25 2.05 7.50 4.30 7.50 66.50 9.25 0.50 0.75 0.00 0.75 0.00 0.75 Gen5-HPCWinterR1 0.00 0.75 0.67 1.57 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75   245  Performance indicator QC SDWS-1 QC SDWS-2 QC SDWS-3 QC SDWS-4 QC SDWS-5 QC SDWS-6 QC SDWS-7 QC SDWS-8 In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. Gen5-HPCWinterR2 0.00 2.50 0.67 3.63 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-HPCWinterR3 0.00 0.75 0.67 1.57 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-HPCSpringR1 0.67 1.57 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 1.33 2.50 Gen5-HPCSpringR2 0.67 3.63 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 1.33 5.00 Gen5-HPCSpringR3 0.67 1.57 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 1.33 2.50 Gen5-HPCSummerR1 2.00 5.00 25.00 0.75 28.67 7.50 0.67 1.57 3.33 7.50 1.33 2.50 0.00 0.75 13.33 7.50 Gen5-HPCSummerR2 2.00 7.50 25.00 2.50 28.67 9.25 0.67 3.63 3.33 9.25 1.33 5.00 0.00 2.50 13.33 9.25 Gen5-HPCSummerR3 2.00 7.50 25.00 0.75 28.67 9.25 0.67 1.57 3.33 7.50 1.33 2.50 0.00 0.75 13.33 9.25 Gen5-TCAutumnR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCAutumnR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-TCAutumnR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCWinterR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCWinterR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-TCWinterR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCSpringR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCSpringR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-TCSpringR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCSummerR1 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-TCSummerR2 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 0.00 2.50 Gen5-TCSummerR3 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 0.00 0.75 Gen5-pHAutumnR1 7.24 3.90 7.23 3.90 7.16 3.90 6.56 5.00 5.12 7.50 7.48 1.16 7.03 5.00 6.01 7.50 Gen5-pHAutumnR2 7.24 5.40 7.23 5.40 7.16 5.40 6.56 7.50 5.12 9.25 7.48 2.78 7.03 7.50 6.01 9.25 Gen5-pHAutumnR3 7.24 5.25 7.23 5.25 7.16 5.25 6.56 7.50 5.12 7.50 7.48 1.24 7.03 7.50 6.01 7.50 Gen5-pHWinterR1 6.98 5.00 6.82 5.00 6.48 5.08 6.91 5.00 5.94 7.50 7.07 4.46 7.05 4.46 5.50 7.50 Gen5-pHWinterR2 6.98 7.50 6.82 7.50 6.48 7.54 6.91 7.50 5.94 9.25 7.07 6.29 7.05 6.29 5.50 9.25 Gen5-pHWinterR3 6.98 7.50 6.82 7.50 6.48 7.50 6.91 7.50 5.94 7.50 7.07 6.31 7.05 6.31 5.50 7.50 Gen5-pHSpringR1 6.81 5.00 7.35 2.33 6.28 5.79 6.23 6.09 5.15 7.50 7.31 3.24 7.05 5.00 5.02 7.50   246  Performance indicator QC SDWS-1 QC SDWS-2 QC SDWS-3 QC SDWS-4 QC SDWS-5 QC SDWS-6 QC SDWS-7 QC SDWS-8 In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. Gen5-pHSpringR2 6.81 7.50 7.35 3.71 6.28 7.99 6.23 8.21 5.15 9.25 7.31 4.60 7.05 7.50 5.02 9.25 Gen5-pHSpringR3 6.81 7.50 7.35 2.79 6.28 7.50 6.23 7.50 5.15 7.50 7.31 4.15 7.05 7.50 5.02 7.50 Gen5-pHSummerR1 6.71 5.00 7.06 4.46 6.54 5.00 6.72 5.00 5.01 7.50 6.72 5.00 6.92 5.00 5.23 7.50 Gen5-pHSummerR2 6.71 7.50 7.06 6.29 6.54 7.50 6.72 7.50 5.01 9.25 6.72 7.50 6.92 7.50 5.23 9.25 Gen5-pHSummerR3 6.71 7.50 7.06 6.31 6.54 7.50 6.72 7.50 5.01 7.50 6.72 7.50 6.92 7.50 5.23 7.50 Gen5-TurbidityAutumnR1 0.10 0.75 0.20 0.75 1.40 5.00 0.20 0.75 0.15 0.75 0.26 0.87 0.42 2.13 0.84 2.50 Gen5-TurbidityAutumnR2 0.10 2.50 0.20 2.50 1.40 7.50 0.20 2.50 0.15 2.50 0.26 2.67 0.42 4.40 0.84 5.00 Gen5-TurbidityAutumnR3 0.10 0.75 0.20 0.75 1.40 7.50 0.20 0.75 0.15 0.75 0.26 0.87 0.42 2.13 0.84 2.50 Gen5-TurbidityWinterR1 0.06 0.75 0.25 0.75 0.26 0.87 0.15 0.75 0.16 0.75 0.15 0.75 0.32 1.41 0.62 2.50 Gen5-TurbidityWinterR2 0.06 2.50 0.25 2.50 0.26 2.67 0.15 2.50 0.16 2.50 0.15 2.50 0.32 3.41 0.62 5.00 Gen5-TurbidityWinterR3 0.06 0.75 0.25 0.75 0.26 0.87 0.15 0.75 0.16 0.75 0.15 0.75 0.32 1.41 0.62 2.50 Gen5-TurbiditySpringR1 0.12 4.69 0.98 4.69 0.22 4.69 0.26 4.69 0.12 4.69 0.60 4.69 0.40 4.69 0.49 4.69 Gen5-TurbiditySpringR2 0.12 2.50 0.98 5.00 0.22 2.50 0.26 2.74 0.12 2.50 0.60 5.00 0.40 4.19 0.49 4.92 Gen5-TurbiditySpringR3 0.12 0.75 0.98 2.50 0.22 0.75 0.26 0.92 0.12 0.75 0.60 2.50 0.40 1.99 0.49 2.46 Gen5-TurbiditySummerR1 0.60 2.50 0.22 0.75 0.83 2.50 0.36 1.74 0.17 0.75 0.14 0.75 0.34 1.59 0.67 2.50 Gen5-TurbiditySummerR2 0.60 5.00 0.22 2.50 0.83 5.00 0.36 3.85 0.17 2.50 0.14 2.50 0.34 3.66 0.67 5.00 Gen5-TurbiditySummerR3 0.60 2.50 0.22 0.75 0.83 2.50 0.36 1.74 0.17 0.75 0.14 0.75 0.34 1.59 0.67 2.50 Gen5-TemperatureAutumnR1 10.80 5.00 13.63 5.08 12.10 5.00 11.78 5.00 12.43 5.00 11.65 5.00 9.68 4.37 12.30 5.00 Gen5-TemperatureAutumnR2 10.80 7.50 13.63 7.54 12.10 7.50 11.78 7.50 12.43 7.50 11.65 7.50 9.68 6.43 12.30 7.50 Gen5-TemperatureAutumnR3 10.80 5.00 13.63 5.08 12.10 5.00 11.78 5.00 12.43 5.00 11.65 5.00 9.68 4.37 12.30 5.00 Gen5-TemperatureWinterR1 1.50 0.75 4.67 2.37 4.33 2.17 1.90 0.75 2.67 0.95 1.83 0.75 3.50 1.68 2.67 0.98 Gen5-TemperatureWinterR2 1.50 2.50 4.67 4.76 4.33 4.45 1.90 2.50 2.67 2.78 1.83 2.50 3.50 3.77 2.67 2.83 Gen5-TemperatureWinterR3 1.50 0.75 4.67 2.37 4.33 2.17 1.90 0.75 2.67 0.95 1.83 0.75 3.50 1.68 2.67 0.98 Gen5-TemperatureSpringR1 1.83 0.75 6.33 2.50 5.93 2.50 3.10 1.37 4.40 2.22 2.67 0.98 7.00 2.50 6.67 2.50 Gen5-TemperatureSpringR2 1.83 2.50 6.33 5.00 5.93 5.00 3.10 3.36 4.40 4.53 2.67 2.83 7.00 5.00 6.67 5.00 Gen5-TemperatureSpringR3 1.83 0.75 6.33 2.50 5.93 2.50 3.10 1.37 4.40 2.22 2.67 0.98 7.00 2.50 6.67 2.50 Gen5-TemperatureSummerR1 16.60 7.50 21.83 7.50 12.17 5.00 18.67 7.50 19.00 7.50 16.33 7.50 11.00 5.00 19.07 7.50   247  Performance indicator QC SDWS-1 QC SDWS-2 QC SDWS-3 QC SDWS-4 QC SDWS-5 QC SDWS-6 QC SDWS-7 QC SDWS-8 In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. In. Out. Gen5-TemperatureSummerR2 16.60 9.25 21.83 9.18 12.17 7.50 18.67 9.25 19.00 9.25 16.33 9.25 11.00 7.50 19.07 9.25 Gen5-TemperatureSummerR3 16.60 7.50 21.83 7.66 12.17 5.00 18.67 7.50 19.00 7.50 16.33 7.50 11.00 5.00 19.07 7.50   

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.24.1-0347281/manifest

Comment

Related Items