Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

The development of the tools to implement evolutionary operation as an operations strategy in wastewater… Coleman, Patrick F. 1992

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


831-ubc_1992_spring_coleman_patrick_f.pdf [ 8.56MB ]
JSON: 831-1.0050493.json
JSON-LD: 831-1.0050493-ld.json
RDF/XML (Pretty): 831-1.0050493-rdf.xml
RDF/JSON: 831-1.0050493-rdf.json
Turtle: 831-1.0050493-turtle.txt
N-Triples: 831-1.0050493-rdf-ntriples.txt
Original Record: 831-1.0050493-source.json
Full Text

Full Text

THE DEVELOPMENT OF THE TOOLS TO IMPLEMENTEVOLUTIONARY OPERATION AS AN OPERATIONS STRATEGY INWASTEWATER TREATMENT PLANTSbyPATRICK F. COLEMANB.ASc. (Civil Engineering) University of British Columbia, 1982A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYinFACULTY OF GRADUATE STUDIESCIVIL ENGINEERINGWe accept this thesis as conformingto theequired standyrdTHE UNIVERSITY OF BRITISH COLUMBIADecember 1991© Patrick F. Coleman, 1991In presenting this thesis in partial fulfilment of the requirements for an advanceddegree at the University of British Columbia, I agree that the Library shall make itfreely available for reference and study. I further agree that permission for extensivecopying of this thesis for scholarly purposes may be granted by the head of mydepartment or by his or her representatives. It is understood that copying orpublication of this thesis for financial gain shall not be allowed without my writtenpermission.Department of Civil EngineeringThe University of British ColumbiaVancouver, CanadaDate DeCeIflbeI 21, 1991DE.6 (2/88)AbstractA technical operator of a wastewater treatment plant responds to changes in performancewhile a fundamentalist operator causes the performance to change. A good operator isboth, responding to maintain control and acting to optimize performance. Most operators are technicians, afraid to experiment on their systems. This state of affairs existsfor three reasons. First, management and regulatory authorities accept consistent sub-optimal performance because they do not know what the plant is capable of. Second,many treatment plants are inflexible and poorly designed because the designers have noway to evaluate their work. Third, the operator does not understand what is going onin his process because the data are unreliable, insufficient, incompatible or unavailable.Computer based solutions to performance limitations have had mixed success becauseresearchers have ignored this situation. The goal of this research is to provide a newway to look at treatment plant data that will free the operator to he both a technician and a fundamentalist. This new view is based on three paradigms. The Structureparadigm uncouples the structure of the system from the task of reasoning about thesystem. The Measurement paradigm maps all data collected on the system, qualitativeand quantitative, into a single space so that it can be analyzed as a single unit. TheOperation paradigm enables the operator to determine the effect of his actions on hisprocess. A computer program that incorporates this approach will enable the operatorto learn about his process while operating his plant.IITable of ContentsAbstract iiList of Tables xiiList of Figures xvAcknowledgement xix1 Introduction 11.1 Objectives 11.2 Summary 51.2.1 Technician Or Fundamentalist7 51.2.2 Technician By Necessity 61.2.3 Fundamentalist By Design 111.2.4 Information: The Basis For Fundamental Operation 131.2.5 Computers- Information Assistant 171.2.6 From Concept To Code: Objectives 171.2.7 Paradigms- A Frame Of Mind 181.2.8 Structure Paradigm 181.2.9 Measurement Paradigm 191.2.10 Operation Paradigm 191.2.11 Technician or Fundamentalist- The Operator Decides 201.3 Layout 21ni2 PLF’s and Computer-Based Solutions2.1 Sources Of Information2.1.1 CPE/CCP2.1.2 Audits2.1.3 Plant Experimentatiou2.1.4 Examination Of Historical Records2.1.5 Summary: Quality and Information2.1.6 Results2.1.7 Summary2.1.8 Equipping The Operator To Improve2.2 Data Analysis2.2.1 Initial Data Analysis2.2.2 Data Processing2.2.3 Data Quality2.2.4 Summary Statistics2.2.5 Relationships2.2.6 Statistical Graphs2.2.7 Advanced Statistical Methods2.3 Modeling2.3.1 Identifiability2.4 Expert SystemsIntegration: Access To Other Data .Expert: Is There One?Rule Conflict: Wait and Short Term Gain2.4.4 Are Expert Systems Needed?2.4.5 Future: The Role Of Expert System Technology232323242727282932Performance 32353535363844.56.586060697071-T722. Automatic Control.2.6 Summaryluference4 Measurement Process4.1 Measurement4.1.1 Measurement Theory4.1.2 Operational Measures4.2 Preservation4.3 Sampling4.3.1 Sampling Protocol4.3.2 Sampling Plan4.3.3 Sampling Viewpoints4.3.4 References4.4 Quality Assurance/Quality Control1101121121191221231241271361381403 Cause and Effect3.1 Temporal Reasoning3.1.1 Introduction3.1.2 Problems and Solutions3.2 Cause And Effect3.2.1 Association Versus Causation3.2.2 Effect To Cause: The Poor Cousin3.2.3 Effect To Cause - Treatment Plants3.2.4 Cause To Effect - Fundamental Problem Of Causal3.2.5 Treatment Plants and Time Series Experiments3.3 Synthesis3.4 Summary: Control Cycle7778808181828585899195100106108v6 Structure Paradigm6.1 Database Management Systems6.2 A Simple Plant6.3 Overview Of The Structure Paradigm6.3.1 Nodes6.3.2 Links6.3.3 Mapping6.3.4 Classvi4.4.1 Concern For Quality4.4.2 References4.4.3 Data Quality4.5 Ivleasurement Model4.5.1 Model Components4.5.2 Preparation Errors4.5.3 Error Correction4.6 Good Data, Good Decisions5 Fuzzy Sets, Logic and Reasoning5.1 Literature5.2 V\7hat Is Fuzziness?5.3 Notion Of Sets5.3.1 Notion Of Crisp Sets5.3.2 Notion Of Fuzzy Sets5.4 Notion Of Variables5.5 Notion Of Fuzzy Controllers5.6 Conclusions1401421431521541561591631641651661671671681761781781791811851941992002032046.3.5 Structural Relationships : Path. Loop and Network 2076.3.6 Level and Plane6.3.7 Stream and Cnrrency6.4 Structure Of Planes6.4.1 Structural Plane6.4.2 Monitoring, Diagnostic and Capacity6.4.3 Quality Assnrance Plane6.4.4 Derived Planes6.4.5 Reasoning Planes6.5 An Example: Construction Of The Structural Skeleton . .Parse In Node Information From A TextConstruct HierarchyParse In Link InformationConstrnct Link SetsBuild Interplanar LinksBuild Sample and Measurement SetsLoop DetectionPhases 8-11: Preparation For PlottingPhases 12-15: Object and Graphic SpaceValidatiow Draw A Plane6.7 Conclusion 2277 Measurement Paradigm 2287.1 Declaration Space: Origin Of A Datum7.1.1 Parameter Context6.6 Algorithms.6.6.1 Phase 1:6.6.2 Phase 2:6.6.3 Phase 3:6.6.4 Phase 4:6.6.5 Phase 5:6.6.6 Phase 6:6.6.7 Phase 7:File208209211211211213216217219223223223224224225...2252262262262266. Sample Context 2347.1.3 Measurement Context 2347.2 Data Space: Derivation Of A Datum 2387.2.1 Quality 2387.2.2 Preference 2417.3 Data Space: Internal Representation Of A Quad 2457.3.1 Manipulated Parameter: A Special Case 2477.4 Primary Mapping: Mapping A Datum Into The Data Space 2487.4.1 Mapping: {Crisp, {Ratio, Interval, Ordinal }} 2487.4.2 Mapping: {Crisp, Nominal} 2497.4.3 Mapping: {Mean/Standard Deviation, {Ratio or Interval}} . . 2517.4.4 Mapping: {Fuzzy Number, {Ratio or Interval}} 2517.4.5 Mapping: {Linguistic \/ariahle, Any Scale} 2527.5 Viewpoint 2537.6 Secondary Mapping: Quad To Series 2577.7 Tertiary Mapping: Grouping Data Series 2597.8 Summary 2618 Operation Paradigm 2648.1 What Is Status7 2668.2 What Is Change? 2708.2.1 Level 2708.2.2 Warn-Alarm . . . 2708.2.3 Limits 2728.2.4 Trend 2728.2.5 Frequency 275viii8.3 Postdiction: Why Was A Manipulated Variable Changed? 275283285286286286291294• . . . 295• . . . 2962962972973003023023043053053053083083083163168.3.1 Reason #1: Anticipate Disturbance8.3.2 Reason #2: Anticipate Performance8.3.3 Reason #3: Optimize Performance8.3.4 Reason #4: Compensate Disturbance8.3.5 Reason #5: Compensate Manipulated8.3.6 Reason #6: Counteract Performance8.3.7 Reason #7: Non-Operational Change8.3.8 Reason #8: Catastrophic Intervention8.4 Prediction: How To Change A Manipulated Variable8.5 Conclusions9 Synthesis9.1 The Relationship Between The Program and The Operator . .9.2 Construction Of A Simple Example9.3 Information Generating System9.3.1 Treatment Process Module9.3.2 Information Gathering Process Module9.3.3 Adjustable Parameters9.3.4 Simulation Scenario9.3.5 Detection Of Change9.3.6 Detection and Response To Change9.4 Simulation: Coping With A Bulking Sludge9.4.1 March 17, 19919.4.2 March 27, 19919.4.3 April 22, 1991ix9.4.4 May ‘20, 1991. 3189.4.5 Postscript 31810 Conclusion: From Box’s EVOP to Evolutionary Operation 322Appendices 324A Abbreviations And Copyrights 324B Glossary 326B.1 Introduction 326B.2 Items 326C Uncertainty In The MCRT and F/M ratio 365C.1 Goal 365C.2 Result 365C.3 Derivation 366D Sludge Quality 374D.1 Plant Observations 375D.1.1 Foam On Surface Of Aeration Basin 375D.1.2 Secondary Clarifier 376D.2 Microscopic Examination 378D.2.1 Morphology 378D.2.2 Size 378D.2.3 Composition 378D.2.4 Protozoa 379D.3 30 Ivlinute Settling Test 381xD.3.1 Floc Formation 381D.3.2 Blanket Formation 81D.3.3 Settling Velocity 381D.4 Laboratory Measurements 383D.4.1 Analytical Measurements 383D.4.2 Derived Measures 384E Structure Paradigm Example 385E.1 Database Schema 385Bibliography 391xiList of Tables2.1 Performance Limiting Factors As Identified By The CPE/CCP Program 292.2 DMR QA Study 2-Average Failure Rate 332.3 Mean, Median, Mode - Common Location Estimators 412.4 Common Width Estimators - Single Values 422.5 Skewness and Kurtosis - Order Statistics 452.6 Correlation Example: Correlation Analysis and Outlier Detection . . 462.7 Holmberg’s Batch Reactor Equations 623.1 Statistical Solution- Analysis Of Variance . 983.2 Time Series Experiment- Cause Archetypes . 1013.3 Time Series Experiment - Effect Archetypes . 1023.4 Mixed Culture/Mixed Substrate Interactions. 1034.1 Measurement Scales 1164.2 Appropriate Statistical Operations 1174.3 Ratio And Interval Temperature Scales . . 1184.4 Effectiveness Of ATU As Effluent BUD5 Sample Nitrification Inhibitor 1224.5 Outline Of A Sampling Protocol 1254.6 Green’s Sampling Design Principles 1264.7 Sample Selection Methodology 1314.8 DMR QA/PESP Effectiveness 1424.9 BOD and Suspended Solids Quality Assurance Results: % Unacceptable 1454.10 Potassium Determination In An Agricultural Laboratory 148xii4.11 Sample Plan and Type Selection Based On The Quality Fluctuation Error 1626.• 2242252492522542562562622702772872872872892899.1 Observed MeasurementsA.1 Copyrights2932933063245.1 Properties Of Crisp Set OperationsPortion Of An IndexPhase 1: Text File SyntaxPhase 2: Section Numbers and Node RelationshipsPhase 3: Text File SyntaxMeasure Paradigm: Effluent COD ExampleLinguistic Variable: Clarifier Condition Is PoorCalculate Exploratory StatisticsTime Series Viewpoint : Display Decision Table7.5 Differenced Time Series Viewpoint : Series Manipulation Table .7.6 Measure Paradigm: Effluent COD SummaryForms Of ChangeWhy Change A Manipulated Variable?Rule For Reason #1: State Of Anticipated DisturbanceRule For Reason #1: State Of Common PerformanceRule For Reason #2: State Of Anticipated PerformanceRule For Reason #3: State Of PerformanceRule For Reason #4: State Of Initiating Disturbance .8.8 Rules For Reason #48.9 Rule For Reason #6:State Of Common Performance Variables.State Of Common PerformancexniA.2 List Of Abbreviations 325D.1 375D.2 376D.3 378D.4 378D.5 379D.6 380D.7 38tD.8 . . . . 38tD.9 382383Plant Observations - Aeration Basin FoamPlant Observations - Secondary ClarifierMicroscopic Examination - Floc MorphologyMicroscopic Examination- Floc SizeMicroscopic Examination - Floc CompositionMicroscopic Examination- Protozoa30 Minute Settling Test- Floc Formation30 Minute Settling Test - Blanket Formation30 Minute Settling Test- Settling VelocityD.10 Laboratory Measurements - 1982 DMR-QA ResultsxivList of Figures1.1 From Concept To Code: Program Development 41.2 Datum As Information 141.3 Elements Of Information: Datum, Series and Relationship . . . 152.1 Statistical Sample Size: {Small, Medium, Large} . . . 342.2 Correlation Example: Filament Length vs SVI 472.3 Anscombe’s Quartet 552.4 Ljung’s System Identification Loop . . 633.1 Scale-up : Volume to Surface Area Ratio Contours 873.2 Hysteresis 933.3 Scientific Solution 973.4 Statistical Solution 994.1 Samples Taken On Linsley Pond 1284.2 Cy’s Sampling Error Model 1554.3 Example Audit Procedure 1615.1 Butterfly Cluster 1715.2 Butterfly Cluster : Fuzzy Clustering 1725.3 Representations Of Uncertainty 1776.1 Book Index Schema 1846.2 Simple Plant 187xvLP.3: Primary Treatment PlaneLP.i: Secondary Treatment PlantLP.i.3: BioreactorOverlap Of Classes Of InformationLinkInfluence MapPMS Plane Owned By Structural Node LP.1LP..3: BioreactorLP.1 Influent: QA/QC Plane For CompositeGraphic SpaceObject Space8.1 Derived Time Series: Level 2676.5 LP.: Treatment Plant Plane1881896.3 Outline Of Plant6.4 Simple Plant: Bird’s Eye View Of A Hierarchical Network .190191192193196201205212214Measure. 215221222Influent6. and CODMeasure Paradigm: Parameter ContextModel Context: An ExampleMeasurement Paradigm: Sample Context . .Measurement Paradigm: Measurement ContextPrimary Mapping: Datum To Data Space .Quality Distribution For COD TestPreference Distribution For COD TestMeasure Paradigm: Effluent COD ExampleSecondary MappingTertiary Mapping231234235236239243244250258260xvi8.2 Derived Time Series: Stability 2688.3 Simple Direction Algorithm 2698.4 Change In Level 2718.5 Change In Trend 2748.6 Change In Frequency 2768.7 Why A Manipulated Parameter Is Changed 2798.8 Stability And Control Actions 2808.9 Causal and Noncausal Change 2818.10 Reason #1: Anticipate A Change In A Disturbance Parameter 2848.11 Reason #2: Anticipate A Change In A Performance Parameter 2858.12 Reason #3: Optimize A Performance Parameter 2888.13 Reason #4: Respond To A Change In A Disturbance Parameter 2908.14 Reason #5: Compensate For A Change In A Manipulated Parameter 2918.15 Reason #6: Compensate For A Change In A Performance Parameter 2928.16 Reason #7: Correct An Operational Error 2949.1 Synthesis: Information Generation and Interpretation 2999.2 Simple Example: Program Layout 3019.3 Simulated Plant’s Flow Sheet 3039.4 Simulation Scenario: Tbie SVI Value 3079.5 Control Cycle: Wastewater Teatment Plant 3099.6 Daily Trend 3109.7 Stability 3119.8 Weekly Trend: Disturbance. Performance and Adjustable Parameters 3129.9 Weekly Trend: Status Parameters 3139.10 Reason #2: Anticipate A Change In A Performance Parameter 315xvii9.11 3179.12 3199.13 3209.14 321Bi. .B.2 . .C.1 . . . . 370C.2 . . . . 371C.3 . . . . 372C.4 . . . . 373Uncontrollable SystemSystem RecoveryEffluent CODWastage Rate ControlSimple Box and Whisker PlotAge Stem and Leaf Plot33036295% Confidence Intervals For MCRTMCRT Range Of Insignificance95% Confidence Intervals For F/M RatioF/M Ratio Range Of InsignificancexviiiAcknowledgementI owe a deep debt of gratitude to the many people who assisted in the completion of thiswork. My greatest debt is to my snpervisor, Dr. W. K. Oldham, who was extremelypatient and supportive through my many mistakes, wanderings and false starts. I amalso grateful for the assistance I received from the members of my committee, my fellowgraduate students, computing services and interlibrary loan staff. I am especially gratefulfor the support given to me by Fred Koch, Susan Liptak, Dave Wareham, Timo Peraand Kerry Seale.The concepts in this thesis began with my stay at the Penticton Pollution ControlCenter. I am grateful to the operator, Bernie Udala, and the staff for taking the time toteach me how to operate a plant. I also acknowledge the financial support provided bythe Natural Sciences and Engineering Research Council of Canada.Without the support of friends, family, church and most important, my wife Myrna,this thesis could never have been completed. This thesis is dedicated to my father whotaught me that a university edncation is a privilege. AMDGxixChapter 1IntroductionThe man who knows what question to ask is on the verge of understanding;the man who is beginning to understand what he does not know is not farfrom knowledge.This chapter consists of three sections: Objectives, Summary and Layout. The purpose ofthese sections is to introduce the thesis material and provide an overview of the structureof this document.1.1 ObjectivesSuccessful control of a wastewater treatment plant is less dependent on plant configuration or sophisticated and expensive equipment than on the operator’s ability to recognizeand influence causal relationships in his plant. For this reason, operators are comingunder increasing pressure to improve or maintain their plant’s effluent quality despiteincreasing flows and shrinking budgets. The operator, therefore, needs a new tool whichwill assist him in improving and interpreting his monitoring data so that he can improvehis plant’s performance. The goal of this thesis is to develop the logical paradigms for acomputer program which could become such a tool.Computers are already being used in the wastewater industry. However, much oftheir function is information storage. This accumulation of monitoring data solely as ahistorical record is a luxury few municipalities can afford. However, with reallocationof some of these computing resources to data analysis, the benefit accrued from each1Abba Issac, Fifth Century1Chapter 1. Introduction 9monitoring dollar would significantly increase. Or a.s Lord Rutherford once said, “Wehaven’t the money, so we’ve got to think”.A new partnership between the operator and the computer must be struck if theoperator is to take on a more aggressive role in operating his plant. This new partnershiprequires new paradigms that gnide the operator in his search for optimal performance.For this reason, this research concentrates on the specification of an appropriate toolrather than the development of a working program. The thesis objectives are defined asfollows:1. Define the Concept of Computer-Assisted EVOP: Explain how expressingprocess information in a cause/effect framework enables the operator to improveboth his information gathering aud treatment process.2. Design the Logical Structure: Develop a set of paradigms that guide the operator and the computer program in organizing the data to best elucidate causalpatterns in the data.3. Demonstrate the Paradigms’ Use: Explain how an operator uses these paradigmsto decide how and wheu to respond to a process change.Figure 1.1 outlines the evolution of a concept to a computer program [319] [206]. Thefirst thesis objective, concept definition, initiates the design process. The definition’scontent provides the basis for the development of a logically correct program. The secondobjective, paradigm development, carries this evolution to level 2: Logical Structure.From then on, the evolution becomes more of an exercise in Computer Science than oneof Environmental Engineering.Implicit in this model is the notion that a de-evolution must take place as well. Thegoal of top-down planning (level 1 5) is to maintain logical consistency (i.e. avoidChapter 1 Introduction 3“bugs”) while the goal of bottom-up planning (level 5 1) is to maintain feasibility(i.e. can the machine do it). In order to ensure that it was feasible to implement theseparadigms as part of a computer program, a number of test programs were written aspart of this research. These programs form the basis of the examples in Chapters 6 and 9.Chapter 1. IntroductionProgramStructureProgramStructureState SpaceAnd ModulesEPTcD4OverviewLogicalStructureData ActivitiesStrategic StrategicOverview OverviewOf Corporate Of CorporateData FunctionsDetailed LogicalLogical RelationshipData AmongModel ProcessesProgram-Level OverallView Of ProgramData StructureProgram DetailedUsage Of ProgramData LogicData FunctionStructure DesignDesignLevel 1Level 2Level 3LevelLevel 5Figure 1.1: From Concept To Code: Program DevelopmentChapter 1. Introduction1.2 SummaryAn operator would prefer to influence and control his process, rather than just react toit. The goal of this thesis is to provide the operator with a tool which could assist himin this decision-making process.1.2.1 Technician Or Fundamentalist?A good operator acts as a fundamentalist or a technician depending on the situationin his plant at a given time. A fundamentalist is an author of cause (i.e. acts) whilea technician is a reader of effects (i.e. reacts). As a technician, an operator assumesthat he does not need to know (or cannot determine) the cause of an effect. Instead,he responds to changes in effects by adjusting his plant’s manipulated parameters, control. As a fundamentalist, an operator exploits causal relationships withinthe plant by changing a manipulated parameter to offset the effect of a change in adisturbance parameter, e.g. feed-forward control.A Technician: Reader Of EffectsThe technical approach is based on three premises;1. A cause worth worrying about will have a detectable effect on the process.2. An operator detects change by monitoring changes in an effect’s quality, preferenceand value.3. An operator can reverse a change in an effect by changing a manipulated parameter,i.e. process is elastic.This approach fails when the change in the system makes the process uncontrollable,i.e. the change in the manipulated parameter no longer affects the process.Chapter 1. Introduction 6A Fundamentalist: Author Of CauseA fundamentalist operates on a different set of premises:1. A change in performance is due to either a change in a disturbance parameter oran operator manipulated parameter, i.e. process is controllable.2. A change in a disturbance can be detected or anticipated early enough that theoperator can act to offset it, i.e. feed-forward control.3. The cause and effect relationships in the plant are known. i.e. an operator knowswhat to change and how to change it.4. An operator knows when the system reaches a stable state, i.e. steady state 2 isdeterminable.This approach fails when the operator’s model of the system fails, i.e. the process responds differently than expected.1.2.2 Technician By NecessityBy necessity, most operators are technicians. There are three reasons for this:1. Management and regulatory authorities are generally willing to accept consistentsuboptimal performance, as long as regulatory demands are met, i.e. .satisfices [168].2To avoid confusion with the microbiological definition of Steady State, the term Stable State will beused to describe the situation when the components of interest in a system stabilize around a constantvalue.Chapter 1. Introduction 72. Wastewater treatment plants often are either inflexibly or poorly designed.3. The operator does not understand what is going on in his process.What Will Management Accept?Operators generally aim for consistent operation rather than optimal performance. Thereason management and regulatory authorities settle for this situation is that they do notunderstand what the treatment plant is capable of doing. There is no easy way for them to“tap” into the plant and monitor its performance. It is very costly in time and resourcesto collect and examine the information required when one uses an optimizing strategy toarrive at a decision. A consequence of the information age is that most managers sufferfrom severe time pressures and information overload [265]. In other words, a manager is“so busy manning the fire hose that he cannot devise a fire prevention program” [168].The tendency is “to go with” the first solution that works (i.e. satisfices) rather thansearch for a solution that is optimal. For this reason, management settles for adherenceto a numerical standard rather than evidence that the operator is making careful andinformed judgements about the plant’s operation.Management needs a method of viewing the treatment plant data that will organize,value and filter information so that a manager is able to understand what is happeningat the plant. A manager and an operator should be able to view the same screen (or atleast the same data in the same way) and discuss what they see.How Does One Learn How To Design A Plant?The one conclusion that can be drawn from the EPA’s Comprehensive Performance Evaluation/Composite Correction Program (CPE/CCP) and numerous other field studies3The CPE/CCP program is discussed in Section 2.1.1Chapter 1. Introduction 8and experiments is that there is not enough communication between those who operateplants and those who design them. This lack of communication has led to the construction of plants based on theories that field studies have shown to be unreliable. This iswhy the CPE/CCP concluded that many plants are over-designed and that design decisions are often the source of operational problems. It is an onerous task for a designer toreturn to the plant and sort through the log book, data files and maintenance databasein the hope of evaluating his design. Neither the client or the designer have the resourcesto fund such an evaluation.The cost is high because of the way the data are stored. Most of the cost associatedwith an evaluation is due to collecting and re-entering the data so that it is in a formatthat the designer understands and the computer can manipulate. If the data were alreadyin such a format, the operator could send the data to the designer on a disk and thedesigner could analyze the data at his office.Why Don’t Operators Know What Is Going On?One of the leading Performance Limiting Factors (PLF) identified during the EPA’sCPE/CCP was operator ignorance. This PLF invariably was cited along with staffingand design problems. The EPA felt that this PLF was too ambiguous because it did notidentify why the operator appeared ignorant to the evaluator. One explanation was thatthe evaluator expected the operator to be a fundamentalist while the conditions in theplant forced the operator to be a technician. This situation occurs for three reasons:• Data Quality: The data the operator receives from the laboratory is often notreliable. Due to inadequate equipment, training or just poor quality control, mosttreatment data is of poor quality [50] [215] [164] [244]. The uncertainty of controlvariables such as MCRT increase as the data quality deteriorates. As well, poorChapter 1. Introduction 9data limits statistical analyses to descriptive statistics and increases the data sizerequirements for inferential statistics. It also renders models non-identifiable onthe system and makes automatic control impossible. In other words, “Data of poorquality are a pollutant of clear thinking and rational decision making” [164, p. 870].• Data Coverage: The operator does not receive enough data to make an informeddecision about “what is causing what”. There are three reasons why the data donot “cover” the process:1. Data is collected hut is either not recorded or recorded but not consideredpart of the process’s data set. Among this missing data is information on personnel, energy usage, sludge management, maintenance, equipment failuresand operating costs [237]. Given that personnel, power and chemical expenditures can account for 80% of a typical plant’s operating costs [89] [253], theseexpenditures should be analyzed.2. The operator relies on qualitative observations and intuition because he lacksthe resources for on-line instrumentation or additional laboratory work. Thisinformation is usually not part of the monitoring data set due to its categoricalor linguistic nature, i.e. data apartheid.3. Data may be collected at different frequencies or not collected at all. Thiscreates a problem because most statistical methods require data pairs, notjust data. Therefore, the loss of a single datum can mean that all the datacollected at that time is also lost to the analysis. In other words, “The bestthing that can be said about missing data is- don’t have any!” [264, p. 188].• Data Apartheid: Data Apartheid occurs when two or more data sets, collected onthe same system at the same time, are stored, analyzed and interpreted separatelyChapter 1. Introduction 10simply because of their form. For example, the EPA CPE/CCP identified thatplant performance was affected by a number of factors:— scheduling of preventive maintenance— equipment failure— staffing levels— quality assurance tests on on-]ine samplers and instruments— industrial wastes— sludge processing streams— in-plant pumping cycles— abnormal influent events (e.g. storms)Because of their form, these data are often spread across log books and databases,or, in some cases, not recorded at all.Operators can manage information if the information is in a digitally compatibleform. Converting information from one form to another is far more time consuming thananalyzing that same information. Once the information is in one place, the operator cancross check to ensure that (1) the information is reliable (i.e. of good quality) and (2) hehas collected all the information he needs (i.e. the data covers his process). The need tomanage information such that its use results in decisions being made wisely is not uniqueto wastewater treatment plant operators. Grace Hopper, throughout her career, has4Rear Adm. Grace Hopper USN Retired, now in her 80’s, is a pioneer in the development of standardized application programming languages, including COBOL. She continues to act as consultant tothe Digital Equipment CorporationChapter 1. Introduction 11argued that any organization that relies on information must put into place a systemthat ensures that the required information is collected, that it’s quality is maintainedand its value is known [161, p. 170]:We have a raw material that is called data. We feed it into a process. In thiscase, the process consists of hardware, software, communications, and trainedpeople. Hopefully, the output product is information. Equally hopefully, thisprocess is under some form of control and there’s a feedback ioop from theinformation to the control to improve the quality of the information.1.2.3 Fundamentalist By DesignIn 1922, Abel Wolman argued that the wastewater treatment profession would benefitgreatly if treatment plant operators applied the Scientific Method to their routine operation and management decisions [313]. Wolman realized that (1) if operators improvedthe performance of their plants through systematic assessment of the plants’ design andoperation and (2) if the design profession had access to these assessments, then boththe design and operation of treatment plants would improve. As a technical community,the wastewater industry has been slow to follow Wolman’s advice [228] 6 and the lack ofcommunication between operators and engineers continues to hurt the industry [30].Since Wolman’s article was published, advances in statistics have provided a new basisfor the application of the Scientific Method to the operation of wastewater treatmentplants. Statistical Sampling Theory provides guidance on how to design an effectivemonitoring program while Statistical Quality Control enables the operator to monitorthe quality of the data collected by this program.The Statistical Design Of Experiments provides the operator with guidance on how to5Abel Wolinan (1893-1989) was one of the founders of the Water Pollution Control Federation.6The Engineering Profession as a whole has been slow to adopt the use of statistical methods. Forexample, one conclusion of the Post-Challenger Evaluation was that “NASA is not adequately staffedwith specialists and engineers trained in the statistical sciences to aid in the transformation of complexdata into information useful to decision makers, and for use in setting standards and goals” [148]Chapter 1. Introduction 12operate his plant to both treat the waste and provide information on how to improve boththe plant’s operation and design. This theory has been applied to treatment plants intwo ways: Plant Experimentation (PLEX) and Evolutionary Operation (EVOP). PLEXinvolves executing one or more experiments in a limited amount of time to learn moreabout the process. PLEX usually involves “getting in and getting out”. A plant experiment is usually carried out when there is additional personnel to closely monitor theprocess and take corrective action at the slightest warning of an upset [143].EVOP is less intrusive and more long term [58]. EVOP is a sequential form ofexperimentation conducted in a treatment plant during normal operation. The principaltheses of EVOP are that knowledge to improve the process should be obtained along witha product and that designed experiments using relatively small shifts in factor levels canyield this knowledge at minimum cost. The range of variation of the factors for anyone EVOP experiment is usually quite small in order to avoid upsetting the process.However, the smaller the change, the smaller the experimental error must he if the effectof the change is to be detected. The smaller the experimental error, the more replicatesare required [6]. The more replicates, the longer the experiment and the more dangerthat something else has changed, e.g. the weather.The North American economy has been hurt by the Engineering Profession’s reluctance to integrate these statistical methods into the day-to-day operation of industry [94][175]. On the other hand, many foreign economies, especially that of Japan, have grownpartly because of their emphasis on statistical methods. They have managed to improvethe quality of their products by improving the design and operation of their processes.When coupled with a sound understanding of the fundamentals of a process, the useof statistical methods increases creativity and improves both productivity and quality.A similar change must come about in the wastewater treatment profession if Wolman’sdream is to become reality. Operators, managers and designers will all benefit onceChapter 1. Introduction 13operators integrate plant experimentation and process fundamentals into their day-today operational decisions. Both the design and operation of plants will improve as thisinformation filters out from each plant into the profession as a whole.1.2.4 Information: The Basis For Fundamental OperationThe goal of a fundamentalist is to lead rather than be led by his process. For this reason,an operator must (1) understand how his process functions and (2) know his process’scurrent state. The last requirement, the need for timely and reliable information, iscritical to an operator’s success. Information is data whose origin and derivation isknown, i.e. where it comes from and how it got here. In this thesis, information is definedwith respect to a datum, a data series or a relationship among data series (Figure 1.3).The origin and derivation of a datum provide the datum with meaning (Figure 1.2),i.e. context. In the case of wastewater treatment plants, a datum’s context derivesfrom where in the system the datum originates (i.e. structural context), how the datumis obtained on the system (i.e. measurement context) and how the operator uses thedatum to make a decision (i.e. operational context).Chapter i. mtroductjo11 14DatumDeclarationOriginStructureMeasureOperationDeni tionDerivationTimePreferenceQualityValueFigure 1.2: Datum As TnformationChapter 1. Introduction 15Time (Or Space)SeriesDatum Datum Datum Datinn Datnm DatumIRelationship4SeriesDatiam Datum Datum DatumDaturj DatmFigure 1.3: Elements Of Information: Datum, Series and RelationshipChapter 1. Introduction 16The origin and derivation of a datum also provide a datum with value (Figure 1.2).For example, Grace Hopper argues that a datum is valuable to an operator if it enableshim to make a control decision [161, p. 169-170].For a couple of decades now, I’ve beenì asking people how they value theirinformation. I haven’t received any answers but I have received a great assortment of blank stares. . . . We have totally failed to consider the criteriafor the value of information. We haven’t even defined onr criteria. And yetwe must know something about the value of the information and data we areprocessing. I think we must create several priorities: the time you have to acton the data and the number of lives and dollars at stake. But there’s anotherone - the importance of that piece of information in making decisions.In the race for faster computers and more powerful software, Hopper argues that theindustry has lost sight of what it is doing with data. All data are not of equal value andthe allocation of monitoring, QA/QC and data analysis resources should reflect this fact.A datum’s worth is described, in part, by its preference and quality. Preferencedescribes how the operator feels about a datum. For example, an effluent COD concentration that is above the permit level would be less preferable than one below. Qualitydescribes how reliable the datum is. For example, a datum derived from what later turnsout to be a contaminated sample is of little use to the operator. Good data are accurate,precise, complete, representative and comparable.A data series describes the history of a characteristic with respect to time (i.e. timeseries) or space (i.e. space series). The history describes change: change in level, changein trend and change in stability. A data series should cover the interval of interest, i.e.describe the important dynamics of a characteristic.A set of data series describe the history of a system, i.e. a treatment plant. The set isinformative if location of the series in the process and the role of the series in the decisionmaking process is known. A set of data series should be comprehensive, i.e. describeanything of importance that might impact on the plant.Chapter 1. Introduction 17An operator will intervene when he knows he can make a difference. The operatorresponds to changes in his information, first by ruling out any explanation other than aprocess change and second, by acting on the information provided. For this reason, anoperator needs more than just data, he needs information- data with meaning and value.If an operator is to do more than just react to change in order to maintain the statusquo, he must be in control of the process that provides him with information. He mustdesign his monitoring program carefully to ensure that (1) he receives the information herequires and (2) he can separate changes in the system from problems in the monitoringprogram. The consequence of not taking control is “infolock”, a situation where anoperator is so busy processing the data that he has no time to analyze it [265].1.2.5 Computers- Information AssistantResearchers who design computer-based solutions to overcome performance limitationsmay be divided into two groups: those who seek to replace the operator and thosewho seek to utilize the operator. This thesis takes the latter approach. The operator’spresence is our insurance against the unpredictable (e.g. toxic spill) and the inevitable(e.g. equipment failure) . In this case, the computer assumes responsibility for tasksthe operator either dislikes doing or does not do well. This frees the operator to do whathe does best- operate his treatment plant.1.2.6 From Concept To Code: ObjectivesThe goal of this thesis is to develop a tool that will enable an operator to implement anEVOP-like operations strategy. The evolution of a concept to a program involves both abottom-up and top-down approach to design (Figure 1.1). The goal of top-down planning7Guariso and Werthner provide a convincing argument of this viewpoint with respect to expertsystems [131]Chapter 1. Introduction 18is to maintain logical consistency (i.e. avoid “bugs”) while the goal of bottom-np planningis to maintain feasibility (i.e. machine can do it,). The tension of design is to translatean organic concept into a digital one. The tension exists because humans and computers“think” differently. For example, imagine the difficulty of the task of translating a workwritten in a language with a vocabulary of over 100,000 words into one with less than 200.This is exactly what happens when a client describes a concept to a programmer andthe programmer translates the concept into a computer program . This man-machinecompromise is what makes the design of an EVOP-like tool so difficult.1.2.7 Paradigms - A Frame Of MindThe success of computer-based solutions has been limited for at least three reasons:• The solutions usually take advantage of only a fraction of the available information.• The solutions usually ignore both the quality of the data and the measurementcharacteristics of the data.• The solutions usually do not take full advantage of the operator’s presence in theplant.In order to deal with these limitations, three paradigms are developed: Structure,Measurement and Operation.1.2.8 Structure ParadigmThe aim of the structure paradigm (Chapter 6) is to construct a physical and causalnetwork of a wastewater treatment plant in both the memory of the operator and the5The Intel ©8086/88 chip has an instruction set of less than 200 triples9A paradigm is a model or pattern that organizes knowledge about a subject, explains phenomena,and serves as the basis for what measurements to take [77).Chapter 1. Introduction 19machine. An operator uses the network to guide his reasoning while the machine uses themodel to construct relationships between parameters. The computer uses the physicalnetwork to follow the passage of mass and the causal network to establish cause and effectrelationships. The causal network, constructed from the physical and influence structureof the plant, forms the skeleton onto which all other knowledge is huug. This knowledgeincludes information on how the process works and how information on the process isobtained.The structural network consists of nodes and links, classed into planes, classes, andstreams. A plane is a level of abstractiou, a class is a form of knowledge and a streamis a the form of flow that passes through the links. The paradigm places information incontext of the system’s structure fixing it in space, time and process continuum. Thestructure forms the basis of communication between the different forms of informationcollected on the process.1.2.9 Measurement ParadigmThe goal of the measurement paradigm (Chapter 7) is to establish a datum’s contextand value (Figure 1.2). The paradigm accomplishes this by mapping information into anumber of spaces. A space is a predefined logical structure that formalizes the meaning ofa datum. The spaces define the meaning of a datum at the datum, series and relationshiplevel.1.2.10 Operation ParadigmThe goal of the operation paradigm is to link a past change to a change now, and achange now to a change in the future. For example, the operator decreases the wastagebecause he detects ammonia in the effluent (no nitrification), in the hope that he willChapter 1. Introduction 20see these ammonia levels drop in the future. These linkages enable the operator to learnfrom his actions.1.2.11 Technician or Fundamentalist- The Operator DecidesIt is clear . . . that in each plant, no matter how small, no matter how crude,phenomena of great importance and of peculiar significance are occurring andrecurring. They are not always observed and still less often reported. It isa special plea . . . that this condition be remedied. for with its remedy, perhaps, many of both scientist and practical people will avoid voyages “boundnowhere, under full sail” [313. p. 14].Over sixty years have passed since Abel Wolman argued that each plant, if operatedscientifically, was an untapped reservoir of process knowledge. This thesis will be counteda success if it moves the wastewater industry one step closer to realizing Wolman’s dream.Chapter 1. Introduction 211.3 LayoutThe thesis is divided into four parts:• PLF’s And Computer-Based Solutions: Chapter 2 consists of two parts: (1)a review of what limits performance and (2) a discussion of problems with existingcomputer-based solutions to these limitations. The goal of this chapter is to showthat the computer’s role is that of an assistant to, rather than a replacement for,the operator.• Theory: Chapters 3, 4, 5 present theory necessary to overcome many of the limitations identified in Chapter 2:— The focus of Chapter 3, “Cause and Effect”, is on how to determine thecause of an effect in wastewater treatment plant where temporal assumptionsmay change, a number of causes may be confounded and the operator mayintervene.— The focus of Chapter 4, “Measurement Process”, is on how the process ofobtaining information on a characteristic affects the detection of cause andeffect relationships.— The focus of Chapter 5, “Fuzzy Sets,Logic and Reasoning”, is on the representation of data that are not crisp numbers.• Paradigm: Chapters 6, 7 and 8 provide a logical basis for an EVOP-hke tool:— The goal of Chapter 6, “Structure Paradigm”, is to reduce the structure ofthe process to a hierarchical network.— The goal of Chapter 7, “Measurement Paradigm”, is to establish the meaningof a datum and a data series.Chapter 1. Introduction 22— The goal of Chapter 8, “Operation Paradigm”, is to establish the operationalmeaning of the relationships among the data series, i.e. cause and effectviewpoint.• Synthesis: Chapter 9 uses a simple example to demonstrate how these paradigmsassist the operator to run his plant.• Conclusions And Recommendations: The last chapter summarizes the research findings and provides suggestions for further research.Chapter 2PLF’s and Computer-Based SolutionsIt is a great fault of descriptive poetry to describe everything1.The purpose of this chapter is to discuss what limits treatment plant performance andhow computer-based solutions can ameliorate the problem. The intent of this chapteris to place this research in context with other research in the area of computer-basedsolutions to performance limitations.2.1 Sources Of InformationThe information in the literature on what limits plant performance comes from four typesof studies:1. Comprehensive Performance Evaluation/Composite Correction Program (CPE/CCP)2. Audits3. Plant Experimentation4. Review Of Historical Records2.1.1 CPE/CCPThe EPA’s Comprehensive Performance Evaluation and Composite Correction Programconsists of two steps: (1) Evaluation Phase and (2) Performance Improvement Phase1Alexander Pope23Chapter 2. PLF’s and Computer-Based Solutions 24[125] [126] [139] [140] [145] [252] [40] [138] [251]. The evaluation phase is a thoroughreview and analysis of a treatment plant’s design capabilities and associated administration, operation and maintenance practices. The performance improvement phase isa systematic approach to eliminating those factors that limit performance in existingtreatment plants.2.1.2 AuditsAn audit involves the comprehensive monitoring of a part of a treatment plant over a shorttime in order to characterize its behaviour. All audits share at least four characteristics:1. An audit is not an experiment.2. An audit involves additional staff and resources not normally available to the operator.3. An audit is (usually) short-term.4. An audit is plant-specific.An audit suffers from the same drawbacks as all short-term observational studies. Forthis reason, if an auditor recommends a change to the operator based on the audit’sresults, the operator should be careful to introduce one change at a time to ensure thatbeneath a short term benefit there is not a long term disaster.Resource AuditA resource audit monitors how a resource is allocated for use in the treatment plant,e.g energy or labour. For example, Steiger et al [282] conducted an energy audit atthe Jackson Pike Treatment Plant 2 The auditors suggested a number of changes that2The reader should refer to [239] [112j and [17] for more information on energy auditsChapter 2. PLF’s and Computer-Based Solutions 25would reduce the plant’s consumption of energy including the use of hot water boilersin lieu of steam boilers, the institution of regular cleaning of air socks, screen and filtersand the elimination of leaks in the air mains. Given that personnel, energy, chemicalexpenditures and sludge disposal account for at least 80% of a typical treatment plant’soperating budget [89], these parameters shonld be audited on a regular basis if they arenot being monitored by the operator.Design or Research AuditsAudits are an effective way to evaluate design assumptions. For example, most clarifier models are based on the assumption that activated sludge flocs settle according toKynch ‘s Theory Of Thickening. However, at least two researchers confirmed that this isnot the case. Laikari [191] compared the settling velocities obtained from a conventionalcylinder test with those obtained in a clarifier during stable operation and concluded thatcylinder test results are misleading. This deviation probably explains the high degree ofvariability Morris et al [216] observed in the Vesilind coefficients (k, V0) on a day-to-daybasis in three Burlington, Vermont, completely mixed activated sludge (CMAS) plants.Most clarifier models operate in, at most, two dimensions and assume that the hydraulic and mass transfer characteristics of a clarifier are fixed. Again, field studies showthat this is not the case. Crosby [83] conducted a series of innovative dye tests on sevendifferent clariflers in seven different plants. Based on these tests, Crosby concluded thatboth the hydraulic and mass transfer characteristics of a clarifier depend on the clarifier’sdesign and the plant’s operation. For example, Crosby found that clarifier performancewas affected by such things as clarifier depth, influent flow distribution, baffles, weirs,sludge rakes and blanket height. This level of complexity is not represented in any existing clarifier model.Chapter 2. FLE’s and Computer-Based Solutions 26The focus of both Crosby’s investigations of seven American plants and Tendaj-Xavierand Hnltgren’s investigations [291] at the Bromma STP (Stockholm) was operationalstability. Both researchers argue that depth along with inlet, underfiow and overflowdesign are critical to stable operation. They also suggest that the design should bechecked and adjusted using dye tests once the clarifier is on-line.Crosby also observed that clarifier performance is sensitive to changes in the hydraulic loading. A change induces a small amplitude wave in the clarifier increasing theturbulence in the settler. Retention time is not related to the time for a flow change topropagate through a plant unless flow equalizing storage volume intervenes. The mainfactor controlling the speed with which a flow change propagates is the small amplitudewave speed: c = /jh where g is the gravitational acceleration and h is the water depth.Crosby estimates a change in the flow into the Orange County Sand Lake Plant wouldtake about 4 minutes to propagate through the plant. However, this dynamic dies outwithin one hydraulic retention time [216].Process AuditA process audit is a “stepping-up” of the monitoring program to define the time or spacevariation of a number of plant characteristics to determine if an operational or designchange is in order [286] [280]. For this reason, process audits usually are used duringstart-up studies or to evaluate the operation of older, often overloaded, treatment plants.For example, Stephenson et al [286] conducted an audit of the Cambridge (Hespeler)Water Pollution Control Plant and identified air flow disturbances and pump stationinduced hydraulic disturbances as causes of problems in the secondary clarifier. TheCPE documentation describes how to design, execute and interpret process audits.Chapter 2. PLF’s and Computer-Based Solutions 27Performance AuditIn some cases. an audit is necessary to provide an independent evaluation of some aspectof the treatment plant. For example, regulatory authorities may audit a laboratoryas part of their quality assurance program [107] [223]. Similarly, legal authorities mayrequire an independent assessment of piece of equipment to settle a dispute between aclient and an equipment supplier.2.1.3 Plant ExperimentationPlant Experimentation (PLEX) involves executing one or more experiments in a limitedamount of time to learn more about the process. Treatment plants with parallel streamsenable the operator to experiment with one stream and use the second stream as acontrol. For example, Curran et al [86] conducted a stress test on one stream in a plantto evaluate its nitrification and clarification capacity. Experimentation on a single streamis also possible. Poole and Biol [245] modified the Mold Treatment Plant (Wales) froma completely mix system to a plug flow system with an anoxic zone. They supplementedtheir experimental work with plant simulations to help them understand the diurnalvariation in nitrification.2.1.4 Examination Of Historical RecordsA researcher cannot make sense of a plant’s historical records without discussing theircontents with the plant staff. Historical records are often incomplete, fragmented and,in some cases, open to misinterpretation. This information may be found in log books,data files and work orders. Berthouex et al [46] went through this process when theyreviewed the operations records of 15 well-operated plants .:3A well-operated plant is a plant that meets its discharge permit using conventional control methods.Chapter 2, PLE’s and Computer-Based Solutions 28One difficulty in working with historical data is judging its accuracy andprecision. Making quality assurance tests today can only show that the analytical work is being done properly or improperly today. It cannot prove thatpast work was of the same quality. The quality of the data used was assessedby meeting with plant chemists and operators, seeing how samples were collected, stored and analyzed, reviewing analytical procedures, and learningwhat quality assurance methods were being used. From this kind of on-siteevaluation, an impression was formed that the data were reliable.In their study, a typical plant was upset 9% of the time and an average upset lastedabout 3.5 days. Together, low DO, high flow, low MLSS, or solids handling problems,caused 60% of these upsets.2.1.5 Summary: Quality and InformationSources must be evaluated in light of their quality, coverage and completeness. Theadvantage of historical data over data from the other three sources is that it is long term.The disadvantage is that the researcher has no control over the process by which its wasgathered. This is a situation that this thesis hopes to help change as well as one that theEPA has expended a great deal of effort to improve (see Section 4.4).Data collected as part of an experiment are easier to interpret than those collectedas part of an audit. For this reason, PLEX data are preferred over CPE/CCP and auditdata. CPE/CCP data are preferred over audit data because the CPE/CCP program includes numerous follow-up studies that ensure that the auditor’s diagnosis of the problemis correct.Chapter 2. PLF’s and Computer-Based Solutions 29Table 2.1: Performance Limiting Factors As Identified By The CPE/CCP ProgramPLF Code Description Of Performance Limiting FactorA Poor understanding and application of process control by operatorB Staffing (too few staff, low pay, turnover, etc.)C Support from municipality (administrative and technical)D Operating Budget and user charge systemE Operability and maintainability considerations (process flexibility, automation, standby units, etc.)F Infiltration and InflowG Construction ProblemsH Process design errors (clariflers, aerators, disinfection, etc.)I Over designJ Under DesignK Solids handling and sludge disposalL Pretreatment, industrial discharges and toxicslvi Operation and maintenance manualN Preventive maintenance program0 Spare parts inventoryP Chemical inventoryQ Laboratory capability for process / NP DES testingR NPDES reportingS Equipment/Unit process broken down or inoperableT Hydraulic overloadU Poor aeration system2.1.6 ResultsThe CPE/CCP EPA studies identified 21 factors that limit a plant’s performance (Table 2.1). These Performance Limiting Factors (PLF’s) can be sorted into one of sixgroups [14]:1. OperatorThe plant’s performance is limited because the operator does not understand hisprocess and/or is unable to use his knowledge to control his process. Because thisChapter 2. PLF’s and Computer-Based Solutions 30PLF did not identifywhy the operator had problems, the EPA feels in retrospectthat this PLF was too ambiguous to be useful [14]. This PLF is often cited with“staffing problems” and “design errors”. This indicates that “operator ignorauce”may sometimes be a symptom of a problem with the plant’s design or management.2. Management, Staff and BudgetsMany PLF’s are due to staffing, management and budget problems. Inadequatestaffing, high staff turnover, non-supportive management, and under funding canall cause suhoptimal plant performance, especially in small plants (< 10 mgd) [288][88] [14].3. Plant Design and ConstructionInflexible or poorly designed/constructed plants do not perform. In some cases,audits can identify the design weaknesses and modifications can be made to theprocess. This is particularly true of aeration and pumping systems. For example,Crosby [83] suggests that a leading cause of clarifier problems is due to hydraulicdisturbances caused by pump cycles. With new pumps and/or controls, this problem can be eliminated.Chapter 2. FLF’s and Computer-Based Solutions 314. Influent CharacteristicsA treatment plant can cope with changes in the influent if the operator can preventthese changes from destabilizing the process, i.e. wash ont the solids. If a problemis acnte and is a serious threat to the plant’s operation, then the problem must bedealt with before that flow enters the plant. For example, in a community where astorm may cause a large increase in flow into the plant, the municipality can use acomputer to divert flow past the plant and/or store flow in the sewerage system toenable the plant to treat as much flow as possible without being overloaded [11].Apart from storm events and infiltration, the predominant cause of problem influents is industrial wastes. Wetzel and Scott [308] observed that 80% of the plantsreporting operation and maintenance problems received industrial wastes. Theseproblems included corrosion, flow obstruction, process upsets and fires. Wetzel argues that industrial wastes are the main cause of permit violations and ongoingplant upsets..5. Preventive Maintenance and Proper OperationEquipment failure is a common PLF. Berthouex et al [46] cited mechanical failure,power interruptions, maintenance activities and system modifications as commoncauses of upsets in well-operated plants. Sensor failure is a common problem inhighly automated plants. A 1984 survey showed that a Georgia Plant had only 20sensors out 700 functioning properly, a Hunstville plant had only 1 sensor in 40working and the Washington DC sanitary commission reported that their sensorswere down 70% of the time [246].Chapter 2. PLF’s and Computer-Based Solutions 326. LaboratoryTo produce good data, a plant needs knowledgeable staff and adeqnate equipment.The issue of data quality is discnssed in detail in Section 4.4. In the mid 1970’s,the EPA realized that the data collected by wastewater treatment plants was verypoor. In response to this finding, the EPA launched a number of programs toimprove the quality of data in treatment plants. Table 2.2 contains a summaryof the 1982 DMR-QA Study 2 [64] [244]. Only 42% of the 7500 treatment plantsstudied passed all the tests.2.1.7 SummaryNo one factor limits performance more than lack of information. The problem is not thatany of the above situations is without a solution, rather the problem is that operators,management and designers lack the confidence to act because they lack information. Inother words, they need evidence that there is a problem and an indication of what thatproblem is. The most cost effective way to overcome this lack of information is to improvethe reliability of the historical record.2.1.8 Equipping The Operator To Improve PerformanceA computer excels at well-defined and tedious tasks. For example, operators currentlyuse computers to keep records, do repetitive calculations, draw graphs, print reports,log data collected on-line, manage preventive maintenance programs, track costs, andcommunicate with other computers [115] [232] [227] [246] [234] [236] [269].The cost of CPU power and storage has dropped considerably giving treatment plantsaccess to much more powerful machiues. As software development usually lags hardwaredevelopment, much of the ongoing research concentrates on developing software to helpChapter 2. PLF’s and Computer-Based Solutions 33Table 2.2: DMR QA Study 2-Average Failure RateParameter % FailurepH 14.1Total Suspended Solids 17.1Oil and Grease 23.1Nutrients 36.9Ammonia-N 37.8Nitrate-N 43.2Kjeldahl-N 29.8Total Phosphorus 37.8Ortho-Phosphorus 34.1Demands 19.7Chemical Oxygen Demauds 25.8Total Organic Carbon 17.0Biochemical Oxygen Demand 17.9Metals 20.8Plants That Failed All Tests 21.6Plants With At Least One Failure 58.0National Discharge Monitoring Report (DMR) Quality Assurance Program (QA) 1982 involved 7500plants [244]. A plant failed if its determination was more than two standard deviations from the averagedetermination of 100 EPA and state laboratories.Chapter 2. PLF’s and Computer-Based Solutions 34the operator run his plant. The following discussion of this new software is broken intofour parts: data analysis, modeling, expert systems and automatic control.atistical Sample Size:Small: N < 30Medium: 30 < N < 100Large: N> 100Figure 2.1: Statistical Sample Size: {Small, Medium, Large}Chapter 2. PLF’s and Computer-Based Solutions 352.2 Data AnalysisThe two areas of statistics that impact the most on treatment plants are Initial DataAnalysis and Statistical Quality Control as they suit the analysis of small data sets(Figure 2.1).2.2.1 Initial Data AnalysisInitial Data Analysis(IDA) [71] [73] or Exploratory Data Analysis (FDA) [72] [217] [294][264] [98] describes the first phase of a statistical analysis and inclndes the following steps:1. Processing of data into a suitable form for analysis.2. Checking data quality.3. Calculating simple descriptive statistics.4. Preparing graphs.2.2.2 Data ProcessingThe goal of this step is to determine the structure of the data. The structure consistsof two components: the system on which the data are collected and the measurementprocess used to collect the data.The number of measures taken on a parameter and the number of parameters determine what types of analysis can be conducted on the data set. Chatfield [73] warns thatany model-fitting is likely to be unreliable if the sample size is less than ten. In a treatment plant, ten data points may represent ten days of data. For this reason, a researcher4lnferential statistics often involves the derivation of a model to which the statisticians fit the data.For example, if the population is normally distributed, then the 95% confidence intervals can be constructed from a sample using the t-distribution.Chapter 2. PLE’s and Computer-Based Solutions 36must be careful not to “over-analyze” the data [50] [44] [45], i.e. use a statistical methodthat the data do not support.The number of parameters is also important. For example, if an analyst is given iiparameters and knows nothing about the structure, then he must look at (n(n — 1))/2relationships. Moreover, the analysis of two parameters is significantly easier than theanalysis of three parameters. If the analyst knows the layout of the system, he may beable to simplify the analysis considerably, i.e. parameter A cannot affect parameter Bbecause there is no pipe that connects their unit processes. If the number of parametersexceeds the number of data points for a given parameter, then multivariate techniquesshould not be used.The type of measure and parameter also affects subsequent analyses. For example,if a measure is ordinal then the mean should not be used. The relationship betweenmeasurement scales and arithmetic manipulations is discussed in Section 4.1. Similarly,the analyst needs to know which parameters are causes or factors and which ones areeffects or responses. This topic is discussed in Chapter Data QualityData are collected from the laboratory, from the plant and on-line. These data pointsmust somehow be entered into a database. Missing values and non-numerical data mustbe coded in such a way that the computer can distinguish them from numerical data .The value of a data set rapidly diminishes as the number of missing values increases.The first impact of a missing datum is loss of power. If the datum forms part of a record(i.e. x,y pair), the record is ignored (i.e. if x is missing, the analysis cannot use y). Ifmissing data form part of a intervention analysis or an experiment, the loss can make5This is a problem if the software does not allow the user some method of distinguishing an emptyfield from a zero. dBASE IV and Lotus 123 suffer from this limitation while BMDP does not.Chapter 2. PLE’s and Computer-Based Solutions 37analysis of the data impossible. Most statistical analyses can work around a few missingvalues but even this has some problems [198]. The best advice with respect to missingdata remains, don’t have any [264].The operator does have one advantage over the statistician- he knows the origin of hisdata. For example, if two measures (e.g. COD and BOD) estimate the same parametere.g. Substrate), the operator can exploit the relationship between the two measures (e.g.use a COD value to derive an estimate for a missing BOD value). Similarly, if a valuefor an operator-set value (i.e. manipulated parameter) is missing, the operator can safelyassume that the value has not changed since it was last set.An outlier may occur due to an error in the measurement process (therefore can beignored) or from a fault in the process (therefore indicates a problem). An operator cantell the difference only if he has a quality assurance/quality control program in place andsoftware that checks the data as it is entered. For example, if the operator notices thatmost of the parameters measured on the same sample are outliers, he can go check ifthere is a problem with the sampler.One purpose of the measurement paradigm described in Chapter 7 is to help theoperator exploit his knowledge of his plant and his monitoring program to overcomesome of the problems associated with missing values and outliers. Other data qualityissues are discussed in Section 4.4.Chapter 2. PLF’s and Computer-Based Solutions 382.2.4 Summary StatisticsThe initial examination of a data set uses robust estimators 6 of population parameters.A robust estimator is one that is insensitive to small departures from the assumptionsfrom which it is derived. The population parameters of interest include location, width,shape and association.LocationA location statistic estimates where the “center” of a population lies, i.e. fixes its locationon a number line. The operator should examine the characteristics of his data set in lightof what he knows about his process in order to determine which estimator to use:• The number of data points in the sample: The different location statisticsgive similar values if the sample size is large but perform with varying degrees ofaccuracy if the sample size is small [12]. For example, if the data set is small, themedian is a more efficient estimator of a population’s mean than is the arithmeticaverage [302].• The distribution of the sample values with respect to magnitude: Locationstatistics perform about the same if the distribution is symmetric and unimodalbut perform differently if this is not the case. For example, given a process witha limit cycle and an upset condition, the median will provide the best estimate ofthe expected performance because it is insensitive to the upset data points [255].6An estimator is a method that estimates the value of a particular population parameter. For example,assume we have a normally distributed population (p, c). The median, mean, mode and ADA are allestimators of p.Chapter 2. PLF’s and Computer-Based Solutions 39• The distribution of sample values over time: Most estimates weight eachsample equally. However, if the sample is taken to estimate an interval’s expectedvalue and the sampling frequency varies, the analyst may have to apply a differentweight to each datum. For example, assume a manager asks the operator to providehim with the average annual influent COD concentration. If the operator measuredthe influent COD daily during the summer (when it was strongest) but only weeklyduring the winter (when it was the weakest), then the arithmetic average wouldover-estimate the influent COD concentration. Instead, the operator should weighteach COD value by the interval it represents and calculate the weighted average.An ideal (robust) estimator exhibits four characteristics:1. Restricts the influence of any fixed fraction of wrong observations on the value ofthe estimate.2. Rejects outliers which are too far away from the bulk of the data.3. Limits the effect of rounding, grouping and other local inaccuracies.4. Performs well under ideal conditions.The three most commonly used location statistics are the mean, mode and median(Table 2.3). The arithmetic mean is a linear combination of sample values, the median isan order statistic and the mode is an occurrence statistic. This is important because thetype of statistic determines the type of data to which the estimator can be applied (seeSection se:meth). The Princeton Monte Carlo Project [12] evaluated these and 65 otherlocation statistics using these criteria. The Project found that the median performedbetter than the mean or mode when the sample size was small and the population’sChapter 2. PLF’s and Computer-Based Solutions 40distribution was unimodal but asymmetric. This is why the Interlaboratory QualityAssurance Program (International Joint Commission) decided to use the median insteadof the mean in their program [16].The mean is not a robust estimator. The Project observed that “...the mean is ahorrible estimator, except under strict normal distribution; it gets rapidly worse evenunder mild deviations from normality, and at some distance from the Gaussian distribution, it is totally disastrous ...“ [12, p. 244]. The mean is difficult to interpret if thedistribution of the data is not known.Despite these problems, the mean is still the most commonly used location statistic.There are at least five reasons for this:1. Application or tradition requires it, e.g. permit written in terms of the averagemonthly discharge.2. Cost of redoing analysis programs is prohibitive, e.g. Lotus 123 © has a functionfor mean but not median or mode.3. Majority of inference statistics use the mean, e.g. Analysis Of Variance.4. Mean is easy to calculate as it is a linear combination of the data, e.g. the meancan be calculated recursively while the median and mode cannot.5. Some populations of data are indeed symmetric and unimodal, e.g. instrumenterror 8The mean should not be used when reviewing data, at least, not by itself. Instead,the median should be used for non-categorical data and the mode, accompanied by7The Project found that the best location statistic for small sample sizes (n < 10) was the Adaptive-MEstimator (ADA). This estimator is described in the glossary (Appendix B).8Unless a standard is near the detection limit of an instrument, repeated measurement of a standardshould result in a Gaussian distribution.Chapter 2. PLF’s and Computer-Based Solutions 41Table 2.3: Mean, Median, Mode- Common Location EstimatorsMeanArithmetica Arithncetic = L1 p(xj)xj where p(Xj) = i/riGeometricb Geometric = Vl1=iHarmonic° XHarmonic = n/(ZL1)xj ifnisoddMedianifnisevenMod& Most frequent value or interval.a If p(xj) 1/n, then we refer to the mean as being a weighted mean.The geometric mean is used when the variable changes at a rate proportional toitself, i.e growth rates [255]. The geometric mean should he used with data with aratio scale.C The harmonic mean is used when the observations of what we wish to expresswith the arithmetic mean are given in inverse proportion, i.e. mean velocity overa proportion of road or mean lifetime [255] . The harmonic mean should he usedwith data with a ratio The mode of a non-categorical measure is an interval rather than a singleton. Thenumber of groups (intervals), m, should satisfy the criterion [134]: n 2”.occurrence, for categorical data. The operator should have no problem interpreting thesestatistics because of their straight-forward definition and robust behaviour. The moreesoteric and efficient estimates, such as ADA, should not be used because they are difficultto calculate and equally hard to explain. The mean shonid be reserved for those caseswhere it is required or the data support it.Chapter 2. PLF’s and Computer-Based Solutions 42Table 2.4: Common Width Estimators - Single ValnesVariance= {z1 (x )2] —1)Absolute Mean Deviation MD= [Z (I x — )J /nAbsolute Median Deviation D = median {I x —Quartile Rangea QR= Qi —a: Quartiles are values that divide a set of observations into 4 equal parts. The values, denoted by Qi,Q2, and Qs, are such that 25% of the data falls below Q, 50% falls below Q2, and 75% falls belowQ [303]. Q2 is the median.WidthA width statistic measures the variation of the samples around the center (of a unimodaldistribution). A location estimate, e.g. mean, should always be accompanied by (atleast) a width statistic and a sample size. Table 2.4 lists four commonly used widthestimators.The most commonly used width statistic is the variance (or standard deviation). Thevariance is calculated using the mean. Therefore, it performs poorly either when thesample size is small and/or the distribution is skewed. The interpretation of the varianceChapter 2. PLF’s and Computer-Based Solutions 43is dependent on the distribution being normal [2641:If the signal is Gaussian, the limit of guaranteed detection is centered at sixstandard deviations from the mean of the background noise. At this point,only 0.13% of the distributions overlap. However, if the background distribution is not Gaussian, this probability (as given by Tschebyscheff inequality)can be as high as 11%.The Project [193] observed that few dispersion estimators work well when the samplesize is below 20. The reason for this is two-fold: (1) most width statistics are calculatedusing location statistics and (2) most width statistics involve the calculation of differences.For example, assume that the mass of solids at the start of an experiment is given byx and the mass at the end by y. Assume that the standard deviation for the mass ofsolids is given by a and a respectively. If the destruction of the solids is given byx—y, then the standard deviation = + u. In other words, the uncertaintyof a sample-based variance is higher than a sample-based mean because the variance iscalculated using a difference that involves the mean.A more effective approach to describing a sample’s dispersion (for small sample sizes)is to provide a histogram that shows how the sample is distributed. An alternative for noncategorical data to the histogram is Tukey s Five Number Summary which consists of themaximum, upper quartile, median, lower quartile, and minimum. A sixth number, samplesize, should also be provided. The Box and Whisker Plot is a graphical representation ofthis summary. Given that most categorical measures in a treatment plant have no morethan ten categories, the best approach would be to provide a histogram for categoricalmeasures and Tukey’s summary for the non-categorical measures.Chapter 2. PLE’s and Computer-Based Solutions 44ShapeA given distribution may deviate from a Normal distribution in three ways:1. The distribution develops another mode.2. One of the two tails lengthens and the distribution skews: The skewness is positiveif the right side skews and negative if the left side skews.3. The maximum occurrence is higher or lower than that of a normal distribution:If the maximum is higher, the distribution is peaked or exhibits positive kurtosiswhile if the maximum is lower, the distribution is flat or exhibits negative kurtosis.Table 2.5 lists three shape statistics that use order statistics. These statistics work bestif the sample size is over 20 and the distribution is unimodal. A more effective way toexamine the shape frequency distribution of a small sample is to use a histogram or astem and leaf plot.2.2.5 RelationshipsThe detection, definition and determination of relationships among data sets are fundamental to treatment plant operation and design. An operator detects the possibilityof a relationship primarily through plotting the data. If the relationship is linear, theoperator may confirm his findings by calculating a correlation coefficient and reasoningwhether such a relationship can exist.Correlation analysis should only be done after the data has been plotted, i.e. screened.For example, Sadalagekar et al [256] determined the SVI and filament length for a numberof activated sludge samples and calculated the Pearson correlation coefficient to be 0.95.However, a plot of their data shows that most of the SVI data points are under 125 mL/g(Figure 2.2). If Tukey’s outlier detection method [41] is applied to screen out the outliers,Chapter 2. PLF’s and Computer-Based Solutions 45Table 2.5: Skewness and Knrtosis - Order StatisticsSkewness II S = DZ9+DZ1-2Skewness II ranges between —1 and 1.Skewness III S111 = Q3+Qa—2Skewness III ranges between —1 and 1.Knrtosis K= 2(DZ9—1)For a normal distrihntion, K = 0.263There exists 3 values which partition a frequency distribution in to 4 equal parts. The central valueis the median, fl; the other two are designated the lower or first quartile, Qi and the upper or thirdquartile, Q3. Three-quarters of the samples are less than the first quartile while one quarter of thesamples are less than the third quartile. The decile, DZ divides the distribution into ten parts. 90%of the values lie beneath the first decile, DZ1 while decile, DZ9. These statistics can only be appliedto non-categorical scales [255].Chapter 2. PLE’s and Computer-Based Solutions 46Table 2.6: Correlation Example: Correlation Analysis and Outlier DetectionCase SVI Pearson’s Coefficient Spearman’s CoefficientRange Calculated Critical Calculated CriticalAll Data (n=30) (500,9.2) 0.95 0.36 0.97 0.36No Extreme Outliers (n=28) (180,9.2) 0.82 0.37 0.83 0.38No Outliers (n=25) (125,9.2) 0.50 0.40 0.74 0.40Critical values is a = 0.05, two-sided test [255, pp. 398 and 425]the correlation coefficient drops considerably (Table 2.6) and the range of interest shrinks.The same pattern is observed if the more robust but less powerful Spearman coefficientis used. Apart from data entry, no part of statistics is as tedious and yet as importantas screening the data before an analysis.In order to construct a scatter plot or to calculate a correlation coefficient, eachdata point on the x-axis (abscissa) must he paired with a single datum on the y-axis(ordinate). This implies that all parameters must be measured at the same time andfrequency, which, from a wastewater treatment plant operator’s viewpoint, is ludicrous.The alternative is to match the series on the basis of the least common intervaL Forexample, assume an operator wishes to plot DO vs MLSS, and DO vs effluent solidsconcentration (TS). The operator obtains the three measures as follows:1. The operator collects the DO probe readings over an hour, and calculates theirmean and standard deviation, i.e one datum every hour.2. The operator takes a grab sample of the mixed liquor every 12 hours aud conductsa single solids test, i.e. one datum every 12 hours.3. The operator uses an automatic sample to form a 24 hour composite sample of theeffluent. The operator condncts a solids test (TS) on this sample, i.e. one datumevery 24 hours.Chapter 2. PLF’s and Computer-Based Solutions 47Filament Length vs SVITukey’s Outlier Detection400,, 350OUTLIER EXTREME OUTLIER300<250_‘ 200a)150R=0.95 All Data Used2 100R=0.82 No Extreme Outlierstt 50 R=0.53 No Outliers0 I I I I I I0 50 100 150 200 250 300 350 400 450 500SVI (mUg)Figure 2.2: Correlation Example: Filament Length vs SVIChapter 2. PLF’s and Computer-Based Solutions 48The least common interval between the DO and MLSS is 12 honrs. Therefore, theoperator pools the DO data to form the best estimate of the 12 honr interval and plotsit against a single MLSS valne. The least common interval between the MLSS andeffluent TS is also 12 hours, not 24 hours. The TS is a physical (versus statistical)24 hour average and is the best estimate of the TS concentration over both 12 hourintervals in a day. If the TS had resulted from a grab sample, the least common intervalwould have been 24 hours. The construction of these derived series is not an exercise astatistician would feel comfortable with. The primary difference between a statisticianand an operator is that a statistician analyzes data while an operator analyses his plant.In other words, an operator is able to make compromises during the analysis of thedata based on his knowledge of his process. This is another manifestation of the ResultEvaluation Principle.What an operator regresses depends on what an operator is looking for. The operatorgroups parameters to determine particular data characteristics. In wastewater treatment,parameters are most commonly grouped to address concerns arising from the monitoring program (PMS), the quality assurance program (QA/QC), the passage of materialthrough the plant and operating decisions:Parameter-Measure-Sample: A datum originates from a measure conducted ona sample to estimate a parameter. A number of measures may be taken on a sample(e.g. COD, Solids and Total Phosphorus taken on the influent composite sample)while a number of measures may he taken to estimate a parameter (e.g. CODand BOD5 estimate substrate concentration). These relationships are formalizedin the PMS plane discussed in Chapter 7. An operator can make at least threecomparisons using this grouping:— An operator may compare two measures (e.g. COD, BOD5) of the sameChapter 2. PLF’s and Computer-Based Solutions 49parameter (e.g. Influent substrate) if he suspects one of the measures or heneeds to replace a missing data point.— An operator may compare two parameters (e.g. influent and effluent Fe) toensure that one parameter is consistently less than the other (e.g. not addingtoo much pickle liquor).— An operator may compare two measures taken on the same sample if he suspects the sample is contaminated, i.e. all values that day appear to be out ofline.• Quality Assurance/Quality Control: A datum’s measure may be comparedagainst a standard or a more reliable measure while a datum’s sample may bechecked against an independent sample. Koopman et al [187] discuss an examplewhere a plant’s inability to meet its discharge permit was traced, using these typesof comparisons, to a problem with the sampling equipment.• Structure: A set of parameters may be grouped by whether they precede or followa given parameter, e.g. what affects the MLSS.• Operation: A set of parameters may be grouped by whether they are causes(i.e. disturbance or manipulated parameters) or effects (i.e. status or performanceparameters).Once a relationship is detected, an operator may decide to reduce the relationship toan equation, i.e. characterize the relationship. Characterization consists of identifyingthe equation’s structure and determining the equation’s coefficients. A discussion ofmodel identification techniques is beyond the scope of this thesis. However, this does notpreclude a discussion of the types of relationships that interest an operator and some ofthe limitations his process places on his analysis.Chapter 2. PLF’s and Computer-Based Solutions 50An operate may regress data for one of three reasons:1. Measure: A regression can be used to summarize data by reducing an aspectof the data to a single value, i.e. statistic. Morris et al [216] showed that theVesilind coefficients provide a unique measure of the settleability of a sludge. Theyrecommend the coefficients be determined every 2-4 days during stable operationand daily when the plant is stressed (e.g. settling problems).2. Prediction and Planning: A regression can he used to formalize an observedrelationship between two or more parameters. Because the basis of the equation isassociation (not causation), the equation will work most of the time as long as thecircnmstances in the plant do not change.For example, Keinath [184] derived an empirical relationship that related stirredS\71 to the Vesilind coefficients [3SmL/g SVI 220rnL/g]:= VoeX= [15.3 — (2.1)Daigger and Roper [87] developed a similar relationship which is suspect becausetheir data included plants that used pickle liquor for phosphorus removal.A relationship may also be characterized via a reference or frequency distribution.A reference distribution (i.e. samples-based histogram) provides a measure of whatcan be expected [45] while a frequency distribution [224] may be used to predictrare events (e.g. equipment failure [271], peak flows [113]).9See Subsection 3.2.1, correlation due to inhomogeneityChapter 2. PLF’s and Computer-Based Solutions 513. Operation: A regression equation between a cause and effect may be used tocontrol the process. The independent variables should include at least one manipulated parameter while the dependent variable may be a status or performanceparameter. A status parameter may also be used as an independent variable. Therange of applicability of the equation should also be provided.For example, White [309] developed the following relationship for allocating clarification capacity and setting the recycle rate:0.68( + MLSS < 31085V1°77 () (2.2)whereA Area [m2]MLSS Mixed Liquor Suspended Solids [g/l]Influent Flow Rate [rn3/h]Recycle Flow Rate [m3/h]SSVI Stirred Specific Volume Index [ml/g]Keinath et al proposed a similar but slightly more complex system that uses anexperimentally determined mass flux curve [185].Three characteristics of treatment plant data limit the utility of regression:1. Treatment plant data are often multi-collinear, i.e independent variables are correlated.2. Treatment plants are under control, i.e. the operator compensates for importantsources of variability.Chapter 2. PLF’s and Computer-Based Solutions 523. Manipulated parameters may change over a limited range, i.e. range of applicabilityof an equation is narrow.Most secondary clarification models use an empirically derived relationship that includesclarifier influent flow rate and MLSS [22.5]. If the operator sets the recycle rate to be afixed fraction of the plant infinent flow rate, then the two flow rates will he correlated.A step-wise regression algorithm will use only one of them. The resulting relationshipwill hold true as long as the operator does not change his operating strategy.The second problem is that the operator compensates for important sources of variation, i.e. disturbance parameters. One possible signature of a well-operated processis that there is no correlation between the inflnent and effluent characteristics. NCASIlooked at the influent and effluent of 39 treatment systems including 17 activated sludgeplants that treat pulp and paper mill wastes [226]:The major observation is that the treatment processes reviewed in this bulletin appeared relatively insensitive to the variation in influent quality. Effluent quality and its variation more than likely reflects the operating state ofthe individual systems.The third problem is a corollary to the second problem. In order to learn what happenswhen you interfere with a process, you must interfere with it, not just passively observeit [135]. Therefore, another possible signature of a well-operated plant (with a fixed influent pattern) is that the data contain little information on how to change a manipulatedparameter.For example, W. G. Hunter recounts the story about a newly graduated statisticianwho analyzed a chemical plant’s data and then prioritized the causes of variance in theplant’s performance. At the end of the presentation, he stated that the least importantvariable is the amount of water present. Much to the statistician’s surprise, his audiencelaughed. It was, in fact, easily the most important one, because, if any water wereChapter 2. PLF’s and Computer-Based Solutions 53allowed to enter this particular plant, the plant would gloriously and very definitelyexplode.” [165].A computer program can aid the operator in determining relationships in three ways:1. Data Management: A computer is faster than an operator at grouping parameters and extracting their respective data series. An operator would gladly pass thistedious task over to a computer.2. Data Manipulation: A computer, if provided with the context of the data, canderive the data series based on the least common interval. A computer excels atthis well-defined repetitious task.3. Guidance: A computer can retrieve information (e.g. interpretive rules) much inthe same as it does data if it is provided with a method of matching the informationneeded with a particular situation.The latter task is the subject of a review paper (followed by a set of discussion papers)by Gerald J. Hahn [136]. Using elements of Hahn’s expert system for Product-Life dataanalysis, we can describe what a data analysis program should be able to do for theoperator:• Setup: The operator indicates what he wants to find out and the computer programsets up the problem, i.e. viewpoint. The computer determines which parametersare involved and extracts their data series.• Verify: The program characterizes the data and matches the data to the statisticalmethod.Chapter 2. PEF’s and Computer-Based Solutions 54• Execute: The program steps the operator through the analysis. When the computer requires the operator’s opinion, the computer queries the operator and provides a list of possible answers.• Interpret: The computer indicates how the operator can interpret the results byweighing the outcome against the data series’ characteristics.Ideally, each stage of the analysis should have a graphical and textual representation. Aswell, a complete report should be geuerated at the end so the operator can review whathe and the computer have accomplished.A good operator is both suspicious and appreciative of statistics. The computershould cultivate these attitudes by pointing out the weaknesses and strengths of ananalysis. George Box summed up this attitude in his axiom [181]: “All models arewrong, but some are useful”.Chapter 2. PLF’s and Computer-Based Solutions 5514 1412 1210 10oD8 .o...° 8..CD6 64 0 42 200 5 10 15 20 00 5 10 15 2014 1412 12C10 1o...—.-::..—-.8086_600 5 10 15 20 00 5 10 15 20Figure 2.3: Anscombe’s QuartetChapter 2. PLF’s and Computer-Based Solutions 562.2.6 Statistical GraphsGraphics reveal data and can be more precise than conventional statistical computations [77]. For example, F. J. Anscombe derived data sets that can be described by thesame linear model (Figure 2.3) [141]:Number of Data Points : 10MeanofX : 9.0Mean of Y : 7.5Coefficient Of Determination : 0.82Equation : Y 3 + 0.5XThis example illustrates why an operator should always plot his data first.The most commonly used plots are listed below [77] [70] [98]:• Scatter Plot: Plot two parameters against each other. A Time Series Plot is aspecial case of a scatter plot in which one parameter is time. A common way toview a number of parameters is as an array of scatter plots. One very powerfulmethod to analyze an array of scatter plots is to allow the user to define a box inone plot that contains a set of data points, then highlight the location of these datapoints in every other plot in the array.• Cusurn Charts, Difference Plots: A cusum plot is a plot of the cumulative sumof differences from a location statistic. Difference plots are plots of first and seconddifferences between elements in a time series.Chapter 2. PLF’s and Computer-Based Solutions 57• Histograms, Percentile, Box and Whisker Plots and Stem and Leaf Plots:These plots show how the data are distributed. Plant operators find that histogramsare an effective way to monitor a process [45] [44]. Figure B.2 contains a simplebox plot and Figure B.2 contains a stem and leaf plot.• Andrew’s Curve, Chernoff Faces and Glyphs: These plots display more thanone parameter in a single plot, e.g. the plant’s effluent characteristics at a particularinstance. An operator assigns each variable to a parameter in an harmonic function.The frequency spectra of this function forms the Andrew’s curve. A Chernoff faceis constructed by assigning the deviation of a measure from its set point to a featureof a face. The further a measure is from its desired value, the more distorted thefacial feature. A Glyph is a circle with rays emanating from its circumference. Adatum’s value is assigned to a ray. The purpose of these plots is to provide animage that an operator can scan quickly but still notice important changes in thedata.• Control Charts: These charts are plots of successive statistical measures or othervalues of a random variable, e.g. mean, range, proportion and trend. Controlcharts are routinely used in QA/QC programs [248]. Berthouex et al [43] usedcontrol charts to the monitor effluent quality.• Autocorrelation Function and Spectral Density: The autocorrelation function and spectral density function enable the analyst to detect cycles in the data.Graphical methods can be abused much in the same way that statistical methods canbecause the interpretation of a plot is invariably based on some assumptions about thedata. Graphical analysis is time consuming unless the software is set up to automate theproduction of plots. For example, given a small data set consisting of ten parametersChapter 2. PLF’s and Computer-Based Solutions 58(one of which is time) there would be ((10(10— l))/2) = 45 plots to analyze. If we assumeit takes 10 minutes to analyze one plot, then it would take 450 minutes or a work day toanalyze the data. And even then, the more complex relationships involving three or morefeatures would be missed [264]. The alternative is to provide a computer program withthe ability to decide which plots the operator may be interested in seeing. The operatorcould page through these plots first, and if wanted to view additional plots, request thecomputer to add these plots to its list.2.2.7 Advanced Statistical MethodsA simple rule in statistics is that the more powerful or complex an analysis, the moredemands the analysis places on the data set. If the data set cannot match these demands,then the results of the analysis are suspect. This fact motivated many universities toredesign their statistics courses to free the student from the rigors of computation so thatthey concentrate on problem solving [71]. The students are given real data, complete withmissing values and asymmetric distributions, and taught how use statistics to characterizethen analyze these data. Similarly, applied statisticians advocate that statistical softwarehe made more “intelligent” [136]. The software would step the user through a completeanalysis starting with IDA and moving onto more sophisticated methods if the dataand problem warrant it. The goal of these two groups is the same to emphasize thatstatistical dollar is better spent on planning a study than on analysis.A number of advanced statistical methods have been tried on treatment plant dataincluding time series analysis. Time series analysis exploits auto- and cross- correlationswithin the data set. The cause of this correlation in wastewater treatment is twofold.The first cause is due to the actions within the unit process, e.g. mixing, aeration andseparation. For example, assume two aeration basins are linked in series with tank Aupstream of tank B. The MLSS in tank A and tank B are correlated because the solidsChapter 2. PLF’s and Computer-Based Solutions 59in A flow into B. Similarly, the MLSS in tank 14 is correlated with the MLSS in tankA an honr later for two reasons: (1) the biological activity is partly dependent on theconcentration of the active mass and (2) the mixing averages the characteristics of thesolids across the reactor.The second cause of antocorrelation is due to auto correlated inputs, e.g. diurnalflow variation. Numerous researchers, including Dehelak and Sims [93], Crowther etal [85] [84], Berthouex et al [8] [50] [47] [48] [49] [51] and Hiroaka et al [146] [147],examined the auto- and cross- correlation functions in wastewater treatment plants. Mostcharacteristics show a 1 to 3 day dependency and a seasonal component. The reason forthis low dimensionality is that the noise to signal ratio of most environmental measuresis high.Chapter 2. PLF’s and Computer-Based Solutions 602.3 ModelingThis section focuses on the use of a model to simulate and control the process. For areview of the current state of wastewater treatment plant modelling, refer to Lessard andBeck’s survey paper [195]. All advanced control schemes require a model of the process.Adaptive controllers usually use a stochastic model that is identified recursively on-linewhile model following (model reference adaptive) systems use a mechanistic model. Forexample, Kabonris and Georgakakos [178] designed a model reference set-point controlsystem for wastage and recycle rate. The controller uses the IAWPRC model [142] todetermine the optimal control settings to maintain the effluent quality and minimizeenergy consumption over a 24 hour period.Before a loop can be closed using a controller, the structure and the parameters ofthe model must be correctly identified. There is a great deal of doubt in the wastewaterand automatic control literature whether this is possible with some control loops.2.3.1 IdentifiabilityThe construction of a model (i.e. structural and parameter identification) from datainvolves three entities [199]:• The Data: A set of ordered pairs of input and output• The Model Set: A set of candidate models containing the real model 10• The Criteria: A method to determine which model is the real oneFigure 2.4 outlines the identification cycle. The data are obtained from an experiment.The type of model in a model set ranges from being mechanistic to being stochastic (i.e.t0Best Model: The real—life system is an object of a different kind than our mathematical models.Mathematical descriptions are templates we place over the real-world to order what we see. Therefore,the real model is one that describes what we see in full.Chapter 2. PLF’s and Computer-Based Solutions 61black box). There are several ways to fit a model to data (i.e. criterion), of which, leastsquares is only one of many [199]. A model is valid if it satisfies at least three criteria:• Model agrees sufficiently with observed data: The model is run alongside thesystem to see if it follows the process. The data set on which a model is identifiedshould not be the same data set on which it is validated [31].• Model is good enough for the purpose it was intended: The model’s performance is compared against the modelling objectives.• Model describes the “true” system: The model’s dynamics are comparedagainst past data and theory (via simulation) “.The identification fails if the model is invalid or if the model is nonidentifiable. The lattercase is the focus of this section as it is the “Achilles heel” of most modelling and controlexercises in wastewater treatment.The problem of deciding how to use the criteria to determine which parameter set bestfits the model to the data is referred to as the parameter identification problem. A modelis considered identifiable if an analyst can use the criteria to determine the best parameterset. Therefore, parameter identifiability is dependent on both the characteristics of themodel and the data set.Godfrey and DiStefano [123] divide parameter identifiability into two classes: (1)deterministic, structural or a priori identifiability and (2) numerical or a posterior identifiability. Deterministic non-identifiability occurs when two parameters are confoundedand cannot be separately identified. Complex nonlinear models are susceptible to thistype of problem. Once a parameter is known to be identifiable, the next step is to determine the accuracy of the parameter estimate given a particular stochastic input function“The true system is an esoteric entity that cannot be attained in practical modelling. We have tobe content with partial descriptions that are purposeful for our applications” [199, p. 430]Chapter 2. PLF’s and Computer-Based Solutions 62Table 2.7: Holmberg’s Batch Reactor Equationsdx=ds 1= —571t8xS— K8+sx : concentration of microorganismsconcentration of growth limiting substratespecific growth rateY : yield coefficientAd : decay rate coefficient[tm : maximum growth rateMichaelis- Menten constant(i.e. numerical identifiability).For example, Holmberg [1.56] published a paper on the identifiability of microbialgrowth models incorporating Michaelis- Menten (Monod) type nonlinearities. Holmbergexamined microbial growth in a batch reactor (see Table 2.7).Holmberg studied the sensitivity functions for the four parameters, {K3,1um, K1,Y}.The sensitivity function, , describes the effect of small perturbations in the parameters,{p : K, ,urn. Kd, Y} on the state variables, {z : x, s}. The sensitivity function of lAm andK3 are indistinguishable with respect to either x or .s. This indicates that these pararneters may be difficult to identify, i.e. will give the analyst the most problem. Pohjanpalo’sidentifiability test confirmed that all the parameters are structurally identifiable.Holmberg introduced noise into the parameters and observed that the standard deviation of K8 and [m could be 200-7007c greater than the standard deviation of s andChapter 2. PLF’s and Computer-Based Solutions 63Figure 2.4: Ljung’s System Identification LoopChapter 2. PEE’s and Computer-Based Solutions 64x. Holmberg warns that with a few noisy nwasurements, an analyst will not be ableto determine K3 and gUm with sufficient accuracy. The paper concludes that (1) theMichaelis-Menten model is at best an empirical model rather than an internally descriptive one and (2) a number of K3 and sUm pairs fit the data set with a moderate amountof noise equally well. Holmberg’s results challenge the validity of wastewater treatmentplant models that consist of a series of Monod type equations.M. B. Beck examined the area of wastewater modeling in a number of papers [31] [32][33] [34] [35] [37] [36]. He observed that modellers take one of two approaches: stochasticand mechanistic. Stochastic models are limited by treatment plant data, which usuallyvary within a narrow range exhibiting very few degrees of freedom. These models aretypically under-parameterized but accurate as long as the conditions under which theywere identified prevail. The second approach is to construct complex nonlinear models.The model parameters are identified in laboratory experiments [114, pp. 347]:Most of our knowledge of the activated sludge system is based on experimentscarried out under so-called controlled laboratory conditions. Such studies canbe much broader and more diversified than experiments in pilot or full-scaledconditions, but are often biased by undne influence of some factors whichcannot be properly controlled or modelled. It is usually taken for grantedthat the behavior of the full-scale plants will be close to that of models studied, and that the relationships among process parameters, observed in smallerscale, will be equally valid in the full-scale plants. However, quite often thefluctuation of wastewater quantity and quality, as well as the instability ofsome process parameters, together with the inadequacies of metering andsampling, are such that in specific cases expected correlations cannot be realized in practice. The data are characterized by such a scatter of informationthat their correlation would be without any practical meaning.In other words, complex mechanistic models tend to be precise but inaccurate.lvi. B. Beck summarized these problems in two dilemmas [35]: Model ComplexityDilemma and Prediction Error Dilemma. The basis of these dilemmas is the over andunder parameterization of the model with respect to the data set [199]. For example,Chapter 2. PLF’.s and Computer-Based Solutions 65assume the true system is giveu by Equation 2.3 (linear case) and Equation 2.4 (nonlinearcase).Y = aX1 + bX2 + cX3 + d (2.3)Y = dXXX (2.4)In order to understand the consequence of Beck’s dilemmas, consider the following scenarios:• Scenario 1: The data set has 2 degrees of freedom (df), i.e. X2,X3 are constantand the model has 2 df:— Linear Case: Y = aX1 + dIn this case, d = d + aX2 + bX3.— Nonlinear Case: Y = dXIn this case, d = dXX, assuming that a a.Scenario 1 is the ideal stochastic case, i.e. the df of the data equal that of the model.If X2 or X3 cease to be constant, then both model equations will give precise butinaccurate predictions, i.e. they will consistently predict the wrong future. If X2and X3 change slowly, the analyst can use some form of recursive identificationscheme [200] to update the estimates of a and b.Chapter 2. PLF’s and Computer-Based Solutions 66• Scenario 2: The data have two degrees of freedom but the model oniy one:— Linear Case: Y = aX1— Nonlinear Case: Y = XfScenario 2 is Ljung’s under parameterized case [199]. In these cases, a may bedifficult to determine and its value will depend on the fit criterion. Scenario 2 willnever provide a good prediction of Y even when X2 and X2 remain constant. Theproblem is similar to trying to describe a plane with a line. Scenario 2 will havethe same problems as scenario 1.• Scenario 3: The data set has 2 df and the model has 3 df:— Linear Case: Y = aX1 + êX + dA step-wise regression algorithms will “kick out” the X term. If the usersets the term a priori (e.g. from a laboratory experiment), then d will changeaccordingly.— Nonlinear Case: Y = dXXfThe identification algorithm can choose any value for d, X and ê as long asit satisfies the following equation:= dXX (2.5)If X is X2, and X2 ceases to be constant, the model has the potential ofpredicting the correct future. However, the user has no way of knowing whenit is doing so since the values of the extra parameters were chosen arbitrarily.Scenario 3 is Ljung’s over parameterization case. Scenario 3 presents a differentproblem to 1 and 2. If the user conducts a series of laboratory tests to determineChapter 2. PLF’s and Computer-Based Solutions 67some of the model coefficients (e.g. kinetic rates, half saturation constants, substrate fractions), then the user must have a method of determining if the laboratoryvalues hold true in the field. The user cannot determine this from the field dataalone and, therefore, can never be certain that the model will predict the correctfuture. Beck [36] argues that this is particularly a problem with carbonaceoussubstrate degradation because the responsible biomass is heterogeneous and thesubstrate measurements are noisy and nonspecific.The consequence of Beck’s dilemmas can be reduced to three points:1. Over-parameterized models that rely on laboratory experiments for determinationof their coefficients cannot be relied on because there is no way to verify that thecoefficient values hold true in the field.2. Any model that uses field data to determine some of its coefficients can be reliedon as long as conditions in the plant do not change, i.e. there is no advantage toover-parameterized models.3. Any model that is used for control must be recursively identified on line and,therefore, should not be over-parameterized.A model cannot be used to control a complex system unless the model’s structnreand parameters are identifiable. The model must be complete in the sense that it enablesthe operator or computer to explain the past and plan the future. 12 To plan means tobe able to define a set of control actions that will move the system from its current state12Kalman Decomposition Theorem [18] states that a linear system must be both observable andreachable before it is identifiable. Observable means that given the input/output history of the system,you can predict its current state while reachable means that given its current state, you can define a setof inputs that will drive it to a another state.Chapter 2. PLF’s and Computer-Based Solutions 68to a desired state. To explain means that given its current state and its input/outputhistory, you can determine what a past state was (see Section 3.1).The point of this discussion to underline how a computer can use a model to assist theoperator. A model can be used to help the operator analyze his data, i.e. act as a basisof comparison. A model may also be used to make short term predictions provided themodel’s parameters are being constantly adjusted to reduce the error in these predictions.However, over-parameterized (and therefore partly laboratory identified) models shouldnot be used as the basis for control.Chapter 2. PLF’s and Computer-Based Solutions 692.4 Expert SystemsAn expert system is an automated process which incorporates the judgement, experience,rules of thumb, and intuition used by a human specialist to emulate that specialist’sproblem solving ability. Usually, the knowledge is stored in a computer in the form offacts and decision rules, although many more complex knowledge representations areavailable (e.g. neural networks). Au expert system consists of two parts:• Knowledge Base: A database of domain specific knowledge (cf basic source code).• Inference Engine: A knowledge interpreter (cf basic interpreter).The inference engine forms part of the shell. A shell is the total system minus theknowledge base. The main advantage of an expert system is that it is an available sourceof knowledge no matter what time of day it is, i.e. human experts unavailable.Expert systems have been applied to a number of tasks in wastewater treatmentplants:• Recommend solutions to wastewater and water treatment process problems [242][174] [203] [230]• Review monitoring data and make control decisions [190]• Recognition and diagnosis of sludge bulking [121]• Detect failure in an anaerobic digester and recommend action [192]• Recommend control actions based ou the DO profile in a PHOSTRIP plant [109]• Evaluate compliance data from a number of plants [312]Chapter 2. PLF’s and Computer-Based Solutions 70• Identification of performance limiting factors following the EPA’s CPE/CCP procednres [69].There are at least fonr reasons why expert systems have met with mixed snccess inwastewater treatment:1. Expert systems are limited by lack of access to other databases, e.g. monitoringand maintenance information [242].2. Expert knowledge is not available. Knowledge is either is too vagne or too specificleading to a mediocre system [241].3. False alarms and rnle conflicts undermine the operator’s confidence in the system [190]4. A good operator does not need an expert system to help him run his plant, i.e .theoperator is the expert [190].2.4.1 Integration: Access To Other DataThe trend in the industry is away from stand alone expert systems to embedding thetechnology into existing programs, e.g. spreadsheets, databases, statistical packages [230].An expert system can be integrated with other software in a number of ways includingusing mixed language programming (e.g. Prolog and C), linkable expert systems 13,shells with “hooks” to other commercial software, and rule compilers. In order not to“cripple” a piece of software, like the one envisioned in this thesis, the expert systemcomponent should he coded as a number of small rule bases that the software’s internallanguage interpreter can process. At this point in time in wastewater treatment, the13e.g. NEXPERTOBJECT © Neuron Data, 165 University Avenue, Palo Alto, CAChapter 2. PLF’s and Computer-Based Solutions 71integration issue far outweighs the issues surrounding how knowledge is represented andinterpreted.2.4.2 Expert: Is There One?To develop an expert system, there must be an expert. This may seem trivial, hut inreality, many experts are not experts at all. An “expert” is a person who can both do andexplain the task [38]. The task should be one that is routinely taught to beginners, andeven after they are trained, the expert should be able to outperform them. The expertmust be willing to communicate his knowledge and to commit himself to the project.Researchers have expressed serious doubts that there are individuals of this caliber forall areas of wastewater treatment operations [241]. The most disconcerting question anengineer can ask a computer scientist is “How do you know your expert is an expert?” 14This question is analogous to the identifiability question posed in the previous section.2.4.3 Rule Conflict: Wait and Short Term GainA rule conflict occurs when two rules suggest opposing actions for the same condition.There are at least three reasons why this may happen:1. A change in a manipulated parameter may have a positive effect on one part of theplant and a negative effect on the other. For example, an increase in the recyclerate increases turbulence in the clarifier, reduces the hydraulic retention time inthe aeration basin, increases the hydraulic loading on the clarifier, decreases thesludge blanket height and increases the plant’s energy consumption.2. A change in a manipulated parameter produces a hydraulic, chemical and biologicaldynamic in the process. Some of these may be preferable and some not.‘4From experience. I never heard from him again.Chapter 2. PLE’s and Computer-Based Solutions 723. The operator is sometimes unsure how to use a manipulated variable to affect hisprocess. One cause of this uncertainty is due to interactions between status andmanipulated parameters, i.e. the effect of a change in the recycle ratio is dependenton the MLSS and the air flow rate into the aeration basin [295].Lai [190] lists a number of guidelines used by his expert system to resolve these conflicts:1. Select the control action that has an immediate effect on protecting effluent quality.2. Examine the trends in the variables. If the variables are headed in the right direction, leave alone.3. If a recent control action was taken, wait (action may influence today’s decision).4. Be cautious about acting on the basis of a single unusually high or low value.5. If there is no risk, wait.The first guideline may lead to accepting immediate (hydraulic) improvement at the costof long term (biological) deterioration. The second guideline explains why the computerneeds to know when a number is preferable, i.e. operator prefers a low effluent BOD5concentration over a high one. The other guidelines advise the operator to wait until theprocess sorts itself out, i.e. when in doubt, do nothing.2.4.4 Are Expert Systems Needed?An expert system application is a success if it meets three criteria [128]:1. The expert system must represent/emulate the decision processes used by knowledgeable person in the field.Chapter 2. PLF’s and Computer-Based Solutions 732. The expert system must produce a sufficient increase in decision efficiency andquality to justify the development and maintenance costs3. The expert system must be accepted by those who will use it.Most wastewater expert systems fail on the first two criteria. In other words, theywere not the best tool for the job. An expert system is a new tool to most wastewaterpractitioners, and therefore, there is a danger it will be used inappropriately. For thisreason, Beckman [38] developed a checklist of 75 qnestions to help a practitioner decidewhether or not to develop an expert system. The list is broken into six categories:Category Points PossibleTask 25Payoff 20Management 20Domain Expert 15System Designer 10User 10Total Possible 100The main problem with many of the stand alone expert systems developed to dateis that there is no payoff because there is no need for such a system. For example, theconsensns in the wastewater literature is that experienced operators do not need expertsystems to help them run their plants [230] [190]. The reason there is no need is becausean expert system is either the wrong tool or not the best tool to use to solve the problemat hand. Most problems in wastewater treatment plants can be placed into two groups:those an experienced operator can solve and those that reqnire outside help. The basicpremise of an expert system is that there is an expert and that the expert is eitherunavailable or expensive. If an experienced operator on staff can solve the problem, thenChapter 2. PLF’s and Computer-Based Solutions 74there is no need for an expert system. If the problem requires outside help, then it usuallyis plant specific and not worth developing an expert system for.In most cases, the operator solves problems by conducting an investigation (e.g. additional analyses, in plant measurements) , and when necessary, will refer to an operationsmanual for help (e.g. [293] ). The nature of the problem is iterative meaning the operatormay decide to take additional measurements as part of his diagnosis. If the problem isgeneric, then an expert system could be used to solve it if the problem requires cognitivereasoning and is neither to complex or simple (i.e. an expert can solve it in 1-8 hours) [38].If the problem is plant specific, the expert system probably will not be able to help theoperator due the the system’s ignorance of the process’s particulars. In this case, it isprobably cheaper to “call out” an experienced operator (if he is not there already) thanto develop a plant specific expert system. The second type of problems happen infrequently and usually require a process audit to solve. The “expert” designs and sometimessupervises the audit to diagnose the problem. Again the expert is available.Expert systems excel at tasks that have a narrow domain and involve ctassificationor heuristics. For example, AT&T developed an expert system, Antoprint, to simplifyprinting out a file under UNIX [263]. This is a successful application because UNIX is“well-behaved” system, i.e. logically consistent and predictable. Biological systems arenot. There is a gap between what the expert system requires and what the operator (orexpert) can provide it with created by the lack of monitoring information and processunderstanding.An expert system should provide a number of benefits including reduced costs, increased autonomy from consultants, and improved quality. The developer of such asystem must look at a minimum of three other criteria. The first criterion is the natureChapter 2. PLE’s and Computer-Based Solutions 75of the platform that is needed to mn the system 15 Ideally, the system should run onthe machines now in nse in the plant. The second criterion is the portability of the product. A developer will not recoup his investment by developing a system that is useableonly at a single plant. The final criterion is whether the product would be useful if onlypartially finished. Al (Artificial Intelligence) projects are notorious for not meeting theirdead-lines. The cost to benefit ratio for a successful project should be approximately1:10 [38]. Given the plant specific nature of process knowledge and the high cost tobenefit ratio, it is not surprising that most wastewater expert systems are developed byuniversities and government departments.Most wastewater expert system projects would not have gone ahead or at least takenthe form they took if the researcher had worked through Beckman’s checklist. Theconsequence of this is that most environmental expert systems sit in a filing cabinetnnused [166]. The fault is not the technology but rather its misapplication.2.4.5 Future: The Role Of Expert System TechnologyIf a researcher works through Beckman’s list [38] or Greathouse’s six questions [128],he will realize that expert systems show their greatest potential when (1) they are partof other systems and (2) their purpose is to support rather than replace the decisionmaker 16 These expert systems should be small, fast, run on conventional hardware andbe imbedded in existing applications [231]:A recent trend is that expert systems are moving away from being “standalone” systems to one interacting with other useful plant software packages,such as database-management systems, spreadsheets, and graphics. Thus,embedibility of expert systems into other programs and the calling of these15For most plants at the writing of this thesis, the system should run on an IBM AT class machine.This is a significant limitation as the POTW Expert [69] is noticeably slow on a machine with an Intel80386 running at 25 Mhz.‘5The use of expert systems to support environmental decision makers is discussed in [131]Chapter 2. PLF’s and Computer-Based Solutions 76programs from the exert system is becoming increasingly important. Theunderlying philosophy is to accept expert-system technology as just anothersoftware tool, and to make a judicious use of this tool in tasks best suited toit.For example, expert systems are being used now in a number of computer programs tomake a piece of equipment “more intelligent”:• On-site Help: A complex piece of equipment would have a small expert systembuilt into it that helps the operator trouble-shoot and/or operate his equipment.This is a common feature in photocopiers.• Failure Detection: A plant with a large number of sensors and other on-lineinstrumentation should have a program that monitors the data sent back to theplant to detect sensor failure. The program could do on-line calibration, takea sensor off-line and/or warn the operator that a sensor needs attention. Thissoftware exists in some chemical plants and nuclear reactors.• Data Analysis: An expert system could make data analysis easier by improving aprogram’s interface, interpreting analytical results, selecting appropriate data analyses, and making suggestions on the basis of the monitoring and maintenance data.This was discussed in Section 2.2 and is the subject of Hahn’s review paper [136].In order to imbed expert system technology into an application, the expert system mustbe made aware of the process and given access to structural and monitoring information.This is the purpose of the structure paradigm discussed in Chapter 6.Chapter 2. PLF’s and Computer-Based Solutions 772.5 Automatic ControlThe main goals in applying control methods to microbiological systems are to improveoperational stability, prodnction efficiency and profit, and to handle dynamic changesduring start-up and shutdown [157]. Most biological processes are difficult to controlautomatically because they exhibit non-linear, time-varying behavior and many of theavailable process state measurements are of poor quality [109]. For this reason, controlin these systems remains a hierarchical two-level problem with the operator active on theupper level [155]. Gustaf Olsson [234] [236] notes that the operator cannot be factoredout of the loop as he is the best person to monitor slow changes in the process, leavingthe computer to deal with the fast ones.Conventional control theory deals predominantly with linear systems having constantparameters [20]. Loops in these systems are usually controlled using a ProportionalIntegral-Derivative (PID) controller. However, if operating conditions change, the controller needs to be re-tuned. This becomes a problem if the system is constantly changing,which is the case with aeration and disinfection systems in wastewater treatment plants.In the last decade, Foxboro (1984), ASEA (1982) and SattControl (1984) introducedauto-tuning PID controllers. In the spring of 1986, Novatune’s product controlled over1000 loops in a wide range of industrial processes including loops in wastewater treatment plants and pulp and paper mills [19]. In 1986, Rundqwist [254] implemented aself-tuning dissolved oxygen controller in the Kappalla Sewage Works in Sweden. Thestandard deviation of the DO was 0.15 mg/l over the test period. Since then, Holmberget al [158] and Bocken et al [54] introduced systems that estimate the oxygen utilizationrate on-line to improve the controller by predicting the demand.Chapter 2. PLE’s and Computer-Based Solutions 782.6 SummaryTreatment plant performance is limited by a number of factors, some of which the computer can ameliorate. The computer is a powerful tool only if it is applied to the rightproblem. The right problem is something the operator either does not do well or dislikesdoing. In addition to the tasks the computer now performs in treatment plants, it couldbe used to analyze, model, reason and control the system. However, there are limitations:• The computer may be used analyze the data as long as the software is sensitive tothe data’s characteristics. In most plants, these characteristics limit the analysisto simple summary statistics and statistical graphs.• The computer may be used to model the process but these models cannot form thebasis of controllers unless they can be identified reliably on-line.• The computer may be used to reason about the process hut these expert systemsshould he coupled to existing software and act in support of the operator. Typically,these expert systems will be small and fast, and perform supervisory tasks suchas monitoring on-line instrumentation for failures, reviewing monitoring data andproviding on-line help with complex pieces of equipment.• The computer may he used to control the system but only under the supervisionof the operator. The computer can control difficult loops using adaptive controlalgorithms such as auto-tuning.Information on what limits and improves performance comes from four sources: PLEX,CPE/CCP, audits and historical data. Because historical data is a long term record ofthe operator’s interaction with his process, it should he the best source of information.Chapter 2. PLF’s and Computer-Based Solutions 79This is not the case because the quality, coverage and completeness of these data is poor.The computer can help the operator overcome these limitations by managing the data insuch a manner that the operator regains control of both his information gathering andtreatment processes.Chapter 3Cause and EffectThe approximate answer to the right question is worth a great deal more thana precise answer to the wrong question [73J.’The purpose of this chapter is to discuss time and cause/effect. The chapter consists ofthree sections:1. Temporal Reasoning: This section provides an overview of Shoham’s TemporalReasoning paradigm and discusses the paradigm’s application to treatment plantoperations.2. Cause and Effect: This section is broken into three parts. The first part discussesthe difference between association and causation. The second part discusses thedifficulty of determining an effect’s cause while the third section discusses experimental design.3. Operation Paradigm: This section outlines the requirements of an treatmentplant operations paradigm.‘John Tukey’s Fzrst Golden Rule Of Statistics80Chapter 3. Cause and Effect 813.1 Temporal ReasoningThe importance of reasoning about cause over time is fundamental in all areas of science.Given this issue’s interdisciplinary nature, the terminology is not standardized. For thisreason, we chose to use the definitions laid out in Yoav Shoham’s book “Reasoning AboutChange” [270].3.1.1 IntroductionWithout the possibility of change, there is no reason to keep track of time. Consequently,knowledge possesses a temporal component, i.e. what is true now may not be true later.A theory describing change in the context of time provides both a language for describingwhat is true or false over an interval and a way of manipulating rules describing lawfulchange.We classify the types of temporal reasoning into four classes 2;• Prediction: Given a description of the world over some time period and a setof rules governing change, predict the state of the world at some future point intime. For example, given the treatment plant’s current state and a calibrated plantmodel, determine what the effluent characteristic will be in a week.• Explanation: Given a description of the world over a time period and a set ofrules governing change, describe the state of the world at a previous point in time.For example, given a set of monitoring data up to the point the clarifier started tobulk and a model, determine when the system started to change.2The notions of planning and explanation are abstractions of the concepts of controllability andobservability in linear systemsChapter 3. Cause and Effect 82• Planning: Given a description of a desired futnre state and a description of thecurrent state, provide a set of actions to achieve this future state. For example,draft a strategy that will fix the bulking problem in the secondary clarifier.• Learning: Given a description of the world at a number different times, providea set of rules that account for regularities in the description. For example. explainhow the Mean Cell Residence Time (MCRT) influences the sludge’s settleability.To plan and to learn require a higher level of thinking than to predict or to explain (cfBloom’s Taxonomy).These four types of reasoning form the basis of plant operations. If an operator canpredict, then he can plan. Similarly, if he can explain, then he can learn. If he canplan and learn, then his ability to operate the plant will improve along with the plant’sperformance. However, the operator must know when he can optimize if he is to planand he must be able to review his decisions if he is to learn. These are the principlesthat form the basis of Box and Draper’s EVOP [58].3.1.2 Problems and SolutionsThe goal of a temporal reasoning algorithm is to reason correctly and efficiently aboutwhat is true over extended periods of time. In other words, the algorithm’s goal isto reach a decision quickly expending the least amount of resources without making amistake. The general problem exists because these two criteria, efficiency and accuracy,are contradictory.The general problem can be split into two problems: the qualification problem andthe extended prediction problem. The qualification problem arises from the relationshipbetween the amount of knowledge required to make a decision and the accuracy of thedecision. For example, an operator observes ash-like particles on the liquid surface of hisChapter 3. Cause and Effect 83secondary clarifiers. This condition may be caused by either the onset of denitrificationin the sludge blanket or high grease levels in the solids in the aeration basin [293]. Theoperator mnst decide how much information he needs to collect before he can decidewhat the canse of the problem is. For example, is the operator willing to conduct agrease analysis on the MLSS (as suggested by Tsngita et al [293]) in order to diagnosehis problem?The extended prediction problem arises from the relationship between length of timeover which a prediction is made and the accuracy of the prediction. A special case of thisproblem is the persistence problem. The persistence problem occurs when we predict onthe basis that a fact remains true over the prediction interval. For example, we may mapout an operations strategy that is based on the assumption that the operator can controlthe SRT over the prediction interval [301].Both the extended prediction and qualification problems must be taken into accountwhen controlling a treatment plant using a model. The amount of data needed to calibratea model may be more than that needed to operate the plant. In this case, it makes senseto calibrate the model intermittently. The length of time between calibrations must beshort enough to ensure the model is reliable but long enough to ensure that the use ofthe model is feasible.The solution to these problems is chronological ignorance. An operator chooses acompromise between efficiency and accuracy that provides him with sufficient accuracyto run the plant given the resources he has available. However, the operator must inclndein his daily decision making an evaluation of the content of this compromise. For example,the operator assumes that by manipulating the MCRT. he can effect a degree of controlon the system. This is true as long as the clarifier is able to separate the solids. However,if he has to stop wasting in order to maintain some solids in the aeration basin, then hecan no longer manipulate either the F/M ratio or the MCRT. What was true when heChapter 3. Cause and Effect 84made his compromise is no longer true, so he must reevaluate his monitoring program toreflect the new state of his system. The is the situation that one researcher encounteredwhen he was developing a process control strategy based on decision theory [301].The consequence of this solution is nonmonotonicity. Nonmonotonic reasoning occurswhen the addition of a rule to the knowledge base can force us to retract an inference.Nonmonotonicity and common sense are two reasons why expert systems have difficultywith treatment plant operations . However, an operator can live with nonmonotonicityas long as he periodically reviews his temporal assumptions.3The problem with common sense is that it leads to nonmonotonic logic. For example, if we knowsomething is a bird, we assume it can fly. If we find out it is an Ostrich, then we retract this conclusion.Chapter 3. Cause and Effect 853.2 Cause And EffectA clear understanding of the cause and effect relationships within a treatment plant is aprerequisite for successful operations. If the operator searches for the effect of a cause, heis an observer, while if he causes an effect, he is an experimenter. All other things beingequal, it is better to be an experimenter than an observer. In both cases, the operatoravoids obscuring these causal relationships by the introducing additional changes or bynot monitoring the process.3.2.1 Association Versus CausationAssume operator plots the effluent COD against a number of plant parameters includingthe volume of supernatant returned by the anaerobic digester to the primary clarifier.Assume he notices a linear relationship and regresses one on the other. In this case, theCoefficient of Determination, R2, is high [137]. The Coefficient of Determination is thatproportion of the dependent variable, effluent COD, that is accounted for by the regression equation in the independent variable, volume supernatant returned. Unfortunately,the Coefficient of Determination gives no indication of whether the lack of perfect prediction is caused by an inadequate model or by experimental uncertainty [95]. On otherwords, a high R2 value does not necessarily indicate a statistically significant equation.All statistical methods are based on a set of assumptions. Therefore, the result ofany statistical test is conditional on the fit between the test’s assumptions and the dataanalyzed. The R2 value is no exception. For example, even though the data may satisfythe test’s assumptions, the significance of the R2 value is still dependent on the numberof data points and the number of parameters in the regression equation .4The intent of this section is not to discuss regression but rather the determination of cause andeffect relationships. The topic of regression is thoroughly discussed in Draper and Smith’s text AppliedRegression Analysis [97] and Mosteller and Tukey’s text Data Analysts and Regression [217]Chapter 3. Cause and Effect 86Similarly, a high R2 value does not necessarily indicate a useful regression equationeither. A large R2 value can result from the data being taken over an unrealistically largerange of the independent variables or a small R2 may result because the system is beingrestrained, i.e. controlled. Because the data do not come from an experiment, a high R2may indicate association but not necessarily causation [255].For example, given the function y = .sirz(x), y is functionally related to x yet thecorrelation between the two is zero [26]. On the other hand, there are at least foursituations where two variables are correlated but not in a causal relationship:1. Chance: Two unrelated phenomena may show a correlation for no reason at all,e.g. an observed association between the circulation of New York public libraryand the changes in the ozone layer.2. Dependence: Two variables may share a common cause, e.g. the decrease in thenumber of stork nests in East Prussia and the decrease in the number of humanbirths, both caused by growing industrialization. A second example would be thedecreasing rate of substrate utilization and the decreasing rate of solids productionduring a batch test, both of which are a function of the metabolism rate.3. Inhomogeneity: The data are taken from three distinct groups. The correlationbetween the two variables is nonexistent within the group but, when the data arepooled, the correlation exists because of the inhomogeneity between groups. Forexample, Horvath, in his text on scale-up of wastewater unit processes, argues thatthe ratio of volume to surface area should be held constant when scaling up abioreactor [162]:Chapter 3. Cause and Effect 87Volume to Surface Area RatioCylindrical Tank25.020.0\\ Scale: Diameter13.7 m and Depth=8.5 m1 5.0Ratio=2.44I.5.0a)U)2_____________________________________2.0c 10.0—0.0-I- I I I I I I I0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0Depth (m)Figure 3.1: Scale-up : Volume to Surface Area Ratio ContoursChapter 3. Cause and Effect 88The wall effect is more or less pronounced in virtually all cases and maymodify not only the hydraulic but the physico-chemical and biologicalconditions. The boundary snrfaces may, for instance, act as a catalyser[sic] and the biological film developing on the wall must also be takeninto account- especially in small tanks. The ratio of the internal (active)boundary surfaces and volumes of the reactors of different size should bepossibly equal.Given a full-scale cylindrical reactor, 8.5 in high and 13.7 rn diameter, the volumeto surface area ratio is 2.44 in. Figure 3.1 shows the relationship between depth anddiameter for a number of volume to surface area ratios. A researcher must maintaina reactor with a height over 4.0 in in order to maintain the full scale ratio of 2.44 in.If the wall effect is significant and if a different ratio is used at bench, pilot andfull scale, then when these data are pooled, a correlation will exist because of thevarying importance of the wall effect in each group of data.4. Dead Soldier: One of the variables is used to derive the other, e.g. plot effluentsolids against Mean Cell Residence Time (MCRT) where the calculation of MCRTincludes unintentional wastage.Chapter 3. Cause and Effect 893.2.2 Effect To Cause: The Poor CousinThe weakest form of scientific investigation is observation. In this case, the researcherconducts measurements on the system and tries to sort out what is a cause and what is aneffect. However, this is often the only way to study a phenomenon because of the natureof what is being studied, i.e. field conditions can not be duplicated in the laboratory. Theunderlying premise of an observational study is that association may indicate causation,i.e. “where there is correlational smoke, there may he causational fire” [154].Different disciplines have formalized guidelines for sorting out cause and effect. Inmedicine, Koch ‘s Postulates are used to implicate an organism as a pathogen. Socialscientists use path analysis and other correlation techniques while economists rely heavilyon time series analysis. One set of guidelines broad enough to be useful in environmentalengineering were proposed by Sir Bradford Austin Hill, who was among one of the firstpublic health officials to argue that there is causal connection between smoking and lungcancer. Hill’s criteria are listed below [154]:• Temporality: The cause should precede the effect, i.e. what happens upstreammay be the cause of what happens downstream. To this we must add one proviso, wecan only reason about monitored causes. For example, we cannot link denitrificationto rising sludge in the secondary clarifier unless we monitor changes in the nitrateconcentration.• Natural Experiment: A Natural Experiment is an unplanned event that approximates a planned experiment. For example, due to a malfunction at a wineryconnected to the sewerage system, the plant is organically overloaded for a shortperiod of time.5A Natural Experiment may approximate a quasi-experiment. A quasi-experiment is an experimentwhere the various treatment groups cannot be presumed to be initially equivalent within the limits ofsampling error [80], e.g. repeated experiments on a treatment plant.Chapter 3. Cause and Effect 90Plausibility, Coherence, Analogy: A causation link must be plausible andcoherent in the sense that it does not conflict with known facts. Similarly, the dynamics of the situation should be analogous to similar situations that have occurredelsewhere. These principles are very important when a number of causes lead tothe same effect. For example, low DO concentration in the aeration basin, nutrient deficiencies, presence of sulfides, low organic loading and fixed film cultures inbench-scale treatment processes (e.g. in tubing) have all been identified as causesof sludge bulking [171].• Strength: It is easier to suspect a casual link if the cause and effect are stronglyassociated. Association may take various forms. For example, a spectral analysis 6of the winery waste and the plant’s effluent characteristics shows that certain dynamics found in the winery waste appear in the effluent characteristics that cannotbe attributed to dynamics in the domestic influent.• Consistency: The same conditions should produce similar results. All good research uses replicates, either through time or in space. The reason for this is thatthere is no guarantee that the same conditions will produce the same results. Forexample, Zaloum [317] applied the same organic loading to two systems at the sameSRT and observed a different response. The difference occurred because the twosystems were operated at different hydraulic retention times (HRT).• Specificity: A specific cause has a specific effect. Based on experience, Junkinset al [177] recommend that the operator maintain a 3 feet (< im) sludge blanketdepth in the secondary clarifier. However, Crosby [83] observed that the effect of5Spectral analysis is a useful exploratory tool as long as person is aware of its limitations. In ourcase, we used spectral analysis to examine the Penticton’s data and identified behaviors in the data thatthe operator later explained. See spectral density in glossary.Chapter 3. Cause and Effect 91the blanket height on the effluent solids depended on the settling properties of thesludge. Crosby recommends the following:— If sludge settleability is poor, and flow rate is steady, then raise the blanketso that the inlet is below the blanket. If the height of the blanket is notcontrollable, increase the recycle to lower the blanket below the inlet.— If the sludge settleability is good, then lower the recycle to lower the blanketbelow the inlet.In this case, the specific cause (blanket height) has a specific effect (effluent solids)depending on the settleability of the sludge. Statisticians refer to this as an interaction.To this list, we need to add one more item. A cause must be a variable that canbe manipulated. An attribute (or an intrinsic variable) cannot be a cause because anattribute’s value changes only when the object it is describing changes. For example,the redox potential of an anaerobic reactor in a biological phosphorus removal plantdoes not cause the bacteria to release phosphorus. However, the redox potential andphosphorus release may be correlated because both are caused by the same mechanism.In other words, “causes are only those things that could, in principle, be treatments inexperiments” [154].3.2.3 Effect To Cause - Treatment PlantsA treatment plant is a complex system. If an operator must determine what causedan effect in the system, he starts by tracing upstream. Possible causes fall into twocategories: disturbances and control actions. A disturbance is a cause whose origin isoutside the plant, e.g. influent, weather, power failure. A control action is something theChapter 3. Cause and Effect 92operator does to the system, e.g. recycle rate, wastage rate, DO set point. The analysisis complicated by three factors: limit cycles, loops and the operator’s actions.A layman’s definition of a limit cycle is a finctuation in the system that the operatorcannot remove. Limit cycles may be induced by predator/prey type relationships inthe biomass or by relay-like controls in the plant, e.g. pump cycles governed by a levelsensor. The operator must be able to distingnish between a “real” effect and a naturalfluctuation in the system.Baltzis and Fredrickson [24] describe a situation where a pure culture exhibits a limitcycle when grown on two limiting nutrients. Assume we have an organism growing ina chemostat where either S or S2 may be the rate limiting nutrient. At dilution rateD (Figure 3.2), there are two steady states, A and B, depending on the direction fromwhich the equilibrium is approached. If we establish a steady state at B, then decreasethe dilution rate slowly maintaining a quasi steady state, we move along line b — d to b.If we move slightly past b, the system will jump to line c— a. If we increase the dilutionrate, we will proceed along c — a until we hit a. If we increase the dilution rate further,the system will jump to line b — d.A loop is a path that returns to its starting point, not crossing any intermediatevariable twice. Feedback loops come in two varieties: negative and positive feedback. Anegative feedback occurs when an increase in the initiator sets into motion processesthat lead to its decrease. For example, if the dissolved oxygen (DO) level in the tankincreases, the DO controller responds by decreasing the air supply leading to a decreasein the DO. Positive feedback occurs when an increase in the initiator sets into motionprocesses that lead to an increase in the initiator. For example, the effluent solids levelrises due to poor settling sludge. The operator reacts by increasing the recycle rate with7The reader should refer to either Astrom and Wittenmark’s Adaptive Control [20] or Beltrami’sMathematics For Dynamzc Modeling [39]. Limit cycles are fine in the process but not in the dataanalysis.Chapter 3. Cause and Effect 93A and B are steady states at dilution rate DSteady Stateu=D /dS2-j/b Hysteresis0SiFigure 3.2: HysteresisChapter 3. Cause and Effect 94view of drawing the blanket down. The increased recycle rate increases the hydraulicloading on the clarifier, increasing the turbulence causing less sludge to settle out. Ifan effect is in a loop, then its cause could be downstream. For example, if the MLSSin the aeration basin is dropping, the cause may be due to problems downstream in thesecondary clarifier.A common identification problem is identifying the open ioop characteristics of asystem when the process is in a closed ioop. We will not discuss this problem in detailbut rather will focus on three points raised in the review paper by Gustavsson at al [132].The first point is that an inherent feedback system with a rate faster than the samplingrate is part of the open loop system. The second point is that the degrees of freedomin the model cannot exceed that in the data collected on the system. These two pointslimit on-line identification in treatment plants to simple models whose response time isin hours or days rather than seconds. The last point is that one cannot identify the openloop unless one knows the structure of the regulator. In automatic control, this meansthere must be a regulator model, e.g. PID model. In other words, in order to determinehow a process reacts to a change, an operator must know why the change occurred.These limitations may he extended to the case where the operator acts as the controller. In order to understand the system, we must understand the operator. If theoperator makes a change, we must know what caused the operator to act. Assume theoperator changes the recycle rate in response to a change in the effluent solids concentration. Physically, there is no causal connection between the two yet the change in effluentsolids caused the recycle rate to change. In this case, the connection between these twoparameters is through the operator.If an operator is aware how difficult it is to determine the cause of an effect, he willdo all he can to not complicate it any more than it already is. The computer knowsthe plant’s structure and understands the monitoring program. If an operator wants toChapter 3. Cause and Effect 95control an effect, the compnter can identify which control variables will influence theeffect. In other words, the computer provides the “road map” and the operator does the“driving”.3.2.4 Cause To Effect - Fundamental Problem Of Causal InferenceThe Fundamental Problem Of Causal inference forms the basis of our discussion of detection of cause and effect. The problem, posed formally by Paul ‘N. Holland [154],frustrates treatment plant operations as much as it does laboratory research:Because it takes at least two treatments (cause present, cause absent) todetermine an effect, and only on treatment can be applied to an unit at atime, then it is impossible to observe the effect of the treatment on the unit.For example, an engineer wants to know if he can improve the operation of an aerobicsludge digester by controlling the pH. However, he cannot simultaneously observe theeffect of two causes. There are two general solutions to this problem: the ScientificSolution and the Statistical Solution. We use data extracted from a thesis by Andersonas the basis of our example [10].Assume our researcher obtains a second reactor, identical in every way to the first.Also assume that he operates both reactors in such a way that they both reach steadystate. Once he is convinced that the two reactors are identical, he introduces pH controlto the second reactor. By comparing the performance of the two reactors, he can inferwhat the effect of pH control is. Our researcher solved the problem by making theuntestable assumption that the two reactors are exactly same in all ways except for thepH controller. This approach is referred to as the Scientific Solution.Figure 3.3 shows the solids destruction data for the two reactors. Given the noise inthe data, the researcher can not detect a difference between the reactors (cf Jenkins [170]).The Analysis Of Variance (ANOVA) Table confirms this (p = 0.229). To overcome thisChapter 3. Cause and Effect 96problem, our researcher decides to repeat the experiment. He redesigns his work usingthe principles underlying Experimental Design [79] [59] [53]:1. Replication: A single treatment is applied to more than one experimental unit.The experimental error is the variation between replicates. Replication is analogousto Hill’s consistency requirement. For example, our researcher repeats the experiment twice, one with sludge from plant A and second, with sludge from plant B.2. Randomization: The decision about which experimental unit shall receive whattreatment should have a random component. The goal here is to avoid confoundingthe researcher’s bias with the experimental results. In our case, randomization isless an issue because we have only two treatment units within each run.3. Control: The researcher exploits the structure of the experimental units to reducethe debate over what caused what:• Balance: Assign the treatments to maintain symmetry, i.e. avoid confoundingand incomplete blocks if possible. For example, the researcher applies bothtreatments twice, once in each run.• Block: Assign the treatment units such that the units within the blocks areidentical in all ways except for the treatment they receive. For example. theonly difference between run 1 and 2 is the source of the sludge.• Group: Placement of experimental units into a homogeneous group to whichthe treatment is applied. With only two reactors, grouping is not an issue.Chapter 3. Cause and Effect 97Scientific Solution - Aerobic DigestionFill and Draw Reactors140_________ _________r.12O I No Control>%____ ______pH=7 I-D____ __ __________- 800)60ci>o 404—.Cl)_ _ci) 2040C,)C,)Cu-20-40 —_____________________________________—60 I-10 0 10 20 30 40 50 60DayFigure 3.3: Scientific SolutionChapter 3. Cause and EffectTable 3.1: Statistical Solution - Analysis Of Variance98Reactor Average Standard(n=34) Destruction Deviation[kg/d] [kg/d]Run 1: No pH Control 6.8 14.3Run 1: pH Control 18.0 16.7Run 2: No pH Control 10.9 17.0Run 3: pH Control 22.4 26.0Analysis Of VarianceTreatment ProbabilitypH Control/No Control 0.065 Moderately SignificantRun or Sludge Source 0.003 Significant• Unit Homogeneity: The characteristics of the reactors remained the same throughout the experiment.• Constant Effect: The effect of the pH controller on the reactors was the same nomatter when it was applied.The experimental layout is a randomized block design with days within a run nestedwithin the treatment. The ANOVA confirms that the pH controlled reactor performsbetter than the uncontrolled one. We used BMDP © General Mixed Model Of Variance(3V) [298] to analyze the data (see Table 3.1).Reactor StatisticsFigure 3.4 is a plot of the results from the two runs. Just as in the Scientific Solutionexample, the measurement error in the solids test obscures the difference between thereactors. However, the researcher made two assumptions when he laid out his experimentthat will enable him to factor out the sampling error:Chapter 3. Cause and Effect 99Statistical Solution-Aerobic DigestionFill and Draw Reactors120100 Run 1: No Control> 80 Runl:pH=7Run 2: No Control0)____60—_____-______________ci)>_____040L. Run2:pH=7C’)_ ___________a)___200Cl)Cl)-20a—40-10 0 10 20 30 40 50 60DayFigure 3.4: Statistical SolutionChapter 3. Cause and Effect 100The above example illustrates what is required to detect the effect on a cause under theideal conditions in the laboratory. In the first example, the measurement error swampedout the experimental error making it impossible to detect any difference between thereactors. In the second example. our researcher replicated the experiment. In this case,the analysis was able to remove the sampling error from the experimental error anddetect the improvement in solids destruction caused by controlling the pH. If the solidsmeasurements were less precise or the researcher did not take care to prevent a secondcause from acting on the reactors, the experiment would have failed.3.2.5 Treatment Plants and Time Series ExperimentsThe previous section may appear to be an argument for constructing treatment plantswith parallel treatment trains. In a sense, it is becanse one treatment train can act asthe control and the other as the experiment. If the experiment fails, the inventory andcapacity of the control can be used to regain control of the system. However, in manycases, having two trains is too costly. In this case, one must be careful not to upset thesystem with a poor control or optimization decision.The question we need to ask is what if our researcher in the previous section had onlyone reactor? Both the Scientific Solution and the Statistical Solution could be appliedif he can assume the characteristics of the sludge do not change from run to run. Ourresearcher can take one of two approaches. The first approach would be to fill the reactorwith fresh sludge, wait for steady state, measure the destruction, empty the reactor andstart over. In this case, there would be no carry over effect from run to run due to theapplication of a treatment to the sludge. Our researcher would lay out his experimentsuch that the application of the treatments is independent of time [91] [90] [173].The second approach would he to use the same sludge throughout. We refer to thisas a Time Series Experiment [122]. Tables 3.2 and 3.3 list cause and effect archetypesChapter 3. Cause and Effect 101Table 3.2: Time Series Experiment- Cause ArchetypesObject ElementsType { Level, Trend, Stability, Frequency }Duration { Pulse, Square, Step }respectively. In classical experimental design, a cause is a change in version (e.g. fertilizerA or B) or a change in level (e.g. high and low dosage of fertilizer A). In a time seriesexperiment, the cause may also be a trend (e.g. increasing flow), instability (e.g. greaterdistance between peak and low flows) or a new dynamic (e.g. an industrial discharge thatfollows a 2 day cycle). A time series experiment must also be concerned with a cause’sduration. Classical experimental design detects an increase in levels of effects. However,a time series effect may also manifest itself as a trend, an instability or a new dynamic.The experimenter must also he concerned with the elasticity of the effect (i.e. if thecause is removed, does the effect remain) and the lag between the start of the cause andthe start of the effect. For these reasons, a time series experiment is much more difficultto interpret than a classical experiment. The discnssion that follows elaborates on thisincrease in complexity.The independence of experimental nnits is a fundamental assumption underlying experimental design. In the hard sciences, the determination of the independence of theexperimental units is trivial unless the layout is a time series experiment. The reasonfor this is twofold: (1) difficulty in determining steady state; and (2) the danger of causing permanent changes to the biomass such that the response is dependent on previonstreatments.A reactor mnst be at steady state before the researcher can measure its performanceand before he can apply a new treatment. The reason for this is that during the transitiontime between steady states, a reactor’s performance may fluctuate. In some cases, theChapter 3. Cause and Effect 102Table 3.3: Time Series Experiment- Effect ArchetypesObject ElementsType { Level, Trend, Stability, Frequency }Elasticity { Elasticity. Inelasticity }Delay { Delay, No Delay }performance may deteriorate before it improves. For example, Olsson and Chapman [235]introduced a step change into a secondary clarifier in both directions and observed thatthe transient response depended on the direction of the step change.A mixed culture is a complex, nonlinear system [28] [278] [2731.8 For this reason, theresearcher must determine1. if the system is at steady state,2. if there is more than one possible steady state, and3. if the steady state is stable.Portions of a complex system may reach a steady state while other portions maystill be in transition. For example, Kucnerowicz and Verstraete [189] wrote that theirlaboratory scale activated sludge system reached a reasonably consistent steady state interms of substrate removal efficiency, effluent suspended solids and sludge volume indexin 2-4 sludge ages. However, it took the same culture at least 10 sludge ages to establish asteady state in terms of nitrification and endogenous respiration. Turk [296] operated onereactor for over a year and did not achieve steady state with respect to nitrite build-up.8For a survey of wastewater microbiology, the reader should refer to references [127] or [218]9A better term may be stable state as steady state is impossible in a mixed culture system driven bya changing input function.Chapter 3. Cause and Effect 103Table 3.4: Mixed Culture/Mixed Substrate InteractionsType Effect On Effect OnSpecies A Species BMutualism + +Competition— —Neutralism 0 0Commensalism + 0Parasitism + —Predation +—Amensalism 0—A mixed culture’s current state is dependent 011 its history as well as the currentenvironmental conditions [305, pp. 642]:and for that matter almost all models presently in use, state that thebehavior of the biomass-substrate system depends only on the present state,and there is no provision for the past history of the microorganism. It hasbeen recognized for a long time, however, that the observed response of acell population at a certain time instant is the composite result of variousbiological processes that were initiated at different time instants in the pastas a response to instantaneous environmental conditions prevailing at eachparticular time.Thble 3.4 lists seven types of interactions found in a mixed culture/mixed substratesystem [65]. The significance of these interactions is that they make the state of theculture dependent on its history, i.e. non-markovian. For example, the shifts betweencompetition, mutualism and predation depend on the density and age structure.For this reason, an operator may not be able to maintain the system at the currentsteady state. Once the dynamics in the culture start to change, so does the system.An excellent example of this is documented in Turk’s thesis [296]. Turk decreased theoxidation of nitrite to nitrate using the inhibitory effects of free ammonia. The systemChapter 3. Cause and Effect 104operated in what appeared to be a steady state for over 4 months. However, once thebiomass became acclimatized to the ammonia, the system’s state began to change. Turkwas unable to maintain control over the system at this point.A history dependent, nonlinear system may have a number of steady states, bothstable and unstable. For this reason, stability analysis should be the part of any laboratory or modelling research. Takamatsu et al [289] conducted a stability analysis ona simple system with two types of organisms, fioc forming and bulking. The organismscompeted for the same substrate. They identified four equilibrium points: normal state,bulking state, coexistence state and washout state Only the normal state was desirablefrom an operations point of view Bertucco et al [52] examined the stabihty of a reactorwith inhibition kinetics and identified conditions under which the system may becomeunstable. Stability is as important to an operator as is performance. This is why mostoperators prefer to respond to change rather than to induce change in their processes,i.e. they are risk adverse.When a treatment plant operator introduces a change, he is conducting a time seriesexperiment. However, unlike the researcher, the operator must be concerned about thetransient as well as steady state response because he must meet his discharge permit atall times. An operator must be careful not to mistake a transient response as a steadystate. An action in a wastewater treatment plant may have a hydraulic, chemical and/ormicrobiological response. For example, an operator may make a change in his plant thatcauses his performance to improve in the short term but deteriorate in the long term.This may occur if the hydraulic response to the change is positive but the biologicalresponse is negative. If the operator assumes that the system has now reached a stablestate and introduces a second change, the long term effect of the first change will beconfounded with the short term effect of the second change.George Box introduced Evolutionary Operation (EVOP) as a sequence of small twoChapter 3. Cause and Effect loslevel factorial experiments run in a completely randomized design. However, EVOP maybe viewed as a methodology rather than a sequence of a particular type of experimentaldesign. EVOP has four distinguishing characteristics [58] [60]:• EVOP is a sequence of statistically designed experiments.• EVOP searches for an optimum on a response surface under the direction of anEVOP committee, i.e. plant staff.• EVOP is a day-to-day operational procedure.• EVOP couples the staff’s knowledge of the process with the results of a statisticalanalysis.EVOP has a number of variants but the underlying principles remain the same.Springer et al [281] applied Box’s EVOP to an activated sludge plant in Miamisburg,Ohio. Box’s EVOP is probably the simplest statistically based procedure that could beapplied to wastewater treatment plants. Even so, EVOP suffered from the same limitation that most sophisticated schemes face— time. For example, a typical run couldtake at least 12 sludge retention times (SRI). At a ten day SRT, a study that began onAugust 1 would end near the beginning of December. If the operator replicates the run,the study would end near the beginning of April. During this eight month period, theplant would have experienced a number of disturbances whose effect may mask the effectof the operator’s actions.The other alternative is to use intervention analysis. At best this analysis requires 30days of data prior to the change followed by at least 30 days of data following the change.If the operator suspects that the system’s response may lag the change, the post changeperiod would have to be even longer [298]. Most fault detection and change detectionroutines have the same appetite for data [27] [310].Chapter 3. Cause and Effect 1063.3 SynthesisThe determination of cause and effect is the basis for the Scientific Operation of wastewater treatment plants. Scientific Operation is a strategy that Abel Wolman [313] advocatedas early as 1922 and is still being advocated today [228]:The scientific method involves observation and experience, formulation of aworking hypothesis, testing the validity of the hypothesis through experimentation, and the acceptance or rejection of the hypothesis. In other words,the integration of past experience and close observation allows an operatorto formulate a conjecture about the most efficient method to run a piece ofequipment or process. From these conjectures, a working hypothesis may bedeveloped that serves as the basis for further investigation.Once a useful working hypothesis has been developed, sampling and testingprograms may be implemented. Such programs will provide relevant information on the validity of the hypothesis. As a final step, the informationgathered during the optimization study can he used as a measurement of thesoundness of the original hypothesis.The scientific method provides a framework for operation and should not be tied toa particular form of statistical analysis (as is EVOP). Instead, an operation’s strategyshould allow the operator to use an analytical method that fits the data.Although the scientific method forms the basis of experimentation, observation andoperation, operation differs from its counterparts in that the operator must maintaincontrol of his system in real time. For this reason, a computer program that assists theoperator must track cause and effect relationship in the plant so that the operator candecide whether to intervene or not. Given the importance of an intervention, the programmust enable the operator to perform three tasks:• To rule out spurious causes: Prevent low quality data or problems in themonitoring program from being mistaken for a change in the system.Chapter 3. Cause and Effect 107• To evaluate his decisions: Store the reason for a change with the change so thatthe operator can determine whether his reasoning was correct.• To determine when a causal link no longer exists: Warn the operator whena control act will no longer effect the system.Chapter 3. Cause and Effect 1083.4 Summary: Control CycleAn operator controls his process by changing an operator-set parameter (i.e manipulatedparameter) to respond to changes in an effect or disturbance parameter. A control cycleincludes the detection, the introduction, and the monitoring of a change (Figure 9.5).An operator detects change through his monitoring program. First, an operator mustdefine change. Change may be a shift in magnitude, trend, stability or hmits of the value,preference or quality of a measure. Second, an operator must decide when he has enoughinformation to confirm that a change has taken place, i.e qualification problem. If he actstoo soon, he may make a mistake; if he waits too long, he may be too late to make aneffective improvement to the biological system. Third, he must decide in which systemthe change took place. A change may indicate a problem in his information gathering orhis treatment processes. Finally, an operator must decide whether he should intervene.If he is too late, an intervention will just perturb the process. An operator should notintervene unless he absolutely has to.If an operator decides to intervene, he must decide what to change. This decisionis based on the operator’s understanding of the cause and effect relationships in hisprocess. First, if the operator is responding to an effect, he must try to determine itscause. Reasoning from effect to cause is fraught with many pitfalls so the operator mustreason carefully. Once the operator constructs a list of possible causes, he must decidewhether they still exist (i.e did they change back?) and whether their effect is elastic orinelastic (i.e is the effect dependent upon the ongoing presence of its cause?).In order to determine how he should intervene, the operator must work back to findoperator manipulated causes of the effects of these disturbances. Once the operatorconstructs a list of operator set parameters, he must decide if a change in one of themwill effect a change in the process and what this change will be. If an operator setChapter 3. Cause and Effect 109parameter’s relationship with its effect is on its limit of existence (i.e. is the processcontrollable), then it is not the parameter to change. An operator cannot assume thatcause and effect relationships remain static; they do change, and in some cases, cease toexist (i.e. nonmonotonicity). If none of the candidates will do and a change must beeffected, then the operator must intervene catastrophically (i.e. push the process backinto being controllable by introducing a new cause).Once an operator effects a change, he must monitor the change to determine if (1)the change in the operator-set parameter did what he thought it would, (2) the situationthat caused him to act has changed and (3) his process has reached a stable state. Anoperator faces two challenges. First, he must define what “stable state” is and second,he must decide what to do if he has not reached a stable state and a new change startsto act on his system.This control cycle forms the basis of evolutionary operation as each cycle providesinformation on the system. Therefore, the operator should keep the causal relationshipsas simple as possible (i.e. limit the number of changes) and follow his decisions throughto their end result to determine if he reasoned correctly.Apart from detecting change and grouping parameters into cause/effect relationships,the role of the computer is twofold: (1) to link the change that caused the operator toact to the change (if any) the operator made, and (2) to link the change the operatormade to the change’s effect. In Chapter 8, we show how a computer accomplishes thesetasks using a network database management system (without an expert system).Chapter 4Measurement Process[Measurement] does not belong to the modelling of a reactor as such. Nevertheless, if theoretical methods are to be used for estimation and control, itis necessary to model the measurements, i.e. how the measurement outputrelates to the states of the process. [108]An operator obtains information on his plant in two ways: observation and measurement.The difference between the two mechanisms is the role of operator: the operator makesan observation but takes a measnrement. The pnrpose of this chapter is to discuss thisprocess in the context of the operation of wastewater treatment plants.This discussion is broken into five sections:1. Measurement2. Sample Handling And Preservation3. Sampling4. Quality Assurance and Quality Control (QA/QC)5. Measurement Process ModelA datum has meaning becanse we know where it was obtained, how it was obtainedand its quality. A datum represents a point or interval in time depending on whether it isan average or originates from a composite sample. How we manipulate a datum dependson its scale while how much emphasis we place on it depends on the datum’s quality.110Chapter 4. Measurement Process 111The measurement process is as important to understanding the meaning of a datum as isknowing where in the plant the datnm originates. However, control systems and modelscontinue to be built that ignore this process altogether. The result is research that isnever used and systems that are either unreliable or unusable. Recognizing this fact,the Measurement Paradigm discussed later in this thesis draws heavily on the materialdiscussed in this chapter.Chapter 4. Measurement Process 1124.1 MeasurementThis section consists of two parts: a discussion of measurement theory and wastewatermeasurements. Two points can be extracted from this discussion:1. The onus is on the analyst to ensure that an unstable measure’s value is correct.2. A datum’s measure determines, in part, how the datum can be analyzed.4.1.1 Measurement TheoryA measurement is the assignment of symbols to attributes of objects according to a predefined set of rules. A method of measurement consists of specifications of the equipmentto be used, the operations to be performed, the sequence in which these operations are tobe executed, and the conditions under which these operations are to be conducted [164].A measurement process is the realization of a method of measurement in terms of particular conditions that, at best, only approximate the conditions prescribed. A method ofmeasurement may he viewed as a template (or a model) which when imposed on realityextracts information. The degree to which the template fits determines the utility of theresult [111, pp. 27]:Measurement then is seen as the construction of a model of some property ofthe world. Like all modelling it involves the establishment of a correspondencebetween an empirical relational system (the world) and a formal relationalsystem (the model), so that one can be said to represent the otherTherefore, the meaning of a measure is determined by the degree of correspondencebetween the conditions under which a method of measurement was derived and theconditions under which the measurement was made [103].A measure may be expressed using any symbol to which we can attach meaning. Weattach meaning by comparing a measure against a scale. In the hard sciences, the1An electrical circuit is a hard system while the politics in the Middle East are a soft system.Chapter 4. Measurement Process 113predominant expression is a number; preferably accompanied with an indication of thenumber’s precision and accuracy. Numbers are a convenient and an “unambiguous wayof delimiting and fixing onr ideas of things” [111]. Alternate forms of expression includelinguistic variables and fuzzy numbers.Rules define how symbols are assigned to attributes. Different rules provide differentmeasures of the same attribute. For example, substrate concentration is an attribute ofan effluent sample. This attribute cau he measured using the Chemical Oxygen Demand(COD) test or the Biochemical Oxygen Demand (BUD) test. The method by whichthe attribute is measured in each test differs, resulting in different numbers for the samesample. The mapping of an attribute into a measure should be homomorphic, meaningthe structure of the attributes appear unmodified in the measure of the attribute, e.g.high substrate concentration means both a high COD and BOD value.Measures of an object’s mass and a person’s Intelligence Quotient (IQ) are at oppositeends of the spectrum of possible measures. The former is based on verifiable scientifictheory and is easy to execute properly. As the measurement of mass is standardized, measures conducted anywhere in the world can be compared. In contrast, because scientistscannot agree on what intelligence is, they cannot agree on what the IQ test measures.Typical of measures in the soft sciences, the correct execution and interpretation of an IQtest requires a high level of training. Consequently, these measures fall under a greaterdegree of scrutiny than measures in the hard sciences. However, until a better test of intelligence is found, IQ is as important to researchers as the measurement of mass because“measurement is a necessary bridge between the real world and our ability to investigateits attributes” [110, pp. 71].Chapter 4. Measurement Process 114Statistical ControlIn order to qualify as a specification of a measurement method, a set of instructions mustbe sufficiently definite to insure statistical stability (of repeated measurements) [103].W. A. Shewart, an early proponent of this concept, viewed a measurement process asbeing in statistical control when samples, cumulated over a suitable time interval, givea distribution of a given shape, time after time. Under these conditions, unaccountedvariation is random in nature [4], [106] [307]. When this is the case, the arithmetic meanof a set of measurements approaches a limiting value which may or may not be the truevalue. Eisenhart expresses this fact in the Postulate Of Measurement [103, N. E. Dorsey,quoted on page 168]:The mean of a family of measurements- of a number of measurements for agiven quantity carried out by the same apparatus, procedure and observer-approaches a definite value as the number of measurements is indefinitelyincreased. Otherwise, they could not be properly called measurements of agiven quantity. In the theory of errors, its limiting mean is frequently calledthe true value, although it bears no necessary relation to the true quaesitum,to the actual value of the quality that the observer desires to measure Letus call it the limiting mean.If an analyst can maintain statistical control during the measurement process, then themeasurement method is considered to be statistically stable.In practice, the impetus to measure comes from the need to know, not the availabilityof acceptable measurement methods. Consequently, not all measures used in wastewatertreatment plants are statistically stable. For example, the BOD5 test, which is usedby regulatory authorities and plant operators, was dropped by ASTM from its list ofstandardized tests because it is statistically unstable. ASTM argued that the test gavedifferent values for the same substrate depending upon which seed was used. Becausethere is no way to characterize the seed bacteria, there is no way to determine the test’saccuracy. When an unstable test is used, the analyst cannot solely rely on the properChapter 4. Measurement Process 115execution of the method to ensure validity. Instead, the analyst must assume moreof the responsibility for determining when the method produces a valid result (ResultEvaluation Principle) [103, pp. 163]:To the extent that complete elimination of the subjective element is not alwayspossible, the responsibility for an important and sometimes difficult part ofthe evaluation is shifted from the shoulders of the statistician to the shouldersof the subject matter expert.In wastewater treatment, this is why a redox measure is much more difficult to interpretthan a dissolved oxygen reading.Scale Of MeasurementA measurement scale may be one of four types: nominal, ordinal, interval and ratio.The minimal requirement of a scale is that the scale consistently assigns a symbol toa particular attribute. Consistent assignment is the basis of the occurrence. If thescale forms a lattice, then a second property, order, is introduced. In order to performlinear transformations in the attribute values, the scale must he delineated with a unitinterval. To perform nonlinear transformations, the scale’s zero must coincide with theattribute’s zero in the real world. Table 4.1 describes the basic empirical operations andthe mathematical group structure for each scale. The table is adapted from [111. pp. 26]and [73].The type of scale partly determines what type of statistical measures and methodologies are appropriate. Parametric tests should only be applied to data from interval andratio scales [255, pp. 133] because one should not use statistics that could be distortedby admissible transformations of the scale values [111]. Table 4.2 lists examples of statistical measures appropriate for measurements made on various scale types. The tableis adapted from a similar table in [111, pp. 31].Chapter 4. Measurement ProcessTable 4.1: Measurement Scales116Scale Operation Group ExampleNominal Determination Permutation SupernatantOf Equality Group AppearanceFor Unordered p = f(x) (see Appendix D)Categorical where f(x)Variables means anyone-to-onesubstitutionOrdinal Determination Isotonic Mohs Hardnessof Greater Group Of Mineralsor Lessy=f(x)Ordering But where f(x)No Implication means anyOf Distance increasingBetween Scale monotonicPositions functionInterval Determination Linear Nephelometricof Equality Affine Method Ofof Intervals Group Turbidityor of MeasurementDifferences p = ax + bwhere a > 0EqualDifferencesBetweenSuccessiveIntegers ButZero PositionArbitraryRatio Determination Similarity MassOf Equality GroupRatiosp = cxThe Highest where c> 0MeasurementWhere One CanComp areDifferencesIn Scores AsWell As RelateMagnitudesChapter 4. Measurement Process 117Table 4.2: Appropriate Statistical OperationsThe simplest type of scale is a nominal or classificatory scale. Attributes are comparedwith archetypes and those that match the archetype are assigned the same symbol asthe archetype. For example, an operator determines the Appearance of the clarifiersupernatant by comparing a sample with six archetypal conditions 2: bulking, clumping,ashing, straggler-floc, pin-floc, and clear. The supernatant appearance is recorded asbeing the archetypal condition that is most indicative of the sample’s condition.Given that the only statistical property these data possess is commonness of occurrence, the only appropriate summary statistics are frequency statistics such as the mode.An ordinal scale is a nominal scale where the sets that form the scale can be ranked.When measuring, attributes are compared with the archetypes that form the scale. If anattribute matches an archetype, the attribute is given the archetype’s symbol. BecauseScale Measures Measures Measures SignificanceOf Of Of TestsLocation Dispersion AssociationNominal Mode Information,H Information ChiSee Appendix B Transmitted SquareContingencyCorrelationOrdinal Median Percentiles Rank Order Sign TestCorrelation Run TestInterval Arithmetic Standard Product- I-testMean Deviation Moment F-testCorrelationRatio Geometric PercentMean VariationHarmonicMean2see Table D.2, Appendix DChapter 4. Measurement Process 118Table 4.3: R.atio And Interval Temperature ScalesScale Reactor A Reactor B Ratio A: BCase 1Celsius 20 C 40 C 1:2.00Fahrenheit 68 F 104 F 1:1.54Rankine 528 R 563 R 1:1.05Kelvin 293 K 313 K 1:1.05Case 2Celsius -162 C -51 C 1:3.13Fahrenheit -260 F -60 F 1:4.35Rankine 200 R 400 R 1:2.00Kelvin 111 K 222 K 1:2.00the scale is ordered, if an attribute does not match any of the archetypes, then it mustlie between two archetypes. If this is the case, then the attribute is given a symbol thatindicates between which two archetypes it lies. Mohs scale of hardness of minerals usesan ordinal scale. The hardness of A is ranked higher than B if A can scratch B, hut Bcannot scratch A.Both commonness of occurrence and rank are important statistical properties of ordinal measures. The appropriate summary statistics are frequency and rank statistics(e.g. median, maximum, minimum, quartile).If the size of the interval between archetypes is known, then when an attribute isbetween two archetypes, a symbol can be assigned that expresses where in the intervalthe attribute lies. If the choice of zero on the scale is arbitrary, we refer to this as aninterval scale.Unlike the previous scale, this scale is divided into units of equal and known size ( Arithmetic operations can be applied to differences between measures sharingthe same interval scale as long as the rank and relative difference between measures isChapter 4. Measurement Process 119preserved. This limits arithmetic manipulations to linear transformations. For example,the arithmetic mean is a valid statistic but not the geometric or harmonic mean.An interval scale becomes a ratio scale when measures of real world attributes haveboth ranking and interval properties and a natural zero. The natural zero implies thatany two intervals on the scale have comparable magnitudes determined by the numberof times one contains the other.Celsius and Fahrenheit are measured on an interval scale and Rankine and Kelvinare measured on a ratio scale. Table 4.3 lists the temperature of two reactors. In case1, the temperature of reactor B is 40 C and A is 20 C giving us a ratio of 2 to 1. Noneof the other measures give this ratio. However, in case 2, the temperature of reactor Bis 400 if and A is 200 if. The only other measure that gives this ratio is degrees Kelvin.The reason for this is that the zero of both the Fahrenheit and Celsius scale is differentfrom the zero of the Rankine and Celsius scale which is absolute zero. Absolute zero isdefined as the temperature at which the thermal energy of random motion of particlesof a system in equilibrium is zero. In other words, absolute zero is a ‘true’ zero. This iswhy we cannot say that reactor B is twice as hot as reactor A based on either a Celsiusor Fahrenheit measure (e.g Case 1).4.1.2 Operational MeasuresMany of the measures that an operator uses to operate his treatment plant are not standardized. A measure is non-standard if (1) the operator cannot determine its accuracy(e.g. BOD5) or (2) the measure’s interpretation is context dependent (e.g. redox). Inboth cases, the operator preserves the quality of these data by (1) applying the measureto the process in a consistent manner and (2) establishing a method to validate thesedata using other measures conducted on the process.The operator also uses derived measures to run his process. Derived measures, suchChapter 4. Measurement Process 120as F/M ratio and SRT, are statistics. A statistic’s interpretation is based upon a model.If the model is meaningless, then so is the statistic. The Mean Cell Residence Time(MCRT) is defined as the average time an organism remains in the system under steadystate conditions. In this case, the growth rate is a function of the dilution rate. Ina chemostat containing a single pure culture, the MCRI is equal to the inverse of thedilution rate. If a solids separation device and a recycle line are added, the MCRT isdefined as the ratio of the number of organisms in the system to the net loss of organismsfrom the system.SRI (Solids Retention Time) is the extension of this concept to a wastewater bioreactor. SRI is a noisy and misleading operational parameter for three reasons: (1) atreatment plant cannot reach steady state conditions as defined by microbiologists, (2)a wastewater bioreactor is a fiocculating mixed culture and (3) the raw measurementsused to calculate SRI are noisy. Vaccari [300] [299] derived an alternative measure, Dynamic Sludge Age (DSA), based on an age distribution function. DSA is founded on amore realistic model and appears to act as a smoothed SRI. However, both measuresstill suffer from noisy raw measures. For example, Figure C.1 (Appendix C) shows thatif the standard deviation of the solids test is 10%, then a measured MCRT of 10 daysindicates that the “real” MCRT is somewhere between 6 and 14 days . For this reason,data analysis should be conducted on raw values rather than derived values.Measures used to monitor and control biological systems have been the subject of anumber of reviews, some of which are listed below:Meyer et al [210] published a review of microbial growth control measures along‘ hen suddenly increased loading results increased sludge wasting and wasting is adjusted tomaintain a removal of a constant fraction of the mass. The traditional sludge age calculation predictsconstant sludge age. In reality, the sludge initially gets younger due to the increased amount of newsludge present.” [300]4The derivation of this plot is in Appendix CChapter 4. Measurement Process 121with a discussion of growth control strategies.• Stephanopoulos et al [285) [257] [130] [258) discussed on-line instrumentation andbioreactor identification in a four paper series.• Wang and Stephanopoulos [304] reviewed real time digital-computer applicationsto fermentation control with an emphasis on using on-line instrumentation.• Briggs and Grattan [62] [63] provide a survey of instrumentation used in the UnitedKingdom.• The EPA published a manual on Wastewater Treatment Plant Instrumentationthat covers installation and maintenance of conventional instrumentation [205].The developments of measures to control wastewater treatment plants and other biological systems is ongoing.5The author found that most of this material was published in IAWPRC specialty conferences,IFAC specialty and general conferences, ISA proceedings and American Automatic Control Conferenceproceedings.Chapter 4. Measurement Process 122Table 4.4: Effectiveness Of ATU As Effluent BOD5 Sample Nitrification InhibitorPercent Decrease In BUD5 When ATU Increased From 0.5 to 2.0 mg/i ATUType Of PlantStatistic Activated Biological BothSludge FiltersMaximum 88% 27% 63 %Upper Quartile 58 % 19 % 46 %Median 46 % 14 % 39 %Lower Quartile 30 % 10 % 33 %Minimum 6% 6% 28%Number 29 9 104.2 PreservationA sample is preserved to protect its integrity between collection and analysis. Preservation techniques attempt to minimize physical changes caused by volatilization, adsorptionand absorption, diffusion and precipitation, and chemical changes caused by chemical reactions, photochemical reactions and microbiological degradation. The changes are minimized with the use of temperature control, type of sample container used and chemicaladdition [240] [247] [42].The validity of the preservation method is usually tested as part of a laboratory’sQA/QC procedure. For example, Tyers and Shaw [297] looked at the effect of addingallythiourea (ATU) to a BOD5 sample to inhibit nitrification. Table 4.4 shows thepercent decreases in the BUD5 concentration when the ATU is increased from 0.5 mg/lATU to 2.0 mg/l ATU.Chapter 4. Measurement Process 1234.3 SamplingBy definition, a sample is a fraction of a stream. A fraction may be delineated by theprocesses of extraction or splitting, or by the interaction between the characteristics ofan in situ measurement device and the system. The size of the fraction must satisfy twocriteria:• The size must be manageable.• The size must allow the analyst to infer from the fraction’s characteristics thecharacteristics of the stream at the time and place the fraction is delineated, i.e.the sample is representative.The aim of sampling is to obtain relevant information on an object, e.g. a wastestream. The analyst must translate a request for information into a monitoring program.This program can be optimized by considering the purpose of the program, what theprogram is seeking information on and how this information will be obtained. The moreprior information that is available on these items, the better a plan can be developed togather the needed process information.By using the theoretical and practical work from other disciplines, the analyst canimprove the monitoring program. Two disciplines that play an important role in thediscussions in this section are Statistical Sampling Theory and Signal Theory. The application of knowledge from these disciplines to the design of treatment plant monitoringprograms is “tricky” because the conditions in a treatment plant are different from theconditions under which both these disciplines were developed. However, these disciplinesprovide a logical framework that can be used to solve the problems at hand.Most objects hold in themselves the proof of their value. Their value can be estimated by measurements carried out on the object itself, i.e. all a good jeweller needs toChapter 4. Measurement Process 124determine the value of a diamond is the diamond itself. This is not true of a sample.The economical value of a sample is its representativeness, i.e. the closeness with whichthe composition of the sample reflects the composition of the stream from which it wasdelineated. The only way to distinguish a representative sample from a useless sampleafter the fact is to examine how the sample was obtained. Over the last decade, manystudies have shown that poor sampling is a commonly identified cause of poor data. Nomatter how elaborate the analysis made on a sample is, the resulting datum is of novalue or worse, misleading, unless the sample is representative [82, pp. 171]:Sampling in environmental analyses is akin to dish washing in analyticalchemistry. If the dishes are dirty, all analytical activity thereafter is fornought. Similarly, nothing is gained from the chemical endeavor if the samplesare not representative of the environment from which they are taken.The discussion that follows this section is broken into four parts. The first sectiondiscusses the EPA’s definition of a sampling protocol. The second section discusses oneaspect of this protocol, the sampling plan. The third section discusses why a sample istaken and the final section lists a number of sampling references.4.3.1 Sampling ProtocolA sampling protocol is a thorough written description of detailed steps and proceduresinvolved in the collection of samples (see Table 4.5). The protocol forms part of a QualityAssurance Project Plan and is usually included in a project’s proposal and report [233].The three requirements for a sampling protocol as established by the American ChemicalSociety Committee on Environmental Improvement, are listed below [25]:1. A proper statistical design that takes into account the goals of the study and itscertainties and uncertainties. This item discussed in the next section.Chapter 4. Measurement Process 125Table 4.5: Outline Of A Sampling ProtocolPurpose SubelementsAnalytes of interest Primary and secondary chemical constituents and criteria for representativenessLocations Site, depth and frequencySampling points Design, construction and performanceevaluationSample collection Mechanisms, materials and methodologySample handling Preservation, filtration, and field controlsamplesField determinants Unstable species and additional samplingvariablesSample storage and transport Preservation of sample integrity2. Instruction for sample collection, labelling, preservation and transport to the analytical facility.3. Training of personnel in the sampling techniques and procedures specified.A sampling protocol is an expression of professional accountability because the protocol enables a researcher’s peers to evaluate the quality (particularly the representativeness) of the data. The objective of requiring a protocol is to prevent a researcherfrom sampling until the questions to be answered have been determined and properlyframed [9]. This is evident in Green’s methodology which is one of the best set of guidelines for developing a sampling plan [129](see Table 4.6).Chapter 4. Measurement Process 126Table 4.6: Green’s Sampling Design PrinciplesI. Be able to state concisely to someone else what question you are asking.Your results will be as coherent and as comprehensible as your initialconception of the problem.2. Take replicate samples within each combination of time, location andany other controlled variable. Differences among can only be demonstrated by comparison to differences between.3. Take an equal number of randomly allocated replicate samples for eachcombination of controlled variables. Putting samplers in representativeor typical places is not random sampling.4. To test whether a condition has an effect, collect samples both wherethe condition is present and where the condition is absent but all else isthe same. An effect can only be demonstrated with a comparison witha control [cf Fundamental Problem Of Causal Inference].5. Carry out some preliminary sampling to provide a basis for evaluationof sampling design and statistical analysis options. Those who skip thisstep because they do not have enough time usually end up losing time.6. Verify that your sampling device or method is sampling the populationyou think you are sampling, and with equal and adequate efficiency overthe entire range of sampling conditions to be encountered. Variation inefficiency of sampling from area to area biases among-area comparisons.7. If the area to be sampled has a large-scale environmental pattern, breakthe area up into relatively homogeneous subareas and allocate samplesto each in proportion to the size of the subarea. If it is an estimate oftotal abundance over the entire area that is desired, make the allocationproportional to the number of organisms in the subarea.8. Verify that your sample size unit is appropriate to the size , densitiesand spatial distributions of the organisms you are sampling. Then estimate the number of replicate samples required to obtain the precisionyou want.9. Test your data to determine whether the error variation is homogeneous, normally distributed, and independent of the mean. If it is not,as will be the case for most field data, then (a) appropriately transform the data. (b) use a distribution-free (nonparametric) procedure,(c) use an appropriate sequential sampling design, or (d) test againstsimulated H0 data.10. Having chosen the best statistical method to test your hypothesis, stickwith the result. An unexpected or undesired result is not a valid reasonfor rejecting the method and hunting for a better one.Chapter 4. Measurement Process 1274.3.2 Sampling PlanA sampling scheme consists of at least five components 6:• Sample Volume: how much sample to take• Sample Delineation And Location: what defines a sample• Sample Selection: random or systematic• Sample Frequency, Number, and Length of Study: how many samples should betaken and how often should they be taken• Sample Type: grab sample, composite sample or probe readingSample VolumeThe population consists of the totality of the observations with which we are concerned,i.e. the effluent over the last 24 hours. The population must be defined before we candesign a sampling scheme because a sample is a subset of the population. The samplingobjective is to obtain a precise and accurate picture of the population’s characteristics.For this reason, the sample size should be large enough that it averages over the compositional heterogeneity yet is still small enough to be handled. Figure 4.1 shows therelationship between sample volume and the precision of the determination [82]. Thesamples were taken from the euphotic zone of a lake using a Van Dorn bottle outfittedwith a graduated steel tape. The 5 Liter volume was the smallest size that gave similarresults for both sides of the boat.61f the reader is unfamiliar with the terminology used in sampling theory, he could refer toASTM D 3370-82 [2], ASTM D 4392-87 [4] and/or ASTM D 4375-84 [3]. The terms pertinent tothis thesis are defined in the glossary (Appendix B).Chapter 4. Measurement Process0C)E100Effect Of Sample VolumeMean and Standard Deviation128., Sfl-7,.’‘I—,60 —50 —:: F FL202L-S 2L-N 5L-S 5VolumeL-N 20 L-S 20 L-N 30 L-S 30 L-Nand Side Of BoatFigure 4.1: Samples Taken On Linsley PondChapter 4. Measurement Process 129Sample Delineation and LocationSample delineation depends on the type of sample the analyst wishes to take:• Increment Sample: An increment sample is a portion of a stream at a giveninstance in time. The analyst must decide where in the stream to take his sample.For example, if the analyst is sampling the aeration basin for MLSS, he would takethe sample in a well-mixed zone away from the edge of the basin.• Split Sample: A split sample is the total volume of the stream over a time interval,i.e. the stream is diverted into the sample container. The analyst must decide whenand how much of the stream to divert. For example, if the analyst wants to samplethe pickle liquor addition during a pump cycle, he should divert the flow at thestart, middle and end of the cycle to form a composite sample. Split samples arecommonly used in the mining and other solids handling industries.• Probe Sample: A probe sample is the volume over which the probe measures.This volume depends on the characteristics of the probe and what is being sampled. For example, a probe in a quiescent basin samples a smaller volume thana probe in a turbulent tank. In this case, the analyst must position the probe toobtain a representative measure of the tank’s contents. One way to determine theapproximate size of this sample is to measure the concentration profile of the tankand plot out the zone around the probe that is within two standard deviations ofthe probe’s measurement error.If more than one probe is used, then the sample value may be either the mean ormedian probe reading. This is the case when an operator attempts to improve datareliability by using two or more probes rather than a single probe [66] ‘.7lnstrument redundancy does not eliminate the need for a QA/QC program. For a complete discussionof DO probe redundancy aud failure, refer to [66]Chapter 4. Measurement Process 130Sample SelectionIn standard sampling theory, the rule is that samples must be chosen at random unlessthere is a very good reason for doing otherwise (see Table 4.7). The reason for this is thata researcher does not know everything about the population being studied. Thereforeat some point, he must make some assumptions. Statistical Sampling Theory reducesthe consequences of a faulty assumption [9] by forcing the effect of what is unknown tomanifest itself as a random variable. Consequently, at some level, sample selection mustbe random [129, pp. 10]:It is critical that at some level the sampling be random or most statisticaltests will be invalid. The assumption of independence of error is the only onein most statistical methods for which violation is both serious and impossibleto cure after the data have been collected. Truly random allocation of samplesis the necessary and sufficient safeguard against this violation.A researcher may decide to restrict or eliminate random selection given prior knowledgeof the population. However, if this knowledge later proves to be inaccurate, so may theend results.In most situations, researchers control sample selection while still retaining a randomcomponent. The case when the selection is completely controlled is referred to as Systematic Sampling. The main advantage of systematic sampling is that the program is easierto execute. The disadvantage is that the system used to select the samples may appear inthe results as being a characteristic of the system. For example, when evaluating a TotalOxygen Demand Meter, researchers observed that the sample-based two month averageof a systematic sample of the plant’s effluent oxygen demand could be out by as muchas 35% depending on the time of day the sample was taken [117]. A similar situationcan occur if the sample time coincides with equipment cycles [105]. Despite its dangers,systematic sampling is commonly used because “to some extent, good judgement is asubstitute for money” [57].C. Cochran provides a list of circumstances under which it is safe to use systematicsampling [78, pp. 229]:Chapter 4. Measurement Process 131Table 4.7: Sample Selection MethodologySelect Units Method Description CommentUsing prior Haphazard No prior information available, so Population must be homogeneousinformation or Arbitrary any sample will do or parameter estimates will be bion ased. The method is used primarpopulation ily in pilot sampling studies, i.e.look at whatever data is available.Judgement Sampler knows population very Same as above. This method iswell so he selects which sam- often used when the sample itselfple will be indicative of the is important, i.e. check suspiciouspopulation. cars at the border.Search Sampler has historical infor- The sampling goal is to findmation on the location being something in a population whichsearched for. is different from studying thepopulation as a whole, i.e. findan illegal connection in a sewer.Using a reg- Systematic The sampler chooses the samples Systematic sampling is used toular pattern systematically. map across space and/or time. Ifor system the population exhibits a cyclicalpattern, the sampler must he surethe system’s resolution is highenough to detect the pattern.Using a ran- Probability The sample selection process has Statistical Sample Theory dealsdom number a random component. with probability sampling almostexclusively. Random selection ofsamples is the preferred methodology, unless there is a very goodreason for doing otherwise.Chapter 4. Measurement Process 132• Population is relatively homogeneous: The population exhibits no systematicpatterns that could bias the program’s results. Over a day, the nature of the fishentering a spawning channel would not change dramatically nor would they exhibitany sort of systematic pattern. Therefore, examining every tenth fish would beequally as effective as sampling at random.• Population is stratified and an independent sample is taken from eachstrata: The population consists of strata. Although the average values betweenstrata may differ, the variation within the strata is random and small when compared to the variation between strata.• Population is stratified but the sampling frequency is high enough tocharacterize the strata: This situation is like the previous one except the variation within the strata is significant. However, the number of samples taken on eachstrata is sufficient to ensure that the results are not biased.• Population is systematically stratified and an error estimate is not required: This situation is similar to the previous example. However, the form of thevariation within the strata is not random and is known, thus the judicious choiceof sample time and location provides a reasonable estimate of the strata’s average.Systematic sampling is also used when subsequent analyses require a systematic dataset (i.e. Time Series Analysis). If systematic sampling is used instead of random sampling, the onus shifts from the statistician to the researcher to prove that the results arerepresentative. This is another manifestation of the Result Evaluation Principle. For thisreason, samples whose results may be used in a court case or to determine complianceshould be selected at random.Chapter 4. Measurement Process 133Sample Frequency and Number Of Samples TakenA researcher uses foreknowledge of the system’s structure to decide on the sample typeand frequency. The Statistical Sampling Frequency is determined from estimates of theunderlying variability and the desired precision. If the system exhibits a regular cyclical behaviour, the researcher should also consider the Signal Sampling Frequency. TheSignal Sampling Frequency should be at least three times faster than the fastest cyclicalcomponent in the system being sampled. The larger of the two sampling frequencies governs. The other considerations that must be examined when determining the samplingfrequency are mentioned below [50, pp. 85]:For selection of an appropriate sampling interval, a careful analysis must takeinto account the natural variability of the process, the precision of controlthat is possible, the required uniformity of performance, the accuracy of measurements used to judge performance, the penalty for failing to detect badperformance, and the cost of sampling, laboratory work, and data analysis.Sample TypeIn pollution control, there are three types of samples that deserve mention: in-situ,grab and composite. An in-situ sample is one where the sample is not removed fromthe population. For example, a redox probe measures the condition of the biomass inthe vicinity of the probe. An in-situ sample is used whenever a parameter cannot bemeasured any other way or when it is cheaper or more convenient to measure in-situ. Agrab sample is one where the sample is removed from the population and the analysis isconducted on a portion of the sample. Berg lists three situations where a grab sampleshould be used [42, pp. 19-21]:1. Sampling batch flows (i.e. on delivery of a tank of waste pickle liquor to a treatmentplant using the liquor for phosphorus removal)2. Sampling parameters that may change when stored or composited (i.e. DissolvedOxygen, PH)Chapter 4. Measurement Process 1343. Sampling a parameter that changes very slowly making a composite unwarranted(i.e. MLSS is an aeration basin).A composite sample is a sample where a number of grab samples are pooled priorto analyses. The volume and time of sampling of the grab samples determines how thesamples are pooled. A composite sample is routinely used to screen populations, toreduce the variability in a sample and to facilitate an analytical procedure [116].The advantage of a composite sample is that it allows the operator to uncouple thesampling frequency from the analytical frequency. For example, assume an operatorwishes to estimate the plant’s daily load. Also assume that the variation in the loadover the day is unimportant but significant. The ideal approach is to take and analyzeenough grab samples within each day to characterize the variation completely, then filterout the within-day dynamics from the data set. A more pragmatic approach wouldbe to composite the grab samples and conduct one analysis each day. Time-basedcompositing effectively filters out frequencies up to the analytical period while distortingthe contributions of components with a period between the analytical period and fourtimes the analytical period.Composite samples reduce analytical costs and reduce inter-sample variance. However, the disadvantage of composite samples warrants some examination:1. Dilution below detection limit: Under some circumstances, compositing maydilute the parameter of interest to the point it is undetectable.2. Loss of representativeness: The parameter of interest should behave like aconservative substance. This excludes parameters that are in equilibrium withother phases such as dissolved oxygen, pH, free cyanide ion or dissolved heavymetals [214].Chapter 4. Measurement Process 1353. Loss of information and possible bias: Information is lost because the grabsamples that form the composite are not measured individually. Bias results fromthree sources:• The nature of the composite function: The simplest composite sampleis a time based composite - samples of equal volume are taken at fixed timeintervals. A more complex composite sample is a flow-based composite - thesize of the aliquot is a determined by the current flow rate. In this case, thestatistic is a ratio. The variance introduced by imperfect knowledge of theflow affects the variance and distribution of the load [260].• The nature of the statistic: A time based composite estimates the meanconcentration while a volume composite estimates the mean load. Becausethe flow and concentration both vary with time, the load estimated from atime-based composite and the mean concentration estimated from a volumebased composite are biased.• The relationship between the analytical frequency and the stream’sdynamics: Bias may result if the dominant dynamic in the stream lies between the Nyquist frequency and the evaluation frequency. The evaluationfrequency is one quarter the analytical frequency.Sample IntervalA sample represents an interval in space or time. The optimal size of this interval isdetermined by the relationship between the characteristics of the sample and of thepopulation.Chapter 4. Measurement Process 1364.3.3 Sampling ViewpointsWe can view sampling from three perspectives:1. Principal and Objectives: A sample is taken on a system to either describe,monitor or control the system. A descriptive measure may be infrequent and noisyas long as it provides the operator with the information he needs, while a controlmeasure must be precise, accurate, fast and frequent if the operator is to maintaincontrol of his system.2. Object and Its Properties: A sample must be representative. Therefore, thesampling process should not modify the sample’s characteristics. In some cases, acomposite sample may be desired but a grab sample necessary because the attributeof interest is in equilibrium with the atmosphere, i.e. pH.3. Analyst and Analytical Procedure: The sampling method depends on theanalytical method. For example, if the attribute is near the detection limit of theanalytical method, then a composite sample should not be used.We will elaborate on the first perspective as the previous sections dealt with the othertwo items in sufficient detail.Principal and ObjectivesA sample is taken for one of three reasons:1. To Describe2. To Monitor3. To ControlChapter 4. Measurement Process 137A description is a measure of the properties of an object to determine the object’saverage characteristics along with some measnre of the precision of these characteristics.The researcher mnst be aware of the type of estimator nsed if he is to nse the properprecision estimator.Monitoring is the measurement of the changing properties of an object in order todetect deviations from a preset value that are considered too large. This is also referred tothis as Threshold ControL The sampler increases the sampling frequency as the propertiesapproach their bounds. In other words, the sampling frequency should be a function ofthe previous value. In this case, the most important sample is the most recent one.We use this type of monitoring when we want to detect abnormal conceutrations, todefine peaks, to detect trends, to detect when a parameter exceeds a limit, or to followa parameter as a unit process approaches failure.Control is the measurement of the changing properties of an object to detect a significant deviation from a set-point aud to adjust the system to correct for these deviations.The difference between control and monitoring is the degree of intervention required tomaintain stability. The turn-arovnd-time of a sample must be faster than the dynamicbeing controlled. If a correction is applied after the dynamic passed through the system,then the correction itself may become a cause of deviation. Turn-around-time should notbe confused with dead-time. Dead-time is a lag in the system while turn-around-time isa lag in the measurement process. Control requires precise, accurate and fast measures.Chapter 4. Measurement Process 1384.3.4 ReferencesA standard reference in Statistical Sampling Theory is W. G. Cochran’s text “SamplingTechniques” . However, statistical sampling theory does not consider sampling and reconstruction of a time series, particularly Shannon’s Sampling Theorem. For this reason,Cochran’s text is complemented with short sections from other time series and automatic control texts [18] [61] [284] [133]. Three texts discuss the application of StatisticalSampling Theory areas which share some characteristics with wastewater treatment:• R. 0. Gilbert’s “Statistical Methods For Environmental Monitoring”: monitoringsurface waters, hazardous wastes and ground water.• R. H. Green’s “Sampling Design and Statistical Methods For Environmental Biologists”: ecology.• P. M. Gy’s “Sampling Of Particulate Materials”: mineral processing, sewage sludgesand solid wastes.The EPA published two sampling manuals for wastewater treatment plants which focuson the methodological aspects of sampling [267] [42].Montgomery and Hart [214], Ellis and Lacey [105] and Katema [182] published sampling reviews. Additional information can be found in chapters 1 and 2 in Examination OfWater For Pollution Control [204] [311]. The balance of the literature discusses specificsampling concerns, some of which are listed below:• Volume Of Sample [82]: The volume of a sample is a compromise between asample being representative and a sample being manageable.5Two of W. 0. Cochran’s texts are considered standard references in applied statistics: SamplingTheory [78] and Experimental Design [79].Chapter 4. Measurement Process 139• Validity Of Composite Samples [2.59], [261], [260]. [116]: A composite sampleloses its advantage over a grab sample when the composite function is noisy.• Mass Discharge Estimation [55]: A stratified random design using a ratio estimator to estimate baseline pollution to the Great Lakes.• Sampling Sludges and Slurries [243], [81]: Application of Gy’s sampling theory.• Scheduling Of Sample Analysis In A Laboratory [169]: If the objective isprocess control, then the most recent sample should be processed first.• Design Of Sampling Plans When Autocorrelation Is Present [221], [275],[276]: The methods exploit the antocorrelation function sample to reduce the number of samples taken.• Sample Collection Routing [266]: The Lincoln Division Of Anglian Water mustcollect samples from over 600 locations. The Division optimized the collectionroutes and was able to collect over 15000 more samples without an increase instaff.• Ecological Sampling [9]: The distribution of a population is as important as themean because most ecological populations are not normally distributed.• Sampling Intervals For Compliance [23], [215], [279], [117]: Optimize the ability to enforce compliance while controlling the cost of monitoring.• Sampling Protocol [25], [233], [101]: The EPA requires a sampling protocol aspart of the Quality Assurance Project Plan.9Autocorrelation of a time series indicates that information is carried over from observation to observation. [201]Chapter 4. Measurement Process 140Automatic Samplers [229], [197], [187]: Reviews of automatic sampling equipment.The bulk of the literature on sampling is published either in ASTM or EPA documentsor in “Water Pollution Control (GB)”. Perhaps, what is more interesting is what isnot in the literature. Apart from M. B. Beck’s work, little reference is made to theimpact of sampling (and measurement) on modelling and automatic control in wastewatertreatment. This is not the case in the biotechnology and automatic control fields.4.4 Quality Assurance/Quality Control4.4.1 Concern For QualityThe data obtained from wastewater treatment plants is used to determine compliance,to control and optimize the process, and to provide information leading to new designsand new regulations. The type and frequency of these measurements must increase if atreatment plant is to meet its new more stringent discharge requirements. Measurementfor these purposes across the industrial and public sector accounts for about 6% of anindustrial nation’s Gross National Product [164]. Although this cost is large, the indirectcost of making poor measurements is staggering.The immediate effect of poor quality data is noise. Although the amount of information published each day is increasing, there is also an increase in confusion caused bypoorly designed experiments, haphazard sampling plans and flawed measurement processes. The cost of this noise is reflected in the wastage of research effort when resultscannot be duplicated, and conflict between regulators and industry over permit violationsand the applicability of new technology [164]. This cost also includes the time readers10”Water Pollution Control” was merged the “Public Health Engineer” in 1987 to form “Water andEnvironmental Management”. The last issue of WPC was 86(2).Chapter 4. Measurement Process 141spend trying to determine the utility of a paper’s findings. Without standardized and fullreporting of results, researcher’s cannot pass judgement on what they read [21]. Similarly,without a standardized and full method for recording operations decisions, designers andmanagers cannot determine how well the plant is being operated.The problem is endemic and concern is growing. For example, consider the evolutionof the EPA’s compliance inspection programs [223]. In the mid 1970’s, the EPA organizedthe Compliance Evaluation Inspection (CEI) and Compliance Sampling Inspection (CSI)programs. CEI involved a short visit by an inspector. The inspector discussed plantoperation, laboratory methodology, record keeping and laboratory quality control. CSI isidentical to CEI except the inspector also collected effluent samples which were analyzedby another laboratory and the results compared with the plant’s records. In the late1970’s concomitant with the National Cause and Effect Survey, the EPA organized thePerformance Audit Inspection (PAT) which is less intensive than the CSI but focuses onplant operation rather than performance. The Cause and Effect Survey provided thebasis for the Composite Correction Program (CCP) [140].As a consequence of the results of previous surveys and some of the initial between-laboratory test sample programs [164] [50], the EPA began to require self-monitoringNPDES sites to analyze reference samples as early as 1980. This program is knownby various names including the Discharge Monitoring Report Quality Assurance (DMRQA) program and the Performance Evaluation Sample Program (PESP). Table 4.8 summarize the program’s effectiveness [7, pp. 6]. By 1985 October 1, the EPA required allenvironmental data used for decision making purposes to be of known (and documented)quality [213]. The next step in this evolution is the Wisconsin plan to certify laboratoriesbased on their QA/QC programs and their performance in a regular reference sampletest program.11National Pollution Discharge Elimination SystemChapter 4. Measurement Process 142Table 4.8: DMR QA/PESP EffectivenessYear 91e Of Permittees % Of DMR QAWith All Data AnalysesAcceptable A ccept able1980-81 30.8 73.91982 41.4 78.91983 49.3 82.81984 54.2 85.41985 55.8 85.51986 52.1 86.11987 54.1 87.11988 56.9 88.0The development of the EPA’s concern over data quality is not unique. An increasingnumber of publications concerned with data quality have emanated from ASTM [290],National Bureau of Standards [164], Bureau of Census [21] and the National Council forAir and Stream Information [223] [226].The quality of data is judged relative to its intended use. In a wastewater treatmentplant, effluent data are collected to satisfy regulatory authorities and to provide an indication of the plant’s performance. Other data are collected to monitor and control theprocess. These data are also used to establish new regulations and expand our understanding of existing processes. The data quality required for these last two purposes maybe different than that required for process monitoring and control.4.4.2 ReferencesQuality Assurance is the system of activities whose purpose it is to provide assurancethat the quality control job is being done effectively. Quality Control is the system ofactivities whose purpose is to provide a quality of product or service that meets the needsof the user. The application of these systems to wastewater and water laboratories isChapter 4. Measurement Process 143clearly explained in three EPA documents:• Handbook For Analytical Quality Control In Water and Wastewater Laboratories [56]• Choosing Cost-Effective QA/QC Programs For Chemical Analysis [248]• Quality Assurance/Quality Control QA/QC for 301h Monitoring Programs: Guidance On Field and Laboratory Methods [22] 124.4.3 Data QualityData Quality can be defined as “the totality of features and characteristics of data thatbears upon its ability to satisfy a given purpose [233]. By definition, a data set may be ofboth poor and excellent quality at the same time depending upon its end use. The EPAformalizes the Quality concept by identifying five characteristics of major importanceaccuracy, precision, completeness, representativeness and comparability.Accuracy And PrecisionAccuracy is a function of precision and bias [103]. Accuracy is a measure of the degree ofcorrectness while precision is a measure of reproducibility. If two measures are withoutbias, then the more precise measurement would be considered the more accurate whileif two measurements are equally precise, the one whose mean value is closest to the truevalue would be considered the most accurate. A precise measure cannot he consideredaccurate unless the bias is negligible. Accuracy, precision and bias are usually reportedwith a description of how they were determined. A reliable measure is defined as beingboth a precise and an accurate measure.12Under section 301(h) of the Clean Water Act, municipalities are required to conduct monitoringprograms to determine the impact of their discharge on marine hiota, to demonstrate compliance withapplicable water quality standards and to measure toxic substances in their discharge [22].Chapter 4 Measurement Process 144To estimate accuracy, an estimate of the bias, imprecision and (strictly speaking)the form of the distribution of individual measures about the sample-based average arerequired. If the measurement process is under statistical control, it is assumed that thedistribution is Gaussian. Accuracy cannot be determined unless the true valne is knownbeforehand.Analytical inaccuracy may be caused by interference, nonselectivity, a biased calibration or an erroneous blank correction [104]. An interference is a biological or chemicalattribute (other than the determinand) of a test sample that positively or negativelyoffsets the measurement result from the true value. For example, reduced inorganicssuch as ferrous iron interfere with the Chemical Oxygen Demand (COD) measurement oforganic material snch as fatty acids. Nonselectivity is the inability to measure all formsof the determinand eqnally. A typical COD test is nonselective because it oxidizes allbiodegradable organics but only partially oxidizes aromatic organics [202].Many environmental measurements use at least one of the following techniqnes intheir measurement methodology: blanks , standard solutions or calibration cnrves. Forexample, the BOD, COD and Suspended Solids test use a blank to provide a zero point.Blank results, like other analytical results, are subject to error. Therefore the use ofblanks may introduce additional variation and/or bias [248].Correction is a less obvious source of analytical inaccuracy. “Corrections applied inpractice are usually of more limited scope than the names that they are given appear toindicate” [103]. For example, rate constants are often corrected for temperature usingthe Arrhenius law. However, the law is only an approximation because enzyme catalysisis a bounded nonlinear function of temperature. For this reason, correction is a form ofextrapolation and extrapolation increases the uncertainty and possibly the inaccuracy ofa measnre.Table 4.9 contains quality assurance data for BOD and Suspended Solids tests [21]Chapter 4. Measurement Process 145Table 4.9: BOD and Suspended Solids Quality Assurance Results: % UnacceptableStudy Number BOD5 SSWisconsin 1978 150Major Municipal 80 20Minor Municipal 72 35Major Commercial 60 70Minor Commercial 100 33Wisconsin 1980 107 22EPA DMR QA 122 20Wisconsin 1982 109 18EPA DMRQA 123 17DMR QA 1982 7500 17.9 17.1[50] [244] [188] [64]. Weber’s study was one of many studies that showed that a vastamount of data being collected by dischargers was inaccurate [50]. The percentage ofinaccurate data dropped dramatically in later studies as the EPA, having once identifieda problem laboratory, took remedial action. The Wisconsin studies also showed that Paper Mill laboratories performed better than other NPDES laboratories. This may be thecase because these laboratories conducted analyses for process control and therefore wereunder greater scrutiny. The discrepancy between the initial percentage of unacceptableresults is indicative of the observation that “it is more difficult to ensure adequate accuracy and comparable results with biological/microbiological determinations than withphysical or chemical determinations [104, pp. 266].Sampling inaccuracy may result from a poor sampling plan, an inappropriate samplingtechnique or an unacceptable sample custody chain. The latter two sources occur becausethe researcher has not evaluated the appropriateness of the sampling process for eachdeterminand. For example, four deficiencies identified by NPDES inspectors at pulp andpaper mills are given below [223]:Chapter 4. Measurement Process 1461. Insufficient linear velocity in automatic sample collection to prevent loss of settleable solids2. Collection of an unrefrigerated sample when a refrigerated sample is required3. Use of dirty sampling equipment and sample containers4. Holding the collected sample prior to analysis longer than the maximum allowableholding timeInaccuracy resulting from the sampling plan was discussed in Section 4.3.A Quality Assurance Project plan addresses the issue of accuracy in two ways [233] [213].First, the project proposal is justified with respect to experimental and sampling designtheory, and standard sampling and sample custody practices. Second, an analytical quality control program is instituted to prove that the analytical method provides accuratedata throughout the project’s life. Quality control usually consists of analyzing reference or spiked samples. These data should be accompanied with supporting informationincluding a description of how these data were collected and the number of data pointsinvolved [274].Precision is meaningless if the measurement process is not under statistical control.An analyst in a wastewater treatment plant maintains statistical control over his measurement process by ensuring that the measurement of blanks, standards and spikes arestatistically stable, i.e. vary randomly about their true value.Chapter 4. Measurement Process 147The precision of an analytical process is often concentration dependent. For example,a 1:1 mixture of glucose and glutamic acid in a total concentration range of 5 to 340mg/l was analyzed by between 86 and 102 laboratories in an interlaboratory study [176,pp. 489]. The precision expressed as a Standard Deviation (5) is given by the followingequation:S = 0.120(Added Level in mg/l) + 1.04This type of dependency must be taken into account when applying Statistical Quality Control methodologies to environmental analyses - particularly when using ControlCharts [56]. A precision measure should be reported with an explanation of how it wasdetermined [233] [5].Precision is dependent upon the analyst, the apparatus, the laboratory and the dateand time of the analysis. Given all the permutations possible. ASTM restricts the use ofthe terms “Repeatability” and “Reproducibility” to the following situations:• Repeatability is a measure of the variability between at least two test results obtained from a single operator on a particular piece of equipment on a particularsample in the shortest length of time. Repeatability is an indication of the highest precision that can be obtained by a single operator using a particular piece ofapparatus on a particular sample.• Reproducibility is a measure of the variability of a measurement method carried outon a single sample (usually by different laboratories). Reproducibility is a measureof the precision of a measurement method carried out on a single sample.By definition, reproducibility is almost always greater than repeatability.Precision of a measurement should be presented as a function of the measured valueacross the range of applicability [274]. At minimum, regression coefficients, means andChapter 4. Measurement Process 148Table 4.10: Potassium Determination In An Agricultural LaboratorySource Of Imprecision Percentage jSampling Error 87.8due to field sampling 84.0due to sampling inhomogeneity 3.8in the laboratoryBetween Laboratories 9.4Sample Preparation 1.4Precision of Ivleasurement 1.4Variability is expressed as a percentage of the total.other statistics should be accompanied by an indication of their precision. The mosteffective way to present the precision of a measurement process is as a Statistical Experiment because the Analysis of Variance Table speaks for itself. For example, a researcherlooking at Table 4.10 knows immediately what sources of imprecision are significant [207].This approach requires that the sampling plan and measurement schedule are laid out toaccommodate this type of analysis. Precision estimates are usually accompanied by somedescriptive information including the type of precision, the components of the processincluded in the estimate, and any outlier identification schemes [274].CompletenessCompleteness is a measure of the amount of valid data obtained from a measurementsystem compared to the amount that was expected to be obtained during planning. Thesignificance of completeness depends on the consequence of not having all the data youhoped for [274]. For example, the impact of the missing data might he a loss of statisticalChapter 4. Measurement Process 149power resulting in large confidence intervals or insensitive statistical tests. When dataare unavailable, common practice dictates that the report identifies in what phases of thework the data were lost, provides an explanation and suggests a remedy. The impact ofmissing data on the study’s utility may be severe (see Section 2.2).Representativeness And ComparabilityRepresentativeness is a geneiic term referring to the degree of correspondence betweenwhat is measured in the laboratory and what occurred in situ The strength of thiscorrespondence determines a study’s utility [29]No matter how elaborate or expensive the analysis made on a sample is, it isof no value or worse, misleading, unless the sample is representative.Representativeness is affected by how and when samples are collected, what happensto the samples between their being collected and analyzed and how the samples areanalyzed [274] [240].For example, inspections of paper industry NPDES self-monitoring facilities identifieda number of deficiencies that possibly affected the representativeness of their effluentBUD5 data [223]:• Sampling— Samples not kept at 4 degrees C during collection— Dirty samplers and sample containers• Storage— Holding time exceeded the maximum recommendedChapter 4. Measurement Process 150Analysis— Improper seed storage— Insufficient thiosulfate standardization frequency— Improper incubation temperatureMost deficiencies occurred in laboratories without a Quality Assurance/Quality Controlprogram in place.A representative sample is not necessarily a random sample. In the literature, thetwo words are sometimes assumed to be synonyms [1]. Random refers to how a sample ischosen while representative refers to how well a sample describes the part of the systemfrom which it was taken.By definition, representativeness like accuracy, is impossible to quantify. For thisreason, the EPA requires that the researcher include a justification of the project planwith respect to representativeness and proof that the plan was implemented [233].Comparability expresses “the confidence with which one data set can be comparedto another” [233]. The comparability of two data sets can only be decided if the conditions under which the data were generated and collected are known. For example, theEPA [233] list six criteria that to determine if two data sets might be comparable:• Similar siting criteria• Same observables measured• Compatible sampling and analysis protocols• Same degree of quality assurance and control• Same units of reportingChapter 4. Measurement Process 151• Correction of measured values to standard conditionsAlthough full disclosure is required to determine whether two data sets are comparable,many articles fall short of providing the information necessary to evaluate comparability.For example, a recent review of three prominent medical journals revealed that only 56%of clinical trials were fully reported. The authors concluded that ‘. . . investigators oftenhave good reason for weaknesses in design, but the reasons for weaknesses in reportingcan he few” [96, pp. 1337].Chapter 4. X’ieasuremerzt Process 1524.5 Measurement ModelPierre M. Gy observed that the failure of many mining or metallurgical ventures can often be attributed to unaccountable sampling errors. Gy developed his General SamplingTheory to improve the accuracy and efficiency of the sampling of particulate solids inthese industries, particularly the sampling of ores and concentrates. His theory is summarized in his text “Sampling Of Particulate Materials” [133]. The theory now formsthe basis of process sampling techniques in a multitude of industries including wastetreatment [81] [243].In retrospect, Gy’s theory evolved from the inadequacy of any existing theory initself to address completely the difficult sampling problems surrounding the monitoringof municipal sludges, sediments, dredge spoils, drilling muds, solid wastes and wastewater streams. For this reason, the theory is a synthesis of Statistical Sampling Theory,Signal Theory and Sampling Physics and Mechanics. Statistical Sampling Theory ormore specifically, Standard Sample Survey and Classical Sampling Theory, was developedto obtain efficient estimates of a population’s characteristic and its uncertainty. SignalTheory was developed to ensure that a digital reconstruction of an analog signal is free ofalias. The physics and mechanics of sampling are concerned with the sample extractionprocess itself.Two models form the basis of Gy’s theory: The Continuous and The Discrete SelectionModels. The Continuous Selection Model describes the variation with respect to timeand space of the characteristic of interest within the investigated material. This modelconsists of two functions:• Qualitative Function: The critical content of the material flowing past the sampling point at an instance in time.Chapter 4. Measurement Process 1.53• Quantitative Function: The rate of flow at an instance in time.If necessary, these functions can be generalized to 2 or 3 dimensions.The second model, the Discrete Selection Model, is concerned with the discrete natureof fragments or particles submitted to the sampling operation. The model describes twotypes heterogeneity:• Constitution or Micro Heterogeneity: The case when there are differences inphysical or chemical properties between or within particles.• Distribution or Macro Heterogeneity: The case when the stream is stratifiedor multi-phasic.Gy limits the sample selection schemes to systematic selection with random positioning, random stratified selection and random selection. The reason for this restriction isthat purposive selection does not lend itself to statistical examination [78, pp. 10-111.However, in most situations, understanding the probabilistic case is a prerequisite tounderstanding the purposive case. The theory discusses two methods for obtaining asample: increment sampling and flow splitting. The difference between the two is thatin flow splitting, the entire stream is diverted for an instance in its entirety, while withincrement sampling, only a portion of the stream is removed at an instance.Gy defines the underlying objective of a sampling scheme as correctness. Correctnessis a measure of the match between a reconstruction of the history of a stream froma set of samples and the history of the stream. The required closeness of the matchdepends on whether the reconstructed history is used for control, failure detection, processidentification, monitoring, or compliance determination.The goal of Gy’s theory is to identify sampling problems and determine if they arecorrectable. The theory accomplishes this by classifying sampling errors into identifiableChapter 4. lVleasurement Process 154units. This scheme forms the framework for this section. The units may be identifiedin part by running a \/ariographic Experiment to obtain data to construct a RelativeSemivariogram. The reader may refer to Gy’s text [133] or Pitard et al’s paper [243] formore information.4.5.1 Model ComponentsFigures 4.2 contain a schematic of Gy’s sampling error model. The error components aredealt with during the planning of the monitoring program (i.e. Quality Assurance) andduring the execution of the program (i.e. Quality Control). The components tagged inFigure 4.2 with an “A” are dealt with at the planning stage, those with a “C” duringthe execution stage and those with an “A,C” at both stages. Gy’s model could form thebasis of a measurement process simulation as well as a measurement process audit.Overall ErrorThe Overall Error, eoE, is the error generated during the measurement process. Theprocess includes sampling, preservation, transport, preprocessing and analysis. All butthe last term is contained in the Total Error, eTE. The Analytical Error, eAE, is the errorgenerated duiing the measurement of the sample’s determinand Given the abysmalperformance of treatment plants during DMR-QA studies, analytical error remains animpoitant concerneoE = eTE + CAE (4.6)Chapter 4. Measurement Process 155A PlanningC ExecutionA, CAKFYStage Error Dealt WithC, AC, AC, ACCAAAA, CA,CFigure 4.2: Gy’s Sampling Error Mode’Chapter 4. Measurement Process 156Total ErrorThe Total Error, 6TE, is the sum of the individual samples’ preparation, epg, and sampling errors, esE.N6TE = E(epEk + esEk) (4.7)k=14.5.2 Preparation ErrorsPreparation errors occur during the preparation of a sample for analysis:• Contamination Error: A foreign substance contaminates the sample altering themeasurement of the determinand, i.e. contamination from a dirty sample bottle.• Loss Error: A fraction of the sample is lost prior to analysis, i.e. the exclusion ofsome colloidal matter because the analyst does not use a wide-mouth pipette.• Alteration Error: Sample characteristics are altered by physical or biological phenomena, i.e. the sample undergoes nitrification in the sampler.• Mistake and Fraud: A mistake is unintentional error introduced due to an accidentor due to ignorance, i.e. confusing sample bottles. Fraud is an intentional mistake.Sampling ErrorSampling Error, esg, arises from the sample selection, delineation and extraction processes. The sampling error is the sum of the Continuous Selection Error, ecE, and theMaterialization Error, eME.eSE = eME + CUE (4.8)Chapter 4. Measurement Process 157Materialization ErrorThe Materialization Error is the sum of the Increment Delineation Error, CDE and theExtraction Error, 6EE The Increment Delineation Error results from the incorrect delineation of the sample in the stream, i.e. the sample is taken at the wrong time or place.The Extraction Error results from the tendency for the sampling device or probe to selectfor different fractions of the stream with different probabilities. For example, a samplerwill select for different portions of the stream depending on the tube size and the samplerpump velocity.eME = eDE + cEE (4.9)Continuous Selection ErrorThe Continuous Selection Error, 60E, is the error that results from the interaction between the stream’s characteristics and the sampling process. The error is the sum of twocomponents, the Weighting Error and the Quality Fluctuation Error. The WeightingError, ewe, results from changes in the stream’s flow or volume.For example, an operator can take either a time-based or a flow-based compositesample. In theory, the flow-based sample provides the best estimate of the mass of aparameter entering the plant over the compositing period because it is a flow-weightedaverage. However, the sample’s accuracy and precision rely on the accuracy and precisionof the flow meter. Consequently, the utility of the sample decreases as the noise in theweighting function increases (Schaeffer’é Composite Sampling Theorem) [259].A time-based composite sample provides a biased estimate of the mass loading intothe plant because each aliquot is weighted equally despite the flow into the plant at thatinstance. However, because flow and strength are inversely correlated in most plants [262],Chapter 4. Measurement Process 158the bias is tolerable for most operating decisions as it is much easier to weight a sampleby time than it is by flow.ecE = ewE + CQE (4.10)Quality Fluctuation ErrorThe Quality Fluctuation Error consists of three components:• QE1: Short Range Quality Fluctuation Error, i.e. noise• QE2: Long Range Quality Fluctuation Error, i.e. trend• QE3: Periodic Quality Fluctuation Error, i.e. diurnalClassical Sampling Theory assumes that QE2 and QE3 are not significant or havebeen compensated for by stratification of the population prior to sampling.eQE = 6QE1 + eQE2 + 6QE. (4.11)Short Range Quality Fluctuation ErrorShort Range Quality Fluctuation Error, 6QE1, consists of two components, FundamentalError, eFE and Segregation or Grouping Error, eGE.Distributional or Macro Heterogeneity is the source of Segregation or Grouping Error.Macro Heterogeneity occurs when the mass of the investigated material is distributed in anon-random manner across the stream. This distribution may take the form of strata (i.e.secondary clarifier solids distribution) or a velocity profile (i.e. the solids distributionacross the secondary sludge recycle line). If the structure of the heterogeneity is notknown, the samples should be delineated either systematically or at random. If enoughChapter 4. Measurement Process 159samples have been taken to plot the profile of the population, the samples can be groupedinto strata after the fact to give the average concentration in each strata.’3Compositional or Micro Heterogeneity is the source of Fundamental Error. MicroHeterogeneity occurs when the stream either contains a dispersed phase or when theproperties of the components of a single phase vary. Primary influent is an example ofthe first situation. The influent, which is mostly water, contains dissolved, colloidal andsettleable solids. Dried secondary sludge is an example of the second situation. Thecharacteristics of the sludge at a single concentration varies depending on its microbialcomposition. Fundamental Error cannot be suppressed by improvements in the measurement process and therefore forms the basis of comparison for determining the importanceof the contribution of other sources of error.4.5.3 Error CorrectionA possible procedure for reducing the overall error is listed below:1. Audit: The operator should familiarize himself with the proper procedures for sampling, preservation, transport, preprocessing and analysis. He should step throughhis monitoring program and try to identify any obvious errors in the system (Figure 4.3). An audit is much easier to do if the plant has a sampling protocol.2. Plan: In some cases, the error may be reduced by using a different sample type orplan. Table 4.11 provides some suggestions on what sampling plan or sample type touse given the quality fluctuation error. The sampling frequency,f3,is the frequencyat which an aliquot is removed from the stream. The analytical frequency. fa. is thefrequency at which the sample produces a value. If the sample is a grab sample, the‘3The reader should not confuse stratification before sampling with stratification after sampling. Theformer improves the precision of the overall mean if the strata are significantly different. The latter doesnot [78, pp. 89-90].Chapter 4. Measurement Process 160two frequencies are equal. If the sample is a probe average or a composite sample,the analytical frequency is the averaging or compositing frequency respectively.The evaluation frequency, f8, may be defined as being a quarter of the analyticalfrequency. These guidelines were derived by the author using Shannon’s SamplingTheorem [18] and Gy’s sampling theory 143. Experiment: The precision of a measure can be improved by replication. However,the operator must determine what to replicate. Table 4.10 contains the resultsof a sampling experiment. Because the sampling error accounts for most of thevariation in the data (87.8%), the operator would gain more precision if he collectstwo samples and conducts an analysis on each than if he collects one sample andreplicates the analysis.14see evaluation frequency in Appendix BChapter 4. Measurement Process 161Errorz:z;F4EioEz::---SampleHandungAuditlCheck Equipment And ReactantsAudit Analytical MethodologyAudit Laboratory ProceduresService Or ReplaceSampling EquipmentCorrect How SampleIs DelineatedAudit SamplingHandling ProceduresFigure 4.3: Example Audit ProcedureChapter 4. Measurement Process 162Table 4.11: Sample Plan and Type Selection Based On The Quality Fluctuation ErrorDominant ActionFormRandom 1. Determine the cause.ErrorIf the cause is due to sample heterogeneity then increase the sample volumeand/or homogenize the sample mechanically.If the cause is due to subsampling or sample preparation then increase the quality control and/or improve the subsampling scheme.If the cause is due to the analytical method then either change the method orperform repetitive determinations and calculate the average.If a less precise but easy measure is available, a third alternative might be touse a double sampling type scheme.Trend 1. Determine or estimate the form of the trend.2. Decide on the sampling interval that gives the optimal resolution of the trend.Cyclic: 1. Determine the cost of measurement(f<fa)2. Determine if compositing is possible3. Determine if the determinand can be measured on-line.If on-line measurement can be used then measure at least 3f and filter out thedynamics you do not want.If measurement cost is low and compositing not possible then use a stratifiedrandom design.If the cost is high and compositing is not possible then use a stratified randomdesign sparingly until a better measurement alternative can be found.If a composite sample can be used then sample at 3f and composite.Cyclic: A grab sample may be all that is needed. The sampling or analytical frequency(f > fe) should be less 3f.Cyclic: Same as first cyclic case. The operator should be aware that a composite sam(L f f€) pler will introduce f into the time series under a somewhat damped alias. Analias is a high frequency component that appears in the sample (reconstructedsignal) as a low frequency component.Definitions f Frequency Of Fastest Significant Componentfa Appraisal Frequencyfe Evaluation Frequency (fa/4)Chapter 4. Measurement Process 1634.6 Good Data, Good DecisionsMeasurement is the Achilles heel of treatment plant data analysis, modelling and control.Most of the problems discussed in this chapter cannot be rectified after the fact. Ifthe data analysis is to produce meaningful results, then the measurement process mustproduce representative data [207]:statistical treatment is no substitute for good data./Chapter 5Fuzzy Sets, Logic and ReasoningComputers give blunt answers. Yes or no, black or white. Researchers inartificial intelligence are trying to teach their machines a little subtlety: toencode the shades of gray characteristic of human thought. One approachthat is producing good results is fuzzy logic, the creation of Lofti Zadeh of theuniversity of California at Berkeley. His brainchild is now running cementkilns and making decisions for corporate managers. [13]The purpose of this chapter is to introduce the concept of a fuzzy set and explain howthe theory is used to overcome data apartheid. The measurement paradigm uses fuzzyset theory in two ways:1. To convert a string into a form the computer understands: For example,the operator tells the computer that the floc size is “large” (Table D.4, Appendix D).For example, the could computer map the string “large” into the following fuzzyset:0 ifx<4O0tmitarge(X) = x;ooo if 400< x < 500um (5.12)1 ifx500RrnThe computer maps the data into a common space so that the data can be analyzedas a unit.2. To convert a datum into a more reliable form: For example, the user tellsthe computer that the effluent Chemical Oxygen Demand (COD) is 22.1 mg/l.164Chapter 5. Fuzzy Sets, Logic and Reasoning 165The computer “knows” that the COD test is unreliable below 50 mg/l. Instead ofstoring the datum supplied by the user, the computer stores the value as being lessthan 25 rng/l. This avoids false alarms, e.g. reporting a trend in COD values below2.5 mg/l.5.1 LiteratureA comprehensive review of fuzzy set literature is beyond the scope of this thesis. Thissection limits itself to listing introductory reviews, a number of texts and a few examplesof applications in wastewater treatment.Lofti Zadeh [314] [315] [316] published two short and accessible introductions to fuzzylogic. ‘. Matthias Otto [238] published a more extensive review discussing fuzzy theory’simpact on the field of ‘hernometries.In the course of this research, a number of texts proved to be helpful. Dubois andPrade [99] and, to a lesser degree, Kandel [180] provide an overview of the subject.Smithson [277] discussed the application of fuzzy set analysis to categorical measureswhile Kaufmann and Gupta [183] dealt with ratio and interval measures (fuzzy numbers).Klir and Folger [186] reviewed fuzzy set theory in the context of information theory.The predominant application of fuzzy set theory in wastewater treatment is in thedevelopment of fuzzy controllers and diagnostic systems. The first controllers were developed by Tong et al [292] and Flanagan [109]. Flanagan used the DO profile in theplant to estimate the F/M activity in the plant and output daily settings for SRT. Recentapplications include control of sewage pumping [160] and chlorination [167]. Fuzzy settheory is also used in a number of wastewater treatment expert systems [174] [190].‘A number of Lofti Zadeh’s papers have been collected into a single edition,”Fuzzy Sets And Applications”, New York:Wiley, 1987Chapter 5. Fuzzy Sets, Logic and Reasoning 1665.2 What Is Fuzziness?Fuzziness is a form of uncertainty. Uncertainty may be due to vagueness or ambiguity. Vagueness is associated with the difficulty in sharp or precise distinctions in theworld [186] or some domain of interest which cannot be delimited by sharp boundaries.Ambiguity is associated with one-to-many relations, that is, when the choice betweentwo or more alternatives is left unspecified. Fuzziness is a form of vagueness that isunambiguous.Possibility is one way to describe fuzziness. Possibility definitely is not probability inthe frequentist sense, however, under some conditions, the difference between subjectiveprobability and possibility is difficult to discern [277]. For example, given a COD valueless than 25 mg/l, the likelihood (belief) and the possibility that the COD is betweeno and 25 mg/l are indistinguishable. The view among those who use fuzzy set theoryin the work place is that there is some overlap between the various versions of fuzzinessand probability. No single uncertainty representation is able to describe every form ofvagueness. Therefore, a researcher must choose the one which gives his problem the bestcoverage. The reader can refer to either Klir [186] or Smithson [277] if he wishes topursue the issue further.Chapter 5. Fuzzy Sets, Logic and Reasoning 1675.3 Notion Of SetsFuzzy Sets are an extension of Crisp Sets. For this reason, this section introduces thenotion of crisp sets and then extends this notion to that of fuzzy sets.5.3.1 Notion Of Crisp SetsIntuitively a set is a collection of objects (elements of the set). Let X denote a classical(or crisp) set of all objects of concern in each particular context or application, and let xdenote a generic element in X. By definition, X is the Universal Set. Subsets are formedon X.Given a set A which contains the dates that a treatment plant does not meet itsdischarge permit, we can delineate the set by one of two methods:1. List Method: name all a sets members.A = {a1,2 a}Days Violated = {Jan2/90, Feb5/90, ..., Mar2/91}2. Rule Method: specify some property the set’s elements must satisfy.A ={aahas propertiesDaysViolated = {Date BUD5 > 3Orng/l or TS> 2.5rng/l}Throughout the balance of our discussion, we will refer to two important universalsets:1. fl: The set of real numbers.2. Ar: The set of positive integers.Chapter 5. Fuzzy Sets, Logic and Reasoning 168We denote a n-dimensional Euclidean Vector space of real nnmbers asA convex set is a set where if a line is drawn between any two members of the set, allthe elements that fall on the line are also members of the set. A set whose members aresets is referred to as an indexed set or a family of sets: B= { A i e I }. For example,if we define B to be the days treatment plants in British Columbia did not meet theirpermit, then I would be the number of plants in the province and A would the daystreatment plant i violated its permit.The Characteristic or Discriminant Function delineates which elements x C X belongto A:11 ifxeAVx C X [LA(X) =10 ifx3AThe characteristic function maps elements of X into a set that consists of 0:Not A Memberor 1:Member: [1A : X —* {0, 1}.The cardinality of A, A , is the number of elements of X that belong to A. ThePower Set of A, P( A), is an indexed set of all possible subsets of A. The cardinality ofP( A) = Table 5.1 lists various properties of crisp set operations [186, pp. 9].5.3.2 Notion Of Fuzzy SetsClassical set theory is governed by a logic that permits a proposition to possess one ofonly two values: true or false. In the real world, the distinction between what is trueand what is false is blurred. Truth is relative in the sense that something can be moretrue than something else or in some circumstances, something is true and in others, it isfalse. For this reason, a system that forces common sense knowledge into being alwaystrue or always false may not be reliable. The solution is to allow the boundaries of aconcept such as truth or set membership to be graded. The consequence of this is thatthe excluded middle law is no longer true, i.e. A U A $ X.Chapter 5. Fuzzy Sets, Logic and Reasoning 169Table 5.1: Properties Of Crisp Set OperationsInvolution = ACommutativity A U B = B U AAnB=BnAAssociativity (A U B) U C = A U (B U C)(AflB)flC=Afl(BflC)Distributivity A fl (B U C) = (A fl B) U (A fl C)Au(BflC)=(AuB)n(AUC)Idempotence A U A = AAflA= AAbsorption A U (A fl A) = AAn(AUA) =AAbsorption of Complement A U (J fl B) = A U BAn(AuB) = AnBAbsorption by X or A U X = XAnø=øIdentity A U 0 = AAn X= XLaw Of Contradiction A fl A = 0Law Of Excluded Middle A U A =XDeMorgan’s Laws A fl B = A UAUB =AflBChapter 5. Fuzzy Sets, Logic and Reasoning 170A crisp set’s characteristic function maps an element either into the set or out ofthe set: flA : X —÷ {0, 1}. A fuzzy set’s characteristic (membership) function mapsan element into a set by degrees depending on how compatible the element is with theunderlying concept of the set: [‘A : X —* [0, 1]. Consequently, the utility of a fuzzy setdepends on the appropriateness of its membership function. Membership describes thepossibility that an object is a member of a given set. Although both possibility andprobability range between 0 and 1, they are very different concepts and should not beconfused with one another.For example, Figure shows a data set of 15 objects in a 2 feature plane [238,pp. 106]. The goal is to cluster the data into two groups. When using a crisp objectivefunction based on Euclidean distance, two clusters are obtained (see Figure wherea membership value of 1 is assigned to the left cluster and 0 to the right cluster. Despitethe symmetrical nature of the data, the two clusters are not symmetric. The reasonfor this is that the middle data point lies between the two data patterns. This pointshould be described as belonging equally to both. If we repeat the analysis using fuzzyclustering, then the middle elements given a membership value of 0.5. Figures 5.2c and5.2d map the membership of the elements to both the right and left cluster respectively.The choice of the interval [0, 1] is arbitrary as any interval (e.g. [0, L] ) could beused. What is not arbitrary is that when elements are mapped into an interval, they areordered by membership. The interval [0, 1] is the natural choice because in binary logic1 represents TRUE or ON while 0 represents FALSE or OFF.Fuzzy Sets should not be confused with Fuzzy Measures. A fuzzy set assigns a degreeof membership to each element in the universal set to indicate how compatible an elementis with the concept of the set. A fuzzy measure assigns a value to each crisp subset of auniversal set signifying the degree of evidence or belief that a particular element belongsin a particular subset.Chapter 5. Fuzzy Sets, Logic and Reasoning 171(A).B .• • B B B B B• BBButte/fly Cluster(W Cluster 1B7• B B B :B B BCluster 2Figure 5.1: Butterfly ClusterChapter 5. Fuzzy Sets, Logic and Reasoning 172(C)• 0.86 0.14W0.94 0.060.86 0.1410.971 0.99W • 0.Oi• o.o0.94 0.061•0.86 0.14WCluster 1(D)•0.14 0.86W.0.06 0.940.14 0.8610.03 •o.oi • • 0.99W 0.97 a0.50W0.06 0.9410.14 0.86WCluster 2Figure 5.2: Butterfly Cluster : Fuzzy ClusteringChapter 5. Fuzzy Sets, Logic and Reasoning 173For example, jury members at a criminal trial are uncertain about the guilt or iunocence of the defendant [186, pp. 107]. The boundary between these two sets is clear inthat given perfect evidence, the jury would have no problem making a decision. Unfortunately, the evidence is rarely perfect, therefore the jury must express its belief that thedefendant is innocent or guilty. Belief is a fuzzy measure. In contrast, assume that inthe course of the trial, a witness states that he saw a tall man run from the crime. Afterthe witness is finished, the prosecutor informs the jury that the defendant is 5’ 10” tall.In this case, the problem is not with the evidence but with determining what is a tallman.Notat ionThe usual way to denote an element in a fuzzy set is t/x where t is the membership andx is the element. For example, 0.8/50 mg/l may denote a possibility of 0.8 that 50 mg/lBOD5 is considered a high value. A less common denotation is to write an element as apair: (x,t).The two most common methods to denote a fuzzy set A are as a union or as a set ofordered pairs:• As a union:A=A(x)/xj where denotes union over discrete universeorA= J 4(x)/x where f denotes union over continuous universe• As ordered pairs:A = {(xi, ILA(X1)), (x2, [IA(X2)),.. . , (x, RA(Xn))}Chapter 5. Fuzzy Sets, Logic and Reasoning 174For example, let X = [0. oc] be a measure of distance. Let A be the fuzzy set Longwhere Xa is the distance beyond which the membership is different from zero. We candenote this set in one of two ways:• Method 1:lo ifx<xaRL0n9(X)=11—c’’ ra) ifx>Xa• Method 2:A= J 0/x + J [1 —X<Xa X>Xawhere + denotes nnion.Basic DefinitionsThe support of A is the crisp set of all the elements A with positive membership:supp(A) = {x I /JA(X) > 0}. The height of A is the highest degree of membershipof any element in A: hght(A) = supxexiUA(X) where sup means supremum. A crossoverpoint is an element in A whose membership is 0.5.The cardinality of a crisp set is the number of objects in the set. The cardinalityof a fuzzy set lies between the number of items with a membership greater than zeroand the number of members with a membership of one, lAlo.o Al lAl=1.o. Thescaler cardinality of a fuzzy set A is given by IA! = Z€X PA(x) where means sum 2The fuzzy cardinality of A is a fuzzy set or a fuzzy number where [tIAI(IAI) = a. Thedefinition of a fuzzy empty set is similar to that of a crisp empty set: A is empty (A =0) if Vx C X,,uA(x) = 0.2Norrnally, means union when discussing fuzzy setsChapter 5. Fuzzy Sets, Logic and Reasoning 175Two fuzzy sets are equivalent if they contain the same items with the same degreeof membership: A equals B (A = B) if Vx€ X,RA(x) = /1B(X). The definition ofnonequivalence follows from the definition of equivalence: A not equals B (A $ B) ifX€X,4(x)One set contains another set if for every item in the first set, the membership in thesecond set of that item is less than the membership in the first set: A contains B (A ç B)if Vx C X, ILA(x) RB(x). B is proper subset of A if (A C B) and supp(A) 0 supp(B).An a-set, Aa, is a crisp set of elements of the fuzzy set A whose membership functionexceeds a threshold value a: Aa : x C X p,A(x) a. A fuzzy set may be represented bya set of a-sets where a is varied from 0 to 1. It is much easier to program a computer tooperate on this representation of a fuzzy set than on the more conventional representationdescribed in the previous section [99].Chapter 5. Fuzzy Sets, Logic and Reasoning 1765.4 Notion Of VariablesA variable’s type is determined by what the variable takes for values (Figure 5.3). A crispvariable’s values are crisp sets, i.e. DaysViolated. A crisp number and interval valuesare restrictions on 7?. A crisp number’s set has one member, i.e. Volume is 100 m3. Aninterval’s set contains all the values on 7? between two limits, i.e. Normal Workday isbetween 7.5 and 8.5 hours long. A probabilistic variable value’s are given by a referencedistributiou. If the reference distribution is Gaussian, the set is defined by a mean anda standard deviation.A fuzzy variable’s value is a fuzzy set, i.e large. A fuzzy number’s sets are convexnormal restrictions on 7?. A linguistic variable’s values are words. A word is meaninglessunless it is defined as part of a language. In fuzzy set theory, a word is equated witha fuzzy variable. For example, Floc Size may take on the values { Large, Mid-Sized,Small } (Table D.4, Appendix D). Each value is associated with a fuzzy set, i.e. large isdefined by equation 5.12.The measurement paradigm maps all values to a crisp number, a mean/standarddeviation or a fuzzy set. Each of these representations may he mapped to a fuzzy set. Inthis way, all variables may be analyzed at the same time.chapter 5. Fuzzy Sets, Logic and Reasoning 1771.0:- Crisp0.01.0- Interval1.0=0.0_Possibility1.0 -0.0Mean/StdLikelihoodFigure 5.3: Representations Of UncertaintyChapter 5. Fuzzy Sets, Logic and Reasoning 1785.5 Notion Of Fuzzy ControllersIn the early seventies when expert systems were being developed in the United States, aparallel development took place in the United Kingdom - the development of rule-basedcontrollers. Although not originally conceived as expert systems, rule-based controllerspossess all the same characteristics of an expert system. The first controllers used fuzzyset theory to represent the rules and rule-based control has been associated with fuzzysets ever since. Control is based on deviations from a set-point, much in the same way asPID control. Efstathiou [100] describes Image Automation’s fuzzy controller for a cementkiln. As of 1988, over a third of the United Kingdom’s cement making capacity operatesusing this controller.A self-organizing rule based controller is one that corrects its rules on the basis ofthe controller’s performance. The controller is the rule-based counterpart to an adaptivecontroller.5.6 ConclusionsFuzzy set theory and fuzzy logic enable the measurement paradigm to eliminate dataapartheid and to represent vague data. Fuzzy set theory may also provide a method toreason about the data analysis and assist the operator in making control decisions.Chapter 6Structure Paradigm24s simple as possible and only as complicated as necessary 1The purpose of this chapter is to explain the Structure Paradigm. The Structure Paradigmplaces various forms of process knowledge into a single data structure providing a datumwith a structural context. The importance of the structural context in managing theinformation gathering and treatment processes was discussed in Chapter 2 2 and Chapter 3 . The primary purpose of the structural paradigm is to enable a computer programto create an image of these processes within its memory so that it can establish processderived relationships among the data.Process knowledge falls into one of four classes:• Structural: Layout of the process• Measurement: Measurements taken on the process (1) to monitor, control, diagnose the process or (2) to validate the measurement process.• Derivation: Statistics and other numbers derived from the data.• Reasoning: Rules and equations provided by the operator that enable the computer to reason about the process.Structure is the language by which information in the other classes is expressed.‘Engineering Maxim2PLF’s and Computer-Based Solutions3Cause and Effect179Chapter 6. Structure Paradigm 180The chapter consists of four parts:• Introduction to Data Base Management System (DBMS) Models: The differencebetween a relational and network databases is explained using a text book’s subjectindex as an example.• Layout of a Simple Plant: The concept of hierarchical network is introduced using asimple activated sludge plant. The plant is simple because it has only one primaryclarifier, one secondary clarifier and one aeration tank. This makes the plant’sschematic easy to draw. This example is referred to throughout the balance of thechapter.• Definition of the Structure Paradigm: The basic elements of the structure paradigmare defined. These include nodes, links, classes, planes, streams and currency.• Plane Types: An introduction to the structural representation of different types ofprocess information.• An Example of How the Structure Database May Be Built: Chapter 1 mentionedthat a number of test programs were written to ensure that the computer can doa task. The example in this section is one of these test programs . This exampleis written in ANSI C [211] and Assembler [212] under MS-DOS. The database wasdeveloped using db_FILE. dh_FILE © [250] is a Database Management System(DBMS) for use by C language application developers.4The example consists of over 20,000 lines of code not including file I/O and list management. Contactthe author about the availability of the source code.Chapter 6. Structure Paradigm 1816.1 Database Management SystemsBecause most database packages that run on personal compnters are relational databases,most people are unfamiliar with the concept of a network database. The differencebetween the two models is discussed in this section because this research uses a networkdatabase. For a more in depth presentation of this topic, one can refer to C. J. Date’sbook “An Introduction To Database Systems” [92].Database Management Systems (DBMS) are based on variations of either the relational or network model. The relational model views a database as a set of two dimensional tables where the columns correspond to fields and the rows form relations betweenthese fields. The advantages of this model are threefold:1. A relational database is easy to query.2. New relations may be formed from existing relations.3. The database structure is simple.A relational database is used when the database structure either is not known aheadof time or may change. The disadvantage of the relational model is that it is difficultto store one-to-many type relations without storing a large body of redundant data. Aone-to-many relation consists of a single owner (i.e parent) and any number of members(i.e. children). dBASE IV © and Paradox © both use the relational model.Chapter 6. Structure Paradigm 182Table 6.1: Portion Of An IndexSludgeconditioning (see Conditioning of sludge)conversion processes, 520-522selection of processes, 70, 71vacuum filters (see Vacuum filtration)The network model extends the relational model by allowing the user to store relationships among records with the records themselves. The advantage of a network DBMSover a relational one is twofold:1. Network databases use storage space more efficiently than relational databases.2. Network databases allow the storage of relationships among the data with the dataitself.A network database is a complex structure. Therefore, once built, its structure is difficultto modify.To illustrate these differences more clearly, consider the following example. An authorwishes to construct a subject index for his book. A portion of a typical index is shownin Table 6.1. Using this table, we can make four generalizations about subject indexes:• A subject cannot be more than 40 characters in length (arbitrary).• A subject may have any number of secondary subjects,. i.e. Sludge has at leastfour.Chapter 6. Structure Paradigm 183• A subject may have any nnmber of page references associated with it, i.e. selectionof processes has two.• A subject may have any number of synonyms. An author uses a synonym to referthe reader to another part of the index, i.e. seeA network DBMS implements this using two records: one to contain a subject anda second to contain a page number. The DBMS constructs two sets. The first setlinks a subject to its synonyms (synonyms) and the second links a subject with a pagenumber (paga.refs) . Figure 6.1 contains the database schema. If the user requires thepage numbers associated with a subject, the user simply makes the subject the ownerof the pagesef set and retrieves all the set’s members. The reader should refer to thedb_FILE [250] manual if they wish to explore this example further.The user could accomplish the same task using a relational DBMS if they took oneof two approaches:• Construct the record to accommodate the extreme case. This approach results ina large, mostly empty file.• Mimic a network DBMS by storing large amounts of redundant information andwriting code to model the set relationships. This approach requires a large numberof database files linked together by user supplied relations.A relational DBMS shifts onto the user (or the programmer) the task of managing relationships among data. Although this burden is manageable in most small businessenvironments, the burden becomes unacceptable when managing treatment plant information. The book indexing example (Table 6.1) provides the reader with a point ofreference when reviewing the example given at the end of this chapter.In this chapter, slanted type is used to indicate a db_FILE set.Chapter 6. Structure Paradigm 184JrSubjectPrimary Name synonymsSecondary Namepage_refsReferencePage NumberFigure 6.1: Book Index SchemaChapter 6. Structure Paradigm 1856.2 A Simple PlantAn operator and a computer view networks differently. An operator views a network asa single entity (e.g. story) while a computer views a network one node at a time (e.gword). A computer can deal with a network of almost any size because it deals with eachnode separately. An operator views a network as an entity in terms of what the linksand nodes describe. For this reason, a computer program that processes a network muststrike a compromise between these two viewpoints.A network is a mnch more complex entity to think abont than a node. Therefore,most operators reason best when the network consists of no more than eight nodes. Thesoftware must both provide a node level description to the computer while providing anetwork level description to the operator. The best way to meet both the needs of theoperator and the computer is to represent the system as a hierarchical network. This isone example of the “organic-digital” compromise discussed in Section 1.2.6.A hierarchical network is a tree (i.e. hierarchy) of networks. A tree is a one-to-manyrelation with one node owning any number of nodes and links, i.e. a network. In thisapplication, the parent node is a more “abstract” concept than its children, i.e. thelevel in the hierarchy is based on the level of abstraction. Like a conventional network,a hierarchical network describes a node by what immediately precedes and follows it.However, a third dimension (i.e. level of abstraction) is added which elaborates on anode using yet another network.The example discussed in this section illustrates this compromise. Figure 6.2 describesa simple activated sludge plant. The level of abstraction of the plant’s componentsvary. For example, “Split Underfiow” is a more concrete concept than “Aeration Basin”.Chapter 6. Structure Paradigm 186Therefore, the first task is to group the nodes iuto sets with the same level of abstraction,i.e an outline 6Figure 6.3 is an outline of the plant. The first level, LP.? , consists of seven nodes.Two of these nodes, LP.3 and LP.4 own children. For example, LP.3 Primary Treatmentowns [P.3.1 - LP.3.4. The networks are formed using nodes at the same level of abstraction. Figure 6.4 shows a “bird’s eye view” of the hierarchical network. A node that isdesignated as being a source or a sink node in the outline marks the edge of the network, i.e. the origin or destination of a link is outside the process. This is analogous toreplacing a connection with a force when drawing a free body diagram.The first network an operator would see provides an overview of the plant (Figure 6.5).The terms “Currency” and “Coordinates” which appear in the figure are defined inSection 6.3.7. The node [P.3 Primary owns a network which describes what primarytreatment entails (Figure 6.6). This is a simple network because the plant has onlyone primary clarifier. The term “Capacity” which appears in the figure is discussed inSection 6.4.2 and Chapter 7. Similarly, the node [P.4 Secondary is described by thenetwork in Figure 6.7 and [P.4.3 is describe further by the network in Figure 6.8.This representation forms the basis of the operator’s interaction with the computer.An operator would begin his analysis of his plant first by taking an overview (Figure 6.5),and then would proceed to focus in on trouble areas by “opening up” a unit process, e.g.Figure 6.5 Figure 6.7 =i Figure 6.8. The computer provides a datum with a structuralcontext by associating it with the part of the plant the datum describes, e.g. MLSS with[P.4.3.2 Aeration Basin.5For example, the input for the example describe in this chapter was developed using Microsoft Word’s© outlining feature. In order to keep the example simple, sludge processing has not been elaborated on.TIn this chapter only, names of nodes which appear in the text will he in sans serif type.Chapter 6. Structure Paradigm 187Figure 6.2: Simple PlantChapter 6. Structure Paradigm 188P. City XYZ Water Pollution Control CenterLP.l Plant Influent [Source]LP.2 MixLP.3 Primary TreatmentLP.3.1 Primary Infinent [Source]LP.3.2 Primary ClarifiersLP.3.3 Primary Effluent [Sink]LP.3.4 Primary Sludge [Sink]LP.4 Secondary TreatmentLP.4.1 Secondary Influent [Source]LP.4.2 Mix Return Sludge With Influent [Join]LP.4.3 Aeration Basin BioreactorLP.4.3.1 Aeration Basin Influent [Source]LP.4.3.2 Aeration BasinLP.4.3.3 Aeration Basin Effluent [Sink]AP.4.3.4 Aeration Basin Air Supply [Contains a Source: Energy]LP.4.4 Secondary ClarifierLP.4.5 Secondary Effluent [Sink]LP.4.6 Wastage From Secondary Clarifier Underfiow [Sink]LP.4. 7 Split Secondary Clarifier Underfiow [Split]LP..5 Plant Effluent [Sink]LP.6 Sludge ProcessingLP.7 Composting [Sink]Figure 6.3: Outline Of PlantChapter 6. Structure Paradigm 1 89InfluentLP.1ISludgeLP.4: Secondary TreatmentFigure 6.4: Simple Plant: Bird’s Eye View Of A Hierarchical NetworkChapter 6. Structure Paradigm 190Influent Compost Currency: Flow {L3/T]LP.1 LP.7Coordinates: [M/L3]Mix 1. SolidsLP.2 2. Substrate3. Ammonium-NPrimary Sludge— 4. Nitrate-NLP.3 LP.6 5. Toxic CompoundSecondaryLP..4EffluentLP.5Figure 6.5: LP.: Treatment Plant PlaneChapter 6. Structure Paradigm 191Currency: Flow [L3/TjClarifierLP.8.2IFigure 6.6:1SludgeLP.3.4Coordinates: [M/L3]1. Solids2. Substrate3. Ammonium-N4. Nitrate-N5. Toxic CompoundCapacity:1. Number: 2 Tanks2. Length: 46.33 m3. Width: 11.58 m4. Depth: 3.10 mInfluentLP.3.1EffluentLP.3.3LP. 9: Primary Treatment PlaneChapter 6. Structure Paradigm 192Mix+IBioreactorLP..3IClarifierLP.JI ICurrency: Flow [L3/TJCoordinates: [M/L3]1. Solids2.3.4.5,SubstrateAmmonium-NNitrate-NToxic CompoundInfluentLP..1SplitLP..7EffluentLP.4.5WastageLP.4.6Figure 6.7: LP..i: Secondary Treatment PlantChapter 6. Structure Paradigm 193<Liquid Stream>Currency: Flow [L3/T]Coordinates: [M/L3]1. Solids2. Substrate3. Ammonium-N4. Nitrate-N5. Toxic CompoundluffuentLP.4.3.1FBasin AirEffluentLP.4.3.3<Air Stream>Currency: Mass [M/T]Figure 6.8: LP.11’.3: BioreactorChapter 6. Structure Paradigm 1946.3 Overview Of The Structure ParadigmThe Structure Paradigm offers a structured and extendible way to process informationobtained on a process whose structnre is relatively stable, e.g. a treatment plant. Theparadigm accomplishes this by creating a data structnre that provides the computer witha description of the process. This data structure forms a skeleton onto which the computercan graft other classes (i.e. types) of process information (e.g. monitoring program,simulation model). For example, assume the operator wants to tell the computer wherea dissolved oxygen probe is located in his process. The computer draws Figure 6.8 on thescreen and asks the operator to point to the probe’s location. The operator points to nodeLP.4.3.2 Aeration Basin. The data structure enables the computer to associate a locationon the screen with a node in the hierarchical network. Now assume the operator entersa rule into the computer that the aeration basin dissolved oxygen should not exceed 3mg/l. The computer can use this rule because the computer can retrieve the aerationbasin’s location in the network from the data structure and then retrieve any dissolvedoxygen data collected at this location.The data structure consists of two components: (1) skeleton and (2) flesh. Theskeleton is a hierarchical network modelled after the structure of the process. The fleshis formed from the remaining forms of process information, e.g. monitoring program,model, diagngstic rules.The first step is to build the skeleton or structure. We define structure as the physicaland influence layout of the plant. The physical layout consists of pipes, unit processes andsampling locations while the influence layout consists of items whose connection cannotbe reduced to a physical entity, e.g. the weather. The computer views this structure as asequence of records in the database that describe nodes, links and loops in the network.The computer gives each of these elements a unique code. All other forms of knowledgeChapter 6. Structure Paradigm 195“talk about” the layout of the treatmeut plant using these codes.The other forms of process knowledge are broken into three classes:• Measurement: Information on the information gathering processes, e,g monitoring program, QA/QC program.• Derivation: Information that tells the computer how to derive information fromthe data and what this information might mean, e.g Sludge Age.• Reasoning: Information that uses information on the process structure, on theinformation gathering processes, on the information derived from the data andthe monitoring data itself to reason about the process, e.g operating rules andsimulation models.These classes of information are grafted onto the network in the order listed becauseeach is expressed in terms of the classes preceding it (Figure 6.9). For example, theSRT is defined using elements from the monitoring program, e.g. MLSS, and the processstructure, e.g. SRT describes an aspect of the LP.4 Secondary Treatment (Figure 6.5).As discussed in the previous section, the skeletal data structure is a cross between anoutline and a network. An outline is a branching structure that groups elements accordingto their level of abstraction. An outline is an example of a one-to-many relationship, aseach section owns one or more subsections. We refer to the section as the parent andthe subsections as children. An outline enables a user to maintain conceptual controlover a complex system. For example, if an operator added a second primary clarifier toFigure 6.2, the figure starts to become “untidy” because the effluent and sludge linkscross each other. With each added degree of complexity, the figure becomes more andmore incomprehensible to the operator. A similar data structure was used by Narayananand Viswan [222] to uncouple failure detection rules from the structure of a system.Chapter 6. Structure Paradigm 196- •---• ---•-• / Measurement7ur11’ I capacity IStructural____Diagnostic/ExplainPredl JLearnPlanReasoningDerivationFigure 6.9: Overlap Of Classes Of InformationChapter 6. Structure Paradigm 197An alternative approach is to construct an outline of the process, (Figure 6.3). nestsiblings beneath their parent and display each family as a separate diagram. This is theapproach taken in the previous section. The user navigates through the plant by travellingup and down the outline and through the network formed by a group of siblings. Therule of thumb is to limit a network (on a given plane) to no more than eight nodes (seeFigures 6.5, 6.6, 6.7 and 6.8). Perman [242] observed that “experts” 8 describe plantsand their components with varying levels of detail, much in the manner described here.The structure paradigm is based on the assumption that structure is the basis ofknowledge. A recent UBC study analyzed the water quality in 12 apartment buildings [272]. The study took the form of a 2x2 Split-Plot Factorial Design which wasblocked by Run. The experimental layout (i.e. structure) determined what buildingswere studied and how the data were analyzed (by associating each datum with the optimal combination of age, building height and location effects). The analysis exploited thestructure of the data in order to extract the maximum amount of information from thedata set. An observational study of the same size would have yielded far less informationthan the study that was done.The reason for this is that the researcher selected the buildings that would give himthe optimal combination of plumbing and age effects. In this way, he used the structure ofthe study to extract from his data the effect of plumbing type, building age and buildingheight and their interactions. In other words, he used the four principles underlyingthe Statistical Design Of Experiments [79] to his advantage. The alternative would havebeen to select the buildings at random and attempted to classify data after it had beencollected.Similarly, the structure of the treatment plant forces a framework on all process information including monitoring data. The buildings had a “natural organization” that the5The definition of an expert is subject to some debate in wastewater treatment (see Section 2.4).Chapter 6. Structure Paradigm 198researcher recognized and exploited in his study. Similarly, because all the informationcollected on a single treatment plant “talks about” the same system, the structure of thesystem provides a framework with which to analyze information. The structure paradigmquantifies this structure so that the data analysis modules can exploit this structure toextract information about the system. Tn other words, if the computer knows where inthe process the data originate, the computer can use the structure to organize the datafor the operator, making the task of data analysis much easier to do correctly.All the information collected on a system share one common denominator: the system’s structure. At the moment, rule bases, models and monitoring data each have aunique and non-portable manner of expressing structure. This leads to duplication ofinformation as well as an inability to access each other’s data. The alternative is todevelop a language that all the classes can nse that expresses structural context. In thisway, the program only needs one copy of the process’s structure.For example, assume a user wishes to diagnose a problem in the aeration basin. Theuser identifies which unit process he wishes to study, and the program then sets upthe problem by constructing a cause-effect network description of the unit process andretrieves the relevant data sets. The next step is to retrieve from a rule-base the rules associated with the unit process. The inference engine is able to access the monitoring data,information on the monitoring program, a simulation model as well as its own rule-baseusing the system’s strncture. The efficacy of such an approach is recognized by expertsystem developers. One of the conclusions of Perman’s work [242] with SLUDGECADETwas that the best way to improve an expert system is to give it the ability to communicatewith other plant databases, e.g. monitoring data and maintenance information.No known existing commercially available software package can implement the paradigmefficiently. For this reason, a new package will have to be developed, and this activityforms the main thrust of this thesis.Chapter 6. Structure Paradigm 1996.3.1 NodesThe paradigm starts with the presumption that the universe of discourse [120] consistsof nodes. The notion of a node is quite broad. A node may be concrete or abstract.A node is concrete if the object it represents exists as an entity in reality. Examples ofconcrete nodes include a tank (e.g. Figure 6.8: LP.4.3 Aeration Basin), a sample and alocation in the plant. A node is abstract if it represents a concept or a grouping, e.g. theweather and the secondary treatment train, e.g. Fignre 6.5: LP.4 Secondary Treatment.The paradigm uses five “built-in” node types:• Source and Sink nodes form the edge of the system. By definition, a source nodehas no inputs and a sink node no outputs. For example, the sewer into the plantconld be an input node (e.g. Figure 6.5: LP.1) and the receiving water an outputnode (e.g. Figure 6.5: LP.5).• A Point node defines a location in the plant. A point node has one input and oneoutput. The location of the effluent sampler could be a point node.• Split and Join nodes describe the action of splitting and joining a stream. Theparadigm assumes that when a stream is split, each of the split streams possessthe same characteristics as the input stream. In other words, a split node reapportions the currency (e.g. flow) but does alter the stream’s coordinate variables(e.g. BUD5,TSS concentrations). Splitting the secondary clarifier underfiow intoa sludge recycle stream and a sludge wastage stream is a split node (Figure 6.7:LP.4.6 Split). The paradigm assnmes that when a number of streams are joined,their contents are completely mixed to form a single exit stream. The joining of theplant influent and the sludge processing supernatant is an example of a join nodeChapter 6. Structure Paradigm 200(Figure 6.5: LP.2 Mix). The paradigm describes the relationships among nodes using a combination of ordered and unordered sets, and one-to-one and one-to-manyrelationships.6.3.2 LinksA link is an ordered set consisting of three nodes (Figure 6.10):• Parent: The parent node defines the highest level of abstraction that a link reaches.• Source: The origin of the link.• Sink: The destination of the link.For example, Figure 6.7 shows that the clarifier’s underfiow moves to the sludge splitterbox where it is split into two streams: wastage and recycle. The parent node is LP.4Secondary Treatment, the source node is LP.4.4 Clarifier and the sink node is the LP.4.7Split. This link is a concrete link in that it describes a pipe that connects the clarifierto the sludge splitter box. A link may also be abstract meaning it describes a causalconnection that is not reducible to a physical entity, i.e. the influence the weather exertson the operation of the plant.A planar link is a link that connects two nodes on the same plane. The link betweenthe secondary clarifier and the sludge splitter box is a planar link: { LP.4 : LP.4.4LP.4.7 } . The link between the primary clarifier effluent and the secondary treatmentinfluent is a cross planar link (Figure 6.4): { LP.: LP.3.3 LP.4.1 }.A concrete link differs from an abstract link in that a concrete link passes materialwhile an abstract link passes influence. For this reason, we talk about a concrete link’scapacity and an abstract link’s strength. If a link does not pass anything, it ceases toexist.9Syntax: { Parent: Source =‘- Sink }201paradigmChapter 6. StrUCtfh\CroSSPa LinkFigure 6.10 LinkPlanar lrnkChapter 6. Structure Paradigm 202Derived LinksA link may be referred to as being as being raw, derived or hierarchicaL A raw link is alink supplied to the program. The other types of links are derived by the program froma raw link in order to accomplish a task, e.g. plot the plane on the screen. These linksare archetypes of the implementation rather than the paradigm.A derived link is a piece of a raw link. For example, if a raw link crosses from one planeto another, the program divides the link into its planar and hierarchical components muchin the same way a physicist divides motion into its vertical and horizontal components.A hierarchical link is a link that travels within a family, i.e. parent to child. For example,the link { LP.: LP.3.3 [P.4.1 } may be broken into the following components:{ LP.3 : LP.3.3 LP.3 } Hierarchical{ LP. : LP.3 LP.4 } Planar{ LP.4 : [P.4 LP.4.1 } HierarchicalChapter 6. Structure Paradigm 2036.3.3 MappingA map is an ordered set that charts information from one space into another. Theparadigm nses maps in three situations:• Information must be communicated from one class or stream to another. For example, a map associates a hierarchical network description of a particular clarifiermodel with LP.3.2 Primary Clarifier (Figure 6.6).• Information must be communicated between two planes with different coordinatesystems. For example, a primary clarifier model may partition Total Solids into{ Inert Settleable Solids, Volatile Settleable Solids, Inert Nonsettleable Solids,Volatile Nonsettleable Solids }. A map associates the primary clarifier influentsolids measure with the model’s solids component.• Influence is inherited from an ancestor (see Figure 6.11). For example, the user (ofthe program) indicates that the weather affects the plant, i.e. LP.. The weatherdata are collected at a nearby weather station and therefore are not associated withany particular part of the plant. The paradigm assumes then that all the planesbeneath this node are influenced by the weather, e.g. LP.3 Primary Treatment andLP.4.3.2 Aeration Basin. An influence map propagates this information down thehierarchy.The example showed that there are a number of situations when a node has onlylinks coming in. If the link is concrete, this signals that the node is a sink, i.e.a part of the system where material passes out of the system into the world. Ifthe link is abstract, the link indicates that the link’s source influences the link’ssink as wells as all its children. An analysis of the example showed that these twosituations must be treated differently. This is why the notion of influence mappingChapter 6. Structure Paradigm 204was added to the structure paradigm.6.3.4 ClassA class groups uodes and links by the type of information they represent. There are fourclasses of process information (Figure 6.9):1. Structural2. Measurement3. Derivation4. ReasoningEach class expresses knowledge in terms of the classes listed above it, e.g. derive isexpressed in terms of measure and structure.The measurement class consists of four forms of measurement knowledge:• Monitoring Program: At various locations in the plant, the staff take samplesand conduct measurements on the process (on a regular basis).• Diagnostic Program: At various times, the staff take additional samples andmeasurements to diagnose a problem.• Quality Assurance/Quality Control: The staff test out the assumptions onwhich their monitoring program is based.• Capacity: The capacity is the maximum amount of material a unit process canprocess. The algorithm cannot interpret the meaning of an allocation measurewithout knowing the unit process’s capacity, e.g. a plant has three clarifiers but isonly using two at the moment.Chapter 6. StrUCt paradigm 205lntluefl UnkChild Inherits LlflkFigure 6.11: Influefl MapChapter 6. Structure Paradigm 206The purpose of the measurement class is to describe the structure of the iuformatiougathering processes.The purpose of the derivation class is to define numbers that are derived from themonitoring data:• Summary Statistics: e.g. running average, Tukey’s five number summary (seeSection 2.2).• Operation Statistics: e.g. SRT, F/M ratio• Mass Balances: e.g. net substrate consumption in aeration basin• Yields: e.g. solids generated per unit of substrate consumed in aeration basin• Relations: e.g. relationship between effluent solids concentration and MLSS, influent flow rate and underfiow flow rate (see Section 2.2.5).Derivation knowledge may be described by a hierarchical network and a macro. A macrois a set of instructions that the program’s internal language can interpret The macrouses a plane as a data structure. The internal language would have functions that performmass balances, calculate summary statistics and fit relationships. The form that thislanguage should take is a topic for further investigation.Over time, an operator may develop a set of operational rules that are based onboth experience and theory. These rules may take the form of a simulation model or arule-base. One way to implement these is to express them as function that the internallanguage interpreter would call or a small rule-base that the interpreter would pass ontoan inference engine. The functions and the rules would be expressed in terms of the other‘°The use of an internal language interpreter is discussed in Chapter 10. An interpreter enables theuser to write macros that the program can execute.Chapter 6. Structure Paradigm 207classes of knowledge. To date, no commercially available treatment plant software hasachieved this level of integration.The hierarchy of these classes of information highlights the fanlt of some of the research in to treatment plant operations. A good operator does not need a model oran expert system to advise him as much as he needs a way to control the gathering ofinformation on his process and a way to discern the effect of his interaction with hisplant [190].6.3.5 Structural Relationships Path, Loop and NetworkA path is an ordered set of consecutive links and nodes that trace the way (along thetime axis) from one node to another. A path event, a path positioned in time, is a threedimensional entity with the source occurring in time before the sink. This concept isimportant when trying to understand how a loop affects a system. A loop is a paththat begins and ends at the same node. For example, Figure 6.7 shows that returning aportion of the clarifier under flow to the aeration basin creates a loop.The paradigm constructs two types of networks from the links and nodes: (1) MassNetwork and (2) Causal Network. The former includes only concrete links while thelatter includes both.The example includes an algorithm for constructing loops given a set of structurallinks and nodes. Once a loop is constructed, the example program stores the loop andassigns it a unique code. The reason for this is that loop detection is a onerous task.Given that the structure of a plant is relatively static, it is more efficient to direct theprogram to store the loop rather to ask the program to reconstruct the loop over andover again.Chapter 6. Structure Paradigm 2086.3.6 Level and PlaneLevels order planes within a class by degree of abstraction. A plane is a set of nodes andtheir relationships that share a common parent and the same level of abstraction. Theabstraction axis follows a tree-like strncture as it emanates downward until it culminatesin the most concrete level of abstraction. For example, a possible tree in the structural class might proceed from plant (Figure 6.5) to secondary treatment (Figure 6.7) toaeration basin (Figure 6.8).Abstraction is one step in the problem solving process [319]. Reality is far too complexfor us to comprehend so we model what is important and ignore the rest. Abstraction istask driven in that the utility of the abstraction is determined by our ability to use it toaccomplish a predefined task. However, abstraction is not easy. Invariably, our first tryat abstraction turns out be a vague generality. A vague generality is a simplified answerthat appears to be good enough but is not. An abstraction is a refinement of a vaguegenerality.An abstraction has five attributes [319]:1. An abstraction abbreviates or simplifies the thing from which it is abstracted. Anabstraction omits irrelevant details.2. An abstraction characterizes a set of things. An abstraction groups similar itemsinto one set.3. An abstraction is precise. The description is exact enough that there is no doubtwhen the question is asked, “Is this thing an example of that abstraction?”4. An abstraction is complete and accurate. An abstraction contains all relevantdetails.Chapter 6. Structure Paradigm 2095. An abstraction may be hierarchical. Abstraction A may be stated in terms ofabstraction B and C.Abstraction enables the paradigm to organize the process and its ancillary structuressuch that the nser can manage the process. An abstraction is the map that guides theuser when he must find bugs in the system and when he must find hooks onto whichchanges can be hung.6.3.7 Stream and CurrencyA Stream is a group of nodes that share a common currency. The best way to illustratethis is to describe some of the streams that may be found in a treatment plant:• Liquid: The main stream in a wastewater treatment plant is the links and nodesthat process the liquid waste. The currency is flow in [L3/T].• Dry or Wet Chemical: This stream maps the use of chemicals such as methanol,pickle liquor, lime, or chlorine. The currency is mass [M/T] or flow {L3/T].• Air: Maps the aeration system from the blowers to the aeration basins and/or gritchambers. The currency is flow in [M/T Air].• Solids: Maps the solids handling in the treatment plant such as composting ordrying beds. The currency is mass in [M/T].• Energy: Maps the use of energy throughout the plant. The currency may be cost.• Labour: Maps the amount of hours personnel spend maintaining a part of theplant. The currency would be man-hours.• Reliability: Maps the amount of time a piece of equipment is available. Thecurrency would be time.Chapter 6. Structure Paradigm 210A stream carries a coordinate variable, i.e. water carries nitrates and solids. A coordinateparameter must be a fundamental quantity [284]. For example, Figure 6.8 contains twostreams: liquid (L) and air (A). The currency for the former is volume flow [L3/T water]and the latter is mass flow [M/T air].Chapter 6. Siucture Paradigm 2116.4 Structure Of PlanesThe paradigm represents all classes of knowledge using planes and macros. A planeconsists of a group of nodes and their relationships (i.e. sets) while a macro is a set ofinstructions that the internal interpreter would execute (i.e. functions). The purpose ofthis section is to describe the layout of these planes, leaving the procedural elements forfuture study.6.4.1 Structural PlaneA structure plane contains a network of nodes connected by links (see Figure 6.7). Allbut the root plane has a parent. The root plane consists of a single node that delineates the start of the hierarchical tree. This node contains the identity of the systembeing modelled, e.g. City XYZ Water Pollution Control Center. Node LP.4 Secondary(Figure 6.5) is the parent of LP.4? (Figure 6.5). The parent to child set (ParTo€thd)defines this relationship.A plane consists of a network. The network is defined by nodes Nodeinfo and linksLinkinfo. The relationship between nodes and links are defined by two sets: Node to LinkNItoLI and Node from Link NffromLL6.4.2 Monitoring, Diagnostic and CapacityA plane that describes the monitoring program is called a Parameter-Measurement-Sample (PMS) plane. The plane consists of three types of nodes (Figure 6.12) 12:• Parameter: Attribute under study, i.e. Substrate11The database schema is in Appendix E.‘2Figures 6.12, 6.13 and 6.14 use a box to describe a node. The box contains three entries. The topentry is the node’s identity or name, the middle entry is a typical value and the bottom entry is thenode type.Chapter 6. Structure Paradigm 212Parameter:Measure SetN/Sample:Measure SetFigure 6.12: PMS Plane Owned By Structural Node LP.1 Influent• Sample: Samples taken to measure attribute, i.e. Influent composite• Measure: Measure conducted on the sample to estimate attribute, i.e. COD orBOD tests.The relationships among the nodes are described by two sets, Parameter:Measure (ParToMeas) and Sample:Measure (SmpToMeas). The first set connects a parameter to itsmeasures, i.e. Substrate owns COD and BUD5. This set allows the program to reasonabout a parameter. For example, the user may provide a rule that requires a BUD5measure. If there is no BUD5 measure available, the program can substitute the CODBUD5- 250 mg/l -.MeasureChapter 6. Structure Paradigm 213measure by establishing a relationship between COD and BOD5. The second set connects a sample to the measures conducted on it. i.e. Influent Composite Sample ownsCOD, BUD5 and TS. This set enables the program to monitor the sampling program.For example, if the values of all the measures conducted on one sample change, theprogram can warn the user that the change could be due to a sampling problem (andtherefore minimize false alarms).The Diagnostic and Capacity planes are similar to the PMS plane. The only differencebetween a Diagnostic Plane and a PMS plane is their occurrence. The program assumesthat a diagnostic event occurs when the need arises while a monitoring event occurs ona regular basis. Normally, the program polls only the monitoring data unless instructedotherwise by the user (or by a rule). A Capacity Plane does not contain any samplenodes. Instead, a one-to-one set links the capacity measure with its allocation measure(see Figure 6.13).6.4.3 Quality Assurance PlaneA Quality Assurance Plane consists of three nodes (see Figure 6.14):• QA/QC Process: Which quality assurance technique is being used.• Observed Value: The measured result.• Compare Value: The value to which it should he compared.An operator conducts QA/QC tests on the following:• Measurement Process: Blank, Spike or Standard• Measure: Spike or Duplicate• Sample: Spike or DuplicateChapter 6. Structure Paradigm 214Parameter-Measure-Sample Plane Capacity PlaneVolume VolumeAeration Basin Aeration BasinParameter ParameterAllocate Number75 97o 4 TanksMeasure CapacityVolumeSame NodeH 8649m3CapacityFigure 6.13: LP.4.3: BioreactorChapter 6. Structure Paradigm 215PMS Plane QA/QC PlanesFigure 6.14: LP.1 Influent: QA/QC Plane For Composite Sample and COD MeasureFor example, Figure 6.14 shows three planes. The PMS plane describe how the substrateparameter is measured as COD on a composite sample of the influent sewage. The figureshows that two QA/QC planes are attached to nodes in this PMS plane.The first plane is attached to the composite sample node. In this case, the operatorused a second composite sampler (i.e. itinerant sampler) to obtain a second compositesample of the influent and measured the COD on both samples. In practice, he wouldconduct the same analyses on the itinerant sample as he would the regular sample toensure that the regular sampler is not altering the characteristics of the sample [187].The second plane is attached to the COD measure node. In this case, the operator takes a second laboratory sample and spikes this sample with potassium hydrogenCon nectTo QA/QCPlaneChapter 6. Structure Paradigm 216pthalate. The observed value is the difference between the two laboratory samples, i.e.CODspiIced — CODnotspikeci.In order to monitor the quality of the analytical methods used in the laboratory, aPMS plane is created and attached to the root node to model the laboratory. This planecontains a list of all the analytical methods used in the laboratory, e.g. COD and BOD5.The parameter nodes represent the parameters (e.g. substrate), the measure nodesrepresent the measurement methods (e.g. BOD5 and COD tests) and the sample nodesrepresent the origin of the laboratory sample (e.g. standard solutions, dilution water).These nodes are connected to QA/QC planes that are used to monitor the quality ofthe measurement methods, e.g. Chemical Oxygen Demand test. The laboratory samplesused by these planes are created in the laboratory, e.g. standard solutions. The purposeof this PMS-like plane is to warn the operator that a datum may be of poor quality dueto a problem with the test. The paradigm accomplishes this by grouping all measuresthat use a particular a measurement method (e.g. Effluent COD, Influent COD) into aset that is owned by the measurement method node in this PMS plane (e.g. COD teston a standard). If the quality of the measurement process deteriorates, the program willnote that this may be one of the reasons a parameter has changed (i.e. minimizing falsealarms).6.4.4 Derived PlanesA derived plane consists of two types of nodes. The first type is a node that identifieswhat is being derived (e.g. F/M ratio) while the second type indicates which measuresthe derived datum is derived from. The derived datum may be derived from other deriveddata (e.g. COD load and mass of volatile solids under aeration) or raw data (e.g. InfluentCOD, Influent flow, Volume of under aeration and MLVSS). A derived plane does notcreate a new node for a raw datum. Instead, it uses the datum’s PMS measure node.Chapter 6. Structure Paradigm 217This is why derived information is grafted onto the structural skeleton after measurementinformation.A set of instructions that describes how to calculate the datum must be associatedwith the datum. The procedure may either be an internal function or a macro. Forexample, an internal function would be a routine that calculates mass balances while amacro would be an operator-supplied procedure written in the program’s internal language. An internal function is analogous to a spreadsheet function (e.g. ©avg) while amacro is analogous to a macro written using the spreadsheet’s macro language.6.4.5 Reasoning PlanesThe best way to describe a reasoning plane is to use an example. Assume the operatorwants to store the following rule as a plane [172]: If the effluent NH3 is high and theeffluent TSS is low, then increase the air flow rate to the aeration basin. The planewould consists of four types of nodes:• Identity: A unique name for the rule or equation. The computer uses this node torefer to the rule. This node is the parent (i.e. owner) of the input, parameter andoutput sets.• Input: A set of both measurement and derived nodes that are used by the rule,e.g. effluent All3 and TSS concentrations. These references would link the rule tonodes in the measurement and derivation classes.• Parameter 13: A set of nodes identify operator provided definitions, i.e. anotherrule, equation or constant. For example, the definition of “high” NH3 and “low”TSS concentrations.‘3The term “parameter” used here is not to be confused with a parameter in a PMS plane. Here, theterm is used to describe numbers or sets of numbers supplied by the operator.Chapter 6. Structure Paradigm 218• Output: A set of nodes that the rule or equation estimates or comments on, e.g.the air flow rate to the aeration basin.The rule or equation should be expressed in terms of the program’s internal language.Chapter 6. Structure Paradigm 2196.5 An Example: Construction Of The Structural SkeletonThe construction of a data structure, like a building, must proceed one step at a time.If two steps are mutually dependent (i.e. either one can only proceed if the other iscompleted), the data structure cannot be built. The more complex a data structurebecomes, the more difficult it is to determine if it is feasible. The objective of theexample discussed in this section is to determine if the database defined by the schemain Appendix E can be built from two operator supplied text files. The purpose of theexample is threefold:1. The example demonstrates that the computer can construct the database. This isan important consideration because it is possible to design a network database thatcannot be built.2. The example enables us to study the paradigm and correct some of its weaknesses,e.g. the difference between abstract and concrete links discussed in Section The example shows that once structure and graphics information is stored in anetwork database, retrieval is simple.An operator can describe a hierarchical network by constructing two text files. Thefirst text file contains an outline of his process much like Figure 6.3 while the second liststhe links between the outline’s elements as the triple: { Parent : Source = Sink }. Theformat of these files is defined in Tables 6.2 and 6.4 respectively.The database consists of two elements: records and sets. A set is a one-to-manyrelation between records. The database schema (Appendix E) divides the records andnodes into three spaces:• Ideal Space: the space used by the program to reason about the structure, i.e.hierarchical network. The spaces consists of three elements: nodes, links and loops.Chapter 6. Structure Paradigm 220The relationship between nodes and links is defined by two sets: a set of incominglinks and a set of outgoing links. The relationship among nodes (or links) and loopsis a many-to-many relationship, i.e. a node may belong to more than one ioop whilea loop consists of one or more nodes. These relationships are modelled using setsby introducing a new record that owns one loop and one node (i.e. intersectionrecord).• Object Space: the space used to map the network from the graphic space into theideal space. An object spaces describes a plane by dividing the plane into node andchannel objects (Figure 6.16). A channel object consists of a number of conduitobjects. A conduit is a space through which one link may pass through a channel.A link is mapped to a set of conduits that defines its path from the link’s sourcenode to its sink node.• Graphic Space: The graphics space consists of two sets of coordinates. The firstset is used to draw and label a box to represent a node while the second set is usedto draw a set of lines to represent a link (Figure 6.15).For example, assume a user “clicks on” the location 1 shown in Figure 6.15. The programdetermines which object this location falls into (see Figure 6.16). The program determinesthat two links pass through this object and lists them on the screen. The user selects thelink of interest and the program locates the link in the hierarchical network (i.e. idealspace).A set of 15 programs consisting of over 20,000 lines of code were written to constructthis database. This exercise is an example of bottom-up design (see Chapter 1).CD C;’Li)CD(ID CDCt, Cz C C-.CIDCt,-o (I) CDJ\Cl) \CD\C 0 z 0 0. CDøzøzChapter 6. Structure Paradigm 223Table 6.2: Phase 1: Text File Syntax<Section Number> • <Name Of Node> (required)<Nick Name> ,<Node Type>, <X-Coord> <Y-Coord> (required)<Description> (optional)(optional)NOTE • denotes one or more blanks6.6 AlgorithmsThe example consists of 15 executable programs described as phases. Each programopens the database, passes through the database performing a function, and closes thedatabase when finished.6.6.1 Phase 1: Parse In Node Information From A Text FilePhase 1 extracts information on nodes from a text file and constructs node informationand node description records. The node description records enable the user to attach anexplanation to a node. Table 6.2 contains the source file’s syntax. A future implementation should include a designation for a node’s class and stream.6.6.2 Phase 2: Construct HierarchyPhase 2 examines each node’s section number, then searches for its children. The algorithm reads a node’s section number and collects its children. The mode of connectingthe child to the parent is dependent on the child’s plane type, i.e. capacity, PMS orstructural. For simplicity, the PMS planes are attached to their parent structural nodeChapter 6. Structure Paradigm 224Table 6.3: Phase 2: Section Numbers and Node Relationships1.34.54.12 Current Node1.34.54 Parent1.34 Ancestor1. Child1. Descendent1.34.54.12.A2 Capacity1.34.54.23 Sibling1.3.45 No Relationshipvia the ParToCthd set. The final copy of the schema would create a new set for this toavoid blurring the distinctions between the two classes. Table 6.3 explains relationshipsamong structural nodes based on their section number.6.6.3 Phase 3: Parse In Link InformationPhase 3 parses a text source file to obtain information to construct link records. Theparent, source and sink node section numbers describe a link. In this implementation,the source file includes PMS set linkages as well.6.6.4 Phase 4: Construct Link SetsPhase 4 constructs two sets, the node’s output links and input links, to model structurallinks. The program accomplishes this matching of the source and sink section numbersin the link record to those in the node records.Chapter 6. Structure Paradigm 225Table 6.4: Phase 3: Text File Syntax<Parent Section Number> .[<Parent Node Name>] (required)<Source Section Number> (required)<Sink Section Number> (required)<Link Type>,<Nick Name> (required)<Description> (optional)(optional)NOTE: • denotes one or more blanks6.6.5 Phase 5: Build Interplanar LinksAn influence may be mapped from a parent to its descendants by either creating anumber of links from the parent to the descendants or by creating a number of links fromthe source to the descendants. In this implementation, the former approach was taken.However, the example showed that the latter approach would have been more efficientand logically safe. Therefore, the next implementation should use the latter approach.6.6.6 Phase 6: Build Sample and Measurement SetsPhase 6 constructs sample and measurement sets using links that model PMS type relationships. In retrospect, it would have been better to process the PMS informationseparately from the structure information. This would simplify the code considerably.Chapter 6. Structure Paradigm 2266.6.7 Phase 7: Loop DetectionPhase 7 detects and stores ioops. The algorithm travels from source nodes to sink nodes.Along the way, the algorithm pushes its path onto a stack. With every move, the algorithm checks to see if a node appears on the stack twice. If so and if the loop is notalready stored in the database, the program dumps the loop into the database.6.6.8 Phases 8-11: Preparation For PlottingThe purpose of phases 8-11 is to construct a set of nodes and links that form the basisof a graphic presentation of the plane. Phase 8 decomposes a cross-planar link into itsconstituents within and across planar components. Phase 9 groups the derived and rawlinks contained in a plane and attaches them to the plane’s parent (PARowrisLI). Phase10 generates a pseudo node to act as source and sink nodes for those derived links missinga source or sink node in the plane. Phase 11 determines the maximum number of linksleaving or entering a node in a plane.6.6.9 Phases 12-15: Object and Graphic SpacePhase 12 maps a link into a set of channel objects, Phase 12a generates the graphicscoordinates of these objects and Phase 14 generates a sequence of points that describe aline from the output edge of a source node to the input edge of a sink node. Phase 13generates the graphic coordinates for a node.6.6.10 Validation: Draw A PlaneA simple routine was written to retrieves the graphics coordinates and draw a plane.This routine was used to ensure that the database resembles the plant it is suppose todescribe.Chapter 6. Structure Paradigm 2276.7 ConclusionThe structure of information forms the vocabulary with which information is best expressed. The structure paradigm formalizes this vocabulary in terms of a hierarchicalnetwork. The skeleton of the network describes the structure of the process being studied. On to this structure, measurement, derivation and reasoning classes of informationare grafted in the order mentioned. This approach forms the basis of an integrated approach to the storage and manipulation of information collected on a process and on itsinformation gathering processes.Chapter 7Measurement ParadigmThe order of evolution is from raw data to information, from information toknowledge, and from knowledge to wisdom. [9]The purpose of this chapter is to explain the Measvrement Paradigm.. The MeasurementParadigm maps process measurements into a common space so that the data can beanalyzed as a single unit. A datum may take on a variety of forms, many of which areincompatible. For example, a string and a number are incompatible data types becausethey must be processed differently. The paradigm avoids data apartheid by mapping alldata to one of three compatible types: Crisp Number, Mean and Standard Deviation andFuzzy Set. Two data types are compatible if they can be stored in the same database andcan be manipulated by the same program. For example, time and currency are Lotus1-2-3 © compatible because a 1-2-3 stores them both as real numbers.This chapter consists of 8 sections:• Declaration Space: Defines the origin of a datum relative to a datum’s PMSplane.• Definition Space: Provides a datum with a preference. quality and (in somecases) value relation that maps a datum to a quad. This quad consists of aninterval, value, preference and quality.• Data Space: Describes how a quad is stored in a database, i.e. internal formats.228Chapter 7• Measuiement Paradigm 229• Primary Mapping. Explai5 how the different forms of data are mapped to oneof the three internal formats.• Viewpo;. Introduces the concept of a viewpoint A viewpoint is a data analysistemplate• Secondary Mapping: Describes how a viewpoint and a datum’s declaration guidethe construction of raw and derived time series• Tertiary Mapping: Describes how a viewpoint and a datum’s declaration guidethe grouping of raw and derived time series.The objectj of this Paradigm is to help the operator gain control of his informationgathering processesChapter 7. Measurement Paradigm 2307.1 Declaration Space: Origin Of A DatumA datum is the product of a measurement process. A measurement process is characterized by three dements: Parameter, Sample and Measure. In the previous chapter,it was explained that the structure of the measurement process is characterized by aPMS (or PMS-like) plane. A datum is placed in the context of the information gatheringprocess by associating the datum with the appropriate measure node in the PMS plane.Because the PMS plane is associated with a structure node, the datum is also placed inthe context of the layout of the treatment plant. This process of attaching a datum toa measure node which in turn is associated with a measurement process and a positionin the plant is referred to as “mapping into the declaration space”. The objective of thissection is to elaborate on the elements of the declaration space.7.1.1 Parameter ContextA Parameter is a characteristic of the stream under study, i.e. Substrate Concentration. The primary purpose of a parameter is to group measures of the same streamattribute together, e.g. COD and BOD5 under Substrate Concentration. A parameterhas the following attributes (Figure 7.1):• Structural Context: The structural context is an unique code that identifieswhich node represents the parameter in the structure database.Chapter 7. Measurement ParadigmParameter Context231—] AdjustmentAllocation—H DisturbanceStatusMeasure Paradigm: Parameter ContextStructural ContextControl ContextCause—1 ManipulatedHEffect—H Status-L Performance— QA/QCH Model Context J—J Currency ObservableCoordinateHHFigure 7.1:Chapter 7. Measurement Paradigm 232• Control Context: A user measures a parameter to make a control decision:— Adjustment or Allocation Parameters: By definition, an adjustable orallocated parameter is not measured, but set (e.g. recycle flow rate). For thisreason, the paradigm assumes their measures are without error and their valueremains the same until they are reset. An adjustable or allocated parameter isan operator determined input, i.e. a manipulated or operator-set parameter.— Disturbance Parameters: By definition, the cause of a change in a disturbance parameter is outside the system described by the structure paradigm(e.g. influent BUD5 concentration). In other words, a disturbance parameteris an externally (versus internally) controlled input.— Performance and Status Parameters: If the operator flags a parameteras being an important effect (i.e. Effluent BUD5), the paradigm refers to thisas a performance parameter. Otherwise, the parameter is a status parameter.A status parameter is not necessarily a state parameter. 1— QA/QC Parameter: A QA/QC parameter is not an attribute of the structure hut of the monitoring program (i.e. described by a QA/QC plane).A parameter whose value is not set by the operator is referred to as being an observable parameter. The value of an observable parameter may only be determinedthrough measnrement.‘Notion Of State: Some qualitative information ( a set of numbers, a function etc.) which is theleast amount of data one has to know about past behaviour of the system in order to predict its futurebehaviour. The dynamics is then described in terms of state transitions, i.e. one must specify how onestate is transformed into another as time passes [179].Chapter 7. Measurement Paradigm 233• Model Context: A user measures a parameter to model the system (Figure 7.2):— Currency Parameter: A currency parameter is the flow or volume in astream, e.g. liquid, mass of air or dollars.— Coordinate Parameter: A coordinate parameter is an attribute of the currency parameter, e.g. solids or substrate concentration. A coordinate measuremust be a fundamental quantity.— Status Parameter: A status parameter is none of the above, e.g. pH orredox potential. A status parameter is not necessarily a fundamental quantityor a state parameter.By definition, a QA/QC parameter cannot be a model parameter.One way to view this classification system is to relate it to a simple model. Assumethat we have a chemostat with a pure culture. The influent flow rate is a manipulatedparameter (assuming it is set by the operator), the infiuent substrate concentration is adisturbance (assuming it is not set by the operator), the reactor pH is a status parameter(assuming it is not how the performance of the process is measured) and the effluentsolids concentration is a performance parameter (assuming the object of the study is togenerate solids).Similarly, given that the liquid volume is the system’s currency, then both the influentflow rate and the volume of liquid in the reactor would be currency parameters. Anyparameter that satisfies a conservation principle is a coordinate parameter, i.e. substrateand solids concentration. The remaining parameters are (model) status parameters.Chapter 7. Measurement Paradigm 234Figure 7.2: Model Context : An Exampledx10 Q a25x912= (yio — xio) + (7.13)cIt 1/ a26+x9Q, V Flow and Volume Currencyx,y Nitrate-N CoordinateNitrite-N CoordinateX12 Nitrobacter Coordinate7.1.2 Sample ContextA Sample is the portion of the stream on which a measure is conducted. A samplehas the following attributes (Figure 7.3):• Structural Context: The structural context is an unique code that identifieswhich node represents the sample in the structure database.• Type Context: The type of sample refers to how the stream is divided up forstudy. Common sample types inciude grab. composite and probe. Sample typeswere discussed in Section 4.3.• Time Context: A composite sample, and sometimes a probe sample, sample thestream over an interval rather than at an instance. The effect of this on the dataanalysis is discussed in Section Measurement ContextA Measure possesses four attributes (Figure 7.4):• Structural Context: The structural context is an unique code that identifieswhich node represents the measure in the structure database.Chapter 7. Measurement Paradigm 235L Sample ContextHFigure 7.3: MeasurementH Structural ContextType ContextH GrabH CompositeH ProbeH Time Context—H InstanceIntervalParadigm: Sample ContextChapter 7. Measurement Paradigm 236L Measurement ContextStructural ContextScale ContextNominalOrdinal—H IntervalH RatioResolution ContextH Crisp-H VagueDefinition ContextFigure 7.4: Measurement Paradigm: Measurement ContextChapter 7. Measurement Paradigm 237• Scale Context: A measure may possess one of four scales: Nominal, Ordinal,Interval or Ratio. Measurement scales were discussed in Section 4.1.• Resolution Context: A measure may be crisp or vague. A crisp measure may beaccompanied by a measure of its precision. The notions of crisp and vague measureswere discussed in Chapter 5.• Definition Context: A measure possesses a definition, e.g. filtered COD.Chapter 7. Measurement Paradigm 2387.2 Data Space: Derivation Of A DatumAll data are not of equal value. The value of a datum (to the operator) depends partly onthe datum’s preference and quality. Preference describes how an operator “feels” abouta datum while quality describes how reliable the datum is. For this reason, the paradigmmaps a datum (and its time) to the quad: { Interval, Value, Preference, Quality }, i.e.Primary Mapping (Figure 7.5). This quad forms the basis of the data space.In order to map a datum into the data space, both a declaration and a definition mustbe associated with the datum’s measure. A definition consists of two, and in some cases,three relations. The first two relations are the preference and quality distribution. Thethird relation alters a datum’s magnitude. This relation is optional. All three relationsare somewhat arbitrary as the operator defines the relations to suit his needs (i.e. therelations are plant specific).7.2.1 QualityThe measurement process and execution affect quality. The effect of the process on adatum’s quality is known (for the most part) before the fact. For example, consider themeasurement specification for the determination of Chemical Oxygen Demand (COD)using the Dichromate Reflux Method [176]. We can derive a quality distribution for a CODmeasure by recognizing that the COD is dependent on two factors. The first factor isthe difference between the amount of Ferrous Ammonium Sulphate (FAS) used to titratethe blank versus that used to titrate the sample. If the sample is weak, the differenceis small. If the sample is strong, the difference is large. Standard Methods [176] suggestthe difference should not be larger than 22.5 mL which is half the sample volume andnot smaller than 1 mL. This means if the COD is over 900 mg/l 2, the sample must21f a 25 mL sample is used, the dilution breakpoint is 450 mg/I COD. A 25 mE sample is used whena 125 mL erleumeyer flask is usedChapter 7. Measurement Paradigm 239DatumInstanceII Data SpaceDefinitionIntervalValueMap FromInput DatumToData SparePreferenceQualityDeclarationFigure 7.5: Primary Mapping: Datum To Data SpaceChapter 7. Measurement Paradigm 240be diluted. Because dilution introduces another source of error, the quality of the testdeteriorates as the dilution required increases.We can derive a quality function, eQQD(COD), using the limitations placed on theCOD test by the FAS difference and dilution restrictions. The function is given inequation 7.14 and plotted in Figure 7.6:1. COD < 25 mg/l: The COD test is unreliable so the Quality is zero. This meansthe value is somewhere below 25 mg/i.2. 25 mg/i < COD < 50 mg/i: Standard Methods suggests the analyst use a0.025 N rather than 0.25 N FAS solution [176]. Standard Methods warns thatthis second test is very sensitive to interferences. A value in this region shouldbe viewed as being in range of 25 to 50 mg/i. Because the two FAS normalitiesrepresent separate tests, the structure paradigm would create a measure node foreach test and group the two measure nodes under a single parameter node.3. 50 mg/i < COD < 900 mg/i : The COD test is reliable over this range. Thevalue should be viewed as being crisp.4. COD > 900 mg/i: When the COD exceeds 900 mg/i, Standard Methods suggestthat the laboratory sample be diluted so that the sample’s COD is within therange of the test. However, the reliability of the test result decreases as the dilutionrequired increases.Chapter 7. Measurement Paradigm 241This quality function,9co, is given below:0 if COD <25 mg/i1 — 50—COD if 25 <COD < 50 mg/i25—OCOD(COD) = (7.14)1 if 50 COD < 900 mg/iif COD > 900 mg/iThe effect of the execution of the measurement process on a datum’s quality shouldbe monitored continuously through a QA/QC program. The paradigm encourages theoperator to flag suspect data to avoid the change detection module from raising a falsealarm. Bad data pose a greater threat to the system than missing data because bad datalead to had conclusions. The secret of dealing with bad data is simple: “don’t have any”.Or, at least, flag the data so that the program will ignore it. Gy’s measurement model,discussed in Section 4.5, could help the operator design his QA/QC program.7.2.2 PreferenceA preference distribution tells the program “how desirable” a datum is. For this reason,the operator wants the preference function to be most sensitive in the range where thedesireability of the datum changes. For example, a treatment plant’s discharge permitstates that the effluent COD must not exceed 50 mg/i. Given the error in the CODtest. the operator would prefer if the measured effluent COD value was less than 40mg/i. The closer the effluent COD is to the permit value, the higher the likelihood thatChapter 7. Measurement Paradigm 242a noncompliant COD will be observed. A typical preference distribution, for theEffluent COD is given below (see Figure 7.7):1 if COD <30 mg/lQGOD(COD) = 1 — 80D if 30 COD <80 mg/i (7.15)0 if COD > 80 mg/iChapter 7. Measurement Paradigm 243Chemical Oxygen Demand TestFAS Normality = 0.25 N1.0• I I III I I I I I 1111 I I I I I I III I I I I I liii I I I I I III1 10 100 1000 10000 100000Sample COD (mg/I)Figure 7.6: Quality Distribution For COD TestChapter 7. Measurement Paradigm 244Effluent Chemical Oxygen DemaiPreference1. 0.6cU)’0.0 I I I I I0 10 20 30 40 50 60 70 80 90 100Sample COD (mg/I)Figure 7.7: Preference Distribution For COD TestChapter 7. Measurement Paradigm 2457.3 Data Space: Internal Representation Of A QuadThe data space consists of the quad: { Interval, Value, Preference aud Quality }. Aquad is stored by the computer as a queue of real numbers. Depending on a datum’ssample type, a datum may represent an interval or an instance. Because an instance canhe described as an interval, the paradigm expresses time as an interval, i.e. two queueentries. By definition, preference and quality are expressed as a crisp numbers between0 and 1. Therefore, they require one queue entry each.A datum’s value may be stored in one of three forms: crisp number, mean andstandard deviation, or fuzzy set. Each of these require a different amount of queuespace:1. Crisp Number: A crisp number is a single value. The number may represent oneof two values: (1) a nominal measure, e.g. superuatant appearance (see Table D.2,Appendix D) or (2) a crisp ordinal, interval or ratio measure, e.g. 75 mg/l COD.Preference (optional : defalllt to 1) 0.10Quality (optional : default to 1) 1.0Value (required) 75.02. Mean and Standard Deviation: A mean/standard deviation consists of twovalues. The numbers represent an interval and, by definition, apply only to intervaland ratio measures. The statistics summarize a number of observations taken overan interval, usually by an on-line device. The interval should be short enoughto ensure the standard deviation is a valid estimate of the mean’s precision. ForChapter 7. Measurement Paradigm 246example, assume we wish to store the dissolved oxygen as measured by an on-lineprobe over the last hour. In thsi case, the mean is 2.0 mg/l and the standarddeviation is 0.8 mg/l:Preference (optional : default to 1) 1.0Quality (optional : default to 1) 1.0Value (required) 2.0Standard Deviation (required) 0.83. Fuzzy Number or Linguistic Variable: Fuzzy numbers and linguistic variableswere discussed in Chapter 5. A fuzzy number may be stored in one of three ways: asa standard function, as an a-set or as a piece wise linear approximation. The firstalternative limits the operator to a small set of standard functions while the secondalternative requires too much space. For these reasons, the last alternative is used.The fuzzy set is represented by a set of membership and magnitude coordinates.If the fuzzy set’s basis set is noncategorical, the paradigm interpolates betweencoordinates by drawing a straight line. For example, the following queue stores“less than 25 mg/l COD”:Preference (optional : default to 1) 1.0Quality (optional : default to 1) 0.0Magnitude (required) 0.0Possibility (required) 1.0Magnitude (required) 25Possibility (required) 1.0Magnitude (required) 35.0Possibility (required) 0.0As many pairs as neededGhapter 7. Measurement Paradigm 247Because a queue represents a quad (which represents a datum), the queue must begrouped under a unique record formed from the datum’s time interval and the datum’smeasure code.7.3.1 Manipulated Parameter: A Special CaseBecause an operator sets rather than measures a manipulated parameter, manipulatedparameters represent a special case. A manipulated datum differs from other data in twoways:1. By definition, the quality is always 1, i.e. good.2. A new datum is only entered when the parameter is changed by the operator.For these reasons, the quality measure is replaced by a flag that indicates why the operatorchanged the parameter (see Table 8.2).Chapter 7. Measurement Paradigm 2487.4 Primary Mapping: Mapping A Datum Into The Data SpaceA datum may take on a variety of forms:• Single or Crisp Number (e.g. 300 mg/i BOD5)• Range of Equally Possible Numbers (e.g. COD is less than 25 mg/i)• Fuzzy Number or Range of Numbers With Different Degrees of Possibility (e.g.The foam covers about half of the basin’s surface)• Arithmetic Average with a Precision (e.g. the average DO over the last hour was2.0 mg/l with a standard deviation of 0.5 mg/i)• Category (e.g. the sludge is buiking)• Linguistic Variable that is not a Fuzzy Number (e.g. the condition of the clarifieris very poor)Each datum is important and should not be excluded from the database (and the dataanalysis) on the basis of their form (i.e. data apartheid). However, these data can notbe analyzed in their current form because their types are incompatible (i.e. string versusnumber). For this reason, the paradigm maps these data into one of the three internalformats modifying their value on the basis of their quality (Figure 7.5).7.4.1 Mapping: {Crisp, {Ratio, Interval, Ordinal }}Given a crisp datum measured on a non-categorical scale, the program calculates thedatum’s preference and quality and stores all three in the database. The time may be aninstance if the sample was a grab, single probe reading or an observation, otherwise, thetime is an interval. For example, the effluent COD is measured on a 24 hours compositeChapter 7. Measurement Paradigm 249Table 7.1: Measure Paradigm: Effluent COD ExampleEntered Quality [ Preference [ Stored20 mg/l 0.0 1.0 Fuzzy Number: Less than 25 mg/l30 mg/l 0.2 0.84 Fuzzy Number: Between 25 and 50 mg/l75 mg/l 1.0 0.10 Crisp : 75 mg/lsample. The program would record the datum as being indicative of the compositeinterval, i.e. from Monday 8:00 AM to Tuesday 8:00 AM.An operator may modify the storage of the Effluent COD by including the followingrules:• IF COD 25 mg/lTHEN COD is a number less than 25 mg/l• IF (COD > 25 mg/l) AND (COD 50 mg/l)THEN COD is a number between 25 and 50 mg/I• IF COD > 50 mg/lTHEN COD is the measured CODTable 7.1 and Figure 7.8 show the results. The advantage of this approach is that theprogram will not detect a change in trend or stability in regions where the quality of thedata is low. This will reduce the number of false alarms and simplify the causal analysis.7.4.2 Mapping: {Crisp, NominallA categorical or nominal datum is stored as a crisp value. If the user supplies the categoryas a string, the program retrieves the corresponding crisp value from the dictionary. Thecrisp value is stored with its quality and preference.Chapter 7. Measurement Paradigm 2501.075 mg/I COD30 mg/I CODI I I10 20 30 40 50 60 70 801.0I41.0I1.0I I I I I I10 20 30 40 50 60 70 80I I I I10 20 30 40 50 60 70 801.01.0I I I I I I10 20 30 40 50 60 70 8010 20 30 40 50 60 70 80I I I I I10 20 30 40 50 80 70 80Figure 7.8: Measure Paradigm: Effluent COD ExampleChapter 7. Measurement Paradigm 2517.4.3 Mapping: {Mean/Standard Deviation, {Ratio or Tnterval}}By definition, a Mean/Standard Deviation datum must have either a ratio or intervalscale. The operator may modify how the datum is stored. If we assume the datum isnormally distributed, we can calculate the preference and quality as a weighted averageover the distribution. For example, assume the operator measures DO on-line. Themonitoring system generates a reading every minute and averages the values over the hourand calculates their standard deviation. This reduces the number of data points from 60to 2. The average and standard deviation are stored in the database as representing thehour. A moving average should not be used for this purpose7.4.4 Mapping: {Fuzzy Number, {Ratio or Interval} }By definition, a Fuzzy Number must have either a ratio or an interval scale. The programretrieves the number’s definition from the dictionary and stores the definition. Thedefinition is stored as a piece-wise function. We calculate the preference and qualityas a weighted average over the possibility distribution. For example, the Effluent CODexample used two trapezoidal fuzzy numbers (x, i)’• COD is less than 25 mg/l:{(0, 1.0), (25, 1.0), (35, 0.0)}• COD is between 25 and 50 mg/h{(20, 0.0), (30, 1.0), (45, 1.0), (55, 0.0)}3A Discrete Average summarizes the data while a Moving Average smooths the data. The distinctionis important because most statistical methods, especially Analysis of Variance, assume the data areindependent of each other.Chapter 7. Measurement Paradigm 252These definitions are arbitrary (i.e. set by the operator).7.4.5 Mapping: {Linguistic Variable, Any Scale}The program retrieves the linguistic variable’s definition from the dictionary, and storesthe definition. We calculate the preference and quality as a weighted average over thepossibility distribution. The basis set of the measure “Clarifier Supernatant Appearance”is contained in Appendix D. We define the condition of the clarifier as being very poor ifthe sludge is clumping, bulking or washing out with less emphasis on the other conditions(see Table 7.2).The supernatant appearance scale is nominal (Table D.2). The definition assigns aninteger to each class, e.g. Bulking=1, Ashing=2, etc. The computer stores the value“poor” by replacing the category with its corresponding integer.Table 7.2: Linguistic Variable: Clarifier Condition Is PoorBase Variable MembershipBulking 1.0Ashing 0.6Straggler Floc 0.6Pin Floc 0.6Clumping 1.0Washout 1.0Normal 0.0Chapter 7. Measurement Paradigm 2537.5 ViewpointIn order to analyze the data, the operator needs three things:• A list of what to analyze (e.g. all influent parameters)• A time interval over which the analysis should take place (e.g. the last month ofdata)• A list of what analyses to perform (e.g. cause and effect, summary statistics,...)Given the context of the data (i.e. each datum’s definition and declaration) and ananalysis framework, a computer program should be able to do the following:1. Connect a task to a data set, a set of functions and a set of conditions:For example, “I want to look at the aeration basin” would mean plot the data as atime series, conduct mass balances on the basin’s coordinate parameters, list whatseries have changed and calculate summary statistics for all the parameters.2. Perform the correct types of manipulations for a given a data set: Forexample, the logic for calculating a set of simple exploratory statistics is based onthe measurement scale and the number of data points (see Table 7.3).This framework is referred to as a viewpoint. By shifting the question specific componentsonto a viewpoint, the program is free to use the same menu system and interface for everyviewpoint.A viewpoint is a template whose purpose is to focus on the relationships among thedata. For this reason, a viewpoint is associated with a question the operator may askabout his data, e.g. which parameters changed? A viewpoint consists of five components:Chapter 7. Measurement Paradigm 254Table 7.3: Calculate Exploratory StatisticsIF Measure(scale) is categoricalCalculate reference distributionELSE IF more than 7 data pointsCalculate Tukey’s five number summaryELSE IF more than 2 data pointsCalculate maximum, median and minimumELSECalculate averageENDIF1. Table: A viewpoint constructs a list of measures that it will look at. The contentsand order of this list may be restricted by the viewpoint. The table consists of fourcomponents:• View Restrictions: What type of measurements may be viewed, i.e. whichraw series to extract from the database.• Order Restrictions: What criteria may be used to control the order inwhich the data sets are to be viewed, i.e. how to group the data series ( parameter).• Duration: The length of time over which to extract data from the database,e.g. month or year.• Windows: How the duration is to be split into windows, i.e. which derivedseries to prepare (e.g. weeks or SRTs).2. Manipulation: A decision table and a set of functions used to massage the datasets into a form used by the viewpoint, e.g. calculate first or second differences.Chapter 7. Measurement Paradigm 2553. Display: A decision table and a set of fnnctions to display the data in text andgraphics mode.4. Comment: A decision table and a set of fnnctions that are valid within the viewpoint and operate on the data series, e.g. highlight preference data below 0.5.5. Report: A list of fnnctions that produce reports, e.g. list duration summarystatistics.The following two examples illustrate how a viewpoint uses the measurement scaleand parameter control type. Table 7.4 contains part of the display decision table toform a viewpoint that plots the data against time and calculates some simple summarystatistics. The viewpoint allows the user to specify where the program should locatethe X-axis. The scatter plot function plots the data points. If the datum represents aninterval, the scatter plot function represents the datum by a bar that stretches over theinterval. Otherwise, the datum is represented by a point. Operator set data are plottedas a post-point step function . The other types of parameters are joined by a solid line.For example, if the parameter is a manipulated parameter (i.e. recycle rate) and thescale is ratio, the operator would see a step function on the screen. He would be able tomove the X-axis to the mode-range, median or mean.Table 7.5 is taken from the manipulation section of viewpoint that takes the first andsecond differences on time series. A transition sequence is a list of intervals over whichthe first difference changes sign. For example, if the parameter is a manipulated variable(i.e. recycle rate) and the measurement scale is ratio, the viewpoint would calculate thefirst difference series and construct a list of points where the derivative changes sign.4A post point step function is one where the datum starts the step rather than ending itChapter 7. Measurement Paradigm 256Table 7.4: Time Series Viewpoint : Display Decision TablePARA(Control):{Adjust,Allocate} T T T T F F F FMEAS(Scale):{ N=Nominal, O=Ordinal, I=Interval, RRatio } N 0 I R N 0 I RStep Fuuction:Post Point X X X XLine:Solid X X X XScatter Plot X X X X X X X XY-Axis Type:Category X X X XX-Axis Y-Position:Mode X X X XX-Axis Y-Position:Mode Range X X X XX-Axis Y-Position:Median X X X X X XX-Axis Y-Position:Mean X X X XX-Axis Y-Position:Interval Weighted Mean X X X XX-Axis Y-Position:Harmonic or Geometric Mean— XTable 7.5: Differenced Time Series Viewpoint : Series Manipulation TablePARA(Control):{Adjust,Allocate} T I’ F FMEAS(Scale):{Nominal,Ordinal} T F T FMEAS(Scale):{IntervaLRatio} F T F TGenerate First Difference Time Series X X X XGenerate Second Difference Time Series XGenerate Transition Sequence X X XGenerate Max, Mm and Inflection Points—— XChapter 7. Measurement Paradigm 2577.6 Secondary Mapping: Quad To SeriesA raw series is a set of quads ordered by interval, i.e. time series. The duration of theseries is the length of time between the start of the first quad’s interval to the end of thelast quad’s interval. The duration of a series may be broken into equally sized windows.A derived series consists of a summary of the quads within each window. For example,an element of a derived series would consist of summaries of the preference, quality andvalue of quads contained within a window. The quality and preference summaries couldconsist of Tukey’s five number summaries (see Section 2.2). The value summary couldconsist of Tukey’s five number summaries for non-categorical data and histograms forcategorical data. The mapping of data quads contained within a duration to a number oftime series is referred to as secondary mapping (Figure 7.9). This mapping is controlledby both a viewpoint and each datum’s declaration.Chapter 7. Measurement Paradigm 258ViewpointRaw SeriesDatum QuadDerived SeriesMap From Derived SeriesData Space____________ToData SeriesDatum QuadDerived SeriesDatum QuadSeries SummaryData Space_________________Data Series SpaceDeclarationFigure 7.9: Secondary MappingChapter 7. Measurement Paradigm 2597.7 Tertiary Mapping: Grouping Data SeriesA viewpoint may require that the data series be organized into different groups. This isreferred to as tertiary mapping (Figure 7.10). For example, a viewpoint that calculatesmass balances and yields around unit processes would group the series first by parameter(e.g. substrate) and second by whether the parameter characterizes a stream entering orleaving a unit process.Chapter 7. Measurement Paradigm 260ViewpointRaw SeriesGroup 1 SeriesvedSeries’_Group 2 SeriesDerived Series I Map FromData SeriesToViewpoint...Derived SeriesGroup 3 SeriesSeries SummaryScc.I________________DeclarationFigure 7.10: Tertiary MappingChapter 7. Measurement Paradigm 2617.8 SummaryThe Measurement Paradigm builds on the Structural Paradigm by providing a datumwith a measurement context. The paradigm relies on three sources of information toguide the paradigm’s extraction of information from a data set. The first two sourcesof information are a datum’s declaration and definition. A summary of the declarationand definition of the effluent COD example is given in Table 7.6. The third source ofinformation is a viewpoint. A viewpoint is associated with a data analysis task ratherthan with a datum and its PMS representation. A viewpoint guides the derivation andgrouping of time series.The derivation of information is accomplished by three mappings. The primary mapping associates a datum with a quad (Figure 7.5). A quad consists of an interval, avalue, a preference and a quality. The interval defines the time span the datum represents. The value may take on various forms, all which represent the datum’s magnitudeand all which are mapped to one of three internal formats. Preference describes how theoperator “feels” about the datum and quality indicates how much weight the operatorcan put on the datum.The secondary mapping maps data to raw and derived time series (Figure 7.9). Aderived time series consists of a series of window summaries. For example, the windowmay be the least common interval between two raw series (see Section 2.2). The tertiarymapping groups time series according to the needs of a viewpoint (Figure 7.10). Forexample, the cause and effect viewpoint (which is discussed in the next chapter) groupsdata series by a datum’s parameter control context, i.e. { adjust or allocate, disturb,perform, status, QA/QC }.Statistical packages and other data analysis programs assume that quality of the datais good. The onus is on the user to ensure that this is the case. However, when data areChapter 7. Measurement Paradigm 262Table 7.6: Measure Paradigm: Effluent COD SummaryComponent Context ExampleDeclaration SpaceParameter Structure Effluent SubstrateControl PerformanceModel CoordinateSample Structure Effluent Composite SampleType CompositeTime 24 HoursMeasure Structure Effluent CODScale RatioResolution CrispDefinition None NeededDefinition SpaceQuality Preprocess see equation 7.14QA/QC Test is checked against a standardSample is checked against a duplicatePreference Distribution see equation 7.15Chapter 7. Measurement Paradigm 263collected on a routine basis and simply archived, there is the tendency to get careless.The problem of data quality was discussed in Chapter 4.The measurement paradigm enables the operator to correct the data for the limitations of the test method, to monitor the quality of the monitoring program and todocument any changes in the sampling or measurement methods. i.e. control the information gathering process. This is important because we cannot distinguish between achange induced by the problems in the monitoring program from changes in the system.An excellent example of this occurred at Beckman and Southwest Treatment Plants inJacksonville where the plants were brought into compliance simply by installing a newcomposite sampler [187]. In this case, the problem was with the information gatheringprocess, not the plant.Chapter 8Operation ParadigmFinally, we note that with messy data and unclear objectives, the problem isnot how to get the optimal solution, but how to get any solution. Once again,philosophical problems seem rather irrelevant. [73]The purpose of this chapter is to explain the Operation Paradigm. The OperationParadigm enables the operator to determine the effect of his actions on the process.The Chapter is divided into five sections;1. What Is Status?2. What Is Change?3. Postdiction: Why Was A Manipulated Variable Changed?4. Prediction: How Should A Manipulated Variable Be Changed?5. ConclusionThe Chapter mentions two viewpoints; Change Viewpoint and Cause/Effect Viewpoint. If an operator changes a manipulated variable, the manipulated variable becomesthe response variable in one of the eight Change Viewpoints (Table 8.2). Other manipu264Chapter 8. Operation Paradigm 265lated variables may be members of the viewpoint depending on their relationship to theresponse and initiating variables. A change viewpoint ceases to exist when one of thefollowing conditions occurs:• The change is deemed insignificant: The time at which the change was madeis now outside the data series duration.• The change is deemed successful or unsuccessful: A change goal is definedby what the operator hopes will happen and/or by what he hopes will not happen.• The viewpoint is no longer relevant: A viewpoint is based on a set of assumptions about the relationship among the variables. If these assumptions are nolonger valid, the viewpoint ceases to exist.A Cause and Effect Viewpoint is nested in each Change Viewpoint. An operatorcan lock on a variable and ask the Cause and Effect Viewpoint to organize the othervariables relative to this variable. For example, the graphics screen is divided into fourplots: Manipulated, Disturbances, Performance and Effect. If the operator locks on adisturbance variable, the plots in the other three windows are grouped as follows:• Perform and Status: The set of effect variables affected by the disturbance, i.ewhat does the disturbance variable affect?• Manipulate and Disturb: The set of cause parameters that also affect the aboveperformance parameter, i.e. what might offset the effect of the disturbance parameter?Because the paradigm relies on the detection of change, the paradigm will fail if thechange detection routines fail. Therefore, the operator is able to overrule the paradigmat any time.Chapter 8. Operation Paradigm 2668.1 What Is Status?A derived time series’ duration is split into equally sized windows. The status of theseries is itself a series that describes the magnitude (location) and stability (width) ofthe data within a window. A status element consists of the following items:• Interval: Size of the window.• Count: Number of data points in the window.• Position: The window’s position relative to the other windows.• Direction: The direction the time series is moving.• Stability: The variability of the time series.The user defines a duration that determines how far back the analysis should lookat the data. The user breaks this duration into a number of windows that define short,medium and long term behavior, i.e. at least three derived time series per raw time series.The program generates a status report for each window. When possible, the programcalculates the position, trend direction and stability for the window summary’s value,preference and quality. The window statistics should be simple (see Section 2.2). Forexample, Figures 8.1 and 8.2 show location defined as Tukey’s five number summary andstability defined as the median difference respectively. Trend direction could be describedusing the simple algorithm described in Figure 8.3 • This algorithm is one example ofa simple trend detection algorithm that will work on small sample sizes [219]. Similaralgorithms for stability need to be developed and tested.1’ is the maximum allowable retracement and is usually set at 0.66 (see Figure 8.3)Chapter 8. Operation Paradigm 267DurationIII—Window—--II‘IWI t IIsI Ij:II $ I14III I• I • • I III I Ici $• I I —-C ‘sC I II I II • -— •‘D ‘ • I $ gII I ssIs I! — I I II IC) I $ I I SI I •-iII II I II, p I IS IWI I p I pg I gIIii I I p II__I I p II I pIl p $II $WI ItWI Box-PlotTimeWindow Series Replaced With Tukey’s Five Number SummaryFigure 8.1: Derived Time Series: LevelChapter 8. Operation Paradigm 268DurationIII’‘—Window--— II I’I III I II II • I Ia4. I II I III I IIII III ‘I IiiS I I $ I 5$ 1$I III IJhlI II • I 5I gI I IC I I II II I SI I IItI I I S II •i-I’IIII •I II III WIII i Ii-i II II I gTimeWindow Series Replaced With Absolute Median DeviationFigure 8.2: Derived Time Series: StabilityChapter 8. Operation Paradigm 269CONSTANT: P = 0.66iF Xt_2 > X_THENIF x_ > XTHENDirection is DownELSE IF x_ <Xtx—z2_1—z_—xt_IF zX > PTHENDirection is NoneELSEDirection is DownENDIFELSEDirection is DownENDIFELSE IF Vt2 <rt—1THENIF X_ < XjTHENDirection is UpELSE IF xt_a > Xg— xt—.xt_1—THENDirection is NoneELSEDirection is UpENDIFELSEDirection is UpEND IFELSETHENIF Xt_1 < XjTHENDirection is UpELSE IF X_ > 3JTHENDirection is DownELSEDirection is NoneENDIFENDIFFigure 8.3: Simple Direction AlgorithmChapter 8. Operation Paradigm 270Table 8.1: Forms Of Change_Form ExplanationLevel The weekly median effluent COD increasedWarn/Alarm The daily effluent COD is above the permit value.Limits Today’s effluent COD is the highest window value.Frequency The diurnal flow cycle has developed a new peak.8.2 What Is Change?A change event occurs when the value, quality or preference between windows changes.Table 8.1 lists the different forms a change may take.8.2.1 LevelFigure 8.4 shows a square wave that steps up at T = 10 and down at T = 20. Thesolid line would represent a change in a manipulated variable because a manipulatedvariable is set, not measured. The other points represent changes in non-manipulatedvariables. If the measurement noise is comparable to the step size, the step is undetectable(STD = 0.40). Warn/Alarm and Limits changes are special cases of a level change.8.2.2 Warn-AlarmAn operator can associate warning and alarm levels with a parameter. A change may hedefined as moving from one region into another (change in level). This type of change isuseful for the case when the operator collects very little data on a parameter that shouldnot vary outside a. predefined region. For example, the operator may assign the effluentChapter 8. Operation Paradigm 271Change In LevelSquare Wave (h=O.50)3.50Square Wave3.00 ±STD = 0.0100*2.50 t÷++±±-I- STD — 0.10C 0 00D JK** 0* STD = 0.402.001.50— 01.o0 I I I I I0 5 10 15 20 25 30 35TimeFignre 8.4: Change In LevelChapter 8. Operation Paradigm 272COD to a member of the following set (assuming the discharge permit is 50 mg/i):• Okay: { COD <40 mg/i }• Warning : { 40 COD 45 mg/i }• Alarm : { COD > 45 mg/i }Alarms may be used for nominal parameters as well. The operator assigus eachcategory to a region. For example. the operator may assign the secondary clarifier “appearance” measures to the following sets (see Appendix D):• Okay : {Normal }• Warning : { Ashing, Straggler-Floc, Pin-Floc }• Alarm : { Bulking, Wash-Out, Clumping }8.2.3 LimitsA change could be defined as the establishment of a new limit, e.g. 30 day maximum orminimum. A window’s maximum and minimum defines the limits for a non-categoricalmeasure while the mode and the set of unreferenced categories defines limits for categorical measures. A change would be defined as either the establishment of a new mode orthe use of a previously unused category (e.g. bulkiug).8.2.4 TrendA trend is a tendency for an observation to increase (or decrease) over time, over andabove what can be attributed to local trends or random variation [219]. Figure 8.5 showsboth a positive and negative trend.Chapter 8. Operation Paradigm 273A trend possesses four physical characteristics [74] [75]:1. Existence: The trend is cansed. The canse may be a disturbance parameter, amanipulated parameter or an undetected causal parameter.2. Stable: The trend is different from random variability and resolvable above thenoise. A caused trend may be unstable if the measurement noise drowns out itscontribution to the time series.3. Uniqueness In Scaling: A scale and window define a trend. For example, assumean operator detects an upward trend in the average daily effluent solids concentration over the past month. The scale is daily, i.e. an individual datum represents a24 hour interval and the window is the month.4. Well-Behaved: A well-behaved trend is smooth and continuous. For example. abuild-up of filamentous bacteria in the system would cause a trend upward in theeffluent solids. Discontinuities are usually due to equipment failures, contaminatedsamples or analytical errors.The state of a trend at time t is defined by the triplet {xQt),, 9}. A trend event isthe ordered set of contiguous trends. The more recent a trend event, the more importantit is.A simple way to detect trend is to assign the “Up” 1, “Down” -1 and “None” 0 (seeFigure 8.3). Trend then could be defined as the sum of the directions divided by thenumber of items in (i.e. windows) in the series. The operator would pick a thresholdvalue for this which he could “tune” for each parameter. Alternatively, the operatorcould use trend lines [219]. A violation of a trend line indicates a change in trend.Chapter 8. Operation Paradigm 274Change In TrendTriangle (h=O.50)3.50Triangle3.00 +STD = 0.010 0*D2.50 STD=0.100**0)2.0000o STD = 0.40*0 0001.50 —______________________________1.000 5 10 15 20 25 30 35TimeFigure 8.5: Change In TrendChapter 8. Operation Paradigm 2758.2.5 FrequencyA change may be also due to a change in the autocorrelation or spectral density function(see Figure 8.6). In order to detect such a change, a large amount of data are required. Forexample, Watnabe et al [306] proposed a two filter scheme to detect parametric change.A time series model is determined on-line during normal operation. A Kalman filter isused to update the model until the innovations sequence becomes colored. An ExtendedKalman Filter is used to correct the model. A change in frequency may manifest itselfas a change in stability. A change in stability is defined as the change in spread betweenwindows. For example, a change in stability may be defined as a change in the windows’median deviation.8.3 Postdiction: Why Was A Manipulated Variable Changed?Table 8.2 summarizes the eight reasons why an operator may change a manipulatedvariable (change viewpoints). In the previous chapter, it was mentioned that qualityis meaningless when speaking of manipulated parameters. For this reason, the qualitymeasure is replaced with a code that indicates why the operator changed a manipulatedparameter. These change viewpoints form the basis of this code. Figure 8.7 shows theinterrelationships between these viewpoints.All viewpoints except Catastrophic Intervention require that the process be controllable. A process is uncontrollable when changes in a manipulated parameter cease toeffect a change in performance. For example, assume that a biological phosphorus removal plant stops removing phosphorus. The introduction of a readily degradable carbonsource, and changes in recycle or wastage cannot effect an immediate process recovery.The operator decides to polish the effluent by using alum until the process starts removing phosphorus again. The addition of alum is a temporary measure in that theChapter 8. Operation Paradigm 276Change In FrequencySine Wave (T=1O)3.50AdS,.w,T=3LI Triangle3.00 ——-——--—LI±STD = 0.012.50STD=0.102.00 oJ STD = 0.40T**0)1.50 LI1.00LI0.50 I0 5 10 15 20 25 30 35TimeFigure 8.6: Change In FrequencyChapter 8. Operation Paradigm 277Table 8.2: Why Change A Manipulated Variable?Why ExampleT Anticipate Disturbance The load on the plant will increase during tourist seasonso decrease wastage now to build up the solids needed toexpand the plants capacity.2 Anticipate Performance The nitrate levels in the effluent rise during the warmsummer months so change your process configuration toallow denitrification.3 Optimize Performance Reduce recycle rate with the hope settling will improve.4 Compensate Disturbance The organic load in the plant is trending upwards so decrease wasting to raise the MLSS concentration.5 Compensate Manipulated Decrease wasting to offset effect of increase in sludge processing streams returned to aeration basin.6 Counteract Performance Increase recycle rate to offset denitrification in secondaryclarifier.7 Non-Operational ChangeFailure The recycle line to the aeration basin was plugged somonitor the aeration basin’s recovery.Error The change in the recycle rate caused a deterioration inperformance not an improvement.Maintenance One clarifier was off-line for three days for maintenance.Disruption The power was out for three hours.8 Catastrophic Intervention Chlorinate secondary clarifier sludge to kill back some ofthe filamentous population.Chapter 8. Operation Paradigm 278plant is designed to remove phosphorus biologically only. Once the process recovers,the operator stops adding alum and monitors his performance controlling the processusing conventional means. The viewpoint changes from Catastrophic Intervention toCounteract Performance.The viewpoints, Anticipate Disturbance, Anticipate Performance and Optimize Performance, are preemptive control actions in that the cause of the change is in the future.The operator can sort out the effect of the change if the process is stable at the timethe change is made. Figure 8.8 shows the distribution of a performance parameter overa window. If recent intervals he within the optimize region, the operator can assume itis safe to “push” the performance to a new level.Figure 8.7 illustrates that the tendency is to move to a control viewpoint after apreemptive change is made to the process. The reason for this is that the operator muststabilize his process before he acts again.When the operator anticipates a disturbance or decides to optimize the plant’s performance, he is making a prediction. The program monitors the performance parametersaffected by the change in the manipulated parameter and watches to see if the operator’sprediction comes true.The Control viewpoints, Counteract Performance, Compensate Disturbance and Compensate Manipulated, are the most common operation modes. The operator respondsto his process in the hope of stabilizing the process performance. Figure 8.8 describesthis as operating on the least preferable tail of a performance distribution. In otherwords, if the operator cannot assume his plant is stable and if the recent intervals areless preferable than older intervals, then the control objective is to stop the deteriorationin performance. Once the deterioration is stopped, then the operator can analyze whathappened and improve the process performance. If the process is not stable but is moving in a preferable direction, the operator should leave his process alone until either theChapter 8. Operation Paradigm 279ControllableFigure 8.7: Why A Manipulated Parameter Is ChangedChapter 8. Operation Paradigm0)PreferenceFrequency Of OccurMagnitude DistributionPreference FunctionFigure 8.8: Stability And Control Actions280, Control.....::... Optimize6Monitor.......................Chapter 8. Operation Paradigm 281AnticipateI1’vianipulateOptimizeIManiPulatejFeedback ControlrmFeedforward ControlturbInteractionManipulateMonitorTimeCauseCauseCauseCauseCauseDisturbPerform1ManipulateIManipulateIManipulateCauseFigure 8.9: Causal and Noncausal ChangeChapter 8. Operation Paradigm 282performance “settles down” or the performance starts to deteriorate. One of the mostdifficult decisions an operator mnst make is deciding when to wait (not intervene). Therule of thumb is that if the change in performance is tolerable then it is better to waitthan to perturb the process further.The reason a change is made determines what the operator should monitor. Whenthe operator reacts to a change in performance, he forms a feedback loop, while whenhe reacts to a disturbance or compensates for another manipulated parameter, he formsa feedforward loop (see Figure 8.9). In both these situation, the operator’s goal is tostabilize or control the process. For this reason, the program monitors performanceparameters to ensure that they do not deteriorate. If the program detects that anothercause is affecting the performance parameters, the program will warn the operator andreset the viewpoint.If the operator makes an error or a piece of equipment fails, the operator’s main concern is with the process’s stability (assuming that if an error is detected, it is corrected).If the performance starts to deteriorate, the viewpoint will suggest he act. However, ifthe operator feels the effect is transient, he may decide to wait the deterioration out andnot intervene.The program assumes that a variable is monitored and that a change in the variableis detected. If this is not the case, the operator will have to assist the program:• Monitored but Not Detected: A change occurred that the program missed.The operator informs the program of its error and the program enters the change.If the data are noisy, the program’s change detection routines may not detect achange that is obvious to the operator. In other words, if the computer and theoperator disagree on whether a significant change took place, the operator shouldbe able to override the computer.Chapter 8. Operation Paradigm 283• Not Monitored but Detected: The program detects a change in a diagnosticvariable. The program obtains the necessary information on the variable from theoperator and includes it in the analysis until the viewpoint changes. This usuallyoccurs when the operator monitors a parameter for a couple months each year, e.g.effluent nitrate concentration or when the operator is attempting to determine if anormally unmonitored parameter is the cause of a process problem, e.g. anaerobicdigester supernatant COD.• Not Monitored and Not Detected: The operator detects a change in a variablethat the program knows nothing about. The operator informs the program andprovides it with a location in the process. The program has no data on the variableso it must rely on the operator to tell it when the variable’s state changes again.For example, the operator observes that his influent turned a milky color and thewet well smells of dry cleaner solvent. He may tell the program when this occurredbut not provide the program with any data.8.3.1 Reason #1: Anticipate DisturbanceFigure 8.10 outlines the case when the operator changes a manipulated parameter tocompensate for an anticipated change in a disturbance parameter 2 For example, theoperator decreases the wasting because he anticipates an increase in load due to thetourism season.If the anticipated disturbance occurs, the action is considered successful and the viewpoint ceases (Table 8.3). If a common performance parameter deteriorates, the actionis considered unsuccessful and the viewpoint switches to Error (Table 8.4). However,2The reason for the change is the operator’s prediction that a change will take place. The predictiontook place before the change, therefore, postdiction.Chapter 8. Operation Paradigm 284Not DeteriorateMust Change cDMust ImproveCause To EffectAnticipate ConfoundingDisturbance DisturbanceCommon:::::::Figure 8.10: Reason #1: Anticipate A Change In A Disturbance ParameterChapter 8. Operation Paradigm 285Not DeteriorateMust Change 0Must Improve QCause To EffectConfounding--- _[ DisturbanceAnticipatending ConfoundingManipulated ManipulatedPerformanceFigure 8.11: Reason #2: Anticipate A Change In A Performance Parameterthe operator may choose to ignore the program’s opinion because he suspects the disturbance is still imminent. The change viewpoint is reset if a confounding manipulated ordisturbance variable changes.8.3.2 Reason #2: Anticipate PerformanceFigure 8.11 illustrates the case when the operator changes a manipulated variable becausehe suspects that a performance variable is about to change.Chapter 8. Operation Paradigm 286The operator is successful if the change in the performance is never detected (Table 8.5). In other words, the absence of failure is success. If a confounding disturbanceor manipulated variable intervenes, the viewpoint is reset.8.3.3 Reason #3: Optimize PerformanceAn operator should not introduce a change in a manipulated variable with the view toimproving the process’s performance if the process is not stable (Figure 8.12). Whenthe change is made under stable conditions and the subsequent performance does notimprove, the action is considered unsuccessful (Table 8.6). The viewpoint is reset if aconfounding disturbance or manipulated variable acts on the performance variable beingoptimized.8.3.4 Reason #4: Compensate DisturbanceFigure 8.13 outlines the case when the operator changes a manipulated parameter to compensate for a detected change in a disturbance parameter. For example, the disturbancemay be an increase in the winery’s COD while the response may be reduced wasting.The operator is successful if the shared performance variables between the disturbanceand the manipulated variable do not deteriorate (Table 8.8). However, if a confoundingdisturbance or manipulated variable change, the viewpoint must be reset. The viewpointis cleared if the initiating change is no longer important or the initiating disturbancechanges (Table 8.7).8.3.5 Reason #5: Compensate ManipulatedA change in a manipulated variable may cause some performance variables to improveand others to deteriorate. To offset this, a second performance variable may be changedChapter 8. Operation Paradigm 287Table 8.3: Rule For Reason #1: State Of Anticipated DisturbanceIF initiating disturbance manifests itselfChange is successfulWHY? Anticipated disturbance detectedChange viewpoint to Compensate DisturbanceENDIFTable 8.4: Rule For Reason #1: State Of Common PerformanceIF at least one of the performance parameters deterioratesChange viewpoint to Compensate DisturbanceChange is unsuccessfulWHY? Manipulated parameter caused process to deteriorateENDIFTable 8.5: Rule For Reason #2: State Of Anticipated PerformanceIF performance variable does not deteriorateIF change in manipulated variable insignificantChange is successfulWHY? Performance did not deteriorateENDIFELSEChange viewpoint to Error or Compensate ManipztlatedChange is unsuccessfulWHY? Performance deterioratedENDIFChapter 8. Operation Paradigm 288Not Deteriorate IMust Change QMust Improve [Cause To EffectDisturbance TargetPerformance fl PerformanceCommonDisturbance IShared ImtiatmgPerformance ManipulatedCommon I IManipulatedManipulatedPerformance jFigure 8.12: Reason #3: Optimize A Performance ParameterChapter 8. Operation Paradigm 289Table 8.6: Rule For Reason #3: State Of PerformanceIF performance deterioratesChange is unsuccessfulWHY? Performance deterioratedChange viewpoint to Error or Compensate ManipulatedELSE IF change in manipulated variable still significantIF performance improvedChange is successfulWHY? Performance improvedELSEChange is unsuccessfulWHY? Performance did not improve.ENDIFENDIFTable 8.7: Rule For Reason #4: State Of Initiating DisturbanceIF No change since initiating change in initiating disturbanceIF Initiating change insignificantViewpoint ceasesWHY? The initiating disturbance insignificantENDIFELSEViewpoint ceasesWHY? Initiating disturbance changed significantlyENDIFChapter 8. Operation Paradigm 290Not DeteriorateMust Change QMust ImproveCause To EffectWarn Only PerformanceInitiating ConfoundmgDisturbance DisturbanceShared___________Performance___________Responding L ConfoundingManipulated ManipulatedManipulatedWarn OnlyPerformanceFigure 8.13: Reason #4: Respond To A Change In A Disturbance ParameterChapter 8. Operation Paradigm 291Not DeteriorateMust Change 0Must Improve lUCause To EffectWarn Only PerformanceSharedResponding L_ ConfoundingManipulatedIManipulatedWarn Only PerformanceFigure 8.14: Reason #5: Compensate For A Change In A Manipulated Parameter(Figure 8.14). The operator is successful if common performance variables do not deteriorate.8.3.6 Reason #6: Counteract PerformanceFigure 8.15 outlines the case when the operator decides to respond to a change in aperformance parameter. For example, the operator increases the wastage rate becausepin-fioc is observed in the clarifier supernatant. If the effect improves or remains theChapter 8. Operation Paradigm 292Not DeteriorateMust ChangeMust Improve QCause To Effectrn OnlyCommonis ur anceIrfifiatingnding___________Performance. ] ManipulatedCommonManipulatedrn OnlyFigure 8.15: Reason #6: Compensate For A Change In A Performance Parametersame, then the operator is successful.If the performance deteriorates (Table 8.9), the control action is unsuccessful. In thiscase, the operator may decide to either compensate for the change in performance or thechange in the manipulated variable.Chapter 8. Operation Paradigm 293Table 8.8: Rules For Reason #4 : State Of Common Performance VariablesIF Performance variables did not deteriorateChange is successfulWHY? Performance did not deteriorateELSEChange is unsuccessfulChange viewpoint to Error or Compensate ManipulatedWHY? Performance variable deterioratedENDIFTaEle 8.9: Rule For Reason #6: State Of Common PerformanceIF at least one of the performance parameters deterioratesChange unsuccessful.Change viewpoint to Error or Compensate ManipulatedWHY? A target performance variable deteriorated.ELSEControl action successful.ENDIFChapter 8. Operation Paradigm 294Not DeteriorateMust Change 0Must ImproveCause To EffectChanged Back To Correct ErrorConfoundingDisturbanceCommonIn ConfoundingManipulated ManipulatedPerformanceFigllre 8.16: Reason #7: Correct An Operational Error8.3.7 Reason #7: Non-Operational ChangeIf a manipulated variable is changed for a non-operational reason (i.e. error or equipmentfailure), the operator has the choice to compensate for the change or to correct theoffending variable (Figure 8.16). If a performance variable deteriorates, the viewpointchanges to Counteract Performance.Chapter 8. Operation Paradigm 2958.3.8 Reason #8: Catastrophic InterventionA catastrophic intervention differs from all the previous interventions in that it onlyoccurs when the operator has lost control of his process. When this occurs, the operatorintroduces a new cause that “jolts” the process back into a state where the operator canmaintain control using conventional control variables. An operator is successful when theintervention is no longer required (i.e. removed).Chapter 8. Operation Paradigm 2968.4 Prediction: How To Change A Manipulated VariableAlthough the focus of this research is postdiction, prediction is required if the operatorrequires assistance in deciding if, when and how to change a manipulated parameter. Inthis case, the program could be coupled to a model. The choice of model is important.An operator may decide that a model is not worth bothering with if the model requiresmaintenance or extensive computing resources. The best approach is to use a simplemechanistic model whose parameters can be recursively identified from the monitoringdata. In this case, the program, not the operator, maintains the model. Simplicityrequires compromise- a simple model cannot estimate the transient and steady stateresponse to the same degree of precision as a complex model. However, when making anoperating decision in a wastewater treatment plant, the transient response is of only passing interest. Therefore, the simple model should concentrate on estimating the correctsteady state response. The program can introduce an additional degree of complexityby using a set of simple models, each of which is valid over a small range of operatingconditions.8.5 ConclusionsThe goal of the Operation Paradigm is to link a set of data in the past to a change now,and change now, to a set of data in the future. These linkages enable the operator tolearn from his actions.Chapter 9SynthesisA reasonable scientific goal is to develop a theory of representation and reasoning that explains how information can be structtired in such a way as tobe efficiently interpretable by machine, yet understandable to humans [124].The objective of this chapter is to demonstrate how an operator can use the paradigms,described in the previous three chapters, to improve his ability run his plant. An operatorcan improve his plant’s performance by gaining control of his information gatheringprocesses and following the effect of his control decisions. The simple example providedin this chapter focuses on the latter case. The structure of the example discussed in thischapter is presented in Chapter 6.9.1 The Relationship Between The Program and The OperatorThe operator uses the analyst module of the program to work through each controlrecursion (Figure 9.1). Because the Treatment Process and the Information GatheringProcess cannot be uncoupled, control must be maintained over both. For this reason, thesystem can be broken into two components: the Information Generating System and theInformation Interpretation System. The operator and the computer program (the analyst)form the latter while the treatment process and the data collection programs make upthe former.The most common approach to treatment plant control is to assume each set-pointis independent. For example, the wastage rate is set to maintain a desired SRI and therecycle rate is set to provide a desired sludge blanket depth in the secondary clarifier. In297Chapter 9. Synthesis 298this case, the operator passively monitors the system’s performance nntil a change occursin either the plant’s performance or the plant’s inputs. Although such an approach iseasily automated, the approach often leads to suhoptimal performance because these twomanipulated parameters are in fact linked.The alternative approach is to recognize that for a given set of input and state conditions, there is at least one optimal set of set-points. The reason for this is that set-pointsin a treatment plant are not independent. The operator must actively monitor the effectof the set-points on the system in the hope that he can determine their optimum values,i.e. delineate a response surface. This approach is not easily automated. The programoutlined in this thesis assists the operator in recognizing the pattern of set-points thatprovides the best over-all plant performance. If the program is used in this manner itwill lead to both an improvement in the plant’s performance and the operator’s processknowledge.Chapter 9. SynthesisTreatment Process Inform att on Gathering ProcessObservableL. CollectionPrograms ParametersInformation Generating System299ViewpointOperatorInformation Interpretation SystemFigure 9.1: Synthesis: Information Generation and InterpretationChapter 9. Synthesis 3009.2 Construction Of A Simple ExampleThe purpose of the example described in this section and analyzed in the following sectionis to demonstrate how the three paradigms defined in the previous three chapters assistthe operator to run his treatment plant. The purpose of the example is not to prove theefficacy of the paradigms over conventional operational strategies nor is it to simulate indetail the interaction between an operator and his treatment plant.The structure of the example parallels Figure 9.1 and is shown in Figure 9.2. Thefirst difference between the two figures is that the treatment plant process and the datacollection programs are replaced with models. The purpose of these models is to simulatethe functions of these two elements of the real system at a very primitive level.The second difference between the figures is that Figure 9.2 contains a model thatdrives the simulation, i.e. Simulation Scenario Function. In the real world, the plantresponds to events in its system that are caused by outside disturbances (i.e. changes inthe weather) and internal state changes (i.e. shifts in the biological population). In theexample, the system responds to changes in model coefficients and inputs made by theSimulation Scenario Function.easuremen ObservedModel ParametersInformation Generating SystemWhat4Changed?JrHow Did ItChange?zzzz:tzzzz’HowToRespond?V%ewpointOperatorInformation Interpretation SystemFigure 9.2: Simple Example : Program LayoutChapter 9. Synthesis 301Treatment Process Information Gathering ProcessSimulationScenarioChapter 9. Synthesis 3029.3 Information Generating SystemThe Information Generating System consists of the Treatment Process and InformationGathering Modules.9.3.1 Treatment Process ModuleThe Treatment Plant module models the layout and the function of a treatment plantindependent of each other. This module constructs a hierarchical network model of thetreatment plant in the computer’s memory that enables it to trace the incoming sewagethrough the plant to the receiving water. The objective of this module is to generatedata that describes the state of the plant’s two streams as they pass through the system.The first stream, the liquid stream, consists of one currency parameter, Liquid Flow,and five coordinate parameters: { Solids, Substrate, NH3 — N Aro3 — N, ConservativeToxic Compound }. The second stream, the air stream, consists of a currency parameteronly: Air Flow. The design of this module is patterned after the Structural Paradigmdiscussed in Chapter 6.•The simulated treatment plant consists of a primary clarifier, an aeration basin anda secondary clarifier. The plant is identical to the one described in Quasim’s book onwastewater treatment plant design [249](Figure 9.3).Influent CharacteristicsThe influent characteristics were modelled by passing yearly average values for each of theliquid stream’s components through a seasonal, weekly and daily hydrograph [209] [283].Chapter 9. Synthesis 303Adjustable ParameterJoin NodeSplit NodeInfluent EffluentCompost00Figure 9.3: Simulated Plant’s Flow SheetChapter 9. Synthesis 304Primary ClarifierThe primary clarifier model consists of a function that estimates the removal of settleablesolids and COD given the clarifier’s hydraulic loading rate [225].Aeration BasinThe aeration basin is modelled using Lessard’s Activated Sludge Model [194]. Lessard’smodel is a subset of the JAWPRC model [142].Secondary ClarifierA secondary clarifier performs two functions: clarification and thickening. Clarificationis modelled using a simple linear function that adjusts the percent of solids spilling overthe weir using the clarifier loading rate and the SVI of the sludge [32]. The modelassumes that the clarifier is a perfect thickener. i.e. the underfiow solids concentration iscalculated using a simple mass balance.Sludge Processing WatersThe example uses Arun’s regression model that estimates the return flow rate, solidsconcentration and COD concentration from the sludge treatment processes using theplant’s influent characteristics [15].9.3.2 Information Gathering Process ModuleThe Information Gathering Module uses the stream parameters estimated by TreatmentProcess Module to generate the data the operator would obtain through his monitoringprogram. The Information Gathering Module accomplishes this using two functions:Sampling and Measurement Error functions. The role of these two functions is bestChapter 9. Synthesis 305described by a simple example. Assume the operator collects a 24 hour composite effiueutsample and determines its COD concentration. The Sampling Function would constructthe 24 hour average to which the Measurement Error Function would add a random error.The preference and quality functions calculate a number between 0 and 1 that represent the COD value’s desirability or quality. The purpose of the preference function is tomake a measure more sensitive to changes around a critical value, i.e. permit limit. Thepurpose of the quality function is to make a measure less sensitive, particularly whenthe measure is outside the test’s detection range. In some cases, the quality functionrounds off the observed value or replaces the observed value with a more realistic number. The Preference and Quality functions used in this example were constructed usinginformation from Standard Methods [176]. Table 9.1 lists the measurements used in thisexample.9.3.3 Adjustable ParametersThe example assumes the operator is able to adjust the recycle flow rate and the wastagerate. The operator decides to fix the recycle rate at 60% of the influent flow rate and tofix the wastage rate to maintain a 15 day MCRT.9.3.4 Simulation ScenarioThe simulation scenario function changes model coefficients to simulate changes in thetreatment plant. Figure 9.4 shows how the true SVI is changed to simulate the occurrenceof a bulking sludge in the treatment plant.9.3.5 Detection Of ChangeThe simulation uses three measures to detect change in a data series:Chapter 9. Synthesis 306Table 9.1: Observed MeasurementsControl Context Measure Name Sample Type Sampling FrequencyDisturbance QI Influent Flow Rate Daily Average DailyCI lnf. COD Composite DailySI Inf. Solids Composite DailyNI Inf. Ammonia Composite MWFAdjustable WASQ Wastage RateRASQ Recycle RatePerformance CE Elf. COD Composite DailyPref. and Qual.SE Eff. Solids Composite DailySEP PreferenceNE Elf. Anunonia Composite DailyStatus PF Sludge Proc. Flow Rate Daily Average DailyPC Sludge Proc. COD Grab MWFPS Sludge Proc. Solids Grab MWFPN Sludge Proc. Ammonia Grab MWFMVS MLVSS Grab DailyUAS Underfiow Solids (RAS) Grab DailyOUR Oxygen Uptake Rate Grab DailySVI Sludge Volume Index (SVI) Grab DailySET Settleability Grab DailyFIN Filament Number Grab DailyChapter 9. Synthesis 307Simulation ScenarioSecondary Clarifier Model Parameter110100 StartCL290 /Bulking Event80 StopCL222-Jan 03-Mar 12-Apr 22-May 01-Jul 10-Aug11-Feb 23-Mar 02-May 11-Jun 21-JulFigure 9.4: Simulation Scenario: True SVI ValueChapter 9. Synthesis 308• Daily trends are detected by a simple direction measure described in Figure 8.3.• Weekly Trends are detected by examining the position of the current week’s values relative to the last four weeks. Five number summaries are used for thesecomparisons. Five number summaries were discussed in Chapter 2.• Changes in stability are detected by examining changes in the Absolute MedianDeviation over the last four weeks. This measure was discussed in Chapter Detection and Response To ChangeThe detection and response to a change in a data set may be broken into two steps.The first step is to rule out the possibility that the change in the data set is due to aproblem in the measurement process. Once this is ruled out, the second step is to tryto reconstruct a reason for the change in the process and to plan out a response if theperformance of the process is deteriorating. Figure 9.5 contains an outline of this process.9.4 Simulation: Coping With A Bulking SludgeThe simulation starts on November 1, 1990 and ends on August 29, 1991. The Measurement Model starts January 1,1991 and the hulking event starts on March 15, 1991.9.4.1 March 17, 1991The program begins its analysis by examining the daily and weekly behavior of the dataseries. The results of these analyses are shown in Figures 9.6, 9.7, 9.8 and 9.9.In this case, the program detects a decrease in the weekly variability of the MLVSS andunderfiow solids concentration. This decrease is associated with a decrease in the averageMLVSS concentration. The operator supplies the program with a qualitative measure ofChapter 9. Synthesis 3094 YesIntervene9CatastrophicicherventionControllableInterveneFigure 9.5: Control Cycle: Wastewater Treatment PlantChapter 9. Synthesis 310Average Direction Of Time SeriesFebruary 18/91 to March 17/91RASQ_____WASQPCPSPFUAS0) OURci FIN_o viMVS I INESEP__ ____ _ ___________SECEP ÷1:IjpwardTrendCE-1: Downward TrendO:NoTrendNI ICI-0.25 -0.15 -0.05 0.05 0.15 0.25-0.20 -0.10 0.00 0.10 0.20Average Direction Over 28 DaysFigure 9.6: Daily TrendChapter 9. Synthesis 311C0>CCt3a)a)D0(I)-QG)>a)AbsoluteFebruaryMedian Deviation18/91 to March 17/91LI I IIMar11 to Mar17Mar 4 to Mar 10Feb 25 to Mar3Feb18 to Feb2416.00-Compare relative size of bars to top bar.14.00- Top bar represents current week and is—normalized to one.12.00-10.00-8.00-6.00- —I ujj i0 CI’ bSI NI CEP SEPUA.. PMVS SET OUR PF PC PASOFigure 9.7: Stability-SeeTable9.1Ct) ccCc)CDCD Cl) Cl)CDCD—f!.sCD‘r+CD—z CCD3C)00CD -‘ C,)CDC 3 3CD °CD—.ci, CD CD ci,11UUCDCCDCD-‘-‘I D CQ CDI D CD CDChapter 9. Synthesis 313Relative Median Deviation(Mar 11-17) compared to (Feb 1 8-Mar 17)ii_PC MinimumPS______1111111125% (Lower Hinge)___I_UAS [datesaweekIYenJ MedianOUR huuuii 75% (Upper Hinge)FIN_____ ____MaximumsvI -uvs________-060 -020 020 0.60-040 0.00 0,40 0.80Re’ative Five Number SummaryFigure 9.9: Weekly Trend: Status ParametersChapter 9. Synthesis 314the sludge’s settleability that is derived from his observations in the laboratory and inthe plant. The measurement paradigm enables the program to integrate this parameterinto the analysis. In this case, the settleability index increased suggesting that the sludgeis beginning to bulk.The program uses the structural paradigm to sort the data sets into four group: disturbance (cause), manipulated (cause), status (cause or effect) and performance (effect).If the operator decides to look at a parameter, the program constructs a list of variables that either affect or are affected by the parameter. This enables the program totravel upstream of the aeration basin to determine what might have caused the changein settleability.The changes the program detects upstream are the cyclic variations in the influent.The program did not detect a change in any of the other disturbance parameters thatcould account the change in the sludge.The operator responds to this warning by changing the way he calculates the wastagerate. Up to now, he calculated his wastage using weekly averages for the solids concentrations. Now, he will use the daily values so that he can respond quickly to changesin the effluent solids. The program notes that the conditions in the plant caused theoperator to change the way he determines what to waste. The program will also monitorthe effluent solids and the sludge’s characteristics to determine if the operator’s decisionsolves the bulking problem.The operator asks the program to monitor the relationship between intentional andunintentional wastage. This situation corresponds to Reason 2: Anticipate A ChangeIn Performance discussed in the previous chapter. Figure 9.10 is derived from Figure 8.11.Chapter 9. Synthesis 315Not Deteriorate-Must Change 0Must Improve jjCause To EffectMonitor InfluentCharacteristicsAnticipate ChangeIn EffluentChange Wastage Do Not Change TheCalculation Method Recycle Rate SetpointWatch For ChangeIn SettleabilityFigure 9.10: Reason #2: Anticipate A Change In A Performance ParameterChapter 9. Synthesis 3169.4.2 March 27, 1991The program detects a deterioration in the sludge and effluent quality. This deteriorationis accompanied by an increase in both filament connt and SVI. The MLVSS concentrationand the wastage rate show a trend downward as well as an increase in stability. Thisindicates an exponential type decay in both quantities. The program warns the operatorthat 94% of the wasting is unintentional (Figure 9.11). Consequently, the MCRT hasdropped from the set point value of 15 days to 10 days because of the uncontrollable lossof solids in the effluent in the last week.If the sludge ceases to settle, then the wastage rate and recycle rate will have littleeffect on the process. To avoid this, the operator decides to use Cl2 to shift the balancebetween fioc forming and filamentous microorganisms. The program refers to this as acatastrophic intervention as it is not part of the plant’s regular control strategy. Theprogram will discourage any other operator induced changes to the parts of the plantaffected by this decision until the intervention ends.The program will consider the operator successful if he manages to regain controlover his settling and stops using chlorine. In Chapter 8, this is referred to as Reason8: Catastrophic Intervention. The importance of checking that a control action is stillvalid given the current circumstances is discussed in Chapter April 22, 1991The stability of the SVI measure has increased while that of the underfiow has decreased.This indicates that the clarifier is regaining some ability to thicken the sludge. This isconfirmed by a decrease in the sludge settleability index. However, the wastage rate isstill a difficult parameter to use due the need to purposively waste very little solids dueto the significant fluctuations in the effluent solids concentration (Figure 9.12).Chapter 9. Synthesis 317Uncontrollable SystemMarch 27, 19911.00:- 7000-oFraction Of Solids Wasted By The Operato/ 60000.80.1(I)fl7fl-:2 \ 50000.600.50- ,..—\ 4000g::::0.1 0 FrauorrOrSoIrds l.os 1a The izmueni0.00 I I I I 100026-Feb 08-Mar 18-Mar 28-Mar03-Mar 13-Mar 23-Mar- Unintentional Loss Intentional LossFigure 9.11: Uncontrollable SystemChapter 9. Synthesis 318The program does not detect changes in the effluent COD because the COD valuesare outside the COD test’s range. The quality function replaces the COD values between25 and 50 mg/L with 38 mg/L and those less than 25 mg/L with 12 mg/L. The reasonfor this is to prevent the change detection routines from mistaking measurement noisefor process variation. Figure 9.13 illustrates this relationship.Given the improvement, the operator decides to stop using chlorine and see if thesystem continues to stabilize. Because there has been no change in any of the disturbanceparameters, the operator reasons that the effect of the removal of the chlorine will notbe confounded with a change elsewhere in the process.9.4.4 May 20, 1991The intervention appears to have worked because the settleability and the effluent qualityboth improved. As well, the proportion of unintentional wastage decreased and theMLVSS is beginning to increase. The operator will now wait until either a new changeoccurs or the program suggests the system be optimized.9.4.5 PostscriptBecause MCRT is a derived parameter, it is very difficult to control its value using thewastage rate if (a) the system is changing or (b) the error in the solids tests is significant(Figure 9.14). In times like these, the program provides the operator with a tool tomonitor the controllability of the process and helps him integrate information from bothquantitative and qualitative measures throughout the process.Chapter 9. Synthesis 319Unintentional Loss Intentional LossSystem RecoveryApril 22, 19911.O00 0.900.800.70CI)p 0.600ci 0.500.40• 0.30°02(tL 0.100.0023-MarFraction 01 Solids loss Via The Efiluent / -Fraction Of Solids Wasted By The Operatot I I I2O.0018.0016.0014.0012.0010.00aoo1’>-DI-a:002-Apr 1 2-Apr 22-Apr28-Mar 07-Apr 17-Apr 27-Apr-IFigure 9.12: System RecoveryChapter 9. Synthesis 320Effluent QuaUtyApril 22, 199170 1.2060 ---------—---•——--------- 1.0050 1 0.80E. LÀo 40____0 6030 0.40C -: j0:3Vw10 0.000 -0.2023-Mar 02-Apr 12-Apr 22-Apr28-Mar 07-Apr 17-Apr 27-AprLi” Effluent COD COD PreferenceFigure 9.13: Effluent CODChapter 9. Synthesis 321WastageMay 22, 1991______1.5020•_ _ _ _ ___18 Iii L kh i.oo, ivL 0.50H 1 2 I ‘Il I “I\’ .. . I—c.€ - -. V86 ncoroIIable II -0.50LI-4 MCRT2 I23-Nov 12-Jan 03-Mar 22-Apr 11-Jun 31-Jul 19-SepIntentional Loss Unintentional LossFigure 9.14: Wastage Rate ControlChapter 10Conclusion: From Box’s EVOP to Evolutionary OperationOne stumbles through unknown regions, is led astray by analogies, is overwhelmed by new possibilities, and knows afterward what one should haveknown before.1EVOP, as first proposed by Box and Draper, was a plant optimization method thatapplied the statistical design of experiments to organize and interpret the effect of smallprocess changes. Box and Draper went beyond most of their colleagues at the time bysuggesting that the results of these experiments should form the basis of discussion amongthe employees who operate the process. These employees, not management, would decidewhat the next change would be.The improvement that this methodology would cause in the performance of the process, was not due solely to the use of statistics and experimental design. Rather, bycasting the operation into a cause and effect framework, the employees became aware ofparameters that were affecting their process; parameters that they had never thought ofbefore.This operation strategy led not only to an evolution in the process’s performance,hut also to changes in what was monitored, how it was monitored, who monitored itand how the data that resulted from this monitoring was interpreted and used. Even incases where experimental design could not be used, this way of viewing the process ledto improvements.1Coristantin Virgil Negoita322Chapter 10. Conclusion: From Box’s EVOP to Evolutionary Operation 323The reason for this, is that this operations strategy concentrates its energy in organizing process information into a form that is readily assimilated by the operator.This thesis has modified and expanded the work of Box and Draper [58] by applyingthe EVOP principle to wastewater treatment processes. The application, as defined inthis thesis, enables an operator to control both his information gathering and treatmentprocesses and to improve his control decisions.Specifically, the objectives listed in Section 1.1 have been achieved as follows:1. A framework has been developed that enables the operator to express the causeand effect relationships in both his information gathering and treatment processesin a form that enables him to improve the performance of both of these processes.The framework, developed from the theory discussed in Chapters 2,3,4 and 5, isdefined by three paradigms, described in Chapters 6,7 and 8 respectively.2. The thesis developed three paradigms that guide the operator and the computerin organizing process information to best elucidate causal patterns. The StructureParadigm, described in Chapter 6, enables the computer to create an image ofthe process so that the computer can define monitoring data, derived variablesand process rules in terms of where in the process they occur. The MeasurementParadigm, described in Chapter 7, maps a datum into a common space providingthe datum with meaning. This enables the computer to analyze the data set as asingle unit. The Operations Paradigm, described in Chapter 8, uses the previoustwo paradigms to enable the operator to determine the effect of his control decisions- leading to evolutionary operation.3. A simple example is used in Chapter 9 to demonstrate how the operator uses theseparadigms to decide how and when to respond to a process change and how tolearn from his past control decisions.Appendix AAbbreviations And CopyrightsBMDP BMDP Statistical Software IncorporateddBASE IV Ashton-TatedbFILE Raima CorporationLotus 123 Lotus CorporationMS-DOS Microsoft CorporationParadox BorlandUNIX AT&TTable A.1: Copyrights324Appendix A. Abbreviations And Copyrights 325ANOVA Analysis Of VarianceAT Artificial IntelligenceANSI American National Standards InstituteASCII American Standard Code For Information InterchangeASTM American Society for the Testing of MaterialsBOD5 or BOD Biochemical Oxygen DemandCCP Comprehensive Correction ProgramCEI Compliance Evaluation InspectionCOD Chemical Oxygen DemandCMAS Completely Mixed Activated SludgeCSI Compliance Sampling InspectionCSTR Completely Stirred ReactorC P E Comprehensive Performance EvaluationDBMS Database Management Systemdf Degrees Of FreedomDMR-QA Discharge Monitoring Report- Quality AssuranceDO Dissolved Oxygene.g. For ExampleEVOP Evolutionary OperationF/M Food To Microorganism RatioHRT Hydraulic Retention TimeIAWPRC International Association on Water Pollution Research and Controli.e. That IsMCRT Mean Cell Residence TimeMLSS Mixed Liquor Suspended SolidsMLVSS Mixed Liquor Volatile Suspended SolidsNPDES National Pollution Discharge Elimination SystemPAl Performance Audit InspectionPESP Performance Evaluation Sampling ProgramPID Proportional-Integral-DerivativePLEX Plant ExperimentationP LF Performance Limiting FactorP MS Parameter-Measurement-SampleQA/QC Quality Assurance/Quality ControlSVI Sludge Volume IndexUBC University Of British ColumbiaWRc Water Research Center (England)Table A.2: List Of AbbreviationsAppendix BGlossaryIf you want to prevent war, define the meaning of your words before speaking.’B.1 IntroductionA glossary is necessary becanse a number of disciplines either use similar terms withdifferent meanings or use terms with unfamiliar definitions. The area from which thedefinition originates is enclosed in square brackets following the term. If a term is notfound in the glossary, it is discussed at length in the main body of the thesis.B.2 Items301(h) Monitoring Program [Wastewater Treatment]: Under section 301(h) of theClean Water Act, municipalities are required to conduct monitoring programs to determine the impact of their discharge on marine biota, to demonstrate compliance withapplicable water quality standards and to measure toxic substances in their discharge [22].Accuracy [Statistics]: Closeness of a measured value to the true or reference value.Accuracy is a measure of correctness. A measure is accurate only if it is both unbiasedand precise. For this reason, ASTM [1] recommends that researchers include a statementabout bias and precision, not accuracy.Adaptive Control [Control Theory]: Control in which automatic means are used tochange the type or influence of control parameters in such a way as to improve theperformance of the control system [220].1326Appendix B. Glossary 327Adaptive M-Estimator (ADA) [Statistics]: A robust location estimator:Solve for Tqs (sd)= 0Where0<x{<aa a<lxI<b= sgn(xj) (xHxiI) a b x <c0 xc= median of the absolute deviations from the medianI 1.0 L<0.44a = O.75L—25 0.44 < L 0.62.5 0.6<LL = average (IxiMI) where — XMb = 4.5c = 8.0Alias [Control Theory]: A high frequency component that appears in the sample (reconstructed signal) as a low frequency component.Allocation, Parameter [Measurement Paradigm]: An allocation parameter describeshow a unit process’s capacity is allocated. By definition, an allocation measure is anumber between 0 and 1 where 1 is 100% of capacity. For this reason, an allocationmeasure is associated with a capacity measure (Figure 6.13).Analysis Of Variance [Statistics]: The Analysis Of Variance (ANOVA) is a tool or amethod of presentation of the quantitative evaluation of the influence of the independentvariables (factors) on the dependent variable. In other words, ANOVA is a body ofstatistical theory, methods and practices in which variation in a set of data is partitionedAppendix B. Glossary 328into identifiable sonrces of variation [1]. The form of the analysis depends on the formof the assumed model. For example, equation B.16 is the model assumed in table 3.1.Xjj[L+7i+/3j+€ij (B.16)xj : Solids Destruction [kg/day]Average Solids Destruction [kg/day]Sludge Source Factor,3 Control/No Control Factore, Expenmental ErrorAndrew’s Curves [Statistics]: Each variable is assigned to parameter in an harmonicfunction. The frequency spectra of this function forms the Andrew’s curve [77].Anticipate [Operation Paradigm]: To act before a disturbance enters the plant or beforea change in performance occurs.Artificial Intelligence (Engineering Definition) [Artificial Intelligence]: The concepts, theory and practice of building intelligent machines.Association, Functional And Stochastic [Statistics]: Two attributes are associatedif there is a relation between them. If the relation is based on theoretical considerations,the association is functional. For example, the area of a circle is a function of its radius.If the relation is based on a joint probability distribution, the relation is stochastic. Astochastic relation is a weaker than a functional relation [255].Audit, Process [Wastewater Treatment]: An audit is an observational study of theoperation of a wastewater treatment plant. The goal of the study is to detect an effectand infer its cause(s).Automatic Control System [Control Theory]: A control system that operates withouthuman intervention [220].Appendix B, Glossary 329Beck’s Model Complexity Dilemma [Modelling]: Given two correctly identified models. the first developed a priori (i.e. from laboratory experiments and first principles) andthe second developed a posterior (i.e. from data collected on the system), then the firstmodel’s predictions may be accurate but imprecise while the second model’s may beprecise but inaccurate [35].Beck’s Prediction Error Dilemma [Modelling]: Given a data set with fewer degreesof freedom than the system, and two models, one with the same degrees of freedom asthe data set and one with the same degrees of freedom as the system, then if both modelsare identified on the data set and validated on a new data set with either more degreesof freedom or different degrees of freedom, then the first model will produce uncertain(and absurd results) while the second model will produce precise inaccurate results [35].Bias [Statistics]: Persistent or systematic error that remains constant over a series ofreplicated measurements.Binary Plot [Statistics]: A graphical method for identifying and displaying patternsin multivariate data sets. The construction requires the calculation of the median foreach variable followed by subtraction of the median from each datum. The residuals areassigned to a binary number. The sorting operation causes samples with the same binarynumbers to form clusters [208].Block [Experimental Design]: see Control [Experimental Design]Bloom’s Taxonomy [Psychology]: A hierarchical classification system for levels of questioning.BMDP [Statistics]: A set of over 43 computer programs designed to perform simple andcomplex statistical analyses on a data set [298]. BMDP (©BMDP Statistical Software,Inc.) runs on micro, mini and mainframe computers. Each program is referred to by anumber and a letter, i.e. 3V performs General Mixed IViodel Analysis of Variance.Box and Whisker Plot [Statistics]: A boxplot displays a rectangle oriented with theaxes of a coordinate system in which the vertical axis has the scale of the batch data.Its top and bottom are drawn at the upper and lower quartiles of the batch. The boxis cut by a horizontal line at the median. A simple Box and Whisker Plot is shown inAppendix B. Glossary 330Maximum75th PercentileInterquartileMedianRange25th PercentileMinimumFignre B.1: Simple Box and Whisker PlotFigure B.2.C Language [Computer Science]: The C language is a general-purpose programminglanguage known for its efficiency, economy and portability. The C programming languagewas developed at AT&T Bell Laboratories in 1972 by Dennis Ritchie. Its roots can befound in a language called BCPL (Basic Combined Programming Language) developedby Martin Richards of Cambridge, England. Scientists at Bell Laboratories developeda variant of BCPL called B for an early version of the UNIX operating system. DennisRitchie improved B by introducing data typing and called the new language C. For thisreason, a C Language Compiler forms part of the UNIX operating system. This versionof C is now referred to as K&R C. In 1989, ANSI developed a new standard for C incooperation with the software industry. This version of C is referred to as ANSI C.Capacity [Structure Paradigm]: The potential of a unit process, e.g. volume, surfacearea, maximum flow rate. An allocation measure is the fraction of the potential currentlybeing used. For example, a plant has 4 clariflers of which only 3 are on-line. The capacityis 4 while the allocation is 0.75 (3).Appendix B. Glossary 331Catastrophic Failure [Control Theory]: Sudden change in operating characteristicsresulting in complete loss of useful performance of the function or unit.Catastrophic Intervention [Operation Paradigm]: A catastrophic intervention is acontrol action taken by the operator when the operator is unable to control the systemusing a manipulated variable. A catastrophic intervention is a short term solution. Forexample, an operator may polish the effluent with alum until the process is able to removeall the phosphorus biologically.Categorical Data [Ivleasurernent Theory]: A nominal value, i.e. a class to which anobject belongs.Central Tendency, Indices Of [Statistics]: Statistics that describe the clustering ofdata e.g. mean, mode and median. By clustering tendency we mean the tendency ofsubgroup(s) of data to have the same value.see also Location StatisticChange [Operation Paradigm]: A change occurs when the state of a measure over atime window shifts. The operation paradigm defines five types of changes: (1) changein level, (2) change in trend, (3) change in warning or alarm status, (4) change in limitsand (5) change in frequency.Change Point Problem [Control Theory, Statistics]: Test for a change in model parameter occurring at an unknown time.Characteristic [Statistics]: A property of items in a sample or population which whenmeasured, counted or otherwise observed, helps to distinguish between items [6].Chebyshev’s Inequality [Statistics]: The probability that a random variable X fallswithin k standard deviations of the mean is at least (1 —— ku <X < ci + ka) 1 —Chemometrics [Analytical Chemistry, Statistics]: The application of statistical andAppendix B. Glossary 332mathematical methods to chemical data. The word chemometrics is the English translation of the Swedish word kemometri, a term first used by Svante Wold in 1972. Thedriving force behind Chemometrics is the increase in the amount of data available toChemists due to advances in automation, instrumentation and computers. Chemometrics, established in the early 1970’s by Brnce Kowalski, Luc Massart and Svante Wold,has more in common with Psychometrics than with Biometrics [118] [119].Chernoff Faces [Statistics]: A datum’s deviation from a statistic (e.g. mean, set-point)is assigned to a feature of a face. The further the datum is way from the desired value,the more distorted is the feature [77].Child [Structure Paradigm]: A child is a structural node that is owned by anotherstructural node, i.e. Parent.Chronological Ignorance [Computer Science]: Assume a set of premises are true untiltold otherwise by the user.Class [Structure Paradigm]: A class is a set of nodes and links that model the sametype of information, e.g. the structure class models the physical and causal layout of theplant.Coefficient Of Determination [Statistics]: The amount of variation that can be accounted for by the regression equation.Coefficient Of Determination Rule, Deming’s [Statistics]: The Coefficient of Determination gives no indication of whether the lack of perfect prediction is caused by aninadequate model or by purely experimental uncertainty [95].Compensate [Operation Paradigm]: To change a manipulated variable to counterbalance the effect of a change in an effect sharing manipulated variable, i.e. decrease thewasting to counterbalance the increase in the organic load due to increasing the wasteaccepted from the winery.Composite Correction Program (CCP) [Wastewater Treatment]: The CCP is asystematic approach to eliminating factors that limit performance in existing treatmentAppendix B. Glossary 333plants. CCP usually follows the snccessful completion of a CPE.Composite Frequency [Measnrement Paradigm]: The composite freqnency is the freqnency at which the composite sample is taken. For example, a sample is taken every 20minntes to form a composite sample. The sampler is emptied and its contents analyzedevery 24 hours. The sampling frequency is once every 20 minutes while the compositefrequency is once a day.Comprehensive Performance Evaluation (CPE) [Wastewater Treatment]: TheCPE is a thorough review and analysis of a treatment plant’s capabilities and associated administration, operation, and maintenance practices. The primary objective isto determine if significant improvements in treatment can be achieved without makingmajor capital expenditures. CPE includes a process audit.Confounding [Statistics]: Combining indistinguishably the main effect of a factor or adifferential effect between factors (interactions) with the effect of other factor(s), blockfactor(s) and interaction(s) [6]. In other words, two factors are confounded with eachother when we can not determine whether the response is due to one of the factors orboth of the factors. For example, given two factors A and B and assume that theyare confounded, then if a significant effect is detected, it may be due to A, B, or aninteraction between A and B.Context Free Grammar [Computer Science]: A system of definitions that can be usedto break up a sentence into phrases solely on the basis of the sequence of strings in theinput sentence [159, 166].Context, Measure [Measurement Paradigm]: The definition, scale and resolution context of a datum.Context, Measure Definition [Measurement Paradigm]: The identity of the datum’smeasurement method, e.g. COD.Context, Measure Resolution [Measurement Paradigm]: A datum’s resolution: { Crisp,Mean/Standard Deviation, Fuzzy Number, Linguistic Variable }.Appendix B. Glossary 334Context, Measure Scale [Measurement Paradigm]: A datum’s scale: { Nominal, Ordinal, Interval, Ratio }.Context, Parameter [Measurement Paradigm]: The control and coordinate context ofa datum.Context, Parameter Control [Measurement Paradigm]: The role of a datum in making a control decision: { Adjust, Allocate, Disturbance, Performance, Status, QA/QC }.Context, Parameter Model [Measurement Paradigm]: A datum’s role in modelling:{ Currency, Coordinate, Status }.Context, Sample [Measurement Context]: The type and time context of a datum.Context, Sample Time [Measurement Paradigm]: The time interval a datum represents. e.g. June 1, 1991 12:00:00 PM to June 2, 1991 12:00:00 PM.Context, Sample Type [Measurement Paradigm]: The type of sample the datum isderived from, e.g. probe, grab or composite.Context, Structural [Measurement and Structure Paradigm]: What part of the processthe datum describes, i.e. its location.Control [Experimental Design]: An experimental unit that is left as it is.Control [Experimental Layout]: The researcher exploits the structure of the experimental units to reduce the debate over what caused what:• Balance: Assign the treatments to maintain symmetry. i.e. avoid confounding andincomplete blocks if possible. For example, the researcher applies both treatmentstwice, once in each run.• Block: Assign the treatment units such that the units within the blocks are identicalin all ways except for the treatment they receive. For example, the only differencebetween run 1 and 2 is the source of the sludge.Appendix B. Glossary 335• Group: Placement of experimental units into a homogeneous group to which thetreatment is applied. With only two reactors, grouping is not an issue.Control Charts [Statistics]: A plot of successive statistical measures or other values ofa random variable, e.g. mean, range, proportion and trend.Controllability [Control Theory]: A system is controllable if it is possible to find acontrol sequence such that the origin can he reached from any initial state in finitetime [18).Coordinate Parameter [Structure Paradigm]: A fundamental quantity.Correlation Analysis [Statistics]: Correlation analysis investigates the relationshipbetween random variables of equal importance on the basis of a sample [255].Correlation Coefficient, Pearson’s [Statistics]: Pearson’s Correlation Coefficient, r,measures the strength of the linear relationship between two variables, X and Y. Thecoefficient is valid only if the variables are distributed equally across the range of interest.If the relationship between the two variables is not linear, r will either under or overestimate the strength of the relationship. If the sample size is small (8 $ ii < 30), rslightly underestimates the strength of relationship. A relationship is significant whenr > rent. rent depends on the sample size and the significance level.Correlation Coefficient, Spearman’s Rank [Statistics]: The difference between theSpearman Rank Correlation Coefficient, r8, and the Pearson Correlation Coefficient, r,is the former operates on the ranks of the measures and the latter on the measures.Counteract [Operation Paradigm]: To change a manipulated variable to stabilize adetected change in a performance parameter, i.e. increase the recycle rate to stop denitrification in the secondary clarifier.Currency [Structure Paradigm]: The means of exchange between a source and sinknode, e.g. liquid flow. A stream consists of a set of links that share a common currency.Appendix B. Glossary 336Data Apartheid [Statistics]: Data apartheid occurs when the form of a datum determines how the datum will be analyzed, splitting the data set into independent analyses.Data Coverage [Statistics]: A measure of how well a monitoring program describes aprocess.Data Quality [Statistics]: see Quality, DataDatum Quad [Measurement Paradigm]: Value, Preference, Quality and Time Interval.Dead Soldier Plot [Statistics]: A plot of a variable against itself [268].Dead Time [Control Theory]: The interval of time between initiation of an input changeand the start of the resulting response [220], i.e. lag in the system’s response.Delay [Operation Paradigm]: A delay or lag is the time interval between a change ina cause and the corresponding change in an effect. A delay may occur for at least forreasons:1. Due to the characteristics of the process, e.g. distance, velocity or biological lag.2. Due to the monitoring program, e.g. turn-around time.3. Due the change detection routines.4. Due to the operator.A delay may occur in at least four different circumstances:1. Causal: A change in a manipulated or distnrbance parameter causes a change in acorres