Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Ozone and aerosol prediction in the Lower Fraser Valley of British Columbia Torcolini, Joel Cullen 1999

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


831-ubc_1999-0629.pdf [ 3.73MB ]
JSON: 831-1.0228813.json
JSON-LD: 831-1.0228813-ld.json
RDF/XML (Pretty): 831-1.0228813-rdf.xml
RDF/JSON: 831-1.0228813-rdf.json
Turtle: 831-1.0228813-turtle.txt
N-Triples: 831-1.0228813-rdf-ntriples.txt
Original Record: 831-1.0228813-source.json
Full Text

Full Text

O Z O N E AND A E R O S O L PREDICTION IN T H E L O W E R F R A S E R V A L L E Y O F BRITISH C O L U M B I A : A N E U R A L N E T W O R K A P P R O A C H by J O E L C U L L E N T O R C O L I N I B.Sc, The Pennsylvania State University, 1997 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in THE FACULTY OF GRADUATE STUDIES Atmospheric Science Programme We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA September, 1999 © Joel Cullen Torcolini UBC Special Collections - Thesis Authorisation Form In presenting t h i s thesis i n p a r t i a l f u l f i l m e n t of the requirements for an advanced degree at the University of B r i t i s h Columbia, I agree that the Library s h a l l make i t f r e e l y a v a i l a b l e for reference and study. I further agree that permission for extensive copying of t h i s thesis for sch o l a r l y purposes may be granted by the head of my department or by his or her representatives. It i s understood that copying or p u b l i c a t i o n of t h i s thesis for f i n a n c i a l gain s h a l l not be allowed without my written permission. The U n i v e r s i t y of B r i t i s h Columbia Vancouver, Canada 1 of 1 8/30/99 4:35 AM Abstract During summer months, the lower Fraser Valley (LFV) of British Columbia experiences elevated concentrations of ground level ozone and aerosol pollution. Environmental agencies often need to make daily air pollution forecasts for public advisories. Previous studies have demonstrated a relationship between pollution episodes and the various weather regimes influencing the LFV. Statistically based, multivariate linear regression models have been applied to capture relationships between meteorological conditions and ambient pollution concentrations. However, the relationship between atmospheric circulation and pollutant concentrations is quite complex and non-linear. Consequently, this study proposes the use of neural network models to capture the inherently complex pollutant-weather relationship, and to investigate their ability to forecast daily pollutant concentrations when compared to traditional regression models. Several neural network and multivariate regression models have been developed for use in Abbotsford, British Columbia, allowing a comparative study of the two approaches. Results illustrate that neural network techniques do outperform those of traditional regression models. However, neural network models do not dramatically increase forecast ability, therefore practical use in air quality management is still in question. Ill Table of Contents Page Abstract ii Table of Contents iii List of Tables v List of Figures vi Acknowledgements viii 1. Introduction 1 1.1 Ozone 1 1.2 Particulate Matter 4 1.3 Rationale of Approach 7 2. Artificial Neural Networks and Linear Regression Models 11 2.1 Artificial Neural Networks 11 2.2.1 Network Structure 12 2.2.2 Training 18 2.2.3 Overtraining 19 2.2.4 Strengths and Limitations 21 2.2 Linear Regression Models 22 3. Background and Selection of Prediction Variables 24 3.1 Study Region 24 3.1.1 Overview 24 3.1.2 Description of Abbotsford 26 3.2 Air Quality Data 28 3.2.1 Ozone 28 3.2.2 Particulate Matter 31 3.3 Meteorological Data 32 3.3.1 Overview 32 3.3.2 Synoptic Classification Procedure 33 3.3.3 Temperature 35 3.3.4 Transport Parameterization 35 3.4 Summary 38 IV Page 4. Development of Air Quality Models for Abbotsford 39 4.1 Overview 39 4.2 Model Evaluation Statistics 42 4.3 Model Development 43 4.3.1 Neural Network Models 43 4.3.2 Linear Regression Models 48 Ozone 48 Particulate Matter 50 4.3.3 Persistence 54 5. Model Evaluation/Intercomparison 55 5.1 Daily Maximum Ozone Models 56 5.2 Daily Maximum PMio Rolling Average Models 65 5.3 Daily Maximum PM 10 Models 72 5.3.1 Case Study of Daily Maximum PMio Concentrations 79 5.3.2 Modifications to Daily Maximum PMio Models 83 6. Conclusions and Recommendations 87 6.1 Conclusions 87 6.2 Recommendations for Future Work 91 6.2.1 Ozone 91 6.2.2 Particulate Matter 92 6.3 Summary 93 References 94 Appendix 1: Appendix 2: Rolling Average Sample Calculation 100 Neural Network Architectures for All Prediction Categories 102 L i s t of T a b l e s P a g e Table 1 GVRD's Current Air Quality Monitoring Parameters - 1999 30 Table 2 Air Quality Statistics for Abbotsford Monitoring Station (T28) 41 Table 3 Input Combinations Used in Air Quality Models 45 Table 4 Neural Network Models' Evaluation Statistics 47 Table 5 Regression Estimates for Daily Maximum Ozone Model 49 Table 6 Regression Estimates for PMio Rolling Average Models 51 Table 7 Regression Estimates for Daily Maximum PMio Models 52 Table 8 Linear Regression Models' Evaluation Statistics 53 Table 9 Daily Maximum Ozone Models' Evaluation Statistics 57 Table 10 Daily Maximum PMio Rolling Average Models' Evaluation Statistics 66 Table 11 Daily Maximum PMio Models' Evaluation Statistics 73 vi List of Figures Page Figure 1 Illustration of a Feed Forward Neural Network 13 Figure 2 Simplified Neural Network Architecture 14 Figure 3 Curve Illustrating the Logistic-Sigmoid Transfer Function 17 Figure 4 Graph of Neural Network's Training Performance 20 Figure 5 Location of the Lower Fraser Valley 25 Figure 5a Map of Downtown Abbotsford 27 Figure 6 G V R D ' s Air Quality Monitoring Network 29 Figure 7 Synoptic Classification Examples 36 Figure 8 Neural Network Model with Five Input Parameters 46 Figure 9 Neural Network and Linear Regression Models' Predicted Daily . 59 Maximum Ozone Concentrations vs. Observed Concentrations (1998-TestData) Figure 10 Scatterplot of Observed vs. Predicted Daily Maximum Ozone 60 Concentrations (Neural Network Model - 1998 Test Data) Figure 11 Scatterplot of Observed vs. Predicted Daily Maximum Ozone 61 Concentrations (Linear Regression Model - 1998 Test Data) Figure 12 Scatterplot of Observed vs. Predicted Daily Maximum Ozone 62 Concentrations (Persistence Model - 1998 Test Data) Figure 13 Neural Network and Linear Regression Models' Predicted Daily 67 Maximum PMio Rolling Average Concentrations vs. Observed Concentrations (1998 - Test Data) Figure 14 Scatterplot of Observed vs. Predicted Daily Maximum PMio Rolling 68 Average Concentrations (Neural Network Model - 1998 Test Data) Figure 15 Scatterplot of Observed vs. Predicted Daily Maximum PMio Rolling 69 Average Concentrations (Linear Regression Model - 1998 Test Data) vn Page Figure 16 Scatterplot of Observed vs. Predicted Daily Maximum PMio Rolling 70 Average Concentrations (Persistence Model - 1998 Test Data) Figure 17 Neural Network and Linear Regression Models' Predicted Daily 74 Maximum PMio Concentrations vs. Observed Concentrations (1998-Test Data) Figure 18 Scatterplot of Observed vs. Predicted Daily Maximum PMio 75 Concentrations (Neural Network Model - 1998 Test Data) Figure 19 Scatterplot of Observed vs. Predicted Daily Maximum PMio 76 Concentrations (Linear Regression Model - 1998 Test Data) Figure 20 Scatterplot of Observed vs. Predicted Daily Maximum PMio 77 Concentrations (Persistence Model - 1998 Test Data) Figure 21 Hourly Time Series of PMio Concentrations 80 May 1-2, 1998 Figure 22 Hourly Time Series of Carbon Monoxide Concentrations 81 May 1 - 2, 1998 Figure 23 Hourly Time Series of Nitrogen Oxide Concentrations 82 May 1 - 2, 1998 Figure 24 Hourly Time Series of Wind Speed in Abbotsford 84 May 1 - 2, 1998 Vlll Acknowledgments This thesis has been a long time in development and I would like to take this opportunity to express my sincere thanks to the many people who have contributed to this work. I would like to express gratitude to my supervisor Dr. Ian McKendry for his consultation, guidance, and constructive criticism throughout this research. In addition, I am grateful to Dr. Douw Steyn and Dr. William Hsieh who were always available for advice and assistance. I must also thank Dr. Timothy Oke for his simple, insightful comments and critique of this work. In recognition of his helpfulness with the myriad of computer problems throughout this project, thanks to Vincent Kujala for your much needed assistance. This study was financially supported by the National Science and Engineering Research Council of Canada as well as Teaching Assistantships from the University of British Columbia's Geography Department; I am grateful that these institutions considered my research worthy of funding. I have been very fortunate to have support from an excellent group of friends and colleagues. To everyone who was a part, I would like to express my appreciation. A heartfelt note of thanks to my family; without all their care and support this opportunity would have never been possible. Finally, and most importantly, I thank Jessica Dayton for teaching me so much and giving me the best two years of my life. 1 C h a p t e r 1 I n t r o d u c t i o n The lower Fraser Valley of British Columbia (hereafter LFV) is one of the fastest growing regions in North America. The population of the L F V is currently about 1.8 million people with an estimated 50,000 persons moving into the region annually. If current trends continue, the population will nearly double to 3 million by 2021 (GVRD, 1999). This rapid population growth is expected to accelerate existing air quality problems due to increased air emissions from commercial, industrial, and motor vehicle activities. As levels of air pollution rise, residents of the L F V will be exposed to an involuntary health risk, and an increasing burden will be placed on British Columbia's resources and health care system. Consequently, the need exists for daily accurate air pollutant prediction in regions where concentrations are pushing the thresholds of regulatory compliance. This study focuses on forecasting ground level ozone (Cb) and aerosol (PMio) concentrations downwind of Vancouver, British Columbia in the central sector of the LFV. 1.1 O z o n e Ozone is a highly reactive gas and strong oxidant. In the short term, it reacts with body fluids and tissue causing irritation of the eyes and respiratory tract. Frequent exposure to elevated concentrations is detrimental to human health, particularly the young, elderly, and infirm (Stern et al., 1984). Clinical studies on the health effects of ozone find increased emergency room visits for asthmatics strongly correlated with increases in daily ozone concentrations. Ozone is more damaging to vegetation than humans. High ozone levels cause damage to plant's leaves, resulting in markedly decreased crop yields. In 1983, the BC Ministries of the Environment, and Agriculture and Fisheries estimated that crop damage due to elevated ozone concentrations in the LFV at $9 million. In the US, nationwide crop damage caused by ground level ozone is estimated at $2-4 billion per year (NRDC, 1998). Ozone is a secondary pollutant. There are no significant emissions directly into the atmosphere, rather O 3 is formed as a result of reactions between other directly emitted pollutants. Ozone precursors (e.g. N O x and hydrocarbons) mainly originate from use of petrochemicals and fossil fuels by industry and automobiles (Comrie, 1997). Under stagnant synoptic conditions, pollutant transport becomes restricted creating a "chemical soup", which in the presence of sunlight (hv) undergoes a series of photochemical reactions to form tropospheric ozone. Ozone accumulation depends on the local meteorology of a region and the mixture of nitrogen oxides ( N O x ) and hydrocarbons present in the atmosphere. In light winds with little dispersion, a relatively low steady-state ozone concentration may be produced though a series of reactions with oxides of nitrogen ( N O x ) : F O R M A T I O N : NO2 + hv N O + O (1.1) 0 + 02 + M ^ 0 3 + M (1.2) D E S T R U C T I O N : N O + O3 -> NO2 + O2 (1.3) where hv represents solar radiation (A, < 420 nm) and M represents a third species in the reaction that stabilizes the O 3 molecule and absorbs excess energy (Robeson & Steyn, 1990). This cycle leads to the highest ozone concentrations outside of urban areas where N O has time to be oxidized to NO2. Within cities, the ozone concentration is kept low by high N O concentrations (from automobile exhaust) which destroy O 3 rapidly via reaction 1.3 (McKendry, 1993). Ground level ozone concentrations are enhanced by the presence of volatile hydrocarbons (RH), such as methane (CHU) or ethane (C2H6). Natural and anthropogenic hydrocarbons present in the troposphere produce ozone by a sequence of "free radical" reactions with other atmospheric species. The net reaction for this free radical cycle: 4 R H + 4 O2 + 2 hv -» R ' O + H2O + 2 O3 (1.4) illustrates that in the presence of sunlight, hydrocarbons add significantly more O 3 to the steady-state concentration produced by the NOx cycle. Furthermore, R ' O is usually a carbonyl compound (formaldehyde, CH2O, if R H is CH4) which subsequently reacts with sunlight to produce additional ozone (net reaction): C H 2 O + 4 O2 + 2 hv -> C O + 2 O H + 2 O3 (1.5) Therefore under stagnant conditions, the urban atmosphere undergoes a series of photochemical reactions which may produce remarkably high ozone concentrations in rural areas. 1.2 Particulate Matter Due to the highly mutagenic and carcinogenic composition of atmospheric aerosols, ambient particular matter is emerging as a critical public health issue (McKendry, 1999). Elevated PMio (particulate matter suspended in air less than 10 urn) concentrations are associated with numerous cardio-respiratory health effects and visibility reduction in urban areas (Mathai, 1990). Several epidemiological studies link high particulate concentrations with increases in deaths from heart and lung disease 5 (Schwartz and Marcus, 1990; Schwartz and Dockery, 1992 a,b; Dockery et al, 1993; Seaton etal, 1995). The composition of ambient PMio in a given region is a function of both the natural and anthropogenic sources within that region. Natural sources include: • Wind blown soil and mineral particles • Wood smoke from forest fires • Marine aerosols (in coastal environments) • Biological material (pollen, spores, bacteria) Generally, natural sources of atmospheric aerosols produce coarse particles (2.5 - 10 prn) and are considered less dangerous due to the body's natural ability to expel them from the respiratory tract. However, anthropogenically produced particulate matter has a bi-modal distribution. These sources produce both coarse and fine (.1 - 2.5 pm) particulates. Fine particles contain a higher proportion of toxic metals and acidic sulfur species (Li, 1998) indicating formation through high-temperature, fossil fuel combustion processes found in automobiles, home heating units, industrial boilers, and electrical power plants. Another important aspect of ambient PMio concentrations is the distinction between those particles emitted directly into the atmosphere (primary species) and those that are formed via chemical processes taking place in the atmosphere (secondary species). Usually, primary species are generated through the physical processes associated with erosion, resulting in rather coarse particle formation (Pryor & Barthelmie, 6 1996). Secondary species, however, have a much finer distribution due to formation by gas-to-particle conversion. The complex intricacies involved with gas-to-particle conversion create continuously changing chemical compositions and particle size distributions. Therefore our detailed understanding of the urban aerosol is limited (Seinfeld, 1989). Local meteorology and seasonal effects also influence ambient aerosol concentrations in a given region. The LFV's eastern communities experience high PMio concentrations during both the summer and winter seasons. In the wintertime, elevated PMio concentrations occur during occasional "gap-wind outflow" events that entrain coarse crustal material (i.e. - road dust and soil from agricultural activities) into ambient PMio concentrations. However during summer months, high concentrations occur when a ridge of high pressure is established and anticyclonic conditions prevail (McKendry, 1994; Pryor et al. 1994). Under these conditions, transport and dispersion of air pollutants is inhibited by reduced mixing layer depths and local thermo-topographic circulation (Banta et al. 1997; McKendry et al. 1998). Due to the source differences of PMio during summer and winter months, accurate representation of the summer/winter formation mechanisms is not possible with the same input parameters. Therefore, forecast models developed for this study are designed for PMio prediction only during the summertime photochemical smog season. 7 1.3 Rationale of Approach The complexity of ground level ozone and PMio formation makes daily operational forecasting of these pollutants quite laborious. Photochemical models (e.g., the Urban Airshed Model) are often used to capture the intricacies of pollutant formation in the urban environment (Comrie, 1997). However, photochemical models are used mostly in research and planning rather than operational forecasting as they demand complex chemical and meteorological inputs in addition to large computer and personnel commitments (Comrie, 1997). Consequently, the need exists for the development of an air quality model that provides daily pollutant forecasts using easily obtained input variables and minimal computational requirements. Statistical forecast models are frequently employed in situations where understanding of physical or chemical processes is incomplete. The intricacies of physical, chemical, and anthropogenic processes that govern surface ozone (Robeson & Steyn, 1990) and aerosol concentrations do not allow the use of linear-analytical techniques to accurately describe their formation mechanisms. Therefore, non-linear methodologies that respond to input stimuli and have the ability to learn from their environment (Patterson, 1996) may provide new approaches to establishing relationships in these complex systems. Artificial neural networks have been receiving increased use in meteorological applications with varying levels of success. Kuligowski & Barros (1998) present an 8 approach for producing short-term precipitation forecasts using neural network techniques. Hewitson and Crane (1994) demonstrate that the non-linearity of neural networks provide substantial improvement over conventional linear regression techniques for translating the physical processes involved in large-scale atmospheric circulation to local precipitation in Southern Mexico. McGinnis (1994) performed a direct comparison of neural networks versus traditional linear techniques when predicting snowfall from synoptic circulation in the Upper Colorado River Basin. This study used identical environmental data sets and found a two-fold increase in explained variance by neural network models, demonstrating their improved predictive power over linear methodologies. Promising results in other meteorological applications have prompted researchers to investigate neural network techniques in the area of air quality forecasting. Comrie (1997) found neural networks to be "somewhat, but not overwhelmingly" better than linear regression techniques for weather-based O 3 forecasting in eight cities across the US. Dorling & Gardner (1999) successfully apply neural networks for hourly NOx and NO2 modeling in London using only basic temporal and station meteorological data. However, no studies have been conducted using neural networks for daily PMio forecasting. Currently, daily maximum ozone concentrations are forecast in the LFV using Robeson & Steyn's (1990) temperature-persistence regression model (TEMPER). However, this model has limited capability and difficulty predicting peak ozone 9 concentrations. Therefore, this study aims to improve upon current ozone forecast techniques and provide daily PMio forecasts in the LFV using neural network based models. A purely statistical approach for daily pollutant concentration forecasting is examined using artificial neural networks. These physically based models focus on the statistical relationship between synoptic circulation, mesoscale meteorological conditions, and surface pollutant concentrations. The objectives associated with this study are: Primary Objective • Investigate the feasibility of a neural network based model for operational O 3 and PMio forecasting during summer months in the central LFV. Secondary Objective • Comparison of traditional linear regression methods with a neural network when applied to the same environmental data set. The details of neural network and linear regression models are examined in the next chapter. Chapter 3 describes the sources and meteorology that produce elevated O 3 and PMio concentrations in the LFV and provides the rationale for model input selection. Chapter 4 presents the O 3 and PMio models developed for this study. In this chapter, several neural network and linear regression models are produced and the best models 10 selected for further evaluation. A direct comparison of the different forecast techniques is presented in Chapter 5 using only the best neural network and linear regression model in each prediction category. The last chapter assesses model predictions, revisits the objectives of the study and presents recommendations for future work. 11 Chapter 2 Artificial Neural Network and Linear Regression Models Limited understanding of the physical and chemical processes involved in the formation of elevated pollution concentrations does not allow accurate representation with numeric values. Often, it is easier to categorize these phenomena into broader classes, and investigate relationships between these classes. Traditionally, stepwise multiple regression analysis has been used when a variety of independent variables influence the prediction of a dependent variable (McGinnis, 1994). However, artificial intelligence may also have considerable potential for establishing patterns in complex systems, and provide new approaches to finding relationships within heterogeneous processes. In this chapter, the basic structure and application of neural network and multivariate regression models for air quality forecasting is investigated. 2.1 Artificial Neural Networks A neural network (NN) is a relatively new data processing system, inspired by the learning and pattern recognition mechanisms of the biological nervous system. Neural networks' roots are biological/psychological, however they are beginning to find increased application in a variety of fields where analytic solutions are hard to find, hidden, or non-existent. NNs are designed after the brain's massive parallel processing structure, which arises from the neurons being interconnected by a network (Hsieh & Tang, 1998). Acting much like a "black box", they are able to learn relationships between 12 a pair of multidimensional data sets and predict new values based on input data (McGinnis, 1994). 2.1.1 Network Structure Simple, feed-forward neural networks are developed for this study, since we are only interested in defining a relationship in one direction: the meteorological surrogates' influence on surface pollutant concentrations. A simplified illustration of a feed forward neural network is presented in Figure 1. There are many possible model configurations, but they all follow the same basic framework resulting in a series of interconnected nodes. These nodes are arranged in layers (input, hidden, and output), and each node is connected to the output node by a synapse - a weight vector that quantitatively describes the relationship between the nodes it connects (Kuligowski & Barros, 1998). During training, the nodes are adjusted so the network learns the desired association between forecasted and observed variables. Figure 2 illustrates a simplified network architecture. Each input node (X) contributes to the network's output as their values are multiplied by the weights of the synapses (F) that connect them to the output node (Kuligowski & Barros, 1998). Within this activation function (Z), the results are tallied, and then a bias term (b) is added to this sum: Zi (X, Y,b)=Xi*Yu +X2* Y2J + + Xi • Yu + b 13 Figure 1: Feed forward neural network illustration with 2-14-7-1 architecture. Hidden layers—perform an arbitrary mapping of an input to output 14 Figure 2: Simplified neural network architecture. TRANSFER FUNCTION (Logistic Sigmoid) 0i =f (Zi(X,Y,b)) 15 The bias term is included to partially mimic threshold characteristics of a neuron, triggering a node only when the signal it receives exceeds a certain level (Abareshi & Schuepp, 1998). The relationship between Z, X, Y, and b is similar to that of traditional linear regressions, however a non-linear function of choice 9, called a transfer function, is applied to this sum before its signal is sent to the output node (Kuligowski & Barros, 1998). The application of this non-linear transfer function is what enables the network to estimate non-linearity between input variables. There are a variety of transfer functions used in neural network modeling: Linear: 6L(Z) = aZ + b b, cc<Z Hard-Limiting : 6H(Z)= { a,Z<a b, Saturating Ramp : &R(Z)= { cZ + d a2<Z a ,<Z<a2 Z<a, a, Logistic - Sigmoid : 6 s (Z) = 1 16 In this study, the logistic-sigmoid (LS) function (illustrated in Figure 3) is the transfer function of choice. It is the most common transfer function used in neural network modeling, because it is smooth and easily differentiable allowing the network to training algorithms to operate most efficiently (Statistica NN, 1998). When the sigmoid function is applied to an input parameter, it is only truly sensitive in a range between about 0.2 and 0.8. Consequently, all data input into neural network models are scaled to provide values between 0.2 and 0.8 following Comrie (1997): ni = 0.2 + 0.6 (% - Xmi" )(X»™- X™) where 77; represents the individual scaled observations, %i the actual pollutant concentration, and %nwx and ^  the maximum and minimum pollutant concentrations in the data set. This is done so input parameters are constrained to the sensitive regions of the sigmoid function, and so that any variable with a large numeric value does not dominate the regression (Comrie, 1997). A typical neural network is composed of a series of interconnected nodes arranged in layers: • Input layer (contains the values of the input variables) • Hidden layer (where the input data is processed) • Output layer (produces the resulting predictand value). Generally, networks have one input and one output layer. However, they may have any number of hidden layers depending on their size and complexity. The hidden layer(s) Figure 3: Logistic-Sigmoid Transfer Function 1 1 0.9 0.8 0.7 / * / 0.6 -0.5; ."0.4 -0.3 -0.2 -0.1 -9-- 4 - 3 - 2 - 1 0 1 2 3 4 18 serves as the learning mechanism that stores weights reflecting patterns in the input data. Additional hidden layers merely represent another set of calculations mapping the input vector to the output vector (Kuligowski & Barros, 1998). Therefore, one might believe that numerous hidden layers lead to increased network performance; this is not the case. Cybenko (1998) (cited in Hsieh and Tang, 1998) illustrates that to approximate any particular set of functions to a specific accuracy, at most two hidden layers are required. 2.1.2 Training Neural networks are essentially pattern recognition systems that learn to associate a specific pattern with a given result, deducing a model from a set of training data. Data sets used in neural network modeling consist of a number of cases (or samples), each containing values for a number of input variables (Statistica: NN, 1998). Data are usually divided into three subsets: • Training • Verification • Test Training data allows the network to learn the desired association between forecasted and observed variables. The verification set provides an independent check on the progress of training (to avoid overtraining, discussed in the next section), and a test data set is constructed to perform a quasi-independent measure of the accuracy of predictions. 19 Networks simulate the brain's learning mechanism as they obtain experimental knowledge, store it in weights, and recall it later (Abareshi & Schuepp, 1998). Initially, weights of individual nodes are randomly assigned (generally between 0.1 and 1.0), then adjusted through training by presenting the network with a series of known cases. This process progresses iteratively, as the entire training data set is sequentially run through the network in a procedure known as an epoch. During each epoch, the network calculates an output value based on each set of inputs, compares this value to the known output value, and adjusts the synapses' weights to produce the lowest error in its calculated output (Kuligowski & Barros, 1998). Training performance (Figure 4) is measured as the root mean square error (RMSE) between the output generated by the network and the target output. Training is supervised and stopped when the RMSE (calculated over the verification data set) reaches a minimum and starts to rise. This "training" process effectively infers a network model from the available training data (Statistica: NN, 1998). 2.1.3 Overtraining Unfortunately, the training procedure outlined above does not minimize the error most important in forecast applications—the error the model will make when new cases are presented to it (Statsitica:NN, 1998). The training algorithm is designed to minimize error of training cases, however cannot minimize the error when "real" data is presented to the model. Theoretically it is possible for a network to learn relationships so well that it only learns relationships specific to the training data (Hewitson and Crane, 1994). This 20 Figure 4: Example of Neural Network Training Graph Number of Epochs 21 demonstrates one of the common problems associated with neural network modeling -over-learning, or overtraining. Overtraining occurs if a network is trained so accurately that it classifies all the training data, yet fails to classify patterns of new data. To circumvent overtraining, a number of cases are extracted from the initial training data, and set aside into a cross-verification data set. These data are not actually used by the training algorithm, rather they are used to keep an independent check on the progress of training. After each training epoch, the RMSE is calculated for this verification set. The verification error is used in conjunction with the training error to establish when training should cease. 2.1.4 Strengths and Limitations Predictably, neural networks are not the ultimate solution for modeling complex, non-linear processes. The main advantages of neural networks over traditional modeling techniques are their ability to estimate non-linear relationships and to "learn by example", which allows the model to generate forecasts without having comprehensive prior knowledge of all processes involved. In addition, they are capable of generalizing relationships between input-output variables and their parallel, distributed architecture gives them some tolerance to input noise (Abareshi & Schuepp, 1998). Perhaps the most important constraint in neural network modeling is the need for a data set that accurately describes the characteristics of a system. It is crucial to obtain 22 representative data for model training because the network learns relationships from the training data; therefore it must adequately represent the behavior of the system. Neural nets require as much training data as possible to span the complexity of the input space (Hewitson and Crane, 1994). Their forecast ability is strongest when presented with inputs within the bounds of their training data. If the network is presented with input data that falls outside the bounds of the training data, it is positioned into an input space never before seen by the network (Hewitson and Crane, 1994). Consequently, the network's predictive ability is hindered because the input is located in some point the network is unfamiliar with. Another major limitation of neural network modeling is that due to structural complexity and nonlinear relationships between variables, interpretation of network patterns is very difficult. In nonlinear systems, interdependencies and redundancies may exist between input parameters. A set of parameters may exhibit no predictive ability individually, however may be extremely influential when used together (Statistica:NN, 1998). Therefore, unlike parameters from a linear regression model, the weights of synapses in neural networks are nearly incomprehensible (Hsieh and Tang, 1998). 2.2 Linear Regression Models In this study, linear regression models are developed for comparative purposes against the neural network approach. The basis for the linear regression methodology is exactly the same principal which the neural network seeks to address: pollutant 23 concentrations in the LFV are a function of the synoptic and mesoscale forcings of the region. Since regression models and neural networks are rather disparate in nature, models are developed in their most straightforward, typical fashion (Comrie, 1997). This is done for two reasons: • To keep model comparison as simple as possible. • To determine the feasibility of using the models in an operational mode. Regression models are developed by deriving a number of transfer functions between synoptic map types, meteorological variables, and observed pollutant concentrations. Transfer functions take the form of "standard" multivariate regression models (i.e. no input variable transformation or stepwise variable-selection scheme) with the pollutant concentration as the dependent variable. To ensure the quality of subsequent model analysis, independent variables used in the linear regression models consist of the exact same time series of variables input into the neural net models. An example of the linear relationship between pollutant concentrations (%t), synoptic map types (SMsoo, SMmsi), and meteorological inputs - temperature (Ti) and transport (W) is illustrated: %t = bo + biW+ bi SMmsi + b3 SMsoo + b4Tt + bs where bs, b4, b3, bi, bi, and bo are coefficients from an ordinary least squares regression. 24 Chapter 3 Background and Selection of Prediction Variables 3.1 Study Region 3.1.1 Overview The LFV is a broad, relatively flat valley located in the southwest corner of British Columbia (see Figure 5). It is bounded to the north and east by the Coast Mountain Range, to the south by the Cascade Mountain Range and to the west by the Straight of Georgia. The northwestern edge is the fastest growing and most densely populated area of the LFV; consequently most anthropogenic emissions are assumed to originate from that area (Robeson & Steyn, 1990). During summer months, the central and eastern sections of the LFV often experience the highest pollutant concentrations. Persistent summertime anticyclones produce stable atmospheric conditions and sunny, hot weather in southwest British Columbia. Under these conditions, coastal areas of the LFV experience a sea-breeze circulation, advecting cool marine air onshore (Steyn & Oke, 1982). This influx of cool air limits the boundary layer mixing depth (Steyn & Faulkner, 1986) and combined with the trapping effect of the mountains, prevents the dispersion of pollutants. Consequently, as airflow is constricted east-west along the Fraser Valley by the Coast Mountains, the central and eastern sections of the LFV experience elevated concentrations as pollutants 25 26 are transported downwind of the urban core. During the rest of the year, the LFV generally experiences good dispersion conditions due to a continuous series of North Pacific cyclonic systems (McKendry, 1999). 3.1.2 Description of Abbotsford Abbotsford (station T28 - exact location of the monitoring site is illustrated in Figure 6) was selected to investigate statistical forecasting methods due to: • The severity of historical pollutant concentrations. • Its urban setting which captures the effects of anthropogenic pollutant production. • It provides a single pollutant time series representative of the central LFV. Abbotsford (pop. - 110,000 people) is located in a rural setting at the eastern end of the Fraser River Valley about 60 km east of Vancouver. The farmland surrounding the urban core supports one of the most productive farming communities in British Columbia. The city of Abbotsford has shown significant growth over the past decade leading to sizeable increases in construction and motor vehicle use (City of Abbotsford, 1999). Therefore, it is believed that background pollutant concentrations (mostly due to regional transport from the Greater Vancouver region) are enhanced by this increased activity surrounding the urban core. 27 Figure 5a: Location of PMio monitoring station in downtown Abbotsford. Station marked with a circled "X". 3.2 Air Quality Data 28 Since 1972, the Greater Vancouver Regional District (GVRD), in conjunction with the federal and provincial governments, has been monitoring airborne pollutants in the LFV. They have assembled a network which currently consists of 39 stations (22 continuous monitoring stations) spanning 18 communities in the region. The spatial distribution of the continuous monitoring stations is illustrated in Figure 6. The range of pollutants monitored at each site varies from station to station; Table 1 lists the network's current monitoring parameters. Ozone and PMio data used in this study have been extracted from this monitoring network. To examine the relationship between synoptic circulation and surface pollutant concentrations, five years (1994-1998) of ozone and PMio data are examined. As this pilot study looks only to investigate mechanisms that produce high pollutant concentrations during the photochemical smog season, only data from April 15th to October 15th of each year is used. 3.2.1 Ozone In the LFV, ozone concentrations are monitored using TECO Model 49 samplers, which use the principles of ultraviolet photometry to measure ozone levels. The samplers measure concentrations based on the ozone molecule's ability to absorb specific wavelengths of light (GVRD, 1998). Measurements are taken 2.5 meters above ground, 30 Table 1: Pollutants continuously monitored in the lower Fraser Valley air quality monitoring network (1999) Station S 0 2 T R S N02 C O 0 3 P M i o C O H T1 X X X X X T2 X X X X X T4 X X X X X X X T5 X X X X X T6 X X X X T7 X X X X X T9 X X X X X X X T10 X X X X T12 X X X X X T13 X X X T14 X X T15 X X X X X T17 X X X X X X T18 X X X X X T21 X T22 X T23 X X T25 X T26 X X X X X T27 X X X X X T28 X X X X X X T29 X X X X * "X" denotes a pollutant that is monitored 31 instantaneously 60 times per hour (Robeson & Steyn, 1990). Data is then transmitted once a minute to a central computer where an hourly average is calculated. For hourly concentrations to be considered valid, at least 75% of the relevant data in a given hour must be transmitted (GVRD, 1998). For the purposes of this study, the daily maximum 1-hour concentration is extracted from each day's record, and used as model input. 3.2.2 Particulate Matter Particulate concentrations are monitored continuously in downtown Abbotsford by a Tapered Element Oscillating Microbalance (TEOM) equipped with a 10 pm size selective inlet. This instrument measures PMio concentrations by drawing heated air through a small filter sitting on the end of a hollow tube. One end of this tube is fixed; therefore it can oscillate like a tuning fork. As particulate matter collects on the filter, the rate at which the tube oscillates changes (Li, 1998). The sampler calculates PMio concentrations based on the oscillation frequency of the tube. PMio data are recorded hourly and used to calculate a 24-hour rolling average concentration for each hour of the day (Appendix 1). PMio levels are reported both as hourly readings and as hourly rolling averages. In this study, separate PMio models are developed to forecast the daily maximum rolling average (PMio RAVG), using the previous day's maximum 24-hour rolling average concentration as model input, and daily maximum 1-hour PMio concentration (PMio DMAX), using the previous day's daily 32 maximum 1-hour concentration as model input. Initially, models were developed only to forecast the maximum daily 24-hour rolling average for three distinct reasons: • The 24-hour rolling average is the standard used by regulatory agencies. • The 24-hour rolling average is a reliable indicator of the limiting environmental and meteorological conditions governing a region on a given day. • Health data suggests that short-term exposure to elevated concentrations is less detrimental than long-term exposure to moderate concentrations (Vedal, 1995). However, only minor adjustments were required to test the feasibility of forecasting daily maximum PMio concentrations with the same techniques. Consequently, two separate model sets are developed to forecast PMio levels in downtown Abbotsford, using the same meteorological parameters as model input. 3.3 Meteorological Data 3.3.1 Overview Prevailing meteorological conditions strongly influence ambient pollutant levels throughout the LFV. Several studies (McKendry, 1999; McKendry, 1994, Prior et. al, 1995; Burrows et. al, 1995) have found that elevated pollutant concentrations in the LFV are driven by local meteorology (i.e. - limited dispersion/accumulation) rather than emission patterns. Therefore, models are designed to forecast pollutant concentrations based on the state of the atmosphere on a given day. 33 Many meteorological variables may be considered that might influence Cb and PMio concentrations in a region. These include: daily maximum temperature, wind speed, wind direction, mixing-layer depth, relative humidity, rainfall, and antecedent soil moisture. But if models are to be used in an operational mode, variables need to be easily forecast or obtained by the user on a daily basis. Therefore, variables such as mixing-layer depth and antecedent soil moisture, which might provide significant insight but are not readily available to the user, cannot be used as model input. To circumvent this problem, daily synoptic classifications are used to capture the overall meteorological situation influencing the LFV on a given day. Pryor et al. (1995), McKendry (1994), and Comrie & Yarnal (1992) demonstrate that the meteorological conditions that govern ground level pollutant concentrations are strongly influenced by the synoptic circulation through the passage of fronts, cyclones and anticyclones. Therefore, classifications are designed to act as surrogates for the overall meteorological conditions that produce elevated pollutant concentrations in the LFV. In addition to synoptic classifications, mesoscale variables (i.e. - daily maximum temperature, wind speed, and wind direction) are used as model input to represent the state of the surface environment on a given day. 3.3.2 Synoptic Classification Procedure A daily synoptic weather map classification scheme is implemented to capture the influence of atmospheric circulation on surface pollutant concentrations. However, when 34 considering atmospheric circulation, it is imperative to characterize flow on a variety of scales (Yarnal, 1993). This study focuses on the influence of upper atmospheric and surface level forcings on ground level pollutant concentrations. To capture these forcings, mean sea level (MSL) and 500 hPa map types at 1200 UTC are extracted from the National Center for Environmental Protection/National Center for Atmospheric Research's (NCEP/NCAR) joint 40-year reanalysis project. The pressure fields bound the northwestern United States and southwestern Canada from 140° W to 115° W and from 40° N to 65° N with 2.5° latitude-longitude resolution. For this study, daily averaged MSL and 500 hPa geopotential height fields were first extracted from the NCEP/NCAR reanalysis project for years 1994 to 1998. To remove any seasonal influence, each grid point value on a given day was taken as the difference from the 13-day centered mean of all grid point values (Hewitson and Crane, 1992). In addition to the MSL and 500 hPa classification sets, a separate COMBINED MSL / 500 hPa classification set is formed by concatenating the filtered MSL and 500 hPa series. Next, days from April 15th to October 15th were extracted from each series and the gridded anomalies standardized. The principal components of the correlation matrices for the MSL, 500 hPa, and combined sets were calculated, and the number of components to retain found using the rule-N test (Overland & Preisendorfer, 1982). The retained principal component scores were then clustered using a batch k-means clustering algorithm (Linde et al., 1980). The number of clusters present in each data set was determined using the Davies-Bouldin cluster separation measure (Davies & Bouldin, 1979). In all cases, the k-means algorithm was initialized using the scheme presented in 35 Katsavounidis et al. (1994). This procedure produced 15 - MSL, 16 - 500 hPa, and 22 -combined classifications (Figure 7), which are then used as input to the neural network and linear regression models. 3.3.3 Temperature Data Several studies (Pryor etal. 1995; Comrie, 1997; Robeson & Steyn, 1990) have found daily maximum temperature to be an excellent meteorological surrogate for the conditions associated with photochemical smog production. Unfortunately, the GVRD does not deploy meteorological instruments at their Abbotsford site. Therefore, daily maximum temperatures are extracted from 1-hour averaged air temperature reading at Langley Central (T27) (Figure 6) and considered representative of the central LFV. 3.3.4 Transport Parameterization In summer, under clear skies and intense solar radiation, the LFV often experiences westerly flow due to the land/sea breeze circulation (Steyn & McKendry, 1988). During the day, strong onshore flow is established due to the differential heating of land and water. Airborne contaminants become entrained in this on-shore flow and transported inland. At night, the diurnal reversal of circulation creates a weak land breeze, which results in pollutants being transported back and forth across the valley with little dispersion (Steyn & Faulkner, 1986). If synoptic conditions persist for several days, the land/sea breeze circulation may create significantly higher pollutant concentrations in Figure 7: Examples of Mean Sea Level and 500 hPa synoptic classifications for the period of June 26 - July 2, 1995. 37 eastern regions of the LFV due to build-up and transport of pollutants from Vancouver's urban core. Daily wind speed and wind direction values are taken from measurements at the Vancouver International Airport (Figure 6), and used to estimate the influence of Greater Vancouver's pollutant production on eastern sections of the LFV. Wind speed (in km/hr) and wind direction (in degrees) measurements are reported hourly; therefore must be parameterized for use as a nominal input into a daily forecast model. A "transport term" is developed using the combined effect of wind speed and direction on the transport of airborne pollutants. If the wind direction on a given day is between 225° < <j) < 315° from 8 a.m. to 3 p.m. and the average wind speed over that time greater than 8 km/hr (sufficient to transport pollutants 60 km to Abbotsford), then transport is assumed. 38 This term is designed to give an indication as to whether there could be transport of pollutants from the downtown core, and input into the models as a YES/NO variable. 3.4 Summary In this chapter, the intricacies of the study region and rationale for the selection of prediction variables was presented. It is felt that meteorological conditions not emission factors are responsible for elevated pollution concentrations in Abbotsford, BC-Therefore numerous meteorological variables are considered for use as model input. However availability constraints of certain meteorological variables (e.g. - mixing-layer depth, antecedent soil moisture) do not allow their use in an operational mode. Consequently, daily synoptic classifications are employed as surrogates for these meteorological parameters. Both neural network and linear regression models are constructed using the synoptic classifications, daily maximum temperature, transport term, and previous day's pollutant concentration. In the next chapter, the development of the various models is presented. 39 Chapter 4 Development of Air Quality Models for Abbotsford 4.1 Overview In statistical modeling, the most crucial step in model development is determining which variables or combination of variables should be used as model input. As there is no predetermined method to resolve which inputs should be considered, various combinations of models and data are constructed to determine which input parameters are most useful to the prediction. It must be noted that in this study separate analysis of methodologies (neural network and linear regression) is conducted for each pollutant forecasted. Therefore, in each prediction category separate neural network and linear regression models are developed for comparative purposes. Models are developed using data extracted from April 15th to October 15th each year from 1994—1998. A total of 902 cases are used for development of the ozone models, however only 766 cases are used for PMio model development due to missing data. No days are used with interpolated or missing data. Data for each model is partitioned into a "training set" (used for development of regression models and neural network training) and a "test set" (used for regression verification and network testing). The 03 models are constructed using a training set of 721 observations from 1994—1997, and tested on 181 observations from 1998. Similarly, PMio models are developed using the 585 available cases from 1994—1997, and tested with 181 cases from 1998. 40 Table 2 shows statistics for the daily maximum ozone data and daily maximum PMio rolling average data used in this study. Comparison of the daily maximum ozone data with Joe et al. (1996) illustrates that the data used in this study is consistent with previous studies of ozone in the LFV. Concentrations above the National Ambient Air Quality Objective (NAAQO) of 82 ppb are rare events at Abbotsford. However, exceedances of the maximum "desirable" level of 51 ppb are becoming more frequent, most likely due to changing emission patterns as the population of Greater Vancouver has been growing rapidly in recent years. An eastward shift in population away from Vancouver's urban core has resulted in an eastward shift of peak concentrations of ozone, and thus an increasing trend in ozone is observed in Abbotsford (Joe et al., 1996). Continuous monitoring of PMio in the LFV began on July 20, 1994. Therefore the data record is still too short to provide a sound statistical analysis of the spatial and temporal trends of PMio concentrations (McKendry, 1999). However, the statistics presented for Abbotsford in Table 2 are consistent with other studies of PMio in the LFV and Canada (McKendry, 1999; Li, 1998; Pryor & Barthelmie, 1996). Since monitoring began in 1994, the annual mean has never exceeded 20 plgm"3 for any of the stations in the LFV (GVRD, 1999). However the Abbotsford station has the second highest annual average of all stations in the LFV at 15.2 p,gm3. PMio concentrations are pushing the threshold of regulatory compliance in Abbotsford. BC Environment set the "acceptable" PMio 24-hour concentration at 50 u\gm"3, which has been exceeded 24 times and 40 ugm"3 has been exceeded 55 times since 1994. 41 T A B L E 2: Air quality data statistics for the Abbotsford station (T28) OZONE Data Period Ozone Average (ppb) Ozone Standard Deviation (ppb) Number of Observations # > 81 ppb # > 51 ppb Probability > 81 ppb Probability > 51 ppb April 15, 1994 to October 15, 1998 33.46 12.93 905 2 73 0.002 0.081 PMio 24-Hour Concentration Data Period PMio Average (ugm-3) PM10 Standard Deviation (ugm-3) Number of Observations # > 50 ugm-3 # > 40 ugm-3 Probability > 50 ugm-3 Probability > 40 ugm-3 July 20, 1994 to October 15, 1998 22.72 10.33 766 24 55 0.031 0.072 42 4.2 Model Evaluation Statistics In order to provide a quantitative comparison of test forecasts between models, the Pearson's correlation coefficient (R2) is calculated for the neural network and linear regression models in each category. Although widely used as a measure of agreement between predicted and observed values, the correlation coefficient is not the most reliable measure of correspondence. For example, consistent over- or under-prediction by the model will result in a high degree of correlation (Gould, 1994); however this result is spurious, as it does not provide a true indication of the model's predictive ability. To avoid misleading evaluation, Wilmott (1981,1982) reports an Index of Agreement (d) indicating the degree to which a model's predictions are error free (Comrie, 1997). This dimensionless index has limits of 0.0 (no agreement) to 1.0 (perfect agreement) and is defined as: E(Pi-Oi) 2 d= 1 - — z( I P M + I O M ) 2 where Pi and Oi are the observed and predicted values, respectively, P/i = Pi- O and 0'\ = Oi- O, with O being the average observed value (Steyn & McKendry, 1988). In this study, both the Index of Agreement and Pearson's correlation coefficient are calculated, then used to determine which network and linear regression are extracted for further evaluation. 43 4.3 Model Development 4.3.1 Neural Network Models In this study, the Statistica: Neural Network software package was used for network development. To determine the appropriate NN architecture, the standard approach has been to conduct a series of experiments, try various network types and architectures, decide which are promising, and concentrate on fine tuning those networks. This method is extremely time consuming, and thus limits the number of network architectures that may be reasonably explored. The Statistica: Neural Network package includes an "Automatic Network Designer" feature that treats the issue of architecture determination as an optimization problem. It employs line-search and simulated annealing search strategies to explore numerous possible architectures and select the best configuration (Statistica: NN manual, 1998). This strategy is similar to the trial and error method described above, however its main advantage is that it does not require user input once started. Therefore, it can investigate far more architecture combinations, producing the most efficient network configuration for a given set of input parameters. The "Automatic Network Designer" was used to develop all network architectures in this study. 44 In each experiment, four different network and data combinations are developed to determine the most effective NN model (Table 3). Initially, networks were designed to use: • Previous day's pollutant concentration • Forecasted daily maximum temperature • Predicted transport term • Forecasted 500-hPa classifications (12:00 UTC) • Forecasted MSL classifications (12:00 UTC) to predict a daily pollutant concentration (Figure 8). However, to evaluate the sensitivity of different input parameters, networks are modified by sequentially excluding variables. By successively removing variables from the training set, one is able to assess their relative importance to the overall ability of the network to train and predict. High sensitivity to specific input parameters would imply relatively high importance and vice-versa. Numerous network architectures and data combinations are developed in this study. Each model's performance is evaluated by forecasting the 181 pollutant observations in the test data set. The R2 and d statistics presented in Table 4 represent the best results obtained in each category. Network architectures and data combinations of models used in all prediction categories ( O 3 DMAX, PMio RAVG, and PMio DMAX) are presented in Appendix 2. 45 TABLE 3: Input parameters used by models to forecast daily maximum O 3 , daily maximum PMio rolling average, and daily maximum PMio concentrations. 5 INPUT MODELS • Previous day's pollutant concentration • Forecasted daily maximum temperature • Forecasted transport term • Forecasted 500-hPa synoptic classification (12:00 UTC) • Forecasted MSL synoptic classification (12:00 UTC) 4 INPUT MODELS • Previous day's pollutant concentration • Forecasted daily maximum temperature • Forecasted transport term • Forecasted COMBINED synoptic classification (12:00 UTC) 3 INPUT MODELS • Previous day's pollutant concentration • Forecasted daily maximum temperature • Forecasted transport term 2 INPUT MODELS • Previous day's pollutant concentration • Forecasted daily maximum temperature 46 Figure 8: Example of a neural network model with 5 inputs and a 5-5-5-1 architecture. Previous Day's Pollutant Concentration Daily Maximum Temperature Transport Term 500 hPa Classification MSL Classification Pollutant Prediction 47 T A B L E 4: Neural network model evaluation statistics O Z O N E DMAX: Number of Inputs Architecture R A 2 d 5 - LS 5 - 5 - 5 - 1 0.501 0.831 4 - LS 4 - 4 - 3 - 1 0.508 0.834 3 - L S 3 - 4 - 2 - 1 0.518 0.836 2 - L S 2-14-7-1 0.552 0.849 PMio R A V G : Number of Inputs Architecture R A 2 d 5 - L S 5 - 2 - 2 - 1 0.645 0.875 4 - L S 4 - 3 - 3 - 1 0.657 0.878 3 - L S 3 - 5 - 2 - 1 0.637 0.870 2 - L S 2 - 11 -11 - 1 0.652 0.877 PMio DMAX: Number of Inputs Architecture R A2 d 5 - L S 5 - 1 3 - 13- 1 0.100 0.431 4 - L S 4 - 8 - 1 8 - 1 0.181 0.481 3 - L S 3 - 3 - 3 - 1 0.136 0.468 2 - L S 2 - 1 3 - 6 - 1 0.190 0.490 * LS - inputs scaled between 0.2 and 0.8 to fit logistic (sigmoid) transfer function ** Highlighted models are chosen for further comparison 48 4.3.2 Linear Regression Models Ozone Although several NN architectures and data combinations are examined for daily maximum O 3 prediction, only one linear regression model is created for comparison against the neural network approach. Robeson & Steyn (1990) developed a bi-variate regression model, using the previous day's daily maximum ozone concentration <-;) and daily maximum air temperature (Ti) as input parameters, that explains a large proportion of the variance in J£ '• The expression takes the form of: %t = bo + bi Tt + b2 X t-i where bo , bi, and bi are the coefficients from an ordinary least squares regression (Robeson & Steyn, 1990). This model has shown consistent performance and is currently used by GVRD to predict daily maximum ozone concentrations in the LFV. Therefore, a new temperature-persistence based (TEMPER) model is derived from data used in this study to evaluate its forecast ability against neural network models. Parameter estimates for the new TEMPER model are presented in Table 5. TABLE 5: Linear regression parameter estimates for daily maximum ozone model ( T E M P E R ) . %t = bo + b i Tt + b2 Xm bo = -2.360 b i = 1.043 b2 = 0.459 50 PMio The stepwise multiple regression procedure is performed for each combination of inputs used in the neural network models, allowing each data combination its own unique regression model. Parameters for the PMio RAVG and PMio DMAX models are provided in Tables 6 and 7, respectively. Again many data combinations were explored, but the regressions presented best emulate the linear relationship between the synoptic circulation, meteorological surrogates (used in this study), and observed PMio concentrations in downtown Abbotsford, BC. Results for each pollutant category (PMio RAVG and PMio DMAX) are presented in Table 8. For models predicting the daily maximum PMio rolling average concentration, the inclusion of additional input parameters produces minimal improvement over a temperature-persistence forecast model. This illustrates the dominance of the persistence signal in the PMio RAVG regression estimates. Again, the PMio daily maximum regression models generate much lower R2 and d values than either the O3 DMAX or PMio RAVG models. It should be noted that analysis of PMio DMAX evaluation statistics reveals that not all the variables presented in Chapter 3 improve the forecast skill of the model. The O 3 DMAX and PMio RAVG regression models have the best performance at the high end (5-input for PMio RAVG) or low end (2-input for O 3 DMAX) of the input range. However, the most effective PMio DMAX model is developed using 4-inputs indicating some confusion in the 5-input PMio DMAX model due to the inclusion of additional parameters. 51 TABLE 6: Linear regression parameter estimates for daily maximum PMio rolling average models 1) 5-inputmodel %t = b o + b i W + bi S M m s i + b 3 SMsoo + b 4 T t + bs %t-i b o = -3.818 b i = 0.573 b 2 = 0.043 bs = 0.034 b 4 = 0.583 bs = 0.629 2) 4 - input model X t = bo + b i W + b2 SMcomb + b3 T t + b4 %t-i b o = -3.714 bi = 0.566 b2 = 0.065 bs = 0.572 b4 = 0.630 3) 3 - input model %t = b o + b i W + b 2 T t + b 3 %t-i b o = -3.066 b i = 0.559 b 2 = 0.581 b 3 = 0.628 4) 2 - input model X t = bo + b i T t + b2 X t - i b o = -3.217 b i = 0.602 b2 = 0.626 52 T A B L E 7: Linear regression parameter estimates for daily maximum PMio models 1) 5-inputmodel %t = bo + b l W + D2 S M m s i + b3 SM500 + b4 T t + bs %t-l bo= -12.106 bi = 4.608 b2= 0.358 bs = 0.095 b4 = 2.256 bs = 0.206 2) 4 - input model %t = bo + b i W + b2 SMcomb + b3 T t + b4 %t-i bo= -12.553 b i = 4.371 b2 = 0.337 b3 = 2.201 b4 = 0.206 3) 3 - input model Xt = bo + bi W + b2 T t + b3 Xn bo = -9.260 bi = 4.345 b2 = 2.238 b3 = 0.205 4) 2 - input model Xt = bo + b i T t + b2 %t-i bo= -10.478 bi = 2.389 b2 = 0.206 53 T A B L E 8: Linear regression model evaluation statistics O Z O N E DMAX: Model R A 2 d TEMPER 0.483 0.810 PMio R A V G : Model R A2 d 5 - input 0.662 0.879 4 - input 0.659 0.878 3 - input 0.658 0.877 2 - input 0.653 0.875 PMio DMAX: Model R A 2 d 5-input 0.186 0.537 4 - input 0.193 0.540 3-input 0.188 0.535 2-input 0.173 0.521 Highlighted models are chosen for further comparison 54 4.3.2 Persistence In this study, a benchmark for model comparison is persistence—the previous day's daily maximum concentration is forecast to occur the next day: x<=x<-> The persistence forecast is a commonly used benchmark for evaluating forecast skill in nearly all atmospheric and environmental sciences (Robeson & Steyn, 1990; Kuligowski & Barros, 1998). The persistence forecast is generated only for the 181 observations in the test data set. In the next chapter, the best models for each methodology (neural network and linear regression) are extracted for further evaluation against the persistence forecast. The sections presented investigate the strengths and limitations of each methodology's ability to predict pollutant concentrations based on information from the 1998 test data set. 55 Chapter 5 Model Evaluation/Intercomparison After model development, separate experiments for Cb DMAX, PMio RAVG, and PMio DMAX are conducted on the trained networks and linear regression models. Daily pollutant forecasts are made for the 181 quasi-independent samples (from the training data) in the test data set. A number of quantitative measures are considered to gauge model performance and determine which of the methodologies (neural network, linear regression, or persistence) produce the most effective forecast model in each category. To complement the Index of Agreement (d) and Pearson's correlation coefficient (R2) from the previous section, an array of model evaluation statistics are calculated for comparative purposes, as recommended by Robeson & Steyn (1990), Comrie (1997), and Wilmott (1981,1982). Additional statistics reported in this study are: predicted and observed means and standard deviations (SD), intercept (a) and slope (b) of an ordinary least squares regression of predicted concentration on observed concentration, mean absolute error (MAE) and root mean squared error (RMSE), and systematic and unsystematic portions of RMSE (RMSEs and RMSEu) (Robeson & Steyn, 1990). The linear relationship: RMSE2 = RMSEs2 + RMSEu 2 56 allows detailed analysis of model error by quantifying the relative contributions of systematic (model-oriented) and unsystematic (data-oriented) components of the RMSE. The model comparisons presented below use the evaluation statistics to determine the feasibility of neural networks in daily operational Cb and PMio forecasting. All statistics reported below refer to the respective predicted (Pi) and observed (Oi) pollutant concentration data produced by the models. 5.1 Daily Maximum Ozone Models The full set of model evaluation statistics for the selected neural network (NN), linear regression (LR), and persistence (PER) models is presented in Table 9. The mean daily maximum ozone concentration ( Jo ) observed for the downtown Abbotsford station during the summer of 1998 was 35.143 ppb. Predicted ozone concentrations ( Po ) for the three models are very good, all falling within ±1 ppb of the observed value. The standard deviation of the observations is 12.952 ppb indicating significant variability in the O 3 time series. Predicted standard deviations range from 9.562 ppb for the LR model to 12.392 for the PER model. These values generally coincide with expectations as nearly all statistical models underestimate variance to some degree because they are designed to produce a generalized or "average" prediction for a specific set of inputs (Comrie, 1997). Certainly, Table 9 indicates that the NN and LR models are not able to capture the original variability of the O3 time series, however the NN model does estimate this variability better than the LR model (especially at peak values). 57 T A B L E 9: Evaluation statistics for daily maximum ozone models MODEL* 1 2 3 Observed Mean (ppb) 35.143 Observed Standard Deviation (ppb) - 12.952 -Predicted Mean (ppb) 34.720 34.540 35.385 Predicted Standard Deviation (ppb) 10.497 9.562 12.392 Regression Line Intercept 13.551 16.502 14.651 Slope 0.602 0.513 0.540 Mean Absolute Error 6.829 7.075 8.867 Root Mean Square Error 8.719 9.346 11.355 Systematic component 5.173 6.337 5.457 Unsystematic component 6.819 6.873 9.942 Correlation Coefficient 0.552 0.483 0.292 Index of Agreement 0.849 0.810 0.731 * Model 1 * Model 2 * Model 3 -- Neural Network (2-LS) --TEMPER Linear Regression ~ Persistence 58 A direct comparison of the neural network and linear regression models' ability to predict elevated ozone concentrations is visually represented in Figure 9. In an operational setting where accurate prediction of peak concentrations is vital, neural network models clearly capture the spikes in the time series better than the linear regression model. Therefore neural network techniques do deserve consideration for use in an operational setting based on their ability to estimate peak concentrations better than the conventional regression technique. The scatterplots provided in Figures 10-12 show the occurrences of predictions (abscissa) versus observations (ordinate) for the 181 test forecasts during 1998. In a perfect model, all points would run along the main diagonal indicating that the observed O 3 concentration is exactly the same as the forecast O 3 concentration. Examination of Figures 10-12 indicates significant scatter about the main diagonal in all three models. However the NN and LR models are both noticeably more constrained about the main diagonal than the PER model. This illustrates the value of including the daily maximum temperature variable as input to these models. Although similar performance is observed between the NN and LR models, again the NN visibly outperforms the LR in the prediction of high-ozone episodes therefore must be considered more valuable in an operational mode. Least squares regression estimates of predicted concentrations on observed concentrations are calculated for the three models to provide information on the systematic over- or under-prediction of the models (Table 9). Values of a range from 59 F i q u r e 9 : N e u r a l n e t w o r k and linear regression models predicted ozone concentrations versus target concentrations for April 15, 1998 to October 15 iyy8. ' 60 Figure 10: Scatterplot for neural network daily maxmium O 3 forecasts (2-LS model) 0 10 20 30 40 50 60 70 80 90 Target 03 Concentration (ppb) 61 Figure 1 1 : Scatterplot for linear regression daily maximum O 3 forecasts (TEMPER) 0 10 20 30 40 50 60 70 80 90 Target O3 Concentration (ppb) 62 Figure 12: Scatterplot for persistence daily maximum O 3 forecasts 63 13.551 ppb for the NN model to 16.502 ppb for the LR model with b values ranging from 0.602 for the NN model to 0.513 for the LR model. In all models, b is less than one and a greater than zero and a trend of higher slopes corresponding to lower intercepts. This indicates a tendency towards underestimation at high concentrations and overestimation at low concentrations (verifying the above findings on variability). In this study, error indices are based on the difference between individual predictions and observations (Pi - Oi) (Comrie, 1997). Model error is reported in Table 9 via the MAE (average forecast error between Pi and Oi) and RMSE (square root of the average of all differences between Pi and Ot). The MAE of models range from 6.829 ppb for the NN to 8.867 ppb for the PER model. The close proximity of these results indicates somewhat consistent performance of all models. However, the MAE provides only a crude indication of forecast error, which is not particularly sensitive to outlier predictions (Comrie, 1997). The RMSE provides a more accurate indication of model error. Values follow the same trend as MAE results (increasing error from NN to PER models), however they are noticeably higher than corresponding the MAE values due to the RMSE's sensitivity to outlying values. An additional advantage of the RMSE statistic is its useful decomposition into systematic (RMSEs) and unsystematic (RMSEu) components (Robeson & Steyn, 1990). Systematic error is designed to give an indication of differences in expected predictions and observations, thus it is often deemed "model-oriented" error. Conversely, unsystematic error provides information on the residuals between actual and expected 64 predictions, and therefore represents "data-oriented" error. Systematic and unsystematic portions of the RMSE are presented in Table 9. RMSEs values range from 5.173 ppb for the NN to 6.337 ppb for the LR model, and RMSEu values range from 6.819 ppb for the NN to 9.942 ppb for the PER model. Although the RMSEu values follow suit, the RMSEs results do not follow the same trends found in the MAE and RMSE results. Interestingly the PER model produces a lower systematic component than the LR model, indicating higher levels of bias in the LR model (however the PER model does produce a much higher RMSEu value than either the NN or LR models). A final comparison of model performance is made through examination of the previously described Pearson's correlation coefficient, R2, and Index of Agreement, d (Table 9). R2 values range from 0.552 for the NN model to 0.292 for the persistence forecast with d values following the same trend: 0.849 for the NN to 0.731 for persistence. Generally, strong results are obtained by the NN and LR methodologies with similar trends observed as from statistics discussed above. The NN model does show improvement over the traditional temperature-persistence based LR model, however the improvement is marginal. It is not clear if the difference in performance is significant enough to warrant use in an operational mode, therefore it would be premature to conclude that NN methodologies are superior to traditional approaches for operational daily maximum 03 forecasting. 65 5.2 D a i l y M a x i m u m PMio R o l l i n g A v e r a g e M o d e l s Table 10 provides model evaluation statistics for the neural network, linear regression, and persistence Daily Maximum PMio Rolling Average models developed in this study. The average daily maximum rolling average concentration ( %PMR ) of the 181 observations used in the test data set is 22.850 Ugm"3. Models produced similar mean concentrations ranging from 22.797 Ugm"3 for the PER model to 23.066 ugm"3 for the LR model. It must be noted that both NN and LR models produce slightly higher average concentrations than observed, which is evident from examination of the time series shown in Figure 13 (general tendency to over-predict at low-mid level concentrations). The standard deviation of observations found to be 10.782 |igm"3, again suggesting notable fluctuations in the observed time series. Predicted standard deviations for the LR and NN models are significantly lower, 8.069 Ugm"3 and 8.134 Ugm"3 respectively, than the PER model at 10.764 |lgm"3. This difference is expected, as the same general pattern of underestimating predicted standard deviation values is evident in each pollutant category. Again it is seen that the empirically derived models (NN and LR) are not able to adequately represent the variability of the pollutant time series (Figure 13). Values of the intercept (a) and slope (b) are obtained from a least squares regression of predicted concentrations on actual concentrations and presented in Table 10. Examination of the scatterplots in Figures 14-16 reveals that the PER model provides a much better least squares regression estimate than either the NN or LR models. In accordance with the lower standard deviations, NN and LR predictions are 66 T A B L E 10: Evaluation statistics for daily maximum PM10 rolling average models M O D E L * 1 2 3 Observed Mean (ugm-3) - 22.850 Observed Standard Deviation (ugm-3) - 10.782 Predicted Mean (ugm-3) 22.955 23.066 22.797 Predicted Standard Deviation (ugm-3) 8.134 8.069 10.764 Regression Line Intercept 8.987 9.155 5.223 Slope 0.611 0.609 0.769 Mean Absolute Error 4.292 4.292 4.852 Root Mean Square Error 6.346 6.313 7.300 Systematic component 4.192 4.224 2.490 Unsystematic component 4.765 4.693 6.862 Correlation Coefficient 0.657 0.662 0.594 Index of Agreement 0.878 0.879 0.874 * Model 1 --* Model 2 --* Model 3 --Neural Network (4-LS) Linear Regression (5 - input) Persistence 67 Neural Network Forecasts for PM10 Daily Maximum Rolling Average Concentrations (4 - LS model) 80 100 # of Days 120 140 -Target •Predicted 160 180 80 100 # of Days 120 140 160 180 Fiqure 13: Neural network and linear regression models predicted PM10 rolling average concentrations versus target concentrations for April 15 1998 to October 15, 1998. 68 Figure 14: Scatterplot for neural network daily maximum PM10 rolling average forecasts (4 - LS model) 69 Figure 15: Scatterplot for linear regression daily maximum PM10 rolling average forecasts (5 input model) 70 Figure 16: Scatterplot for persistence daily maximum PM10 roll ing average forecasts 71 much more constrained about the main diagonal; however, the NN and LR models both significantly under-predict high concentrations. This bias is not evident for PER model forecasts. This leads one to conclude that the NN or LR methodologies might be more appropriate for predicting low-to-mid level PMio rolling average concentrations, however persistence forecasts should outperform them at high concentrations. Consistent performance is observed between the three models, as the range of MAE and RMSE for all model predictions is rather small. Trends follow those of the standard deviation predictions where NN and LR models have slightly lower MAE and RMSE values than persistence forecasts. The decomposition of the RMSE into model- and data-oriented error illustrates the degree to which a model can be improved within the constraints of variability in the observed data (Comrie, 1997). Again, the PER model produces a significantly lower RMSEs than either the NN or LR models confirming the lack of significant bias in persistence forecasts. Consequently, it is believed that the NN and LR models lack sufficient information to generate accurate forecasts as they both exhibit -70% more systematic error than the persistence forecast. Perhaps the best quantitative measure of model performance in this study is provided by the Pearson's correlation coefficient, R2, and Index of Agreement, d. The R2 and d values presented in Table 10 illustrate some inconsistencies in the model evaluation statistics presented above, as both the NN and LR models produce higher R2 and d values than the PER model. However, the difference between these values is rather small, illustrating the importance of persistence information relative to the other 72 meteorological variables. The lack of any significant improvement by either the N N or L R models over a straight persistence forecast does not demonstrate applicability of these methodologies for operational daily maximum PMio rolling average concentration forecasting. 5.3 D a i l y M a x i m u m PMio Models Statistical models are developed to forecast daily 1-hour maximum PMio concentrations in Abbottsford, B C . In this section, the accuracy of each model's forecasts is evaluated through an array of model evaluation statistics presented in Table 11. For the 181 samples in test year 1998, the observed mean daily maximum PMio concentrations is 41.987 ugm"3. The persistence forecast (41.829 pigm"3) was the only model able to adequately reproduce this mean concentration, as both the N N and L R models produce significantly higher mean concentrations (~ 4 |igra"3). The observed standard deviation of 31.868 |igm"3 illustrates the extreme variability in the daily maximum PMio concentration time series (Figure 17). The N N and L R models produce much more conservative standard deviations at 16.455 ( igm 3 and 14.693 Ugm"3, respectively. This implies a fundamental lack of information in the N N and L R models, as they are generating only -50% of the variability in the observed time series. The scatterplots presented in Figures 18-20 illustrate the N N and L R models' inability to produce reasonable predictions for daily maximum PMio concentrations. The majority of forecasts is over-predicted (at low concentrations) and peak concentrations 73 T A B L E 11: Evaluation statistics for daily maximum PMio models M O D E L * 1 2 3 Observed Mean (ugm-3) 41.987 Observed Standard Deviation (ugm-3) - 31.868 -Predicted Mean (ugm-3) 45.586 46.002 41.829 Predicted Standard Deviation (ugm-3) 16.455 14.693 31.876 Regression Line Intercept 38.419 37.486 24.680 Slope 0.171 0.203 0.408 Mean Absolute Error 20.359 19.393 20.779 Root Mean Square Error 30.864 28.903 34.670 Systematic component 26.672 25.716 18.850 Unsystematic component 15.530 13.196 29.099 Correlation Coefficient 0.109 0.193 0.167 Index of Agreement 0.490 0.540 0.623 * Model 1 -* Model 2 -* Model 3 - Neural Network (2-LS)  Linear Regression (4 - input)  Persistence 74 F i q u r e 1 7 N e u r a l n e t w o r k and linear regression models predicted daily maximum PM10 concentrations versus target concentrations for April 15 1998 to October 15, 1998. 75 Figure 18: Scatterplot for neural network PM10 daily maximum forecasts (2-LS model) 76 Figure 19: Scatterplot for linear regression PM10 daily maximum forecasts (4 input model) Target PM10 Concentration (ugm3) 77 Figure 20: Scatterplot for persistence PM10 daily maximum forecasts 78 severely under-predicted, behavior certainly not desirable in an operational model. The time series in Figure 17 visually illustrates this behavior. A persistence forecast is an improvement over the NN and LR methodologies, however it still does not provide an adequate forecast. Examination of slope and intercept values in Table 11 indicate that all models have difficulty providing satisfactory least square regression estimates. For each model the MAE is -20 ugm"3, indicating rather large residuals between predicted and observed concentrations. RMSE results are substantially higher than the MAE, demonstrating the models' inability to accurately predict at higher concentrations (i.e. outliers). Furthermore, the NN and LR models exhibit high proportions of systematic (model-oriented) error illustrating that other factors are more important to the production of local, elevated PMio concentrations than the meteorological surrogates used in this study. The R2 and d values in Table 11 illustrate the rather poor performance of all three models. The persistence forecast certainly outperforms both the NN and LR models, but R2 values ranging from 0.109 - 0.190 and d value in the 0.490 - 0.623 indicate rather weak relationships between predicted and observed PMio concentrations in all models. Obviously, the meteorological variables used in this study do not provide an adequate model for explaining elevated PMio concentrations in downtown Abbottsford, BC. 79 5.3.1 Case Study of Daily Maximum PMio Concentrations In an attempt to explain the poor performance of the daily maximum PMio models, a case study of hourly PMio concentrations at Abbotsford is examined. Figure 17 illustrates several spikes in the PMio daily maximum time series, which neither the neural network or linear regression models are able to adequately reproduce. The hourly PMio concentration time series for the first spike in Figure 17 (~ day 20) is presented in Figure 21. Examination of this graph indicates that concentrations remain quite low through most of the day, however during a one to two hour time period around 11:00 PM short-lived, extreme PMio concentrations are observed. It is felt that local emissions and micro-meteorological factors might be governing these spikes in the PMio time series. To investigate the processes contributing to these elevated 1-2 hour PMio episodes, hourly time series of carbon monoxide (CO) and nitrogen oxide (NO) concentrations are examined during the same time period and presented in Figures 22 & 23. In the LFV, nearly all ambient CO (~ 97%) and NO x concentrations are a result of motor vehicle emissions (McKendry, 1999); furthermore the transportation sector accounts for the largest source of particulate matter (Monn et al. 1995). The CO and NO time series both exhibit similar patterns to the PMio time series with peak concentrations occurring within the same 1-2 hour time period. The corresponding spikes in the PMio, CO and NO time series, along with the proximity of the monitoring station to motor vehicle traffic (Figure 5a), provide solid evidence that localized anthropogenic emissions are partially responsible for these elevated concentrations. 80 Figure 21: Time Series of 1-Hour PM10 Concentrations May 1 -2,1998 180 -1 0 -I 1 1 1 1 1 1 1 1 5/1/98 5/1/98 5/1/98 5/1/98 5/2/98 5/2/98 5/2/98 5/2/98 5/3/98 0:00 6:00 12:00 18:00 0:00 6:00 12:00 18:00 0:00 Time (starting midnight May 1st) 81 Figure 22: Time Series of 1-Hour Carbon Monoxide Concentrations May 1 -2,1998 3000 0 "I 1 ! 1 , 1 , , 1 5/1/98 5/1/98 5/1/98 5/1/98 5/2/98 5/2/98 5/2/98 5/2/98 5/3/98 0:00 6:00 12:00 18:00 0:00 6:00 12:00 18:00 0:00 Time (starting midnight May 1st) 82 Figure 23: Time Series of 1-Hour Nitrogen Oxide Concentrations May 1 - 2 , 1 9 9 8 120 5/1/98 5/1/98 5/1/98 5/1/98 5/2/98 5/2/98 5/2/98 5/2/98 5/3/98 0:00 6:00 12:00 18:00 0:00 6:00 12:00 18:00 0:00 Time (starting midnight May 1st) 83 Examination of the MSL and 500 hPa synoptic classifications indicates anti-cyclonic conditions influencing the LFV on May 1 -2, 1998. In the summer months under high-pressure systems, Abbotsford generally experiences light winds complements of the sea breeze circulation. During the evening, the surface environment becomes rather calm due to a weakening sea breeze circulation and the onset of the nocturnal boundary layer. At night, a 1 - 2 hour transitional period occurs before the surface can emit enough radiation to allow the land breeze to develop. During this transitional period, the surface layer becomes quite stagnant; consequently little wind is available to disperse pollutants (Figure 24). Under these conditions, pollutants accumulate in the downtown core explaining the observed elevated PMio concentrations. Within an hour or two, the easterly land breeze develops transporting pollutants away from the urban core, and PMio concentrations return to background levels. Therefore, it is felt that in this urbanized setting these late evening PMio spikes are most likely a result of local anthropogenic emissions (i.e. - motor vehicle exhaust and road dust) in conjunction with reduced nighttime dispersion caused by the transition between the land-sea breeze circulation. 5.3.2 Modifications to the Daily Maximum PMio Models In hindsight, it was probably unreasonable to expect the methodologies described above to find any significant relationship between localized PMio concentrations in downtown Abbottsford and large-scale synoptic classifications or the parameterized 84 Figure 24:Time series of 1-hour averaged wind speed in Abbotsford May 1 - 2 , 1 9 9 8 5 40 45 85 transport term (which is based on wind measurements recorded 60 km away at the Vancouver International Airport). Recognizing that there was surely room for improvement in NN and LR model performance, new models are constructed to forecast the 1-hour daily maximum PMio concentration using more localized inputs. The synoptic classifications and transport term were replaced with daily wind speed and precipitation data, both recorded at the Abbotsford Airport. In addition, the day of week is included to capture the weekly cycle of local anthropogenic emissions. Persistence information and daily maximum temperature are retained as model input; therefore additional neural network and linear regression models were constructed using these input parameters: • Previous day's 1-hour maximum PMio concentration • Forecasted daily maximum temperature • Previous day's precipitation data • Forecasted wind speed (as a range) • Day of week Numerous NN and LR models were developed using different combinations of the new input parameters. However when comparative runs of the models were made, the performances of the new models provided no improvement on those previously described. It appears reasonable to assume that the formation mechanisms for elevated PMio concentrations are sufficiently complex and require detailed information dealing with the variety of sources influencing the region in question (e.g. secondary particle formation, influence of marine aerosols, fugitive dust from construction, roads, and agriculture). It is 86 seen that the meteorological parameters used in this study have little influence on PMio concentrations in downtown Abbotsford. Therefore it is assumed that the relationship between daily maximum PMio concentrations and meteorological variables contains a level of random variation which cannot be adequately represented by these empirically derived models. The poor performance demonstrated by both the neural network and linear regression techniques creates a strong argument for future work investigating other methods for daily maximum PMio prediction. 87 Chapter 6 Conclusions and Recommendations 6.1 Conclusions This study investigates the ability of neural networks to produce quantitative, daily forecasts of O3 and PMio in the lower Fraser Valley of British Columbia. Neural network, linear regression, and persistence models have been used to predict daily maximum ozone and PMio concentrations in Abbotsford, BC. Models are developed on data from 1994 - 1997 and tested on data from 1998 (a set of 181 observations not used in model development). The research produced mixed results. For daily maximum O3 prediction, the NN technique showed improvement over the linear regression and persistence models, however the significance of this improvement (with respect to operational use) is questionable. For daily PMio prediction, neither the neural network nor linear regression models demonstrated meaningful improvement over a simple persistence forecast illustrating the difficulties associated with short-term PMio forecasting. Certainly, the neural network models do not produce the level of results one might desire, but significant implications are drawn from characteristics evident across all prediction categories about the feasibility of NN modeling in an operational mode. Neural network performance on test data from all three categories varied from good (O3 DMAX) to poor (PMio DMAX). One common thread linking all networks developed for this study is the underestimation of peak pollutant concentrations. This is relatively common problem associated with the neural network modeling since they are specifically designed to generalize functions. During model development, networks are provided with a set of training cases that implicitly represent a sampling of some continuous function (Hewitson and Crane, 1994). Networks generalize this function by creating a specific input-output mapping function that is defined solely by the training data. Therefore, the training set must contain an adequate number of extreme events to allow the mapping function sufficient representation. If this is not the case, when the conditions that produce peak concentrations are presented to the network they are seen as anomalous values falling outside the realm of the network's mapping function; hence forecast ability is hampered. This demonstrates one of the most common problems associated with statistical forecasting; the necessity for data used in model development to accurately represent behavior of the system. The network used for daily maximum ozone forecasting certainly reproduces the observed time series reasonably well with temperature and persistence information. Conversely, the extreme variability found in the daily maximum PMio time series does not allow accurate representation with the meteorological variables used in this study. It would be interesting to repeat this investigation with a much longer daily maximum PMio time series, which might allow networks to generalize relationships between variables more effectively. However, it is felt that a more detailed understanding 89 of the processes that lead to elevated PMio concentrations is necessary before empirically derived models may be used successfully for operational PMio forecasting. Another thread common to models in all prediction categories is the sensitivity to the role of persistence in aiding forecast ability. The significance of persistence information is particularly evident in models forecasting the daily maximum PMio rolling average concentration. The marginal improvement made by incorporating meteorological variables in the NN and LR models illustrates the dominance of the persistence variable as a predictor. However, this is not terribly surprising after considering the calculation of a rolling average concentration (Appendix 1). The rolling average is, by definition, an average of the previous 24 one-hour PMio concentrations. Consequently, the averaging process dampens any large fluctuations in the 1-hour time series. This results in gradual changes in the daily maximum PMio rolling average concentration with time. The luxury of using the rolling average statistic to monitor PMio levels now becomes evident. As previously mentioned, we are not able to adequately represent the inherent complexities associated with elevated PMio concentrations numerically. Therefore, it is assumed that attempts to forecast PMio using the traditional time series approach (mathematical decomposition of the time series) have produced similar unsatisfactory results. This poses a prediction problem for regulatory agencies, most of which have recently recognized PMio as a priority pollutant. However, when PMio levels are reported using a rolling average concentration, the statistic's inherent reliance on persistence information provides a strong indication (d = 0.874) of what the next day's 90 maximum 1-hour rolling average concentration should be. Consequently, relatively accurate forecasts may be estimated without requiring a detailed understanding the controlling factors that lead to the accumulation of ambient particulate matter. The last common thread among all models derived in this study is the lack of significant information provided by synoptic classifications. This is particularly evident in models predicting daily maximum ozone concentrations. Table 4 illustrates the NN models decrease in forecast accuracy with inclusion of synoptic information as inputs. In hindsight, this is not a surprising result considering that models are trying to forecast pollutant concentrations at one site in the LFV based on classifications of synoptic systems over the entire LFV. Consequently, pollutant concentrations are assumed influenced more by local processes than large-scale atmospheric features. Since networks are specifically designed to generalize, it is felt that NNs have potential use for investigating relationships between large-scale circulation and less variable pollutant time series. Therefore, it would be interesting to further this research by investigating the NN's ability to translate synoptic features to spatially averaged pollutant concentrations in the LFV. 91 6.2 Recommendations for Future Work 6.2.1 Ozone Results of this study indicate that neural networks are more effective at matching the day-to-day changes in daily maximum ozone concentrations than the presently used linear regression model. Trial results of NNs with different combinations of input parameters confirm Robeson & Steyn's (1990) findings that a significant amount of the variability in the O 3 time series may be explained using temperature and persistence information. The improved predictions by the neural network model illustrate that this relationship definitely contains a non-linear component. But, the NN model only produces R2 and d values of 0.552 and 0.849, which still leaves substantial room for improvement. It is felt that the limits of the logistic-sigmoid transfer function to capture non-linearity between temperature and persistence information have been attained by this study. However, this begs the obvious question: What if the non-linearity in the relationship between daily maximum temperature and persistence is better represented by another transfer function? Consequently, it is recommended that a number of transfer functions be explored to establish the best representation of the relationship between daily maximum temperature and persistence. Use of a different transfer function might demonstrate the significant improvement necessary to warrant use of neural networks for daily maximum ozone forecasting in the LFV. 92 6.2.2 PMio This study certainly demonstrates the problems associated with modeling ambient PMio concentrations in a coastal environment. This is in large part due to difficulty quantifying the variety of influences that govern the formation of elevated PMio concentrations. To circumvent the problems associated with PMio modeling, it is recommended that regulatory agencies begin full-scale monitoring of PM2.5. Continuous monitoring of PM2.5 would be very beneficial from a modeling perspective since it would provide a pollutant time series that has narrower range of processes influencing concentrations. Recent studies of particulate matter in the LFV (McKendry, 1999 and Li, 1999) indicate the finer fraction (PM2.5) contains a much higher portion of toxic metals and acidic sulfur species than coarser fraction (PMio). This provides solid evidence for anthropogenic production of the majority of particulates found in the PM2.5 size range. Therefore if long enough time series was developed, it is believed that stronger relationships could be established between observed PM2.5 concentrations and external variables due to an increased understanding of the factors controlling the variability in the time series. This would allow researchers to focus input selection based on one or two source parameters (anthropogenically based) rather than the five to six sources influencing PMio concentrations. 93 6.3 Summary This research illustrates the strengths and limitations of neural network based modeling for operational O 3 and PMio prediction in the L F V of British Columbia. A direct comparison between the neural network and linear regression methodologies provides evidence for neural networks' use for operational O 3 prediction based on their ability to estimate peak concentrations better than the presently employed regression model (TEMPER) . The results of this study are consistent with other studies investigating neural networks' potential for air quality forecasting (Comrie, 1997; Boznar et al, 1993, Dorling & Gardner, 1999) in that neural network models demonstrate improved performance over traditional regression techniques. But neural nets do not dramatically improve models' overall forecast ability, therefore it may be argued that relatively simple, familiar regression models are a more practical choice in an operational setting. However, advances in neural network software technology (e.g. - Statsitica, Matlab) provide the user with an excellent interface allowing a very clear, straightforward presentation of the modeling technique. Therefore, due to their relative ease of use and improved predictive power at critical peak concentrations, neural network based models are recommended for operational O 3 prediction in Abbotsford, B C . 94 References Abareshi, B. and Scheupp, P.H. 1998: Sensible heat flux estimation over the FIFE site by neural networks. American Meteorological Society, 55, 1185-1197. Banta, R.M. , L.D. Oliver, C.-J. Zhu, K.G. Anlauf, H.A. Wiebe, A.J. Gallant, J.W. Bottenheim, D.G. Steyn, I.G. McKendry, J.Pottier, T. Biesenthal, and P.B. Shepson, 1997: Nocturnal cleansing flows in a tributary valley. Atmos.Env. 31 (4), 22147-2162. Boznar, M . , Lesjak, M . and Mlakar, P. 1993: A neural network-based method for short-term predictions of ambient SO2 concentrations in highly polluted industrial areas of complex terrain. Atmospheric Environment, 27B, 221-230. Burrows, W.R., Benjamin, M . , Beauchamp, S., Lord, E.R., McCollor, D. and Thomson, B. 1995: CART decision-tree statistical analysis and prediction of summer season maximum surface ozone for the Vancouver, Montreal and Atlantic regions of Canada. J. Applied Meteorology, 34, 1848-1862. City of Abbotsford Web Page. 6 August 1999. <> Comrie, A .C. 1997: Comparing neural network and regression models for ozone forecasting. J. Air & Waste Manage. Assoc. 47: 653-663. Comrie, A .C. and Yarnal, B. 1992: Relationships between synoptic-scale atmospheric circulation and ozone concentrations in metropolitan Pittsburgh, Pennsylvania. Atmospheric Environment, 26B, 301-312. Cybenko, G. 1998: Continuously valued neural networks with two hidden layers are sufficient. Medford, MA, Dept. of Computer Sci., Tufts University. Davies, D.L. and Bouldin, D.W. 1979: A cluster separation measure. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-1 (2), 224-227. 95 Dockery D.W., Pope C.A. , X u , X. , Spengler, J.D., Ware J.H. , Fay M.E. , Ferris B .G . , and Spiezer F .E. 1993: A n association between air pollution and mortality in six U.S. cities. New Engl. J. Med. 329, 1753-1759. Dorling, S.R. and Gardner, M.W. (In Press A): Neural network modelling and prediction of hourly NOx and NO2 concentrations in urban air in London. Accepted for publication in Atmospheric Environment. Greater Vancouver Regional District (GVRD) , 1994: Ambient Air Quality Annual Report. Greater Vancouver Regional District (GVRD) , 1999: Ambient Air Quality Annual Report. Greater Vancouver Regional District Website. 6 August 1999. <> Gould, P. 1994: Neural computing and the AIDS pandemic: the case of Ohio. B.C. Hewtison and R.G. Crane (eds.), Neural Nets: Applications in Geography, Kluwer Academic Publishers, Boston, 101-119. Hammerstrom D. 1993. Neural Networks at Work. IEEE Spectrum. Hertz, J . , Krogh A . , and R.G. Palmer 1991. Introduction to the theory of neural computation. Addision-Wesley, 327. Hewitson, B. and Crane, R. 1992: Regional-scale prediction from the GISS G C M . Paleogeography, Paleoclimatology, Paleoecology (Global Planetary Change Section), 97, 249-267. Hewitson, B. and Crane, R. 1994a. Neural Nets: Applications in Geography. Kluwer Academic Publishers, Boston, 1-52. Hewitson, B. and Crane, R. 1994b: Precipitation controls in southern Mexico. B.C. Hewtison and R.G. Crane (eds.), Neural Nets: Applications in Geography, Kluwer Academic Publishers, Boston, 121-143. Hsieh, W.W. and Tang, B. 1998: Applying neural network models to prediction and analysis in meteorology and oceanography. Bulletin of the American Meteorological Society, 79 (9), 1855-1870. 96 Joe, H., Steyn, D.G. and Susko, E. 1996: Analysis of trends in tropospheric ozone in the lower Fraser Valley, British Columbia. Atmospheric Environment, 30 (20), 3413-3421. Katsavounidis, I., Kuo, C.J. and Zhang, Z. 1994: A new initialization technique for generalized Lloyd iteration. IEEE Signal Processing Letters, 1 (10), 144-145. Kuligowski, R.J. and Barros, A.P. 1998: Experiments in short-term precipitation forecasting using artificial neural networks. Mon. Weather Review, 126, 470-482. Li , K.H. 1998: Spatial and temporal analysis of ambient hourly PMio in Vancouver. Unpublished M.S. thesis, Dept. of Statistics, University of British Columbia, Vancouver, Canada, 87 pp. Linde, Y., Buzo, A. and Gray R.M. 1980: An algorithm for Vector Quantizer Design. IEEE Transactions on Communications, 28 (1), 84-90. Mathai, C.V. 1990: Visibility and Fine Particles. A summary of the A&WMA/EPA International Specialty Conference, J. Air & Waste Manage. Assoc., 40 (11), 1486-1494. McKendry, LG. 1993: Surface ozone in Montreal, Canada. Atmospheric Environment. 27B, 93-103. McKendry, LG. 1994: Synoptic circulation and summertime ground-level ozone concentrations at Vancouver, British Columbia. /. Appl. Meteor., 33, 627-641. McKendry, I.G., D.G. Steyn, R.M. Banta, W. Strapp, K. Anlauf, and J.L. Pottier, 1998: Daytime photochemical pollutant transport over a tributary valley lake in Southwestern British Columbia. J. Appl. Meteor., 37 (4): 393-404. McKendry, I.G. (In Press): PMio levels in the lower Fraser Valley, BC, Canada: An overview of spatio-temporal variations and meteorological controls. Accepted for publication in Journal of the Air & Waste Management Association. McGinnis, D.L. 1994: Predicting snowfall from synoptic circulation: a comparison of linear regression and neural network methodologies. B.C. Hewtison and R.G. 97 Crane (eds.), Neural Nets: Applications in Geography, Kluwer Academic Publishers, Boston, 77-99. Monn, CH., Braendli, O., Schaeppi, G., Schindler, CH., Ackermann-Liebrich, U., Leuenberger, PH., and Sapaldia Team 1995: Particulate matter < 10 um and total suspended particulates (TSP) in urban, rural, and alpine air in Switzerland. Atmospheric Environment, 29, 2565-2573. Natural Resources Defense Council Website. 15 October 1998. <http ://w w w. nrdc. org/nrdc/> Overland^  J.E. and Preisendorfer, R.W. 1982: A significance test for principal components applied to a cyclone climatology. Mon. Weather Review, 110, 1-4. Patterson, D.W. 1996. Artificial Neural Networks, Theory and Application. Prentice Hall. Pryor, S.C, and Barthelmie, RJ. 1996: PMio in Canada. The Science of the Total Environment, 177, 57-71. Pryor, S.C, McKendry, I.G., and Steyn, D.G. 1994: Synoptic-Scale meteorological variability and surface ozone concentrations in Vancouver, British Columbia. J. Appl. Meteor., 34, 1823-1833. Robeson, S.M. and Steyn D.G. 1989: A conditional probability density function for forecasting ozone air quality data. Atmospheric Environment, 23, 689-692. Robeson, S.M. and Steyn D.G. 1990: Evaluation and comparison of statistical forecast models for daily maximum ozone concentrations. Atmospheric Environment, 24B, 303-312. Rumelhart D.E., and McCelland, J.L. 1986: Parallel Distributed Processing PDP: Explorations in the Microstructure of Cognition, 1, MIT Press, Cambridge, MA. Schwartz, J. and Dockery, D.W. 1992a: Particulate air pollution and daily mortality in Steubenville, Ohio. American Journal of Epidemiology, 135, 12-19. 98 Schwartz, J. and Dockery, D.W. 1992b: Increased mortality in Philadelphia associated with daily air pollution concentrations. American Review of Respiratory Disease, 145, 600-604. Schwartz, J. and Marcus, A. 1990: Mortality and air pollution in London: A time series analysis. American Journal of Epidemiology, 131, 185-194. Seaton A., MacNee W., Donaldson K., and Godden D. 1995: Particulate air pollution and acute health effects. Lancet. 345, 176-178. Seinfeld, J.H. 1989: Urban air pollution: state of the science. Science, 243, 745-752. Statistica: Neural Networks (Quick Reference) 1998: StatSoft, Inc. Tulsa, Oklahoma, 1-28. Stern A.C., Boubel R.W., Turner D.B., and Fox D.L. 1984. Fundamentals of Air Pollution: Second Edition, Academic Press, Inc., Florida. 110-111. Steyn, D.G. and Faulkner, D.A. 1986: The climatology of sea-breezes in the Lower Fraser Valley. B. C. Clim. Bull, 20 (3), 21-39. Steyn, D.G. and McKendry, I.G. 1988: Quantitative and Qualitative Evaluation of a Three-Dimensional Mesoscale Numerical Model Simulation of a Sea Breeze in Complex Terrian. Monthly Weather Review. 116 (10), 1914-26. Steyn, D.G. and Oke T.R. 1982: The depth of the daytime mixed layer two coastal sites:a model and its validation. Boundary-Layer Met. 24, 161-180. Vedal, S. 1995. Health effects of inhalable particles: implications for British Columbia. Vancouver Hospital and Health Sciences Center for BC Environment. Willmott, C.J. 1981: On the validation of models. Physical Geography, 2, 184-194. Willmott, C.J. 1982: Some comments on the evaluation of model performance. Bulletin of American Meteorological Society, 62, 599-609. Willmott, C.J., Ackelson, S.G., Davis, R.E., Feddema, J.J., Klink, K.M., Legates, D.R., O'Donnell, J., and Rowe, CM. 1985: Statistics on the evaluation and comparison of models. Journal of Geophysical Research, 90 (C5), 8995-9005. 99 Yarnel, B. 1993. Synoptic Climatology in Environmental Analysis, Bell Haven Press, Florida and London. 100 Appendix 1 Rolling Average Sample Calculation 101 R o l l i n g A v e r a g e S a m p l e C a l c u l a t i o n 24-Jun 25-Jun Hours 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0 1 2 3 4 5 6 7 8 9 10 Average past 24 24.37077 PMio Concentration 31.12 36.68 41.04 42.24 38 33.84 20.68 — • 31.92 40.04 -4-28.28 15.36 19.24 21.56 29.64 29.04 23.6 19.84 15.76 16.36 15.44 17.16 20.96 25.92 25.52 25.56 24.84 25.08 27.84 27.2 27.84 28.16 28.48 • 23 14.84 <4— 23.04 25.84 Average past 24 23.71385 Rolling Average @ t=7 hrs Rolling Average @ t = 8 hrs 102 Appendix 2 Neural Network Architectures for All Prediction Catego Network Architecture for Daily Maximum Ozone Prediction Inputs: 5 Architecture: 5 -5-5-1 R2 = 0.501 d = 0.831 Previous Day's Ozone Concentration Daily Maximum Temperature Transport Term 500 hPa Classification M S L Classification Ozone Prediction 104 Network Architecture for Daily Maximum Ozone Prediction Inputs: 4 Architecture: 4 -4-3-1 R 2 = 0.508 d = 0.834 105 Network Architecture for Daily Maximum Ozone Prediction Inputs: 3 Architecture: 3 -4-2-1 R2 = 0.518 d = 0.836 Previous Day's Ozone Concentration Daily Maximum Temperature Transport Term Ozone Prediction 106 Network Architecture for Daily Maximum Ozone Prediction Inputs: 2 Architecture: 2-14-7-1 R2 = 0.552 d = 0.849 Previous Day's Ozone Concentration Daily Maximum Temperature Ozone Prediction 107 Network Architecture for Daily Maximum PMio Rolling Average Prediction Inputs: 5 Architecture: 5-2-2-1 R2 = 0.645 d = 0.875 Previous Day's Daily Maximum PMIO Rolling Average Concentration Daily Maximum Temperature Transport Term 500 hPa Classification M S L Classification Daily Maximum — 0 PM10 Rolling Average Prediction 108 Network Architecture for Daily Maximum PMio Rolling Average Prediction Inputs: 4 Architecture: 4 - 3 - 3 - 1 R 2 = 0.657 d = 0.878 Previous Day's Daily Maximum 0 P M 1 ° Rolling Average Prediction 109 Network Architecture for Daily Maximum PMio Rolling Average Prediction Inputs: 3 Architecture: 3 -5-2-1 R2 = 0.637 d = 0.870 Previous Day's Daily Maximum PMIO Rolling Average Concentration Daily Maximum Temperature Transport Term Daily Maximum PMio Rolling Average Prediction 110 Network Architecture for Daily Maximum PMio Rolling Average Prediction Inputs: 2 Architecture: 2-11-11-1 R2 = 0.652 d = 0.877 I l l Network Architecture for Daily Maximum PMio Prediction Inputs: 5 Architecture: 5-13-13-1 R2 = 0.100 d = 0.431 Previous Day's Daily Maximum PMIO Concentration Daily Maximum Temperature Transport Term 500 hPa Classification M S L Classification Daily Maximum PM10 Prediction 112 Network Architecture for Daily Maximum PMio Prediction Inputs: 4 Architecture: 4-8-18-1 R2 = 0.181 d = 0.481 Previous Day's Daily Maximum PMIO Concentration Daily Maximum Temperature Transport Term Daily Maximum PMIO Prediction 113 Network Architecture for Daily Maximum PMio Prediction Inputs: 3 Architecture: 3 -3-3-1 R2 = 0.136 d = 0.468 114 Network Architecture for Daily Maximum P M i o Prediction Inputs: 2 Architecture: 2-13-6-1 R2 = 0.190 d = 0.490 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items