Science, Faculty of
Earth, Ocean and Atmospheric Sciences, Department of
DSpace
UBCV
Tangang, Fredolin T.
Doctor of Philosophy - PhD
University of British Columbia
Neural network models were used to seasonally forecast the tropical Pacific sea surface temperature anomalies (SSTA) in the Nino 3.4 region (6°S - 6°N, 120°W - 170°W). The inputs to the neural networks (i.e. the predictors) were the first seven wind stress empirical orthogonal function (EOF) modes of the tropical Pacific (20°S - 20°N, 120°E - 70°W) for four seasons and the Nino 3.4 SSTA persistence for the final season. The period 1952-1981 was used for training the neural network models, and the period 1982-1992 for forecast validation. At 6-month lead time, neural networks attained forecast skills comparable to the other El Nino - Southern Oscillation (ENSO) models. The results suggested that neural network models were viable for ENSO forecasting even at longer lead times of 9 to 12 months. It appeared that at these longer leads, the underlying relationship between the wind stress and Nino 3.4 SSTA became increasingly nonlinear. Two types of neural network models were further compared for forecasting the SSTA over several standard equatorial Pacific regions (Nino 3, 3.4, 3.5, 4, P2, P4 and P5). The first type used the sea level pressure (SLP) as predictor, while the second one used the wind stress. By ensemble averaging over 20 forecasts with random initial weights, the resulting forecasts were much less noisy than those in the earlier models. The wind stress models had better forecast skill at short lead times, while the SLP models generally had better skill at lead times of 6 months or longer. The western-central regions of the equatorial Pacific Ocean were best forecasted, with the Nino 4 region (6°S - 6°N, 160°E - 150°W) having the highest skiU, foUowed by Nino 3.4 and 3.5 (10°S - 6°N, 120°W - 180°), then Nino 3 (6°S - 6°N, 90° W - 150°W). The eastern boundary regions P4 (0 - 10°N, 80°W - 100°W) and P5 (10°S - 0, 80°W - 100° W) had much lower skill, while the western boundary region P2 (10°S - 10°N, 140°E - 180°) had no forecast skill. In an attempt to understand the inner working of the models, smaller networks were constructed with the extended empirical orthogonal functions (EEOF) of the SLP field as inputs. These smaller networks delivered forecasting skills similar to those of earlier models. By network pruning and spectral analyses, four important inputs were identified: modes 1, 2 and 6 of the SLP EEOFs and the SSTA persistence. Mode 1 characterized the low frequency oscillation (LFO, with 4 - 5 years period), and was interpreted as the typical ENSO signal, while mode 2, with a period of 2 - 5 years, appeared to characterize the quasi-biennial oscillation (QBO) plus the LFO. Mode 6 was dominated by decadal and interdecadal variations. Thus, forecasting ENSO seems to require information from the QBO, and the decadal-interdecadal oscillations. The nonlinearity of the networks tended to increase with lead time, and to become stronger for the eastern regions of the equatorial Pacific Ocean.
8401603 bytes
application/pdf
F O R E C A S T I N G E L NINO - S O U T H E R N OSCILLATION (ENSO) E V E N T S : A N E U R A L N E T W O R K A P P R O A C H by Fredolin T. Tangang B. Sc. (Mathematics) University of Malaya, 1985 M . Sc. (Statistics) Iowa State University, 1988 A T H E S I S S U B M I T T E D IN P A R T I A L F U L F I L L M E N T O F T H E R E Q U I R E M E N T S F O R T H E D E G R E E O F D O C T O R O F P H I L O S O P H Y in T H E F A C U L T Y O F G R A D U A T E S T U D I E S E A R T H A N D O C E A N S C I E N C E S We accept this thesis as conforming to the required standard T H E U N I V E R S I T Y O F B R I T I S H C O L U M B I A July 1997 © Fredolin T. Tangang, 1997 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Earth and Ocean Sciences The University of British Columbia 2075 Wesbrook Place Vancouver, Canada V6T 1Z1 Date: Abstract Neural network models were used to seasonally forecast the tropical Pacific sea surface temperature anomalies (SSTA) in the Nino 3.4 region (6°S - 6°N, 120°W - 170°W). The inputs to the neural networks (i.e. the predictors) were the first seven wind stress empirical orthogonal function (EOF) modes of the tropical Pacific (20°S - 20°N, 120°E -70°W) for four seasons and the Nino 3.4 SSTA persistence for the final season. The period 1952-1981 was used for training the neural network models, and the period 1982-1992 for forecast validation. At 6-month lead time, neural networks attained forecast skills comparable to the other E l Nino - Southern Oscillation (ENSO) models. The results suggested that neural network models were viable for ENSO forecasting even at longer lead times of 9 to 12 months. It appeared that at these longer leads, the underlying relationship between the wind stress and Nino 3.4 SSTA became increasingly nonlinear. Two types of neural network models were further compared for forecasting the SSTA over several standard equatorial Pacific regions (Nino 3, 3.4, 3.5, 4, P2, P4 and P5). The first type used the sea level pressure (SLP) as predictor, while the second one used the wind stress. By ensemble averaging over 20 forecasts with random initial weights, the resulting forecasts were much less noisy than those in the earlier models. The wind stress models had better forecast skill at short lead times, while the SLP models generally had better skill at lead times of 6 months or longer. The western-central regions of the equatorial Pacific Ocean were best forecasted, with the Nino 4 region (6°S - 6°N, 160°E - 150°W) having the highest skiU, foUowed by Nino 3.4 and 3.5 (10°S - 6°N, 120°W -180°), then Nino 3 (6°S - 6°N, 90° W - 150°W). The eastern boundary regions P4 (0 -10°N, 80°W - 100°W) and P5 (10°S - 0, 80°W - 100° W) had much lower skill, while the u western boundary region P2 (10°S - 10°N, 140°E - 180°) had no forecast skill. In an attempt to understand the inner working of the models, smaller networks were constructed with the extended empirical orthogonal functions (EEOF) of the SLP field as inputs. These smaller networks delivered forecasting skills similar to those of earlier models. By network pruning and spectral analyses, four important inputs were identified: modes 1, 2 and 6 of the SLP EEOFs and the SSTA persistence. Mode 1 characterized the low frequency oscillation (LFO, with 4 - 5 years period), and was interpreted as the typical ENSO signal, while mode 2, with a period of 2 - 5 years, appeared to characterize the quasi-biennial oscillation (QBO) plus the L F O . Mode 6 was dominated by decadal and interdecadal variations. Thus, forecasting ENSO seems to require information from the QBO, and the decadal-interdecadal oscillations. The nonlinearity of the networks tended to increase with lead time, and to become stronger for the eastern regions of the equatorial Pacific Ocean. in Table of Contents Abstract ii List of Tables vii List of Figures viii Acknowledgements xv 1 Introduction 1 1.1 ENSO Phenomenon 1 1.2 ENSO Models 2 1.3 Motivation 4 2 Neural network models 6 2.1 Introduction 6 2.2 Multi-layer feedforward neural networks 6 2.3 Training with backpropagation method 9 2.4 Overfitting and generalization 11 3 Forecasting the Nino 3.4 SSTA 13 3.1 Introduction 13 3.2 Data and preliminary analysis 14 3.3 Model designs 18 3.4 Results 20 iv 3.4.1 Forecasts with neural network models 20 3.4.2 Comparison with a stepwise multiple linear regression model . . . 29 3.4.3 The effects of using wind stress of different latitudinal extent . . . 33 3.4.4 Seasonality of the skills 38 3.5 Sensitivity Analyses 39 3.6 SST as predictor 45 3.7 Summary and conclusion 48 4 Forecasting regional SSTA in the equatorial Pacific 51 4.1 Introduction 51 4.2 The data and preliminary analyses 51 4.3 Model Designs 60 4.4 Results 61 4.4.1 Retroactive forecasts 61 4.4.2 Cross-validated forecasts 75 4.5 Summary and conclusion 83 5 Forecasting regional SSTA: Neural network - extended E O F approach 89 5.1 Introduction 89 5.2 Preliminary data processing 90 5.3 Model design 95 5.4 Neural network forecasts 99 5.4.1 Forecasting SSTA of individual regions 99 5.4.2 Forecasting EOFs of SSTA 103 5.5 Network pruning analysis 114 5.6 Spectral analysis of the trained network 120 5.7 Summary and conclusion 138 v 6 Overall conclusion and summary 140 6.1 Neural network models and forecast results 140 6.2 Operational format, and future research 143 Appendices 152 A The E O F and E E O F Analyses 152 v i List of Tables 4.1 The percentage of variation explained by the first 7 E O F modes 54 5.1 The weights, w, and biases, b, of the hidden layer, for the network fore-casting Nino 4 SSTA at 6-month lead 117 5.2 The pruning statistics for forecasting Nino 4 SSTA for the 1982-93 period at 6-month lead. The values shown are the correlation and RMS between the outputs of the hidden neurons (NI, N2, and N3) of the pruned network and the corresponding outputs of the unpruned network. The correlation and RMS values between the pruned network outputs and the correspond-ing unpruned outputs are given in the last two columns 118 5.3 The weights, w, and biases, b, in several neural network models for forecast-ing the SSTA in various regions at 0, 6 and 12 months leads (the weights wi, 1V2, W3 and W4 correspond to modes 1, 2, 6 of the SLP E E O F and the SSTA persistence, respectively). The models were all trained with data from 1952-1981. Region P4 is at the eastern boundary (0 - 10°N, 80°W -100°W) 132 5.4 The ratio (in percentage) of the power from nonlinear interactions to the total power for various regions at 0, 6 and 12 months leads. The models were all trained with data from 1952-1981 134 V l l List of Figures 2.1 An example of a feedforward neural network model, where there are 4 neurons in the input layer, 3 in the hidden layer, and 1 in the output layer. The wij and Wj are the weights and the bj and b axe the biases. The bj and the b can be regarded as the weights for constant inputs of magnitude 1 8 3.1 First E O F mode of wind stress field. The dashed box centered around the equator indicates the Nino 3.4 region, in which the SSTA is forecasted. The amplitude has been divided by the standard deviation 16 3.2 Second E O F mode of wind stress field. That this mode is strongly related to ENSO can be seen by its amplitude, which rises sharply during E l Nino years 17 3.3 Schematic diagram showing the timing of the predictor-predict and periods at 6-month lead forecasting the onset (Nov(-l) - Jan(O)), peak (Mar(O) - May(O)) and mature (Dec(O) - Feb(+1)) phases of an E l Nino event. The year 0 refers to the E l Nino years (in which anomalous warming first appears and then amplifies in the tropical Pacific); year -1 and +1 refer to the previous and subsequent years, respectively. The predictors are enclosed by rectangular boxes, and the predictands by oval boxes 20 vi i i 3.4 (a) Correlation and (b) root-mean-square (rmse) between neural network model forecasts (individual and ensemble), and observation for the Nino 3.4 SSTA during 1982 - 1992 for various lead times. Persistence is shown as the dot-dash curve. The rmse was calculated for standardized values (i.e. values inflated to zero mean and normalized to their standard deviations). 21 3.4 (Cont.) 22 3.5 A comparison of correlation skills of the neural network model for the Nino 3.4 SSTA during the training period (1952-1981) and the forecast validation period (1982-1992) at various lead times 24 3.6 Standardized SST anomalies of the neural net forecasts (dash-dotted curves for individual forecasts, dotted curves for the ensemble forecasts) of the Nino 3.4 SSTA compared with the observed anomalies (solid curves) during 1982-1992 at lead times of a 3, b 6, c 9, d 12 months 26 3.6 (Cont.) 27 3.7 A comparison of correlation skills between the neural network model and the multiple linear regression model during (a) training period (1952-1981) and (b) forecasting period (1982-1992), for various lead times 30 3.7 (Cont.) 31 3.8 Standardized SST anomalies of the neural network hindcasts (dashed curve) and the multiple linear regression hindcasts (dotted curves) for the training data at 12-month lead 34 3.9 • A comparison of forecast correlation skills between neural network models using wind stress of 20°S-20°N, 10°S-10°N and 6°S-6°N for the forecasting period (1982-1992), at various lead times 37 3.10 The forecast correlation skills for various predictand seasons at 0, 3, 6, 9, 12 and 15-month leads 40 IX 3.11 The forecast correlation skills at 0, 3, 6, 9, 12-month leads, as the number of neurons in the hidden layer varied from 2 to 5 42 3.12 The forecast correlation skills at 0, 3, 6, 12-month leads for various dura-tion of wind stress data as predictors 43 3.13 The forecast correlation skills at 0, 3, 6, 9, 12-month leads, as the number of input wind stress EOFs varied from 4 to 7 46 3.14 Nino 3.4 SSTA forecast correlation skills at 0, 3, 6, 9, 12-month leads for neural network models with different sets of predictors 47 3.15 The standardized SST anomalies of observations (solid curve) versus fore-casts at 0-month lead for a model using SST EOFs (dashed curve) and a model using wind stress EOFs (dotted curve) as predictors 49 4.1 The location of the seven regions of interest 53 4.2 First E O F mode of the wind stress field. The temporal amplitude shows a decadal trend plus ENSO signals. The vectors correspond to the cor-relations between the temporal amplitude and the zonal and meridional components of the wind stress anomaly field. Both the spatial and tempo-ral patterns are similar to Fig. 3.1, with the exception of larger magnitude of the vectors and enhanced westerlies in the western-central equatorial Pacific 55 4.3 Second E O F mode of the wind stress field. Both the spatial and temporal patterns are similar to Fig. 3.2, with the exception of stronger westerlies. 56 4.4 First E O F mode of the SLP field. Both the spatial and temporal patterns indicate ENSO pattern. The contours represent the correlations between the temporal amplitude and the SLP anomaly field at each grid point. . . 58 x 4.5 Second E O F mode of the SLP field. The spatial pattern indicates a low negative center in the central tropical Pacific between 0 - 15°N. The am-plitude shows no clear association with ENSO events 59 4.6 The mean correlation skills of retroactive forecasts (1982 - 1993) for the SSTA of each region in Fig. 4.1, using SLP as predictor. The vertical lines indicate ± 1 standard deviation. No skill was found for the P2 region, hence it is not included 63 4.7 As in Fig. 4.6, but for the wind stress as predictor 64 4.8 Comparison between the forecast skills of the SLP and wind stress models for the Nino 4 region. Persistence correlations were also plotted, but with negative values omitted 66 4.9 As in Fig. 4.8, but for the Nino 3.5 region 67 4.10 As in Fig. 4.8, but for the P4 region 68 4.11 Standardized SST anomalies of the neural net forecasts (dashed curves, averaged from 20 individual forecasts) for the Nino 4 SSTA compared with the observed anomalies (solid curves) during 1982-1993 at lead times of 3, 6, 9 and 12 months, with the SLP as predictor 70 4.11 (Cont.) 71 4.12 As in Fig. 4.11, but for the wind stress as predictor 72 4.12 (Cont.) 73 4.13 The forecast skills for various decades which roughly represent the 1950s, 1960s, 1970s and the 1980s for the Nino 4 region, using the SLP as predictor. 76 4.14 As in Fig. 4.13, but for the wind stress as predictor 77 4.15 The forecast skills for various decades which roughly represent the 1950s, 1960s, 1970s and the 1980s for the Nino 3.5 region, using the SLP as predictor 78 x i 4.16 As in Fig. 4.15, but for the wind stress as predictor 79 4.17 The forecast skills for various decades which roughly represent the 1950s, 1960s, 1970s and the 1980s for the P4 region, using the SLP as predictor. 82 4.18 The cross-validated skills (based on withholding 10 consecutive years in the cross-validation process) using the SLP or wind stress as predictor for the Nino 4 region 84 4.19 The seasonal cross-validated skills (based on withholding 10 consecutive years in the cross-validation process) using the SLP as predictor for the Nino 3.5 region 85 4.20 The monthly cross-validated skills (based on withholding 10 consecutive years in the cross-validation process) using the SLP as predictor, for the Nino 3.4 region, for forecasts of 0 - 12 months lead 86 5.1 The spatial pattern of the SLP E E O F Mode 1 at lags 9, 6, 3, and 0 months. The contours represent correlation between the E E O F amplitude and the SLP anomaly at each grid point 92 5.1 (Cont.) 93 5.2 The amplitude for the SLP E E O F mode 1 94 5.3 The spatial pattern of the SLP E E O F Mode 2 at lags 9, 6, 3, and 0 months. The contours represent correlation between the E E O F amplitude and the SLP anomaly at each grid point 96 5.3 (Cont.) 97 5.4 The amplitude for the SLP E E O F mode 2 98 5.5 The mean correlation skills for the forecast period of 1982-93, for Nino 4, Nino 3.5 and Nino 3 regions. The error bars indicate ± 1 standard deviation of the 20 forecasts 100 xii 5.6 Standardized SSTA of the neural net forecasts (dashed curves), averaged from 20 individual forecasts of Nino 4 region, compared with the observed anomalies (solid curves) for the forecast period of 1982-93 at lead times of 3, 6, 9 and 12 months 101 5.6 (Cont.) 102 5.7 The forecast correlation skills for various decades, corresponding roughly to the 1950s, 1960s, 1970s, and the 1980s for the Nino 4 SSTA. The error bars indicate ± 1 standard deviation of the 20 forecasts 104 5.8 The cross-validated correlation skills calculated over all decades for the Nino 4, Nino 3.5 and Nino 3 SSTA. The error bars indicate ± 1 standard deviation of the 20 forecasts 105 5.9 The seasonal cross-validated correlation skills for the the Nino 4 SSTA. The error bars indicate ± 1 standard deviation of the 20 forecasts 106 5.10 The spatial pattern and amplitude of SSTA E O F mode 1 of the equato-rial Pacific Ocean (20°S - 20°N). The contours represent the correlation between the amplitude and the SSTA at each grid point 108 5.11 As in Fig. 5.10 except for the second mode 109 5.12 As in Fig. 5.10 except for the third mode 110 5.13 The forecast correlation skills for 1982 - 1993 forecast period at lead times of 3, 6, 9, and 12 months. The SSTA forecasts were reconstructed from the forecasted first four modes of SSTA EOFs 112 5.13 (cont.) 113 5.14 Standardized SSTA of the neural network forecasts (dashed curves), spa-tially averaged from the forecasted SSTA field, of Nino 4 region, compared with the observed anomalies (solid curves) for the forecast period of 1982-93 at lead times of 3, 6, 9 and 12 months 115 xiii 5.14 (cont.) 116 5.15 The correlation skills for three different networks, the full model network trained with all the eight inputs, the network trained with modes 1, 2, 6 of the SLP E E O F and the SSTA, and the network trained with modes 3, 4, 5, and 7, for the training period (1952-1981) 121 5.16 As in Fig. 5.15 except for the forecast period (1982 - 1993) 122 5.17 The comparison between forecast skills of various neural network models for the 1982-93 period for the Nino 4 region (see text for explanation). The pruned models have only SLP E E O F modes 1, 2, 6 and the SSTA as inputs, with either 3 or 1 hidden neurons 123 5.18 The spectrum for the SLP E E O F mode 1 125 5.19 As in Fig. 5.18 except for SLP E E O F mode 2 126 5.20 As in Fig. 5.18 except for SLP E E O F mode 6 127 5.21 The spatial pattern of the SLP E E O F Mode 6 at lags 9, 6, 3, and 0 months. The contours represent correlation between the E E O F amplitude and the SLP anomaly at each grid point 128 5.21 (Cont.) 129 5.22 The standardized amplitude for the SLP E E O F mode 6. The thick line indicates the 6-year low-passed time series 130 5.23 The spectra for the neural network outputs for the Nino 3 region when fed with four pure sinusoidal signals at frequencies of 40, 55, 75 and 100 u>0 (see text for explanation) at 0-month lead time. The square root of the spectral density was plotted to render the weaker signals more visible. . . 135 5.24 As in Fig. 5.23 except for lead time of 6 months 136 5.25 As in Fig. 5.23 except for lead time of 12 months 137 xiv Acknowledgements I would like express my gratitude to my supervisor, Dr.William Hsieh, for his encourage-ment, guidance and support during this endeavour. I also would like to thank Dr.Benyang Tang for his comments and suggestions. To the other members of my supervisory com-mittee, Dr.Paul LeBlond and Dr.Ian McKendry, I am thankful for the roles they played. I would like to acknowledge my colleagues- Adam Monahan and Steven Bograd for fruitful discussions and valuable comments. I also received much help from Renee Jacques on the usage of L A T E X . Many thanks also go to Roy Hourston for supplying the COADS data tape. I am indebted to the system manager, Denis Laplante, for all the instructions on computing. I am grateful to the National University of Malaysia and the Malaysian Goverment which financially supported me and my family throughout our stay in Vancouver, British Columbia. I am especially grateful to my beloved wife, Zainab Batin, for the encouragement and support she gives. She is my 'co-pilot' and this study would not be possible without her. To my two lovely daugthers, Aneeza and Atika Oksana: you're daddy's source of inspiration. And finally, many thanks go to my parents in Sabah (Borneo), Malaysia, who con-stantly and remotely gave me encouragement and advice. xv Chapter 1 Introduction 1.1 E N S O Phenomenon The ENSO phenomenon, an acronym derived from its oceanographic component, E l Nino, and its atmospheric component, the Southern Oscillation, is a prominent climatic oscillation at the interannual time scale. ENSO is best viewed as an oscillation between warm and cold events, commonly referred to as E l Nino and La Nina (or E l Viejo), respectively. E l Nino is characterized by anomalous warming of surface waters in the equatorial Pacific, extending from the coastal region off Peru and Ecuador to the central region near the date line. Typically, the warming first appears late in boreal spring or summer, progressing to a peak at the end of the calendar year, and disappearing by the following summer. The warming produces significant circulation anomalies in the tropical Pacific and around the globe, e.g. redistribution of heat in the Pacific [Wyriki, 1985], reversal of current systems [Firring et al., 1983], shifts in major rain systems [Ropelewski and Halpert, 1989], and variation in global sea level pressure [Barnett, 1985]. The La Nina can be viewed as the opposite of an E l Nino. Occurring irregularly at intervals of typically 3 to 5 years, ENSO events exert dra-matic influence on the global climate, causing not only economic devastation but also a toll in human life. The E l Nino has been blamed for major climate related disasters such as flooding along the Pacific coast of South America and prolonged drought in the Asian-Australian region. The failure of the Indian monsoon to develop is also attributed 1 Chapter 1. Introduction 2 to this warm event [Kiladis and Sinha, 1991]. It also affects temperature and precipita-tion distributions in North America, especially during winter [Horel and Wallace, 1981; Rasmusson and Carpenter, 1983; Ropelewksi and Halpert, 1987]. During the 1982/83 extreme E l Nino, the Asian-Australian region experienced its worst recorded drought, resulting in the burning of millions of acres of rainforest in Borneo. Other regions, such as the Nordeste region of Brazil, the Sahel, and southern Africa, experienced similar drought. Meanwhile, the normally arid regions of Peru and Ecuador were hit by ma-jor flooding, with great loss of life and destruction of infrastructure. The warming also caused widespread mortality of fish and guano birds, damaging the local economy in these regions. Overall, the 1982/83 E l Nino caused US $8 billion in damage and the loss of 2,000 lives [Cane, 1992]. While the aftermath of the warm event has long been documented and appreciated, the consequences of the cold event have only been realized quite recently. Trenberth et al. [1988] linked the cold event of the 1988 La Nina to the severe drought in North America. The 1988 drought and its related heat waves constituted the worst natural disaster in United States history, with an estimated US $30 billion in agricultural losses and 10,000 deaths from heat stress [Trenberth and Branstator, 1992]. With such global implications, ENSO has become a very active research area, with much effort going into forecasting ENSO events. Prediction of ENSO has considerable economic and social benefits, as well as scientific interest. 1.2 E N S O Models Over the last ten years, many models have been developed for forecasting the ENSO phenomenon. A recent survey by Barnston et al. [1994] shows that there have been several ENSO prediction schemes developed and routinely used. These schemes can be Chapter 1. Introduction 3 divided into 4 distinct classes: (a) simple dynamical coupled models, e.g. at the Lamont-Doherty Earth Observatory [Cane et al. 1986; Cane and Zebiak, 1987; Zebiak and Cane, 1987], (b) coupled GCMs (general circulation models), e.g. at the National Center for Environmental Predictions (NCEP) [Ji et al. 1994a,b], (c) hybrid coupled models, where a simple empirical atmosphere is coupled to a dynamic ocean model, e.g. Barnett et al. [1993], and (d) statistical models, e.g. Canonical Correlation Analysis (CCA) [Barnston and Ropelewski, 1992], and the constructed analog method [Van den Dool, 1994]. More recently, another linear statistical model which has been used to predict ENSO is the Principal Oscillation Pattern (POP) analysis [Tang, 1995 ; von Storch et al., 1995]. As our physical understanding increases, dynamical models have the potential to give more skillful forecasts than purely statistical models [Barnston et al., 1994; Cane, 1992]. Despite the relative simplicity of statistical models, Barnston's survey indicated that there was no significant difference between the performances of these four types of models when forecasting at a lead time of 6 months. When used to forecast the sea surface temperature anomaly (SSTA) in the east-central equatorial Pacific for the period of 1982-1993 at a lead time of 6 months, these models all performed with comparable correlation skills (in the 0.6 - 0.7 range). For longer lead time forecasts, the dynamical models appeared to have better skills than the statistical models. Latif et al. [1994] showed that the simple coupled models [e.g. Zebiak-Cane model] had significant prediction ability at lead times exceeding one year. Recent findings have also indicated that data assimilation offer further forecast improvements, e.g. Chen et al. [1995] found skill improvement when a 'nudging' technique was used to assimilate data into the Zebiak-Cane model, with correlation skills greater than 0.60 for lead times up to 12 months. In principle, ENSO is predictable at least one year in advance. Chapter 1. Introduction 4 1.3 Motivation Most statistical models used to forecast ENSO in the past were linear [Barnett, 1984; Graham et al, 1987b; Barnston and Ropelewski, 1992; Tang, 1995]. The C C A model of Barnston and Ropelewski [1992], in particular, was designed to linearly relate the near global sea level pressure (SLP) variation to changes in the sea surface temperature (SST) field. Despite its relative simplicity, this model performed as well as the dynamical models at 6 months lead. At longer leads, the model had less skill than dynamical models. The poor performance at longer lead times may be partially due to the inability to capture nonlinear features in the ENSO dynamics. Latif et al. [1994] addressed the question of the factors ultimately limiting ENSO predictability. One of the possible candidates is the nonlinearity of the coupled systems. They further argued that the quasi-cyclic behaviour of ENSO suggested that nonlinearities played a significant role at longer lead times, presumably longer than one year. Zebiak and Cane [1987] argued that the relationship between the thermocline depth and wind stress was not simply linear. Many processes come into play in relating thermocline motions to surface wind stress, e.g. zonal temperature advection in the oceanic surface layer and moisture convergence effects in the atmosphere. Hence, the objective of this thesis is to investigate the viability of a nonlinear statistical model in forecasting ENSO, particularly at long lead times. In the last ten years, a nonlinear empirical modeling technique has emerged in the field of artificial intelligence- neural network models are now widely applied to numerous scientific, engineering and financial areas. Artificial neural networks are computing sys-tems containing many simple nonlinear computing nodes (or neurons) interconnected by links. This architecture is designed to simulate to some extent the structure and function of the brain [Rumelhart et al., 1986]. If trained properly, a neural network model is ca-pable of 'learning' the linear as well as the nonlinear features in a given data set. Eisner Chapter 1. Introduction 5 and Tsonis [1992] demonstrated that neural networks could perform better than linear statistical models when dealing with chaotic plus random noise systems, as the neural networks were able to distinguish between the seemingly random (but deterministic) behavior of chaos from random noise. Despite the wide use of neural networks, the application of neural networks to climate forecasting is still in its infancy. Recent studies by Tang et al. [1994], Grieger and Latif [1994] and Navone and Ceccatto [1994] have shown positive development towards their application in short-term climate prediction. In this study, neural networks will be applied to forecast the SSTA in the equatorial Pacific, and the results will be compared to the ENSO forecast models described in Barnston et al. [1994]. Chapter 2 Neural network models 2.1 Introduction The development of artificial neural networks has been inspired by the desire to mimic hu-man brain function. In recent years, neural networks have been used to perform complex tasks in various fields of application including artificial intelligence, parallel computers, condensed matter physics, statistical physics, pattern recognition, optimization and fi-nancial forecasting. Artificial neural networks are computing systems containing many interconnected nonlinear neurons, capable of extracting linear and nonlinear regularity in a given data set. The networks learn the features in the data and make generaliza-tions by changing the strength of the connections (i.e. the weights and biases) between neurons. These weights and biases are adjusted and optimized during training periods. While a number of architectures have been proposed, the feedforward back-propagation neural network is by far the most widely used. A brief description of this architecture is presented in the next section. A thorough explanation of all methods can be found in Lippmann [1987]. 2.2 Multi-layer feedforward neural networks A multi-layered architecture has one layer receiving input, one layer broadcasting output, and one or more hidden (intermediate) layers in between. The number of neurons in the input and output layers depends on the number of predictor and predictand variables, 6 Chapter 2. Neural network models 7 while the number of hidden layers and the number of neurons in each hidden layer depends on the complexity of the problem. Each neuron in the input layer is connected to all the neurons in the hidden layer while, each neuron in this layer is connected to all neurons in the output layer. Fig. 2.1 shows an example of a network with 4 neurons in the input layer, 3 neurons in the hidden layer and 1 neuron in the output layer. In this architecture, the information is propagated from the input layer through the hidden layer(s) to the output layer. If X{ is the input from the i-th neuron in the input layer, then yj is the output of the j-th. hidden neuron: yj = tanh^WijXi + bj), (2.1) t where u>ij and bj are called the weight and bias parameters respectively. Here the hyper-bolic tangent function is used as the transfer function, though this choice is not unique. If there is only one neuron in the output layer, then * = £ « > ; % + &, (2-2) 3 where Wj and b are again the weight and bias. In the training phase, the strength of the connection, i.e. the weights between the neurons, are adjusted so that the network learns the desired association between the predictand and predictor variables. This training process proceeds iteratively and stops when an 'optimal' solution is reached. Given enough hidden units, this architecture can perform any arbitrary mapping of an n-dimensional input to an m-dimensional output. It turns out that to approximate any function to arbitrary accuracy, at most two hidden layers (with enough units per layer) are needed [Cybenko, 1988]. For approximating any continous function, only one hidden layer is needed [Cybenko, 1989; Hornik et al., 1989]. Hence it is also called a universal function approximator. Chapter 2. Neural network models 8 Input Layer Figure 2.1: An example of a feedforward neural network model, where there are 4 neurons in the input layer, 3 in the hidden layer, and 1 in the output layer. The W{j and Wj are the weights and the bj and b are the biases. The bj and the b can be regarded as the weights for constant inputs of magnitude 1. Chapter 2. Neural network models 9 2.3 Training with backpropagation method To properly relate the inputs to the desired outputs, the strength of the connection between neurons (i.e. the weights) needs to be adjusted. The iterative adjustment process is often referred to as network training, and a standard algorithm is the 'back-propagation' rule [Rumelhart et al, 1994; Smith, 1993]. The word 'back-propagation' refers to the process of propagating the error information across the network from the output layer back to the hidden layer(s). The idea is that each output neuron determines the direction in which its weights would have to move to reduce the error (i.e. the difference between actual and targeted output), which then propagated back through its links to the hidden neurons. This is a minimization technique based on a steepest-descent algorithm for finding the minimum of a cost function (or error function) J , where the cost J is reduced by modifying the weights, proceeding backwards from the output layer to the input layer connections. Each weight is updated by a constant proportion of the partial derivative of the error with respect to the weight: w(t + l) = w(t)-eVJ(t), (2.3) where w(t) is the vector of weights at time t, e is the learning rate parameter, and V / ( i ) is the vector of first partial derivatives of the cost function. VJ(t) = 8J{t) _ _ 8J{t) dwt (t) > " " ' > dwN(t) (2.4) and N is the number of weights. The disadvantage of this method is its slow convergence. Jacobs [1988] modified the procedure by introducing momentum and adaptive learning rate, resulting in faster convergence. With this modification, each weight is updated as follows: Aw(t) = -(l-a)e(t)^ + aAw(t-l) (2.5) Chapter 2. Neural network models 10 > f jdJjt-i) = - ( l - a ) ^ e ( t - t ) a ^ r T y where a is the momentum factor that determines the relative contribution of the current and past partial derivatives to the current weight change. A value of a — 0.9 is commonly used. Smith [1993] suggested that the behaviour of the system is not highly sensitive to the value of a. The e{t) denote the learning rate corresponding to w(t) at time t, and is updated as follows: ' ic i£8(t-l)8(t) > 0 -<f>t(t) if 8(t- l)S{t) < 0, (2.6) 0 otherwise Ae(t) where xm - d J { t ) and S(t) = {l-6)8(t) + 66{t-l). In these equations, 8(t) is the partial derivative of the error with respect to w at time t and 8(t) is an exponential average of the current and past derivatives, with 9 as the base and time as the exponent. Smith [1993] suggested the values of 0.1, 0.5 and 0.7 for K, (f>, and 0, respectively, work well across a variety of problems. Nevertheles, the performance of the system is not highly sensitive to the choice of values for these parameters. When the consecutive derivatives of a weight in Eq.(2.5) possess the same sign, the exponentially weighted sum grows large in magnitude and the weight is adjusted by a large amount. In contrast, if the signs were different, then the sum becomes small in magnitude and the weight is adjusted by a small amount. In Eq.( 2.6), if the direction in which the error decreases at current weight change is the same as in the direction in Chapter 2. Neural network models 11 which it has been decreasing recently, the learning rate is incremented by a constant, K; if the direction in which the error currently decreases is opposite to the recent direction, the learning rate is decremented by a portion, </>, of its current value. 2.4 Overfi t t ing and generalization Generally, a successful training allows the network to capture the essential underlying relationship between the predictor and predictand variables. In such cases, the trained network shows remarkable generalization capabilities, being able to correctly relate inputs to outputs not included in the training set. However, the size of the architecture is the crucial element that needs to be considered. A smaller-than-needed network does not learn the examples well, while a bigger-than-required one overfits the data, learning undesirable noisy features in the data. A general rule of thumb for obtaining a good generalization is to employ the smallest network that fits the data. However, determining the best size is not obvious. The approach is to train a network that is larger than necessary and then remove parts that are not needed. This process is called network pruning [e.g. Reed, 1993; Xu et al., 1992]. Following Chauvin [1989], the cost function adopted here is of the form 2 ij 1 + {wij/aA)2 y 1 + (w j / a 4 ) 2 where the first term on the right hand side (RHS) is basically the sum squared error (SSE), with the z and zQ are the network output and the target, respectively. The other two terms are penalty terms (weight decay) against excessive weights, with the CZJ parameters to be chosen. As aA becomes large, Eq.( 2.7) simplifies to J=JzZ(z- *o? + a2 J>? , + a3 (2.8) ij j Chapter 2. Neural network models 12 similar to what was used in Tang et al. [1994]. This limiting form is known as ridge regression in the statistical community. This form of a cost function, representing a trade-off between residual error and model complexity, has been shown to produce robust networks [Hanson and Part, 1989; Ji and Psoitis, 1990; Weigend et al., 1990, 1991]. The addition of the penalty terms allows the network to perform auto-pruning. The idea is to force unimportant weights to approach zero and thus be implicitly deleted to obtain minimum network. The values of a-i and a^ can be interpreted as 'adjustments' towards a linear or nonlinear network, as the hidden neurons have nonlinear transfer functions (Eq. 2.1) while the output neuron has a linear transfer function (Eq. 2.2). For the case of 02, more weight restriction is imposed on the output layer and hence the resulting network tends to be more nonlinear. Chapter 3 Forecasting the Nino 3.4 SSTA 3.1 Introduction The equatorial sea surface temperature anomalies (SSTA) in regions such as the Nino 3 (6°S - 6°N, 90°W - 150°W), Nino 3.4 (6°S - 6°N, 120°W - 170°W) and Niiio 4 (6°S - 6°N, 160°E - 150°W) are commonly used as E l Niiio indices. Two previous ENSO C C A models [i.e. Graham et al., 1987b, and Barnston and Ropelewski, 1992] have shown the central equatorial Pacific region of Nino 3.4 (see Fig. 3.1) to be their best forecasted region. As a preliminary attempt to forecast ENSO using a neural network modeling technique, this chapter describes the application of neural network models to forecasting the Niiio 3.4 SSTA with wind stress as the main predictor. This chapter focuses on model building aspects. The design of the models incorporates the space and time domains of the equatorial Kelvin waves and the off-equatorial Rossby waves relevant to ENSO, as outlined in the delayed-oscillator theory [e.g. Battisti and Hirst, 1989; Graham and White, 1988]. To reduce the degrees of freedom in the system, the predictor fields were first compressed by the empirical orthogonal functions (EOF) analysis to several dominant modes. The dominant modes were then fed as inputs to the neural network models. Several experiments were performed in an attempt to establish the best model architecture. The neural network forecasts were interpreted in fight of current ENSO theories, e.g. the role of the off-equatorial Rossby waves in triggering the onset of an ENSO event and the delayed-oscillator theory in the development and 13 Chapter 3. Forecasting the Nino 3.4 SSTA 14 termination of an ENSO event. The results in this chapter have been published, and are hereafter referred to as Tangang et al. [1997a]. 3.2 Data and preliminary analysis The data used in this study, covering the period from 1952 to 1992, come from the Comprehensive Ocean-Atmosphere Data Set (COADS) [Woodruff et al, 1987; 1993]. The COADS monthly data are available in 2° latitude by 2° longitude grids. The tropical Pacific sea surface temperature (SST) is one of the major players in the development of the ENSO phenomenon. In this chapter, attention will be focused on the eastern-central region of the tropical Pacific (6°S - 6°N, 120°W-170°W). This study region (Fig. 3.1) was referred to as Nino 3.4 in Barnston et al. [1994] (where the latitudinal extent is actually 5°S - 5°N). The monthly SST data in this region were obtained by spatially averaging the SST in the 2° x 2° boxes. Sea surface temperature anomalies (SSTA) were obtained by subtracting from the monthly SST data the climatological monthly means which were calculated based on the entire record of the data set (1952-1992). These anomalies were then smoothed in time by a 3-point running average. The wind stress field covering the region of 20°S - 20°N, 120°E - 70°W was calculated based on the drag coefficient formula given in Trenberth et al. [1989] and Large and Pond [1981]. The zonal and meridional components of the wind speed also come from COADS. This region is wide enough to include the equatorial Kelvin and the off-equatorial Rossby wave activity that could be associated with the ENSO phenomenon. The wind stress data were further averaged from 2° by 2° grids to 2° latitude by 6° longitude grids. The traditional E O F analysis [Preisendorfer, 1988] was applied to the wind stress anomaly fields. The EOFs were calculated from the covariance matrix C = XTX, (3-1) Chapter 3. Forecasting the Nino 3.4 SSTA 15 with where rx and ry are the zonal and meridional components of the wind stress anomaly, re-spectively. The dimensionally smaller covariance matrix was used in the E O F analysis as implemented in Lagerloef and Bernstein [1988](see Appendix A) . The resulting temporal coefficients of the EOFs were also smoothed in time using a 3-point running average. Figs. 3.1 and 3.2 show the spatial pattern and the temporal coefficients of first and second modes of wind stress E O F . Each vector in the maps corresponds to the correlation values between the amplitude (i.e. temporal coefficient) and the zonal and meridional components of the wind stress anomaly fields. The spatial and temporal patterns of the first mode are similar to those in Legler [1983]. The first mode accounts for 4.1% of the total variance as compared to 6.8% in Legler's analysis, where the seasonal cycle was not removed. The temporal pattern of this mode shows an increasing trend in the post-1977 period. While there have been conflicting arguments in recent years on whether the increasing trend in the trade winds or the SST in the Pacific Ocean are real [Posmentier et al., 1989; Cardone et al., 1990], recent results of Wang [1995] seem to indicate that this trend is real and important in characterizing the pre- and post-1977 ENSO events. The second mode, with 3.0% of the variance, has a spatial pattern which indicates large anomalous westerly winds along the equator in the western-central region of the tropical Pacific, a feature associated with an E l Niiio event. The temporal coefficient also clearly indicates an ENSO connection, i.e. it is high during E l Niiio and low during La Nina. These patterns resemble the third mode of Legler [1983] (which accounted for 4.8% of the variance), and also the first mode of Graham et al. [1987a], where the wind stress covered the period 1947 - 1984 and the region 30°S - 30°N, 30°E - 70°W, averaged to 2° latitude by 10° longitude grids. The third mode (not shown), with 2.4% Chapter 3. Forecasting the Nino 3.4 SSTA 16 MODE 1 EOF of Wind Stress (4.1 perc.) 20°N Q 3 10°S 20°S 140 'E 100°W LONGITUDE -> 1.00 -7—1—i—rn—i i i i i i i i i i i i i i i i i i—i—i i i I i i i i i i i 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 Figure 3.1: First E O F mode of wind stress field. The dashed box centered around the equator indicates the Nino 3.4 region, in which the SSTA is forecasted. The amplitude has been divided by the standard deviation. Chapter 3. Forecasting the Nino 3.4 SSTA 17 MODE 2 EOF of Wind Stress (3.0 perc.) 20°N ' ' / /1 / " - / / / / * ' / ' / j 140°E \ I \ \ 180° 140°W LONGITUDE 100°W -> 1.00 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 Figure 3.2: Second E O F mode of wind stress field. That this mode is strongly related to ENSO can be seen by its amplitude, which rises sharply during E l Niiio years. Chapter 3. Forecasting the Nino 3.4 SSTA 18 of the variance, also shows some ENSO connection. The fourth and higher modes have no obvious physical interpretations. 3.3 Model designs The objective in this study is to simulate real-time forecasting, where estimating the forecast skill would be an issue. According to Barnston et al. [1994], a purely objective and bias free method for estimating the forecast skill of any given method in a truly independent (future) forecast setting does not exist. There are two common methods for estimating forecast skills: the cross-validation method and the retroactive method. For the cross-validation method [Michaelsen, 1987], the forecast skill is estimated by withholding the data each year in turn from the model's developmental sample and making forecasts for the withheld data period. With this method, skill inflation can still occur when there are interannual autocorrelations in the data, as in ENSO- related time series [Barnston et al., 1994]. This problem can be overcome by withholding groups of consecutive years for each set of forecast trial as suggested by Barnston et al. [1994]. Skill deflation can also occur when the true skill is low, due to a degeneracy inherent in cross validation [Barnston and Van den Dool, 1993]. In the retroactive set-up, no data occurring later than the forecast time are made available to the model. According to Barnston et al. [1994], this is perhaps more realistic in simulating real-time forecasts. In this chapter, the approach was based on the retroactive real-time forecast method, where the networks were trained with the record of 1952 - 1981 and the forecasts were made for 1982 - 1992. Since the E O F analysis was done over the entire record (1952-1992), the forecast period did not represent a true out-of-sample period. However, Tang [1995] found that this approach did not significantly inflate the forecast skills over a true out-of-sample analysis. Chapter 3. Forecasting the Nino 3.4 SSTA 19 A direct forecast method (also known as a 'jump forecast') was adopted [Weigend, 1995]. This approach allows the network to directly project from the present desired point h steps into the future, without involving any intermediate predictions or feedback. The goal of network training is to minimize the iterated /t-step forecasts. This method leads to a different network for a different step (or lead time). The lead time was varied from 0 to 15 months (the definition of lead time follows that of Barnston and Ropelewski [1992] and Barnston et al. [1994], i.e. the time between the end of the latest observed period and beginning of the predictand period). The use of low-frequency variations in the atmosphere, e.g. the sea level pressure (SLP) or wind stress, to predict the SST in the tropical Pacific has been demonstrated in Graham et al. [1987b] and Barnston and Ropeleweski [1992]. As indicated in Graham et al. [1987b], the timing and duration of the wind stress to be included in the predictor seem to be crucial. Their time scale of a 6- month block of wind data was intended to capture the equatorial Kelvin and reflected Rossby wave activities associated with ENSO events. Barnston and Ropelewski [1992] used one year of SLP in the predictor. In an attempt to estimate the best timing, several different durations of the wind stress in the predictor were considered, the results in which will be presented in Section 3.5. The results in the following section are based on the models using 12 months of wind stress plus the Nino 3.4 SSTA persistence. Using the most recent four 3-month averaged values of first 7 wind stress EOFs (accounted for 16.4% of the variance), the inputs in the models totalled 29 (i.e. 4x7+1). With 3 neurons in the hidden layer, the resulting networks had a total of 94 degrees of freedom (i.e. 29x3+3+3+1). The forecast settings, similar to that of Barnston and Ropelewski [1992], are illustrated in Fig. 3.3 for forecasting the onset, peak and mature phases of an E l Nino event. The terminology of Rasmusson and Carpenter [1982] was adopted in refering to these phases. Chapter 3. Forecasting the Nino 3.4 SSTA 20 J J F M A M J J A S O N D | J F M A M J J A S 0 N D J F M A M J J A S O N(D J F ) 1 J F J F J F M A M J J A | S O N D ] J F M A M J J A S 0 N D J F ^ A M ^ J J A S O N D t J J A J F M A | M J J||A S 0||N D|J| |F M A |M s O(N D J ) F M A M J J A S O N D 1 I Figure 3.3: Schematic diagram showing the timing of the predictor-predict and periods at 6-month lead forecasting the onset (Nov(-l) - Jan(0)), peak (Mar(0) - May(0)) and mature (Dec(0) - Feb(+1)) phases of an E l Nifio event. The year 0 refers to the E l Nino years (in which anomalous warming first appears and then amplifies in the tropical Pacific); year -1 and +1 refer to the previous and subsequent years, respectively. The predictors are enclosed by rectangular boxes, and the predictands by oval boxes. 3.4 Results 3.4.1 Forecasts with neural network models The skills of the retroactive forecasts for 1982 -1992 are shown in Fig. 3.4. The three curves represent the control (i.e. persistence) forecasts, the individual forecasts and the ensemble forecasts. The ensemble forecasts were calculated by averaging six individual forecasts. For example, the ensemble forecast for the DJF period of 1982 at 3-month lead was averaged from the individual forecast of the same period at 3-month lead and those at 4, 5, 6, 7, and 8-month leads. As indicated in Fig. 3.4, this method of averaging tends to improve the skills in the medium range (5 to 10-month) but to degrade the skills for forecasts of very short leads and those longer than a year. At 6-month lead, the correlation and the root-mean-square error (rmse) are 0.66 and 0.83, respectively, for Chapter 3. Forecasting the Nino 3.4 SSTA 21 ( a ) c o ro o> i_ i_ o o 1 0 .8 0 .6 0 .4 0 .2 0 0 .2 0 . 4 i 1 1 1 1 1 r — • — N N ( E n s e m b l e f o r e c a s t ) - - e - - N N ( l n d i v i d u a l f o r e c a s t ) - -x- - P e r s i s t e n c e I I I L Xs. J l Jl 0 1 2 3 4 5 6 7 8 9 1 0 1 1 1 2 1 3 1 4 1 5 L e a d t i m e ( m o n t h ) Figure 3.4: (a) Correlation and (b) root-mean-square (rmse) between neural network model forecasts (individual and ensemble), and observation for the Nino 3.4 SSTA during 1982 - 1992 for various lead times. Persistence is shown as the dot-dash curve. The rmse was calculated for standardized values (i.e. values inflated to zero mean and normalized to their standard deviations). Chapter 3. Forecasting the Nino 3.4 SSTA 22 ( b ) 1 .8 1 .6 1 .4 1 .2 1 0 .8 0 0 .4 0 .2 "i r "i r -X' "i i r -X— -X-_ e -— • %•— •o o — • — N N ( E n s e m b l e f o r e c a s t ) « _ _ e - - N N ( l n d i v i d u a l f o r e c a s t ) - x- - P e r s i s t e n c e J I I I I I I I L I I I I I 0 1 2 3 4 5 6 7 8 9 1 0 1 1 1 2 1 3 1 4 1 ! Figure 3.4: (Cont.) Chapter 3. Forecasting the Nino 3.4 SSTA 23 the individual forecasts. The ensemble forecasts appear to be better, with a correlation of 0.71 and an rmse of 0.73. These values are comparable to those of existing models described in Barnston et al. [1994], where the correlations were in the 0.6 - 0.7 range and the rmse greater than 0.80. As lead time increases beyond 6 months, the forecast skill shows stability and minimal decay up to a lead time of a year, with the correlation skills greater than 0.6 and rsme values between 0.83 - 0.85. Also, the correlation at 12-month lead is 0.65, which seems to suggest an increase of skill from that of the 9-month lead for the individual forecasts. This trend agrees with that of a hybrid coupled model shown in Fig.6 of Latif et al. [1994]. Their forecast skills, which were based on the period 1976-1990, clearly showed an increase as lead time approached a year. Beyond 12-month lead, the neural network forecast skill resumes its decline. At 15-month lead, the correlation skill for the individual forecasts is 0.57. Fig. 3.5 compares the correlation between the neural network outputs and the ob-servations over the training period (1952-1981) with the correlation over the forecast validation period (1982-1992). The correlations over the training period are greatest at short leads, and decay as lead time increases. Beyond a 9-month lead the fitting seems to improve. The correlations over the validation period appear to be slightly higher than over the training period for lead time 3 months or shorter. As lead time increases beyond a 3-month lead, the training skills are higher than the forecast skills. Fig. 3.6 compares forecasts with observations at 3, 6, 9 and 12-month leads. The individual forecasts are somewhat noisy, similar to the forecasts by the C C A and analog models discussed in Barnston et al. [1994]. By ensemble averaging the forecasts, the high frequency noise is removed, particularly improving the skills for lead times of 6 and 9 months. At 3-month lead (Fig. 3.6a), the model forecasts the main events of the 1982/83, 1986/87 and the 1991/92 E l Nifios and the 1988 La Nina. The ensemble forecasts give a rather small amplitude for the 1982/83 E l Niiio, while both individual and ensemble Chapter 3. Forecasting the Nino 3.4 SSTA 24 T i i i i i i i i i i i i i r — • — T r a i n i n g — * - - F o r e c a s t 0 1 2 3 4 5 6 7 8 8 9 1 0 1 1 1 2 1 3 1 4 1 5 L e a d t i m e ( m o n t h ) Figure 3.5: A comparison of correlation skills of the neural network model for the Nino 3.4 SSTA during the training period (1952-1981) and the forecast validation period (1982-1992) at various lead times. Chapter 3. Forecasting the Nino 3.4 SSTA 25 forecasts are late for this warm event. Barnston and Ropelewski [1992, Fig. 11] showed similar problems for this warm event. The onset of the 1986/87 and 1991/92 E l Ninos are well forecasted. However, the model fails to forecast the second peak of the 1986/87 E l Nino. The 1988 La Nina is also well forecasted except for a shallower trough. The spin-down of the events are reasonably well forecasted for the 1982/83 and 1986/87 events, with a delay of a few months for the 1991/92 event. As for the individual forecasts, there are two obvious false alarms: the model falsely predicts (1) a cold event in the summer of 1984 and (2) a high peak with a duration of several months in 1990. The incorrect forecast for a cold event in the summer of 1984 is similar to that made by the C C A of Barnston and Ropelewski [1992, Fig.11], who argued that their C C A model 'learned' only from the predictor-predict and relationship over the provided record and 'performed' what an empirical (and linearly) minded human might do. As the neural network model forecast resembles the C C A forecast, this supports the contention (to be discussed later) that for short lead times the neural network model 'learns' only the linear relationship, which appears to be the optimal solution. The high peak in 1990 was not described in Barnston and Ropelewski [1992], as their data ended in 1989. It was, however, clearly visible in the analog model for forecasts at 6-month lead as described in Barnston et al. [1994]. For a 6-month lead (Fig. 3.6b), the neural network model forecasts the 1986/87 and 1991/92 E l Nino rather well, but underestimates the 1982/83 E l Niiio, with the forecasted amplitude half that of the observed plus a delay of onset. The C C A model discussed in Barnston et al. [1994, Fig.2d] performed better, forecasting the phase rather well, but the amplitude was also about half of the observed. The peak of the 1991/92 E l Niiio is better forecasted by the neural network model than the C C A . In contrast, the amplitude of the 1988 La Nina is not as well forecasted by the neural network model, although there was a slight delay in the decay of the 1986/87 E l Niiio in the C C A forecast. Some Chapter 3. Forecasting the Nino 3.4 SSTA 26 ( a ) ( 3 - m o n t h l e a d ) • i i • a o 2 0 ~ rk (Individual: r=0.79, rmse = 0.65, Ensemble: r=0.75, rmse = 0.68) 1982 1984 1986 1988 1990 1992 ( b ) ( 6 - m o n t h l e a d ) • i i • 2.0 H (Individual: r=0.66, rmse=0.83, Ensemble: r=0.71, rmse=0.73) -3.0 1 1 1 ' I I' 1982 1984 1986 1988 Observations Ensemble forecasts 1990 1992 Individual forecasts Figure 3.6: Standardized SST anomalies of the neural net forecasts (dash-dotted curves for individual forecasts, dotted curves for the ensemble forecasts) of the Nino 3.4 SSTA compared with the observed anomalies (solid curves) during 1982-1992 at lead times of a 3, b 6, c 9, d 12 months. Chapter 3. Forecasting the Nino 3.4 SSTA 27 (c) ( 9 - m o n t h lead) 1982 1984 1986 1988 1990 1992 (d) (12-month lead) 1982 1984 1986 1988 1990 1992 Observations Individual forecasts Ensemble forecasts Figure 3.6: (Cont.) Chapter 3. Forecasting the Nino 3.4 SSTA 28 time in the fall of 1984, the model also incorrectly forecasts a cold event. The C C A model also predicted a relatively high peak in 1985 which did not appear in the neural network forecast. The differences between the C C A and the neural network forecasts could be due to: (1) the models used different predictor sets, and (2) the C C A searched for linear combinations of the predictor and the predictand data fields with the highest correlation, while the neural network model searched for the best linear or non-linear relationship between the predictors and the predictand. At 9-month lead (Fig. 3.6c), the individual forecast skills are marginally worse than those at 6-month lead, while the ensemble forecasts have correlation and rmse values identical to those at 6-month lead. A direct comparison cannot be made between the C C A and the neural network model as no details of the C C A forecasts at 9-month lead and beyond were given in Barnston et al. [1994]. The results of Barnston and Ropelewski [1992] and Graham et al. [1987b] seemed to indicate low skills for the C C A method at 9-month lead or beyond. At 12-month lead (Fig. 3.6d), the ensemble forecasts do not improve on the individual ones since the individual forecasts degrade rapidly beyond a 15-month lead. Overall, the results seem to indicate that a neural network model is viable for forecasting the ENSO phenomenon, with reliable forecasts up to a year ahead. At 6-month lead, the neural networks models perform comparably to the existing models discussed in Barnston et al. [1994]. The model performances at 9 and 12-month leads are rather encouraging as previous studies of Barnston and Ropelewski [1992] and Graham et al. [1987b] seemed to suggest lower skills for the C C A at these leads. At these longer leads the skills from the empirical models would be expected to be lower than those of the dynamical models [Cane, 1992; Barnston et al., 1994; Chen at al, 1995]. The relatively high skills attained by neural network models at longer lead time could be attributed to two factors: (1) the predictors (i.e. the wind stress and the SSTA persistence) and the predictand (i.e. the Chapter 3. Forecasting the Nino 3.4 SSTA 29 SSTA) could be increasingly nonlinearly related as the lead time increases, with neural networks having the ability to model these nonlinear relations, and (2) the wind stress fields cover the region from 20°S to 20°N, which is wide enough to include the activities of the off-equatorial oceanic Rossby waves that may be associated with ENSO. Graham et al. [1987b] used the wind fields only from 9°S to 9°N, which did not reach far enough off the equator, as noted by Graham and White [1988]. 3.4.2 Comparison with a stepwise multiple linear regression model To test (1), a stepwise multiple linear regression model was used to fit the data. The linear model was set up in exactly the same way as the neural network except that the predictors were related to the predictand by linear regression. This simple method was not intended to be representative of an optimal linear model but merely as a way to see whether the neural network model learns linear or nonlinear relationships between the predictor and the predictand. Nonetheless, the Nino 3.4 SSTA forecasts of this regression model would be expected to be comparable to those from C C A models. Fig. 3.7 compares the neural network model and linear regression model in fitting the training data (Fig. 3.7a) and forecasting the validation data (Fig. 3.7b). For 0-month lead, there is no difference between the two models. As lead time increases, the fitting by the simple regression method becomes poorer, whereas the neural network fittings show very slow decay (Fig. 3.7a). In fact, the fit improved as the lead time approached 12 months, where the correlation is above 0.70. The failure of this linear regression model to fit the training data at longer lead of 9 to 12 months is consistent with the results presented by Barnett [1984], who showed that his linear statistical model produced no skillful hindcasts beyond 5-month leads. What does the neural network learn from the training data? It does not just learn the linear relationship in the data, as it does much better than the regression model at 9 and Chapter 3. Forecasting the Nino 3.4 SSTA 30 ( a ) 1 0 3 6 9 1 2 Lead time (month) Figure 3.7: A comparison of correlation skills between the neural network model and the multiple linear regression model during (a) training period (1952-1981) and (b) forecast-ing period (1982-1992), for various lead times. Chapter 3. Forecasting the Nino 3.4 SSTA . ( b ) 0 3 6 9 1 2 Lead time (month) Figure 3.7: (Cont.) Chapter 3. Forecasting the Nino 3.4 SSTA 32 12-month leads. That leaves two possibilities: either the neural network learns nonlinear relationships, or it fits to the noise in the training data. With a very flexible method like neural networks, any degree of fitting can be achieved by adding more neurons in the hidden layer. Better fitting with the training data does not guarantee better forecasting with the validation data. Fig. 3.7b shows that over the validation data, the neural network model is able to produce better skills than the regression model, particularly at longer leads of 9 to 12 months. Fig. 3.8 compares the fittings in the training data at 12-month lead. The three curves show the corresponding observations (solid fine), the neural network hindcasts (dashed) and those of the regression model (dotted). Overall, neural network hindcasts are better than those of the regression model, with a correlation of 0.72 versus 0.38. One particularly interesting feature in Fig. 3.8 is the period of 1974/75, known as 'aborted E l Nino'. Latif et al. [1994] noted that most prediction schemes failed to forecast the anomalously cold year of 1975, as the atmosphere and ocean conditions were favourable for a warm episode to occur. This is the case with the regression model which hindcasted a warm event in 1975. The neural network model is able to hindcast the cold event, though the trough is somewhat underestimated. These results seem to suggest that nonlinearity could be important in forecasting at longer leads of 9 to 12 months, even though at shorter leads nonlinearity is negligible. How much nonlinearity is there in ENSO? Munnich et al. [1991] used a nonlinear atmosphere/ocean model to show that irregular interannual variability can result from the coupling of the atmosphere and the ocean. Mantua [1994] and Mantua and Battisti [1995] analyzed the Zebiak-Cane numerical model and demonstrated that the irregularity in the model ENSO cycle was due to the interaction between two coupled unstable atmosphere/ocean modes. They also concluded that the irregularities in the model ENSO events were caused by deterministic, nonlinear processes in the model associated with Chapter 3. Forecasting the Nino 3.4 SSTA 33 the interactions between the unstable modes. These modeling results seem to suggest that nonlinearity plays an important role in the irregularities of ENSO events, and are consistent with the notion that nonlinearity could be important in long lead forecasting. 3.4.3 The effects of using wind stress of different latitudinal extent To test (2), (i.e. the effect of using wind stress fields of different latitudinal extent), the process was repeated by re-calculating the wind stress EOFs for the two smaller bands of 10°S- 10°N and 6°S-6°N. The neural networks were again trained with the first seven wind stress EOFs for these regions plus the SSTA itself, and to the point where all performed comparably in the training data. The forecast skills were then assessed on the validation data. The forecast skills of models using three different wind stress domains i.e. 20°S-20°N, 10°S-10°N and 6°S-6°N, show that for forecasting at 3-month lead or shorter, no distinction can be made from using the three domains (Fig. 3.9). As lead time increases, the skills of the model using the wind stress of 6°S-6°N decay rather rapidly. There are no differences between the skills of the model using wind stress from 20°S-20°N and that from 10°S-10°N up to 9-month lead. Overall, the model using wind stress from 20°S-20°N seems to be the best. While these experiments give some insight on how the latitudinal width of the wind stress field affects the forecasting skills, they do not provide a definitive answer on whether the off-equatorial Rossby waves are important for the ENSO oscillation. The relative importance of these off-equatorial Rossby waves in. driving ENSO has been a topic of debate in recent years [Graham and White, 1988; Battisti, 1989; Kessler, 1991; Graham and White, 1991; Battisti, 1991; White and Tai, 1992]. The recent study of Schneider et al. [1995] identified the low-order freely propagating Rossby waves, excited by wind stress forcing near the date fine, as being important. Battisti and Sarachik [1995] con-cluded that presently, the theory for ENSO events which best fits the observations is the Chapter 3. Forecasting the Nino 3.4 SSTA 34 - 3 . 0 - « -1954 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 Observation Neural Network MLR Figure 3.8: Standardized SST anomalies of the neural network hindcasts (dashed curve) and the multiple linear regression hindcasts (dotted curves) for the training data at 12-month lead. Chapter 3. Forecasting the Nino 3.4 SSTA 35 'delayed-oscillator theory' of ENSO, put forward by Suarez and Schopf [1988] and Bat-tisti and Hirst [1989]. According to this theory, the easterly (westerly) wind anomalies over the western Pacific prevailing during the La Nina (El Nino) event force upwelling (downwelling) Kelvin waves which propagate eastward along the equator, causing cooling (warming) at the sea surface in the eastern Pacific, where the thermocline is shallow. The oceanic reponse to the easterly (westerly) winds in the west also consists of downwelling (upwelling) Rossby waves, which propagate westward. As the thermocline is deep in the western Pacific, the SST is not strongly influenced by these waves. The Rossby waves then reflect at the western boundary into a packet of downwelling (upwelling) Kelvin waves which propagate eastward. Once the Kelvin waves have propagated far enough eastward, they are able to affect the SST, and a positive (negative) SSTA develops which can grow further by unstable air-sea interactions into an E l Nino (La Nina). This theory describes a perfectly periodic oscillation which supports a never-ending succession of al-ternating warm and cold events. In nature, the frequency of ENSO events is irregular, a characteristic not explained by the delayed oscillator theory. Nonetheless, Mantua and Battisti [1994] stressed that the evolution of wind and SST anomalies in each (warm or cold) event remains consistent with the major features of the theory. They further argued that the only missing ingredient is the initial perturbation for the instability in the central and eastern Pacific to act on. White et al. [1989] presented evidence of positive thermocline depth anomalies, associated with wind-driven baroclinic Rossby waves, along the western boundary during the onset phase of an ENSO event. Graham and White [1990] found that most of the magnitude in this wave activity occurred in response to the wind stress curl forcing in the central and western Pacific, amplified by resonance. They further argued that these anomalies actually triggered the ENSO event. These effects are likely to be dominant only during the onset of an event. The equatorial oceanic response to increasingly anomalous westerly wind activities is responsible for the Chapter 3. Forecasting the Nino 3.4 SSTA 36 subsequent development in the peak and mature phases. The wind stress forcing in the off-equatorial domain occurs 1 to 2 years prior to the onset phase, whereas that in the equatorial domain occurs during the later part of the onset phase and continues into the peak and mature phases, lasting for 9 months or less. The interpretation of the model forecasts can be considered in conjunction with the theory and evidence presented in these earlier studies. Fig. 3.3 illustrates the predictor-predictand relation for forecasting a warm event at 6-month lead time. Such predictor-predictand relation can be easily extended for other lead times. At all lead times, to forecast the onset phase the model utilizes anomalous wind stress 1-2 years prior to the onset phase. For forecasting the peak and mature phases, the model could have used the anomalous winds responsible for the off-equatorial signals and/or the anomalous winds resulting from the ongoing event. The model skills in forecasting the onset, peak and mature phases reflect the model's ability to detect the off- equatorial wind signals 1 -2 years prior to the onset phase and the equatorial wind effects as the event developed. This tendency is consistent not only with the evidence of baroclinic Rossby waves presented by White et al. [1989], but also with the delayed oscillator theory in which Kelvin wave effects dominated the progression of the event from the peak through the mature phase until terminated by the delayed Rossby wave effect. The model's failure or underestimation for the 1982/83 E l Nino at 6 to 12-month leads is puzzling. This could be due to the unusual evolution of the 1982/83 E l Nino [Philander, 1990], which does not resemble any of the E l Nifios in the period 1952-1981 used to train the model. Another plausible explanation is that the winds used as predictor at 9 to 12-month leads did not contain the anomalous off-equatorial winds, which could have occured late for this E l Nino. White et al. [1987] showed the evolution of the heat content propagating as baroclinic Rossby waves from the central tropical North Pacific in mid-1981 to the maritime coast of Asia in late 1981 and early 1982, and Chapter 3. Forecasting the Niiio 3.4 SSTA 37 0 3 6 9 1 2 Lead time (month) Figure 3.9: A comparison of forecast correlation skills between neural network mod-els using wind stress of 20°S-20°N, 10°S-10°N and 6°S-6°N for the forecasting period (1982-1992), at various lead times. Chapter 3. Forecasting the Nino 3.4 SSTA 38 generating the equatorial Kelvin wave in middle-to-late 1982 (see their Fig.3). Also, their comparison of the observed and modeled anomalous dynamic heights showed that the observed generation of the Rossby waves at the source region (near the date line) came much later (mid to late 1981) than the model result (mid 1980). These off-equatorial anomalous wind effects were therefore delayed by about a year before the 1982/83 E l Nino occurred. With this delay, there would be little or no precursor patterns in the wind stress data for forecasting at longer leads of 9 to 12 months (Fig. 3.3) and hence the model would fail to forecast the event. In comparison with the 1982/83 E l Nino, White et al. [1989] noted that the generation of the Rossby waves responsible for the onset of 1986/87 E l Nino took place sometime in 1984-1985. In section 3.6, the neural network model was run with the SST EOFs as predictors, which contained little off-equatorial signal, and the model tended to give late forecasts for the onset phase even at a lead of 0 month. 3.4.4 Seasonality of the skills The seasonality of the forecasting skills was first noted by Hasselmann and Barnett [1981] and subsequently observed in many studies. Barnston et al. [1994] discussed the seasonality of the skills of five models based on the retroactive forecasts of the period 1982-1992. Their forecasts at 6-month lead were also seasonally dependent, with the forecasts made in the fall (or winter) for the following summer (or fall) having lower skills than those made at other times. This is related to the so-called 'spring barrier', which is difficult to traverse in a forecast [Ji et al., 1994b; Latif et al., 1994]. Chen et al. [1995] however found no spring barrier when they used a 'nudging' procedure with the Zebiak-Cane model. To examine the seasonality of the forecast skills, the forecast correlation skills at 0, 3, 6, 9, 12 and 15-month leads for various seasons are displayed in Fig. 3.10. Due to the shortness of the record here (1982-1992), a detailed discussion Chapter 3. Forecasting the Nino 3.4 SSTA 39 is not warranted, except to note that the forecast skills of the models are also seasonally dependent, with forecasts for spring and winter having higher skills than those for summer and fall. 3.5 Sensitivity Analyses The steepest descent with momentum and adaptive learning rates have been used as the optimization technique in training the networks [Jacobs, 1988]. The use of momentum decreases the backpropagation's sensitivity to small details in the error surface. This would minimize the possibility of the networks getting stuck in shallow minima. The use of adaptive learning rate reduces the training time where the learning rate size is kept as large as possible while keeping learning stable. This method was also adopted in the studies of Grieger and Latif [1994] and Navone and Ceccato [1994]. Extra experiments were performed in which the networks were trained with different initial learning rate and momentum. The result shows the forecast results to be rather insensitive to these training parameters. However, it appears that this optimization scheme is somewhat sensitive to the initial weights, as was also found by Kolen and Pollack [1991]. Hence the optimized solutions may not be the global ones as noted by Grieger and Latif [1994]. The effect of the weighting parameters in Eq.( 2.7) was also investigated. With a\ set to 1, the value of a 4 was fixed to 100, as this allows Eq.( 2.7) to approach Eq.( 2.8). The other two parameters, a% and a^, were the adjusted and the values of 0.1 and 2.5 for a2 and a 3 , respectively, appeared to be reasonable with no obvious overfitting. These chosen parameters put more restriction on the neurons in the output layer than those in the hidden layer in trying to contain the networks' excessive capability for fitting the training data. Choosing the right internal representation in the network is a crucial element in neural Chapter 3. Forecasting the Nino 3.4 SSTA 40 i t I III mm • S p r i n g E2 S u m m e r • F a l l W i n t e r 3 6 9 Lead time (month) 1 2 1 5 Figure 3.10: The forecast correlation skills for various predictand seasons at 0, 3, 6, 9, 12 and 15-month leads. Chapter 3. Forecasting the Nino 3.4 SSTA 41 network modeling. The number of neurons in the hidden layer were adjusted from 2 to 5 for all lead times. For all models, 7 E O F modes and 12 months duration of the wind stress plus the Nino 3.4 SSTA persistence were used as predictors. The networks were trained to the point where they had comparable skills in fitting the training data (1952-1981), then the forecast skills were estimated on the validation data (1982-1992). Fig. 3.11 compares the forecast skills of these models for 0, 3, 6, 9 and 12-month leads. For short leads, 3 months or less, varying the number of hidden neurons has minor effects on the forecast skills. For 6 and 9-month leads, there is little difference between using 3 and 4 neurons. For 12-month lead, the model with 3 neurons performs best. Overall, the models with 3 neurons in the hidden layer seem to be the best choice. The effects from using different durations of the wind stress in the predictor set were also investigated. Graham et al. [1987b] used 6 months of wind stress to predict the SST whereas Barnston and Ropelewski [1992] used 12 months of SLP as predictors. Here an experiment was conducted to compare the models using 6, 9 or 12 months of wind stress. In all models, 7 wind stress EOFs and the Niiio 3.4 SSTA persistence were used as predictors, with 3 neurons in the hidden layer. A l l the models were trained to the point where all attained comparable fitting skills in the training data, and then the forecast skills were assessed on the validation data (Fig. 3.12). For all lead times, the model using 12 months wind stress generally has the best forecast skills, especially at longer lead times. To test how many wind stress E O F modes were needed as predictors, the number of EOFs were varied from 4 to 7. For all models, 12 months of wind stress EOFs plus the Nino 3.4 SSTA persistence were used as predictors, with 3 neurons in the hidden layer. The models were again trained to the point where all achieved comparable fitting skills with the training data. The forecast skills of these models (Fig. 3.13) show that at lead times up to 6 months, there are only minor differences between the models. For longer Chapter 3. Forecasting the Nino 3.4 SSTA 42 Lead time (month) Figure 3.11: The forecast correlation skills at 0, 3, 6, 9, 12-month leads, as the number of neurons in the hidden layer varied from 2 to 5. Chapter 3. Forecasting the Nino 3.4 SSTA 43 0 3 6 9 1 2 Lead time (month) Figure 3.12: The forecast correlation skills at 0, 3, 6, 12-month leads for various duration of wind stress data as predictors. Chapter 3. Forecasting the Nino 3.4 SSTA 44 leads, the model with 4 EOFs does not produce usable skills. Overall, using 7 EOFs in the model seems to give better forecast skills at all lead times. The consistency of the forecast skills based on the forecast period of 1982 to 1992 with other forecast periods was also investigated. The time series were partitioned into 4 parts with roughly 10- year duration for each. These 4 periods are 1952-1962, 1963-1972, 1973-1982 and 1983-1992, approximately representing the 1950s, 1960s, 1970s and 1980s data. One part of the data was withheld in turn as forecast validation data, and the neural network was trained with the remaining 3 parts. The forecast skills based in the 1980s period are similar to those presented earlier in Section 3.4.1. The model performances during the other periods are not as good. At 6-month lead, the forecast correlation skills are 0.02, 0.34, 0.25 and 0.70 for the four periods, respectively. The fitting skills with the training data are comparable, with a correlation of 0.76, 0.79, 0.78 and 0.63, respectively. That the higher skills for the 1980s are not duplicated for the other decades could well be caused by the following factors: (1) The 1980s period experienced rather strong ENSO activities. In particular, the ENSOs of the 1980s had stronger warming in the central equatorial Pacific (in contrast to the stronger warming in the eastern equatorial Pacific during the earlier ENSOs; see Wang, 1995). The quality of the data in the 1980s was also better than the other decades. These factors could make forecasting the Nino 3.4 SSTA (in the central equatorial Pacific) in the 1980s more accurate. (2) Compared to the other decades, the 1960s had the weakest ENSO fluctuations. In general, ENSO forecast schemes do not perform well when there are no strong E l Nino or La Nina events. (3) The data quality in the 1950s and 1960s was relatively poor, and would decrease the forecast skills. Despite the poor data quality in the earlier decades, the networks were apparently able to filter out the noise when using these data for training, thus yielding high forecast skills for the 1980s period. The results of Chen et al. [1995] showed a similar trend in which their skills tended to be lower in 1970s but higher in 1980s. One caveat is that Chapter 3. Forecasting the Nino 3.4 SSTA 45 there is clearly decadal dependence in the skills, but stationarity is implicitly assumed in the neural network models. 3.6 SST as predictor Barnston and Ropelewski [1992] showed that the inclusion of SST as a predictor would increase the forecast skills for SST, as the thermal inertia of SST would help stabilize the forecasts at short leads. Here a comparison between the model using wind stress over four seasons plus the Niiio 3.4 SSTA persistence and the model using only the wind stress was also made. Both models were trained to the point where they had comparable fitting skills on the training data. The forecast skills in Fig. 3.14 indicate only minor improvement by including the SSTA persistence as predictor. As ENSO is a coupled phenomenon, and the result showed that it can be forecasted using wind data, an interesting question is whether it can be forecasted using only ocean data, e.g. SST. To test this, the models were trained with the SST EOFs replacing the wind stress EOFs. The spatial coverage of the SST EOFs was the same as those of the wind EOFs i.e. 20°S - 20°N, 120°E - 70°W. 7 SST EOFs (accounted for 29.6% of the variance) for the four predictor seasons plus the Niiio 3.4 SSTA persistence used as predictors. For comparison, another model with just the 7 SST EOFs as predictors was also trained. Both models were trained to the point where they had fitting skills comparable to those models fed using wind stress or wind stress plus Niiio 3.4 SSTA. The forecast skills of these two models (Fig. 3.14) are much lower than the models using wind stress, with no usable skills for forecasting at 6 months or longer. These results seem to suggest that Nino 3.4 evolution cannot be tracked well based on the basinwide evolution of SST. This supports the earlier contention that the models, with wind stress as the predictor, utilize the effects of the off-equatorial and/or the equatorial anomalous winds. Chapter 3. Forecasting the Nino 3.4 SSTA 46 0 3 6 9 1 2 Lead time (month) Figure 3.13: The forecast correlation skills at 0, 3, 6, 9, 12-month leads, as the number of input wind stress EOFs varied from 4 to 7. Chapter 3. Forecasting the Nino 3.4 SSTA 47 Lead time (month) Figure 3.14: Nino 3.4 SSTA forecast correlation skills at 0, 3, 6, 9, 12-month leads for neural network models with different sets of predictors. Chapter 3. Forecasting the Nino 3.4 SSTA 48 It is unlikely that the model using SST EOFs as predictors captured the off-equatorial signals, since the signatures of the Rossby waves are less likely to be visible in the SST. This is particularly true in the western Pacific Ocean, where the thermocline is deep. White et al. [1987, Fig.3] showed that these waves were visible in vertically-averaged temperature but not in the SST. In contrast, as it was the anomalous wind which forced these waves, the signatures should appear in the wind stress EOFs. Fig. 3.15 displays the observations versus the forecasts at 0-month lead for models using only SST EOFs (dashed curve) or wind stress EOFs (dotted curve) as predictors. The correlation skills are 0.91 and 0.56 for the wind stress model and the SST model, respectively. With the absence of the off-equatorial Rossby waves signals in the SST E O F predictors, the model tends to give late forecasts for the onset of all the events. 3.7 Summary and conclusion This chapter demonstrated the use of neural network models to seasonally forecast the tropical Pacific SST in the Nino 3.4 region (6°S - 6°N, 20°W - 70°W). The inputs to the neural networks were the first seven wind stress EOFs of the tropical Pacific (20°S -20°N, 120°E - 70°W) for four seasons and the Nino 3.4 SSTA persistence. The period of 1952-1981 was used for training the neural network models, and the period 1982-1992 for forecast validation. At 6-month lead time, neural network models attained comparable skill to the models surveyed by Barnston et al. [1994] (where the forecasts were validated over the period 1982-1993). The results also suggested that the neural network models were viable for ENSO forecasting even at longer leads of 9 to 12 months leads. The advantage of neural network models over linear statistical models (e.g. the C C A ) appears to lie in the longer lead time regime of 9 to 12 months. The result shows that the ability of a simple linear model in relating the wind stress and the Nino 3.4 SSTA declined Chapter 3. Forecasting the Nino 3.4 SSTA 49 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 Observation SST EOFs Wind stress EOFs Figure 3.15: The standardized SST anomalies of observations (solid curve) versus fore-casts at 0-month lead for a model using SST EOFs (dashed curve) and a model using wind stress EOFs (dotted curve) as predictors. Chapter 3. Forecasting the Nino 3.4 SSTA 50 as lead time increased. At 9 to 12-month leads, the simple linear regression model could not fit the Nino 3.4 SSTA in the training data well. On the other hand, neural network models explained a significant fraction of the variability in Nino 3.4 SSTA at these longer leads and performed well in forecasting the validation data. This comparison leads to a hypothesis that the underlying relationship between the wind stress and Nino 3.4 SSTA becomes increasingly nonlinear as the lead time increases. This nonlinearity becomes possible since many processes could affect the relation at longer lead times. Zebiak and Cane [1987] argue that many processes come into play in relating the thermocline depth to the surface wind stress, and others act somewhat independently, e.g. zonal temperature advection in the oceanic surface layer and moisture convergence effects in the atmosphere. Despite the ability of neural network models in simulating nonlinear relationships, there is no easy way to interpret the physical meaning of the weights in the hidden and output layers of the network. In the C C A model, however, this can be done rather easily by examining its canonical loading patterns. Also, the issue of confidence intervals for the neural network forecasts was not addressed, as there is no easy way of estimating them. A recent study by Chryssoloris et al. [1996] indicated the possibility of estimating the intervals. The model forecasts were also interpreted in light of current dynamical theories, e.g. the role of off-equatorial baroclinic Rossby waves in triggering the onset of an ENSO event, as proposed by White et al. [1989], and the delayed oscillator theory with respect to the development of the event from the peak through the mature phase (positive Kelvin wave effect) and the termination of the event (delayed negative Rossby wave effect). Chapter 4 Forecasting regional SSTA in the equatorial Pacific 4.1 Introduction As a sequel to Tangang et al. [1997a], in this chapter, three new investigations are described: (1) the relative importance of the sea level pressure (SLP) field versus the wind stress field as a predictor for the equatorial Pacific SSTA in neural network models; (2) the forecast skills attainable by the neural network models over various standard regions of the equatorial Pacific (Niiio 3, Niiio 3.4, Niiio 3.5, Niiio 4, P2, P4 and P5); and (3) the use of ensemble averaging to improve the somewhat noisy neural network forecasts made by Tangang et al. [1997a]. The results in this chapter have been submitted for publication, and are hereafter referred as to Tangang et al. [1997b]. The rest of the chapter is organized as follows. Section 4.2 decribes the data and the preliminary analyses, Section 4.3 outlines the model design, and Section 4.4 presents the forecast results. Section 4.5 gives a summary and conclusion. 4.2 The data and preliminary analyses Figure 4.1 shows the regions where the sea surface temperature anomaly (SSTA) forecasts are made. The name and location of each region, except for Niiio 3.5, are the same as these defined in Barnston and Ropelewski [1992, Table 1]. The Nino 3.5 is a new region proposed by Trenberth and Hoar [1996]. This region covers the area from 120°W to 180° and 10°S to 5°N, which straddles part of the Nino 3 and Nino 4 regions and extends 51 Chapter 4. Forecasting regional SSTA in the equatorial Pacific 52 farther into the Southern Hemisphere. The SSTA for each region was calculated by first averaging the sea surface temperature (SST) in the 2° by 2° boxes contained in each region. The climatological means were then removed and the anomalies were smoothed with a 3-point running average in time. The SST data were obtained from a combination of Smith's SST and Reynolds' SST data sets. The Smith's SST data were reconstructed using the empirical orthogonal functions (EOF) method [Smith et al, 1996] and were available in 2° latitude by 2° longitude grids over the period from 1950-1992. The Reynolds' SST data were derived from the in situ (ship and buoy) and satellite SST using optimum interpolation [Reynolds and Smith, 1994]. This data set is available in 1° latitude by 1° longitude grids and over the period from 1982 to the present. The combined SST used in this study comprises Smith's SSTs from 1952 -1992 and the averaged 2° x 2° 1993 Reynolds' SSTs. Prior to combining these two data sets, preliminary analyses were performed to determine their compatibilty over the overlapping period of 1982 - 1992. These analyses include the calculation of the climatology for the equatorial Pacific (20°S - 20°N) and the SSTA indices of the regions of interest (Fig. 4.1). The results confirm that the climatologies (not shown) are almost identical and the indices are highly correlated (0.98 - 0.99). A comparison between the SSTA indices calculated from this combined data set and those calculated from the comprehensive ocean-atmosphere data set (COADS) [Woodruff et al, 1987;1993] shows that the indices are highly correlated (0.94 - 0.99). The SLP and wind stress data from COADS are available in 2° latitude by 2° longitude grids. The chosen wind field covers a region of 20°S - 20°N and 120°E - 70°W of the tropical Pacific. This region is latitudinally wide enough to include the off-equatorial precursors for forecasting the SSTA [Tangang et al, 1997a]. Prior to the E O F analysis, the wind components were averaged to coarser grids of 2° x 10° and the wind stress field was calculated according to the formula in Trenberth et al [1989] and Large and Chapter 4. Forecasting regional SSTA in the equatorial Pacific 120°E 160°E 160°W 120°W 80°W LONGITUDE Figure 4.1: The location of the seven regions of interest. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 54 M O D E Wind SLP % Var Cum % Var Cum 1 16.8 16.8 36.1 36.1 2 12.2 29.0 13.7 49.8 3 8.1 37.1 7.1 56.9 4 6.2 43.3 4.8 61.7 5 4.2 47.5 4.5 66.2 6 3.7 51.2 3.8 70.0 7 2.8 54.0 3.0 73.0 Table 4.1: The percentage of variation explained by the first 7 E O F modes. Pond [1981]. The wind stress anomaly at each grid was calculated by removing the climatological means, and smoothed by a 3-point running average in time and a 1-2-1 filter in both spatial dimensions prior to the E O F analysis. The resulting EOFs are similar to those presented in Tangang et al. [1997a], except that the percentages of variance explained are higher. (In Tangang et al. [1997a] the wind stress data covered the period from 1952 - 1992, the wind components were averaged to 2° x 6° and no filtering was done prior to E O F analysis). Hence the smoothing procedure increases the percentage of explained variance. Table 4.1 gives the percentage of explained variance for the first 7 EOFs. Fig. 4.2 shows the spatial and temporal patterns for the first mode of the wind stress E O F . The vectors in the map correspond to the correlation values between the temporal coefficient and the zonal and meridional components of the wind stress anomaly field. This mode is dominated by an increasing decadal trend plus some ENSO signals. Wang [1995] also observed a trend which distinguished the pre- and post-1977 ENSO events. For the second mode (Fig. 4.3), the spatial pattern indicates large anomalous westerly winds along the equator in the western-central region of the tropical Pacific. This feature is associated with the ENSO, as indicated by the temporal coefficient. Higher modes have no simple physical interpretation. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 55 MODE 1 EOF of Wind Stress (16.8 perc.) Figure 4.2: First E O F mode of the wind stress field. The temporal amplitude shows a decadal trend plus ENSO signals. The vectors correspond to the correlations between the temporal amplitude and the zonal and meridional components of the wind stress anomaly field. Both the spatial and temporal patterns are similar to Fig. 3.1, with the exception of larger magnitude of the vectors and enhanced westerlies in the western-central equatorial Pacific. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 56 MODE 2 EOF of Wind Stress (12.2 perc.) 20°N H 10°N 10°S 20°S ' ' > y 1, /, /, / 140°E 180° 140°W LONGITUDE 100°W 1.00 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 Figure 4.3: Second E O F mode of the wind stress field. Both the spatial and temporal patterns are similar to Fig. 3.2, with the exception of stronger westerlies. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 57 The SLP data covers a wider region, 20°N - 30°S and 50°E - 90°W, which includes both the tropical Pacific and Indian Oceans. Prior to E O F analysis, the SLP fields were also averaged to coarser grids of 4° x 10°. The anomalies were calculated by removing the climatological means at each grid point, and then smoothed by a 3-point running average in time and a 1-2-1 filter in both spatial dimensions. Table 4.1 also shows the percentage of explained variance for the first seven SLP modes. Despite the larger domain of the SLP fields, the percentage of explained variance of the SLP EOFs is higher than those of the wind stress EOFs, indicating that the wind stress data are noisier than the SLP data. A similar result was observed in Graham et al. [1987a], where their first SLP E O F accounted for 14.6 % of the total variance versus 4.8% for the wind stress E O F . Fig. 4.4 shows the first mode of the SLP E O F . The contour values represent the correlation values between the temporal E O F coefficients and the SLP anomaly at each grid point. The spatial pattern clearly shows the ENSO pattern, with the positive center in the Asian-Australian region and the opposite (negative) center located in the southeastern tropical Pacific. The temporal coefficient also clearly indicates ENSO signatures with high values during E l Nino and low ones during La Nina events. This time series is also highly correlated with the temporal coefficient of mode 2 of the wind stress E O F (0.75). Despite differences in temporal and spatial coverage, the first E O F is remarkably similar to the first near-global SLP E O F of Graham et al. [1987a]. The spatial pattern of the second mode (Fig. 4.5) displays a negative center in the central tropical Pacific between 0-15°N. The temporal coefficients of this mode show no clear association with ENSO. The higher modes also have no simple physical intrepretation. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 58 MODE 1 EOF of SLP (36.0 perc.) 60°E 100°E 140°E 180° 140°W 100°W LONGITUDE 1952 1954 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 Figure 4.4: First E O F mode of the SLP field. Both the spatial and temporal patterns indicate ENSO pattern. The contours represent the correlations between the temporal amplitude and the SLP anomaly field at each grid point. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 59 MODE 2 EOF of SLP(13.7 perc.) 60°E 100°E 140°E 180° 140°W 100°W L O N G I T U D E 1 6. —I I l I I I I I i I I I I I I I I I I I I I i I I I I I i _ j 1 I I I I I I I I 1 I L — 16. —\ i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i r 1952 1954 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 Figure 4.5: Second E O F mode of the SLP field. The spatial pattern indicates a low negative center in the central tropical Pacific between 0 - 15°N. The amplitude shows no clear association with ENSO events. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 60 4.3 Model Designs The same model design used in Tangang et al. [1997a] will be used here. Fig. 3.3 shows the predictor-predictand setting for forecasting the onset (November of year -1 to January of year 0), peak (March - May of year 0) and mature (December of year 0 to February of year +1) phases of an E l Niiio event at 6-month lead. Tangang et al. [1997a, Fig. 3.12] found that the forecast skills were higher using 12 months of predictor data. This longer duration ensured the inclusion of the equatorial Kelvin and Rossby wave activity associated with ENSO. Also, following Tangang et al. [1997a, Figs. 3.11 and 3.13], the first 7 EOFs in four 3-month periods (SLP or wind stress) were used as the predictor, with 3 neurons in the hidden layer. These 7 EOFs accounted for 54.0% of the total variance for the wind stress field and 73% for the SLP field. The SSTA was also included in the final predictor period. Barnston and Ropelewski [1992], and Tangang et al. [1997a, Fig. 3.14] found that there was a slight increase in the forecast skills when the SSTA persistence was included in the predictor sets. Hence there is a total of 29 predictors- 7x4 from the wind stress or SLP EOFs and 1 from the SSTA persistence. As in Tangang et al. [1997a], the values a\ and a 4 in Eq.( 2.7) were fixed to 1 and 100, respectively. Tangang et al. [1997a] showed that the relationship between the wind stress and the SSTA tended to be increasingly nonlinear as lead time increased. Hence in this study the value of a2 was set to 0.1 and 03 was linearly increased with lead time from 2.0 at 0 month lead to 6.0 at 15 months. In estimating the forecast skills, two methods were employed: the retroactive and the cross-validation methods. The first method allows no data occurring later than the forecast time to be made available in building the model [Barnston et al., 1994]. The networks were trained similarly as in Tangang et al. [1997a] using data from 1952 - 1981 and making the forecasts for the 1982 - 1993 period. The cross-validation method, on the Chapter 4. Forecasting regional SSTA in the equatorial Pacific 61 other hand, was implemented by withholding the data each year in turn from the record during network training and making forecasts for the withheld period [Michaelsen, 1987]. Skill inflation can occur with interannually autocorrelated data, as in the case of ENSO time series. Barnston et al. [1994] suggested correcting this problem by withholding groups of consecutive years from the data record. In this chapter, an experiment was performed to compare the results of withholding groups of 10, 5 and 3 consecutive years. As found by Tangang et al. [1997a] and others [e.g. Kolen and Pollack, 1991; Grieger and Latif, 1994], the forecast results were somewhat sensitive to initial random weights used in the backpropagation algorithm. This was more pronounced for longer lead times. As a remedy, the forecast skills were ensemble averaged from 20 individual forecasts, each with different random initialization of the weight and bias parameters. 4.4 Results 4.4.1 Retroactive forecasts The skills of retroactive forecasts for the various SSTA regions for the period of 1982-1993 were obtained with the SLP (Fig. 4.6) and wind stress (Fig. 4.7) as predictors. These skills represent the mean values calculated from 20 individual forecasts, with the error bars indicating ± 1 standard deviation. Both figures show that the Nino 4, Nino 3.4 and Nino 3.5 regions are best forecasted, with the Nino 4 region having the highest correlations. The Nino 3 region appears less well forecasted, while the eastern regions of P4 and P5 have minimal skill. Earlier studies also showed low skill in the eastern part of the equatorial Pacific [Graham et al., 1987b; Barnett, 1984]. No skill was found for the western region of P2, hence it is excluded from both Figs. 4.6 and 4.7. Such spatial distributions of the forecast skills were also observed by Barnston and Ropelewski, [1992]. Their forecast skills, however, showed that the Nino 3.4 region (they referred to Chapter 4. Forecasting regional SSTA in the equatorial Pacific 62 this region as P3) appeared to be the best forecasted region. The forecast skills for Nino 3.4 using the wind stress predictor are comparable to the skills presented in Tangang et al. [1997a]. The skills for the regions in the eastern equatorial Pacific tend to decline rapidly at 3 to 6 month leads but increase as lead time increases, particularly for the forecasts with the SLP as the predictor. This tendency is also somewhat visible for the Nino 3 region. In contrast, the skills for the western-central regions (Niiio 4 and Niiio 3.4) do not show such behavior. This tendency might be explained by the delay in the response, in the eastern regions, to equatorial Kelvin waves generated in the west. Wang [1995] indicated that the post-1977 ENSO events had the initial warming in western-central equatorial Pacific, and the anomaly spread to the eastern equatorial Pacific. For 0-month lead forecast, the skill shown for the eastern regions could come from the persistence (i.e. the inclusion of the SSTA itself in the final predictor period). This persistence became less relevant when the lead time increased, as found by Tangang et al. [1997a]. As a result, the skills declined at 3 to 6 months leads, particularly for the forecasts using the SLP as predictor. As the responses in the eastern Pacific developed, the forecasts began to improve to the point where the skills were roughly the same for all the regions at lead times of 12-15 months. Figs. 4.8 - 4.10 compare the forecast skills of models using the SLP predictor and the wind stress predictor for the Nino 4, Niiio 3.5 and P4 regions. These regions roughly represent the western-central, the central and the eastern parts of the equatorial Pacific Ocean. For the Niiio 4 and Niiio 3.5 regions, the skills appear to be superior to persistence except at very short lead times. It also appears that the skills of the wind stress model are slightly higher than those the SLP model for shorter lead times of 0 to 3 months. However, the differences might not be statistically significant. At 6-month lead, the skills for* both models are comparable. As the lead time increases beyond 6 months, the SLP Chapter 4. Forecasting regional SSTA in the equatorial Pacific 63 • Nino 4 • Nino 3 L E A D T I M E ( M O N T H ) Figure 4.6: The mean correlation skills of retroactive forecasts (1982 - 1993) for the SSTA of each region in Fig. 4.1, using SLP as predictor. The vertical lines indicate ± 1 standard deviation. No skill was found for the P2 region, hence it is not included. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 64 L E A D T I M E ( M O N T H ) Figure 4.7: As in Fig. 4.6, but for the wind stress as predictor. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 65 model appears to have better skill. Why is the SLP a better predictor than the wind stress at long lead times, but vice versa at shorter lead times? Even though the ocean is more directly forced by the wind stress than by the SLP, and hence one would expect wind stress to be the better predictor, the SLP does have the advantage of being a less noisy variable than wind stress (e.g. the first 7 E O F modes for the SLP explained 73% of the variance versus 54% for the wind stress). The presence of noise in the initial conditions invariably causes the forecast to diverge from the true state at long lead times, hence it is not surprising that the wind stress becomes less effective than the SLP at long lead times. This result suggests that it is probably better to use the wind stress model for shorter lead forecasts (6 months or less) and the SLP model for longer leads (6 months or longer). Graham et al. [1987b] also designed their models which used wind stress as predictor for shorter leads and the SLP field for longer leads. The forecasts skills for the P4 region (Fig. 4.10) are not as good as the skills for the western-central tropical Pacific regions. Both models fail to beat persistence at 0-month lead forecast. The forecast skills of the wind stress model are much better than the SLP model at 3 to 9 months forecasts. Such a mismatch may be due to the inclusion of the 'local wind' effect in the wind stress data. Barnett [1984] stressed that the local characteristics of the wind field appeared to be important, particularly in the spindown phase of the warm event. Both models also show a trend of increasing skill (more pronounced in the SLP model) as lead time increased from 6 to 12 months. Figs. 4.11 and 4.12 display the ensemble averaged forecasts versus observations of the SSTA in the Nino 4 region at 3, 6, 9 and 12-month leads for the SLP and wind stress models, respectively. The correlations shown here are not identical to those shown in Fig. 4.8. Since the values in Fig. 4.8 are the mean of 20 individual correlations, they appear 0.02 - 0.04 smaller than those in Figs. 4.11 and 4.12. At 3-month lead, both Chapter 4. Forecasting regional SSTA in the equatorial Pacific 66 L E A D T I M E ( M O N T H ) Figure 4.8: Comparison between the forecast skills of the SLP and wind stress models for the Nino 4 region. Persistence correlations were also plotted, but with negative values omitted. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 6 7 o . s h T c O 0 .6 o O 0.4 0 .2 rn+i 1 T T • SLP EOF + SSTA • Wind EOF + SSTA E3 Persistence 1 2 L E A D T I M E ( M O N T H ) 1 5 Figure 4.9: As in Fig. 4.8, but for the Niiio 3.5 region. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 68 0.8h 0.6 0.4 0.2 i i T SLP EOF + SSTA Wind EOF + SSTA Persistence 1 2 L E A D T I M E ( M O N T H ) 1 5 Figure 4.10: As in Fig. 4.8, but for the P4 region. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 69 models forecast the 1982/83 and 1986/87 E l Ninos. There is, however, a slight delay of the 1982/83 forecast for the wind stress model, which was also observed in Tangang et al. [1997a]. The initial rise of the SSTA for the 1986/87 event is also forecasted somewhat early by the SLP model. Both models forecast a cold event in 1984, with similar forecasts also appearing in Tangang et al. [1997a] and Barnston and Ropelewski [1992]. Both forecast the 1988 La Nina but with smaller amplitudes than observed. The SLP model does not track the SSTA evolution from 1990-1993 well. In contrast, the wind stress model forecasts the 1988 La Nina and the 1991/92 E l Niiio better. The model, however, gives a false forecast of a cold event in 1992/93. Overall, the forecasts from the wind stress model are slightly better than the forecasts from the SLP model (correlation skill of 0.80 for wind stress model versus 0.76 for the SLP model). At 6-month lead, the forecasts from the SLP model for the 1991/92 event are slightly improved. The wind model forecasts for the 1982/83 E l Nino are delayed, but they capture the 1986/87 event rather well. The SSTA evolutions in 1990-92 are better tracked by the wind model than by the SLP model. However, the wind stress model gives erroneous forecasts of cold SSTA in 1992/93. Some of the models surveyed by Barnston et al. [1994] had a similar problem in their Nino 3.4 SSTA forecasts. Overall, the forecasts from the SLP model are marginally better than the wind stress forecasts. This 6-month lead seems to indicate the turning point where the SLP as predictor started yielding better forecast skills than the wind stress. Indeed, at 9-month lead, the SLP model gives better forecasts than the wind stress model with a correlation of 0.77 versus 0.67. As in the 6-month lead forecasts, the wind stress model gives late forecasts for the 1982/83 E l Nino and a false cold event forecast in 1992/93. However, the SLP model also gives a slightly delayed forecast for the 1982/83 event. Tangang et al. [1997a] attributed the failure of the wind model at the 1982/83 event to the short appearance time of the off-equatorial winds which triggered this event. The short appearance means that the wind Chapter 4. Forecasting regional SSTA in the equatorial Pacific 70 (3 -month lead)(r=0.76) 2.0 I 1 ' ' ' ' ' ' ' ' ' i i I I I I I I I I 1982 1984 1986 1988 1990 1992 (6 -month lead)(r=0.75) 2 o I ' ' i i i i i i i ' ' - 3 . 0 I i i i i i i i i i i i 1982 1984 1986 1988 1990 1992 Observations NN forecasts Figure 4.11: Standardized SST anomalies of the neural net forecasts (dashed curves, averaged from 20 individual forecasts) for the Nino 4 SSTA compared with the observed anomalies (solid curves) during 1982-1993 at lead times of 3, 6, 9 and 12 months, with the SLP as predictor. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 71 ( 9 - m o n t h lead)(r = 0.77) 1982 1984 1986 i i • 1990 1992 ( 1 2 - m o n t h lead)(r=0.73) 1982 1984 1988 Observations 1990 1992 NN forecasts Figure 4.11: (Cont.) Chapter 4. Forecasting regional SSTA in the equatorial Pacific 72 ( 3 - m o n t h lead)(r=0.80) 2.0 I ' ' i i i i i i ' ' ' i i i i i: i i i i I 1982 1984 1986 1988 1990 1992 ( 6 - m o n t h lead)(r = 0.72) 2.0 I ' ' ' ' i ' ' i i i i 1982 1984 1986 1988 1990 1992 Observations NN forecasts Figure 4.12: As in Fig. 4.11, but for the wind stress as predictor. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 73 ( 9 - m o n t h l e a d ) ( r = 0 . 6 7 ) 2.0 I 1 ' ' ' ' ' I 1982 1984 1986 1988 1990 1992 ( 1 2 - m o n t h lead)(r=0.61) 2.0 - 3 . 0 I i i i i i i i i i i i 1982 1984 1986 1988 1990 1992 Observations NN forecasts Figure 4.12: (Cont.) Chapter 4. Forecasting regional SSTA in the equatorial Pacific 74 data do not contain enough precursor patterns for forecasting at the longer leads of 9 to 12 months. The SLP data, which covers both the tropical Indian and Pacific Oceans, would likely contain more precursor patterns. Several studies indicated that the atmospheric variability, which eventually caused most of the changes in the SST associated with ENSO, originated from the equatorial Indian Ocean and propagated eastward into the Pacific [Bamett, 1985; Yasunari, 1990; Gutzler and Harrison, 1987]. At 12-month lead, the SLP model forecasts are only marginally worse than those at 9-month lead. The wind model continues giving delayed 1982/83 E l Niiio forecasts, and a false forecast of a cold event in 1992/93, with an overall correlation skill of 0.61, much lower than the correlation of 0.73 attained by the SLP model. The forecasts for the Nino 3.5 SSTA resembled those of Nino 4. For the eastern P4 region, the SLP model forecasts are worse than those of the Nino 4 region, with corre-lation skills of 0.43, 0.30, 0.45 and 0.57 for the 3, 6, 9 and 12-month leads, respectively. Interestingly, the correlations between the forecasted P4 SSTA and the observed Niiio 4 SSTA (i.e. 0.64, 0.60, 0.72, 0.70 for the 3, 6, 9 and 12-month leads, respectively) are higher than the correlations between the forecasted and the observed values of the P4 SSTA. This seems to indicate that the models, given the information in the predictor field, tries to produce the anomalies in the western-central region. In other words, the model expects the SSTA evolution in the western-central and eastern equatorial Pacific to be similar. Wang [1995] noted that this was not the case, as no initial warming took place in the eastern equatorial Pacific in the forecast period. In fact, there appeared only one direction for anomaly propagation, i.e. from west to east. The forecasts for the P4 region (not shown) reveal that, for shorter leads, there is a mismatch between the phases of the forecasts and the observations, with the forecasts leading the observations. The phases become better matched as lead time increases, which one would expect as the longer lead Chapter 4. Forecasting regional SSTA in the equatorial Pacific 75 time would give enough time for the SSTA to propagate to the eastern equatorial Pacific. 4.4.2 Cross-validated forecasts This section presents the forecast skills for various decades which roughly represent the 1950s, 1960s, 1970s and 1980s. The forecast skills in the 1950s were based on the models trained using the data in the 1960s, 1970s and 1980s. A similar setup was adopted for the other decades. This is basically a cross-validation forecasting method, by which four groups of approximately 10 consecutive years were withheld one at a time. First, the skill for each decade is considered, with Figs. 4.13 and 4.14 showing the forecast skills for the SLP and wind stress models, respectively, for the Nino 4 region. The skills for the Nino 3.5 region are shown in Figs. 4.15 and 4.16. The forecast skills for the SLP model for the 1950s and 1980s (Figs. 4.13 and 4.15) are comparable, with useful skills up to a 15 month lead time. In contrast, the skills for the 1960s and 1970s are lower than the other two decades and decay rapidly. For the wind model (Figs. 4.14 and 4.16) the skills for the 1950s decay rapidly, as for the 1960s and 1970s, consistent with Tangang et al. [1997a]. In summary: (1) the forecast skills for the wind model in the 1950s are much lower than those of the SLP model, and (2) the skills for the 1950s and 1980s are higher than for the 1960s and 1970s for the SLP model. The first point might be caused by noisy wind stress data, as the wind data in the 1950s could be of much poorer quality than the SLP data. The trend toward higher skills in the 80s than in the 60s and 70s is consistent with other studies. It has been highlighted in several studies that the forecast skills of ENSO models have decadal dependency. Chen et al. [1995] showed that their forecast skills were higher if the forecast period included the 80s and 90s. Their forecast skills seemed to be lower for the 70s. They attributed the lower skills for the 70s to poorer data quality and lower signal-to-noise ratio during this period. A similar trend was observed in Tangang et al. [1997a], where forecast Chapter 4. Forecasting regional SSTA in the equatorial Pacific 76 L E A D T I M E ( M O N T H ) Figure 4.13: The forecast skills for various decades which roughly represent the 1950s, 1960s, 1970s and the 1980s for the Nino 4 region, using the SLP as predictor. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 77 L E A D T I M E ( M O N T H ) Figure 4.14: As in Fig. 4.13, but for the wind stress as predictor. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 78 T 0.8H .2 0.61 O 0.4| O 0.2 I . / A / A , \ \ \ / A IJ- •r v ^ 5 2-61 6 2-71 • 7 2 8 2 8 1 9 3 i 1 \ \ 4 l " V " V V 1 2 " V ^ V ^ V ^ V 1 5 L E A D T I M E ( M O N T H ) Figure 4.15: 1960s, 1970s The forecast skills for various decades which roughly represent the 1950s, and the 1980s for the Nino 3.5 region, using the SLP as predictor. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 79 L E A D T I M E ( M O N T H ) Figure 4.16: As in Fig. 4.15, but for the wind stress as predictor. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 80 skill was minimal for the 50s, 60s and the 70s. Tangang et al. [1997a] argued that the lower skills for the earlier decades could be due two factors: (1) the quality of the data in the 80s and 90s was better than the earlier decades, and (2) the ENSO activities were frequent and much stronger in the 80s, where strong warmings took place in the central equatorial Pacific. In contrast, the ENSO events in earlier decades tended to have warming further east than later events [Wang, 1995]. There is also evidence [Gill and Rasmusson, 1983; Kousky and Leetmaa, 1989] that the warm events during the 1980s were different in timing, compared to the canonical E l Nino event composited by Rasmusson and Carpenter [1982] for earlier decades. The higher skills in the 50s with the SLP model are rather surprising. Balmaseda et al. [1995] noted that the 1970s had more pronounced seasonal variations than the 50s and 80s. This is related to the degree of phase locking of an E l Nifio event to the annual cycle, in which a strong phase locking leads to a strong seasonal barrier in the forecasts. The weak seasonal barrier (and hence weak seasonality) in the 50s and 80s may contribute to higher overall skills for these periods. To further investigate this, the seasonal skills for the 1950s, 1960s and the 1970s were calculated and compared. The results (not shown) show less conclusive evidence of stronger seasonality in the 1960s and 1970s compared to the other two periods. The length of each record (i.e. 10 years for each period) might be too short to establish a seasonal trend. The differences in spatial and temporal evolution patterns between the pre-1977 and post-1977 ENSO events may also affect the forecasting skills in the eastern regions. Wang [1995, Fig. 12] noted that there was no coastal warming in the eastern equatorial Pacific for the post-1977 events in the development phase (Feb(0) - May(0)). The subsequent development in the transition phase (Jun(O)-Oct(O)) involved only the eastward propaga-tion of the western SSTA. In contrast, the pre-1977 ENSO events had a strong warming Chapter 4. Forecasting regional SSTA in the equatorial Pacific 81 in the coastal region of the eastern equatorial Pacific in the development phase. The sub-sequent development involved both the eastward propagation of the equatorial western SSTA and westward propagation of the eastern SSTA. Hence one would expect higher forecast skills for the western-central regions than the eastern regions of the equatorial Pacific for the post-1977 events. Also, as it takes some time for the SSTA to propagate from the western to the eastern equatorial region, one would expect that the skills in the eastern region would increase with lead time (as noted in Figs. 4.6 and 4.7). One would expect the forecast skills in the eastern equatorial Pacific for the pre-1977 period to be higher than the skills for the post-1977 period. Also, there should not be any trend of increasing skills with lead time for the pre-1977 periods, as both the western-central and the eastern equatorial Pacific warmed during the initial stages. Fig. 4.17 shows the forecast skills of the SLP model for the P4 region at various decades. The skills for the 1950s appear consistent with this expectation, i.e. the skills are relatively high, and do not show any increasing trend as lead time increased. The skills for the 1960s appear stable for lead times 6 - 1 5 months, while in the 1970s the skills decrease with lead time. Both periods, however, have lower skills than the 1950s. The overall correlation was also calculated by combining the forecasts from all periods. These represent the cross-validated forecast skills. An investigation was also carried out to determine whether the number of withheld consecutive years in the cross-validation process would affect the cross-validated skills. The experiment was repeated twice by withholding 5 and 3 consecutive years of data, respectively. The results (not shown) show that the cross-validated forecast skills are very similar among the three experiments, hence the skills are not significantly affected by the number of years withheld in the cross-validation process. Fig. 4.18 shows the cross-validated skills for the Nino 4. The SLP model has higher skills than the wind stress model for lead time of 6 months or longer. Similar skills are obtained for the Nino 3.5 region (not shown). For the P4 region (not Chapter 4. Forecasting regional SSTA in the equatorial Pacific 82 0.8-X .2 0 .6H 0.4H 0.2H / A 5 2-61 6 2-71 •A • 7 2 8 2 8 1 9 3 1 2 St AS 1 5 L E A D T I M E ( M O N T H ) Figure 4.17: The forecast skills for various decades which roughly represent the 1950s, 1960s, 1970s and the 1980s for the P4 region, using the SLP as predictor. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 83 shown), persistence appears to be a tough competitor for both models for lead times 3 months or shorter. For 6 months or longer, both models have low cross-validated skills. The cross-validated seasonal skill was also calculated. For the Nino 3.5 region (Fig. 4.19), the winter season has the highest skills at longer leads, with the fall season perhaps second best. A similar trend is observed for the Nino 4 region (not shown). The monthly cross-validated skills for the Nino 3.4 region, using the SLP as predictor, are also plotted (Fig. 4.20). These skills can be compared to those of Barnston and Ropelewski [1992, Figs.7a,b]. The skills at 0 and 3-month lead forecasts are similar to the corresponding skills of Barnston and Ropelewski [1992] for all seasons, except during summer season when the neural network model appears to have better skill. Their C C A skills at 3 months lead (i.e. equivalent to one season) in the summer were between 0.30 - 0.40. In contrast, the neural network skills are between 0.50 - 0.60. Also, for the 6-month forecasts, there were no skills for the C C A except in winter months, where the correlations were exceptionally high (> 0.70). The neural network forecast skills, however, appears to be stable all year round with the correlation between 0.50 - 0.60. There appeared to be no skills for the C C A at 9 and 12 months leads, whereas the neural network model shows usefull skill in the winter and fall months, with the correlations between 0.50 - 0.60. Overall, the skills presented here appear to offer some improvements over the C C A , particularly at longer lead times. 4.5 Summary and conclusion Two types of neural network models were constructed and compared for forecasting the SSTA in various regions of the equatorial Pacific Ocean, one using the wind stress field as predictor, and the other using the SLP field. The wind stress models were designed to include the space and time domains of the equatorial and the off-equatorial waves Chapter 4. Forecasting regional SSTA in the equatorial Pacific 84 L E A D T I M E ( M O N T H ) Figure 4.18: The cross-validated skills (based on withholding 10 consecutive years in the cross-validation process) using the SLP or wind stress as predictor for the Nino 4 region. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 85 L E A D T I M E ( M O N T H ) Figure 4.19: The seasonal cross-validated skills (based on withholding 10 consecutive years in the cross-validation process) using the SLP as predictor for the Nino 3.5 region. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 86 1 1 T o 1 2 - m o n t h F E B A P R J U N A U G O C T D E C M O N T H Figure 4.20: The monthly cross-validated skills (based on withholding 10 consecutive years in the cross-validation process) using the SLP as predictor, for the Nino 3.4 region, for forecasts of 0 - 12 months lead. Chapter 4. Forecasting regional SSTA in the equatorial Pacific 87 relevant to ENSO, as outlined in the delayed-oscillator theory [e.g. Battisti and Hirst, 1989; Graham and White, 1988]. The SLP models were constructed to encompass a larger spatial domain, which included the tropical Indian and Pacific Oceans. By ensemble averaging over 20 forecasts with different random initial weights, the resulting forecasts were less noisy than those in Tangang et al. [1997a]. At shorter leads, the wind stress models had slightly better skill than the SLP models for the 1982-93 forecasting period. At longer leads, the wind stress models also produced useful forecasting skills for the 1982-93 period, consistent with Tangang et al. [1997a]. Both the retroactive and cross-validated forecast skills showed that although the SLP models generally had less skills than the wind stress models at shorter leads, the SLP models delivered better forecasting skills than the wind stress models at lead times of 6 months or longer. Only at the eastern boundary region of P4 was this reversal of skill at long lead time not observed. Why did the SLP models deliver better skills at longer leads? While the ocean is more driven by the wind stress than by the SLP, the SLP data have the advantage of being less noisy than the wind stress. The presence of noise causes the forecast to diverge from the true state at long lead times, hence the drop in skill for the wind stress models. Also, the SLP had a wider spatial coverage which included both the equatorial Indian and Pacific Oceans. The higher skill was consistent with the idea that the atmospheric variability, responsible for the SST changes associated with ENSO, originated from the equatorial Indian Ocean and propagated eastward into the Pacific [e.g. Barnett, 1985; Yasunari, 1990; Gutzler and Harrison, 1987]. The spatial distribution of the forecast skill indicated higher skills in the western-central regions of the equatorial Pacific. The Niiio 4 had the highest skill, and the lowest skill was in the eastern boundary regions, consistent with Graham et al. [1987b] and Barnston and Ropelewski [1992]. The forecasting skills during the 1982-1993 period Chapter 4. Forecasting regional SSTA in the equatorial Pacific 88 in the eastern boundary regions appeared to increase with lead time. This trend may be explained by the delay in the response in these regions to equatorial Kelvin waves generated in the west. Wang [1995] noted there was only one direction for anomaly propagation, i.e. eastward, as no initial warming took place in these regions during the post-1977 ENSO events. The lower forecasting skills in these regions, particularly for the SLP model, may also be due to local effects not included in the predictors [e.g. Barnett, 1984]. Chapter 5 Forecasting regional SSTA: Neural network - extended E O F approach 5.1 Introduction Tangang et al. [1997a] forecasted the SSTA in the Nino 3.4 region with neural network models using wind stress as predictor. Tangang et al. [1997b] broadened the investigation by: (1) comparing the SSTA forecasts skills over various standard regions (Nino 3, 3.4, 3.5, 4 etc.) in the tropical Pacific, (2) comparing the sea level pressure (SLP) versus the wind stress as predictor fields, where the SLP emerged as the better predictor at long lead times, and (3) using an ensemble average of 20 forecasts with different initial random weights to greatly reduce the fluctuations in the forecasts, resulting in improved skill. Despite the encouraging results, the models in these earlier works were rather large (with many degrees of freedom), rendering an interpretation of the inner workings of the models most difficult. This is a common problem with neural network modelling. In this chapter, the objective is to greatly simplify and clarify the neural network models by 4 new initiatives: (1) The extended empirical orthogonal functions (EEOF) of the SLP were used as input to the neural networks, which turned out to be much more effective in compressing the predictor fields than the empirical orthogonal function (EOF) method used in the earlier models. As the E E O F allows temporal lag compression in addition to space compression, this enabled a much smaller network to be employed. (2) In addition to forecasting SSTA of discrete regions in the equatorial Pacific, the forecast scheme 89 Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 90 was extended to forecasting the first four dominant modes of equatorial Pacific SSTA E O F . From the forecasted modes, the forecasted SSTA field was reconstructed, and the results were compared to those forecasts for individual regions. (3) A pruning technique was introduced to trim the networks of unnecessary structure, which further reduced the size of the networks. (4) Spectral analysis was applied to better understand how the network functioned, particularly in relation to its nonlinearity. It was found that in the predictor-predict and relation, nonlinearity tended to increase with lead time, and to become stronger for the eastern regions of the equatorial Pacific. The results in this chapter have been accepted for publication, and are hereafter referred to as Tangang et al. [1997c]. The rest of this chapter is organized as follows: Section 5.2 describes the E E O F anal-ysis of the SLP field, while Section 5.3 briefly illustrates the neural network model design. Section 5.4 presents the forecast results, Section 5.5 describes the network pruning, and Section 5.6 descrices spectral analysis of the networks. 5.2 Preliminary data processing The same SST and SLP data used in Tangang et al., [1997b] are used in this chapter. Prior to the E E O F analysis, the SLP fields were averaged to coarser grids of 4° by 10° from the 2° by 2° COADS grids. The anomalies were calculated by removing the climatological means at each grid point, followed by smoothing with a 3-point running average in time and a 1-2-1 filter in each spatial dimension. The use of the E E O F technique in the pre-processing of the SLP field allows lag as well as spatial compressions of the data field into several dominant modes [Weare and Nasstrom, 1982; Graham et al., 1987a]. This is particularly advantageous in relation to neural network modeling since fewer inputs are required. For example, the network requires only 7 inputs (if the first 7 modes were Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 91 used) for one year compression of the predictor data field, as oppossed to 28 inputs (i.e. 7 x 4 , for 4 3-month lags) if ordinary EOFs were used as in Tangang et al. [1997 a,b]. Following Tangang et al. [1997 a,b], the SLP data field that evolved over a period of one year was used as the main predictor. To characterize the evolution over this period, the E E O F at 3 month intervals was calculated for the period of 1952-93. This was achieved by 'stacking' the four temporal sequences of the SLP field (see Appendix A for details). The E E O F was then calculated by an E O F analysis on the covariance of the stacked matrix. The eigenvectors provided a sequence of 4 'snapshots' (each covering a 3-month period) characterizing the evolution of a spatial pattern over a one-year period. As was found in Graham et al. [1987a], the ENSO signal appeared to be divided between the first two modes, each displaying different, partially overlapping, portions of the SLP variations associated with ENSO. Fig. 5.1 shows the spatial patterns of the first mode, which accounted for 26.5% of the total variance. The contours represent the correlation values between the amplitudes (i.e. the temporal coefficients) and the SLP anomaly at each grid point. The spatial patterns clearly indicate an ENSO association, with a positive center of action in the Indian-Asian region and the opposite center (negative) located in the southeastern tropical Pacific. The patterns for the subsequent lags (i.e. lags 6, 3 and 0) resemble the spatial pattern for lag 9 with the intensification and slight eastward displacement of the Indian-Asian center. Hence these patterns characterized the seesaw oscillations between these two centers of action. The temporal coefficient shown in Fig. 5.2 also clearly indicates typical ENSO signatures with high values during E l Nifios and low values during La Ninas. The second mode (Fig. 5.3), which explained 10.4% of the total variance, also displays two centers, with, at lag 9, a high pressure cell centered in the southeastern equatorial Pacific and a low covering the Asian-Australian region. The patterns appear to propagate eastward as they evolve to the most recent lag (i.e. lag 0), at which time a complete Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 92 MODE 1 of SLP EEOF (26.5 perc.) 60°E 100°E 140°E 180° 140°W 100°W LONGITUDE Figure 5.1: The spatial pattern of the SLP E E O F Mode 1 at lags 9, 6, 3, and 0 months. The contours represent correlation between the E E O F amplitude and the SLP anomaly at each grid point. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 93 60°E 100°E 140°E 180° 140°W 100°W LONGITUDE 60°E 100°E 140°E 180° 140°W 100°W LONGITUDE Figure 5.1: (Cont.) Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 94 Figure 5.2: The amplitude for the SLP E E O F mode 1. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 95 reversal of the pressure pattern occurs (i.e. high in the west and low in the east, with resemblence to the first mode). This evolution seems to suggest that this mode was also associated with ENSO. Mode 1 has spatial and temporal patterns similar to the SLP E O F mode 1 in Tangang et al. [1997b]. Both the E O F and the E E O F produce similar results since both modes characterize the standing oscillation. However since the E O F analysis cannot capture propagating features with a single mode, the SLP E O F mode 2 in Tangang et al. [1997b] has different spatial and temporal patterns than the SLP E E O F mode 2. Interpretations of these modes will be given in a later section, particularly in relation to the quasi-biennial oscillation (QBO) and the low-frequency oscillation (LFO) described, e.g., in Barnett [1991], Ropelewski et al. [1992] and Goswami [1995]. 5.3 Model design The model design used here is similar to the previous models (see Fig. 3.3), with the exception of using the EEOFs. The first 7 modes of SLP EEOFs, which accounted for 53.2% of the total variance, plus the SSTA persistence were used as inputs. With 3 neurons in the hidden layer, the resulting networks were much smaller, with only 31 degrees of freedom (i.e. 8x3+3+3+1 d.f.) versus 94 in Tangang et al. [1997a,b]. As in Tangang et al. [1997b], the values of ai , a 2 , and a 4 in Eq.( 2.7) were fixed to 1, 0.1 and 100 respectively. However, since the size of the networks was smaller, a% was set to 2.0, a smaller value than that used in Tangang et al. [1997b]. Two methods were employed in estimating the forecast skills: the retroactive [Barnston et al., 1994] and cross-validation [Michaelsen, 1987] methods. As in Tangang et al. [1997b], the retroactive method was implemented by training the networks over the period of 1952 -1981 and used the subsequent period from 1982 to 1993 as forecast validation. For the cross-validation method, a group of 10 consecutive years was withheld in turn from the Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 96 MODE 2 of SLP EEOF (10.4 perc.) 60°E 100°E 140°E 180° 1 40°W . 100°W LONGITUDE 60°E 100°E 140°E 180° 140°W 100°W LONGITUDE Figure 5.3: The spatial pattern of the SLP E E O F Mode 2 at lags 9, 6, 3, and 0 months. The contours represent correlation between the E E O F amplitude and the SLP anomaly at each grid point. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 97 Figure 5.3: (Cont.) Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 98 1952 1954 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 Figure 5.4: The amplitude for the SLP E E O F mode 2. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 99 record, and the networks were trained with the rest of the data. Forecasts were then made for the withheld period. 5.4 Neural network forecasts 5.4.1 Forecasting SSTA of individual regions The mean correlation skills (averaged from 20 forecasts, each with different random initial weight initializations) of retroactive forecasts for the period of 1982 - 1993, for three regions, Niiio 4, Nino 3.5 and Niiio 3, are shown in Fig. 5.5, with the error baxs indicating ± 1 standard deviation of the 20 forecasts. The western-central region (Nino 4 and Nino 3.5) appears to be the best forecasted region, with Nino 4 having the highest skills. The central-eastern region of Nino 3 is less well forecasted, particularly for longer lead times. Fig. 5.6 shows the forecasts versus observations at 3, 6, 9 and 12-month lead time for the Nino 4 region. These were ensemble forecasts (averaged from the 20 forecasts), and hence the correlation skills of the ensemble forecasts shown were not identical to the ensemble average of individual forecast skills shown in Fig. 5.5, but were 0.01 - 0.04 higher. These skills are significant at the 95% level, assuming 19 independent realizations of forecast skills over the 12-year period. The assumption of 19 independent realizations is a rough approximation based on the decorrelation (autocorrelation e-folding) time of approximately 7.5 months of the observed SSTA. These forecasts resemble those presented in Fig. 4.11, with improvements over the period of 1990 - 1992. Fig. 5.7 shows the correlation skills for forecasting various decades for the western-central region of Niiio 4. The skills show strong decadal dependence, with higher skills in the 1950s and 1980s and lower skills in the 1960s and 1970s, similar to those in Fig. 4.13. Fig. 5.8 shows the cross-validated correlation skills calculated over all decades for the Chapter 5., Forecasting regional SSTA: Neural network - extended EOF approach 100 L E A D T I M E ( M O N T H ) Figure 5.5: The mean correlation skills for the forecast period of 1982-93, for Nino 4, Nino 3.5 and Nino 3 regions. The error bars indicate ± 1 standard deviation of the 20 forecasts. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 101 ( 3 - m o n t h lead)(r=0.80) 2 Q I ' i i ' i i i i i ' ' — 3.0 I i i i i i i i i " " i i | 1982 1984 1986 1988 1990 1992 ( 6 - m o n t h lead)(r = 0.76) i i i i i i i i i i 1982 1984 1986 1988 1990 1992 Observations NN forecasts Figure 5.6: Standardized SSTA of the neural net forecasts (dashed curves), averaged from 20 individual forecasts of Nino 4 region, compared with the observed anomalies (solid curves) for the forecast period of 1982-93 at lead times of 3, 6, 9 and 12 months. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 102 ( 9 - m o n t h lead)(r=0.75) 2.0 I ' ' i ' i i '••••i i ' i i i I I I I I I I 1982 1984 1986 1988 1990 1992 ( 1 2 - m o n t h lead)(r = 0.67) ' i i ' i i i i i i i i i i i i i 1982 1984 1986 1988 1990 1992 Observations NN forecasts Figure 5.6: (Cont.) Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 103 three regions. The cross-validated skills are higher in the western-central region than in the eastern region. The cross-validated seasonal skills for the western-central region of Nino 4 are shown in Fig. 5.9. The seasonality in the skills is apparent, with the winter season having the highest skills, useful up to 12 months lead. In general, for all regions, the smaller networks adopted here had similar skills as the larger networks in Tangang et al. [1997b]. 5.4.2 Forecasting EOFs of SSTA In this section, the SSTA field for the entire equatorial Pacific ocean (20°S - 20°N, 120°E - 70°W) was forecasted. This was done by first decomposing the SSTA field, T(x,t), using the E O F analysis, as T(x,t) = YJ<*n{t)en(x) (5.1) n where a and e represent the temporal and spatial coefficients. The forecasts for the first few dominant modes of the SSTA E O F were then made. From Eqs.( 2.1) and ( 2.2), the forecasted fc-th mode is given as + T) = Y.Wjtanh^wMt) + 6,.) + b (5.2) 3 * where ui{j, uij, bj, and b are the weights and biases, r is the lead time and (3%(t) is the temporal coefficient of the SLP E E O F mode i. The forecasted SSTA fields, T(x,t), were obtained by substituting d*. from Eq.( 5.2) into a*, in Eq.( 5.1). This approach of forecasting the dominant modes, and then reconstructing the SSTA field forecasts, was also implemented by Zhang et al. [1993], using a forecast method based on a temporal E O F iteration scheme. A similar network configuration used for forecasting individual regions was employed, with the first seven modes of SLP E E O F as inputs and 3 neurons in the hidden layer. The first four modes of the SST E O F were Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 104 L E A D T I M E ( M O N T H ) Figure 5.7: The forecast correlation skills for various decades, corresponding roughly to the 1950s, 1960s, 1970s, and the 1980s for the Nino 4 SSTA. The error bars indicate ± 1 standard deviation of the 20 forecasts. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 105 L E A D T I M E ( M O N T H ) Figure 5.8: The cross-validated correlation skills calculated over all decades for the Nino 4, Nino 3.5 and Nino 3 SSTA. The error bars indicate ± 1 standard deviation of the 20 forecasts. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 106 L E A D T I M E ( M O N T H ) Figure 5.9: The seasonal cross-validated correlation skills for the the Niiio 4 SSTA. The error bars indicate ± 1 standard deviation of the 20 forecasts. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 107 forecasted, with a different network for each mode. These four modes accounted for 82.4% of the total variance. Each network was trained using the data from the period of 1952 - 1981, and the trained network was used to forecast the corresponding mode for the period 1982 - 1993. A total of 20 forecasts (each with different weight initialization) were for each mode. The mean forecasts for each mode were calculated and the SSTA forecast patterns were constructed based on these mean forecasts. Figs. 5.10 - 5.12 show the spatial and temporal patterns for the first three EOFs of the SSTA. The contour values represent the correlation between the amplitude and the SSTA at each grid point. The first E O F , associated with ENSO events, accounted for 62.0% of the total variance. The spatial pattern of this mode indicates a warming in the central and eastern equatorial Pacific ocean, resembling the anomaly composite of Rasmusson and Carpenter [1982] during the transition and mature phases (refer to their Figs. 20a and 21a). This mode thus appears to be associated with the late development of ENSO events. The second E O F , which accounted 10.8% of the total variance, also shows an association with ENSO events. The spatial pattern indicates a warming in the eastern equatorial Pacific and a cooling in central equatorial Pacific. The cooling pattern also appears to be prominent between 10°N - 20°N in the central - eastern part of the tropical Pacific ocean. This mode resembles, to some degree, the anomaly composite of Rasmusson and Carpenter [1982] for the peak phase (refer to their Fig. 19a). The third E O F indicates a cooling in the central Pacific and a warming trend in the western Pacific, explaining 5.9% of the total variance. This spatial pattern resembles the onset phase anomaly composite of Rasmusson and Carpenter [1982] (see their Fig. 18a), hence may also be associated with an early development of ENSO events. The fourth and higher modes have no obvious physical intrepretation. These dominant patterns, particularly for the first two modes, are similar to those of much earlier studies [Weare et al., 1976; Weare, 1982]. Also, these first four modes are similar to those of the COADS SST Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 108 MODE 1 of SSTA EOF (62.0 perc.) 1952 1954 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 Figure 5.10: The spatial pattern and amplitude of SSTA E O F mode 1 of the equato-rial Pacific Ocean (20°S - 20°N). The contours represent the correlation between the amplitude and the SSTA at each grid point. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 109 Figure 5.11: As in Fig. 5.10 except for the second mode. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 110 J 1 1 1 I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I L T i i i i i i i i i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — r 1952 1954 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 Figure 5.12: As in Fig. 5.10 except for the third mode. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 111 (not shown), except that the CO ADS SST modes accounted for less variance. This is expected, since the Smith/Reynolds SST field is smoother than the COADS SST field [Reynolds and Smith, 1994 Smith et al, 1996] Fig. 5.13 shows the forecast correlation skills for lead times of 3, 6, 9 and 12 months for the period of 1982 - 1993. Most of the skill came from the first mode, with minimal skill in the other three. The lower skill for the second mode, which characterizes the warming in the eastern equatorial Pacific, is consistent with Tangang et al. [1997b]. For all leads, the western-central region of the equatorial Pacific appears to be the best forecasted region. The skill decreases eastward, with somewhat lower skill in the central-eastern region and minimal or no skill in the coastal eastern regions. Also, no skill in the far western region is found. The spatial distribution of the skill resembles the skill distribution from forecasting the individual regions presented earlier. Also, there appears to be an eastward 'propagation' of the skills as lead time increases. The correlation skills in the eastern region, at 3 months lead, are less than 0.4. The 6-months forecasts show better correlations, with the 0.6 contour having moved eastward, and the 0.4 contour expanded both southward and northward. At 9 months, the 0.6 contour covers much of the region within 5° of the equator of the eastern Pacific Ocean. Such an increasing trend of forecast skill was also noted in Tangang et al. [1997b] for the eastern region of P4. Tangang et al. [1997b] hypothesized that this trend was due to the delay of the Kelvin wave effects, generated from the west, to develop in the east. Such a trend is not observed in the western-region as the signals travel from west to east. Wang [1995] indicated that for the post-1977 ENSO, the SSTA propagation appeared to be one way, i.e. eastward. The initial warming was confined in the western-central equatorial Pacific, and the anomaly spread to the eastern part of the equatorial Pacific. Another region that appears to have better skill at shorter lead times is the western-northern region of 10°N - 20°N centered at 150°E. This is part of the south-westward Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 112 ( 3 - m o n t h lead) 140°E 180° 140°W . 100°W LONGITUDE Figure 5.13: The forecast correlation skills for 1982 - 1993 forecast period at lead times of 3, 6, 9, and 12 months. The SSTA forecasts were reconstructed from the forecasted first four modes of SSTA EOFs. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 113 (9—month lead) 140°E 180° 140°W 100°W LONGITUDE 140°E 180° 140°W 100°W LONGITUDE Figure 5.13: (cont.) Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 114 extension of the cold tongue originating from the North Pacific. The cold anomaly pattern is likely to appear in the North Pacific Ocean whenever E l Niiio occurs. Fig. 5.14 displays the forecasts versus observation for the Niiio 4 region, at lead time 3, 6, 9 and 12 months, with the forecasts calculated from the forecasted SSTA field. The forecasts resemble those based on individual region forecasting shown in Fig. 5.6. Thus, both methods of forecasting delivered similar skills. 5.5 Network pruning analysis Tangang et al. [1997a,b], used large networks (with 94 degrees of freedom) with the EOFs as inputs. With such large networks, the weight intrepretation can be very difficult. The smaller networks used here (with 31 d.f.) produced comparable forecast skills and very consistent results. This shows that the lag compression via the E E O F analysis of the predictor data provides an effective way to reduce the size of the networks. With smaller networks, the intrepretations of the weights are more lucid. In addition to lag compression by E E O F , a pruning scheme that further trimmed the networks to minimal size was also introduced. To demonstrate this, the weights and biases of the hidden layer, for the network forecasting Niiio 4 SSTA at 6 months lead, are presented in Table 5.1. This forecast was one of the 20 forecasts used to calculate the mean forecasts presented in the last section. The correlation skills for the training and forecast periods were 0.69 and 0.76, respectively, comparable to the mean skills. The corresponding weights, uij, for the output layer were -0.3474, -0.6110, and -0.7401 for the first, second and third neurons, respectively. The bias, b, for the neuron in the output layer was -0.0725. If one considers the magnitude of the weights, one may conclude that some of the inputs, for example the SLP E E O F modes 5 and 7, may not contribute significantly. These weights, however, need to be intrepreted with caution as Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 115 ( 3 - m o n t h lead)(r=0.81) - 3 . 0 I i i i • i i i i i i i i 1982 1984 1986 1988 1990 1992 ( 6 - m o n t h lead)(r=0.79) 2o I ' ' i i i ' i ' ' ' ' - 3 . 0 I i i i i " i i i i i r " i 1982 1984 1986 1988 1990 1992 Observations NN forecasts Figure 5.14: Standardized SSTA of the neural network forecasts (dashed curves), spatially averaged from the forecasted SSTA field, of Nino 4 region, compared with the observed anomalies (solid curves) for the forecast period of 1982-93 at lead times of 3, 6, 9 and 12 months. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 116 ( 9 - m o n t h lead)(r = 0.75) 2.0 i i. 1990 NN forecasts Figure 5.14: (cont.) Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 117 Input Hidden Neuron 1 2 3 Mode 1 -.0776 .4946 -.0363 Mode 2 -.2230 .0117 -.2511 Mode 3 .0527 .1630 .0244 Mode 4 .2738 .2255 -.3460 Mode 5 -.1287 -.0057 -.0894 Mode 6 -.1426 .1986 -.4514 mode 7 -.1078 .0200 .0663 SSTA -.2085 -.0776 -.5923 Bias .6380 -.4411 .0188 Table 5.1: The weights, w, and biases, b, of the hidden layer, for the network forecasting Nino 4 SSTA at 6-month lead. the modeling strategy learned by the network cannot be readily understood by visually inspecting weights and biases. Different signal patterns may interact with the weight patterns in subtle ways, hence the signals themselves must be included in the analysis. In particular, it is important to determine how the signals activate the hidden neurons. It is also crucial to consider how these neurons interact in the output layer, since the response for a particular input may be large for several neurons in the hidden layer but the effects cancel out in the output layer. To show this effect, the pruning statistics for the forecast period of 1982 - 1993 are displayed in Table 5.2. Similar statistics were obtained for the training period. These statistics were obtained by pruning each input in turn (replacing it with zeros) and feeding the pruned inputs into the trained network. Equivalently, this can be achieved by setting the weights associated with that particular input to zero. The correlation and root mean square (rms) values between outputs of each hidden neuron in the unpruned network and those of the pruned ones were then calculated. The same statistics were also calculated for the final output neuron. These statistics provide some clues on the Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 118 INPUT P R U N E D Corr RMS Output NI N2 N3 NI N2 N3 Corr R M S Mode 1 .98 .50 1.0 .0697 .4584 .0292 .91 .2387 Mode 2 .84 1.0 .95 .1942 .0111 .1943 .92 .2008 Mode 3 .99 .96 1.0 .0428 .1438 .0157 .98 .1115 Mode 4 .80 .92 .93 .2187 .1745 .2434 .99 .0759 Mode 5 .96 1.0 1.0 .1034 .0051 .0585 .99 .0793 Mode 6 .93 .92 .87 .1353 .1904 .3260 .93 .1820 Mode 7 .97 1.0 1.0 .0895 .0166 .0449 1.0 .0201 SSTA .98 1.0 .96 .0854 .0322 .1929 .94 .1890 3-4-5-7 .72 .90 .92 .2567 .2224 .2654 .95 .1589 1-2-6-SSTA .50 .62 .66 • 3275 .4634 .5383 .35 .4596 Table 5.2: The pruning statistics for forecasting Nino 4 SSTA for the 1982-93 period at 6-month lead. The values shown are the correlation and RMS between the outputs of the hidden neurons (NI, N2, and N3) of the pruned network and the corresponding outputs of the unpruned network. The correlation and RMS values between the pruned network outputs and the corresponding unpruned outputs are given in the last two columns. importance of each input in activating individual hidden neurons and how they affect the final outputs. Both the correlation and rms values suggest that modes 2, 4 and 6 of the SLP E E O F gave relatively larger responses than the other inputs for the first hidden neuron (denoted as N I in Table 5.2), as their removal led to a drop in the correlation and a rise in the rms value. The SSTA, however, produced a smaller response despite its larger weight than mode 6 (see Table 5.1), since the magnitude of the SSTA input was smaller than mode 6, and hence the overall response on the neuron activation was smaller for the SSTA than mode 6. The second neuron was dominated by mode 1, with modes 4 and 6 also showing relatively important contributions. Modes 2, 4, 6 of the SLP E E O F and the SSTA had impacts on the third neuron. It appears that only 5 inputs (i.e. modes 1, 2, 4, 6 and the SSTA) made important contributions in activating the hidden neurons. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 119 The statistics for the output neuron (both the correlation and rms), however, show that only four inputs, i.e. modes 1, 2, 6 and the SSTA, appear to have relatively important contributions. The mode 4 input, despite larger responses on all three hidden neurons, produced negligible net effect on the final output. This can be explained by considering that the weights for mode 4 (see Table 5.1) were .2738, .2255 and -.3460 for the first, second and third hidden neurons respectively, while all the weights for the output layer (-0.3474, -0.6110, and -0.7401) were of the same sign. Such a combination of weight patterns allowed the mode 4 signal to essentially cancel out in the final output layer. Modes 3, 4, 5 and 7, which when removed individually produced no net effect on the network output, were further simultaneously omitted. The net effect of removing these 4 inputs simultaneously appeared to be minimal, smaller than the effect of removing mode 1 alone. The network however completely failed when modes 1, 2, 6 and the SSTA were simultaneously removed from the inputs. The means (based on 20 forecasts) for these pruning statistics were also calculated, and similar results prevailed. These results suggest that only four of the eight original inputs were important, and as modes 3, 4, 5 and 7 made no significant contribution, they can be ignored. To further investigate this, two new networks were retrained, with the first using modes 1, 2, 6 and the SSTA as inputs, and the other using modes 3, 4, 5 and 7. Figs. 5.15 and 5.16 show the correlation skills for the training period (1952 - 1981) and the forecast period (1982 - 1993) for these two networks. For comparison, the skills for the full model (i.e. with 8 inputs) are also plotted. The skills of the first pruned model are slightly lower than the full model during the training period, but slightly higher during the forecast period. The slightly higher skill for the full model in the training period was expected since the full model had more free parameters to fit the data. The pruned model also shows less sensitivity to initial weights by having smaller standard deviations with ensemble forecasts. On the other hand, the model with modes 3, 4, 5 and 7 as inputs shows no Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 120 skills in both the forecast and training periods. These further substantiate that modes 1, 2, 6 and the SSTA were the only relevant inputs, while the other inputs had minimal or no important contributions. The pruned networks (with modes 1, 2, 6 and the SSTA) were further examined in relation to how the three hidden neurons were being activated. It appeared that the outputs of one of the neurons were highly correlated with the network outputs, whereas the other two neurons tended to cancel each other. This seems to suggest that one neuron in the hidden layer would be sufficient. To test this idea, another network with only a single neuron in the hidden layer was constructed and retrained. This represents a minimal network with 4 inputs, 1 hidden neuron and 1 output neuron, for a total of 7 degrees of freedom. Fig. 5.17 shows the forecast skills for the 1982-93 forecast period, for the Niiio 4 SSTA. For comparison purposes, the forecast skills of the pruned model with 3 hidden neurons, the full model (all 8 inputs) and those of larger networks using SLP E O F as inputs presented in Tangang et al. [1997b] are also shown. The result shows that this minimal network gave a slight improvement of the forecast skills. The minimal networks also show less sensitivity to the selection of the initial random weights. Such improvements were also seen for other decades and for other equatorial regions (not shown). 5.6 Spectral analysis of the trained network The results in the previous section showed that only modes 1, 2, 6 and the SSTA persis-tence were important for forecasting the SSTA. The SSTA persistence has been shown to improve the skills [Tangang et al. 1997a; Barnston and Ropelewski 1992]. In this section, a spectral analysis is presented, which provides supporting evidence of the importance of modes 1, 2 and 6. Figs. 5.18, 5.19, and 5.20, show the spectra of modes 1, 2, and 6 of the Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 121 0 . 8 ~ 0 . 6 O 0 . 4 O 0 . 2 • Full Model • Pruned model (1,2,6,SSTA) Pruned model (3,4,5,7) §3 33 m 6 9 L E A D T I M E ( M O N T H ) 1 2 1 5 Figure 5.15: The correlation skills for three different networks, the full model network trained with all the eight inputs, the network trained with modes 1, 2, 6 of the SLP E E O F and the SSTA, and the network trained with modes 3, 4, 5, and 7, for the training period (1952-1981). Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 122 1 L E A D T I M E ( M O N T H ) Figure 5.16: As in Fig. 5.15 except for the forecast period (1982 - 1993). Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 123 0 . 8 c .2 0 . 6 re <D O 0 . 4 O -ri 0 . 2 'hi • i l l M 1 EOF EE0F(Full Model) EEOF(Pruned, 3 hidden EEOF(Pruned, 1 hidden T Mi 111 neurons) neuron) 1 2 m mi 1 5 L E A D T I M E ( M O N T H ) Figure 5.17: The comparison between forecast skills of various neural network models for the 1982-93 period for the Nifio 4 region (see text for explanation). The pruned models have only SLP E E O F modes 1, 2, 6 and the SSTA as inputs, with either 3 or 1 hidden neurons. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 124 SLP E E O F , respectively. The spectrum for mode 1 shows a peak at 4 - 5 year period. This agrees with the earlier interpretation that mode 1 was due to ENSO. Mode 2 has a broad spectrum, with several peaks around 2 - 5 years. These two modes were consis-tent with the low-frequency oscillation (LFO) and the quasi-biennial oscillation (QBO) described in, e.g., Barnett [1991], Ropelewski et al. [1992] and Goswami [1995]. Mode 1, with a 4 - 5 year period and a spatial pattern indicating a standing wave number-1 oscil-lation, fitted the description of the L F O . Mode 2 appeared to be a mixture of the QBO and the L F O , with an eastward propagating spatial pattern. Barnett [1991] concluded that the ENSO process is partially due to the nonlinear interaction between these two longer time scale oscillations. Mode 6 shows high energy peaks at low frequencies (10-year period or longer). Figs. 5.21 and 5.22 show the spatial and temporal patterns of mode 6, respectively. At a lag of 9 months, the spatial pattern indicates two opposite centers, a relatively strong center located in the central equatorial Pacific about 170°W, and a much weaker one in the Australian-Indonesia region, suggesting an east-west oscillation. The subsequent pat-terns have no obvious physical intrepretation. The rapid changes of the spatial patterns from one lag to another may be caused by high frequency variations. Also shown is the 6-year low-pass filtered time series (generated by successive applications of centered 25-and 37-month running means), clearly revealing the decadal oscillation. This filtered time series resembles the 6-year low-pass filtered time series of the SST over the Pacific Ocean shown in Zhang et al. [1996, Fig.3]. While both modes 1 and 2 were expected to be important, since both explained rela-tively large fractions of the total variance and appeared to be associated with ENSO, it was surprising to see mode 6, explaining only 2.7% of the total variance, play a significant role in forecasting the SSTA. Wang [1995] hypothesized that the interdecadal SSTA are coupled with the interdecadal atmospheric circulation anomalies, where the changes in Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach F r e q u e n c y ( c y c l e s / 4 9 5 m o n t h s ) 0 5 1 0 1 5 2 0 2 5 3 P e r i o d s ( y e a r s ) Figure 5.18: The spectrum for the SLP E E O F mode 1. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 126 F r e q u e n c y ( c y c l e s / 4 9 5 m o n t h s ) 0 5 1 0 1 5 2 0 2 5 3 0 P e r i o d s ( y e a r s ) Figure 5.19: As in Fig. 5.18 except for SLP E E O F mode 2. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 127 F r e q u e n c y ( c y c l e s / 4 9 5 m o n t h s ) 0 5 1 0 1 5 2 0 2 5 3 0 P e r i o d s ( y e a r s ) Figure 5.20: As in Fig. 5.18 except for SLP E E O F mode 6. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 128 MODE 6 of SLP EEOF (2.7 perc.) 60°E 100°E 140°E 180° 140°W 100°W LONGITUDE Figure 5.21: The spatial pattern of the SLP E E O F Mode 6 at lags 9, 6, 3, and 0 months. The contours represent correlation between the E E O F amplitude and the SLP anomaly at each grid point. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 129 60°E 100°E 140°E 180° 140°W 100°W LONGITUDE 60°E 100°E 140°E 180° 140°W 100°W LONGITUDE Figure 5.21: (Cont.) Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 130 - 2 . 0 - 3 . 0 - I f 1 I 1 1 ! 1 1 1 1 L lllll 1 l 1 1 1 1 f N i l h M I I I I I 1 1 , / \ i i i / I 1 \ I I I 1 1 i I V 1 i i i i i if1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 i i ft i i i I 1 1 1 1 ill i i V 1 1 -1952 1954 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 Figure 5.22: The standardized amplitude for the SLP E E O F mode 6. The thick line indicates the 6-year low-passed time series. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 131 the atmospheric circulation may, in turn, feed back into the ocean, effectively influencing the ENSO cycle. Wang [1995] also argued that the conditions during the warm state of the interdecadal mode favor the formation of positive SSTA stretching south-westward from Baja California to the central equatorial Pacific, weakening the trades in the central Pacific. Wang [1995, Fig.12] suggested the two ways this condition may influence the ENSO onset: (1) The warming can offset the cool condition prior to an E l Niiio, resulting in a near-normal Walker circulation over the western and central Pacific and enhanced trades in the southeastern Pacific. The strengthening of the southeast trades in the on-set phase could prevent the South American coastal warming in the subsequent phase, resulting in an absence of coastal warming. Also, the warming in the central equatorial Pacific (near the date line) can trigger the eastward movement of convection and asso-ciated westerly anomalies from the western to the central equatorial Pacific, leading to a warming there. (2) The condition favors the formation of the Philippine sea onset cy-clone, believed to be associated with the equatorial westerly anomalies, which eventually cause the Pacific warming. Despite the profound impacts of the 1976-77 'regime shift', Zhang et al. [1996] speculated that the small shifts, such as those in 1957-58 and 1990-91, are no less important in terms of climate assessment, diagnosis and prediction. Wang [1995] also suggested that the anomaly models that are used to forecast ENSO should incorporate the secular changes in the background 'mean' state. The results presented here, which show the importance of the interdecadal oscillation in forecasting the SSTA, support this contention. To further demonstrate the relative importance of each of the contributing inputs (i.e. modes 1, 2, 6 and the SSTA persistence) for forecasting the SSTA in different regions at different lead times, the weights and biases are presented in Table 5.3. The weights and biases were obtained from training the minimal networks without the weight decay terms in Eq.( 2.7). The networks converged to a unique solution for different random Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 132 Weights and Bias Lead time Region Wi ^3 W4 b Nino 4 -.06 .18 .08 .98 .15 0-month Nino 3.5 -.07 .15 .06 .63 -.05 Nino 3 -.06 -.19 .09 .39 -.47 P4 -.07 .26 .11 .76 -.49 Nino 4 -.35 -.25 .31 .89 -.27 6-month Nino 3.5 -.28 .27 .27 .40 -.62 Nino 3 -.24 -.40 .30 .00 -.78 P4 -.46 -.25 .41 .77 -.83 Nino 4 -.15 .08 .22 .10 -.32 12-month Nino 3.5 -.22 .08 .27 .05 -.61 Nino 3 -.38 .07 .46 -.06 -1.13 P4 -.40 .10 .39 -.02 -1.17 Table 5.3: The weights, w, and biases, b, in several neural network models for forecasting the SSTA in various regions at 0, 6 and 12 months leads (the weights wi, u>2, w3 and wA correspond to modes 1, 2, 6 of the SLP E E O F and the SSTA persistence, respectively). The models were all trained with data from 1952-1981. Region P4 is at the eastern boundary (0 - 10°N, 80°W - 100°W). initializations. Since there is only one neuron in the hidden layer, the magnitude of the weight reflects its contribution. At 0-month lead, the SSTA and mode 2 have the most important contributions. The importance of persistence (as measured by W4) declined eastward from Nino 4 to Nino 3 but then increased at P4. At 6 months lead, all the 3 E E O F modes have roughly equal contributions, while the persistence (w 4) declined (except at P4). For a longer lead time of 12 months, only mode 1 and mode 6 had significant contributions. Also, there appears to be a trend in which the biases increase in magnitude as the regions shift eastward and as lead time increases. For the Taylor series expansion of the hyperbolic tangent function in Eq.( 2.1) about x = 0, the linear, quadratic and cubic parts of the expansion series are sech2(b)(%2i W{Xi), tanh(b)sech2(b)(J2i ^2jwiwjxixj), a n d sech2(b)(l — Ztanh2 (6))(E)i ^2j^2kwiwjwkXiXjXk), Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 133 respectively. Clearly, the contribution of each term depends on the weights, the bias, and the inputs. Since the weights in Table 5.3 are small, the quadratic and cubic parts are likely to be smaller than the linear ones. Also, as the bias approaches zero, there would be no quadratic component as tanh(b) vanishes. On the other hand, the cubic component will be suppressed at b around 0.61 for which the expression (1 — 3tanh2(b)) vanishes. For large values of 6, both the quadratic and cubic components will contribute. What kind of nonlinearity is embedded in the weights and biases of Table 5.3? To in-vestigate, the networks were fed with four pure sinusoidal functions, each with a different frequency. The sine waves, with frequencies of 40, 55, and 75uia and amplitude 3, were used to substitute for mode 1, mode 2 and mode 6, respectively, where ui0 represents the lowest frequency, i.e. 1024 _ 1 mo _ 1 . Amplitude 3 was chosen to match the maximum and minimum of the three original inputs that he between +3 and -3. This is crucial since the networks would produce much stronger nonlinear responses if fed with inputs with much larger amplitudes than those of the original ones. For the SSTA input, a frequency at 100u;o with amplitude 2 was used to approximately match the magnitude of the SSTA input. These four frequencies were chosen so that the frequencies arising from quadratic interactions would not overlap the original input frequencies. The networks were then fed with these four sine waves and the outputs were subjected to spectral analysis. If the network is linear, then only four spectral peaks will appear representing the four original frequencies. If the network is nonlinear, there will be additional peaks for the nonlinear interactions. As an example, Figs. 5.23 - 5.25 show the spectra of the neural network outputs for the Niiio 3 region at 0, 6 and 12-month leads. At 0-month lead, most of the energy is peaked at 100 and 55u>0 correponding to the SSTA and mode 2 inputs, with the other two inputs having relatively smaller peaks, in agreement with the weights in Table 5.3. Also seen is the interaction between the two dominant inputs i.e. at 45(=100-55)u;o and and 155(=100+55)u>o. At 6 months lead, all inputs but the SSTA Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 134 Region Lead time 0-month 6-month 12-month Niiio 4 9.4 12.2 3.9 Niiio 3.5 4.5 9.0 10.4 Nino 3 9.4 14.9 22.4 P4 11.2 19.8 22.6 Table 5.4: The ratio (in percentage) of the power from nonlinear interactions to the total power for various regions at 0, 6 and 12 months leads. The models were all trained with data from 1952-1981. have an important contribution. Also, there appears to be nonlinear interactions between these modes. The interaction between mode 1 and 2 may reflect the interaction between the QBO and L F O envisioned by Barnett [1991]. At a longer lead of 12 months, only modes 1 and 6 appear to be important, with strong nonlinear interaction among them manifested by the peaks at 35 and 115a»0. These results show that the networks, fed with the original inputs, produced nonlinear responses. To roughly estimate the strength of the nonlinearity, the ratio between the power due to nonlinear interactions (i.e. total area underneath the quadratic and cubic peaks) and the total power (i.e. total area under all peaks) was calculated. The results shown in Table 5.4 (in percentages) are for the Niiio 4, Nino 3.5, Nino 3, and P4 regions at 0, 6 and 12 months leads, with region P4 at the eastern boundary (0 - 10°N, 80°W - 100°W). East of Niiio 4, there appears to be a trend where the ratio increases eastward and with lead time. These results support the earlier contention that the relationship between the predictor and the predictand becomes more nonlinear as lead time increases. Also, the nonlinearity appears to be more prominent in the eastern region. The trend for the western-central region of Niiio 4 is somewhat surprising, in that the ratio at 12-month lead was smaller than the ratio at 6-month lead. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 135 (0 c 0 Q (0 2 0 1 5 h o o a w 1 o 0 1 5 (0 1 I 1 I I I I I I I 1 I 1 I 1 I I 1 I 1 I 1 I 1 I 1 I 1 I 0 2 0 4 0 6 0 8 0 1 0 0 1 2 0 1 4 0 1 6 0 1 8 0 2 0 0 F r e q u e n c y ( c y c l e s / 1 0 2 4 m o n t h s ) Figure 5.23: The spectra for the neural network outputs for the Nino 3 region when fed with four pure sinusoidal signals at frequencies of 40, 55, 75 and 100 u}0 (see text for explanation) at 0-month lead time. The square root of the spectral density was plotted to render the weaker signals more visible. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 136 2 0 4 0 6 0 8 0 1 0 0 1 2 0 1 4 0 1 6 0 1 8 0 2 0 0 F r e q u e n c y ( c y c l e s / 1 0 2 4 m o n t h s ) Figure 5.24: As in Fig. 5.23 except for lead time of 6 months. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 137 2 5 | 2 0 (0 8 1 5 a o 1 0 o o 0) 5 U i 1 i 1 i 1 i 1 i 1 i 1 i 1 i 1 i 1 i 1 i 1 i 1 i 1 i 1 i 1 i 1 0 2 0 4 0 6 0 8 0 1 0 0 1 2 0 1 4 0 1 6 0 1 8 0 2 0 0 F r e q u e n c y ( c y c l e s / 1 0 2 4 m o n t h s ) Figure 5.25: As in Fig. 5.23 except for lead time of 12 months. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 138 5.7 Summary and conclusion The neural network technique was applied to forecast the SSTA of the three regions, Nino 4, Nino 3.5 and Nifio 3, representing the western-central, central and central-eastern parts of the equatorial Pacific Ocean, respectively. Nifio 4 was the best forecasted region, with useful forecasting skills up to one-year lead time. Also, two forecasting methods, i.e. a direct forecasting of the SSTA of individual regions, and a forecasting of the first four dominat modes of the SSTA E O F , delivered similar skills. By using the E E O F technique to compress the SLP predictor field, the resulting neural networks were much smaller but produced similar or even slightly higher forecast skills than those of larger models, which used the SLP EOFs as inputs (as found in Tangang et al. [1997b]). In the earlier models, there were 94 degrees of freedom (d.f.); by introducing the E E O F , the size was reduced to 31 d.f, and applying the pruning scheme, the models were trimmed to only 7 d.f. It was discovered that only four of the eight original inputs (i.e. modes 1, 2, 6 of the SLP E E O F and the SSTA persistence) made important contributions to forecasting the SSTA, thus the four unimportant inputs were eliminated. At 0-month lead, mode 2 and the SSTA persistence made the important contributions. At 6-month lead, all four inputs made roughly comparable contributions, while at 12-month lead, modes 1 and 6 were most important. Mode 1, with a 4 - 5 year period and a spatial pattern indicating an east-west seesaw oscillation, fitted the description of the low frequency oscillation (LFO). This mode is usually seen as a typical ENSO signal with high amplitudes during E l Nifios and low ones during La Ninas. Mode 2 displayed an eastward propagating pattern with spectral peaks between 2 - 5 years, indicating a mixture between the QBO and the L F O . The importance of mode 6, characterized by decadal and interdecadal oscillations, showed that very long period oscillations affect ENSO forecasting. Chapter 5. Forecasting regional SSTA: Neural network - extended EOF approach 139 By appying spectral analysis to neural networks, the important inputs and the non-linear responses were further identified. The spectral power due to nonlinear interactions relative to the total power was also calculated, which showed that east of Niiio 4, non-linearity appeared to increase with lead time, thus supporting the earlier contention that the relationship between the atmospheric field and the SST becomes increasingly nonlin-ear as the lead time increases. Nonlinearity also tends to grow stronger eastward from Niiio 3.5. Chapter 6 Overall conclusion and summary This study of forecasting ENSO events using neural network models was carried out in three stages: (1) Tangang et al. [1997a] forecasted the Nifio 3.4 SSTA with the wind stress as the main predictor, focusing on model building aspects (Chapter 3). (2) Tangang et al. [1997b] compared the performances of wind stress models and SLP models in forecasting the SSTA in various standard regions in the equatorial Pacific Ocean (Chapter 4). (3) With the E E O F analysis, Tangang et al. [1997c] introduced smaller models that allowed a better understanding of the inner workings of the neural network models (Chapter 5). 6.1 Neural network models and forecast results The first attempt to forecast the Nifio 3.4 SSTA in Tangang et al. [1997a] (i.e. Chapter 3) focused on model building aspects. It was found that models fed with the first seven E O F modes of tropical Pacific (20°S - 20°N, 120°E - 70°W) wind stress for four seasons and the Nino 3.4 SSTA itself (i.e. the persistence) in the final season prior to the forecasting period delivered the best forecasting skills. At 6-month lead time, these models attained comparable forecasting skills to the existing ENSO models surveyed by Barnston et al. [1994]. The results also show that the models delivered useful forecasting skills even at longer leads of 9 to 12 months, which is an improvement over conventional statistical models. A comparison of neural network models versus multiple linear regression models of the same inputs was also performed. The result shows that neural network models 140 Chapter 6. Overall conclusion and summary 141 perform better than regression models for both the training and forecasting periods at the longer lead time regime of 9 to 12 month leads. This result led to the hypothesis that the underlying relationship between the wind stress and Niiio 3.4 SSTA becomes increasingly nonlinear as the lead time increases. This idea is consistent with Latif et al. [1994], who argued that nonlinearity plays a significant role at longer lead times. The latitudinal extent and the temporal duration of the wind stress predictor in the models matched the space and time domains of the equatorial Kelvin waves and the off-equatorial Rossby waves relevant to ENSO. With this, the model forecasts were intrepreted in light of the current understanding of ENSO, e.g. the role of off-equatorial baroclinic Rossby waves in triggering the onset of an ENSO event, as outlined by White et al. [1989], and the delayed oscillator theory [e.g., Battisti and Hirst, 1989; Graham and White, 1988] with respect to the development of the event from the peak through the mature phase (positive Kelvin wave effect) and the termination of the event (delayed negative Rossby wave effect). As an extension to the previous works, Tangang et al. [1997b] (i.e. Chapter 4) pre-sented SSTA forecasts of various regions extending from the western to eastern equatorial Pacific Ocean. Two types of models where the predictor fields were the wind stress and the SLP, respectively, were also compared. The wind stress models were the same as those in Tangang et al. [1997a], whereas the SLP models used the SLP fields covering the tropical Indian and Pacific Oceans. The inclusion of the tropical Indian Ocean SLP was based on findings that the atmospheric variations, responsible for the SST changes in the Pacific Ocean associated with ENSO, may originate in the Indian Ocean [e.g., Barnett, 1985; Yasunari, 1990; Gutzler and Harrison, 1987]. The method of ensemble averaging on the model forecasts was also introduced as a remedy for the sensitivity to initial weights. By ensemble averaging from 20 individual forecasts with initial random weights, the forecasts appeared less noisy than those in Tangang et al. [1997a]. It was Chapter 6. Overall conclusion and summary 142 found that the SLP models delivered better overall forecasting skills than the wind stress models for lead times longer than 6 months. The difference was attributed to the noisier nature of the wind stress data, and the wider spatial coverage of the SLP field. Both models showed similar skill distribution where the skills decreased eastward, and with the Nifio 4 region having the highest skills. Interestingly, the P2 region, which was located slightly to the west and partially overlapped with Nino 4, had no skills. This is consistent with the idea that the SST in the far western region of the equatorial Pacific is largely decoupled from atmospheric variations. Tangang et al. [1997c] (i.e. Chapter 5) focused on understanding the inner working of the neural network models. By employing the E E O F that allows both spatial and temporal compressions of the predictor field, smaller models were constructed. It was found that these smaller models delivered similar skills as the larger models in Tangang et al. [1997a,b]. A pruning scheme was further employed, allowing the removal of the unimportant inputs. The trimmed models had even slightly better skills. Only four inputs, i.e. modes 1, 2, 6 of the SLP E E O F and the SSTA persistence itself, were identified to have made significant contributions in forecasting the SSTA. A l l three modes had plausible physical associations: Mode 1, indicating an east-west seesaw pressure oscillation with a 4 - 5 year period, fitted the description of the low frequency oscillation (LFO). Mode 2 characterized a propagating pattern with spectral peaks between 2 - 5 years, suggesting a mixture between the LFO and the quasi-biennial oscillation (QBO). Mode 6 appeared to be dominated by decadal and interdecadal oscillations with a period of 10 years or longer. This result suggests that forecasting ENSO requires information from the QBO and the decadal-interdecadal oscillations with periods different from the typical 4 - 5 years of ENSO. The nonlinear responses of the neural network models were also investigated and estimated. This was done by substituting the original inputs with sinusoidal functions Chapter 6. Overall conclusion and summary 143 of different frequencies, and inputting them to the trained networks. With the aid of spectral analysis, it was found that east of Nifio 4, nonlinearity appeared to increase with lead time, and tended to grow eastward. This result is consistent with Tangang et al. [1997a,b], and may indicate that the eastern region of the equatorial Pacific Ocean is more affected by nonlinear processes. Overall, this study has shown that neural network models provide useful forecasting skills up to 12-months lead time, which is a substantial improvement over conventional statistical models. 6.2 Operational format, and future research The ultimate goal of this research is to have these models set up to run in operational mode. This can be implemented by projecting the new predictor data into the existing E O F space, and feeding the resulting amplitudes to the models. The unavailability of the most current wind stress or SLP data, however, could be a problem. In the future, this study can also be extended to the extratropics, where the relationship between atmospheric forcing and SST is more subtle. Bibliography Balmaseda, M . A . , M . K . Davey, and D.L. Anderson (1995). Decadal and seasonal depen-dence of ENSO prediction skill. J. Climate, 8:2705-2715. Barnett, T.P. (1984). Prediction of the E l Nifio of 1982-83. Mon. Wea. Rev., 112:1043-1047. Barnett, T.P. (1985). Variations in near global sea level pressure. J. Atmos. Sci., 42:478-501. Barnett, T.P. (1991). The interaction of multiple time series in the tropical climate system. J. Climate, 4:269-285. Barnett, T.P., M . Latif, N . Graham, M . Flugel, S. Pazan, and W. White (1993). Predic-tion of the E l Nifio of 1982-83. J. Climate, 6:1545-1566. Barnston, A . G . , and O F . Ropelewski (1992). Prediction of ENSO episodes using Canon-ical Correlation Analysis. J. Climate, 5:1316-1345. Barnston, A . G . , and H . M . van den Dool (1993). A degeneracy in cross-validated skill in regression-based forecasts. J. Climate, 6:964-977. Barnston, A . G . , H . M . van den Dool, S.E. Zebiak, T.P. Barnett, M , J i , D.R. Rodenhuis, and Others (1994). Long-lead seasonal forecasts- where do we stand? Bull. Am. Meteorol. Soc, 75:2097-2114. Battisti, D.S. (1989). On the role of off-equatorial oceanic Rossby waves during ENSO. J. Phys. Oceanogr., 19:551-559. Battisti, D.S. (1991). Reply. J. Phys: Oceanogr., 21:461-465. Battisti, D.S., and A . C . Hirst (1989). Interannual variability in a tropical atmosphere-ocean model: Influence of the basic state, ocean geometry and nonlinearity. J. Atmos. Sci., 46:1687-1712. Battisti, D.S., and E.S.Sarachik (1995). Understanding and predicting ENSO. Rev. Geophy., Supp., pages 1367-1376. Cane, M . A . (1992). Tropical Pacific ENSO model - ENSO as a mode of coupled sys-tem. In Trenberth, K . , editor, Climate System Modeling, pages 583-614. Cambridge University Press. 144 BIBLIOGRAPHY 145 Cane, M . A . , and S.E. Zebiak (1987). Prediction of E l Nino events using a physical model. In Cattle, H . , editor, Atmospheric and Oceanic Variability, pages 153-181. Royal Meteolorogical Society. Cane, M . A . , S.E. Zebiak, and S.C. Dolan (1986). Experimental forecasts of E l Nino. Nature, 321:827-832. Cardone, V . J . , J .G. Greenwood, and M . A . Cane (1990). On trends in historical marine wind data. J. Climate, 3:113-127. Chauvin, Y . (1989). Generalization performance of overtrained backpropagation. In A l -media, L. and Wellekens, O , editors, Proc. Eurasip. Workshop on Neural Networks, pages 46-55, New York. Springer. Chen, D., S.E. Zebiak, A . J . Busalacchi, and M . A . Cane (1995). A n improved procedure for E l Nifio forecasting: Implications for predict ability. Science, 269:1699-1702. Chryssoloris, G., M . Lee, and A. Ramsey (1996). Confidence interval prediction for neural network models. IEEE Trans. Neur. Netw., 7(Nl):229-232. Cybenko, G. (1988). Continous valued neural network with two hidden layers are suffi-cient. Technical report, Department of Computer Science, Tufts University, Medford. Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Math Control Signal Sys., 2:303-314. Eisner, J .B. , and A . A . Tsonis (1992). Nonlinear prediction, chaos, and noise. Bull. Am. Meteorol. Soc, 73:49-60. Firring, E . , R. Lukas, J . Sadler, and K . Wyrtki (1983). Equatorial undercurrent disap-pears during 1982-1983 E l Nifio. Science, 222:1121-1123. Gil l , A . E . , and E . M . Rasmusson (1983). The 1982-83 climate anomaly in the equatorial Pacific. Nature, 306:229-234. Goswami, B . N . (1995). A multiscale interaction model for the origin of the tropospheric QBO. J. Climate, 8:524-534. Graham, N .E . , and W.B . White (1988). A natural oscillator of the Pacific ocean-atmosphere system. Science, 240:1293-1302. Graham, N .E . , and W.B . White (1990). The role of the western boundary in the ENSO cycle: Experiments with coupled models. J. Phys. Oceanogr., 20:1935-1948. Graham, N . E . , and W . B . White (1991). Comments on "on the role of off-equatorial Rossby waves during ENSO". J. Phys. Oceanogr., 21:453-460. BIBLIOGRAPHY 146 Graham, N .E . , J . Michaelsen, and T.P. Barnett (1987a). An investigation of the E l Nino-Southern Oscillation cycle with statistical models: 1, predictor field characteristics. J. Geophys. Res., 92(C13):14,251-14,270. Graham, N .E . , J . Michaelsen, and T.P. Barnett (1987b). An investigation of the E l Nino-Southern Oscillation cycle with statistical models: 2, model results. J. Geophys. Res., 92(C13):14,271-14,290. Grieger, B . , and M . Latif (1994). Reconstruction of the El-Nino attractor with neural networks. Clim. Dyn., 10(6):267-276. Gutzler, D.S., and D.E. Harrison (1987). The structure and evolution of seasonal wind anomalies over the near-equatorial eastern Indian and western Pacific Oceans. Mon. Wea. Rev., 115:169-192. Hanson, S.J., and L . Y . Part (1989). Comparing biases for minimal network construction with back-propagation. In Touretzky, D., editor, Advances in Neural Information Processing Systems 1, pages 177-185, San Mateo, California. Morgan Kaufmann. Hasselmann, K . , and T.P. Barnett (1981). Techniques for linear prediction for systems with periodic statistics. J. Atmos. Sci., 38:2275-2283. Hirose, M . , and J .E. Kutzbach (1969). An alternate method for eigenvector computations. J. Appl. Meteorol, 6:701. Horel, J.D., and J . M . Wallace (1981). Planetary-space atmospheric phenomena associ-ated with the Southern Oscillation. Mon. Wea. Rev., 109:813-829. Hornik, K . , M . Stinchcomb, and H . White (1989). Multilayer feedforward networks are universal approximator. Neural Networks, 2:359-366. Jacobs, R . A . (1988). Increased rates of convergence through learning rate adaptation. Neural Networks, 1:295-308. J i , C , and D. Psaltis (1990). Generalizing smoothness constraints from discrete samples. Neural Computa., 2(2):188-197. J i , M . , A . Kumar, and A. Leetmaa (1994a). A multiseason climate forecast system at the National Meteorological Center. Bull. Am. Meteorol. Soc, 75(4):569-577. J i , M . , A . Kumar, and A. Leetmaa (1994b). An experimental coupled forecast system at the National Meteorological Center: Some early results. Tellus, 46A:398-418. Kessler, W.S. (1991). Can reflected extra-equatorial Rossby waves drive ENSO? J. Phys. Oceanogr., 21:444-452. BIBLIOGRAPHY 147 Kiladis, G .N. , and S.K. Sinha (1991). ENSO, monsoon and drought in India. In Glantz, M . , Katz, R., and Nicholls, N . , editors, Teleconnetions linking the worlwide climate anomalies, pages 431-458. Cambridge University Press. Kolen, J.I., and J .B. Pollack (1991). Backpropagation is sensitive to initial conditions. In Lippmann, R., Moody, J . , and Touretzky, D., editors, Advances in neural informa-tion processing system 3, pages 860-867, San Mateo, California. Morgan Kaufmann. Kousky, V . E . , and A. Leetmaa (1989). The 1986-87 Pacific warm episode: Evolution of oceanic and atmospheric anomaly fields. J. Climate, 2:254-267. Lagerloef, G.S., and R.L . Bernstein (1988). Empirical orthogonal function analysis of advanced very high resolution radiometer surface temperature patterns in Santa Barbara channel. J. Geophys. Res., 93(C6):6863-6874. Large, W.G. , and S. Pond (1981). Open ocean momentum flux measurements in moderate to strong winds. J. Phys. Oceanogr., 11(3):324—336. Latif, M . , T.P. Barnett, M . A . Cane, M . Flugel, N .E . Graham, and Others (1994). A review of ENSO prediction studies. Clim. Dyn., 9(4):167-179. Legler, D . M . (1983). Empirical orthogonal function analysis of wind vectors over the tropical Pacific region. Bull. Am. Meteorol. Soc, 64(3):234-241. Lippmann, R.P. (1987). A n introduction to computing with neural nets. IEEE ASSP Mag., 4:4-22. Mantua, N . J . (1994). Numerical modeling studies of the El Nino/Southern Oscillation. PhD thesis, University of Washington, Seattle. Mantua, N . J . , and D.S. Battisti (1994). Evidence for the delayed oscillator mechanism for ENSO: The "observed" oceanic Kelvin mode in the far western Pacific. / . Phys. Oceanogr., 24:691-699. Mantua, N . J . , and D.S. Battisti (1995). Aperiodic variability in the Zebiak-Cane coupled ocean-atmosphere model: ocean-atmosphere interaction in the western equatorial Pacific. J. Climate, 8:2897-2927. Michaelsen, J . (1987). Cross-validation in statistical climate forecast models. J. Clim. Appl. Meteor., 26:1589-1600. Munnich, M . , M . A . Cane, and S.E. Zebiak (1991). A study of self-excited oscillations of the tropical ocean-atmosphere system. II. Nonlinear cases. J. Atmos. Sci., 48:1238— 1248. BIBLIOGRAPHY 148 Navone, H.D. , and H.A. Ceccatto (1994). Predicting Indian Monsoon rainfall - a neural network approach. Glim. Dyn., 10(6):305—312. Philander, S.G. (1990). El Nino, La Nina, and the Southern Oscillation. Academic Press, San Diego. Posmentier, E.S., M . A . Cane, and S.E. Zebiak (1989). Tropical Pacific climate trends since 1960. J. Climate, 2:731-736. Preisendorfer, R .W. (1988). Principal Component Analysis in Meteorology and Oceanog-raphy. Elsevier, New York. Rasmusson, E . M . , and T .H. Carpenter (1982). Variations in tropical sea surface tem-perature and surface wind fields associated with the Southern Oscillation/El Nino. Mon. Wea. Rev., 110:354-384. Rasmusson, E . M . , and T .H. Carpenter (1983). The relationship between eastern equa-torial Pacific sea surface temperatures and rainfall over India and Sri Lanka. Mon. Wea. Rev., 111:517-528. Reed, R. (1993). Pruning algorithm- a survey. IEEE Trans. Neur. Netw., 4(5):740-747. Reynolds, R.W. , and T . M . Smith (1994). Improved global sea surface temperature anal-yses using optimum interpolation. J. Climate, 7:929-948. Ropelewski, C.F. , and M.S. Halpert (1987). The relationship between eastern equato-rial North American precipitation and temperature patterns associated with the E l Nino/Southern Oscillation (ENSO). Mon. Wea. Rev., 114:2352-2362. Ropelewski, C.F. , and M.S. Halpert (1989). Precipitation patterns associated with the high index phase of the Southern Oscillation. / . Climate, 2:268-284. Ropelewski, C.F. , M.S. Halpert, and X . Wang (1992). Observed tropospheric biennial variability and its relationship to the Southern Oscillation. J. Climate, 5:594-614. Rumelhart, D.E. , G.E. Hinton, and R.J . Williams (1986). Learning internal represen-tations by error propagation. In Rumelhart, D., McClelland, J . , and Group, P. R., editors, Parallel Distributed Processing 1, pages 318-362, Cambridge, M A . MIT . Rumelhart, D.E. , R. Durbin, R. Golden, and Y . Chauvin (1994). Backpropagation: The basic theory. In Chauvin, Y . and Rumelhart, D., editors, Backpropagation: Theory, Architectures, and Applications, Hillsdale, NJ . Lawrance Earlbaum Associates. Schneider, E . K . , B . Huang, and J . Shukla (1995). Ocean wave dynamics and E l Nino. J. Climate, 8:2415-2439. BIBLIOGRAPHY 149 Smith, M . (1993). Neural networks for statistical modeling. Van Nostrand Reinhold, New-York. 235 pp. Smith, T . M . , R .W. Reynolds, R .E . Livezey, and D.C. Stokes (1996). Reconstruction of historical sea surface temperatures using empirical orthogonal functions. J. Climate, 9:1403-1420. Suarez, M . J . , and P.S. Schopf (1988). A delayed action oscillator of ENSO. / . Atmos. Sci., 45:3283-3287. Tang, B . (1995). Periods of linear development of the ENSO cycle and P O P forecast experiment. J. Climate, 8:682-691. Tang, B . , G . M . Flato, and G. Holloway (1994). A study of Arctic sea ice and sea-level pressure using POP and neural network methods. Atmos.-Ocean, 32:507-529. Tangang, F .T. , B . Tang, A . H . Monahan, and W.W. Hsieh (1997c). Forecasting ENSO events: a neural network - extended E O F approach. / . Climate, (accepted). Tangang, F.T. , W.W. Hsieh, and B . Tang (1997a). Forecasting the equatorial sea surface temperatures by neural network models. Clim. Dyn., 13:135-147. Tangang, F .T. , W.W. Hsieh, and B. Tang (1997b). Forecasting regional sea surface temperatures of the tropical Pacific by neural network models, with wind stress and sea level pressure as predictors. J. Geophys. Res. (submitted). Trenberth, K . , and G.W. Branstator (1992). Issues in establishing causes of the 1988 drought over North America. J. Climate, 5:159-172. Trenberth, K . , and T.J . Hoar (1996). The 1990-1995 E l Nifio-Southern oscillation event: Longest on record. Geophys. Res. Let., 23:57-60. Trenberth, K . , G.W. Branstator, and P.A. Arkin (1988). Origins of the 1988 North American drought. Science, 242:1640-1645. Trenberth, K . , W. Large, and J . Olson (1989). The effective drag coefficient for evaluating wind stress over the ocean. J. Climate, 2:1507-1516. Van den Dool, H . M . (1993). Searching for analogues, how long must we wait? Tellus, 46A:314-324. Von Storch, H . , and G. Hannoschock (1984). Comments on empirical orthogonal function analysis of wind vectors over the tropical Pacific region. Bull. Am. Meteorol. Soc, 65:162. BIBLIOGRAPHY 150 von Storch, H . , G. Burger, R. Schnur, and J.S. von Storch (1995). Principal oscillation patterns: A review. J. Climate, 8(3):377-400. Wang, B . (1995). Interdecadal changes in E l Niiio onset in the last four decades. J. Climate, 8:267-285. Weare, B . C . (1982). E l Nino and tropical Pacific ocean surface temperatures. J. Phys. Oceanogr., 12:17-27. Weare, B . C . and J.S. Nasstrom (1982). Examples of extended empirical orthogonal functions. Mon. Wea. Rev., 110:481-485. Weare, B .C . , A .R . Navato, and R .E . Newell (1976). Empirical orthogonal analysis of Pacific sea surface temperatures. J. Phys. Oceanogr., 6:671-678. Weigend, A.S. (1995). Time series analysis and prediction. In Smolensky, P., Mozer, M . , and Rumelhart, D., editors, Mathematical Perspectives on Neural Networks, Hillsdale, N J . Erlbaum Associates. Weigend, A.S. , D .E . Rumelhart, and B . A . Hubermann (1990). Predicting the future: A connectionist approach. Int. J. Neur. Sys., 1:193-209. Weigend, A.S. , D .E . Rumelhart, and B . A . Hubermann (1991). Generalization by weight-elemination with application to forecasting. In Lippmann, R., Moody, J . , and Touret-zky, D., editors, Advances in neural information processing system 3, pages 875-882, San Mateo, California. Morgan Kaufmann. White, W . B . , and C.K. Tai (1992). Reflection of interannual Rossby waves at the mar-itime western boundary of the tropical Pacific. J. Geophys. Res., 97:14,305-14,322. White, W.B . , S.E. Pazan, and M . Inoue (1987). Hindcast/forecast of ENSO events based upon the redistribuation of observed and model heat content in the western tropical Pacific, 1964-86. J. Phys. Oceanogr., 17:264-280. White, W.B . , Y . He, and S.E. Pazan (1989). Off-equatorial westward propagating Rossby waves in the tropical Pacific during the 1982-83 and 1986-87 ENSO events. J. Phys. Oceanogr., 19:1397-1406. Woodruff, S.D., R . J . Slutz, R .L . Jenne, and P . M . Steurer (1987). A comprehensive ocean-atmosphere data set. Bull. Am. Meteorol. Soc, 6:1239-1250. Woodruff, S.D., S.J. Lubker, K . Wolter, S.J. Worley, and J.D. Elms (1993). Comprehen-sive ocean-atmosphere data set (COADS) release la: 1980-92. Earth Syst. Monitor, 4(l) : l -8. BIBLIOGRAPHY 151 Wyrtki , K . (1985). Water displacements in the Pacific and the genesis of E l Nino cycles. J. Geophys. Res., 90(C4):7129-7132. X u L. , Klasa, S., and Yuille, A . (1992). Recent advances on techniques of statics feed-forward networks with supervised learning. Int. J. Neur. Sys., 3(3):253-290. Yasunari, T. (1990). Impact of Indian monsoon on the coupled atmosphere/ocean system in the tropical Pacific. Meteor. Atmos. Phys., 44:29-41. Zebiak, S.E., and M . A . Cane (1987). A model E l Nino - Southern Oscillation. Mon. Wea. Rev., 115(10):2262-2278. Zhang, B . , J . Lie, and Z. Sun (1993). A new multidimensional time series forecasting method based on the E O F interation scheme. Advan. Atmos. Sci., 10:243-247. Zhang, Y . , J . M . Wallace, and D.S. Battisti (1996). ENSO-like interdecadal variability: 1900-93. J. Climate, (in press). Appendix A The E O F and E E O F Analyses In this study the empirical orthogonal functions (EOF) method was employed in com-pressing the predictor fields. This technique allows the dominant modes of data variabil-ity to be extracted, thus the original data sets can be condensed from the larger number of grid or temporal points to the smaller number of E O F modes retained. For climate variables, the first few dominant modes may be associated with major climatic events. Detail descriptions of this technique may be found in Preisendorfer [1988], and a brief discussion is presented in this section. In this study, the E O F analysis was implemented on the dimensionally smaller covariance matrix. Basically, E O F analysis calculates the orthogonal linear combination of variables that account for the maximum possible amount of variance in a data set. For a given matrix of a data set X(i,j) i = l,---,nx j = l,---,nt (A. l ) where nx is the number of grid points and nt is the number of time intervals, the E O F can be calculated from the dimensionally smaller covariance matrix C (of dimension nt by nt), i.e. C = XTX if nx > nt (A.2) or from the covariance matrix R (of dimension nx by nx) R = XXT if nx < nt . (A.3) It is computationally cheaper and more efficient to implement E O F analysis with the 152 Appendix A. The EOF and EEOF Analyses 153 smaller covariance matrix [Lagerloef and Bernstein, 1988]. It has been shown that the maximum number of nonzero modes is the smaller of either nx or nt [ Von Storch and Hannoschock, 1984; Hirose and Kutzbach, 1969]. For a case where nx > nt, the eigenvalues, A;, are the solution to XiBi = BiC (A.4) where the Bi is the dimensionless eigenvector forming the i-th row of B , and A; is the i-th mode eigenvalue. The modal amplitude functions are computed from A = XBT. (A.5) Similarly, for the case of nx < nt, the eigenvalues are the solutions of the equation XiAi = RAi (A.6) where A; is the i-th column of A, and B becomes the modal amplitudes given by B = ATX. (A.7) The eigenvalues of the larger and smaller matrices are identical, in which solving the larger matrix produces \nx — nt\ zeros eigenvalues. Also, the eigenvectors and modal amplitudes of the two matrices are equivalent. For each E O F mode, the eigenvector of the larger matrix equals the modal amplitude of the smaller matrix divided by the square root of the eigenvalue, and the modal amplitude of the larger matrix equals the eigenvector of the smaller matrix multipled by the squared root of the eigenvalue [Hirose and Kutzbach, 1969]. The eigenvectors, modal amplitudes and eigenvalues are the useful end products of this analysis. The spatial pattern of each mode can be visualized by plotting the eigenvector of each mode. The time-varying amplitudes give the temporal variability pattern and the Appendix A. The EOF and EEOF Analyses 154 relative percentage of the variability of each mode can be obtained from the eigenvalues. Alternatively, the spatial pattern of the variability of each mode can also viewed by plotting the correlation values between the original data (i.e. the X) and the modal amplitude. In contrast to the EOFs which an eigenvector gives a single map in the variable space, the extended empirical orthogonal function (EEOF) eigenvector represent a sequence of 'snapshots'. In this study, this technique allows spatial and temporal lag compres-sions, hence suitable for highlighting standing and propagating anomalies. Tangang et al. [1997c] calculated the E E O F of the SLP anomaly of the tropical Indian and Pacific Oceans at 3 month intervals over one year period. This was implemented by stacking four temporal sequences of the SLP field into a matrix of the form / X Xitl X\t2 XNP,I XNP,2 XlJ Xlt8 Xitio Xitu * 1 , A f - 9 \ XNP,M-9 XiTM-6 M-3 M (A.8) \ XNP^O -XJVP.II • • • XNP^M ) where the NP (=299) and M (=504) were the number of spatial and monthly points respectively. The standard E O F analysis was then applied to the dimensionally smaller covariance of this stacked matrix.
Thesis/Dissertation
eng
Oceanography
Vancouver : University of British Columbia Library
For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.
Graduate
Forecasting El Niño - Southern Oscillation (ENSO) events : a neural network approach
Text