Nonlinear Complex Principal Component Analysis: Theory and Applications to Tropical Pacific W i n d Velocity Anomalies by Sanjay S! P. Rattan B.Sc. (Maths & Physics) University of the South Pacific, 1997 Meteorologist Certification New Zealand Meteorological Services/Victoria University of Wellington, 1998 M. Sc. (Wind Engineering) University of the South Pacific, 2001 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF T H E REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in T H E FACULTY OF GRADUATE STUDIES DEPARTMENT OF EARTH AND OCEAN SCIENCES We accept this thesis as confirming to the required standard T H E U N T V E K S I T Y OF BRITISH COLUMBIA April 2004 Sanjay SP Rattan, 2004 Library Authorization In presenting this thesis in partial fulfillment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Name of Author (please print) Date (dd/mm/yyyy) Title of Thesis: /JortttNf fi/t fewest 6**,/i*f &y-r / fry^ y^U.' toy Q^J fijplt i+fayJ •£ 'top? /e^S^U Degree: Year: Department of The University of British Columbia Vancouver, BC Canada Sci 11 Abstract Principal component analysis (PCA) has been generalized to complex principal component analysis (CPCA), which has been widely applied to complex-valued data, 2-D vector fields, and complexified real data through the Hilbert transform. Nonlinear PCA (NLPCA) can also be performed using auto-associative feed-forward neural network (NN) models, which allows the ex-traction of nonlinear features in the dataset. This thesis introduces a nonlinear complex PCA (NLCPCA) method, which allows nonlinear feature extraction and dimension reduction in complex-valued datasets. The NLCPCA uses the architecture of the NLPCA network, but with complex variables (including complex weight and bias parameters). Applications of NLCPCA to two test problems confirm its ability to extract nonlinear features missed by the CPCA. With complexified real data, the NLCPCA performs well as nonlinear. Hilbert PCA. The NLCPCA is also applied to the tropical Pacific wind velocity data to study the nonlinear seasonal, interannual, decadal and decadal variability of El Nifio and La Nina. The nonlinear mode of NLCPCA for the analysis of data at all these frequencies is found to be explaining more variance and features simultaneously than the equivalent linear approach of CPCA with the same dimensionality. For the interannual variability the NLCPCA mode 1 is able to characterise the whole ENSO phenomenon in a single mode whereas it took CPCA to do the same with at least 2 modes. The variances explained by NLCPCA (17.4%) is certainly higher than CPCA mode 1 (15.3%). The data set with the seasonal variability is found to show a nonlinear mode that explains the full seasonal cycle. The CPCA, in contrast, has the winter/summer stationary anomaly pattterns manifested in the first mode whereas the second mode is the signal for the stationary spring/fall anomaly patterns. The interdecadal background wind velocity anomaly patterns are also found to be more pronounced in the NLCPCA output. The NLCPCA mode 1 analysis of pre-and post-interdecadal regime shift shows that the El Nifio structure and amplitude have changes while there has been no significant differences in the La Nina. T A B L E O F C O N T E N T S Abstract ii Acknowledgements v Chapter 1 Introduction 1 Chapter 2 N L C P C A : Theory and Transfer Functions 6 2.1 Introduction * 6 2.2 Complex Principal Component Analysis 6 2.3 NN Implementation of CPCA 7 2.4 Nonlinear Complex Principal Component Analysis 8 2.5 Transfer functions . 10 2.6 Implementation of NLCPCA 13 Chapter 3 Testing N L C P C A on synthetic data sets 15 3.1 Introduction 15 3.2 NLCPCA of test data set 15 3.3 NLCPCA as Nonlinear Hilbert PCA 17 3.4 Conclusion 20 Chapter 4 Nonlinear Interannual Variability of Tropical Pacific W i n d Anomaly 23 4.1 Introduction 23 4.2 Data 25 4.3 Complex PCA 26 4.4 Method 27 4.5 NLCPCA Mode 1 29 4.6 Higher modes of NLCPCA 33 4.7 Conclusion 35 iv Chapter 5 Nonlinear Seasonal Variability of Tropical Pacific Wind Anomaly 37 5.1 Introduction 37 5.2 Complex P C A 38 5.3 Nonlinear Complex P C A 39 5.4 Conclusion 40 Chapter 6 Nonlinear Decadal Variability of Mean Tropical Pacific Wind Anomaly 6.1 Introduction 42 6.2 Data 43 6.3 N L C P C A mode 1 44 6.4 Conclusion 46 Chapter 7 Nonlinear Decadal Variability of ENSO Wind Anomaly 49 7.1 Introduction 49 7.2 Data 50 7.3 N L C P C A Mode 1 50 7.4 Conclusion 52 Chapter 8 Sumnmary and Conclusion 53 Bibliography 55 V Acknowledgments I wish to thank firstly my supervisor Professor William Hsieh. I am indebted to him for his guidance in the production of this thesis and the wealth of knowledge he shared with me on an understandable level. His quick comments has also made it possible for the submission of a paper from the thesis for journal publication and for which I am quite grateful. For his continued assistance in providing helpful comments especially relating to GRADS plotting software I would like to thank Dr. Aiming Wu. The following members of our research group are also acknowleged: Zhengqine Ye, Alexandre Laine, Beiwei Lu and Shuyong Li for keeping me lively throughout the research. This work is supported by the Natural Sciences and Engineering Research Council of Canada and the Canadian Foundation for Climate and Atmospheric Sciences. Finally, many thanks to all those who supported me and whose names I have missed in men-tioning. This study wouldn't have been at all successful without your continual support and encouragement. CHAPTER 1 Introduction Principal component analysis (PCA) also known as empirical orthogonal function analysis (EOF) (Jolliffe, 2002; Preisendorfer, 1988) has been widely used in the study of climate data. These data sets are generally of large dimensions and preprocessing by PCA to reduce the dimensionality (while preserving as much of the original information as possible) is common. PCA is a statistical tool that can preserve linear features into a few modes and enjoys the advantage of being easily implemented using the singular value decomposition technique (Strang, 1988). The PCA modes thus obtained are uncorrelated in time and orthogonal in space, and the first few modes may identify meaningful patterns that can be related directly to some climatic phenomenon. In the PCA of tropical Pacific sea surface temperature (SST), for example, the first three modes describe I the El Nino-Southern Oscillation (ENSO) phenomenon (Hsieh, 2001). Despite the wide applications of the PCA technique, it is limited to real variables. An encounter with 2D vector fields like horizontal winds and ocean currents is almost inevitable for meteorologists and oceanographers and a complex generalization of PCA called the complex principal component analysis (CPCA) is utilized. Like PCA, CPCA has been also widely used and standard references like Von Storch and Zwiers (1999) provide the theory and applications to climate data. One of the applications of the CPCA has been on tropical Pacific wind stress data (Legler, 1983). The CPCA of the tropical Pacific wind stress field by Legler (1983) compressed the data into three modes which described the seasonal and the ENSO variability. An equivalent scalar approach by PCA does not provide the full insight into the variability of the tropical Pacific wind stress field. Several other advantages of treating the 2D vector fields as complex variables and performing CPCA on them instead of PCA has been documented, for example, in Horel (1984) and Von Storch and Zwiers (1999). In Horel (1984) the CPCA technique has been shown to extract travelling and standing waves in 2D geophysical data sets, in contrast to the PCA approach that was only able 1. INTRODUCTION F I G U R E 1.0.1. The single-layer perceptron neural network model for nonlinear regression. The input neurons are denoted by and the output neurons by (t/j). For i — 1 : m there are m input and output neurons. Sandwiched between the input and output layers is a single hidden layer f(xi) containing k neurons. The network is composed of two parts: The first part from the input to the hidden layer is a nonlinear mapping / usually made through the hyperbolic tangent function. The second part from the hidden layer to the output yi uses a linear transfer function. For auto-associative networks, the target for the output neurons are simply the input data. Increasing the number of neurons in the hidden layer increases the nonlinear modelling capability of the network. to capture standing waves. Von Storch and Zwiers (1999) also detail another application of CPCA technique - the Hilbert PCA which is the CPCA applied to real data complexified by the Hilbert transform. PCA and CPCA techniques, nevertheless, remain linear tools. They cannot simultaneously explain the maximum global variance of the dataset while at the same time approaching local data clusters (Hsieh, 2001). For real variables, there are now a number of ways to nonlinearly generalize PCA (Cherkassky and Mulier, 1998). A common way is to use multi-layer perceptron neural network (NN) models. This type of NN has an input layer, one or more hidden layers and an output layer. A single hidden layer perceptron NN model that can be used for nonlinear regression, for example, is shown in Fig. 1.0.1. A nonlinear transfer function, usually the hyperbolic tangent, maps the input layer to the first hidden layer while the output layer uses a linear transfer function. The weights (including bias parameters) in the network are found by optimization algorithms like the backpropagation algorithm. The use of auto-associative multi-layer perceptron NN models to perform nonlinear PCA (NLPCA) was first introduced in the chemical engineering literature by Kramer (1991). Generally, the climate data to be used for input into the NLPCA model are pre-processed by PCA. The 1. INTRODUCTION 3 first few modes (principal components or PCs) are then used as the input time series, thereby significantly reducing the number of NN parameters (Hsieh, 2001). The NLPCA model then looks for a nonlinear combination of the inputs to yield a nonlinear PC. This nonlinear PCA method has been shown to have explained more variance and extracted more realistic features in a number of studies including non-climate studies (eg. Fotheringhame and Baddeley, 1997; Stamkopoulos et al, 1998). As far as the climate data is concerned, the ID NLPCA approximation of the tropical Pacific SST, for instance, has explained 63.3% of total variance of the original data set in contrast to 57.6% for the first PCA mode (Monahan, 2001). The NLPCA method further yields a nonstationary spatial pattern that is able to show the asymmetry between El Nino and La Nina, whereas the patterns captured by PCA remained stationary (Monahan, 2001). NLPCA complements other non-NN approaches to generalizing PCA, e.g. principal curves (Dong and MacAvoy, 1996; Hastie and Stuetzle, 1989; Malthouse, 1998) and independent component analysis (Orja, 1997). Apart from NLPCA, other important statistical tools have been nonlinearly generalized too: from linear regression to nonlinear regression, from canonical correlation analysis (CCA) to nonlin-ear canonical correlation analysis (NLCCA) and from singular spectral analysis (SSA) to nonlinear singular spectral analysis (NLSSA). The NLCCA method (Hsieh, 2000) has been shown to have greater prediction skills than the CCA method because the former contains correlated nonlinear structures. Similarly, one of the advantages of NLSSA over SSA has been its ability to extract strongly anharmonic signals (Hsieh and Wu, 2002). A review of the nonlinear generalizations of multivariate statistical techniques in the real domain is well documented with applications to cli-mate data in Hsieh (2004). Thus there appears to be important advantages in real climate data analysis of a nonlinear generalization of linear statistical methods. The NN has also been generalized to the complex domain. Real domain NN models are widely based on multi-layer perceptron networks which rely on real backpropagation algorithms for nonlinear optimization (Bishop, 1995). The real backpropagation algorithm has been extended to complex variables so that it can be used in a complex-valued NN (Georgiou and Koutsougeras, 1992; Hirose, 1992; Kechriotis and Manolakos, 1994; Leung and Simon, 1991; Piazza and Benvenuto, 1992). The complex-valued NNs have been used in a number of applications including blind 1. INTRODUCTION 4 equalization schemes (You and Hong, 1998) and digital radio links (Uncini et al., 1999). They have also been shown to outperform their real counterparts in many ways, for example, in learning linear geometric transformations like rotation, parallel displacement and similarity transformation (Nitta, 1997). For example, see Fig. 1.0.2 for the learning ability of both networks to perform parallel displacements of a triangle. A line with embedded dark circles is served as input data, whereas a line (with parallel displacement of —45° at a distance of \/2/2 units) with embedded unfilled circles is presented as the target data. The real-valued and the complex-valued networks are trained using this input and target data. The real-valued neural network structure has a single hidden layer of neurons between two inputs and two outputs. A similar network structure is used for a complex-valued network with a single complex input and a single complex output. After the training of the networks, new data representing a triangle was presented to the network. The complex neural network showed that it had learned parallel displacement after displacing the figure —45° at a distance of -v/2/2 units, while preserving all the features of the figure. However, the real-valued network mapped the triangle onto the original target data, showing that it has not learned to recognize parallel displacement. In the same manner, it is anticipated that with appropriate nonlinear transfer functions the complex valued NLPCAs would outperform their real-valued counterparts in learning nonlinear transformations. Therefore, since the NN has been generalized to the complex domain and several multivariate statistical schemes (in the real domain) as mentioned earlier have been nonlinearly generalized, there is a need for a nonlinear generalization of the complex PCA (called the NLCPCA). Hence the objective of this thesis is to develop and implement the theory of nonlinear CPCA on climate data sets using a NN approach. The theory of CPCA is introduced in Chapter 2 followed by a theory for its generalization to NLCPCA. Chapter 2 also discusses the problems in the hyperbolic transfer function arising from singularities and nonboundedness properties in the complex domain. A solution for this is provided and finally the implementation of NLCPCA is discussed. In Chapter 3, the NLCPCA is applied to a simple test data to highlight its easily visible significance in terms of skills and feature extraction capability over CPCA of the same dimensionality. Then Gaussian noise is added to this data set to investigate the performance of NLCPCA in noisy data sets. NLCPCA used as a nonlinear Hilbert PCA is also discussed in this chapter. Chapters 4 1. I N T R O D U C T I O N 5 (a) (b) -1 -0.5 0 0.5 -1 -0.5 0 0.5 1 (C) 0.5 feoocx oooooo X X 3 0 0 C [ X X O C X "-1 -0.5 0 0.5 1 F I G U R E 1.0.2. Neural network being trained to simulate displacement in (a) in which the line with black filled circles is the input data to the NN model whereas the line with unfilled circles is the target data. The trained NN model is then applied to displace a triangle (in black filled circles) by (b) a real-valued network and (c) a complex-valued neural network (from Nitta, 1997). The real and cmplex-valued approximations in (b) and (c) are shown as lines with unfilled circles. to 7 discuss the application of NLCPCA on actual climate data set: the horizontal wind velocity anomalies of the tropical Pacific. The nonlinear interannual variability of the tropical Pacific wind anomaly is investigated in Chapter 4 while Chapters 5,6 look into the nonlinear seasonal and decadal variability. The impact of decadal variability on ENSO tropical Pacific wind anomalies is further looked into in Chapter 7. Finally, the summary of the thesis and conclusions are presented in Chapter 8. CHAPTER 2 Nonlinear Complex Principal Component Analysis: Theory and Transfer Functions 2.1. Introduction In problems of dimensionality reduction a standard tool like the singular value decompo-sition (SVD) (Strang, 1988) is utilized to reduce the dimensionality of the original data. When applied to a complex field for dimensionality problems, this method is known as the complex prin-cipal component analysis (CPCA). Apart from the SVD method there exists an alternative neural network (NN) method that can be used for implementing CPCA. The advantage that the NN method offers over the SVD is that the former can be easily nonlinearly generalized. In this chap-ter the CPCA method as derived from the viewpoint of linear algebra (SVD) is presented. Then the NN implementation of CPCA is outlined and its nonlinear generalization called the NLCPCA is established. Other subtle technicalities like the complex transfer functions and the details about the operational NLCPCA method are also discussed in this chapter. 2.2. Complex Principal Component Analysis Let Z = X + iY be a complex matrix with dimension TO x n (where m = number of variables, n = number of observations, and the row mean of Z is zero). CPCA of this matrix constructs a solution that is r-dimensional (with r <m and r < n) : (2.2.1) Z = U A r V H , where U and V are complex unitary matrices (V H is the complex conjugate transpose of V) in which the columns of U (TO X r) are eigenvectors of Z Z H and the columns of V (n x r) are eigenvectors of Z H Z (Strang, 1988). The jth column of U is usually referred to as the jth spatial 6 2.3. N N I M P L E M E N T A T I O N O F C P C A 7 pattern, loading, or empirical orthogonal function. A r (r x r) is a real diagonal matrix containing the singular values cri, ...,oy, which are the square root of the r nonzero eigenvalues of both Z Z H and Z H Z . The dimension of Z is < m because there are m — r columns of Z which are linearly dependent. The jth row of A r V H gives the complex principal component (CPC) or score of the jth mode. Since all the features explained by Z can be described by a subspace spanned by the r linearly independent columns of V , there exists a transformation described by a complex function G which projects the r coordinates of the row subspace of Z given by A r V H (r x n) back onto a matrix Zpred in m x n space: (2.2.2) Z p r e d = G ( A r V H ) . For the CPCA, the transformation G ( A r V H ) yields r (m x n) matrices of rank one corre-sponding to each eigenvector and their related singular value and CPC as described by: r (2.2.3) Z p r e d = 5~J<7iujv?) J'=I where Uj and V j are the jth columns of U and V respectively and the matrix UjUjV^ is the jth complex PCA mode. The first complex PCA mode explains the largest variance of the data matrix, Z, followed by the second CPCA mode and up to CPCA mode r which explains the least. From (2.2.1), the mapping G (in the case of the CPCA) is given simply by the linear transformation U . 2.3. N N Impiementation of C P C A The transformation G is also related to the least squares problem (Malthouse, 1998; Strang, 1988) since the idea is to find a minimum length solution between the predicted value Z p r e a and Z. This is achieved if the column space of the error matrix Y = (Z - G ( A r V H ) ) lies perpendicular to the column space of G ( A r V H ) . In the least squares sense, this is equivalent to minimizing the sum of the square of the errors via the objective function or the cost function J: 2.4. NONLINEAR COMPLEX PRINCIPAL COMPONENT ANALYSIS 8 (2.3.1) J = K - - ( G ( A ^ H ) ) « f • t= l j= l The CPCA can be performed by a simple neural network containing a single hidden layer and linear transfer functions (Fig. 2.3.1). The higher dimensional space (in) of the input is reduced linearly to a one-dimensional space at the bottleneck layer given by / : C m —> C 1 and a linear inverse mapping g : C 1 —>Cm maps from the bottleneck layer to the m-dimensional output z', such that the least squares error function: n n (2.3.2) J = J2h, - z;||2 = D < z i - e ^ ) ) ) i i 2 j = l j = l . is a minimum (with Zj the jth column of Z). For any input vector z, the bottleneck neuron is given by: (2.3.3) /(z)=wHz,. where wH is the weight vector between the inputs and the bottleneck layer. The NN approach extracts leading CPCA modes one at a time. After the first mode has been removed from the data, the second mode can be extracted by inputting the residual into the same network, and the process repeated for the higher modes. 2.4. Nonlinear Complex Principal Component Analysis Kramer's (1991) auto-associative neural network structure adapted to the complex domain (Fig. 2.4.1) can be used to nonlinearly generalize CPCA. Other methods like principal curves (Hastie and Stutzle, 1989) and IT-networks (Tan and Mavrovouniotis, 1995) can be employed too. In NLCPCA, there are 3 hidden layers of neurons, with the first layer called the encoding layer, the second, the bottleneck layer (with a single complex neuron), and the third, the decoding layer. The network in Fig. 2.4.1 can be regarded as composed of 2 mappings. The first mapping 2.4. NONLINEAR COMPLEX PRINCIPAL COMPONENT ANALYSIS 9 Z m m ,m c C ,tn c F I G U R E 2.3.1. The neural network model of CPCA for linear dimensionality re-duction from C m —> C 1 and back from C 1 -> C m . There are m input and output neurons or nodes corresponding to the m variables. The bottleneck layer is repre-sented by a single neuron /(z) which compresses the data into a one-dimensional complex time series. The weights between the two layers are given by w, and w'. / : C m —» C 1 is represented by the network from the input layer to the bottleneck layer, with the bottleneck neuron giving the nonlinear CPC. The second mapping g : C 1 —+Cra is represented by the network from the bottleneck neuron to the output layer. This is the inverse mapping from the nonlinear CPC to the original data space. Dimension reduction is achieved by mapping the multi-dimensional input data through the bottleneck with a single complex degree of freedom. It is well known that a feed-forward NN only needs one hidden layer of neurons for it to model any nonlinear continuous function / (Bishop, 1995). This hidden layer is provided by the encoding layer, while for g, it is provided by the decoding layer. For the typical 1-hidden layer feed-forward NN, the transfer function from the input to the hidden layer is nonlinear, while the transfer function from the hidden layer to the output is usually linear (Bishop, 1995). Hence the 4 transfer functions from the input to the output in Fig. 2.4.1 are respectively nonlinear, linear, nonlinear and linear. In NLCPCA, the additional encoding and decoding layers (Fig. 2.4.1) allow the modelling of nonlinear continuous functions / and g. The A;th complex neuron tki at the ith layer is given by the neurons in the previous [the(i — l)th] layer via a transfer function cTj with complex weights (w) and biases (b) : (2.4.1) 2.5. T R A N S F E R F U N C T I O N S 10 Z g(M) F I G U R E 2.4.1. The complex-valued NN model for nonlinear complex PCA (NL-CPCA) is an auto-associative feed-forward multi-layer perceptron model. There are m input and output neurons or nodes corresponding to the m variables. Sand-wiched between the input and output layers are 3 hidden layers (starting with the encoding layer, then the bottleneck layer and finally the decoding layer) contain-ing q, 1 and q neurons respectively. The network is composed of two parts: The first part from the input to the bottleneck maps the input z to the single non-linear complex principal component (NLCPC) /(z). The second part from the bottleneck to'the output z' is the inverse mapping g(/(z)). For auto-associative networks, the target for the output neurons are simply the input data. Increasing the number of neurons in the encoding and decoding layers increases the nonlinear modelling capability of the network. with i = lto4 denoting, respectively, the encoding, bottleneck, decoding and output layers, (and i = 0 the input layer). A nonlinear transfer function (described in detail in the next section) is used at the encoding and decoding layers (criandcra), whereas <72and<74 are linear (actually the identity function). 2.5. Transfer Functions In the real domain, a common nonlinear transfer function is the hyperbolic tangent function, which is bounded between -1 and +1 and analytic everywhere. For a complex transfer function to be bounded and analytic everywhere, it has to be a constant function (Clarke, 1990), as Liouville's theorem (Saff and Snider, 2003) states that entire functions (functions that are analytic on the whole complex plane) which are bounded are always constants. The function tanh(a;) in the complex domain has singularities at every ( | + I S N. Using functions like tanh(z) (without any constraint) leads to non-convergent solutions (Nitta, 1997). For this reason, early researchers (e.g. Georgiou and Koutsougeras, 1992) did not seriously consider such functions as suitable complex transfer functions. 2.5. TRANSFER FUNCTIONS 11 Hence a definition for the complex transfer function for use in the complex domain was sought. The earliest definitions were put forward by Georgiou and Koutsougeras (1992) which defined five desirable properties in choosing an appropriate complex transfer function a: (i) cr(z) = u(x, y) + iv(x, y) is nonlinear in x and y (z = x + iy; x, y£ R), (ii) a(z) is bounded, (iii) the partial derivatives ux, uy, vx, and vy exist and are bounded (where ux — uy = ^ = f ,and vx = %), (iv) cr(z) is not entire, (v) uxvy ^ vxuy. The transfer function that satisfied all the above properties as proposed by Georgiou and Koutsougeras (1992) was: (2.5.1) o-(z) = Z where c and r are real positive constants. To satisy condition (v) the constraint \z\ / 0 was also utilized. Later, Benvenuto and Piazza (1992) proposed another transfer function: (2.5.2) CT(Z) = cr(Re(z)) + ia(lm(z)) = u(x) + iv(y) This transfer function is bounded and non-analytic. For example, if a = tanh then the real and imaginary parts of this transfer function are bounded between +1 and -1 i.e. the magnitude of the transfer function is bounded by a square in the complex domain with cor-ners at (1, i), (—1, i), (—1, — i), (1, — i). Further, it is non-analytic because the Cauchy-Riemann equations = f ,^ | ^ = - f^ (Saff and Snider, 2003) are not satisfied since (2.5.2) yields | i = 1 - tanh2(a;), | ^ = 1 - tanh2(j/), | | = 0, | H = 0. The transfer function given by (2.5.2) is also known as the "split" transfer function because it separately transforms the real and the imaginary components. This has been the traditional transfer function utilized by researchers in the 1990s. Kechriotis and Manolakos (1994) successfully applied it to complex communication 2.5. TRANSFER FUNCTIONS 12 channel equalization problems. Nitta (1997) also used the same transfer functions for geometric transformations using complex-valued neural networks (although he could have simply used the linear transfer function since all his transformations were linear). Another transfer function that is also non-analtyic and bounded was proposed by Hirose (1992) and is defined as: I z I (2.5.3) a(z) = tanh( — )exp[i arg(z)] in which m is a constant inversely related to the gradient of the absolute function value \a\ along the radius direction around the origin of the complex co-ordinate. In spite of the successful applications of the three transfer functions on specific problems, they still have some disadvantages. In (2.5.1) the phase of z is the same as that of a(z) because z is basically scaled by a positive constant to yield CT(Z). Hence this transfer function scales the amplitude of the signal but preserves the phase. Thus it is less efficient in learning the nonlinear phase variations between the input and the output of the NN. The transfer function given by (2.5.3) also preserves the phase. The most widely used transfer function (2.5.2), however, has the ability to learn phase and amplitude variations but it does not have the ability to learn as effectively since it is a non-analytic transfer function. Actually, all the three transfer functions have focussed on the boundedness property, totally neglecting the requirement for analyticity. Recently, a set of elementary transfer functions has been proposed by Kim and Adali (2002) with the property of being almost everywhere (a.e.) bounded and analytic in the complex domain. The complex hyperbolic tangent tanh(z), is among them, provided the complex optimization is performed within certain constraints on z. If the magnitude of z is within a circle of unit radius ^, then the singularities do not pose any problem, and the boundedness property is also satisfied. In reality the dot product of the input and weight vectors may be > ^. Thus a restriction on the magnitudes of the input and the weights has to be considered. This method had been used earlier by Leung and Haykin (1991) in which they constrained the magnitude of the complex sigmoidal transfer function to some region in the complex plane and successfully applied their method to simple pattern classification problems. However, other authors in the last decade did not utilize 2.6. IMPLEMENTATION OF NLCPCA 13 this method since their algorithms (backpropagation algorithm) were based on a gradient descent method which does not have any mechanism for constraining the weights in the NN. In this thesis, as will be described later, a quasi-Newton method instead of the traditional gradient descent method alleviates this problem since it allows the use of penalty terms that keep a restriction on the maximum values of the NN weights. For the NLCPCA model (Fig. 2.4.1), the magnitude of input data can be scaled (e.g. by dividing each element of the rth row of Z by the maximum magnitude of an element in that row, so each element of Z has magnitude < 1). The weights at the first hidden layer are randomly initialized with small magnitude, thus limiting the magnitude of the dot product between the input vector and weight vector to be about 0.1, and a weight penalty term is added to the objective function J to restrict the weights to small magnitude during optimization. The weights at subsequent layers are also randomly initialized with small magnitude and penalized during optimization by the objective function (2.5.4) j = J2h-*'A2+p E K l + r ( i 3 = 1 \l=l 1=1 2 2 ,(3) where denotes the weight vectors (including the bias parameters) from layers i = 1,2,3 and p is the weight penalty parameter (with typical values from 0.01 to 0.1 used). 2.6. Implementation of N L C P C A Since the cost function J is a real function with complex weights, the optimization of J is equivalent to finding the minimum gradient of J with respect to the real and the imaginary parts of the weights. All the weights (and biases) in Fig. 2.4.1 are combined into a single weight vector s. Hence the gradient of the cost function with respect to the complex weights can be split into (Georgiou and Koutsougeras, 1992): 2.6. IMPLEMENTATION OF NLCPCA 14 where s R and s 1 are the real and the imaginary components of the weight vectors respectively. During optimization the real and the imaginary components of the weights were separated and kept in a single real weight matrix while optimization was done by the MATLAB function "fminunc", a quasi-Newton algorithm. For the input test data sets described later, the input variables were normalized by removing their mean and the real components were divided by the largest standard deviation among the real variables while the imaginary components were divided by the largest standard deviation among the imaginary components. Division by the individual variable's standard deviation was not done in order to avoid exaggerating the importance of some variables. The number of hidden neurons, q, in the encoding/decoding layer of the NN model (Fig. 2.4.1) was varied from 2 to 10. Large values of q had smaller mean square errors (MSE) during training but led to overfitted solutions due to excessive number of network parameters. Based on a general principle of parsimony, q = 3 to 6 was found to be an appropriate number for the NN in this study. Suitable values of penalty parameter p ranged from 0.01 to 0.1. For q = 6, an ensemble of 25 randomly initialized neural networks was run. Also, 20% of the data was randomly selected as test data and withheld from the training of the NN. Runs where the MSE was larger for the test data set than for the training data set were rejected to avoid overfitted solutions. The NN. with the smallest MSE over the test data was selected as the solution for NLCPCA mode 1 and compared with the CPCA mode 1. CHAPTER 3 Testing N L C P C A on synthetic data sets 3.1. Introduction To develop intuition about the implementation of the NLCPCA model and its results a test data set is first constructed. This data set is chosen in such a way that all the variables are nonlinearly related and the magnitudes of the three vectors are relatively close to each other as they would be for a real application. The NLCPCA is applied to a test problem with 3 complex variables given by: (3.1.1) zi(t) = -+ i cos (^ ) , 77 Z (3.1.2) z2(t) = -+tsin(5). 7T Z (3-1.3) „ ( , ) = ^ + , ( f i 2 ! ^ ) ! _ I ) , where t denotes a real value from —7r to 7r at increments of 7r/100. This noiseless dataset is analyzed by the NLCPCA and CPCA methods. 3.2. N L C P C A of test data set The method described in section 2.6 was used to carry out the NLCPCA. The input variables (rows of Z from 3.1.1 to 3.1.3) were first normalized. The scaled data were then bounded within a circle of magnitude \ as described in section 2.5. NLCPCA was then performed on this processed data. The number of hidden neurons, q, used in the hidden layers of the neural network 15 3.2. N L C P C A O F T E S T D A T A S E T 16 Ja) z. (b) z, (NL) (c) z, (L) ; l d ) \ (fi) \ (NL) ffl z? (L) : (91. V .. (h) z3 (NL) 0) z3(L) j N v N . . v 7 • i T i l 1 i 1 . i . . t . . i i . . . ( . . . ( . . . ] . . . . [ . . . } . . / . . . . / . • k - . U . f r I" f t-]•'!'• i •••i 4 i i i i i i K"' J L 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 Time step F I G U R E 3.2.1. Plot of the complex time series (for every 10th data point) of. z\,Z2,zz and their predicted series by the NLCPCA mode 1 (NL) (with q = 6 and p = 0.01) and by the CPCA mode 1 (L). Each complex number is plotted as a vector in the complex plane. The time series have been vertically displaced by multiples of 10 for better visualization. model (Fig. 2.4.1) was 6 and the penalty term used was 0.01. An ensemble of 25 neural networks (with random weights of magnitude as outlined in section 2.5) were run. Also, 20% of the data was randomly selected as test data and witheld from the training of the neural networks. Runs where the mean square error was larger for the test data set than for the training data set were rejected to avoid overfitted solutions. The neural network with the smallest mean square error was selected as the solution. The NLCPCA mode 1 results are shown in Figs. 3.2.1, 3.2.2 together with CPCA mode 1 for comparison. In Fig. 3.2.1 the features of the complex time series in all the three variables have been more accurately captured by NLCPCA mode 1 than by the CPCA mode 1. In all the three variables, the magnitude and argument of the complex time series extracted by the NLCPCA mode 1 resemble the input signal, whereas the CPCA shows considerable difference. For example, for zi between the 10th to 50th time points, the CPCA mode 1 results are smaller in magnitude and are oriented more horizontally when compared to the original signal. Fig. 3.2.2, a plot in the complex plane of the three variables, also shows clearly the close resemblance to the original signal by the NLCPCA mode 1 as opposed to CPCA mode 1. The NLCPCA mode 1 accounts for 99.2% of the total 3.3. N L C P C A A S N O N L I N E A R H I L B E R T P C A 17 I n -1: S i \ \ h T\ ~" i \ v 1/ \\ / - - \ \ / \ x / \ / \ A \ / -V / 0 Re (z,; F I G U R E 3.2.2. Complex plane (imaginary component versus real component) plots of the complex time series (a) zi(t) (b) z2(t) and (c) z3(t) (all in solid line). The NLCPCA mode 1 (dot-dash line) is also shown together with the CPCA mode 1 (dashed line). variance, versus 53.6% for the CPCA mode 1. The root mean square error (RMSE) is 0.057 for NLCPCA mode 1, versus 0.631 for CPCA mode 1. The performance of NLCPCA in a slightly noisy data set (with 10% Gaussian noise added to 3.1.1 to 3.1.3) was studied next. Again the NLCPCA mode 1 was able to capture the essential features of the underlying signal in the noisy data set. The NLCPCA mode 1 explained 97.7% of the total variance of the noisy data set, in contrast to 53.3% by the CPCA mode 1. When the CPCA and NLCPCA modes recovered from the noisy data set were compared with the original noiseless data set, the RMSE of 0.107 for the NLCPCA compares favorably with 0.633 for the CPCA. For comparisons the original noiseless time series were also correlated with the retrieved signals by NLCPCA and CPCA mode 1 from the noisy data showed that the NLCPCA attained correlations of 0.99 for all six dimensions (i.e.xi, yi, x 2 , y 2 , x 3 , y3) while the CPCA correlations ranged from 0.52 to 0.80. The NLCPCA was further tested at higher noise levels, where the amount of Gaussian noise added was. increased from 10% to 50% that of the signal, with the extracted mode remaining satisfactory. 3.3. N L C P C A as Nonlinear Hilbert P C A Another common application of CPCA is in Hilbert PCA, where a real data set is first complexified by a Hilbert transform, and then analyzed by CPCA (Horel, 1984; Von Storch and Zwiers, 1999). Here we shall use NLCPCA to nonlinearly generalize Hilbert PCA. 3.3. N L C P C A A S N O N L I N E A R H I L B E R T P C A 18 A Hilbert transformation complexifies a real time series by denning the imaginary component to be the original real time series phase-shifted by ^ at each frequency w. Suppose a real time series x(t) has the Fourier representation (Von Storch and Zwiers, 1999) (3.3.1) i(t) = ^ d ( W ) e - 2 , i u l CO Its Hilbert transform is: (3.3.2) y(t)=xiiT(t) = z2^THe-U> where aHT(u;) is given as: •2nitjt a H T H = < ia(tj) for u) < 0 -ia(oj) for w > 0. For a simple test problem consider three stations recording waves coming to the shore. The first station is far from the coast, so the wave (xi(t)) looks sinusoidal; the second measurement (x2(t)) is closer to the shore, so the wave is steeper, with a tall, narrow crest and a shallow, broad trough; and the third one (x3(t)) is closest to the shore, so the wave is very steep due to strong nonlinear dynamics. Let ui = IT/12, and (3.3.3) xi(t) = sin(ti)t), (3.3.4) x 2 ( t ) = f 2 ( t J t - j ) , (3.3.5) x3(t) = /3(wt - | ) , where 3.3. NLCPCA AS NONLINEAR HILBERT PCA 40 35 28 21 14 i 1 1 1 1 1 1 1 i r >...in A A HL_A A A J"-20 40 60 80 100 Time 120 140 160 180 200 F I G U R E 3.3.1. The real (x) and the imaginary (y) components of the complexified time series by Hilbert transform in three dimensions (only first 200 points shown). The time series have been vertically displaced by multiples of 7 for better visual-ization. f2{B) h(0) 2 for 0 < 6 < ^ -1 for ^ < 9 < 2TT, 3 for 0 < 0 < § -1 for f < 6 < 2ir. The real time series together with their Hilbert transforms are shown in Fig. 3.3.1. To each of the real time series of length 360 points, 10% Gaussian noise was added. The time series were then complexified via the Hilbert transform. These complex time series (Hilbert 3.4. C O N C L U S I O N 20 transforms) were then analyzed by NLCPCA and CPCA. The noisy time series and the extracted mode 1 solutions by NLCPCA and by CPCA are shown in Fig. 3.3.2. The NLCPCA mode 1 explained 97.7% of the total variance of the noisy data set, in contrast to the 85.4% explained by the CPCA mode 1. For comparisons the original noiseless time series were correlated with the retrieved signals by NLCPCA and CPCA mode 1 from the noisy data showed that the NLCPCA attained correlations of 0.99 for all six dimensions (i.e. x\, y\, x2, y2, x3, yf) while the CPCA correlations ranged from 0.85 to 0.95. The RMSE between the mode 1 solution extracted from the noisy data and the original noiseless signal was 0.269 for the NLCPCA, versus 0.736 for the CPCA. Further tests with the added Gaussian noise increased from 10% to 50% shows the signal extraction remaining satisfactory. 3.4. Conclusion An application of the NLCPCA on test data sets shows that NLCPCA has better skill; it explains more variance than the CPCA method and has lower RMSE. Even under noisy conditions NLCPCA is able to extract the underlying signal fairly accurately while explaining better skill. A summary of the skills explained by modes 1 of CPCA and NLCPCA are given in Table 1 for the test problem described in section 3.2 and 3.3. CPCA NLCPCA Non-noisy data % Variance 53.6 99.2 Non-noisy data RMSE 0.631 0.057 Noisy data (10% Gaussian noise) % Variance 53.3 97.7 Noisy data (10% Gaussian noise) RMSE 0.633 0.107 T A B L E 1. Skills (% variance and root mean square error (RMSE)) explained by mode 1 of CPCA and NLCPCA of the problem in section 3.2. CPCA NLCPCA Noisy data (10% Gaussian noise) % Variance 85.4 97.7 Noisy data (10% Gaussian noise) RMSE 0.736 0.269 T A B L E 2. Skills (% variance and root mean square error (RMSE)) explained by mode 1 of CPCA and NLCPCA of the problem is section 3.3. The features extracted by NLCPCA are also much closer to the underlying signal than the CPCA method. The NLCPCA method captures even fine details of the original signal in its 3.4. C O N C L U S I O N 200220 420440 640 F I G U R E 3.3.2. The real and the imaginary components of the noisy complex time series (200 points only) in the three dimensions z i , Z 2 , Z 3 and the corresponding predicted time series by NLCPCA mode 1 (NL) (with q = 6, and p = 0.01) and by the CPCA mode 1 (L). The time series have been vertically displaced by multiples of 7 for better visualization. The length of each time series is 200 points, with each component starting from 0 for the noisy data, from 220 for the NLCPCA result, and from 440 for the CPCA result. There is a horizontal gap of 20 points between them for better visualization. 3.4. C O N C L U S I O N 22 approximation; an ability lacked by CPCA. This ability of NLCPCA is further confirmed by a nonlinear Hilbert transformation technique. In this study the nonlinear Hilbert transform is only briefly introduced and its rigorous theory together with an application on climate data is an area for further research. In the next four chapters the success of the NLCPCA method is further explored from its application to the tropical Pacific wind variability. CHAPTER 4 Nonlinear Interannual Variability of Tropical Pacific Wind Anomaly 4.1. I n t roduc t i on The tropical Pacific is manifested with an interannual variability pattern known as the El Nino-Southern Oscillation, abbreviated as ENSO (Busalacchi and O'Brien, 1981). This is a combined oceanic and atmospheric phenomenon in which El Nino is the oceanic part and the Southern Oscillation is the atmospheric component (Philander, 1990). The term El Nino refers to the unusual interannual warmings that occur in the equatorial Pacific and are responsible for global changes in climate patterns. The opposite of El Nino called the La Nina refers to an unusually cold equatorial Pacific. On the other hand, the Southern Oscillation which is also an interannual variability is measured by the seesaw in the sea level pressure (SLP) between Darwin and Tahiti. El Nino is signalled by an unusually high SLP near Darwin and low SLP near Tahiti whereas for La Nina it is the opposite. The ENSO is a coupled ocean-atmosphere interannual phenomenon. Its signature is evident in both oceanic variables like the sea surface temperature (SST) and sea level height and atmospheric variables like the wind velocities and the sea level pressure (SLP). Several studies (e.g. Philander, 1990) outline the changes in the variables associated with the positive and negative phases of ENSO. Prior to El Nino the equatorial easterlies and southeasterlies (Wyrtki and Meyers, 1975a,b) are stronger than normal west of the dateline (Rasmusson and Carpenter, 1981). Consequently, this produces a pileup of water in the western half of the Pacific basin that occurs normally during October-November prior to El Nino (Rasmusson and Carpenter, 1981; Wyrtki and Meyers, 1976). The subsequent changes in the ocean and the atmosphere have been summarized by Rasmusson and Carpenter (1981) as sub-El Nino phases called onset, peak, transition and mature. Later, Wang 23 4.1. INTRODUCTION 24 and Weisberg (2000) changed the terminology "peak" to "development" and added a final phase called the "decay" phase. During the onset of El Nino the easterlies in the western Pacific suddenly relax and induce an eastward propagating Kelvin wave. This in turn is responsible for the eastward migration of the warm pool of water that had accumulated in the western Pacific. Subsequently the central Pacific SST increases. In the development stage, the equatorial westerly wind anomalies in the western Pacific are further developed both in magnitude and fetch (Wang and Weisberg, 2000) and warm SST approach the far eastern Pacific. In the transition phase a large scale warming occurs in the central and eastern equatorial Pacific and the equatorial westerly winds continue their development and now penetrate eastward to 120°W. In the mature phase the equatorial eastern Pacific shows maximum SST anomalies. There is also an anomalous off-equatorial high SLP in the western Pacific (Wang and Weisberg, 1999) that induces easterly wind anomalies in the western equatorial Pacific. This easterly wind burst forces an eastward propagating upwelling Kelvin wave that assists in the decay of El Nino. As these easterly wind bursts increase in magnitude and fetch the final stage of El Nino called the decay phase is reached. In this phase the warm SSTs in the eastern Pacific begin to decay and the region of westerly wind maximum in the central Pacific is pushed further south of the equator. Several dynamical models (eg. Zebiak and Cane, 1987) and statistical models (eg. Wu and Hsieh, 2001) have been implemented to simulate and predict ENSO. The Zebiak and Cane model is a coupled ocean-atmosphere model which does not have any anomalous external forcing and reproduces certain key features of the observed phenomenon. The ENSO characteristic features like the irregular period and the SST and wind anomaly patterns are adequately represented by this model. The statistical model in Wu and Hsieh (2002) is a nonlinear canonical correlation analysis (NLCCA) model (Hsieh, 2001) that uses SLP as the predictor and the SST as the predictands. This model is still under experimentation but has been seen to be yielding reasonable long term predictions. Other hybrid NN-dynamical models also exist (e.g. Tang and Hsieh, 2002) that have comparable prediction skills to the traditional dynamical models. Some models also give further fine details in ENSO development, for example, the role of westerly and easterly bursts in the western Pacific have been well modelled in conceptual and ocean-atmosphere models (e.g. Belamari et al., 2003; Wang et al., 1999). 4.2. DATA 25 The dynamical and the statistical E N S O models until early last decade utilized linear statistical tools like the P C A for E N S O or interannual variability modelling. However, with the advent of N N , this limitation has been removed and several multivariate statistical methods have been nonlinearly generalized. As far as the complex fields are concerned the C P C A method had been utilized as early as in the late 1970s by Hardy (1977) to extract linear features from the horizontal wind field. More detailed applications of C P C A technique to the horizontal wind field is found in Hardy and Walton (1978) and Legler (1983). The application of C P C A by Hardy and Walton (1978) is confined to a mesoscale region where the wind velocity data length is just a 12-month record at ten spatial points. The method of C P C A was shown to have provided a reasonable linear approximation of the spatial and temporal patterns presented in the original data. A further direct approach of C P C A to tropical horizontal wind variability is evident in Legler (1983). However, since the seasonal cycle was not removed from this data it falls short of fully extracting the linear interannual variability from the tropical Pacific horizontal wind anomalies. Hence, to the knowledge of the author, there does not appear to have been any studies that extracted the interannual wind variability from the tropical Pacific wind velocity data using the C P C A method. In this chapter, the C P C A and the N L C P C A models are applied to the tropical Pacific wind velocity data. As shown in the test problems of the earlier chapter, the superiority of N L C P C A over C P C A is once again established in this chapter for a climate data set. The characteristics of C P C A mode 1 are compared with the mode 1 of N L C P C A . Higher modes called the modal and the nonmodal 2D C P C A and 2D N L C P C A are also compared in terms of the features and the skills they represent. 4.2. Data The data used in this thesis was monthly wind data from the Florida State University (FSU) pseudo-stress analysis (Stricherz et al., 1997) of ship and buoy data. Consider a wind field Z = X + iY where X and Y are mx n matrices of the zonal and meridional components of the wind, respectively. These components are calculated using the zonal and meridional wind stress data (L x and L„): X = ^ / ( L ^ + L j ) 1 / 4 , Y = Ly/(L2X + L2y)^4 (Wang and Weisberg, 2000; Wyrtki n 4.3. C O M P L E X P C A 26 401 1 i 1 1 • 1 i—: 1 r -601- . • . -80 ' 1 1 1 1 1 1 1 1 . -100 -80 -60 -40 -20 0 20 40 60 80 Real (CPC 1) F I G U R E 4.3.1. The complex PCA mode 1 of the tropical Pacific monthly wind anomaly with imaginary CPC 1 plotted against its real component. The new real axis after rotation is shown by the dashed line. and Meyers, 1975). The data period is January 1961 through December 1999, covering the whole tropical Pacific from 124°E to 70°W, 29°S to 29°N with a grid of 2° by 2°. 4.3. Complex P C A Prior to NLCPCA, traditional complex PCA is performed on the data matrix to reduce the dimensions of the data and thus take advantage of the data compression ability of CPCA. Prior to CPCA (section 2.2), the data had the climatological monthly mean removed as described in section 4.2 and were then smoothed with a 3-month, running mean. The first two CPCs obtained accounted for 15.3% and 10.7% of the variance of the total data set. The first CPCA mode is shown in Fig. 4.3.1. This figure suggests that, without loss of orthonormality, the CPC can be rotated slightly clockwise so that most of the variance lies along the real axis as shown in Fig. 4.3.1 (Hardy and Walton, 1978) with a dashed line. The angle of rotation, 13°, was chosen so that the mean value of the argument of the rotated CPC 1 was nearly zero (Hardy and Walton, 1978). The second CPC was also rotated clockwise but by 64°. The higher modes did not require rotation since the mean of the unrotated CPCs were already near zero. 4.4. M E T H O D 27 F I G U R E 4.3.2. The first 2 CPCA modes (i.e., eigenvectors) showing (a) mode 1 and (b) mode 2, of the tropical Pacific monthly wind anomaly (where the seasonal cycle has been removed). The wind field spatial patterns (eigenvectors) of CPCA modes 1 and 2 are shown in Fig. 4.3.2. These patterns when multiplied with their corresponding CPCs yield a standing oscillation. CPCA mode 1 is manifested with the maximum El Nino-La Nina SST anomaly patterns whereas the CPCA mode 2 is the signal during El Nino-La Nina SST decay. During maximum El Nino, for example, the equatorial Pacific has predominantly anomalous westerly winds (Rasmusson and Carpenter, 1982) and this pattern is evident in eigenvector 1 (Fig. 4.3.2a). In the decay phase of El Nifio, for example, there is an anomalous easterly wind burst (Wang and Weisberg, 2000) in the western Pacific and an anomalous westerly winds elsewhere near the equatorial Pacific (Fig. 4.3.2b). That CPCA mode 2 is indeed the decay signal is evident from the real components of the two CPCs (Fig. 4.3.3) which depict that mode 2 peaks normally after mode 1, hence indicating that mode 2 is the decay signal of ENSO. The real component is usually chosen for relating ENSO variability because after rotation the real component contains most of the variance. Along the imaginary time coefficients, there is almost insignificant change in the El Nino and La Nina patterns. 4.4. Method The CPCA and the NLCPCA models as described in detail in chapter 2 are used in this chapter for the extraction of linear and nonlinear interannual variability, respectively from the tropical Pacific wind velocity data. The main difference in the analysis is that for the analysis of wind data the spatial dimensions are quite large (m = 485 in this chapter as opposed to m = 3 4.4. METHOD 28 5 4h - 3 1 — 1960 1965 1970 1975 1980 Year 1985 1990 1995 2000 F I G U R E 4.3.3. Plot of the normalized real components of CPC 1 (in solid line) and CPC 2 (in dash line) from January 1961 to December 1999 indicating that mode 2 generally peaks after mode 1. The tickmarks indicate the January of that year. in the test problems of chapter 3). Utilizing the CPCs instead of the raw data enables these large data dimensions to be reduced before being input into the NLCPCA model. Hence it reduces the number of parameters to be used in the NN model and thus avoids unwanted over-fitting. For the matrix Z = X + iY as described earlier with dimension (485 x 600) a complex principal component analysis (CPCA) of Z seeks a solution that contains r-dimensional (with r < 485) linearly.independent complex unitary vectors or eigenvectors in the columns of Q i (485 x r) such that: in which the rows of A (r x 600) are the complex principal components. For NLCPCA in this chapter a nonlinear combination of the complex pricincipal components (CPCs) is derived (using the structure shown in Fig. 2.4.1 but now the inputs are CPCs instead of the original data) and then multiplied with Q i to yield the predicted data. Usually the first 3 to 6 CPCs are used as the input a into the NLCPCA model to yield a nonlinear CPC /(a) given by the bottleneck neuron. (4.4.1) Z = Q i A 4.5. NONLINEAR COMPLEX PCA MODE 1 29 The transformation mapping /(a) back to the data space is given by g(/(a)) so that the objective function (J) as in (2.5.4) is minimized. 4.5. Nonlinear Complex P C A Mode 1 The six leading CPCs (obtained from section 4.3) are used as the input to the NN model (of Fig. 2.4.1). Further, these six CPCs contain sufficient El Niho-La Nina variability. Higher modes contain low variability and noise, neglecting them do not cause any problem since the interest is to study the El Nino-La Nina phenomenon. The input CPCs were also scaled for optimal performance of the optimization algorithm. These input variables were normalized by removing their mean and the real components were divided by the largest standard deviation among the 6 real CPCs while the imaginary components were divided by the largest standard deviation among the 6 imaginary CPCs. Division by the individual CPCs standard deviation was not done in order to avoid the exaggeration of some variables. The number of hidden neurons, q, used in the encoding/decoding layers of the neural network model (Fig. 2.4.1) was varied from 2 to 10. Larger number of hidden neurons had lower mean square error during the neural network training. However, large values of q led to overfitted solutions and increased the number of network parameters. Based on a general principle of parsimony, q = 3 to 6 were used as the suitable number of neurons for the neural network model in this study. Suitable values of penalty terms used ranged from 0.01 to 0.1. For q = 6, an ensemble of 25 neural networks (with random initial weights of magnitude as outlined in section 2.5) were run. Also, 20% of the data was randomly selected as test data and withheld from the training of the neural networks. Runs where the mean square error was larger for the test data set than for the training data set were rejected to avoid overfitted solutions. The neural network with the smallest mean square error was selected as the solution. The first NLCPCA mode was rotated like the CPCs in section 4.3. Let the rotated NLCPCs be given by: (4.5.1) c = f{a)e 4.5. NONLINEAR COMPLEX PCA MODE 1 30 where 6 is the angle of rotation, then due to nonlinear transfer functions in the network, the weights between the bottleneck and the third hidden layer, w ' 3 ' , needs to be rotated instead of the eigenvectors. The new set of weights between the bottleneck and the third hidden layer is then given by: (4.5.2) w< 3 ) , = w < 3 > e - j e \ " / (new) The first NLCPC mode was rotated by - 9 0 ° while the weights w^ 3 ) were rotated by 90°. The NLCPCA mode 1 explains 17.4% of the total variance of data compared to 15.3% explained by CPCA mode 1. The NLCPCA mode 1 yields nonstationary spatial patterns in contrast to CPCA mode 1 which yields a standing oscillation pattern, with the amplitude varying in time. Hence no single pattern can be associated with the NLCPCA in contrast to the CPCA. This feature extraction capability of NLCPCA lies in its ability to capture lower-dimensional structure in data sets as was observed in NLPCA for the real domain SST data (Hsieh, 2001). Four spatial patterns of NLCPCA mode 1 corresponding to points near miniumum real /(a), half minimum real /(a), half maximum real /(a), and maximum real /(a), are shown in Figs. 4.5.1,4.5.2. These four patterns represent strong La Nina, about half the extreme La Nina, about half the extreme El Nino and strong El Nino. In Fig. 4.5.2a (strong La Nina conditions) the equatorial Pacific displays anomalous easterly winds, with the strongest winds in the equatorial western Pacific. As the negative real component of NLCPC 1 decreases to about half its magnitude, the easterly wind anomalies weaken over the equatorial Pacific as shown in Fig. 4.5.2b to about half the maximum La Nina wind velocity magnitudes. These wind velocity magnitudes decrease further until they reach the neutral condition where the real component of /(a) is near zero. The positive real /(a) represents El Nino conditions. Under El Nino conditions, the tropical Pacific wind field has reversed in direction (Figs. 4.5.2c,d). In Fig. 4.5.2d during maximum positive real /(a), an easterly wind burst is observed in the western Pacific together with strong westerly anomalies to the east. This easterly burst causes the SST anomalies in the eastern Pacific to decay due to the eastward propagation of an upwelling Kelvin wave (Wang and Weisberg, 2000). In this stage, a westerly wind anomaly maximum of 4.5. N O N L I N E A R C O M P L E X P C A M O D E 1 31 T ! x (d) •.••<-jiwvi.;.:v> •».(•=) • -0.5 0 0.5 1 Re (NLCPC 1) F I G U R E 4.5.1. The NLCPCA mode 1 shown in the complex plane showing (a) minimum real /(a) (strong La Nina), (b) half minimum real /(a) (weak La Nina), (c) half maximum real /(a) (weak El Nino) and (d) maximum real /(a) (strong El Nino). about 8 m s - 1 is observed in the central Pacific. The role of the westerly and the easterly wind bursts in the western equatorial Pacific in initiating the onset and the decay of El Nino has been explained in detail by a western Pacific oscillator model (Weisberg and Wang, 1997) and also by using a coupled ocean-atmosphere model (Wang et al., 1999). The NLCPCA mode 1 describes an asymmetric nonstationary evolution of El Nino from La Nina without any fixed number of spatial patterns. The asymmetry in strong El Nino and La Nina is evident from Fig. 4.5.2a (La Nina centre near 0°N 175°E) and Fig. 4.5.2d (El Nino centre near 5°S 160°W). In contrast, the CPCA mode 1 yields symmetrical stationary patterns for El Nino and La Nina. During minimum real CPC 1 (Fig. 4.5.3a) the patterns for strong La Nina are merely a 180° rotation of El Nino patterns (Fig. 4.5.3d). The CPCA centres for both strong 0.5 o Q_ o 4.5. N O N L I N E A R C O M P L E X P C A M O D E 1 32 (a) 6 m / s (b) 3 m / s 150E 180 150W 120W 90W 150E 180 150W 120W 90W (c) 5 m / s (d) 10 m / s F I G U R E 4.5.2. The spatial patterns of the NLCPCA mode 1 showing spatial patterns near (a) minimum real / ( a ) (strong La Nina), (b) half minimum real / ( a ) (weak La Nina), (c) half maximum real / ( a ) (weak El Nino) and (d) maximum real / ( a ) (strong El Nino). Note that different scalings are used, as indicated at the top right corner of each panel. El Nino and La Nina are near 0°N 175°E (Fig. 4.5.3a). Thus this shows that the CPCA mode 1 completely failed to characterize the asymmetry in El Nino and La Nina. The spatial asymmetry existing in El Niho-La Nina episodes as mentioned earlier could further explain the nonstationarity in the NLCPCA mode (Monahan, 2001) . The CPCA mode 1 has the traditional role of approximating standing functions (Horel, 1984) and thus misses any asymmetries between the positive and the negative phases of a function. The NLCPCA, on the other hand, also has the ability to represent more general lower-dimensional structure. For the tropical Pacific SST data (Hsieh, 2001; Monahan, 2001) the explanation for the asymmetrical NLPCA mode 1 was a mixing of PCA modes 1 and 2. Since CPCA modes 1 and 2 have complex time series, it is difficult to relate the explanation directly to the complex domain. Nevertheless, the ability to represent more variance in the real components of CPCA modes 1 and 2 through rotation allows the imaginary components of the two modes to be discarded. Thus it is now possible to plot the two real components against each other. Not surprisingly, the pattern captured is similar to Fig. 4.6. H I G H E R M O D E S O F N L C P C A 33 3 m/s F I G U R E 4.5.3. The spatial patterns of the CPCA mode 1 showing spatial patterns near (a) minimum real CPC 1 (strong La Nina), (b) half minimum real CPC 1 (weak La Nina), (c) half maximum real CPC 1 (weak El Nino) and (d) maximum real CPC 1 (strong El Nino). Note that different scalings are used , as indicated at the top right corner of each panel. 4.5.2 and depicts that both strongly positive and negative values of the real component of CPCA mode 1 are associated with the positive real component of CPCA mode 2. This implies that the asymmetry in NLCPCA mode 1 is largely due to a mixing of CPCA modes 1 and 2, and also because the El Nino-La Nina patterns in CPCA mode 2 enters NLCPCA mode 1 with the same (positive) sign. 4.6. Higher Modes of N L C P C A Modal and nonmodal higher modes can be extracted from wind velocity datasets using the NLCPCA model. For the modal mode 2, the input to the neural network was simply the residual after the extraction of NLCPCA mode 1 (i.e. the difference between the original 6 CPCs and the output of the neural network of the NLCPCA mode 1). The model parameters and the methodology remains the same as in section 4.5. NLCPCA mode 2 explained 9.2% of the variance of the original wind velocity data set. The associated standardized time series of NLCPCA mode 2 4.6. H I G H E R M O D E S O F N L C P C A 34 (a) 1960 1965 1970 1975 1980 1985 1990 1995 2000 Year F I G U R E 4.6.1. The normalized time series of real components of (a) NLCPC 1 (thick solid line) and NLCPC 2 (thin solid line), and imaginary components of (b) NLCPC 1 (thick solid line) and NLCPC 2 (thin solid line). The tickmarks indicate the January of that year. is shown in Fig. 4.6.1 together with the standardized NLCPCA mode 1. The correlations between mode 1 and mode 2 for the real and the imaginary components are low (0.03 and 0.3 respectively). Four spatial patterns of NLCPCA mode 2 corresponding to points near minimum real /(a), half minimum real /(a), half maximum real /(a), and maximum real /(a) are shown in Fig. 4.6.2. There is an equatorial and slightly stronger off-equatorial signal in the patterns; strong anomalies are evident in each of the panels near north of 20°N and south of 20°S and along the equator especially in the western Pacific. The time series and the spatial patterns both indicate that they are El Nino and La Nina features. During El Nino, for example in Fig. 4.6.2d, the South Pacific has strong southeasterly trade winds and during La Nina the southeasterlies are reversed. This pattern of wind anomalies is consistent with past observations. 4.7. C O N C L U S I O N 35 F I G U R E 4.6.2. The spatial patterns of the N L C P C A mode 2 showing spatial pat-terns near (a) minimum real / ( a ) , (b) half minimum real / ( a ) , (c) half maximum real / ( a ) , and (d) maximum real / ( a ) . 4.7. Conclusion A n application of the N L C P C A model on the tropical Pacific wind anomaly has shown that N L C P C A is a more powerful algorithm in producing ID and 2D approximations of the data set than C P C A of the same dimensionality. The patterns extracted by N L C P C A are nonstationary and there is no single pattern associated with it. The first N L C P C A mode has been seen to fully capture the E l Niho-La Nina variability. The asymmetrical positions of the wind maxima during E l Nino and L a Nina and their magnitudes are also realistic when compared with past events. In contrast, the C P C A modes are orthogonal stationary patterns that further lack the ability to characterize the asymmetry in patterns. N L C P C A , on the other hand, is able to show the asymmetry by mixing the C P C A modes. Another advantage of N L C P C A over C P C A is its ability to simultaneously describe more variance together with the features. For the ID approximation, the N L C P C A mode 1 variance (17.4%) is higher than C P C A (15.3%). The variance of 2D modal (i.e. first two ID N L C P C A modal) approximation of N L C P C A is only marginally better than the 2D C P C A approximation. The differences are not that striking as observed in the SST analysis 4.7. CONCLUSION 36 CPCA Mode % Var Explanation 1 15.3 Real CPC shows symmetrical stationary strong E N / L N 2 10.7 Real CPC shows stationary decay phase of E N / L N 2D CPCA 26.0 Shows combined features of CPCA modes 1 and 2 NLCPCA 1 17.4 Real NLCPC shows asymmetrical nonstationary E N / L N 2 9.2 Off-equatorial anomalies related to E N / L N 2D modal 26.6 Shows combined features of NLCPCA modes 1 and 2 TABLE 1. Comparison of the CPCA and the NLCPCA with the seasonal cycle removed. "EN" refers to El Nino and "LN" refers to La Nina. of Monahan (2001) due to the poor variance captured by NLCPCA mode 2. This could be due to the wind velocity data being much noisier than the SST data. A summary of the ID and 2D CPCA and NLCPCA with their variances and physical interpretations are given in Table 1. CHAPTER 5 Nonlinear Seasonal Variability of Tropical Pacific Wind Anomaly 5.1. Introduction The tropical Pacific region has seasonal changes associated with its wind fields. This is quite obvious from the well known fact that this region has variations in dominant wind patterns called the trade winds that are considered to be one of the largest and the most consistent wind fields on our globe (Wyrtki and Meyers, 1975a,b). In the Northern Hemisphere during winter and spring seasons the northeast trades are the strongest and this is also true for the Southern Hemisphere (in their winter and spring seasons) except for the direction of the wind anomalies which is southeasterly. During the summer and the fall of the respective hemispheres the anomalous trade winds weaken and reverse in direction. Another property that has been observed by Wyrtki and Meyers (1975) is that the area covered by northeast trades is smaller than the area covered by southeast trades although the northeast trades have a stronger mean wind stress. Feature extraction of the seasonal variability from the tropical Pacific wind stress fields by CPCA has been performed earlier by Legler (1983). In this analysis Legler (1983) used the data period from 1961 to 1978 with the long term mean removed. The variance explained by CPCs 1 and 2 were 35.4% and 6.8%. CPCA mode 1 was the signal for winter and summer wind stress anomaly patterns whereas mode 2 was the signal for fall and spring. This shows that it took up to 2 CPCA modes to show the patterns of all the four seasons. Further, since the 2 CPCA modes were stationary they discretised the seasonal wind anomaly patterns into two distinct patterns as was the case for the interannual variability. In chapter 4 the simplified NLCPCA model was used to study the nonlinear interannual variability of the tropical Pacific wind anomalies. However, if the seasonal mean is not pre-filtered 37 5.2. COMPLEX PCA 38 from the data set then the nonlinear seasonal variability can be studied. In this chapter the seasonal cycle is retained in the data for this purpose and the patterns extracted by CPCA and NLCPCA are compared. The solution extracted by the NLCPCA is expected to be a closed curve since the seasonality in the data imposes a periodic constraint. In R" the original Kramer's (1991) NLPCA model was incapable of extracting the periodic phenomenon from data sets and a circular node at the bottleneck as outlined in Kirby and Miranda (1996) solved the problem. The application of this for SST data is outlined in Hsieh (2001). In the complex domain, periodic solutions can be extracted without a "circular node" since the bottleneck, being a complex variable, already contains the angular information. The abilities of CPCA and the NLCPCA of equal dimensionality to model the seasonal variability is presented in this chapter. 5.2. Complex P C A The data that is used for CPCA in this section is the same as that used earlier (chapter 4) except that only the long term mean is removed as in Legler (1983) and the seasonal variability is retained. The first two CPCA modes contain 33.3% and 7.7% variance of the total data. The CPCs were again rotated so that most of the variability was captured by the real component. The physical interpretations of the first three modes are given in detail in Legler (1983). Fig. 5.2.1 shows the spatial patterns (eigenvectors) of the first two modes. Mode 1 (Fig. 5.2.1a) is the summer wind anomaly pattern for the Northern Hemisphere. When multiplied with the coefficients of CPCA mode 1 having negative real component, it yields the winter wind anomaly pattern for the Northern Hemisphere. Hence, CPCA mode 1 is the signal for the Northern Hemisphere summer and winter wind anomaly patterns. A plot of the real components of the two CPCs (Fig. 5.2.2) shows that CPCA mode 2 is a precursor event to CPCA mode 1 (mode 2 peaks before mode 1). Fig. 5.2.1b is the spring wind anomaly pattern for the Northern Hemisphere and is a precursor to Fig. 5.2.1a (summer). When multiplied with the coefficients of CPCA mode 2 with negative real component, it yields the fall wind anomaly pattern for the Northern Hemisphere. Thus CPCA mode 2 is the signal for spring and fall. Hence the CPCA modes 1 and 2 has yielded four stationary patterns corresponding to the four seasons. 5.3. N O N L I N E A R C O M P L E X P C A 39 (a) 0.1Sm/s (b) 0.15m/s 150E 180 150W 120W 90W 150E 180 150W 120V 90V F I G U R E 5.2.1. The first 2 CPCA modes (i.e. eigenvectors) showing (a) mode 1 and (b) mode 2, of the tropical Pacific monthly wind anomaly (where the seasonal cycle has been retained). -2 - 1 1 1 — 1 1 — 1 — 1 1 1 I J — I 'i l i f i i i i l i iffffifPffiffifffl i l l ' i i I I i I I i 1965 1970 1975 1980 Year 1985 1990 1995 F I G U R E 5.2.2. Plot of the normalized real components of CPC 1 (in solid line) and CPC 2 (in dash line) from January 1961 to December 1999. 5.3. Nonlinear Complex P C A The six leading CPCs from the previous section were again used as the input to the NL-CPCA model. The model parameters were same as those used in chapter 4.5. The NLCPCA modes 1 and 2 thus obtained explained variances of 35.2% and 12.4% respectively of the original data set in contrast to CPCA modes 1 and 2 variances of 33.3% and 7.7% respectively. NLCPCA mode 1, with the real component plotted against the imaginary component shown in Fig. 5.3.1 is a closed curve solution as expected due to the annual variability. The NLCPCA mode 1 patterns are again not associated with any single spatial pattern in contrast to the stationary patterns in the CPCAs. The NLCPCA mode 1 approximation consists of a series of patterns as in chapter 4.5 that can be visualized cinematographically. Moving around the closed curve in Fig. 5.3.1 thus 5.4. C O N C L U S I O N 40 X F .r E -0.5 0.5 Ra (NLCPC 1) F I G U R E 5.3.1. The NLCPCA mode 1 of the tropical Pacific wind anomaly (sea-sonal cycle not removed) with the imaginary component plotted against its real component. The data points corresponding to W, Sp, Su, F refer to the approx-imate position of the four seasons (winter, spring, summer, fall) in the complex plane. shows NLCPCA mode 1 spatial wind anomaly patterns that gradually change from one season to another. In contrast the CPCA mode 1 changes abruptly from the Northern Hemisphere summer to winter and misses the fall spatial patterns altogether (that is captured separately by CPCA mode 2). The spatial patterns corresponding to the four points marked in Fig. 5.3.1 that rep-resent closely each of the four seasons are shown in Fig. 5.3.2. A composite of the anomalous seasonal wind anomalies for each of the seasons from January 1961 to December 1999 (not shown) shows similar spatial patterns during each of the seasons retrieved by the NLCPCA. Finally, the fact that NLCPCA mode 1 describes all the four seasons in a nonstationary mode whereas it takes. CPCA modes 1 and 2 to do the same shows that NLCPCA is a nonstationary mixture of CPCA modes 1 and 2 (and higher modes). Once again, in the study of the nonlinear seasonal variability of the tropical Pacific wind anomaly, advantages of NLCPCA over CPCA similar to that during the nonlinear interannual variability study are evident. The NLCPCA mode 1 variance (35.2%) is also higher than the CPCA approximation (33.1%). Further, another important point that NLCPCA mode 1 is a 5.4. Conclusion 5.4. CONCLUSION 41 10 m/s 20N ION EQ 10S 20S 7/A (b) 1 \ , S S S S " < i \ \ \ \ \ \ y'/j i i 'i'} i 10 m/s "/•/•//•///ss 20N ION EQ 10S 20S Mil 'in vl i ,,,,,, . 120V 90V 10 m/s 11 r 11 t.t r r / / r r'r > > ' ' ' ' '.' > ' ' ' ' ' i ' ' ' '•' ' '.' ' l ^ V V V W ^ . . - . . — . . « \ . t i m . u r ^ . . . ,.iv>. 150E 180 150V 120V 90V 150E 180 150V 120V 90V F I G U R E 5.3.2. The spatial patterns of NLCPCA mode 1 during the Northern Hemisphere (a) winter (b) spring (c) summer and (d) fall. CPCA Mode % Var Explanation 1 33.3 Stationary winter/summer patterns for NH 2 7.7 Stationary spring/fall patterns for NH NLCPCA 1 35.2 Nonstationary full seasonal variability T A B L E 1. Comparison of the CPCA and the NLCPCA with the long term mean removed (seasonal cycle retained). "EN" refers to El Nino and "LN" refers to La Nina and "NH" for Northern Hemisphere. mixture of CPCA modes 1 and 2 is clearly highlighted for this data set. CPCA mode 1 is the signal for winter and summer patterns whereas CPCA mode 2 is the pattern for fall and spring. NLCPCA mode 1, on the other hand, is a combination of all the four seasons. A summary of the CPCA and NLCPCA modes with their variances and physical interpretations are given in Table 1. CHAPTER 6 Nonlinear Decadal Variability of Mean Tropical Pacific Wind Anomaly 6.1. Introduction There has been an increasing interest recently in the decadal oscillation of the tropical Pacific. The 1976-77 climatological regime shift which caused anomalous warming of the equatorial Pacific (Giese and Carton, 1999) has been of concern for further research in this field. The exact time of the shift is slightly different for different variables and for different parts of the Pacific. Some studies such as Biondi et al. (2001) discuss the interdecadal shifts that occurred in 1947 and 1977 while Latif et al. (1997) also explored the decadal shift of the anomalous 1990s. During these shifts, these studies have confirmed changes in the background wind climatology. Further, the properties of El Nino have also been observed to have changed notably (Arthur et al., 1998; Latif et al., 1997; Wang, 1995). In this chapter the nonlinear background wind anomaly changes of the tropical Pacific associated with the decadal shift is retrieved using the NLCPCA model. The decadal shift shows strong variability in the sea surface temperature (SST), heat content and wind stress fields in the equatorial Pacific (Giese and Carton, 1999) and (unlike ENSO) they also have a strong signature in the North Pacific (Giese and Carton, 1999; Wang and An, 1995; Zhang et al., 1999). One of the signatures of decadal variability in the SST field is the horseshoe pattern that extends to the northeast and the southeast from the western equatorial Pacific (Latif et al., 1997). The main difference between the ENSO and the decadal variability in the SST field is that the decadal variability is more influential in the western Pacific whereas the ENSO is predominant in the eastern tropical Pacific. Nevertheless, the tropical climate impact of both variabilities are similar (Latif et al., 1997). 42 6.2. DATA 43 Some theories (Latif et al., 1997) state that the origin of the decadal variability is related to independent ocean-atmosphere interactions in the midlatitude and the tropics, whereas others involve both the extratropical and the tropical regions through ocean-atmosphere teleconnections. In one theory, the decadal very low frequency (VLF) (Jin, 2001) is said to have transformed into coupled modes of the Pacific decadal variability through the ocean-atmosphere interaction in the tropics. In another theory, Liu (2003) states that the resonance of planetary wave basin modes may provide a mechanism for the generation of the decadal variability in the tropical ocean, and potentially, in the coupled ocean-atmosphere system. In this theory, the extratropics provides the forcing and the memory. Solomon et al. (2003) also suggest an extratropical forcing for the decadal variability which is carried to the equator by variations in transport rather than temperature of the North Pacific cell. Wang and An (1995) also suggest that the decadal signals from the extratropics are propagated to the equatorial Pacific. However, Wu et al. (2003) excludes any modulation of the tropical variability by the extratropics. Their study suggests that the tropical decadal variability is solely internally generated in the tropics irrespective of the North Pacific decadal variability generated in the extratropics. Thus these various theories on the origin of equatorial decadal variability shows that it is still not yet fully explored and further study is required. In several studies of decadal variability (Arthur et al., 2002; Giese and Carton, 1999; Zhang et al., 1997; Zhang et al., 1999) PCA has been utilized widely. As far as the decadal variability is concerned, the nonlinear subsurface thermal decadal variability has been studied (Tang and Hsieh, 2003) by the NLPCA model and has shown improvements over the PCA. Hence for the study of the nonlinear decadal variability of the tropical Pacific wind velocity in this chapter the NLCPCA model is used. The feature extraction capability and skills of NLCPCA are compared with the CPCA model of the same dimensionality. 6.2. Data The Florida State University (FSU) pseudo-stress analysis (Stricherz et al., 1997) data used in the earlier chapters is smoothed (after conversion to corresponding wind velocity data) with a 85-month running mean lowrpass filter (and annual cycle is removed) to be used for the study of the background wind variability at decadal scales. Then the CPCA is performed to extract the 6.3. NLCPCA MODE 1 44 (a) 0 . 1 5 m / s (b) 0 . 1 5 m / s F I G U R E 6.3.1. Eigenvectors of (a) CPCA model and (b) CPCA mode 2. first three leading modes (Hardy, 1977; Hardy and Walton, 1978; Horel, 1984). These CPCs are used as the input to the neural network model (Fig. 2.4.1). The input CPCs were also scaled for optimal performance of the optimization algorithm. These input variables were normalized by removing their mean and the real components were divided by the largest standard deviation of the real CPCs while the imaginary components were divided by the largest standard deviation of the imaginary CPCs. Division by the individual C P C s standard deviation was not done in order to avoid the exaggeration of some CPCs. 6.3. N L C P C A M o d e 1 The first 3 CPCs obtained from the low-pass data had variances of 47.7%, 18.1% and 11.9%. Their eigenvectors are shown in Fig. 6.3.1. The large amplitudes in the tropics of CPCA modes 1 and 2 wind field patterns suggest the association of these modes with some tropical variability (Giese and Carton, 1999). For the NLCPCA model, the NN parameters and model run were the same as in earlier chapters. The nonlinear mode 1 obtained explained 69.3% of the variance, in contrast to CPCA mode 1 variance of 47.7%. The decadal variability with respect to the time domain can be seen in the Hovmoller diagrams. Since the decadal variability has a strong signal in the equatorial Pacific (Zhang et al., 1999), a time longitude plot of the zonal component of NLCPCA mode 1 is shown in Fig. 6.3.2b near equator (latitude 1°N). For comparison, the CPCA mode 1 plot is also shown in Fig. 6.3.2a. For NLCPCA, the zonal wind anomaly (Fig. 6.3.2, 6.3.3) shows changes in anomaly signs near 1977 (turning from predominantly easterlies to westerlies in the western Pacific 6.3. NLCPCA MODE 1 45 (a) (b) — ' i ' — i r - * 'i ' » ' " " i ' ' i *i " i ' - i - " 150E 180 150W 120W 90W 150E 180 150W 120W 90W F I G U R E 6.3.2. Time-longitude plot of the reconstruction along 1°N (near equator) of mode 1 zonal wind anomalies of (a) CPCA (b) NLCPCA. The contour interval is 0.5 m/s, and areas with absolute values over 0.3m/s are shaded. while the opposite in the central and eastern Pacific). The post-1977 period favouring a westerly zonal component would have enabled a warmer central Pacific equatorial SST (Giese and Carton, 1999) and hence stronger post-1977 El Nino episodes. The CPCA plots (Fig. 6.3.2a, 6.3.4), on the other hand, show generally similar features but weaker anomalies. For example, the change in the wind anomalies from easterly to westerly near 1977 (Figs. 6.3.3,6.3.4) are much stronger in the NLCPCA results than the CPCA results. 6.4. CONCLUSION 46 2 m/s 1995 1990 1985 1980 1975 1970 1965 150E 180 150V 120V 90V F I G U R E 6.3.3. Time-longitude plot of the reconstruction along 1 ° N (near equator) of mode 1 N L C P C A wind vectors. 6.4. Conclusion The N L C P C A model sheds more light on the decadal variability than the C P C A method with the same dimensionality. The N L C P C A mode 1 for the study of the background wind anomalies described 66.8% of the total variance whereas the C P C A described 39.9% of the variance. Due to its ability to store more variance, the patterns captured by N L C P C A yield more important features than C P C A . 6.4. CONCLUSION 47 2 m/s 1995 H 1990 H 1985 H 1980 H 1975 H 19651 H I I I HI M M * 5 5 S 1 1 II I I I 4 & M v« N« *N -S I l l l i l l l i i II S S ^ S I I 1 I 1 l l l l I I £ £ & !§ j | | 1 4-g £ g 5= t= E= %%%%%% i 5 2 * 3 5 = 150E 180 150V 120W 90V F I G U R E 6.3.4. Time-longitude plot of the reconstruction along 1°N (near equator) of mode 1 CPCA wind vectors. Apart from other minor features captured by the NLCPCA model, of particular interest is its ability to highlight the 1976-77 interdecadal shift, especially in Figs. 6.3.2,6.3.3. The NLCPCA results show a sharp contrast in the sign of the anomalies during pre- and post-1975 periods. The magnitudes of these changes are also quite significant as compared to the CPCA results. Finally, ability of the NLCPCA to extract interdecadal signals could be made more obvious if a larger 6.4. C O N C L U S I O N 48 data set that includes few other interdecadal shifts (eg. 1947) is utilized. Meantime, this could be dealt as the subject of further research. CHAPTER 7 Nonlinear Decadal Variability of ENSO Wind Anomaly 7.1. Introduction During a decadal shift, since the background wind fields on which El Nino develops changes (as shown in the previous chapter) the features of El Nino could also be changing. The period, amplitude, structure and propagation have changed (Wang and An, 1995). As far as the structure of El Nino is concerned, the onset (Wang, 1995) and mature (Wu and Hsieh, 2003) stages have undergone a significant change. The horizontal wind anomaly fields of the onset stage of El Nino shows a weakening of the trade winds in the southeast Pacific during the pre-decadal shift period and opposite during the post-shift period (Latif et al., 1997; Wang, 1995). Further, the westerly wind bursts (Belamari et al., 2003) which are an indication of the onset of El Nino are related to two different anomalous features in the northwestern and southwestern Pacific. During the pre-shift an anomalous cyclone over eastern Australia is the source of westerly wind bursts in the western Pacific whereas the post-shift period westerly wind bursts are due to the presence of an anomalous cyclone over the Philippine Sea (Wang, 1995). As far as the mature state of El Nino is concerned, the wind field patterns of the post-shift mature stage has moved eastward by about 25° with respect to the pre-shift period (Wu and Hsieh, 2003). This shift has been shown with a nonlinear canonical correlation analysis (NLCCA) model and supported with a hybrid coupled model and the delayed oscillator theory in Wu and Hsieh (2003). Using a coupled ocean-atmosphere model Wang and An (1995) have also shown that the ENSO wind changes during the decadal shifts were related to the decadal shifts in the mean wind field. In this chapter the NLCPCA and CPCA models are used to extract the nonlinear decadal variability of ENSO wind anomaly patterns. 49 7.3. NLCPCA MODE 1 50 (a) 6 m/s ' (b) 6 m/s F I G U R E 7.3.1. The spatial patterns of the NLCPCA mode 1 for the 1961-76 regime for (a) minimum real /(a) (strong La Nina), (b) maximum real /(a) (strong El Nino) and for the 1981-1999 regime for (c) minimum real /(a) (strong La Nina) and (d) maximum real /(a) (strong El Nino). Note that different scalings are used, as indicated at the top right corner of each panel. 7.2. Data The long-term mean and the seasonal means were removed from the data in section 4.2 and then smoothed with a 3-month running mean to remove the seasonal variability. CPCA was performed on this dataset and and the six leading CPCs obtained were divided into 2 subsets: 1961-1976 and 1981-99 which were analyzed by NLCPCA separately. The NLCPCA model parameters, and methodology are same as that described in section 4.5. 7.3. N L C P C A Mode 1 Since there has been a large decadal shift near 1975-77 as outlined in the previous chapter, it may have had an impact on the pre- and post-1976 ENSO events since the background conditions are different for ENSO during these two periods (Wang and An, 1995). The spatial patterns corresponding to the minimum and maximum values of the real component of NLCPCA mode 1 for the pre- and post-1976 ENSO events are shown in Fig. 7.3.1. 7.3.. NLCPCA MODE 1 51 The maximum El Nino events for the two regimes from Figs. 7.3.1b,d appear to differ sig-nificantly. The pre-decadal shift maximum wind anomaly center is near 0° 175°W whereas the post-decadal shift has the center near 9°S 152°W. Thus there is a southeast shift in the peak phase of El Nino after the 1976 decadal shift (Wang, 1995) by about 30°. Further, these figures also suggest that the El Nino events after the 1976 shift have been stronger in magnitude. On the other hand, the La Nina features and magnitude have not changed significantly between the two periods. The 1961-76 El Nino features (Fig. 7.3.1b) shows similarity to the CPCA mode 1 patterns in Fig. 4.5.3d suggesting that the 1961-76 period had been mainly linear. Further there is one more important feature in the post-1976 decadal shift which is not evident in the pre-shift period. The post-shift period has strong easterly anomalies in the western Pacific and strong southeasterly anomalies in the southern Pacific. There are several studies which have documented on these easterly wind bursts. In one study, Wang et. al. (1999) used the Zebiak and Cane atmospheric model to show the western Pacific variability during ENSO. A conceptual western Pacific oscillator model (Weisberg and Wang, 1997) sheds further light on western Pacific westerly wind burst variability. Belamari et al. (2003) mentioned twin anticyclones in the opposite hemispheres that are responsible for the westerly wind burst. The unified conceptual oscillator model (Wang, 2001) further shows that this is just a special case in modelling the oscillatory nature of the ENSO. In contrast, the evidence of strong westerly anomalies is not evident during maximum La Nina and is a contradiction to Wang et al. (1999) but confirms the result of Wu and Hsieh (2003). This further confirms the results from the previous section that in the latter regime the strengthening of the anomalous equatorial westerly winds provided the shifting of the El Nino patterns southeastwards by about 30°. The increase in the southeasterlies enhanced the conver-gence near the equator that led to an increase in the amplitude of El Nino (Wang and Weisberg, 2000). Further, the 1981-1999 warm episodes are stronger than the 1978-1987 period is evidenced also by a southward extent of the southeasterly anomalies. The La Nina patterns, however, for the two regimes do not appear to have undergone any significant change. The maximum wind anomaly center during the two regimes remain near equator and 175°W. 7.4. C O N C L U S I O N 52 7.4. Conclusion The decadal variability of the background wind anomalies has influenced E l Nino, particu-larly its amplitude and structure. The maximum L a Nina events do not appear to have undergone any significant change while there has been important changes in the pre- and post-1975 E l Nino events. The post-1976 maximum E l Nino centres have moved southeastwards by about 30°. The background southeasterlies in the South Pacific have also increased due to the decadal shift in the mean wind field as discussed in the previous chapter. The magnitude of the maximum E l Nino are also larger after the post-1976 period. Further, the changes in the E l Nino features for the pre-and post-1976 periods also suggest that the post-1976 period has been more nonlinear. Thus it can be deduced that the interdecadal shift of 1976 has been quite influential on the structure and amplitude of the post-1976 E l Nino events. CHAPTER 8 S u m m a r y a n d C o n c l u s i o n PCA is widely used for dimension reduction and feature extraction. Its generalization to complex variables led to CPCA, which is also widely used when dealing with complex variables, 2-D vector fields (like winds or ocean currents), or complexified real data (via Hilbert transform). Both PCA and CPCA are linear methods, incapable of extracting nonlinear features in the data. To perform nonlinear PCA (NLPCA), Kramer (1991) used an auto-associative feed-forward neural network with 3 hidden layers. When a linear method such as PCA is applied to data with nonlinear structure, the nonlinear structure is scattered into numerous linear PCA modes- a confusing result which is largely alleviated when using NLPCA (Hsieh, 2004). While complex NNs have already been developed for nonlinear regression problems, this thesis extends complex NNs to the role of nonlinear CPCA (NLCPCA). The NLCPCA uses the basic architecture of the Kramer NLPCA model, but with complex variables (including complex weights and biases). Nonlinear transfer functions like the hyperbolic tangent can be used, though the argument of the tanh function in the complex plane must have its magnitude within a circle of radius | to avoid the singularities of the tanh function. This is satisfied by initializing with weights (including biases) of small magnitudes, and using weight penalty in the objective function during optimization. The implementation of NLCPCA on actual climate data (tropical Pacific wind velocity) to extract nonlinear interannual, seasonal, decadal and decadal ENSO variability also shows improved results over the CPCA method. In most of these cases the NLCPCA mode 1 is a mixture of CPCA modes 1 and 2 (and higher modes), is nonstationary and explains asymmetrical features. For interannual variability, the first NLCPCA mode has been seen to more fully capture the El Nino-La Nina variability. In contrast, for CPCA significant EI Nino-La Nina features are distributed to at least mode 2. For seasonal variability, CPCA mode 1 is the signal for winter and summer 53 '' 8. S U M M A R Y A N D C O N C L U S I O N 54 patterns whereas CPCA mode 2 is the pattern for fall and spring. In contrast NLCPCA mode 1 is capable of representing four seasons. The ability of NLCPCA to combine the leading CPCA modes has the additional advantage that the resultant patterns can capture asymmetry. For example, for the interannual variability data the asymmetrical positions of the wind maxima during El Nino and La Nina and their magnitudes are clearly reproduced by NLCPCA. In contrast, the CPCA modes show stationary patterns. Another advantage of NLCPCA over CPCA is its ability to simultaneously describe more variance together with better feature extraction capability. For all the datasets in this study, the NLCPCA model explained more variance than the CPCA method with the same dimensionality. In some data sets the contrast in the variance was not that dramatic, implying that this characteristic is data dependent. Another important feature extraction property of NLCPCA is observed in the study of inter-decadal variability and its influence on ENSO. The 1976-77 interdecadal shift is more pronounced in the NLCPCA output than the CPCA output. The NLCPCA model also shows important changes in the pre- and post-1976 El Nino events. The post-1976 maximum El Nino centres have moved southeastwards by about 30°. The background southeasterlies in the South Pacific have also increased. The magnitude of the maximum El Nino is also larger after the post-1976 period. Thus the interdecadal shift of 1976 has been quite influential on the structure and amplitude of the post-1976 El Nino events. The results of this thesis on tropical Pacific wind variability suggest that the NLCPCA model is better than the CPCA model in terms of simultaneous feature extraction and variance explaining capability. These advantages hence pave the way for nonlinear generalizations of other complex multivariate analysis schemes. The nonlinear Hilbert transformation briefly introduced in this study can be applied, for example, on the complexified tropical Pacific SST data. The singular spectral analysis method (Hsieh and Wu, 2002) can be also generalized to the complex domain and eventually to a nonlinear complex domain. B i b l i o g r a p h y [1] A r t h u r J M , D a n i e l R C , W a r r e n B W (1998) A w e s t w a r d - i n t e n s i f i e d d e c a d a l c h a n g e i n t h e N o r t h P a c i f i c T h e r -m o c l i n e a n d G y r e - S c a l e C i r c u l a t i o n . J Clim 11 : 3 1 1 2 - 3 1 2 7 . [2] B e l a m a r i S , R e d e l s p e r g e r J L , P o n t a u d M (2003) Dynamic role of a westerly wind burst in triggering an equatorial Pacific warm event. 16 : 1 8 6 9 - 1 8 9 0 . [3] B e n v e n u t o N , P i a z z a F (1992) O n t h e c o m p l e x b a c k p r o p a g a t i o n a l g o r i t h m . IEEE Trans Signal Proc 4 0 : 9 6 7 -9 6 9 . [4] B i o n d i F , C a y a n D R , G e r s h u n o v A (2001) N o r t h P a c i f i c d e c a d a l c l i m a t e v a r i a b i l i t y s i n c e 1 6 6 1 . J Clim 14 : 5 -10 . [5] B i s h o p C M (1995) Neural networks for pattern recognition. O x f o r d u n i v e r s i t y p r e s s . [6] B u s a l a c c h i A J , O ' B r i e n J J (1981) I n t e r a n n u a l v a r i a b i l i t y o f t h e e q u a t o r i a l P a c i f i c i n t h e 1960s . J Geophys Res 8 6 : 1 0 9 0 1 - 1 0 9 0 7 . [7] C h e r k a s s k y V , M u l i e r F (1998) Learning from Data. N e w Y o r k : W i l e y , 4 6 4 p p . [8] C l a r k e T (1990) G e n e r a l i z a t i o n o f n e u r a l n e t w o r k s t o t h e c o m p l e x p l a n e . Proc of Int Joint Conf on Neu Net 2: 4 3 5 - 4 4 0 . [9] D o n g D , M c A v o y T J (1996) N o n l i n e a r p r i n c i p a l c o m p o n e n t a n a l y s i s b a s e d o n p r i n c i p a l c u r v e s a n d n e u r a l n e t w o r k s . Comp Chem Eng 2 0 : 6 5 - 7 8 . [10] F o t h e r i n g h a m e D , B a d d e l e y R (1997) N o n l i n e a r p r i n c i p a l c o m p o n e n t a n a l y s i s o f n e u r o n a l s p i k e t r a i n d a t a . Biol Cybern 77 : 2 8 3 - 2 8 8 . [11] G e o r g i o u G , K o u t s o u g e r a s C (1992) C o m p l e x b a c k p r o p a g a t i o n . IEEE Trans. Circuits and Systems II 3 9 : 3 3 0 -3 3 4 . [12] G i e s e B , C a r t o n J (1999) I n t e r a n n u a l a n d d e c a d a l v a r i a b i l i t y i n t h e t r o p i c a l a n d m i d l a t i t u d e P a c i f i c o c e a n . J Clim 12 : 3 4 0 2 - 3 4 1 8 . [13] H a r d y D M (1977) E m p i r i c a l e i g e n v e c t o r a n a l y s i s o f v e c t o r o b s e r v a t i o n s , Geophys Res Lett 4 : 3 1 9 - 3 2 0 . [14] H a r d y D M , W a l t o n J J (1978) P r i n c i p a l c o m p o n e n t a n a l y s i s o f v e c t o r w i n d o b s e r v a t i o n s . J Appl Met 17: 1153 -1 1 6 2 . [15] H a s t i e T , S t u e t z l e W (1989) P r i n c i p a l c u r v e s . J Am Statis Assoc 8 4 : 5 0 2 - 5 1 6 . [16] H i r o s e A (1992) C o n t i n u o u s c o m p l e x - v a l u e d b a c k p r o p a g a t i o n l e a r n i n g . Elec Letters 2 8 : 1 8 5 4 - 1 8 5 5 . [17] H o r e l J D (1984) C o m p l e x p r i n c i p a l c o m p o n e n t a n a l y s i s : t h e o r y a n d e x a m p l e s . J Clim Appl Met 2 3 : 1 6 6 0 - 1 6 7 3 . [18] H s i e h W W (2000) N o n l i n e a r c a n o n i c a l c o r r e l a t i o n a n a l y s i s b y n e u r a l n e t w o r k s . Neural Networks 13 : 1 0 9 5 - 1 1 0 5 . 55 BIBLIOGRAPHY 56 [19] H s i e h W W (2001) N o n l i n e a r p r i n c i p a l c o m p o n e n t a n a l y s i s u s i n g n e u r a l n e t w o r k s . Tellus 53A: 5 9 9 - 6 1 5 . [20] H s i e h W W , W u A (2002 ) N o n l i n e a r m u l t i c h a n n e l s i n g u l a r s p e c t r a l a n a l y s i s o f t h e t r o p i c a l P a c i f i c c l i m a t e v a r i a b i l i t y u s i n g a n e u r a l n e t w o r k a p p r o a c h . J Geophys Res 1 0 7 : 3 0 7 6 . [21] H s i e h , W W ( 2 0 0 4 ) . N o n l i n e a r m u l t i v a r i a t e a n d t i m e ser ies a n a l y s i s b y n e u r a l n e t w o r k m e t h o d s . Review of Geophysics ( i n p r e s s ) . [22] J i n F F (2001) L o w f r e q u e n c y m o d e s o f t r o p i c a l o c e a n d y n a m i c s . J Clim 14 : 3 8 7 4 - 3 8 8 1 . [23] J o l l i f f e , I T ( 2 0 0 2 ) . Principal component analysis. B e r l i n : S p r i n g e r , 502 p p . [24] K e c h r i o t i s G , M a n o l a k o s E S (1994) T r a i n i n g f u l l y r e c u r r e n t n e u r a l n e t w o r k s w i t h c o m p l e x w e i g h t s . IEE Trans Circuits Systems-II Analog Digital Signal Proc 4 1 : 2 3 5 - 2 3 8 . [25] K i m T , A d a l i T (2002) F u l l y c o m p l e x m u l t i - l a y e r p e r c e p t r o n n e t w o r k f o r n o n l i n e a r s i g n a l p r o c e s s i n g . J Very Large Scale Integrated (VLSI) Sig Proc 32 : 2 9 - 4 3 . [26] K i r b y M J , M i r a n d a R (1996) C i r c u l a r n o d e s i n n e u r a l n e t w o r k s . Neural Comp 8: 3 9 0 - 4 0 2 . [27] K r a m e r M A (1991) N o n l i n e a r p r i n c i p a l c o m p o n e n t a n a l y s i s u s i n g a u t o a s s o c i a t i v e n e u r a l n e t w o r k s . Amer Inst of Chem Eng (AIChE) 3 7 : 2 3 3 - 2 4 3 . [28] L a t i f M , K l e e m a n R , E c k e r t C (1997) G r e e n h o u s e w a r m i n g , d e c a d a l v a r i a b i l i t y , o r E l N i n o ? A n a t t e m p t t o u n d e r s t a n d t h e a n o m a l o u s 1990s . J Clim 10 : 2 2 2 1 - 2 2 3 9 . [29] L e g l e r D M (1983) E m p i r i c a l o r t h o g o n a l f u n c t i o n a n a l y s i s o f w i n d v e c t o r s o v e r t h e t r o p i c a l P a c i f i c r e g i o n . Am Met Soc 64 : 2 3 4 - 2 4 1 . [30] L e u n g H , S i m o n H (1991 ) T h e c o m p l e x b a c k p r o p a g a t i o n a l g o r i t h m . IEE Trans Sig Proc 3 9 : 2 1 0 1 - 2 1 0 4 . [31] L i u Z (2003) T r o p i c a l o c e a n d e c a d a l v a r i a b i l i t y a n d r e s o n a n c e o f p l a n e t a r y w a v e b a s i n m o d e s , P a r t I: t h e o r y . J Clim 16: 1 5 3 9 - 1 5 5 0 . [32] M a l t h o u s e E C (1998) L i m i t a t i o n s o f n o n l i n e a r p r i n c i p a l c o m p o n e n t a n a l y s i s as p e r f o r m e d w i t h g e n e r i c n e u r a l n e t w o r k s . IEEE Trans Neural Net 9 : 1 6 5 - 1 7 3 . [33] M o n a h a n A H (2001) N o n l i n e a r p r i n c i p a l c o m p o n e n t a n a l y s i s : t r o p i c a l I n d o - P a c i f i c s e a s u r f a c e t e m p e r a t u r e a n d s e a l eve l p r e s s u r e . J Clim 14: 2 1 9 - 2 3 3 . [34] N i t t a T (1997) A n e x t e n s i o n o f t h e b a c k p r o p a g a t i o n a l g o r i t h m t o c o m p l e x n u m b e r s . Neural Net 10 : 1 3 9 1 - 1 4 1 5 . [35] O r j a E (1997) T h e n o n l i n e a r P C A l e a r n i n g r u l e i n i n d e p e n d e n t c o m p o n e n t a n a l y s i s . Neurocomputing 17 : 2 5 - 4 5 . [36] P h i l a n d e r S G (1990) El Nino, La Nina, and the Southern Oscillation. A c a d e m i c P r e s s , 2 8 9 p p . [37] P i a z z a F , B e n v e n u t o N (1992) O n t h e c o m p l e x b a c k p r o p a g a t i o n a l g o r i t h m . IEE Trans Signal Proc 4 0 : 9 6 7 - 9 6 9 . [38] P r e i s e n d o r f e r R W ( 1 9 9 8 ) . Principal component analysis in meteorology and oceanography.. E l s e v i e r , 4 2 5 p p . [39] R a s m u s s o n E M , C a r p e n t e r T H (1982) V a r i a t i o n s i n t r o p i c a l s e a s u r f a c e t e m p e r a t u r e a n d s u r f a c e w i n d fields a s s o c i a t e d w i t h t h e s o u t h e r n o s c i l l a t i o n / E l N i n o . Mon Weather Rev 110 : 3 5 4 - 3 8 4 . [40] S a f f E B , S n i d e r A D (2003) Fundamentals of complex analysis with applications to engineering and science. E n g l e w o o d C l i f f s , N . J . BIBLIOGRAPHY 57 [41] S o l o m o n A , M c C r e a r y J P , K l e e m a n R , K l i n g e r B A (2003 ) I n t e r a n n u a l a n d d e c a d a l v a r i a b i l i t y i n a n i n t e r m e d i a t e c o u p l e d m o d e l o f t h e P a c i f i c r e g i o n . J Clim 16: 3 8 3 - 4 0 5 . [42] S t a m k o p o u l o s T, D i a m a n t a r a s K , M a g l a v e r a s N , S t r i n t z i s M (1998) E C G a n a l y s i s u s i n g n o n l i n e a r p r i n c i p a l c o m p o n e n t a n a l y s i s n e u r a l n e t w o r k s fo r I s c h e m i a d e t e c t i o n . IEE Trans Sig Proc 4 6 : 3 0 5 8 - 3 0 6 7 . [43] S t r a n g G (1988) Linear algebra and its applications, S a n D i e g o : H a r c o u r t , B r a c e , J o v a n o v i c h P u b l i s h e r s . [44] S t r i c h e r z J N , D a v i d M L , O ' B r i e n J J (1997) TOGA pseudostress atlas 1985-1994, vol II:Pacific Ocean, F l o r i d a S t a t e U n i v e r s i t y T a l l a h a s s e e F L , U S A , p p 1-155. [45] T a n S , M a v r o v o u n i o t i s (1995) R e d u c i n g d a t a d i m e n s i o n a l i t y t h r o u g h o p t i m i z i n g n e u r a l n e t w o r k i n p u t s . AIChE 4 1 : 1 4 7 1 - 1 4 8 0 . [46] T a n g Y , H s i e h W W (2002) H y b r i d c o u p l e d m o d e l s o f t h e t r o p i c a l P a c i f i c - E N S O p r e d i c t i o n . CH Dynam 19 : 3 4 3 - 3 5 3 . [47] T a n g Y , H s i e h W W (2003) N o n l i n e a r m o d e s o f d e c a d a l a n d i n t e r a n n u a l v a r i a b i l i t y o f t h e s u b s u r f a c e t h e r m a l s t r u c t u r e i n t h e P a c i f i c O c e a n . J Geophys. Res 108 : 3 0 8 4 . [48] U n c i n i A , V e c c i L , C a m p o l u c c i P , P i a z z a F (1999) . C o m p l e x - v a l u e d n e u r a l n e t w o r k s w i t h a d a p t i v e s p l i n e a c t i v a t i o n f u n c t i o n fo r d i g i t a l r a d i o l i n k s . IEE Trans Sig Proc 4 7 : 5 0 5 - 5 1 4 . [49] V o n S t o r c h H , Z w i e r s F W (1999) Statistical analysis in climate research. C a m b r i d g e U n i v e r s i t y P r e s s . [50] W a n g B (1995) I n t e r d e c a d a l c h a n g e s i n E l N i f i o onse t i n t h e l as t f o u r d e c a d e s . J Clim 8 : 2 6 7 - 2 8 5 . [51] W a n g B , A n S I (1995) A m e c h a n i s m fo r d e c a d a l c h a n g e s o f E N S O b e h a v i o u r : r o l es o f b a c k g r o u n d w i n d c h a n g e s , CH Dynam 8: 2 6 7 - 2 8 5 . [52] W a n g C (2001) A u n i f i e d o s c i l l a t o r m o d e l fo r t h e E N S O . J CU 14: 9 8 - 1 1 5 . [53] W a n g C , W e i s b e r g R H (2000) T h e 1997 -98 E l N i n o a n d e v o l u t i o n r e l a t i v e t o p r e v i o u s E l N i n o e v e n t s . J Clim 1 3 : 4 8 8 - 5 0 1 . . [54] W a n g C , W e i s b e r g R H , V i r m a n i J I (1999) W e s t e r n P a c i f i c i n t e r a n n u a l v a r i a b i l i t y a s s o c i a t e d w i t h t h e E l N i i i o -S o u t h e r n O s c i l l a t i o n . J Geophys Res 104: 5 1 3 1 - 5 1 4 9 . [55] W e i s b e r g R H , W a n g C (1997) A w e s t e r n P a c i f i c o s c i l l a t o r p a r a d i g m f o r t h e E l N i n o - S o u t h e r n O s c i l l a t i o n . Geophys Res Lett 24 : 7 7 9 - 7 8 2 . [56] W u A , H s i e h W W (2001) F o r e c a s t i n g t h e t r o p i c a l P a c i f i c s e a s u r f a c e t e m p e r a t u r e b y n o n l i n e a r c a n o n i c a l c o r r e l a t i o n a n a l y s i s . Experimental Long Lead Forecast Bulletin S e p t 2 0 0 1 . [57] W u A , H s i e h W W (2002) N o n l i n e a r c a n o n i c a l c o r r e l a t i o n a n a l y s i s o f t h e t r o p i c a l P a c i f i c w i n d s t ress a n d s e a s u r f a c e t e m p e r a t u r e . J Clim Dynam 19 : 7 1 3 - 7 2 2 . [58] W u A , H s i e h W W (2003) N o n l i n e a r i n t e r d e c a d a l c h a n g e s o f t h e E l N i n o - S o u t h e r n O s c i l l a t i o n , submitted to J Clim. [59] W u L , L i u Z , G a l l i m o r e R , J a c o b R , L e e D , Z h o n g Y (2003) P a c i f i c d e c a d a l v a r i a b i l i t y : t h e t r o p i c a l P a c i f i c m o d e a n d t h e N o r t h P a c i f i c m o d e . J Clim 16: 1 1 0 1 - 1 1 2 0 . BIBLIOGRAPHY 58 [60] W y r t k i K , M e y e r s G (1975a ) The trade wind field over the Pacific ocean, part I, The mean field and the mean annual variation Rep HIG-75-1, H a w a i i Ins t G e o p h y s , U n i v e r s i t y o f H a w a i i , 26 p p . [61] W y r t k i K , M e y e r s G ( 1 9 7 5 b ) The trade wind field over the Pacific ocean, part II, Bimonthly fields of wind stress 1950-1972. Rep HIG-75-2, H a w a i i Ins t G e o p h y s , U n i v e r s i t y o f H a w a i i , 16 p p . [62] W y r t k i K , M e y e r s G (1976) T h e t r a d e w i n d field o v e r t h e P a c i f i c O c e a n . J Appl Met 15 : 6 9 8 - 7 0 4 . [63] Y o u C , H o n g D (1998) N o n l i n e a r b l i n d e q u a l i z a t i o n s c h e m e s u s i n g c o m p l e x - v a l u e d m u l t i l a y e r f e e d f o r w a r d n e u r a l n e t w o r k s . IEE Trans Neural Net 9: 1 4 4 2 - 1 4 5 5 . [64] Z e b i a k S E , C a n e M A (1987) A m o d e l E l N i n o - S o u t h e r n O s c i l l a t i o n . Mon Weather Rev 1 1 5 : 2 2 6 2 - 2 2 7 8 . [65] Z h a n g R H , R o t h s t e i n L M , B u s a l a c c h i A J (1999 ) I n t e r a n n u a l a n d d e c a d a l v a r i a b i l i t y o f t h e s u b s u r f a c e t h e r m a l s t r u c t u r e i n t he P a c i f i c o c e a n : 1 9 6 1 - 9 0 , CU Dynam 15 : 7 0 3 - 7 1 7 . [66] Z h a n g Y , W a l l a c e J M , B a t t i s t i D S (1997) E N S O - l i k e i n t e r d e c a d a l v a r i a b i l i t y . J CU 10 : 1 0 0 4 - 1 0 2 0 .
- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Nonlinear complex principal component analysis : applications...
Open Collections
UBC Theses and Dissertations
Featured Collection
UBC Theses and Dissertations
Nonlinear complex principal component analysis : applications to tropical Pacific wind velocity anomalies Rattan, Sanjay S.P. 2004
pdf
Page Metadata
Item Metadata
Title | Nonlinear complex principal component analysis : applications to tropical Pacific wind velocity anomalies |
Creator |
Rattan, Sanjay S.P. |
Date Issued | 2004 |
Description | Principal component analysis (PCA) has been generalized to complex principal component analysis (CPCA), which has been widely applied to complex-valued data, 2-D vector fields, and complexified real data through the Hilbert transform. Nonlinear PCA (NLPCA) can also be performed using auto-associative feed-forward neural network (NN) models, which allows the extraction of nonlinear features in the dataset. This thesis introduces a nonlinear complex PCA (NLCPCA) method, which allows nonlinear feature extraction and dimension reduction in complexvalued datasets. The NLCPCA uses the architecture of the NLPCA network, but with complex variables (including complex weight and bias parameters). Applications of NLCPCA to two test problems confirm its ability to extract nonlinear features missed by the CPCA. With complexified real data, the NLCPCA performs well as nonlinear. Hilbert PCA. The NLCPCA is also applied to the tropical Pacific wind velocity data to study the nonlinear seasonal, interannual, decadal and decadal variability of El Niño and La Niña. The nonlinear mode of NLCPCA for the analysis of data at all these frequencies is found to be explaining more variance and features simultaneously than the equivalent linear approach of CPCA with the same dimensionality. For the interannual variability the NLCPCA mode 1 is able to characterise the whole ENSO phenomenon in a single mode whereas it took CPCA to do the same with at least 2 modes. The variances explained by NLCPCA (17.4%) is certainly higher than CPCA mode 1 (15.3%). The data set with the seasonal variability is found to show a nonlinear mode that explains the full seasonal cycle. The CPCA, in contrast, has the winter/summer stationary anomaly pattterns manifested in the first mode whereas the second mode is the signal for the stationary spring/fall anomaly patterns. The interdecadal background wind velocity anomaly patterns are also found to be more pronounced in the NLCPCA output. The NLCPCA mode 1 analysis of pre- and post-interdecadal regime shift shows that the El Niño structure and amplitude have changes while there has been no significant differences in the La Niña. |
Extent | 3494244 bytes |
Genre |
Thesis/Dissertation |
Type |
Text |
FileFormat | application/pdf |
Language | eng |
Date Available | 2009-11-20 |
Provider | Vancouver : University of British Columbia Library |
Rights | For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use. |
IsShownAt | 10.14288/1.0052553 |
URI | http://hdl.handle.net/2429/15354 |
Degree |
Master of Science - MSc |
Program |
Atmospheric Science |
Affiliation |
Science, Faculty of Earth, Ocean and Atmospheric Sciences, Department of |
Degree Grantor | University of British Columbia |
GraduationDate | 2004-05 |
Campus |
UBCV |
Scholarly Level | Graduate |
AggregatedSourceRepository | DSpace |
Download
- Media
- 831-ubc_2004-0288.pdf [ 3.33MB ]
- Metadata
- JSON: 831-1.0052553.json
- JSON-LD: 831-1.0052553-ld.json
- RDF/XML (Pretty): 831-1.0052553-rdf.xml
- RDF/JSON: 831-1.0052553-rdf.json
- Turtle: 831-1.0052553-turtle.txt
- N-Triples: 831-1.0052553-rdf-ntriples.txt
- Original Record: 831-1.0052553-source.json
- Full Text
- 831-1.0052553-fulltext.txt
- Citation
- 831-1.0052553.ris
Full Text
Cite
Citation Scheme:
Usage Statistics
Share
Embed
Customize your widget with the following options, then copy and paste the code below into the HTML
of your page to embed this item in your website.
<div id="ubcOpenCollectionsWidgetDisplay">
<script id="ubcOpenCollectionsWidget"
src="{[{embed.src}]}"
data-item="{[{embed.item}]}"
data-collection="{[{embed.collection}]}"
data-metadata="{[{embed.showMetadata}]}"
data-width="{[{embed.width}]}"
async >
</script>
</div>
Our image viewer uses the IIIF 2.0 standard.
To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0052553/manifest