UBC Faculty Research and Publications

Nonlinear complex principal component analysis of the tropical Pacific interannual wind variability Rattan, Sanjay S. P.; Hsieh, William W. 2004

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata


Hsieh_AGU_2004_2004GL020446.pdf [ 265.33kB ]
JSON: 1.0041787.json
JSON-LD: 1.0041787+ld.json
RDF/XML (Pretty): 1.0041787.xml
RDF/JSON: 1.0041787+rdf.json
Turtle: 1.0041787+rdf-turtle.txt
N-Triples: 1.0041787+rdf-ntriples.txt
Original Record: 1.0041787 +original-record.json
Full Text

Full Text

Nonlinear complex principal component analysis of the tropical Pacificinterannual wind variabilitySanjay S. P. Rattan and William W. HsiehDepartment of Earth and Ocean Sciences, University of British Columbia, Vancouver, British Columbia, CanadaReceived 6 May 2004; accepted 7 October 2004; published 2 November 2004.[1] Complex principal component analysis (CPCA) is alinear multivariate technique commonly applied to complexvariables or 2-dimensional vector fields such as winds orcurrents. A new nonlinear CPCA (NLCPCA) method hasbeen developed via complex-valued neural networks.NLCPCA is applied to the tropical Pacific wind field tostudy the interannual variability. Compared to the CPCAmode 1, the NLCPCA mode 1 is found to explain morevariance and reveal the asymmetry in the wind anomaliesbetween El Nin˜o and La Nin˜a states. INDEX TERMS: 4215Oceanography: General: Climate and interannual variability(3309); 3339 Meteorology and Atmospheric Dynamics: Ocean/atmosphere interactions (0312, 4504); 3309 Meteorology andAtmospheric Dynamics: Climatology (1620); 4522 Oceanography:Physical: El Nino; 4504 Oceanography: Physical: Air/seainteractions (0312). Citation: Rattan, S. S. P., and W. W.Hsieh (2004), Nonlinear complex principal component analysis ofthe tropical Pacific interannual wind variability, Geophys. Res.Lett., 31, L21201, doi:10.1029/2004GL020446.1. Introduction[2] Principal component analysis (PCA) also known asempirical orthogonal function (EOF) analysis [von Storchand Zwiers, 1999; Jolliffe, 2002] is a multivariate statisticalmethod widely used to compress datasets and to extractfeatures. Complex PCA (CPCA) is PCA generalized tocomplex variables. Ithas been used toanalyze 2-dimensionalvector fields such as winds [Legler, 1983] and currents[Stacey et al., 1986], where the 2-D vectors are expressedas complex variables. CPCA has also been used to analyzereal data complexified first by the Hilbert transform[Horel, 1984].[3] Linear methods such as PCA has a tendency toscatter the energy of a single oscillatory phenomenon intonumerous unphysical modes [Hsieh, 2004]. Nonlinear PCA(NLPCA) via a neural network (NN) approach [Kramer,1991] has been applied to meteorological/oceanographicdatasets, where it has largely alleviated the scatteringproblem associated with PCA and has revealed the under-lying nonlinear structure of the data (see the review byHsieh [2004]).[4] For nonlinear feature extraction in the complexdomain, the nonlinear CPCA (NLCPCA) method hasrecently been proposed using a complex-valued NN andapplied to the tropical Pacific sea surface temperatures[Rattan and Hsieh, 2004]. This research letter will be thefirst application of the NLCPCA to a 2-D vector field, themonthly tropical Pacific wind data.2. Method and Data2.1. Method[5] Let Z = X + iY be a complex matrix with dimensionm C2 n.Wetaken to be the number of time points and m thenumber of spatial points, with zero mean in time. A CPCAof Z seeks a solution that contains r (r C20 m, n) linearlyindependent complex unitary vectors or eigenvectors in thecolumns of Q (m C2 r) such that [Strang, 1988]:Z ¼ QA; ð1Þwhere the rows of A (r C2 n) contain the r complex principalcomponent (CPC) time series. The first l CPC can serve asinput to the NN for NLCPCA.[6] The Kramer [1991] auto-associative NN for NLPCAcan be adapted to the complex domain (Figure 1) to non-linearly generalize CPCA. After the layer of input neuronscame 3 ‘‘hidden’’ layers of neurons, with the first layercalled the encoding layer, followed by the bottleneck layer(with a single complex neuron), then by the decodinglayer. A nonlinear transfer function f1maps from a,theinput column vector of length l, to the first hidden layer,h(a), a column vector of length q with elementshaðÞk¼ f1WaðÞaþ baðÞC16C17khi; ð2Þwhere W(a)is a q C2 l weight matrix, b(a)is a column vectorof length q containing the bias parameters, and k =1,..., q.The neurons at the bottleneck, the decoding layer and theoutput layer are given respectively byu ¼ f2waðÞC1haðÞþC22baðÞC16C17; ð3ÞhuðÞk¼ f3wuðÞu þ buðÞC16C17khi; ð4Þa0j¼ f4WuðÞhuðÞþC22buðÞC16C17jC20C21; j ¼ 1; ...; l; ð5Þ(see Rattan and Hsieh [2004] for details of the NLCPCAmethod). It is well known that a feed-forward NN onlyneeds one hidden layer of neurons for it to model anynonlinear continuous function [Bishop, 1995]. For theforward mapping u = f(a), where u is the nonlinear CPC(NLCPC), this hidden layer is provided by the encodinglayer, while for the inverse mapping a0= g(u), with a0theNLCPCA model output, it is provided by the decodinglayer. For the typical 1-hidden layer feed-forward NN, thetransfer function from the input to the hidden layer isGEOPHYSICAL RESEARCH LETTERS, VOL. 31, L21201, doi:10.1029/2004GL020446, 2004Copyright 2004 by the American Geophysical Union.0094-8276/04/2004GL020446$05.00L21201 1of4nonlinear, while the transfer function from the hidden layerto the output is usually linear [Bishop, 1995]. Hence thetransfer functions f1, f2, f3, f4are respectively nonlinear,linear, nonlinear and linear, where the linear function issimply the identity function.[7] The nonlinear complex transfer function that is usedis the hyperbolic tangent (tanh(z)), with certain constraintson z. In the complex plane tanh(z) has singularities at(12+ p)pi, p 2 N and these have to be removed to achieveconvergence [Kim and Adali, 2002]. If the magnitude of z isconstrained within a circle of radiusp2then the singularitiesdo not pose any problem and the transfer function isbounded. This requires a restriction on the magnitudes ofthe input data and the (weight and bias) parameters: Eachelement of the rth row of Z was divided by the maximummagnitude of an element in that row, so each element ofZ has magnitude C201. The parameters were randomlyinitialized with magnitude C200.1, and a weight penalty termwas added to the objective function J, i.e.,J ¼1nXnj¼1k ajC0 a0jk2þ pXqk¼1k w1ðÞkk2þkw2ðÞk2þXqk¼1k w3ðÞkk2 !; ð6Þwhere the first term on the right hand side is the meansquare error between a0and a, and the second term is theweight penalty term, with wk(1), w(2)and wk(3)denotingrespectively the vectors containing all the weight and biasparameters from the hidden layers 1, 2 and 3, and the weightpenalty parameter p having typical values from 0.01 to 0.1.During the optimization of J, the real and the imaginarycomponents of the weight and bias parameters wereseparated and kept in a single real vector while optimizationwas done by the MATLAB function ‘‘fminunc’’. Afteroptimization, the predicted CPC a0from the model outputcan be multiplied by the spatial eigenvectors from Q to givethe predicted values.2.2. Data[8] The monthly ship and buoy wind data fromthe Florida State University (FSU) pseudo-stress analysis[Stricherz et al., 1997] were used. Consider a wind field Z =X + iY where X and Yare m C2 n matrices of the zonal andmeridional components of the wind respectively. Thesecomponents are calculated from the zonal and meridionalwind stress data (Lxand Ly): X = Lx/(Lx2+ Ly2)1/4, Y = Ly/(Lx2+ Ly2)1/4[Wang and Weisberg, 2000]. The data period isJanuary 1961 through December 1999, covering the wholetropical Pacific from 124C176Eto70C176W, 29C176Sto29C176N with agrid of 2C176 by 2C176. After the climatological monthly mean wasremoved, the data were smoothed by a 3-month runningmean.3. Results[9] Prior to NLCPCA, traditional CPCA (i.e., complexEOF analysis) was first performed to reduce the dimensionsof the data. The first two CPCs accounted for 15.3% and10.7% of the total variance. The first and the second CPCswere also rotated in the complex plane by 13C176 and 64C176respectively so that the mean value of the argument of therotated CPCs were nearly 0C176, i.e., the variance is mainlyalong the real axis [Hardy and Walton, 1978]. The spatialanomalies associated with the first 2 CPCA modes areshown in Figure 2, with Figure 2a showing the windanomalies during maximum El Nin˜o.[10] The six leading CPCs (with 46% of the total vari-ance) were used as the inputs to the NN model (Figure 1).These input variables were first normalized by removingtheir mean and the real components were divided by theFigure 1. The complex-valued NN model for nonlinearcomplex PCA (NLCPCA) is an auto-associative feed-forward multi-layer perceptron model. There are l input andoutput neurons or nodes corresponding to the l CPCs or thenumber of rows of A used as input. Sandwiched betweenthe input and output layers are 3 hidden layers (starting withthe encoding layer, then the bottleneck layer and finally thedecoding layer) containing q, 1 and q neurons respectively.The network is composed of two parts: The first part fromthe input to the bottleneck maps the input a to the singlenonlinear complex principal component (NLCPC) u by thefunctions f1and f2. The second part from the bottleneck tothe output a0is the inverse mapping by the functions f3andf4. For auto-associative networks, the target for the outputneurons are simply the input data. Increasing the number ofneurons in the encoding and decoding layers increases thenonlinear modelling capability of the network.Figure 2. The spatial patterns of the CPCA (a) mode 1 and(b) mode 2 (plotted when the real component of thecorresponding CPC is maximum).L21201 RATTAN AND HSIEH: ANALYSIS OF INTERANNUAL WIND VARIABILITY L212012of4largest standard deviation among the 6 real CPCs whilethe imaginary components were divided by the largeststandard deviation among the 6 imaginary CPCs. Divisionby the individual CPC’s standard deviation was not done inorder to avoid exaggerating the importance of the highermodes.[11] Thenumberqofhiddenneuronsusedintheencoding/decoding layer of the NN model was varied between 2 and10. While a relatively large q tends to give smaller meansquare error during the NN training, it also tends to giveoverfitted solutions due to the relatively large numberof network parameters. Based on a general principle ofparsimony, q = 6 was chosen in this study. Values of thepenaltyparameterpusedrangedfrom0.01to0.1.Foreachp,25 randomly initialized runs were made. Also, 20% of thedatawasrandomlyselectedastestdataandwithheldfromthetraining of the NN model. Runs where the mean square errorwas larger for the test data set than for the training data setwere rejected to avoid overfitted solutions. Among theremaining NN runs, the one with the smallest mean squareerror was selected as the solution.[12] The first NLCPC shown in Figure 3 had been rotatedby C090C176 in the complex plane (while the weights in thethird hidden layer had also been rotated by 90C176). TheNLCPCA mode 1 explained 17.4% of the total variancecompared to 15.3% explained by the CPCA mode 1. As theNLPC varies, the NLCPCA mode 1 yields nonstationaryspatial anomaly patterns, in contrast to the CPCA mode 1which yields a standing oscillation pattern with the ampli-tude varying according to the CPC.[13] Four spatial patterns of NLCPCA mode 1 corres-ponding to points near the minimum Re(u), half minimumRe(u), half maximum Re(u), and maximum Re(u) are shownin Figure 4. In Figure 4a (strongest La Nin˜a conditions) theequatorial Pacific displays anomalous easterly winds, withthe strongest winds in the equatorial western Pacific. As thenegative real component of NLCPC 1 decreases to about halfits minimum, the easterly wind anomalies weaken over theequatorial Pacific as shown in Figure 4b to about half themaximum La Nin˜a wind magnitude.[14] Under El Nin˜o conditions, the tropical Pacific windfield has reversed in direction (Figures 4c and 4d). InFigure 4d, during maximum El Nin˜o, an easterly windanomaly is observed in the far western equatorial Pacifictogether with strong westerly anomalies in the centralequatorial Pacific. In contrast to the two La Nin˜a pictures(Figures 4a and 4b) which look quite similar except forthe magnitude, the weak El Nin˜o state (Figure 4a) isquite different when compared to the strong El Nin˜o state(Figure 4d), e.g., Figure 4c shows westerly anomalieslocated further west with much less than half the magnitudeof Figure 4d, as well as missing the easterly anomalies atthe far western equatorial Pacific and the off-equatorialanomalies.Figure 3. The first NLCPC u shown in the complex planeas dots, with crosses indicating the (a) minimum Re(u)(strongest LaNin˜a),(b)halfminimumRe(u)(weakLaNin˜a),(c) half maximum Re(u) (weak El Nin˜o) and (d) maximumRe(u) (strongest El Nin˜o). The four corresponding spatialanomaly patterns are shown in Figure 4.Figure 4. The spatial patterns of the NLCPCA mode 1 showing spatial patterns near the (a) minimum Re(u) (strongest LaNin˜a), (b) half minimum Re(u) (weak La Nin˜a), (c) half maximum Re(u) (weak El Nin˜o) and (d) maximum Re(u) (strongestEl Nin˜o). Different scalings are used, as indicated at the top right corner of each panel.L21201 RATTAN AND HSIEH: ANALYSIS OF INTERANNUAL WIND VARIABILITY L212013of4[15] The asymmetry between strong El Nin˜o and strongLa Nin˜a is evident from Figure 4a (with anomaly centernear 0C176N175C176E) and Figure 4d (with center near 5C176S160C176W). In contrast, the CPCA mode 1 yields anti-symmetrical stationary patterns for El Nin˜o and La Nin˜a.During maximum real CPC 1 (Figure 2a) the patterns forstrong El Nin˜o are captured whereas the minimum realCPC 1 represents the maximum La Nin˜a features. The LaNin˜a spatial patterns when plotted involves a 180C176 rotationof the El Nin˜o wind directions and look similar to Figure 4a.HencetheCPCAcentresforbothstrongElNin˜oandLaNin˜aare near 0C176N175C176E,i.e., the CPCA mode 1 completely failedto characterize the asymmetry between El Nin˜o and La Nin˜awhich results in the asymmetric El Nin˜o-Southern Oscilla-tion (ENSO) features being scattered into CPCA mode 2(Figure 2b) and higher modes. Compared to NLCPCAmode 1, CPCA mode 1 also substantially underestimatedthe magnitude of the maximum El Nin˜o (Figure 2a), aswell as missing the easterly anomalies in the far westernequatorial Pacific and the off-equatorial anomalies foundin Figure 4d. Figure 5 shows the difference between theNLCPCA mode 1 and CPCA mode 1 during the strongestLa Nin˜a and the strongest El Nin˜o, revealing the differenceduring the latter to be much greater than during the former,i.e., the CPCA mode 1 does not accurately describe the windanomalies during strong El Nin˜o conditions.[16] To test whether the El Nin˜o and La Nin˜a asymmetryhave been biased by outliers, we removed the two strongestEl Nin˜o episodes and the two strongest La Nin˜a episodes(i.e., a total of 4 C2 12 monthly values) from the input databefore NLCPCA was again performed. The resultant spatialpatterns again exhibited the asymmetry between El Nin˜oand La Nin˜a.[17] The NLCPCA mode 2 was extracted from theresidual. Again with the NLCPC and CPC rotated so theirvariance is mainly along the real axis, we found that thecorrelation between the Southern Oscillation Index (SOI)and Re(NLCPC) is 0.80 for mode 1 and 0.22 for mode 2. Incontrast, the correlation between SOI and Re(CPC) is 0.77for mode 1 and 0.52 for mode 2. In other words, for CPCA,the second mode also contains significant ENSO signal, asthe nonlinear ENSO mode cannot be described by a linearmode and is scattered into higher modes, but the NLCPCAmode 1 has been much more effective in extracting theENSO signal, so the second nonlinear mode is less corre-lated with the SOI than the second linear mode is.4. Conclusions[18] Linear statistical methods such as PCA are often toosimplistic to describe real-world systems, with a tendency toscatter a single oscillatory phenomenon into numerousunphysical modes [Hsieh, 2004]. Two-dimensional vectorfields like the horizontal wind and ocean currents havecommonly used the linear CPCA method for featureextraction. By using a neural network approach, the newNLCPCA method allows a nonlinear generalization of theCPCA. Applied to the tropical Pacific horizontal windanomaly data, the NLCPCA mode l explained 17.4% ofthe total variance (versus 15.3% for the CPCA mode 1), andgave an accurate description of the ENSO oscillation fromstrong La Nin˜a to strong El Nin˜o, revealing the considerableasymmetry in the oscillation. The NLCPCA code (written inMATLAB) is downloadable from http://www.ocgy.ubc.ca/projects/clim.pred/download.html.[19] Acknowledgments. Dr.AimingWukindlyassistedontheGRADS plotting software. This work was supported by the NaturalSciences and Engineering Research Council of Canada and the CanadianFoundation for Climate and Atmospheric Sciences.ReferencesBishop, C. M. (1995), Neural Networks for Pattern Recognition, OxfordUniv. Press, New York.Hardy, D. M., and J. J. Walton (1978), Principal component analysis ofvector wind observations, J. Appl. Meteorol., 17, 1153–1162.Horel, J. D. (1984), Complex principal component analysis: Theory andexamples, J. Clim. Appl. Meteorol., 23, 1660–1673.Hsieh, W. W. (2004), Nonlinear multivariate and time series analysisby neural network methods, Rev. Geophys., 42, RG1003, doi:10.1029/2002RG000112.Jolliffe, I. T. (2002), Principal Component Analysis, Springer-Verlag, NewYork.Kim, T., and T. Adali (2002), Fully complex multi-layer perceptron networkfor nonlinear signal processing, J. VLSI Signal Process., 32, 29–43.Kramer, M. A. (1991), Nonlinear principal component analysis usingautoassociative neural networks, AIChE J., 37, 233–243.Legler, D. M. (1983), Empirical orthogonal function analysis of windvectors over the tropical Pacific region, Bull. Am. Meteorol. Soc.,64(3), 234–241.Rattan, S. S. P., and W. W. Hsieh (2004), Complex-valued neural networksfor nonlinear complex principal component analysis, Neural Networks,inpress.Stacey, M. W., S. Pond, and P. H. LeBlond (1986), A wind-forced Ekmanspiral as a good statistical fit to low-frequency currents in a coastal strait,Science, 233, 470–472.Strang, G. (1988), Linear Algebra and its Applications, Jovanovich, SanDiego, Calif.Stricherz, J., D. M. Legler, and J. J. O’Brien (1997), TOGA pseudostressatlas 1985–1994, vol. 2, Pacific Ocean, Florida State Univ., Tallahassee.von Storch, H., and F. W. Zwiers (1999), Statistical Analysis in ClimateResearch, Cambridge Univ. Press, New York.Wang, C., and R. H. Weisberg (2000), The 1997–98 El Nin˜o and evolutionrelative to previous El Nin˜o events, J. Clim., 13, 488–501.C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0W. W. Hsieh and S. S. P. Rattan, Department of Earth and OceanSciences, University of British Columbia, 6339 Stores Road, Vancouver,British Columbia, Canada V6T 1Z4. (whsieh@eos.ubc.ca)Figure 5. The NLCPCA mode 1 spatial pattern minusthe CPCA mode 1 pattern during the (a) strongest La Nin˜a,and (b) strongest El Nin˜o. Different scalings are used inFigures 5a and 5b.L21201 RATTAN AND HSIEH: ANALYSIS OF INTERANNUAL WIND VARIABILITY L212014of4


Citation Scheme:


Usage Statistics

Country Views Downloads
United States 7 10
France 5 0
China 3 22
Germany 2 0
Finland 2 0
City Views Downloads
Unknown 5 1
Ashburn 5 0
Roubaix 2 0
Shenzhen 2 22
Espoo 2 0
Mountain View 1 0
Sunnyvale 1 0
Beijing 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}
Download Stats



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items