UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A Monte Carlo comparison of regression estimators in the presence of autocorrelation and collinearity Gosling, Barbara J. 1983

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
831-UBC_1983_A4_6 G67.pdf [ 3.07MB ]
Metadata
JSON: 831-1.0095269.json
JSON-LD: 831-1.0095269-ld.json
RDF/XML (Pretty): 831-1.0095269-rdf.xml
RDF/JSON: 831-1.0095269-rdf.json
Turtle: 831-1.0095269-turtle.txt
N-Triples: 831-1.0095269-rdf-ntriples.txt
Original Record: 831-1.0095269-source.json
Full Text
831-1.0095269-fulltext.txt
Citation
831-1.0095269.ris

Full Text

A MONTE CARLO COMPARISON OF REGRESSION ESTIMATORS IN THE PRESENCE OF AUTOCORRELATION AND COLLINEARITY by BARBARA J . GOSLING B.Comm., The University Of Alberta, 1979 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE(BUS.ADMIN.) in THE FACULTY OF GRADUATE STUDIES Department Of Commerce We accept t h i s thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA June 1983 © Barbara J. Gosling, 1983 In p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t of the requirements f o r an advanced degree at the U n i v e r s i t y of B r i t i s h Columbia, I agree that the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r r e f e r e n c e and study. I f u r t h e r agree that p e r m i s s i o n f o r e x t e n s i v e copying of t h i s t h e s i s f o r s c h o l a r l y purposes may be granted by the Head of my Department or by h i s or her r e p r e s e n t a t i v e s . I t i s understood that copying or p u b l i c a t i o n of t h i s t h e s i s f o r f i n a n c i a l gain s h a l l not be allowed without my w r i t t e n p e r m i s s i o n . Department of Commerce The U n i v e r s i t y of B r i t i s h Columbia 2075 Wesbrook Place Vancouver, Canada V6T 1W5 Date: J u l y 14, 1983 Abstract In time series regression modelling, f i r s t - o r d e r autocorrelated errors are often a problem. When the data also suffers from c o l l i n e a r independent variables, generalized least squares estimation i s no longer the best alternative to ordinary least squares. The Monte Carlo simulation i l l u s t r a t e s that ridge estimation using data transformed according to the generalized least squares method provides estimates of the regression c o e f f i c i e n t s which are superior to generalized least squares, ridge and ordinary least squares estimates. The analysis of a set of ' t y p i c a l ' econometric data further supports the application of th i s method referred to as generalized ridge when both autocorrelation and c o l l i n e a r i t y are present. i i i T a b l e o f C o n t e n t s A b s t r a c t i i L i s t o f T a b l e s i v L i s t o f F i g u r e s y A c k n o w l e d g e m e n t s v i I . I N T R O D U C T I O N 1 I I . P R E L I M I N A R I E S 4 I I I . A U T O C O R R E L A T I O N A N D G E N E R A L I Z E D L E A S T S Q U A R E S 6 I V . C O L L I N E A R I T Y A N D R I D G E R E G R E S S I O N 8 V . G E N E R A L I Z E D R I D G E R E G R E S S I O N 11 5 . 1 T h e E x p e c t e d M e a n S q u a r e d E r r o r 1 2 5 . 2 T h e E s t i m a t i o n O f T h e A u t o c o r r e l a t i o n P a r a m e t e r . . . 1 4 5 . 3 S t a n d a r d i z a t i o n 18 5 . 4 T h e E s t i m a t i o n O f T h e R i d g e P a r a m e t e r 2 0 V I . T H E M O N T E C A R L O S I M U L A T I O N 2 3 6 . 1 D a t a G e n e r a t i o n 2 3 6 . 2 P a r a m e t e r E s t i m a t i o n 2 7 6 . 3 D i a g n o s t i c s 3 0 6 . 4 R e s u l t s 3 2 6 . 4 . 1 S e t A 3 4 6 . 4 . 2 S e t B 3 8 6 . 4 . 3 S e t C 41 6 . 4 . 4 S e t D 4 4 6 . 4 . 5 G H W ' 4 6 6 . 4 . 6 K V a l u e s 4 8 6 . 4 . 7 D i s c u s s i o n 4 9 V I I . A N A P P L I C A T I O N O F G E N E R A L I Z E D R I D G E R E G R E S S I O N 5 2 V I I I . C O N C L U S I O N 6 3 A P P E N D I X A - T H E L O N G L E Y D A T A 6 5 A P P E N D I X B - R E S I D U A L V S . T I M E P L O T S : L O N G L E Y D A T A 6 6 B I B L I O G R A P H Y 6 7 iv L i s t of Tables I. E i g e n v a l u e s and E i g e n v e c t o r s of Simulated Data 24 I I . C o r r e l a t i o n M a t r i x : Longley Data 53 I I I . V a r i a n c e Decomposition: Longley Data 54 IV. Regression R e s u l t s without GLS: Longley Data 56 V. Gen. Ridge R e s u l t s u s i n g MLE p: Longley Data 57 V I . Gen. Ridge R e s u l t s u s i n g Durbin p: Longley Data ...61 V L i s t of F i g u r e s 1 . Set A: /3max 34 2. Set A: /3min 36 3. Set A: 02 37 4. Set B: 0max 38 5. Set B: /3min . 40 6. Set C: /3max 41 7. Set C: 0min 42 8. Set C: 02 43 9. Set D: 0max 44 10. Set D: 0min 45 11. Set A i n c l u d i n g GHW: 0max 46 12. Set D i n c l u d i n g GHW: 0max 47 13. Set D i n c l u d i n g GHW: 0min 47 14. O r d i n a r y Ridge T r a c e : L o n g l e y Data 58 15. Gen. Ridge Trace u s i n g MLE p: L o n g l e y Data ....59 16. Gen. Ridge T r a c e using' D u r b i n p: L o n g l e y Data 61 vi Acknowledgement I would l i k e to thank Dr. Martin Puterman for his support throughout the past two years. Marty's suggestions, c r i t i c i s m s , and concern have been dauntless. I would also l i k e to thank Dr. Piet De Jong for his assistance in the f i n a l revisions of th i s paper and Dr. Nancy Reid for w i l l i n g l y completing my committee. Very special 'thank you's must go to Derek, Nadine, Helene, the PhD.'s, P a t t i , Sharon, Alison, and la s t but not least, my Grandmother, for 'being there'. 1 I . I N T R O D U C T I O N T h e o b j e c t i v e o f t h i s t h e s i s i s t o p r e s e n t r e s u l t s o f a M o n t e C a r l o s i m u l a t i o n c o m p a r i n g s e v e r a l v a r i a n t s o f r i d g e e s t i m a t o r s w h e n b o t h c o l l i n e a r i t y a n d f i r s t - o r d e r a u t o c o r r e l a t i o n a r e p r e s e n t i n a r e g r e s s i o n m o d e l . C o l l i n e a r i t y o c c u r s w h e n t h e i n d e p e n d e n t v a r i a b l e s i n a m u l t i p l e r e g r e s s i o n m o d e l a r e s t r o n g l y i n t e r r e l a t e d . T h i s m a k e s i n t e r p r e t a t i o n o f t h e o r d i n a r y l e a s t s q u a r e s ( O L S ) e s t i m a t e s o f t h e r e g r e s s i o n c o e f f i c i e n t s d i f f i c u l t . T h e y o f t e n h a v e u n e x p e c t e d s i g n s a n d o v e r - e s t i m a t e d v a r i a n c e s w h i c h c a n l e a d t o t h e i n c o r r e c t r e m o v a l o f v a r i a b l e s f r o m t h e m o d e l . R i d g e r e g r e s s i o n ( O R 1 ) w a s p r o p o s e d b y H o e r l a n d K e n n a r d ( 1 9 7 0 ) a s a b i a s e d a l t e r n a t i v e t o O L S w h e n s e v e r e c o l l i n e a r i t y i s p r e s e n t . F i r s t - o r d e r a u t o c o r r e l a t i o n r e f e r s t o a o n e p e r i o d l a g r e l a t i o n s h i p b e t w e e n t h e c o n s e c u t i v e e r r o r s i n a t i m e s e r i e s r e g r e s s i o n m o d e l . T h i s a v i o l a t i o n o f t h e a s s u m p t i o n t h a t t h e e r r o r s a r e u n c o r r e l a t e d a n d i n t h i s s i t u a t i o n , t h e b e s t l i n e a r u n b i a s e d e s t i m a t o r ( B L U E ) o f t h e r e g r e s s i o n e q u a t i o n i s t h e g e n e r a l i z e d l e a s t s q u a r e s ( G L S ) e s t i m a t o r . T h e r e h a s b e e n v e r y l i t t l e w o r k d o n e o n t h e c o - e x i s t e n c e o f t h e s e p r o b l e m s . P e r h a p s t h e l a c k o f i n t e r e s t c a n b e a t t r i b u t e d t o t h e p u r p o s e o f t h e m o d e l s w h e r e a u t o c o r r e l a t i o n i s l i k e l y t o b e f o u n d . A u t o c o r r e l a t e d e r r o r s o c c u r m o s t f r e q u e n t l y i n e c o n o m e t r i c s a n d o t h e r t i m e s e r i e s m o d e l l i n g b e c a u s e o f t h e O r d i n a r y R i d g e - d e f i n e d t o d i f f e r e n t i a t e t h e u s u a l r i d g e r e g r e s s i o n e s t i m a t i o n f r o m g e n e r a l i z e d r i d g e r e g r e s s i o n . 2 t r e n d p r e s e n t i n t h e c u m u l a t i v e e f f e c t s o f t h e o m i t t e d v a r i a b l e s i n t h e r e g r e s s i o n s e t t i n g . T h e s e r e g r e s s i o n e q u a t i o n s a r e o f t e n u s e d f o r p r e d i c t i o n a n d f o r e c a s t i n g . A s l o n g a s a n y c o l l i n e a r r e l a t i o n s h i p s a m o n g t h e i n d e p e n d e n t v a r i a b l e s a r e m a i n t a i n e d i n t h e f o r e c a s t a n d t h e p r e d i c t i o n e r r o r i s r e a s o n a b l e , t h e i n s t a b i l i t y a n d p o s s i b l y u n e x p e c t e d v a l u e s o f t h e c o e f f i c i e n t s d u e t o c o l l i n e a r i t y a r e o f t e n i g n o r e d . G e n e r a l i z e d r i d g e r e g r e s s i o n a s d e f i n e d b y H S U ( 1 9 8 0 ) p r o v i d e s a n a l t e r n a t i v e w h e n t h e v a l u e s o f t h e c o e f f i c i e n t s a n d s t a b i l i t y o f t h e m o d e l a r e c o n s i d e r e d i m p o r t a n t . I n i m p l e m e n t i n g g e n e r a l i z e d l e a s t s q u a r e s , t h e d a t a i s t r a n s f o r m e d s o t h a t o r d i n a r y l e a s t s q u a r e s c a n b e p e r f o r m e d . T h e r e f o r e , w h e n b o t h c o l l i n e a r i t y a n d a u t o c o r r e l a t i o n o c c u r , i t i s p o s s i b l e t o t r a n s f o r m t h e d a t a f o l l o w i n g G L S a n d t h e n t o a p p l y r i d g e r e g r e s s i o n . T h i s m e t h o d i s r e f e r r e d t o a s g e n e r a l i z e d r i d g e r e g r e s s i o n ( G R ) . H o e r l a n d K e n n a r d ( 1 9 7 0 ) s h o w e d t h a t a r i d g e e s t i m a t e c a n b e f o u n d w h i c h h a s a s m a l l e r e x p e c t e d m e a n s q u a r e d e r r o r ( E M S E ) o f t h e e s t i m a t e t h a n O L S . G o s l i n g , H s u a n d P u t e r m a n ( 1 9 8 2 ) d e r i v e d a n e x p r e s s i o n f o r t h e E M S E o f t h e u s u a l r i d g e e s t i m a t o r w i t h a u t o c o r r e l a t e d e r r o r s . T h e y s h o w e d t h a t a g e n e r a l i z e d r i d g e e s t i m a t e c a n b e f o u n d w h i c h h a s a s m a l l e r E M S E t h a n G L S . T h e c o m p a r i s o n s o f t h e E M S E { O L S } t o t h e E M S E { G L S } a n d t h e E M S E { O R } t o t h e E M S E { G R } -\ d o n o t s i m p l i f y t o a l l o w f o r i n t e r p r e t a t i o n . T h i s l e d t o t h e n e c e s s i t y o f t h e M o n t e C a r l o s i m u l a t i o n p r e s e n t e d i n t h i s t h e s i s . T h e s i m u l a t i o n n o t o n l y e v a l u a t e d g e n e r a l i z e d r i d g e . 3 C o n s i d e r a b l e e f f o r t w e n t i n t o t h e c h o i c e o f t h e r i d g e p a r a m e t e r b e c a u s e a s a t i s f a c t o r y m e t h o d f o r f i n d i n g t h e b e s t v a l u e h a s y e t t o b e d e f i n e d . T h r e e m e t h o d s f o u n d t o p e r f o r m w e l l b y G i b b o n s ( 1 9 8 1 ) w e r e u s e d a n d e v a l u a t e d i n t h i s s i m u l a t i o n . A s e t o f d a t a w a s o b t a i n e d t o i l l u s t r a t e t h e g e n e r a l i z e d r i d g e m e t h o d . T h e d a t a w a s o r i g i n a l l y u s e d b y L o n g l e y ( 1 9 6 7 ) i n a d i s c u s s i o n o f t h e c o m p u t a t i o n a l p r o b l e m s t h a t c a n o c c u r w h e n e x t r e m e c o l l i n e a r i t y i s p r e s e n t . A m o d e l u s i n g a s u b s e t o f t h e d a t a w a s c o n s t r u c t e d w h e r e b o t h c o l l i n e a r i t y a n d a u t o c o r r e l a t i o n a r e s i g n i f i c a n t . 4 I I . P R E L I M I N A R I E S T h e l i n e a r r e g r e s s i o n m o d e l i s d e f i n e d a s ¥=X0 + c w n e r e ¥ i s a n ( n x 1 ) v e c t o r o f r a n d o m o b s e r v a t i o n s o f t h e d e p e n d e n t v a r i a b l e ; X i s a n ( n x ( p + l ) ) m a t r i x o f k n o w n o b s e r v a t i o n s o f t h e i n d e p e n d e n t , e x p l a n a t o r y v a r i a b l e s x 0 , x , , . . . , x p ( x 0 = 1 ) . 0 i s a ( ( p + 1 ) x 1 ) v e c t o r o f u n k n o w n r e g r e s s i o n c o e f f i c i e n t s . c i s a n ( n x 1 ) v e c t o r o f r a n d o m e r r o r s s a t i s f y i n g E t e ) = 0 a n d E ( e e ' ) i s o 2 I . I n t h i s s i t u a t i o n , i t i s k n o w n t h a t | 3 { 0 L S } = ( X ' X ) - 1 X ' Y i s t h e b e s t l i n e a r u n b i a s e d e s t i m a t o r ( B L U E ) o f / 3 . 2 T h e m o d e l i s o f t e n s t a n d a r d i z e d s o t h a t a l l t h e v a r i a b l e s h a v e z e r o m e a n a n d u n i t v a r i a n c e . T h e t r a n s f o r m a t i o n t o s t a n d a r d i z e i s : x = ( x - x ) / J\\ ( x - x ) 2 j i j i j k=1 j k j Y = ( Y - y) / / z (y - Y ) 2 (2.D i i k = 1 k T h e ' p + 1 ' v a r i a b l e s a r e r e d u c e d t o ' p ' v a r i a b l e s b e c a u s e t h e c o n s t a n t d i s a p p e a r s . X ' X i s t h e m a t r i x o f c o r r e l a t i o n s b e t w e e n t h e i n d e p e n d e n t v a r i a b l e s a n d X ' Y i s t h e v e c t o r o f c o r r e l a t i o n s o f t h e d e p e n d e n t v a r i a b l e w i t h e a c h o f t h e i n d e p e n d e n t v a r i a b l e s . X a n d Y w i l l b e a s s u m e d t o b e s t a n d a r d i z e d u n l e s s 2 T h r o u g h o u t t h i s p a p e r , ' { } ' r e f e r s t o a s u b s c r i p t . I n e q u a t i o n s , ' ( a ) ( b ) ' m e a n s ' a ' m u l t i p l i e d b y ' b ' i f ' a ' a n d ' b ' a r e s c a l a r . I f t h e y a r e v e c t o r s o r m a t r i c e s i t i n d i c a t e s t h e i n n e r p r o d u c t o f ' a ' a n d ' b 1 . 5 o t h e r w i s e s t a t e d . D e f i n e G as the (p x p) o r t h o g o n a l m a t r i x of e i g e n v e c t o r s such t h a t GX'XG' = A, where A i s a d i a g o n a l m a t r i x w i t h the e i g e n v a l u e s of X'X, X 1 f X 2,...»X p on the d i a g o n a l . A p p l y i n g the o r t h o g o n a l t r a n s f o r m a t i o n , G, t o the s t a n d a r d i z e d d a t a , the above model can be r e w r i t t e n as Y=Wa + e where W=XG' i s the dat a m a t r i x e x p r e s s e d i n r o t a t e d c o o r d i n a t e s . The columns of W are the l i n e a r l y independent t r a n s f o r m e d columns of X and a = G/3 i s the v e c t o r of c o r r e s p o n d i n g r e g r e s s i o n c o e f f i c i e n t s . The OLS e s t i m a t e of a i s g i v e n by a{OLS} = (W'WJ-'W'Y = G/3{0LS} (2.2) where t h e ' i ' t h component i s a{OLS,i} = G{i}|3{OLS}; G { i } i s the ' i ' t h row of G. 6 I I I . AUTOCORRELATION AND GENERALIZED LEAST SQUARES In time series regression models, the errors are often dependent on the errors of previous periods. These errors are autocorrelated. First-order autocorrelation occurs when there is a one period lag relationship between consecutive errors. In p a r t i c u l a r , e{t} = pe{t-l} + u{t} where |p| < 1; E(u)=0; E(uu') = a 2 I . Here E(ee') = a 2V where V = 1 P P 1 P' P 1 >ri- i , n - 2 1 (3.1 ) If ordinary least squares estimation i s applied to a model whose errors are autocorrelated 3 the estimates of /3 w i l l be unbiased but the sampling variances of these estimates are larger than i f generalized least squares estimation i s used(c.f. Johnston(1972)). For OLS to be BLUE, the E(ee') must equal a 2I which i s v i o l a t e d here. As a r e s u l t , the usual least-squares formulae for the sampling variances of the regression c o e f f i c i e n t w i l l be i n v a l i d and w i l l l i k e l y severely underestimate these variances. Another major consequence is that predictions w i l l have unnecessarily large sampling variances. For the remainder of this paper, 'autocorrelation* w i l l refer to f i r s t - o r d e r autocorrelation. Other forms of autocorrelation are not dealt with. 7 G e n e r a l i z e d l e a s t s q u a r e s e s t i m a t i o n i s t h e a l t e r n a t i v e t o O L S w h i c h p r o v i d e s t h e B L U E o f j3. T h e G L S e s t i m a t e o f 0 i s 0 { G L S } = ( X ' V ' X ) - ' X ' V 1 Y ( 3 . 2 ) w h e r e X a n d Y a r e n o t s t a n d a r d i z e d . V " 1 i s g i v e n b y V i _ 1 - p -1 - p 0 p 1 + p 2 - p 0 - p 1 + p 2 1 + p 2 V " 1 c a n b e w r i t t e n a s P ' P w h e r e : P = 0 - P 1 0 - p 0 0 . . 0 0 - p 1 • p 1 ( 3 . 3 ) ( 3 . 4 ) I f t h e d a t a i s t r a n s f o r m e d t o X * = P X a n d Y * = P Y t h e r e g r e s s i o n m o d e l c a n b e w r i t t e n a s Y * = X * / 3 + u ( 3 . 5 ) w h e r e u h a s a l l t h e p r o p e r t i e s o f e r e q u i r e d f o r O L S t o b e B L U E . T h u s , / 3 { G L S } i s e q u i v a l e n t t o t h e O L S e s t i m a t e u s i n g X * a n d Y * . T h e g e n e r a l i z e d l e a s t s q u a r e s e s t i m a t o r i n t r o d u c e s a n e w p a r a m e t e r , p , w h i c h m u s t b e e s t i m a t e d . V a r i o u s p r o c e d u r e s f o r t h e e s t i m a t i o n o f p a r e s u m m a r i z e d i n C h a p t e r 5 . 2 . 8 I V . C O L L I N E A R I T Y A N D R I D G E R E G R E S S I O N I n t h e i n t e r p r e t a t i o n o f a m u l t i p l e r e g r e s s i o n e q u a t i o n , a n i m p l i c i t a s s u m p t i o n i s t h a t t h e i n d e p e n d e n t v a r i a b l e s a r e n o t s t r o n g l y r e l a t e d . O r t h o g o n a l i n d e p e n d e n t v a r i a b l e s a r e i d e a l , b u t e x t r e m e l y r a r e , i n m o s t r e g r e s s i o n a p p l i c a t i o n s . W h e n t h e d a t a i s c l o s e r t o b e i n g l i n e a r l y r e l a t e d t h a n o r t h o g o n a l , t h e i n d e p e n d e n t v a r i a b l e s a r e s a i d t o b e c o l l i n e a r . A r e g r e s s i o n c o e f f i c i e n t i s o f t e n i n t e r p r e t e d a s t h e e f f e c t o f a o n e u n i t c h a n g e o f a n i n d e p e n d e n t v a r i a b l e o n t h e v a l u e o f t h e d e p e n d e n t v a r i a b l e w h e n a l l o t h e r ' v a r i a b l e s r e m a i n c o n s t a n t . T h i s m a y n o t b e a v a l i d s t a t e m e n t i f t h e r e i s a s t r o n g l i n e a r r e l a t i o n s h i p a m o n g t h e i n d e p e n d e n t v a r i a b l e s . T h e r e m a y b e n o i n f o r m a t i o n i n t h e d a t a c o n c e r n i n g t h e r e s u l t o f i n c r e a s i n g t h e v a l u e o f o n e v a r i a b l e w h i l e k e e p i n g t h e o t h e r s c o n s t a n t . I n f a c t , t h e n e w c o m b i n a t i o n o f v a l u e s m a y n e v e r o c c u r i n r e a l i t y . C o l l i n e a r i t y s h o u l d n o t b e c o n s i d e r e d a m o d e l l i n g e r r o r ; i t i s a c o n d i t i o n o f d e f i c i e n t d a t a . S e v e r e c o l l i n e a r i t y c a u s e s t h e e s t i m a t e s o f t h e r e g r e s s i o n c o e f f i c i e n t t o b e u n s t a b l e a n d s e n s i t i v e t o s l i g h t c h a n g e s i n t h e d a t a a n d t o t h e a d d i t i o n o r d e l e t i o n o f v a r i a b l e s i n t h e e q u a t i o n . T h e v a r i a n c e s o f t h e c o e f f i c i e n t e s t i m a t e s a r e i n f l a t e d , o f t e n t o t h e p o i n t w h e r e a v a r i a b l e i s i n c o r r e c t l y r e m o v e d f r o m t h e m o d e l o n t h e b a s i s o f a l o w t - s t a t i s t i c . W h e n t h e v a l u e a n d s t a b i l i t y o f t h e e s t i m a t e d c o e f f i c i e n t s a r e o f i n t e r e s t , c o l l i n e a r i t y s h o u l d n o t b e i g n o r e d . O f t e n , t h e p r o b l e m i s ' s o l v e d ' b y t h e r e m o v a l o f o n e o r m o r e o f t h e v a r i a b l e s b e c a u s e t h e y a p p e a r t o r e p r e s e n t t h e s a m e t h i n g . 9 H o e r l a n d K e n n a r d ( 1 9 7 0 ) p r e s e n t e d a n a l t e r n a t i v e t o t h i s a p p r o a c h k n o w n a s r i d g e r e g r e s s i o n . A w i d e l y u s e d c r i t e r i a f o r c h o o s i n g a n e s t i m a t o r o f t h e r e g r e s s i o n c o e f f i c i e n t s i s t h e m i n i m i z a t i o n o f t h e e x p e c t e d m e a n s q u a r e d e r r o r o f t h e e s t i m a t e d e f i n e d a s f o l l o w s : E M S E = E ( £ - 0 ) ' ( £ - 0 ) ( 4 . 1 ) = Z [ v a r ( j ? ) + B i a s 2 ] i = 1 L i i J w h e r e B i a s = E($ ) - 0 i i " ' C l a s s i c a l l y , a n e s t i m a t o r h a s o n l y b e e n c o n s i d e r e d i f i t i s u n b i a s e d " . I n t h i s c a s e , m i n i m i z i n g t h e s u m o f t h e v a r i a n c e s o f t h e c o e f f i c i e n t e s t i m a t e s i s e q u i v a l e n t t o m i n i m i z i n g t h e m e a n s q u a r e d e r r o r . R i d g e r e g r e s s i o n i s a b i a s e d a l t e r n a t i v e t o o r d i n a r y l e a s t s q u a r e s . T h e r i d g e e s t i m a t o r i s : 0 { O R } = ( X ' X + k l ) " 1 X ' Y ( 4 . 2 ) w h e r e X a n d Y a r e s t a n d a r d i z e d a n d ' k ' i s t h e u n k n o w n r i d g e p a r a m e t e r u s u a l l y c h o s e n t o b e p o s i t i v e a n d l e s s t h a n o n e . R i d g e r e g r e s s i o n a i m s a t r e d u c i n g t h e s u m o f t h e v a r i a n c e s o f 0 b y m o r e t h a n t h e a m o u n t o f s q u a r e d b i a s i t i n t r o d u c e s . H o e r l a n d K e n n a r d ( 1 9 7 0 ) s h o w t h a t t h e r e a l w a y s e x i s t s s o m e k > 0 w h e r e E M S E { O R } < E M S E { O L S } . A d i f f i c u l t a n d y e t u n s o l v e d , e m p i r i c a l p r o b l e m h a s b e e n f i n d i n g t h i s v a l u e o f k i n t h e . a p p l i c a t i o n o f * T h e e s t i m a t e , 0, i s u n b i a s e d i f t h e E(/3) = 0 1 0 ridge regression. The Monte Carlo simulation evaluates some of the more popular methods suggested to date. 11 V . G E N E R A L I Z E D R I D G E R E G R E S S I O N G e n e r a l i z e d r i d g e r e g r e s s i o n i s a c o m b i n a t i o n o f g e n e r a l i z e d l e a s t s q u a r e s a n d r i d g e r e g r e s s i o n a n d i s a n a l t e r n a t i v e t o o r d i n a r y l e a s t s q u a r e s w h e n a u t o c o r r e l a t i o n a n d c o l l i n e a r i t y e x i s t . I n t h i s c h a p t e r , a s u m m a r y o f r e s u l t s a b o u t t h e e x p e c t e d m e a n s q u a r e d e r r o r i n t h i s s e t t i n g i s f o l l o w e d b y a s u m m a r y o f t h e t e c h n i q u e s a v a i l a b l e f o r t h e e s t i m a t i o n o f t h e a u t o c o r r e l a t i o n p a r a m e t e r , p . A d i s c u s s i o n o f t h e s c a l i n g a n d c e n t e r i n g o f t h e d a t a i n r i d g e r e g r e s s i o n a n d t h e e s t i m a t i o n o f t h e r i d g e p a r a m e t e r , k , c o m p l e t e a r e v i e w o f t h e a p p l i c a t i o n o f g e n e r a l i z e d r i d g e . 1 2 5 . 1 T h e E x p e c t e d M e a n S q u a r e d E r r o r T h e e x p r e s s i o n f o r t h e e x p e c t e d m e a n s q u a r e d e r r o r w h e n t h e i n d e p e n d e n t v a r i a b l e s a r e c o l l i n e a r a n d r i d g e r e g r e s s i o n i s u s e d w a s d e r i v e d b y H o e r l a n d K e n n a r d ( 1 9 7 0 ) . I n t h e c a s e o f a u t o c o r r e l a t e d e r r o r s , t h e f o l l o w i n g i s t h e e x p r e s s i o n f o r t h e E M S E d e r i v e d b y G o s l i n g , H s u , a n d P u t e r m a n ( 1 9 8 2 ) : E M S E { G R } = 7 i + 7 2 + 7 3 ( 5 . 1 . 1 ) w h e r e : 7 i P r 2 Z X { i } i = 1 ( X { i } + k ) 2 7 2 = p k 2 z a 2 { i } i = 1 ( X { i } + k ) 2 7 3 = 2 a 2 Z j = 1 n - 1 n Z Z w w p 1=1 i = l + 1 i j l j j - l < A { j } + k ) a n d w i s t h e i ' t h o b s e r v a t i o n o f t h e j ' t h i j v a r i a b l e e x p r e s s e d i n t h e r o t a t e d c o o r d i n a t e s o f C h a p t e r 2 . W h e n t h e r e i s n o a u t o c o r r e l a t i o n , t h e t h i r d t e r m d r o p s o u t a n d t h e E M S E i s a s d e f i n e d b y H o e r l a n d K e n n a r d ( 1 9 7 0 ) . I f o r d i n a r y r i d g e r e g r e s s i o n i s p e r f o r m e d , t h e e f f e c t o f t h i s a d d i t i o n a l t e r m d e p e n d s o n t h e v a l u e o f p a n d a l s o t h e t i m e s e r i e s b e h a v i o r o f t h e t r a n s f o r m e d v a r i a b l e s w { i } . F o r e x a m p l e ; • I f p i s p o s i t i v e ( n e g a t i v e ) a n d l a r g e i n a b s o l u t e v a l u e , a n d t h e i n d e p e n d e n t v a r i a b l e s , e s p e c i a l l y t h o s e c o r r e s p o n d i n g t o t h e s m a l l e s t e i g e n v a l u e s , a r e 1 3 p o s i t i v e l y ( n e g a t i v e l y ) s e r i a l l y c o r r e l a t e d , then t h i s a d d i t i o n a l term w i l l be p o s i t i v e and produce l a r g e r EMSE than i n the case of u n c o r r e l a t e d e r r o r s . • I f p i s p o s i t i v e and l a r g e and the independent v a r i a b l e s are not s e r i a l l y c o r r e l a t e d , then t h i s a d d i t i o n a l term w i l l be c l o s e to zero and the EMSE w i l l be about the same as t h a t i n the case of E(ce') = o 2 I . • I f p i s negative and the independent v a r i a b l e s have p o s i t i v e s e r i a l c o r r e l a t i o n , then t h i s term w i l l be n e g a t i v e , i t s magnitude depending on the s i z e of p. If p i s near - 1 , c o n s i d e r a b l e c o r r e l a t i o n between the terms i n d i f f e r e n t powers of p would be expected and the r e s u l t i n g e x p r e s s i o n should be c l o s e to z e r o . I f p i s such that p 2 << |p| then the negative terms should dominate and t h i s e x p r e s s i o n w i l l be nonzero. 14 5.2 The Estimation Of The Autocorrelation Parameter There have been several d i f f e r e n t procedures developed to estimate p. The f i r s t procedure was introduced by Cochrane and Orcutt(1949). The o r i g i n a l regression model with autocorrelated errors can be rewritten as Y{t} - pY{t-l} = a(1-p) + 0,(x,{t}-px,{t-1}) + 02(x 2(t}-px 2{t-1}) + ... + (5.2.1) 0p(x p{t}-px p{t-l}) + e{t} where the X matrix and Y vector are unstandardized and e{t} i s as previously defined. Minimizing the sum of squared residuals leads to nonlinear equations from which analytic expressions for 0 and p cannot be obtained. The Cochrane and Orcutt i t e r a t i v e procedure i n i t i a l l y c alculates the OLS estimates of 0 by assuming p = 0. The residuals are then used to fin d the OLS estimate of p for the equation, e{t} = pe{t-l}. This estimate i s then used to find a GLS estimate of 0 resulting in a new set of residuals. New estimates of 0 and p are a l t e r n a t i v e l y calculated u n t i l a predefined convergence c r i t e r i a i s s a t i s f i e d . Another i t e r a t i v e method used to estimate p is based on maximum l i k e l i h o o d estimation(MLE). It assumes the Y has a multivariate normal d i s t r i b u t i o n with mean vector X0 and covariance matrix o 2V. The MLE of 0, p, and a2 are those which maximize the l o g - l i k e l i h o o d function for 0, p, and a2 given Y = y. The resulting equations are again nonlinear in the 1 5 p a r a m e t e r s ; a n i t e r a t i v e a l g o r i t h m h a s b e e n d e v e l o p e d b y B e a c h a n d M a c K i n n o n ( 1 9 7 8 ) t o f i n d t h e m a x i m u m . F o r a d e t a i l e d d e s c r i p t i o n o f t h i s m a x i m u m l i k e l i h o o d m e t h o d s e e J u d g e e t a l ( p p . 4 4 6 - 4 4 8 ) . T h e C o c h r a n e - O r c u t t a n d M L E m e t h o d s u s u a l l y p r o v i d e a l m o s t i d e n t i c a l e s t i m a t e s . I f t h e f i r s t o b s e r v a t i o n i s d r o p p e d , t h e s a m p l e s i z e i s s m a l l , a n d | p | i s v e r y c l o s e t o o n e ; i t i s p o s s i b l e t h a t t h e C o c h r a n e - O r c u t t p r o c e d u r e w i l l f i n d a n e s t i m a t e o f p g r e a t e r t h a n o n e . ( T h e O L S e s t i m a t i o n o f p d o e s n o t r e q u i r e | p | ^ 1 . ) T h e i m p l e m e n t a t i o n i f g e n e r a l i z e d r i d g e p r e s e n t e d h e r e r e q u i r e s t h e f i r s t o b s e r v a t i o n t o b e d r o p p e d . T h e r e f o r e , t h e u s e o f t h e m a x i m u m l i k e l i h o o d e s t i m a t e i s a d v i s a b l e , e s p e c i a l l y w h e n i t d i f f e r s s u b s t a n t i a l l y f r o m t h e C o c h r a n e - O r c u t t . D u r b i n (1969) s u g g e s t e d a n o t h e r p r o c e d u r e w h i c h G r i l i c h e s a n d R a o (1969) f o u n d t o p e r f o r m a l m o s t a s w e l l a s t h e C o c h r a n e -O r c u t t p r o c e d u r e i n s m a l l s a m p l e s . D u r b i n ' s a p p r o a c h i s t o p e r f o r m l e a s t s q u a r e s r e g r e s s i o n u s i n g a f o r m o f ( 5 . 2 . 1 ) : Y { t } = a ( 1 - p ) + p Y { t - U + 0 , ( x , { t } - p x , { t - 1 } ) + ($ 2 ( x 2 ( t } - p x 2 ( t - 1 } ) + . . . + ( 5 . 2 . 2 ) • / J p ( x p { t } - p x p { t - l } ) + e { t } T h e e s t i m a t e o f p i s t h e l e a s t s q u a r e s e s t i m a t e d c o e f f i c i e n t o f Y { t - 1 } . T h e o n l y c o m p u t e r p a c k a g e s a v a i l a b l e a t t h e U n i v e r s i t y o f B r i t i s h C o l u m b i a w h i c h c o n t a i n p r o c e d u r e s f o r t h e e s t i m a t i o n o f p a r e S H A Z A M a n d T S P . B o t h o f f e r t h e C o c h r a n e - O r c u t t a n d M L E m e t h o d s b u t n o t t h e D u r b i n m e t h o d . A n o t h e r s h o r t c o m i n g o f t h e 16 D u r b i n p r o c e d u r e i s t h a t w h e n ' t i m e ' i s a n i n d e p e n d e n t v a r i a b l e ; i n t h e r e g r e s s i o n t o e s t i m a t e p , t h e X m a t r i x c o n t a i n s ' t i m e ' a n d ' t i m e - 1 ' r e s u l t i n g i n a s i n g u l a r X ' X m a t r i x w h i c h c a n n o t b e i n v e r t e d . A l s o , w h e n t h e i n d e p e n d e n t v a r i a b l e s a r e h i g h l y a u t o c o r r e l a t e d , t h e X m a t r i x u s e d i n ( 5 . 2 . 2 ) i s h i g h l y c o l l i n e a r . A n a d v a n t a g e o f t h e D u r b i n m e t h o d i n M o n t e C a r l o s t u d i e s i s t h a t i t i s i n e x p e n s i v e t o c o m p u t e . A l s o , t h e s i m u l a t e d d a t a a v o i d s t h e o b s e r v e d w e a k n e s s e s o f t h e m e t h o d b e c a u s e i t i s u s u a l l y ' c l e a n ' a n d w e l l - b e h a v e d . T h e e s t i m a t i o n o f p i s n o t t h e o n l y d i f f i c u l t y e n c o u n t e r e d i n g e n e r a l i z e d l e a s t s q u a r e s . D e f i n i n g X * = P X a n d Y * = P Y , a s i n 3 . 4 , t h e O L S e s t i m a t e ( X * ' X * ) - 1 X * ' Y * w i l l y i e l d ( 3 . 2 ) e x a c t l y . W h e n C o c h r a n e a n d O r c u t t i n i t i a l l y s u g g e s t e d G L S , t h e f i r s t o b s e r v a t i o n w a s d e l e t e d f r o m X * a n d Y * . T h i s w i l l u s u a l l y l e a d t o a c l o s e a p p r o x i m a t i o n t o ( 3 . 2 ) a n d t h e r e a r e t w o d r a w b a c k s t o i n c l u d i n g t h e f i r s t r o w . A n y d i s s i m i l a r i t i e s b e t w e e n t h e f i r s t o b s e r v a t i o n a n d t h e r e s t o f t h e d a t a m a y b e m a g n i f i e d b y i t s w e i g h t i n g o f / 1 - p 2 . M o r e i m p o r t a n t t o t h e m e t h o d o f g e n e r a l i z e d r i d g e , i s t h e f a c t t h a t t h e ' c o n s t a n t ' c o l u m n i n t h e X m a t r i x i s n o l o n g e r c o n s t a n t . T h i s m e a n s t h a t X * a n d Y * c a n n o t b e s t a n d a r d i z e d . T h e u s e o f g e n e r a l i z e d r i d g e r e g r e s s i o n p r e s e n t e d h e r e r e q u i r e s s t a n d a r d i z a t i o n a n d t h e r e f o r e t h e f i r s t r o w i s n o t i n c l u d e d . T h e f i r s t o b s e r v a t i o n i s r e m o v e d i n t h e C o c h r a n e - O r c u t t a n d t h e m a x i m u m l i k e l i h o o d e s t i m a t i o n o f p . I f t h e e s t i m a t e i s c a l c u l a t e d i n c l u d i n g t h e f i r s t r o w i n t h e G L S s t e p o f e a c h i t e r a t i o n ; o n c e c o n v e r g e n c e i s r e a c h e d , i t 17 would be i n c o n s i s t e n t to d e l e t e the f i r s t row i n the f i n a l i t e r a t i o n . 18 5.3 S t a n d a r d i z a t i o n S t a n d a r d i z a t i o n can be used i n o r d i n a r y l e a s t squares because the unbiased e s t i m a t o r i s i n v a r i a n t to s c a l i n g a;nd c e n t e r i n g . The usual p r a c t i c e i n r i d g e r e g r e s s i o n i s t o s t a n d a r d i z e the data before i n t r o d u c i n g the r i d g e parameter. U n f o r t u n a t e l y , b i a s e d e s t i m a t o r s such as the r i d g e e s t i m a t o r are not i n v a r i a n t to s c a l i n g and c e n t e r i n g . T h i s t e d (1976) and B e l s l e y , Kuh, and Welsch (1980) both d i s c u s s t h i s problem i n d e t a i l whereas others such a s Gibbons (1981) and Wichern and C h u r c h i l l (1978) do not q u e s t i o n the c h o i c e of s t a n d a r d i z a t i o n . When the components of the X matrix d i f f e r i n s c a l e , the c o n d i t i o n number 5 of the X:'X matrix w i l l be m i s l e a d i n g l y h i g h . C o l l i n e a r i t y may then fee suspected when a c t u a l l y there i s none. S t a n d a r d i z a t i o n s c a l e s the v a r i a b l e s to z e r o mean and u n i t standard d e v i a t i o n . T h i s s c a l i n g and c e n t e r i n g has been used i n most a p p l i c a t i o n s of ridge r e g r e s s i o n and thus the d e c i s i o n t o s t a n d a r d i z e not only r e s u l t s i n c o n f o r m i t y but a l l o w s f o r comparison with p r e v i o u s s t u d i e s . By c e n t e r i n g , the c o n d i t i o n number of the X matrix i s a u t o m a t i c a l l y reduced implying that any i l l - c o n d i t i o n i n g due to the c o n s t a n t i s not a problem and i s ' n o n - e s s e n t i a l 1 . A reasonable value f o r the constant i s r a r e l y known i n p r a c t i c e and i n t r o d u c i n g b i a s to change the estimate fof the c o n s t a n t may not r e s u l t i n a b e t t e r estimate. However, when 5 The c o n d i t i o n number of a matrix i s i n d i c a t e d by the r a t i o of the maximum to minimum e i g e n v a l u e s . The matrix i s more i l l -c o n d i t i o n e d (or n e a r l y s i n g u l a r ) as the r a t i o i n c r e a s e s . 1 9 t h e v a l u e o f t h e c o n s t a n t i s k n o w n ' a p r i o r i ' , c e n t e r i n g i s n o t r e c o m m e n d e d . N o r m a l i z i n g t h e d a t a t o v e c t o r s o f u n i t l e n g t h i s a n o t h e r s c a l i n g t e c h n i q u e s o m e t i m e s u s e d i n r i d g e r e g r e s s i o n . I t i s p r e f e r r e d t o s t a n d a r d i z a t i o n w h e n t h e r e g r e s s i o n m o d e l d o e s n o t i n c l u d e t h e c o n s t a n t t e r m . N o r m a l i z a t i o n i s a p p r o p r i a t e w h e n e s t i m a t i o n i s n o t i n v a r i a n t t o c e n t e r i n g ; f o r e x a m p l e , w h e n t h e r e i s n o c o n s t a n t t e r m . I f t h e G L S t r a n s f o r m a t i o n i s a p p l i e d , c h o o s i n g b e t w e e n s t a n d a r d i z i n g a n d n o r m a l i z i n g n e c e s s a r i l y i m p l i e s a d e c i s i o n w h e t h e r o r n o t t o i n c l u d e t h e f i r s t r o w o f t h e P m a t r i x i n t h e t r a n s f o r m a t i o n . A n i m p o r t a n t f a c t o r w h i c h e n t e r s a t t h i s p o i n t i s t h e e s t i m a t i o n m e t h o d o f t h e r i d g e p a r a m e t e r , k . T h e r e s u l t s o f p a s t s t u d i e s w e r e r e l i e d u p o n ; i n p a r t i c u l a r , t h e s t u d y b y G i b b o n s ( 1 9 8 1 ) . I n a l l t h e s t u d i e s w h i c h w e r e r e v i e w e d , t h e d a t a w a s s t a n d a r d i z e d a n d i t d o e s n o t n e c e s s a r i l y f o l l o w t h a t t h e i r c o n c l u s i o n s w i l l h o l d f o r s o m e o t h e r t r a n s f o r m a t i o n s u c h a s n o r m a l i z i n g t o u n i t l e n g t h . I f t h e c h o i c e o f k i s t o o s m a l l t o b e n o t i c e b l e o r t o o l a r g e , i n t r o d u c i n g u n n e c e s s a r y b i a s , t h e p e r f o r m a n c e o f t h e g e n e r a l i z e d r i d g e r e g r e s s i o n m a y a p p e a r t o b e p o o r f o r t h e w r o n g r e a s o n s . B a s e d o n t h e s e l a s t c o m m e n t s , t h e r e c o m m e n d e d a p p r o a c h i n t h e a p p l i c a t i o n o f g e n e r a l i z e d r i d g e i s t o s t a n d a r d i z e t h e d a t a . I f t h e r e i s ' a p r i o r i ' k n o w l e d g e c o n c e r n i n g t h e c e n t e r i n g o f t h e d a t a , t h e d a t a s h o u l d b e a t l e a s t s c a l e d t o h a v e a u n i t s t a n d a r d d e v i a t i o n . 20 5.4 The Estimation Of The Ridge Parameter The selection of the best value for the k parameter in ridge regression has been the subject of many empirical studies. The purpose of thi s study i s not to survey a l l the available methods for k but to only consider those which performed well in the usual ridge setting. The results of Gibbons (1981) are used for guidance. The methods she showed to perform best o v e r a l l are referred to as RIDGM, HKB and GHW. RIDGM also performed best o v e r a l l in an extensive study of Dempster, Schatzoff and Wermuth (1977) and HKB performed well in the study of Hoerl, Kennard, and Baldwin (1973). Wichern and C h u r c h i l l (1978) could not recommend using HKB without further study. The t h i r d method, GHW, performed equally as well as HKB and RIDGM for Gibbons and was recommended in the simulation of Golub, Heath, and Wahba (1979). They noted that this method did not require an estimate of o 2 and can be used when the number of degrees of freedom i s small. The HKB method minimizes the expected mean squared error with respect to k. Considering the general ridge estimator 0{R} = ( X ' X + G'KG)" 1 X ' Y (5.4.1) where: G i s as defined in Section 2; K = diag(k,,k 2,...,kp). The k{i}'s minimizing EMSE are k{i} = o 2/a 2{i} where a{i} i s as defined in (2.2). k i s chosen to be the harmonic mean of the k's and i s given by k = pa2//3'/3. The estimates 0{OLS} and s 2 are used for /3 and a 2 where s 2 i s defined by 21 s 2 =( Y -x3)'(Y -x3)/(n-p-1) (5.4.2) therefore: k{HKB} = ps 2 / ( /3{OLS} ) ' (J3{OLS} ) . (5.4.3) RIDGM uses a Bayesian approach to select k since ridge estimators are Bayes estimators under appropriate prior d i s t r i b u t i o n s . Dempster, Schatzoff, and Wermuth (1977) define the RIDGM procedure as choosing the k to s a t i s f y P r aMi} i I = ps 2 (5.4.4) i = 1 L d/k) + (l/X{i}) J where a{i} i s as defined in (2.2) and s 2 i s defined as above. The t h i r d estimator, GHW, was derived by Golub, Keath and Wahba (1979) using the techniques of cross-validation in smoothing problems. Rewritten in the context of the k parameter, k i s chosen to minimize | |[l-X(X'X+kI)- 1X']Y| | 2 V{k} = (5.4.5) {trace[l-X(X'X+kI)- 1X']} 2 where ||•I I refers to the Euclidean norm and the trace is the sum of the diagonal elements. When k i s equal to zero, V(0) i s simply the sum of the squared residuals divided by (n-p) 2. Although these estimators have a l l been shown to perform well in simulations, the ridge trace i s s t i l l more popular in the application of ridge regression in real s i t u a t i o n s . The ridge trace i s a plot of k versus the standardized c o e f f i c i e n t 22 e s t i m a t e s w i t h k u s u a l l y r a n g i n g f r o m z e r o t o o n e . T h i s m e t h o d i s v e r y s u b j e c t i v e b e c a u s e k i s c h o s e n a t a p o i n t w h e r e t h e e s t i m a t e s a p p e a r t o ' s t a b i l i z e ' a n d h a v e t h e ' a p r i o r i ' e x p e c t e d s i g n . T h i s w e a k n e s s h a s e n c o u r a g e d t h e c o n t i n u i n g s e a r c h f o r a n o b j e c t i v e , a n a l y t i c a l m e t h o d f o r e s t i m a t i n g t h e r i d g e p a r a m e t e r . 23 V I . T H E M O N T E C A R L O S I M U L A T I O N 6 . 1 D a t a G e n e r a t i o n T h e p r o c e d u r e u s e d f o r d a t a g e n e r a t i o n p r o v i d e s l i n e a r m o d e l s w i t h c o l l i n e a r i n d e p e n d e n t v a r i a b l e s a n d a u t o c o r r e l a t e d e r r o r s . T h e d a t a i s g e n e r a t e d u s i n g m e t h o d s s i m i l a r t o t h o s e o f M c D o n a l d a n d G a l a r n e a u ( 1 9 7 5 ) , W i c h e r n a n d C h u r c h i l l ( 1 9 7 8 ) , a n d G i b b o n s ( 1 9 8 1 ) . T h e g e n e r a t e d m o d e l s c o n t a i n 3 i n d e p e n d e n t v a r i a b l e s ( p = 3 ) a n d 4 0 o b s e r v a t i o n s . F o u r s e t s o f i n d e p e n d e n t v a r i a b l e s a r e c o n s i d e r e d . T h e i n d e p e n d e n t v a r i a b l e s a r e g e n e r a t e d f o r t h e f i r s t t h r e e s e t s a c c o r d i n g t o x { t , j } = {\ZTzirr)z{t, j } + i / / z { t , 4 } t = 1 , . . . 4 0 ( 6 . 1 . 1 ) w h e r e j = 1 , 2 , 3 ( S e t A a n d B ) , j = 1 , 2 ; x { t , 3 } = z { t , 3 } ( S e t C ) a n d z { t , j } ; t = 1 , . . . , 4 0 ; j = 1 , 4 a r e N ( 0 , 1 ) r a n d o m v a r i a b l e s . T h e r e s u l t i n g c o r r e l a t i o n b e t w e e n e a c h p a i r o f x ' s i s i>z w h i c h w a s c h o s e n t o b e . 9 8 f o r t h e f i r s t a n d t h i r d s e t s a n d . 7 2 f o r t h e s e c o n d s e t . T h e f i r s t a n d s e c o n d s e t s a r e b e r e p r e s e n t e d b y o n e ' l a r g e ' a n d t w o ' v e r y s m a l l ' e i g e n v a l u e s . T h e t h i r d s e t w a s g e n e r a t e d t o p r o v i d e m o r e e q u a l l y s p a c e d e i g e n v a l u e s ( T a b l e 1 ) . I n t h e f o u r t h s e t , S e t D, t h e i n d e p e n d e n t v a r i a b l e s a r e b o t h c o l l i n e a r a n d p o s i t i v e s e r i a l l y c o r r e l a t e d , a c o m m o n o c c u r r e n c e i n t i m e s e r i e s r e g r e s s i o n . H e r e , t h e a d d i t i o n a l t e r m i n t h e f o r m u l a f o r t h e E M S E o f r i d g e ( 1 . 2 ) i s p o s i t i v e a n d t h e i m p r o v e m e n t i n t h e E M S E f r o m u s i n g g e n e r a l i z e d r i d g e c o m p a r e d t o 2 4 S E T A X 1 , X 2 , X 3 p a i r w i s e c o r r e l a t e d S E T C X 1 , X 2 p a i r w i s e c o r r e l a t e d E i g e n v a l u e 0max /32 0min 2 . 9 3 6 . 0 2 8 . 0 1 6 p 'max 0 2 0min 2 . 0 1 4 . 9 7 0 . 0 1 6 E i g e n v e c t o r 0, . 5 7 8 - . 4 4 8 . 6 8 2 . 5 7 8 - . 3 6 5 - . 7 3 0 . 5 7 6 . 8 1 6 . 0 4 8 . 6 9 8 . 1 1 2 . 7 0 8 . 6 9 6 . 1 2 8 - . 7 0 6 - . 1 6 9 . 9 8 5 . 0 1 1 P e r c e n t a g e o f v a r i a b i l i t y . 9 8 6 . 0 0 9 . 0 0 5 . 6 7 1 . 3 2 3 . 0 0 5 T a b l e I - E i g e n v a l u e s a n d E i g e n v e c t o r s o f S i m u l a t e d D a t a O L S i s e x p e c t e d t o b e g r e a t e r t h a n w h e n t h e r e i s n o s e r i a l ' c o r r e l a t i o n . T h e Z ' s a r e g e n e r a t e d a s b e f o r e e x c e p t t h a t f i f t y o b s e r v a t i o n s a r e g e n e r a t e d a c c o r d i n g t o ( 6 . 1 . 2 ) . T o a v o i d b i a s f r o m i n i t i a l i z a t i o n o f z " { t , j } , t h e f i r s t t e n o b s e r v a t i o n s a r e d i s c a r d e d . T h e s c h e m e i s g i v e n b y z " { 0 , j } = 0 " z " { t , j } = 7 z " { t - 1 , j } + z { t , j } t = 1 , . . . , 4 0 ; j = 1 , . . . , 4 . ( 6 . 1 . 2 ) x { t , j } = ( / T ^ F ) z " { k , j } + ^ z " { k , 4 } t = 1 , . . . , 4 0 ; k=1 1 , . . . , 5 0 ; j = 1 , 2 , 3 . w h e r e \p2 e q u a l s . 9 8 a n d 7 e q u a l s . 8 , a s i g n i f i c a n t p o s i t i v e l e v e l o f s e r i a l c o r r e l a t i o n . T h e v a l u e o f t h e d e p e n d e n t v a r i a b l e i s d e t e r m i n e d b y t h e f o l l o w i n g : y { t } = 0 O + 0 , x { t , O + 0 2 x { t , 2 } + 0 3 x { t , 3 } + c { t } ( 6 . 1 . 3 ) 25 w h e r e e { t } f o l l o w s t h e f i r s t o r d e r a u t o r e g r e s s i v e s c h e m e , e { t } = p e { t - l } + u { t } a n d u { t } i s a n i n d e p e n d e n t N ( 0 , o 2 ) r a n d o m v a r i a b l e . T h e g e n e r a t i o n o f t h i s v a r i a b l e i n v o l v e s t h r e e p a r a m e t e r s ; t h e o r i g i n a l |3 v e c t o r , t h e v a r i a n c e o f t h e e r r o r t e r m , a n d t h e a u t o c o r r e l a t i o n f a c t o r , 7. C h o o s i n g 0 O t o b e i d e n t i c a l l y z e r o a n d 0,, 0 2 , a n d 0 3 a s t h e a p p r o p r i a t e e i g e n v e c t o r v a l u e s c o r r e s p o n d i n g t o t h e m a x i m u m a n d m i n i m u m e i g e n v a l u e s p r o v i d e s e x t r e m e s i n t e r m s o f t h e E M S E . A s w e l l a s u s i n g 0max ( c o r r e s p o n d i n g t o t h e m a x i m u m e i g e n v a l u e ) a n d 0 m i n ( c o r r e s p o n d i n g t o t h e m i n i m u m e i g e n v a l u e ) , t h e e i g e n v e c t o r i n b e t w e e n ( 0 2 ) i s u s e d f o r t h e c a s e s o f $ 2 = . 9 8 w i t h o n e a n d t h r e e p a i r s o f c o r r e l a t e d i n d e p e n d e n t v a r i a b l e s ( S e t A a n d S e t C ) . T h i s i s d o n e b e c a u s e w h e n a l l t h r e e v a r i a b l e s a r e p a i r w i s e c o r r e l a t e d , t h e s e c o n d e i g e n v a l u e i s n o t m u c h l a r g e r t h a n t h e t h i r d a n d i t w a s s u s p e c t e d t h a t s o m e o f t h e r e s u l t s o f e a r l i e r s i m u l a t i o n s m a y b e a t t r i b u t e d t o t h i s c o n s t r u c t i o n . W h e n o n l y x , a n d x 2 a r e c o r r e l a t e d , t h e s e c o n d e i g e n v a l u e i s l a r g e c o m p a r e d t o t h e t h i r d a n d b y c h a n g i n g t h e d e s i g n , t h e e f f e c t o f a m o r e e v e n l y s p a c e d s p e c t r u m i s e x p l o r e d . T o a v o i d e f f e c t s o f i n i t i a l i z a t i o n , 5 0 o b s e r v a t i o n s o f t h e e r r o r t e r m , e { t } , a r e g e n e r a t e d b u t t h e f i r s t t e n a r e d i s c a r d e d . F i v e c h o i c e s o f a2 a r e c o n s i d e r e d ; a2 = . 0 1 , . 1 , . 5 , 1 . 0 , a n d 4 . 0 . S i n c e t h e e r r o r s a r e n o t i n d e p e n d e n t , t h e s i g n a l - t o - n o i s e r a t i o i s d e f i n e d a s 0 ' 0 / o 2 w h e r e a2 i s t h e v a r i a n c e u s e d t o c o n s t r u c t t h e a u t o c o r r e l a t e d e r r o r s . T h e c o r r e s p o n d i n g r a t i o s a r e 1 0 0 , 1 0 , 2 , 1 , a n d . 2 5 . T h e a u t o c o r r e l a t i o n p a r a m e t e r , p i s k e p t c o n s t a n t a t . 9 b e c a u s e t h i s s t u d y d e a l s w i t h t h e p r o b l e m o f 26 autocorrelation. Adding the si t u a t i o n of ne g l i g i b l e or low autocorrelation would double the volume of output and would only provide v e r i f i c a t i o n of known results that the GLS transformation should not be used in the non-autocorrelated case. One hundred samples are generated for a fixed X matrix and 0 vector by changing the random error and hence the dependent variable. To generate the random errors as well as the Z's used in the creation of the independent variables, a FORTRAN subroutine based on Marsaglia's Rectangle-Wedge-Tail-Method ( c f . Knuth (1973)) is used. Both the random errors and the X matrix are generated using FORTRAN. APL i s used to perform the Monte Carlo simulation and subsequent c a l c u l a t i o n s . A l l computations were performed on the UBC AMDAHL 470-V8 computer. 27 6 . 2 P a r a m e t e r E s t i m a t i o n T h e D u r b i n m e t h o d w a s u s e d i n t h e s i m u l a t i o n t o e s t i m a t e t h e a u t o c o r r e l a t i o n p a r a m e t e r , p . T h e l a r g e n u m b e r o f e s t i m a t e s c a l c u l a t e d m a d e t h e i t e r a t i v e m e t h o d s o f C o c h r a n e - O r c u t t a n d m a x i m u m l i k e l i h o o d t o o c o s t l y . B e c a u s e t h e d a t a w a s s i m u l a t e d , i t w a s ' c l e a n ' , m i n i m i z i n g t h e p o t e n t i a l f o r p o o r e s t i m a t e s f r o m t h e D u r b i n m e t h o d . A l s o , t h e D u r b i n m e t h o d h a s p e r f o r m e d w e l l i n o t h e r s i m u l a t e d r e g r e s s i o n s ( G r i l i c h e s a n d R a o ( l 9 6 9 ) ) . T h e D u r b i n m e t h o d d i d u n d e r e s t i m a t e p i n t h e p r e s e n t s t u d y b u t n o t b y a n a m o u n t w h i c h a f f e c t e d t h e s u p p o r t f o r t h e u s e o f g e n e r a l i z e d l e a s t s q u a r e s . T h e C o c h r a n e - O r c u t t p r o c e d u r e w a s r u n o n s o m e o f t h e s i m u l a t e d d a t a a n d a s e x p e c t e d , i t s e s t i m a t e o f p w a s c l o s e r t o t h e a c t u a l v a l u e o f p t h a n t h e D u r b i n e s t i m a t e . T h e e s t i m a t i o n o f t h e r i d g e p a r a m e t e r w a s i n v e s t i g a t e d m o r e t h o r o u g h l y . T h e m e t h o d s o f H K B a n d R I D G M w e r e c o m p a r e d t o t h e ' b e s t ' k . T h e ' b e s t ' k i s d e f i n e d a s t h a t w h i c h m i n i m i z e s t h e E M S E : ( 0 { O R } - 0 ) ' ( 0 { O R } " P) o r ( 0 { G R } " 0 ) ' ( 0 { G R } - 0 ) . T h e ' b e s t ' k c a n b e d e t e r m i n e d e x p l i c i t l y i n t h e s i m u l a t i o n b e c a u s e 0 i s k n o w n . 0 i s , o f c o u r s e , n o t a v a i l a b l e i n p r a c t i c e . H o w e v e r , i t i s u s e f u l t o s e e h o w t h e k e s t i m a t i o n m e t h o d s p e r f o r m i n r e l a t i o n t o t h e ' b e s t ' m e t h o d s ( r e f e r r e d t o a s B E S T ) . I n a d d i t i o n , c o m p a r i n g t h e ' b e s t ' r i d g e e s t i m a t e s u s i n g o r d i n a r y v e r s u s g e n e r a l i z e d l e a s t s q u a r e s p r o v i d e s i n s i g h t i n t o t h e p e r f o r m a n c e o f g e n e r a l i z e d r i d g e . B o t h R I D G M a n d B E S T r e q u i r e n o n l i n e a r m i n i m i z a t i o n s . T h e s e a r c h m e t h o d o f b r a c k e t i n g i s u s e d a s d e t a i l e d i n A v r i e l ( 1 9 7 6 ) . 2 8 I t i s an i t e r a t i v e method which f i n d s two f u n c t i o n values which bracket a smaller f u n c t i o n value, narrowing the width of the bracket u n t i l i t reaches a predefined convergence c r i t e r i a . I f the values of k are g e t t i n g large (greater that 9.9999) the search i s stopped and a k value of 9.9999 i s s e l e c t e d . This means that when RIDGM or BEST produce many large k's, the methods could perform b e t t e r than i n d i c a t e d i n the s i m u l a t i o n . For each sample i n the s i m u l a t i o n , s i x t e e n regressions were performed; eight with the a u t o c o r r e l a t i o n t r a n s f o r m a t i o n , eight without. Within each s e t , the regressions use 'no ri d g e ' , RIDGM, HKB, 'best' k, and four constant k's(.1, .3, .5 and .7). The constants were included to observe the performance of the EMSE i f k i s the same i n every parameter s e t t i n g . The t h i r d method f o r e s t i m a t i n g k, GHW, was not included i n the o r i g i n a l s i m u l a t i o n because i t was far too c o s t l y . A f t e r completion of the Monte C a r l o study, the alg o r i t h m used to c a l c u l a t e the GHW estimate was r e w r i t t e n to be more e f f i c i e n t and cheaper to run. These r e s u l t s have been included i n t h i s paper. In the o r i g i n a l a l g o r i t h m , The bra c k e t i n g method was used f o r the non-linear search for the minimum V ( k ) . This i s not the most e f f i c i e n t method a v a i l a b l e but i t d i d not perform poorly enough to s i g n a l a change i n the other algorithms w i t h i n the s i m u l a t i o n . The al g o r i t h m was improved by r e p l a c i n g b r a c k e t i n g by the'.more e f f i c i e n t Golden-Section search method ( A v r i e l (1976)). This method f i n d s one 'bracket' and then uses unbalanced b i s e c t i o n . The b i s e c t i o n i s f u r t h e r explained by A v r i e l (1976) and continues u n t i l convergence. The Golden-2 9 S e c t i o n s e a r c h r e d u c e d t h e n u m b e r o f i t e r a t i o n s b y u p t o f i f t y p e r c e n t , a d e f i n i t e i m p r o v e m e n t o v e r ' b r a c k e t i n g ' . U n f o r t u n a t e l y , t h e c o m p u t a t i o n s i n e a c h i t e r a t i o n w i t h i n t h e s e a r c h w e r e s t i l l v e r y c o s t l y s o o n l y t h r e e o f t h e t e n s i m u l a t i o n s w e r e r u n u s i n g G H W . 30 6 . 3 D i a g n o s t i c s T h e M S E i s c a l c u l a t e d u s i n g t h e s t a n d a r d i z e d p " s a n d / 3 ' s . T h e o r i g i n a l / 3 ' s a r e u n s t a n d a r d i z e d s o t h e y a r e t r a n s f o r m e d t o s t a n d a r d i z e d f o r m b y m u l t i p l y i n g b y t h e s t a n d a r d d e v i a t i o n o f t h e i r r e s p e c t i v e i n d e p e n d e n t v a r i a b l e d i v i d e d b y t h e s t a n d a r d d e v i a t i o n o f Y . T h e / 3 ' s a r e t h e u n a l t e r e d r e s u l t s o f t h e r e g r e s s i o n s . U s i n g t h e s t a n d a r d i z e d / 3 ' s a n d / 3 ' s f o r t h e M S E c a l c u l a t i o n i g n o r e s t h e p r o b l e m o f e s t i m a t i n g t h e c o n s t a n t t e r m b u t i s c o n s i s t e n t w i t h p e r f o r m i n g r i d g e r e g r e s s i o n o n s t a n d a r d i z e d d a t a . F o r e a c h r e p l i c a t i o n ( j = 1 , . . . , 1 0 0 ) w e c o m p u t e M S E = [ I (/3 - /3 ) 21 ( 6 . 3 . 1 ) j <-i=1 i i J j a n d t h e e x p e c t a t i o n o f t h e M S E i s e s t i m a t e d b y M S E = L M S E / 1 0 0 ( 6 . 3 . 2 ) Lj = 1 j-l I t s s t a n d a r d e r r o r i s • 1 0 0 1 S E = / | L ( M S E - M S E ) 2 / ( 1 0 0 - 1 ) ( 6 . 3 . 2 ) - j =1 j J I n t e r v a l e s t i m a t e s f o r E M S E o f t h e f r o m M S E ± 2 S E a r e t h e b a s i s o f c o m p a r i s o n b e t w e e n e s t i m a t i o n m e t h o d s . T h e e x p e c t a t i o n a n d s t a n d a r d e r r o r s o f t h e u n s t a n d a r d i z e d / 3 ' s , a n d a s u m m a r y o f t h e k - v a l u e s a r e o b t a i n e d . I n a d d i t i o n , t h e R 2 o f b o t h O L S a n d 31 GLS estimates are calculated, providing insight as to which situations are most p r a c t i c a l . To simplify notation EMSE refer to MSE for the remainder of the paper. 32 6 . 4 Results The autocorrelation transformation and ridge regression are performed with the aim of obtaining estimates with a smaller MSE than OLS and ordinary ridge estimators. Since one hundred t r i a l s were performed for each simulation i t i s very l i k e l y that the i n t e r v a l of EMSE ± 2SE w i l l contain the actual value of the mean squared error. Ten figures are included, each containing two graphs. The f i r s t graph represents the upper and lower l i m i t s for the eight estimators as a function of a. In these graphs, the v e r t i c a l axes do not have a common scale. To simplify comparisons, the second graph presents the r a t i o of the EMSE for a p a r t i c u l a r autocorrelation transformation and/or ridge regression to the EMSE of OLS as a function of a. This measure allows for the results of a l l combinations of varied parameters to be represented on the same scale. As the r a t i o decreases, the performance of the new estimator r e l a t i v e to the OLS estimator improves. The estimators for the constant k-values are not included in these figures because they are not meant to be a recommended method for choosing k but were analysed to provide insight as to what occurs when k i s kept constant as the other parameters vary. Each of the ten figures represents a 0 vector and X matrix combination. Included in each figure, the average R2 of the GLS regression are given for each o. Figures 1 , 2, and 3 (Set A) correspond to using 0max, 0min, and' 02 respectively as the 0 vector. The three pairs of independent variables have co r r e l a t i o n s of \p2 = .98 and there i s 33 no s e r i a l c o r r e l a t i o n in any of the independent variables. These results are compared to one another and then are compared to three other configurations, each d i f f e r i n g from Set A in one respect only. Figures 4 and 5 (Set B) have a pairwise c o r r e l a t i o n between a l l variables of i£2=.72; figures 6, 7, and 8 (Set C) have only one pair of correlated independent variables, X, and X 2; and figures 9 and 10 (Set D) have s e r i a l l y correlated independent variables. Results for 02 in Sets A and D are presented to study the effe c t of the di f f e r e n t eigenvalue structure. The following i s a legend explaining the abbreviations and graphs used in the simulation r e s u l t s : Without autocorrelation transformation:. H (. OLS - no ridge. i h ORIDGM - ridge using RIDGM. + — — — — — i - OHKB - ridge using HKB. H 1- OGHW - ridge using GHW. H y OBEST - ridge using 'best' k. With autocorrelation transformati'on: H 1- GLS - no ridge. + 1- GRIDGM - ridge using RIDGM. + + GHKB - ridge using HKB. 4 _ _ _ _ _ _ _ _ + GGHW - ridge using GHW. + - - - - - - - - H - GBEST - ridge using 'best' k. 'L' refers to lower i n t e r v a l bound. 'U' refers to upper i n t e r v a l bound. OGHW and GGHW replace OBEST and GBEST respectively in figures 11, 12, and 13. 3 4 6.4.1 Set A X's Pairwise Correlated with \p2 = .98. 0max: CO Figure 1 - Set A: /3max i . For every a: a. Every estimator performs better than OLS. b. GHKB and GRIDGM perform better than OHKB and ORIDGM and GLS. c. ORIDGM performs better or the same as OHKB. d. GRIDGM performs better or the same as GHKB. e. GBEST outperforms OBEST. i i . The EMSE of GLS r e l a t i v e to the EMSE of OLS increases as o increases. This i s because the error term i s made up of proportionately more of the random error 3 5 component than the autocorrelation component as a increases. Thus, the autocorrelation e f f e c t becomes dominated by the larger random e f f e c t . i i i . The r a t i o of the EMSE of OHKB and ORIDGM to that of OLS decreases as a increases. This i s because the k determined by HKB and RIDGM i s an increasing function of a. Since both of these methods estimated a k value smaller than that of the BEST results, increasing a decreases this discrepancy. i v . The EMSE of GHKB and GRIDGM r e l a t i v e to the EMSE of OLS do not change s i g n i f i c a n t l y as a increases. This is because of the combined e f f e c t s discussed in ( i i ) and ( i i i ) above. 36 /Jmin: oo .—.o ' cn • _i o on= /TV v GLSL _OH>3: _OMKEJ 4/ i on ' LiJo ' I I I I I I I I : 0. -a 0 6 1 SiGSR 91 .117 .032 V, 4-4-w \ i V,/ +" \\\ _0M<£ - \ - +• _ OP:DC>" i i i I i i i—i i i i— r - r o 0.4 o.e. „ j .2 S i G K -i 1 . 11 Figure 2 - Set A: /3min » ORIDGM and OHKB do not perform better than OLS for small o. Except for GRIDGM when a i s near . 3 , GHKB and GRIDGM perform better than GLS which performs better than OLS for a l l sigma. i i i . No ridge estimator in either the ' 0 ' or 'G' cases has a consistently lower interval estimate, than any other for a l l o. However, for a close to 2 , GRIDGM performs better than ORIDGM and OHKB. i v . OBEST performs better than GBEST for large o. 3 7 02: CO _) o LU • c o o \ 1 \ - i ^ ^ S X LU 1 \ , \\ \ ^ - -/ + - s LU CO LUo ' _GLS _ ~i i i i i i i i i i r 0 .0 b.a 0 8 S I G N ; I I ! F i g u r e 3 - S e t A : 0 2 i . E x c e p t f o r G R I D G M a n d G H K B w h e n a = . 1 , a l l e s t i m a t o r s p e r f o r m b e t t e r t h a n O L S f o r a l l a. i i . T h e ' G ' e s t i m a t o r s p e r f o r m b e t t e r t h a n t h e ' 0 ' e s t i m a t o r s f o r s m a l l a . i i i . F o r l a r g e a , G R I D G M p e r f o r m s b e t t e r t h a n O R I D G M a n d O H K B a n d G H K B p e r f o r m s a s w e l l a s O R I D G M a n d b e t t e r t h a n O H K B . 38 6.4.2 Set B X's Pairwise Correlated, \p2 = .72. p'max: Figure 4 - Set B: p'max i . Every estimator has a s i g n i f i c a n t l y lower EMSE than Set A. i i . The EMSE of OHKB and ORIDGM r e l a t i v e to the EMSE of OLS decreases at a faster rate for increasing o than was obtained in Set A. For o near 2, their performance i s sim i l a r to Set A. However, for small a there is l i t t l e improvement over OLS. The OLS estimator performs much better when the co r r e l a t i o n is low. i i i . The EMSE of GHKB and GRIDGM r e l a t i v e to the EMSE of OLS increases at a faster rate as c increases compared 39 to Set A. The increase in the random error component has a stronger e f f e c t than the increase in the estimated k value as a increases when the corr e l a t i o n i s low. i v . OBEST and GBEST do not perform as well as they do in Set A. This may be caused by the extremely small EMSE in Set C which a f f e c t s the stopping c r i t e r i a of the bracketing method for selecting k. 40 0 m i n : CX . SI a i T i—i—i—i—i i i m r c o o c o e .968 .755 . 398 . 267 .r? f \ \ \ T V "1 .—.cr V, 1 1 o 1 11 v><= 51 t-i_j v . * : c LU LI -21 _ r\j Luc / f _ / I I I I T i i i i r 0 4 0.6 F i g u r e 5 - S e t B : /3min 11. i . A l l t h e e s t i m a t o r s h a v e s i g n i f i c a n t l y l o w e r E M S E t h a n S e t A. O R I D G M a n d O H K B p e r f o r m b e t t e r t h a n O L S o n l y f o r a > . 7 0 7 1 b u t t h e y p e r f o r m b e t t e r t h a n O L S f o r o > .1 i n S e t A. G L S h a s a n E M S E c o m p a r e d t o t h e E M S E o f O L S t h a t i s s l i g h t l y b e t t e r t h a n S e t A e x c e p t f o r o = . 3 1 6 2 . G R I D G M d o e s n o t p e r f o r m a s w e l l a s i n S e t A. W h e n 0min i s u s e d , O R I D G M a n d G R I D G M b o t h h a v e a l a r g e n u m b e r o f s a m p l e s w h e r e k i s r e s t r i c t e d t o 9 . 9 9 9 9 . T h i s m e a n s t h a t t h e e s t i m a t o r s a r e n o t a s g o o d a s t h e y c o u l d b e a n d a n y n e g a t i v e c o n c l u s i o n s c o n c e r n i n g t h e s e i n i v , 41 estimators cannot be made, v. The performance of GHKB worsens for large a and improves for small a compared to Set A. 6.4.3 Set C X, and X2 have \/>2 = .98, X 3 i s uncorrelated with X, and X2. ^max: Figure 6 - Set C: 0max i . The change in the performance of each estimator as a varies i s similar to Set A but the EMSE is larger in Set A. i i . OHKB and ORIDGM have smaller EMSE r e l a t i v e to that of OLS than in Set A. 4 2 /?min: UJ>- • cr i \ i 1 1 1 \ ; \ — -' • \^ x^ _:v%..Vw'--"- ~"- - -5- -' + Gf :x" —* - GBtfTJ — _, / ^ -^K - _ GEIST I J _, 1 _ SETT., r"0***"'--— — — ^ _,' ' ~ ~1 "1 4 y „ - o l 4 , r ' r f i i i i i i i i i i ^ ' " ' i ' i i i > •• c i f i r i i i i i i C- 0 0.<J 0 e 12 ': .£ 2 u CO 0 « l P • ; : f ^ .Ml .228 .110 . 093 .Of Figure 7 - Set C: /3min • i . The performance in terms of EMSE is similar to Set A except that the EMSE of OLS i s a b i t smaller than in Set A for a l l o. i i . For a near . 3 , ORIDGM performs worse than OLS but improves s i g n i f i c a n t l y as a increases or decreases. 4 3 02: CT U_i —j i _ J 1 or- ! 0R1DO-J CSTbj--0 0 0 ^ .992 .92 0 6 J .731 'i57E ] c .27i to H _ J c -1 * \ ± 1 U J 1 w GLS V . t amen. +. (BEST — 4- iJG-Sj-— -_—- . -+- — ^ 4 QrlOK = i i i i i—r C O 0 . 4 G I I ! I 5 ] bi"l Figure 8 - Set C: 02 i . The results in Set C resemble 0max whereas in Set A they resemble 0 m i n . This can be expected considering the percentage of v a r i a b i l i t y in the X's explained by 02 i s close to that of 0min in Set A and closer to that of 0max in Set C. 44 6.4.4 Set D X's S e r i a l l y Correlated, 7=.8; \p2=.9B j3max: Figure 9 - Set D: 0max i . The EMSE i s larger for each estimation method than when there i s no s e r i a l c o r r e l a t i o n . For the ordinary ridge methods, t h i s i s as predicted in Theorem 1.2. i i . Neither OHKB nor ORIDGM make s i g n i f i c a n t improvements over OLS in estimating /3 for any o. i i i . GLS, GHKB, GRIDGM markedly reduce the EMSE compared to OLS. However, the amount of improvement is somewhat less for small a. i v . OBEST and GBEST perform s i m i l a r l y to Set A for a l l a. v. The average k values are smaller for OBEST, OHKB and ORIDGM; l i t t l e changed for GBEST and GHKB; and higher for GRIDGM compared to when there i s no s e r i a l 45 c o r r e l a t i o n . (Smin: V UI -j i o L>:1 L T = I ~ZL -fJ2r__ _ \ • V . "Iis"ic7" 1 I I I C 0 n i i—i—i—r O . i C 6 i i i i i i i r 1 2 1 5 F i g u r e 1 0 - S e t D : / 3 m i n i . T h e E M S E i s l a r g e r f o r e a c h e s t i m a t i o n m e t h o d t h a n w h e n t h e r e i s n o s e r i a l c o r r e l a t i o n . i i . F o r s m a l l o, O H K B a n d O R I D G M d o n o t p e r f o r m a n y b e t t e r t h a n O L S . A s a i n c r e a s e s , t h e i r r e s p e c t i v e E M S E i m p r o v e r e l a t i v e t o O L S . H o w e v e r , t h e i m p r o v e m e n t i s n o t a s g r e a t a s w h e n t h e r e i s n o s e r i a l c o r r e l a t i o n . i i i . B o t h G H K B a n d G R I D G M h a v e m u c h s m a l l e r E M S E t h a n O L S f o r a l l a w h e r e a s t h e y d o n o t f o r s m a l l a w h e n t h e r e i s n o s e r i a l c o r r e l a t i o n . i v . F o r O R I D G M , G R I D G M , O B E S T , a n d G B E S T t h e r e a r e m a n y k v a l u e s g r e a t e r t h a n 9.9999. A s a r e s u l t n e g a t i v e c o n c l u s i o n s c o n c e r n i n g t h e s e m e t h o d s a r e d i f f i c u l t t o 46 make. However, i t i s worth n o t i n g i s t h a t GBEST pe r f o r m s as w e l l , i f not b e t t e r than OBEST f o r a l l a. I t does not p e r f o r m as w e l l as OBEST f o r a > .7071 when t h e r e i s no s e r i a l c o r r e l a t i o n , v. The average k v a l u e s f o r OHKB are s m a l l e r when the X's a r e s e r i a l l y c o r r e l a t e d . They a r e l a r g e r f o r GHKB exc e p t when a = 2.0. 6.4.5 GHW The GHW e s t i m a t e of k was c a l c u l a t e d f o r o n l y t h r e e of the /3 v e c t o r ' and X m a t r i x c o m b i n a t i o n s . F i g u r e s 11, 12, and 13 d u p l i c a t e the f i g u r e s 1, 9, and 10 r e s p e c t i v e l y e xcept t h a t GHW r e p l a c e s the BEST r e s u l t s . T h i s a l l o w s comparison of GHW t o the o t h e r e s t i m a t o r s . F i g u r e 11 - Set A i n c l u d i n g GHW: p'max 47 \ CUDGH 11 1 i i f i I I I i I i I I i I I I 0 0 0 4 0 6 1 2 1 .6 2 0 SIGMA ~i i i i—i—i—i—;—i—i—r~i—i i i—:—i—r 0 4 o ? 1 2 1 f GiGi'^ F i g u r e 12 - Set D i n c l u d i n g GHW: /3max -i 1 a~ 1 CO c i f x c — • GO*" " wor — - * GGHHJ */.- .* \-G1AJ. • — -t 1—i—i—i—i—i—i—i—i—rn i i i i i i r~i i 0 4 o e 1 2 1 6 2 0 / ^ - . _0 i "3 I I I l~7 I I I i I I I I I I I I I I I 0 0 0 4 O B 1 2 1 6 2 0 5 I GMfi F i g u r e 13 - Set D i n c l u d i n g GHW: |3min i . When the x's are c o r r e l a t e d with \\i2 = .98 and /3max i s used r e g a r d l e s s of whether the x's are s e r i a l l y 48 correlated (Figures 11 and 12), OGHW and GGHW are more invariant in their performance r e l a t i v e to OLS as o changes than the other estimators. For small o, OGHW performed better than OHKB and ORIDGM. GGHW also performed better than GHKB and GRIDGM for small a. On the other hand, for larger a, OGHW and GGHW did not perform as well as ORIDGM and GRIDGM respectively. One of the a t t r a c t i v e properties of GHW i s that i t does not require an estimate of o in i t s c a l c u l a t i o n . This may account for i t s consistency over varying a. i i . When 0min was used and the x's were correlated with \p2 = .98 and s e r i a l c o r r e l a t i o n of 7 = .8, GHW does not make any consistent improvements over either HKB or RIDGM. In fact, the int e r v a l estimates indicate that they are not s i g n i f i c a n t l y d i f f e r e n t from one another. i i i . The cost of simulating GHW, stopped any further runs. The cost should not deter the use of GHW. The estimator did appear to be more invariant to the estimate of 0 and therefore i f the estimate of sigma i s in doubt, such as in small samples, GHW may be preferred to HKB or RIDGM. 6.4.6 K Values i . As 0 increases, the variance of the k's for the RIDGM and BEST methods increases. i i . The average k values are smaller when /3max is used than when /3min i s used for the k estimation methods 4 9 R I D G M a n d H K B b u t a r e l a r g e r f o r t h e B E S T m e t h o d s . i i i . T h e k v a l u e s a r e a l l o w e d t o b e n e g a t i v e i n b o t h t h e R I D G M a n d B E S T c a l c u l a t i o n s . G B E S T a l w a y s h a s s o m e k ' s w h i c h a r e n e g a t i v e w i t h e x t r e m e s s u c h a s i n S e t A w i t h / 3 m i n a n d a = . 3 1 6 2 w h e r e 5 4 o u t o f 100 s a m p l e s h a v e a n e g a t i v e k v a l u e . O B E S T h a s o n l y s l i g h t l y f e w e r s a m p l e s w i t h n e g a t i v e k v a l u e s . N e i t h e r G R I D G M n o r O R I D G M p r o d u c e d a n y n e g a t i v e v a l u e s . T h e i m p l i c a t i o n s o f t h e s e n e g a t i v e v a l u e s w i l l b e l e f t f o r f u t u r e s t u d y . S o m e t h e o r e t i c a l r e s u l t s o n r i d g e e s t i m a t i o n - w i t h n e g a t i v e k a r e c o n s i d e r e d b y H u a a n d G u n s t ( l 9 8 2 ) . i v . L a r g e k v a l u e s o n l y o c c u r w h e n / 3 m i n i s u s e d a n d t h e n u m b e r o f o c c u r r e n c e s i n c r e a s e s a s a i n c r e a s e s . v . T h e c o n s t a n t k v a l u e s e v a l u a t e d a r e c h o s e n t o b e . 1 , . 3 , . 5 , a n d . 7 . T h e i r p e r f o r m a n c e i s c l o s e t o t h e B E S T p e r f o r m a n c e i n s o m e c a s e s b u t w o r s e t h a n O L S i n o t h e r s . A l s o , o n e p a r t i c u l a r k i s n o t a l w a y s b e t t e r t h a n t h e o t h e r s . C o n s t a n t k v a l u e s p e r f o r m b e s t w h e n t h e /3 v e c t o r i s p 'max a n d o i s l a r g e . U n f o r t u n a t e l y , n e i t h e r o f t h e s e a r e k n o w n i n r e a l a p p l i c a t i o n s s o a c o n s t a n t k r u l e c a n n o t b e p r a c t i c a l l y u s e d . 6 . 4 . 7 D i s c u s s i o n E v e r y c o n f i g u r a t i o n i n t h e s i m u l a t i o n h a d a n a u t o c o r r e l a t e d e r r o r s t r u c t u r e w i t h p = . 9 . A l l s u p p o r t e d t h e u s e o f g e n e r a l i z e d l e a s t s q u a r e s e v e n t h o u g h t h e D u r b i n m e t h o d c o n s i s t e n t l y u n d e r e s t i m a t i n g p t o b e a p p r o x i m a t e l y . 7 8 . C h a n g e s 50 i n o t h e r m o d e l p a r a m e t e r s a f f e c t e d b o t h t h e p e r f o r m a n c e o f G L S a n d t h e r i d g e e s t i m a t o r . T h e i m p r o v e m e n t m a d e b y G L S d i m i n i s h e d a s t h e a u t o c o r r e l a t i o n c o m p o n e n t o f t h e e r r o r t e r m w a s s w a m p e d b y n o i s e . T h a t i s , t h e E M S E o f G L S r e l a t i v e t o t h e E M S E o f O L S i n c r e a s e d w i t h a. A l s o , t h e k e s t i m a t o r s H K B a n d R I D G M , a r e i n c r e a s i n g f u n c t i o n s o f t h e e s t i m a t e o f a. W h e n p 'max w a s u s e d , t h e y c h o s e a k s m a l l e r t h a n t h e ' b e s t ' k . A l a r g e r e s t i m a t e o f a, d u e t o a l a r g e r a, i m p r o v e d t h e i r p e r f o r m a n c e . T h i s s u g g e s t s t h a t t h e s e e s t i m a t o r s p e r f o r m b e s t i n m o d e l s w h o s e e x p l a n a t o r y p o w e r i s l o w a n d m a y q u e s t i o n t h e i r u s e f u l n e s s . V a r y i n g t h e j3 v e c t o r h a d t h e e f f e c t o f c h a n g i n g t h e e x p l a n a t o r y p o w e r o f t h e r e g r e s s i o n e q u a t i o n . p 'max a c c o u n t s f o r m o s t o f t h e v a r i a n c e i n t h e i n d e p e n d e n t v a r i a b l e s a n d p r o v i d e s a m u c h b e t t e r f i t t i n g m o d e l t h a n / 3 m i n . A m o d e l w i t h a /3 v e c t o r s i m i l a r t o 0 m i n m a y h a v e l i m i t e d u s e a n d t h e p e r f o r m a n c e o f a n y e s t i m a t o r m a y n o t b e g o o d e n o u g h t o e n c o u r a g e u s e o f t h e m o d e l . T h e i n t r o d u c t i o n o f s e r i a l c o r r e l a t i o n i n t h e i n d e p e n d e n t v a r i a b l e s a l s o a f f e c t e d t h e p e r f o r m a n c e o f t h e r e g r e s s i o n e s t i m a t o r s . I t i n c r e a s e d t h e E M S E o f t h e O L S e s t i m a t e a n d t h e i m p r o v e m e n t m a d e b y g e n e r a l i z e d l e a s t s q u a r e s . W h i l e a d j u s t i n g f o r t h e a u t o c o r r e l a t i o n , G L S a l s o r e m o v e d m o s t o f t h e s e r i a l c o r r e l a t i o n . L o w e r i n g t h e p a i r w i s e c o r r e l a t i o n f r o m . 9 8 t o . 7 2 s h o u l d h a v e t h e e f f e c t o f d i m i n i s h i n g t h e b e n e f i t o f r i d g e e s t i m a t i o n . T h e ' b e s t ' k r e s u l t s a g r e e w i t h t h i s h y p o t h e s i s a n d t h e E M S E o f O L S d i d d r o p f r o m 4 . 8 t o . 3 6 w h e n o = 2 . H K B a n d R I D G M p e r f o r m e d b e t t e r t h a n O L S o n l y f o r l a r g e r a a n d i n t h e s e c a s e s , t h e 51 already small EMSE of OLS may not warrant the addition of bias. A f i n a l comment i s that in Sets A, B, and C, generalized least squares resulted in the transformed independent variables having negative s e r i a l c o r r e l a t i o n of approximately .4. Since there i s s t i l l small autocorrelation in the errors, the additional term in (3.1.1) has an effect on the EMSE of the generalized ridge methods. 52 VII. AN APPLICATION OF GENERALIZED RIDGE REGRESSION When a regression model contains autocorrelation and c o l l i n e a r i t y , the Monte Carlo simulation supported the claim that generalized ridge regression provides estimates of the regression c o e f f i c i e n t s superior to ordinary least squares. This chapter i l l u s t r a t e s the application of generalized ridge using time series data. Longley(1967) constructed a model so that computational problems arose from highly c o l l i n e a r independent variables. The data represents ' t y p i c a l ' econometric data with l i t t l e t h e o r e t i c a l j u s t i f i c a t i o n for a model. It consists of the t o t a l derived employment(EMPLYMNT) and seven major economic factors observed yearly for the United States from 1947 u n t i l 1962 (see Appendix A for the derivation of employment and the raw data.) Using employment as the dependent variable, combinations of the seven economic factors were considered in the search for a model with s i g n i f i c a n t c o l l i n e a r i t y and autocorrelation. The f i n a l choice was to use three independent variables: GNPDEF: Gross National Product Implicit Price Deflator (Nominal GNP / Real GNP). GNP: Gross National Product (Nominal; in current dollars) POP: Noninstitutional Population 14 Years of Age and Over. The nominal GNP i s the t o t a l price of the goods produced in a given year in current d o l l a r s . The real GNP i s calculated by multiplying the number of goods produced in a given year by their value in a base year, in t h i s case, 1954. The GNP deflator i s the r a t i o of the nominal GNP in a given year to the 53 real GNP. It i s a measure of i n f l a t i o n from-1954 to the current period. When GNP i s used in econometric modelling, the real GNP is recommended and the GNP def l a t o r i s included to account for . any in f l a t i o n a r y trend. The nominal GNP i s used here to provide a serious c o l l i n e a r i t y problem and i t i s noted that this would l i k e l y not be done in any p r a c t i c a l modelling. Combined with the n o n - i n s t i t u t i o n a l population 14 years and over, these two variables provided the most serious problems of both autocorrelation and c o l l i n e a r i t y found in any subset of the Longley data. The following c o r r e l a t i o n matrix i l l u s t r a t e s the strong pairwise relationships between the independent variables as well as with employment: GNP .9916 POP .9792 .991 1 EMPLYMNT .9709 .9836 .9604 GNPDEF GNP POP Table II - Correlation Matrix: Longley Data When GNP, GNPDEF and POP are a l l included in the regression model, GNPDEF i s not s i g n i f i c a n t . T h i s i s a clear and simple indication of a c o l l i n e a r i t y problem. Belsey, Kuh, and Welsch (pp. 85-151) deal extensively with c o l l i n e a r i t y and emphasize that the c o r r e l a t i o n matrix only indicates pairwise c o l l i n e a r i t y . They encourage the use of the singular value decomposition of the X matrix. The X matrix may be decomposed (Golub(1969)) as X = UDV where U'U = V'V = I and D i s diagonal with non-negative diagonal elements u{i), 54 i=1,2,...,p c a l l e d the singular values of X. This holds true regardless i f X has been centered or scaled. However, for the purpose of c o l l i n e a r i t y diagnostics, i t i s desirable to scale X to have equal column lengths (normalize). The condition indices, the rati o s of the largest u to each p{i}, w i l l a l l have a value of one i f the columns of X are orthogonal. Belsey, Kuh and Welsh use t h i s index to indicate near dependencies amongst variables in the X matrix. As a rough guideline, they suggest that weak dependencies are associated with condition indices around 5 or 10, whereas moderate to strong relationships are associated with condition indices of 30 to 100. The Longley data d e f i n i t e l y has strong i n t e r -relationships in the X matrix as indicated be the condition index of 142.418 as well as the secondary value of 12.550. CONDITION VARIANCE -DECOMPOSITION INDEX PROPORTIONS var 0 1 var/3 2 var/33 1 .000 .00003 .00040 .00005 12.550 .00070 .12182 .00679 142.418 .99928 .87777 .99316 Table III - Variance Decomposition: Longley Data The ' i ' , f j ' t h variance-decomposition proportion i s the proportion of the variance of the ' i ' t h regression c o e f f i c i e n t associated with the ' j ' t h component of i t s decomposition. To further support the existance of severe c o l l i n e a r i t y , observe that a l l three c o e f f i c i e n t variances are almost t o t a l l y 55 a c c o u n t e d f o r b y t h e s a m e c o m p o n e n t . I n t e r m s o f e i g e n v a l u e s ; X , = 2 . 9 8 0 9 , X 2 = 0 . 0 1 8 9 , a n d X 3 = . 0 0 0 1 . T h e c o n d i t i o n n u m b e r , t h e s q u a r e o f t h e l a r g e s t c o n d i t i o n i n d e x , i s 2 0 2 8 2 . 9 w h i c h i s a g a i n a s t r o n g i n d i c a t i o n o f c o l l i n e a r i t y . T h e D u r b i n W a t s o n t e s t i s t h e m o s t c o m m o n d i a g n o s t i c f o r i n v e s t i g a t i n g t h e p r e s e n c e o f a u t o c o r r e l a t i o n . A n a p p r o x i m a t i o n f o r t h e t e s t h a s , u n t i l r e c e n t l y , b e e n t h e o n l y w a y t h a t c o m m o n u s a g e w a s f e a s i b l e b e c a u s e t h e c r i t i c a l v a l u e d e p e n d s o n t h e p a r t i c u l a r X m a t r i x i n v o l v e d . T h e D u r b i n - W a t s o n s t a t i s t i c f o r t h e e m p l o y m e n t m o d e l , i s 1 . 1 6 9 8 w h i c h i s i n t h e ' i n c o n c l u s i v e ' r a n g e a c c o r d i n g t o t h e a p p r o x i m a t e t a b l e s a v a i l a b l e i n m o s t t e x t b o o k s . A n e x a c t t e s t i s a v a i l a b l e a t t h e U n i v e r s i t y o f B r i t i s h C o l u m b i a i n t h e S H A Z A M c o m p u t e r p a c k a g e b y K . W h i t e ( l 9 8 2 ) . T h e f o r m u l a t i o n o f t h e e x a c t t e s t c a n . b e f o u n d i n C h a p t e r 1 5 . 5 o f J u d g e e t a l . T h e p - v a l u e o f t h i s t e s t f o r t h e e m p l o y m e n t m o d e l i s . 0 0 8 2 7 i n d i c a t i n g t h a t t h e e r r o r s a r e a u t o c o r r e l a t e d . T h e M L E a n d C o c h r a n e - O r c u t t p r o c e d u r e s e s t i m a t e d p t o b e . 3 9 3 2 . N o t e a g a i n , t h e c o r r e c t e s t i m a t e t o u s e i s c a l c u l a t e d a s s u m i n g t h e f i r s t o b s e r v a t i o n i s d r o p p e d t o b e c o n s i s t e n t w i t h t h e g e n e r a l i z e d r i d g e p r o c e d u r e . T h e D u r b i n p r o c e d u r e g a v e p = . 6 0 3 9 w h i c h i s s u b s t a n t i a l l y h i g h e r t h a n t h e M L E e s t i m a t e . G i v e n t h e p r e v i o u s d i s c u s s i o n o f s h o r t c o m i n g s o f t h e D u r b i n p r o c e d u r e , t h e M L E e s t i m a t e i s m o s t l i k e l y s u p e r i o r . H o w e v e r , r e g r e s s i o n s u s i n g b o t h e s t i m a t e s a r e i n c l u d e d t o o b s e r v e t h e e f f e c t o f c h a n g i n g t h e a u t o c o r r e l a t i o n p a r a m e t e r o n t h e r e s u l t s 56 of generalized ridge. The s e r i a l c o r r e l a t i o n 6 of the independent variables are .7980, .7938, and .7981 for GNPDEF, GNP, and POP respectively. These high l e v e l s of s e r i a l c o r r e l a t i o n are sim i l a r to Set D in the Monte Carlo study. Referring to the po s i t i v e results when generalized ridge was used on Set D, the potential for improvement over OLS using the employment model should be greater than i f the independent variables were not s e r i a l l y correlated. The results of the regressions using 'no ridge', HKB, RIDGM, GHW, and the ridge trace for both ordinary and generalized least squares using the MLE estimate for p follow: OLS HKB RIDGM GHW TRACE k-value R2 St.Error .0000 .9824 521 .2 .00068 .9819 528.0 .00079 .9818 530.0 .00194 .9799 556.3 .070 .9560 823 .6 Ord.Beta: Constant GNPDEF • GNP POP 102031 . -148.2 .082 -.456 93469. -107.2 .073 -.389 92301 . -101.7 .072 -.379 82318. -55.3 .061 -.299 37030. 101.9 .016 .101 St.Beta GNPDEF GNP POP -.455 2.331 -.904 -.329 2.072 -.770 -.312 2.036 -.751 -.170 1 .737 -.593 .313 .444 .200 Table IV - Regression Results without GLS: Longley Data 6 S e r i a l c o r r e l a t i o n of x = (COV(x{t},x{t-1})/VAR(x{t}) 5 7 p = . 3 9 3 2 G L S H K B R I D G M G H W T R A C E k - v a l u e R 2 S t . E r r o r . 0 0 0 0 . 9 6 2 6 7 8 9 . 0 . 0 0 1 6 8 . 9 6 1 8 7 9 7 . 3 . 0 0 1 8 5 . 9 6 1 7 7 9 8 . 9 . 0 0 1 9 4 . 9 6 1 6 7 9 9 . 7 . 2 0 0 . 8 7 4 0 1 4 4 8 . 3 O r d . B e t a : C o n s t a n t G N P D E F G N P P O P 1 0 5 9 9 5 . - 1 0 1 . 1 . 0 8 4 - . 5 3 6 9 9 4 1 9 . - 7 6 . 1 . 0 7 7 - . 4 8 0 9 8 8 1 3 . - 7 3 . 8 . 0 7 7 - . 4 7 5 9 8 5 0 0 . - 7 2 . 9 . 0 7 6 - . 4 7 3 3 8 5 9 9 . 1 0 0 . 9 4 . 0 1 7 . 0 8 5 S t . B e t a : G N P D E F G N P P O P - . 2 7 4 2 . 2 2 6 - 1 . 0 2 6 - . 2 0 6 2 . 0 5 1 -.919 - . 2 0 0 2 . 0 3 5 - . 9 0 9 - . 1 9 7 2 . 0 2 7 - . 9 0 4 . 2 7 4 . 4 4 0 . 1 6 3 T a b l e V - G e n . R i d g e R e s u l t s u s i n g M L E p : L o n g l e y D a t a W h e n O L S i s u s e d f o r a m o d e l w i t h a u t o c o r r e l a t e d e r r o r s t h e s t a n d a r d e r r o r o f t h e e s t i m a t e i s u n d e r e s t i m a t e d a n d o f t e n l e a d s t o t h e i n c o r r e c t a c c e p t a n c e o f t h e r e g r e s s i o n m o d e l . I n t h i s e x a m p l e , t h e i n c r e a s e i n t h e s t a n d a r d e r r o r w h e n G L S i s u s e d i s n o t e n o u g h t o q u e s t i o n t h e s i g n i f i c a n c e o f t h e m o d e l . N i n e t y -s i x p e r c e n t o f t h e v a r i a n c e o f E M P L Y M N T i s s t i l l e x p l a i n e d b y t h e i n d e p e n d e n t v a r i a b l e s . T h e p r e v i o u s l y d i a g n o s e d a u t o c o r r e l a t i o n i n d i c a t e s t h a t t h e O L S e s t i m a t e s a r e n o t a p p r o p r i a t e . H o w e v e r , t h e y a r e i n c l u d e d h e r e t o a l l o w f o r c o m p a r i s o n w i t h t h e G L S e s t i m a t e s . T h e a n a l y t i c a l ' k ' e s t i m a t i o n p r o c e d u r e s a l l e s t i m a t e d k t o b e v e r y c l o s e t o z e r o i n b o t h o r d i n a r y a n d g e n e r a l i z e d r i d g e . I n f a c t , t h e c h a n g e i n t h e r e g r e s s i o n e s t i m a t e s w e r e n e g l i g i b l e . A p o s s i b l e e x p l a n a t i o n f o r t h e p o o r p e r f o r m a n c e o f H K B a n d R I D G M i s t h e e x c e l l e n t f i t o f t h e m o d e l i n d i c a t e d b y t h e R 2 f o r G L S o f . 9 6 2 6 . T h e e s t i m a t e o f t h e v a r i a n c e u s e d i n c a l c u l a t i n g H K B a n d 58 R I D G M w o u l d b e v e r y s m a l l l e a d i n g t o s m a l l e s t i m a t e s o f k . R e g a r d l e s s , H K B , R I D G M , a n d G H W h a v e p e r f o r m e d w e l l i n s i m u l a t i o n s b u t n o t i n r e a l a p p l i c a t i o n s . T h i s i s w h y t h e r e i s c o n t i n u i n g e f f o r t t o s o l v e t h e p r o b l e m o f a s a t i s f a c t o r y a n a l y t i c a l m e t h o d f o r f i n d i n g t h e r i d g e p a r a m e t e r . T h e r i d g e t r a c e r e m a i n s t h e m o s t p o p u l a r m e t h o d f o r t h e a p p l i c a t i o n o f r i d g e r e g r e s s i o n . I t i s i n s t r u c t i v e t o o b s e r v e t h e r i d g e t r a c e u s i n g o r d i n a r y r i d g e a s w e l l a s g e n e r a l i z e d r i d g e t o s e e t h e e f f e c t o f p o n t h e c h o i c e o f t h e r i d g e p a r a m e t e r a n d t h e v a l u e s o f t h e r e g r e s s i o n e s t i m a t e s . o • J O o ' — o " U J ORDINRRY RIDGE TRPCE LONGLEY D^Q .uN^EF. -1 1 1 1 1 1 1 1 1 1 1 1—1 I 1 1 1 1 1 ! 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.6 0.9 1.0 K VALUE F i g u r e 1 4 - O r d i n a r y R i d g e T r a c e : L o n g l e y D a t a O n e o f t h e i n d i c a t i o n s o f c o l l i n e a r i t y i s a n a p p a r e n t l y i n c o r r e c t s i g n o f o n e o r m o r e o f t h e c o e f f i c i e n t e s t i m a t e s . O n e m i g h t h y p o t h e s i z e t h a t G N P a n d P O P s h o u l d h a v e p o s i t i v e r e l a t i o n s h i p s w i t h e m p l o y m e n t b e c a u s e m o r e e m p l o y e e s a r e u s u a l l y n e c e s s a r y a s p r o d u c t i o n a n d d e m a n d s o n p r o d u c t i o n i n c r e a s e s . T h e G N P d e f l a t o r i s a n i n d i c a t o r o f i n f l a t i o n a n d r e c a l l i n g t h e 59 LU CD'M ' GENERALIZED RIDGE TRACE LONGLEY DflTfi \CNP .CNPOEf-PtlP "> 1 1 1 i I I i 1 I — l — l — l — i — i — i — i — i — i — i 00 0.3 0.2 0.3 0.4 0 5 0.6 0 7 0 6 0 9 10 K VALUE Figure 15 - Gen. Ridge Trace using MLE p: Longley Data 'unemployment r a t e - i n f l a t i o n rate' trade-off, as the i n f l a t i o n rate increases the employment rate should also increase. Since POP i s increasing, an increase in the employment rate.means an increase in EMPLYMNT and therefore, a p o s i t i v e relationship between EMPLYMNT and GNPDEF would be expected. The OLS and GLS c o e f f i c i e n t estimates for GNPDEF 'and POP are negative. K values as low as .02 and .07 for 'ordinary' ridge and generalized ridge respectively provide positive estimates for a l l three c o e f f i c i e n t s . This almost immediate change to the hypothesized 'correct' sign supports the use of the ridge estimation technique. T h e • d i f f i c u l t task now i s to select the k where the estimates appear, to s t a b i l i z e . At f i r s t glance, i t i s obvious that the generalized ridge estimate requires a higher value of k before the system s t a b i l i z e s than ordinary least squares. As k increases the amount of bias introduced also increases therefore i t is desirable to select as small a value as possible. On the 60 o t h e r h a n d , i f t h e k i s c h o s e n a s s o o n a s a l l t h e e s t i m a t e s h a v e t h e c o r r e c t s i g n , a t l e a s t o n e o f t h e e s t i m a t e s w i l l n o t b e s i g n i f i c a n t l y d i f f e r e n t f r o m z e r o . W i t h t h e s e c o m m e n t s i n m i n d , a k v a l u e o f . 0 7 w a s s e l e c t e d f o r ' o r d i n a r y ' r i d g e a n d . 2 0 f o r g e n e r a l i z e d r i d g e . A s t h e l e v e l o f c o l l i n e a r i t y i n a r e g r e s s i o n m o d e l i n c r e a s e s , t h e i n i t i a l r a t e o f c h a n g e o f t h e c o e f f i c i e n t s i n t h e r i d g e t r a c e a l s o i n c r e a s e s . O b s e r v i n g t h e r i d g e t r a c e s w i t h a n d w i t h o u t t h e t r a n s f o r m a t i o n f o r a u t o c o r r e l a t i o n s u g g e s t s t h a t t h e c o l l i n e a r i t y i n t h e e m p l o y m e n t m o d e l d e c r e a s e d a f t e r t h e t r a n s f o r m a t i o n . T h i s i s v e r i f i e d b y a d r o p i n t h e l a r g e s t c o n d i t i o n i n d e x f r o m 1 4 2 . 4 t o 1 0 6 . 1 . C o l l i n e a r i t y i s s t i l l a p r o b l e m b u t i s l e s s s e v e r e . T h i s s u g g e s t s t h a t i n s o m e s i t u a t i o n s , t h e a d j u s t m e n t f o r a u t o c o r r e l a t i o n m a y c o r r e c t t h e c o l l i n e a r i t y p r o b l e m t o t h e p o i n t w h e r e r i d g e r e g r e s s i o n i s n o l o n g e r r e q u i r e d . O n e j u s t i f i c a t i o n f o r t h e d e c r e a s e i n c o l l i n e a r i t y c o u l d b e t h a t t h e s e r i a l c o r r e l a t i o n s o f t h e . i n d e p e n d e n t v a r i a b l e s i s a s o u r c e o f c o l l i n e a r i t y a n d a r e l e s s e n e d w h e n t h e a u t o c o r r e l a t i o n i n t h e e r r o r s i s r e m o v e d . U n f o r t u n a t e l y , a f t e r b e i n g t r a n s f o r m e d b y t h e G L S m e t h o d , t h e s e r i a l c o r r e l a t i o n s o f . 8 1 1 1 , . 7 5 4 7 , a n d . 7 8 3 1 f o r G N P D E F , G N P , a n d P O P r e s p e c t i v e l y d o n o t s u p p o r t t h i s s u g g e s t i o n . T h e y m a y b e d u e t o t h e r e l a t i v e l y s m a l l p o f . 3 9 3 2 a n d t h e n u m b e r o f o b s e r v a t i o n s d e c r e a s i n g b y o n e . T h e r i d g e t r a c e o f t h e g e n e r a l i z e d l e a s t s q u a r e s e s t i m a t e s u s i n g t h e D u r b i n e s t i m a t e f o r p i l l u s t r a t e t h a t a s t h e e s t i m a t e 61 o f p i n c r e a s e s , t h e c o l l i n e a r i t y p r o b l e m d e c r e a s e s . T h e s u m m a r y o f t h e r e g r e s s i o n r e s u l t s a n d r i d g e t r a c e f o l l o w : p = . 6 0 3 9 G L S H K B R I D G M G H W T R A C E k - v a l u e R 2 S t . E r r o r . 0 0 0 0 . 9 3 0 7 1 2 4 2 . 8 . 0 0 3 6 7 . 9 2 9 4 1 2 5 4 . 4 . 0 0 3 9 9 . 9 2 9 2 1 2 5 6 . 4 . 0 0 1 9 4 . 9 3 0 3 1 2 4 6 . 3 . 3 0 0 . 7 6 3 2 2 2 9 7 . 5 O r d . B e t a : C o n s t a n t G N P D E F G N P P O P 1 0 6 9 7 1 . - 1 1 0 . 1 . 0 8 5 - . 5 4 3 1 0 0 9 1 0 . - 8 9 . 1 . 0 8 0 - . 4 9 0 1 0 0 4 1 9 . - 8 7 . 4 . 0 7 9 - . 4 8 6 1 0 3 6 7 4 . - 9 8 . 5 . 0 8 2 - . 5 1 4 4 2 1 6 8 . 8 9 . 0 0 . 0 2 0 . 0 5 3 S t . B e t a : G N P D E F G N P P O P - . 2 6 0 2 . 0 4 8 - . 9 4 2 - . 2 1 1 1 . 9 0 8 - . 8 5 0 - . 2 0 7 1 . 8 9 6 - . 8 4 3 - . 2 3 3 1 . 9 7 1 - . 8 9 2 . 2 1 0 . 4 8 0 . 0 9 3 T a b l e V I - G e n . R i d g e R e s u l t s u s i n g D u r b i n p : L o n g l e y D a t a LU' Li_ . o<=>-(_> <x™ 1—o' LU o. I D ' >GND • .GNPD'EF POP GENERALIZED RIDGE TRC LONGLEY' DATA —1 1 1 1 1 1 1 1 1 1 1 1 1 T 1 1 1 1 I 1 0 0 1 0 2 0 3 0.4 0.5 0.6 0.7 0.6 0.9 1.0 K VALUE F i g u r e 1 6 - G e n . R i d g e T r a c e u s i n g D u r b i n p : L o n g l e y D a t a T h e r i d g e t r a c e i s m u c h m o r e g r a d u a l w h i c h a g r e e s w i t h t h e d e c r e a s e i n c o l l i n e a r i t y i n d i c a t e d b y t h e l a r g e s t c o n d i t i o n 6 2 i n d e x o f 6 9 . 4 . T h e s e r i a l c o r r e l a t i o n s , o f . 7 5 6 7 , . 6 9 0 4 , a n d . 7 8 0 7 f o r G N P D E F , G N P , a n d P O P r e s p e c t i v e l y d o d e c r e a s e m o r e t h a n w h e n t h e M L E p i s u s e d . T h i s m a y g i v e s o m e s u p p o r t f o r a r e l a t i o n s h i p . b e t w e e n c o l l i n e a r i t y a n d t h e s e r i a l c o r r e l a t i o n i n t h e i n d e p e n d e n t v a r i a b l e s . T h e g e n e r a l i z e d r i d g e e s t i m a t e u s i n g t h e D u r b i n p r e q u i r e s a h i g h e r k v a l u e b e f o r e i t s t a b i l i z e s w h i c h m e a n s a h i g h e r l e v e l o f b i a s . A l s o t h e s t a n d a r d e r r o r o f t h e e s t i m a t e i s o n e a n d a h a l f t i m e s g r e a t e r t h a n w h e n g e n e r a l i z e d r i d g e i s u s e d w i t h t h e M L E p . T h e s e f a c t o r s f u r t h e r s u p p o r t t h e M L E e s t i m a t e o f p i n p r e f e r e n c e t o t h e D u r b i n e s t i m a t e , C o n c l u d i n g t h e a n a l y s i s , t h e r e g r e s s i o n e q u a t i o n c h o s e n t o b e s t e s e m p l o y m e n t m o d e l u s e s t h e g e n e r a l i z e d r i d g e e s t i m a t e w i t h t h e M L E p = . 3 9 3 2 a n d t h e k v a l u e o f . 2 o b t a i n e d f r o m t h e r i d g e , t r a c e . A s a f i n a l v e r i f i c a t i o n o f t h e e q u a t i o n , t h e r e s i d u a l p l o t s w e r e f o u n d t o b e s a t i s f a c t o r y . P l o t s o f t h e r e s i d u a l s v e r s u s t i m e f o r O L S a n d G R ( p = . 3 9 3 2 ) a r e f o u n d i n A p p e n d i x B . T h e r e g r e s s i o n e q u a t i o n s f o l l o w s : EMPLYMNT = 3 8 5 9 9 . + 1 0 0 . 9 4 ( G N P D E F ) + .017(GNP) + . 0 8 5 ( P O P ) 63 V I I I . C O N C L U S I O N A M o n t e C a r l o s i m u l a t i o n w a s p e r f o r m e d t o v e r i f y t h e u s e f u l n e s s o f a g e n e r a l i z e d r i d g e p r o c e d u r e p r o p o s e d b y H s u a s a m e t h o d f o r o b t a i n i n g a s m a l l e r m e a n s q u a r e d e r r o r t h a n t h e u s u a l r i d g e a n d g e n e r a l i z e d l e a s t s q u a r e s p r o c e d u r e s i n t h e p r e s e n c e o f b o t h c o l l i n e a r i t y a n d a u t o c o r r e l a t i o n . I n m o s t s e t t i n g s t h e g e n e r a l i z e d r i d g e e s t i m a t o r p r o d u c e s . e s t i m a t e s w i t h s i g n i f i c a n t l y s m a l l e r E M S E t h a n t h e u s u a l O L S , G L S a n d r i d g e e s t i m a t o r s . I n t h e f e w p r o b l e m d e s i g n s w h e r e t h e g e n e r a l i z e d r i d g e e s t i m a t e s w e r e ' n o t s u p e r i o r , t h e 0 v e c t o r w a s i n t h e d i r e c t i o n a c c o u n t i n g f o r t h e l e a s t v a r i a n c e o f t h e i n d e p e n d e n t v a r i a b l e s , r e s u l t i n g i n a l o w R 2 . H e r e g o o d p e r f o r m a n c e w o u l d n o t b e e x p e c t e d f r o m a n y e s t i m a t o r . I n t i m e s e r i e s r e g r e s s i o n p r o b l e m s , s e r i a l l y c o r r e l a t e d i n d e p e n d e n t v a r i a b l e s a r e c o m m o n . I f o n e o f t h e s e v a r i a b l e s i s o m i t t e d f r o m t h e m o d e l , i t s s e r i a l c o r r e l a t i o n b e c o m e s ' p a r t ' o f t h e e r r o r t e r m c a u s i n g t h e e r r o r s t o b e a u t o c o r r e l a t e d . I n t h i s c o m m o n s i t u a t i o n ( s e r i a l l y c o r r e l a t e d i n d e p e n d e n t v a r i a b l e s a n d a u t o c o r r e l a t e d e r r o r s ) t h e s i m u l a t i o n s h o w e d t h e i m p r o v e m e n t m a d e b y g e n e r a l i z e d r i d g e o v e r o r d i n a r y r i d g e w a s m o s t s i g n i f i c a n t . I n s u c h s i t u a t i o n s t h e u s e o f g e n e r a l i z e d r i d g e i s r e c o m m e n d e d . T h e m e t h o d s o f D e m p s t e r , S c h a t z o f f , a n d W e r m u t h ( 1 9 7 7 ) , R I D G M ; a n d H o e r l , K e n n a r d , a n d B a l d w i n ( 1 9 7 5 ) , H K B ; h a v e p e r f o r m e d w e l l i n t h i s s e t t i n g , i n a g r e e m e n t w i t h r e s u l t s o f p r e v i o u s s t u d i e s . R I D G M u s u a l l y p e r f o r m e d b e t t e r t h a n H K B . F u r t h e r , t h e m e t h o d s w e r e f o u n d t o p e r f o r m b e s t w h e n u s e d i n 64 c o n j u n c t i o n w i t h t h e g e n e r a l i z e d r i d g e p r o c e d u r e s . I n t h e t h r e e s i t u a t i o n s w h e r e t h e m e t h o d o f G o l u b , H e a t h , a n d W a h b a , G H W , w a s u s e d , i t p e r f o r m e d a s w e l l a s b o t h H K B a n d R I D G M . T h e s i m u l a t i o n i n d i c a t e d t h a t G H W w a s m o s t i n v a r i a n t t o c h a n g e s i n a . T h e r e f o r e , w h e n t h e e s t i m a t e o f a i s i n q u e s t i o n , G H W m a y b e p r e f e r r e d t o H K B a n d R I D G M . T h e L o n g l e y d a t a p r o v i d e d f u r t h e r s u p p o r t f o r g e n e r a l i z e d r i d g e . T h e D u r b i n m e t h o d f o r s e l e c t i n g t h e a u t o c o r r e l a t i o n p a r a m e t e r i s n o t s u g g e s t e d b e c a u s e i t s e s t i m a t e o f p w a s d i f f e r e n t t h a n t h e M L E a n d C o c h r a n e - O r c u t t e s t i m a t e a n d r e q u i r e d a m o r e b i a s e d r i d g e r e g r e s s i o n e s t i m a t e . T h i s a p p l i c a t i o n d i d n o t f i n d t h e a n a l y t i c a l m e t h o d s o f c h o o s i n g t h e r i d g e p a r a m e t e r t o b e u s e f u l , w h i c h c o u l d b e d u e , i n p a r t , t o t h e e x c e l l e n t f i t o f t h e m o d e l . T h e H K B , R I D G M , a n d G H W v a l u e s f o r k w e r e e s s e n t i a l l y z e r o a n d d i d n o t p r o v i d e r e g r e s s i o n e s t i m a t e s s i g n i f i c a n t l y d i f f e r e n t f r o m O L S o r G L S . T h e r i d g e t r a c e w a s u s e d f o r s e l e c t i n g t h e r i d g e p a r a m e t e r . T h e r i d g e t r a c e s f o r d i f f e r i n g v a l u e s o f t h e a u t o c o r r e l a t i o n p a r a m e t e r i n d i c a t e d t h a t t h e l e v e l o f c o l l i n e a r i t y a f t e r t h e t r a n s f o r m a t i o n f o r a u t o c o r r e l a t i o n i s a d e c r e a s i n g f u n c t i o n o f p . 65 APPENDIX A - THE LONGLEY DATA. GNP Noninst. Total Implicit Gross Unem- Size of Popula- Derived Price Nat ional ploy- Armed t ion Year Employ-Deflator Product ment Forces (14yrs+) ment 1954=100 GNPDEF GNP not not POP not EMPLYMNT used used used 83.0 234289 2356 1590 107608 1 947 60323 88.5 259426 2325 1456 108632 1948 61 1 22 88.2 258054 3682 1616 109773 1949 60171 89.5 284599 3351 1650 110929 1950 61 187 96.2 328975 2099 3099 112075 1951 63221 98. 1 346999 1932 3594 113270 1952 63639 99.0 365385 1870 3547 115094 1953 64989 100.0 363112 3578 3350 116219 1954 63761 101.2 397469 2904 3048 • 117388 1955 66019 104.6 419180 2822 2857 118734 1956 67857 108.4 442769 2936 2798 120445 1957 68169 110.8 444546 4681 2637 121950 1958 66513 112.6 482704 381 3 2552 123366 1959 68655 114.2 502601 3931 2514 125368 1960 69564 1 15.7 518173 4806 2572 127852 1 961 69331 1 16.9 554894 4007 2827 130081 1962 70551 Source of Data: For a complete l i s t i n g of the Implicit Price Deflators for Gross National Product see Council of Economic Advisors, ECONOMIC REPORT OF THE PRESIDENT, January 1964, Table C-6,p.214. For Gross National Product (in current d o l l a r s ) see U.S. Department of Commerce, o f f i c e of Business Economics, "Survey of Current Business", July 1963, p.12. For Unemployment, Size of Armed Forces, and Noninstitutional Population see "Employment and Earnings" dated September 1963, Vol. 10, No. 3, Table A-1, p. 1 . Total Derived Employment i s the sum of Census A g r i c u l t u r a l Employment, Self-employeds, Unpaid Family Workers, and Domestics plus Bureau of Labour Non-agricultural Private Number of Jobs, Federal Government, and State and Local Government. For s e l f -employeds, unpaid family workers, and domestics see United States Department of Labor, "Manpower Report of the President and a Report on Manpower Requirements, Resources, U t i l i z a t i o n , and Training," March 1963, Table A-4, p. 141. For A g r i c u l t u r a l employment see United States Department of Labor, "Employment and Earnings," Vol. 10, No. 3, September 1963. For Bureau of Labor S t a t i s t i c s employment see Table B-1, p. 13, of the same issue of "Employment and Earnings." 66 APPENDIX B - RESIDUAL VS. TIME PLOTS: LONGLEY DATA O 993 31 or. tSi *O0 99 O © o 3 0 « IT © o © © © 1 OOOO 4 . 3 3 3 3 7 6667 " OOO «* 333 3 6 6 « 7 C OOOO 9 3333 '3 661 t « OOO © © TIME © © I OOOO . 3333 7 m l 11 OOO 14 333 J « M 7 « OOOO 9 3333 13 «*7 « 0 0 0 TIME 67 BIBLIOGRAPHY 1. A v r i e l , M., Nonlinear Programming, Analysis and Methods, Prentice-Hall, Inc., Englewood C l i f f s , N.J., 1976. 2. Beach, C. , MacKinnon, J . , "A Maximum Likelihood Procedure for Regression and Autocorrelated Errors", Econometrica , 46: 51-58, 1978. 3. Belsey, P., Kuh, E., and Welsch, R., Regression  Diagnostics , John Wiley and Sons., New York, 1980. 4. Cochrane, D., Orcutt, G.,."The Application of Least Squares Regression to Relationships Containing Autocorrelated Error Terms", Journal of the American  S t a t i s t i c a l Association , 44, 32-61, 1949. 5. Dempster, A., Schatzoff, M. and Wermuth, N., " A Simulation Study of Alternatives to Ordinary Least Squares, Journal of the American S t a t i s t i c a l Association. , 72, 77-106, 1977. 6. Durbin, J., "Estimation of Parameters in Timeseries Regression Models", Journal of the Royal S t a t i s t i c a l  Soc iety Series B, 22, 39-53, 1960. 7. Gibbons, D.G., "A Simulation Study of Some Ridge Estimators," Journal of the American S t a t i s t i c a l  Association , 76, 131-139, 1981. 8. Golub, G., "Matrix Decompositions and S t a t i s t i c a l Calculations", S t a t i s t i c a l Computation , Academic Press, New York, pp.365-397, 1969. 9. Gosling, B., "A Monte Carlo Computer Program for the Analysis of Regression Estimators for Data with C o l l i n e a r i t y and Autocorrelation Problems.", An Unpublished Computer Package, The University of B r i t i s h Columbia, 1983. 10. Gosling, B.,Hsu, J . , Puterman, M., "Ridge Estimation in Regression Problems With Autocorrelated Errors", University of B r i t i s h Columbia Working Paper, 1982. 11. Golub, G., Heath, M.,Wahba, G., "Generalized Cross-v a l i d a t i o n as a Method for Choosing a Good Rdge Parameter", Technometries 21, 215-223, 1979. 12. G r i l i c h e s , Z., and Rao, P., "Small Sample Properties of Several Two Stage Regression Methods in the Context of Autocorrelated Disturbances," Journal of the American  S t a t i s t i c a l A s s o c i a t i o n s , 64, 253-272, 1969. 68 13. Hoerl, A., and Kennard, R., "Ridge Regression, Biased Estimation for NonOrthogonal Problems", Technometrics , 12, 56-67,1970. 14. Hsu, J . , " M u l t i c o l l i n e a r i t y , Autocorrelation and Ridge Regression", Unpublished M.Sc. Thesis, Faculty of Commerce and Business Administration, University of B r i t i s h Columbia, 1980. 15. Hua, T., and Gunst, R. "Generalized Ridge Regression: A Note on Negative Ridge Parameters", Working Paper, Department of S t a t i s t i c s , Southern Methodist University, 1982. 16. Johnston, J . , Econometrics , McGraw-Hill, New York, N.Y., 1972. 17. Judge, J . , G r i f f i t h s , W., H i l l , R., Lee, T., Lutkepohl, H., Introduction to the Theory and Practice of  Econometrics. , Wiley, 1982. 18. Knuth, D., The Art of Computer Programming; Vol.2 - Semi- Numerical Algorithms, Addison Wesley, Reading, Ma., 1973. 19. Longley,J., "An Appraisal of Least Squares Programs for the E l e c t r o n i c Computer From the Point of View of the User", Journal of the American S t a t i s t i c a l Association, 62, 819-840, 1967. 20. McDonald, G., "Ridge Estimators as Constrained Generalized Least Squares," Proceedings of the Purdue Symposium on S t a t i s t i c a l Decision Theory, to appear, 1981. 21. McDonald, G. and Galarneau, D., "A Monte Carlo Evaluation of Some Ridge-Type Estimators", Journal of the American  S t a t i s t i c a l Association ,70, 407-416, 1975. 22. Margolis, M., "Perpendicular Projections and Elementary S t a t i s t i c s " , The American S t a t i s t i c i a n , 33, 131-135, 1979. 23. Prais, S. and Winsten, C , "Trend Estimators and S e r i a l C o r r e l a t i o n " , Unpublished Cowles Commission Study Paper, Stat. No. 383, Chicago, 1954. 24. Thisted, R., "Ridge Regression, Minimax Estimation and Empirical Bayes Methods," Ph.D. Thesis, Department of S t a t i s t i c s , Stanford University, 1976. 25. White, K., UBC SHAZAM ^ An Econometrics Computer Program, Version 4.5, The University of B r i t i s h Columbia Computing Center, 1982. 26. Wichern, D. and Ch u r c h i l l G., "A Comparison of Ridge 6 9 E s t i m a t o r s " , T e c h n o m e t r i e s , 2 0 , 3 0 1 - 3 1 1 , 1 9 7 8 . o 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            data-media="{[{embed.selectedMedia}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0095269/manifest

Comment

Related Items