UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Collinearity in generalized linear models Mackinnon, Murray J. 1986

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1986_A4_6 M26_5.pdf [ 3.27MB ]
Metadata
JSON: 831-1.0096659.json
JSON-LD: 831-1.0096659-ld.json
RDF/XML (Pretty): 831-1.0096659-rdf.xml
RDF/JSON: 831-1.0096659-rdf.json
Turtle: 831-1.0096659-turtle.txt
N-Triples: 831-1.0096659-rdf-ntriples.txt
Original Record: 831-1.0096659-source.json
Full Text
831-1.0096659-fulltext.txt
Citation
831-1.0096659.ris

Full Text

COLLINEARITY IN GENERALIZED LINEAR MODELS by MURRAY J MACKINNON M.Sc. U n i v e r s i t y of Otago N.Z. A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in FACULTY OF GRADUATE STUDIES Commerce and Business A d m i n i s t r a t i o n We accept t h i s t h e s i s as conforming to the r e q u i r e d standard THE UNIVERSITY OF BRITISH COLUMBIA A p r i l 1986 © Murray J Mackihnon, 1986 In p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t of the requirements f o r an advanced degree at the THE UNIVERSITY OF BRITISH COLUMBIA, I agree that the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r ref e r e n c e and study. I f u r t h e r agree that p e r m i s s i o n f o r e x t e n s i v e copying of t h i s t h e s i s f o r s c h o l a r l y purposes may be granted by the Head of my Department or by h i s or her r e p r e s e n t a t i v e s . I t i s understood that copying or p u b l i c a t i o n of t h i s t h e s i s f o r f i n a n c i a l gain s h a l l not be allowed without my w r i t t e n p e r m i s s i o n . Commerce and Business A d m i n i s t r a t i o n THE UNIVERSITY OF BRITISH COLUMBIA 2075 Wesbrook Pl a c e Vancouver, Canada V6T 1W5 Date: A p r i l 1986 A b s t r a c t The c o n c e p t o f c o l l i n e a r i t y f o r g e n e r a l i z e d l i n e a r m o d e ls i s i n t r o d u c e d and compared t o t h a t f o r s t a n d a r d l i n e a r m o d e l s . Two a p p r o a c h e s f o r d e t e c t i n g c o l l i n e a r i t y a r e p r e s e n t e d and shown t o l e a d t o t h e same d i a g n o s t i c p r o c e d u r e . T h e s e a r e a n a l y s e d f o r t h e P o i s s o n , gamma, i n v e r s e G a u s s i a n , p t h o r d e r , b i n o m i a l p r o p o r t i o n and n e g a t i v e b i n o m i a l m o d e l s . A bound i s d e r i v e d f o r t h e d e g r e e o f c o l l i n e a r i t y i n a g e n e r a l i z e d l i n e a r model i n t e r m s o f t h a t o f t h e s t a n d a r d l i n e a r m o d e l . E s t i m a t i o n methods b a s e d on r i d g e , p r i o r l i k e l i h o o d and p r i n c i p a l components a r e p r o p o s e d , and b r i e f l y i l l u s t r a t e d w i t h a Monte C a r l o s i m u l a t i o n o f a gamma model. i i i Table of Contents 0.0 Int.roduct.ion 1 1.0 Col l i n e a r i t y in Standard Linear Models 3 1.1 D e f i n i t i o n of C o l l i n e a r i t y . . . 3 1.2 Sources of C o l l i n e a r i t y 5 1.2.1 Large Pairwise Correlations 5 1.2.2 Data C o l l e c t i o n 6 1.2.3 Model S p e c i f i c a t i o n 8 1.2.4 Overdefined Model 8 1.2.5 O u t l i e r s 9 1.3 E f f e c t s of C o l l i n e a r i t y 10 1.3.1 111 conditioning of X 10 1.3.2 Estimate E f f e c t s 11 1.3.3 Inference E f f e c t s 13 1.3.4 Predictor E f f e c t s 14 2.Q C o l l i n e a r i t y l n Generalized Linear Models 16 2.1 D e f i n i t i o n of a Generalized Linear Model . . . . 16 2.2 D e f i n i t i o n of C o l l i n e a r i t y i n Generalized Linear Models 22 2.2.1 L i n e a r i s a t i o n of the Link Function. . . . 23 2.2.2 I t e r a t i v e l y Reweighted Least Squares Approach 25 2.2.3 Choice of Approach 25 2.3 Relationship of the Standard Linear Model and the Generalized Model C o l l i n e a r i t y D e f i n i t i o n s . . . 26 2.4 Sources of C o l l i n e a r i t y in a Generalized Linear Model 29 2.5 E f f e c t s of C o l l i n e a r i t y in a Generalized Linear Model 32 2.5.1 Estimation E f f e c t s 32 2.4.2 Inference and Predictor E f f e c t s 33 2.6 Appendix 2A Maximum Likelihood and the Generalized Linear Model 35 2.7 Appendix 2B I t e r a t i v e l y Reweighted Least Squares Algorithm 40 3.0 Diagnostics for C o l l i n e a r i t y in Generalized Linear Models 42 3.1 Desirable Properties for Diagnostic Measures 42 3.2 Measures of C o l l i n e a r i t y 43 3.3 Model Dependency in C o l l i n e a r Systems 46 4.0 Estimation for Generalized Linear Models in the Presence of C o l l i n e a r i t y 50 4.1 Remedies f o r C o l l i n e a r Sources 50 4.1.1 Large Pairwise Correlations i n X 51 4.1.2 Data C o l l e c t i o n 51 LEAF IV OMITTED IN PAGE NUMBERING. FEUILLET IV NON UNCLUS DANS LA PAGINATION. V 4.1.3 Model S p e c i f i c a t i o n 51 4.1.4 Overdefined Model 52 4.1.5 O u t l i e r s 52 4.2 Remedies f o r C o l l i n e a r E f f e c t s 52 4.3 Estimation Methods 53 4.3.1 Ridge Estimation 53 4.3.1.1 Standard Linear Case 54 4.3.1.2 General Linear Case 55 4.3.2 Bayesian Estimation 60 4.3.2.1 Standard Linear Case 60 4.3.2.2 General Linear Case 61 4.3.3 P r i n c i p a l Component Estimation 62 4.3.3.1 Standard Linear Case 62 4.3.3.2 General Linear Case 63 4.4 Appendix 4A Maximum Likelihood Estimates of the Posterior D i s t r i b u t i o n 65 5.0 I l l u s t r a t i v e Example 67 5.1 Introduction 67 5.2 Scope of the Simulation 67 5.3 Generation of C o l l i n e a r Data 68 5.3.1 Standard Linear Case 68 5.3.2 General Linear Case 68 5.4 Simulation 71 5.4.1 Simulation Setup 71 5.4.2 Simulation Implementation and D i f f i c u l t i e s 73 v i 5.4.3 Simulation Results and Conclusions. . . . 74 6.0 Summary and Conclusions 81 Bibliography 83 0 . 0 I n t r o d u c t i o n C o l l i n e a r i t y has l o n g been r e c o g n i s e d a s c a u s i n g s i g n i f i c a n t p r o b l e m s i n t h e c o m p u t a t i o n and e s t i m a t i o n o f t h e p a r a m e t e r s i n a s t a n d a r d l i n e a r m odel. However, t h e e f f e c t s w i t h a g e n e r a l i z e d l i n e a r model a r e r e l a t i v e l y u n e x p l o r e d , w i t h t h e e x c e p t i o n o f l o g i s t i c r e g r e s s i o n [ S C H A E F E R , 7 9 3 . I t i s t h e p u r p o s e o f t h i s t h e s i s t o s e e k a r e a s o n a b l e d e f i n i t i o n f o r c o l l i n e a r i t y i n t h e g e n e r a l i z e d l i n e a r model, t o examine i t s c o n s e q u e n c e s and t o p r o p o s e e s t i m a t i o n methods. The f i r s t c h a p t e r i d e n t i f i e s c o l l i n e a r i t y i n t h e s t a n d a r d l i n e a r model and r e v i e w s i t s s o u r c e s and e f f e c t s , s o a s t o p r o v i d e an i n t r o d u c t i o n and c o m p a r i s o n f o r t h e g e n e r a l s e t t i n g . The s e c o n d c h a p t e r s e e k s a r e a s o n a b l e d e f i n i t i o n f o r c o l l i n e a r i t y i n t h e g e n e r a l i z e d l i n e a r model by c o n s i d e r i n g two a p p r o a c h e s . The f i r s t i s b a s e d on a l i n e a r i s a t i o n o f t h e l i n k f u n c t i o n and t h e s e c o n d , w h i c h i s c h o s e n , i s m o t i v a t e d by t h e i n s t a b i l i t y o f m a t r i c e s u s e d i n t h e i t e r a t i v e l y r e w e i g h t e d l e a s t s q u a r e s e s t i m a t i o n . U s i n g t h i s d e f i n i t i o n , i t i s shown t h a t c o l l i n e a r i t y i s d e p e n d e n t on t h e model c h o s e n . The t h i r d c h a p t e r d e a l s w i t h c o l l i n e a r i t y d i a g n o s t i c s i n t h e g e n e r a l s e t t i n g . D e s i r a b l e p r o p e r t i e s a r e p r o p o s e d , and a c r i t e r i o n and i d e n t i f i c a t i o n scheme, e x t e n d i n g t h e a p p r o a c h o f i"BELSEY,KUH,WELSCH,80], i s u s e d . To q u a n t i f y t h e i n d e t e r m i n a n c y due t o model d e p e n d e n c y , weak bounds f o r t h e g e n e r a l c r i t e r i o n a r e d e v e l o p e d i n t e r m s o f t h e c r i t e r i o n f o r t h e s t a n d a r d l i n e a r model. The f o u r t h c h a p t e r i n v e s t i g a t e s e s t i m a t i o n methods. I t o p e n s by s u g g e s t i n g r e m e d i e s f o r c o l l i n e a r s o u r c e s and e f f e c t s , w h i l e c a u t i o n i n g a g a i n s t t h e r o u t i n e u s e o f " m e c h a n i c a l " methods. T h r e e methods a r e p r o p o s e d . The r i d g e a p p r o a c h i s shown t o g e n e r a l i z e , by f o l l o w i n g t h e l o g i s t i c work o f CSCHAEFER,791. The s e c o n d method i s b a s e d on t h e p r i o r l i k e l i h o o d p h i l o s o p h y o f [EDWARDS,691, w h i l e t h e t h i r d g e n e r a l i z e s t h e c l a s s i c a l p r i n c i p a l components s o l u t i o n . The f i f t h c h a p t e r b r i e f l y i l l u s t r a t e s t h e i d e a s o f t h e p r e v i o u s c h a p t e r s w i t h a r e s t r i c t e d Monte C a r l o s i m u l a t i o n o f a gamma mod e l . The p r o b l e m s and o b j e c t i v e s a s s o c i a t e d w i t h t h e c o n s t r u c t i o n o f s u c h a s i m u l a t i o n a r e d i s c u s s e d . The s i m u l a t i o n w i t h i n i t s a r t i f i c i a l s e t t i n g d e m o n s t r a t e s t h e a d v a n t a g e s o f a b i a s e d e s t i m a t o r i n t h e p r e s e n c e o f c o l l i n e a r i t y . 3 1.0 C o l l i n e a r i t y ln Standard Linear Models Throughout the t h e o r e t i c a l development of t h i s thesis a constant p a r a l l e l w i l l be drawn between the standard l i n e a r model and the generalized l i n e a r model. Let the standard l i n e a r model of a response y in terms of predictor variables X be defined as follows. D e f i n i t i o n 1.1 Standard Linear Model y = Xfl + € where y nxl vector of responses X nxp matrix of predictor variables fi pxl vector of c o e f f i c i e n t s € nxl vector of errors where € ~ Nv, (0, cr^I) 1.1 D e f i n i t i o n of C o l l i n e a r i t y C o l l i n e a r i t y , as considered in c l a s s i c a l l inear regression, i s said to be present when : two or more predictor variables are highly correlated c o e f f i c i e n t estimates are d i f f e r e n t in magnitude and/or sign from those hypothesised c o e f f i c i e n t estimates are highly variable 4 t h e s a m p l e d r a n g e o f some p r e d i c t o r s may be s m a l l m a t r i c e s u s e d i n t h e c o m p u t a t i o n o f e s t i m a t e s a r e i l l c o n d i t i o n e d B u t t h e s e a r e o n l y p o s s i b l e symptoms o r c a u s e s o f t h e more b a s i c p r o b l e m o f a p p r o x i m a t e l i n e a r d e p e n d e n c i e s amongst t h e c o l u m n s o f t h e p r e d i c t o r v a r i a b l e s . T h i s u n d e r l y i n g p r o b l e m i s t h e f o u n d a t i o n o f t h e a p p r o a c h t a k e n by [GUNST,831, and u s e d h e r e . W i t h t h i s a p p r o a c h i t i s p o s s i b l e t o t i e t o g e t h e r d i f f u s e c a u s e s s u c h a s d e f i c i e n t s a m p l i n g r a n g e and o u t l i e r s , and t o g i v e v a l i d i t y t o symptoms s u c h as h i g h l y v a r i a b l e e s t i m a t e s and s e n s i t i v e row d e l e t i o n s t a t i s t i c s . The f o l l o w i n g d e f i n i t i o n o f c o l l i n e a r i t y i s a d a p t e d f r o m [GUNST,831. D e f i n i t i o n 1.2 C o l l i n e a r i t y i n t h e S t a n d a r d L i n e a r Model y = Xfl + € i s p r e s e n t when : F o r some s u i t a b l y c h o s e n 6 i 0 t h e r e e x i s t s a p x l v e c t o r c * 0 s u c h t h a t |Xc| L 6|c| I t i s t o be n o t e d t h a t t h i s d e f i n i t i o n o n l y i n v o l v e s t h e d a t a m a t r i x X and makes no m e n t i o n o f t h e d e p e n d e n t v a r i a b l e y, o r t h e u n d e r l y i n g m odel. So f o r t h e s t a n d a r d l i n e a r model, t h e d e t e c t i o n o f c o l l i n e a r i t y i n v o l v e s o n l y t h e X m a t r i x o f p r e d i c t o r s . T h i s d e f i n i t i o n i s s u b j e c t i v e b e c u a s e o f t h e l a c k o f g u i d a n c e i n t h e s e l e c t i o n o f 6. However i t i s no d i f f e r e n t f r o m t h e o t h e r s i n t h e l i t e r a t u r e (eg.CBELSEY,KUH,WELSCH,801 r e q u i r e s u b j e c t i v e s e l e c t i o n o f t h e c o n d i t i o n i n d e x o f t h e m a t r i x X ) . The CGUNST,831 d e f i n i t i o n i s p r e f e r r e d b e c a u s e t h e i n t u i t i v e n o t i o n s o f c o l l i n e a r i t y l i s t e d a bove c a n be de d u c e d more d i r e c t l y f r o m i t . The f o l l o w i n g s e c t i o n s b a s e d on CGUNST,831 shows how t h e s e s o u r c e s and e f f e c t s c a n be e x p l a i n e d i n t e r m s o f t h e d e f i n i t i o n . 1.2 Sourcaa of C o l l i n a a r i t y C o l l i n e a r i t y c a n stem f r o m any one o f a number o f s o u r c e s , and a l l o f t h e s e c a n be c a p t u r e d by D e f i n i t i o n 1.2. F u r t h e r , i t i s o f c o n s i d e r a b l e p r a c t i c a l i n t e r e s t t o have i n f o r m a t i o n on t h e s o u r c e when t r y i n g t o a l l e v i a t e c o l l i n e a r i t y . 1.2.1 L a r g e P a i r w i s e C o r r e l a t i o n s S u p p o s e X has been s c a l e d and c e n t r e d s o t h a t X T X i s i n c o r r e l a t i o n f o r m <ie. t h e c o r r e l a t i o n c o e f f i c i e n t between X j and XK i s r J k , where x j i s t h e j t h column o f X ) . L e t X <j > be t h e m a t r i x w i t h t h e j t h column o f X d e l e t e d and fi«j> t h e 6 c o r r e s p o n d i n g d e l e t e d p r e d i c t o r . Now f r o m l i n e a r r e g r e s s i o n t h e o r y i t i s known ( e g . [MONTGOMERY,PECK,823 p430) 6 < j > — (X < j > X < j > ) X ( j ) x j L e t c be d e f i n e d a s Xc = X j - X t ) > f t < j > Now by t h e e x t r a sum o f s q u a r e s p r i n c i p l e |Xc| = (1 - R..,&-)^ where Rj' S ; i s t h e c o e f f i c i e n t o f m u l t i p l e c o r r e l a t i o n . So i f f o r some 6 i 0 <1 - R_,K >^  i 6 (1 + R R ( J > ) t h e n we have | Xc j £ 6 | c: j s o a c o l l i n e a r i t y e x i s t s i n t e r m s o f D e f i n i t i o n 1.2. S p e c i f i c s l l y i f p = 2 t h e n ft<e> = f i e and s o c i s g i v e n by Xc = x t - riftXfflj and |Xc| = (1 - r i e e > ^ So i f f o r some & > 0 we have <1 - r l e a A i 6(1 + r i a 8 ) ^ t h e n |Xc| 1 6|c| and s o a c o l l i n e a r i t y e x i s t s a s p e r D e f i n i t i o n 1.2. In o t h e r words f o r r i l s " l a r g e enough" a c o l l i n e a r i t y e x i s t s . I t i s i m p o r t a n t t o n o t e t h o u g h t h a t o n l y one d e p e n d e n c y need be s t r o n g f o r a l l t h e c o r r e l a t i o n s t o t e n d t o w a r d s u n i t y . S p e c i f i c a l l y CBELSEY,KUH,WELSCH,803 show when t h e s i n g u l a r v a l u e , c o n n e c t e d w i t h j u s t one d e p e n d e n c y , t e n d s t o z e r o a l l p a i r w i s e c o r r e l a t i o n s t e n d t o u n i t y . Hence, h i g h c o r r e l a t i o n s c a n n o t be u s e d t o d e t e r m i n e m u l t i p l e c o l l i n e a r i t i e s . 1.2.2 Data C o l l e c t i o n I f o n l y a s u b s e t o f t h e p r e d i c t o r v a r i a b l e s p a c e i s s a m p l e d , c o l l i n e a r i t y c a n a r i s e . F o r example, w i t h t h e two v a r i a b l e c a s e i n t h e s c a t t e r p l o t below, t h e d a t a i s h i g h l y p o s i t i v e l y c o r r e l a t e d and c o n s e q u e n t l y c o l l i n e a r . T h i s s o u r c e o f c o l l i n e a r i t y c a n be c a u s e d by e x t r i n s i c f a c t o r s ( e g . s a m p l i n g d e f i c i e n c y due t o c h a n c e , l a c k o f a v a i l a b l e d a t a ) o r more l i k e l y , a s i n an o b s e r v a t i o n a l s t u d y , by i n t r i n s i c f a c t o r s ( e g . one v a r i a b l e i s r e l a t e d t o a n o t h e r , o r one v a r i a b l e j o i n t l y i n f l u e n c e s two o t h e r v a r i a b l e s ) . 1.2.3 Model S p e c i f i c a t i o n O f t e n i n a p p l i e d m o d e ls t h e r e e x i s t i m p l i c i t c o n s t r a i n t s w h i c h m a n i f e s t t h e m s e l v e s i n t h e d a t a ( e g . m i x t u r e m o d e l s ) . However, t h e y need n o t be e x a c t d e p e n d e n c i e s . [BRADLEY,SRIVASTAVA,793 c o n s i d e r t h e f i t t i n g o f a low o r d e r p o l y n o m i a l i n t h e c a s e o f a s i n g l e p r e d i c t o r v a r i a b l e . They show t h a t t h e c o r r e l a t i o n c o e f f i c i e n t between t h e v a r i a b l e s , c o r r e s p o n d i n g t o any two powers, c a n be e x t r e m e l y c l o s e t o u n i t y , i f t h e d i f f e r e n c e s i n powers i s e v e n ( e v e n a l l o w i n g f o r t h e improvement due t o c e n t r i n g ) . 1.2.4 O v e r d e f i n e d Model T h i s i s a s p e c i a l c a s e o f d a t a c o l l e c t i o n , w h i c h o c c u r s when t h e r e a r e more v a r i a b l e s t h a n o b s e r v a t i o n s <ie. n < p ) . The l i n e a r d e p e n d e n c y o f t h e c o l l i n e a r i t y c a n be s e e n a s f o l l o w s . S u p p o s e t h e r a n k o f X i s r . Now r i. m i n ( n , p ) = n. By s i n g u l a r v a l u e d e c o m p o s t i o n t h e r e e x i s t U o r t h o g o n a l , V o r t h o n o r m a l and D d i a g o n a l s u c h t h a t X = UDV T B u t U, V a r e o r t h o g o n a l s o r a n k ( D ) = r a n k ( X ) = r . X = U [ D ± 1 0"|VT where D l t : r * r [o oJ X l V t V a ] = [Ut Uffi] [ D n Cf| [ p o s t m u l t i p l y by V] LO oJ IV o r t h o n o r m a l ] XV r a = O So f o r any 6 i O t h e r e e x i s t s a v e c t o r c , namely tVa-lj s u c h t h a t |Xcj = 0 i e . |Xc| £ 6 | c j . 1.2.5 O u t l i e r s C o l l i n e a r i t y c a n be i n d u c e d by o u t l i e r s <ie. o b s e r v a t i o n s w h i c h a r e a t y p i c a l i n t h e p r e d i c t o r v a r i a b l e s p a c e , a s i s shown i n t h e f o l l o w i n g argument f r o m CGUNST,83]. The l i n e a r d e p e n d e n c y c a n be i l l u s t r a t e d by c o n s i d e r i n g t h e h y p o t h e t i c a l example o f p=3 p r e d i c t o r v a r i a b l e s where X i i and x e e a r e a t y p i c a l l y l a r g e i n m a g n i t u d e . S u p p o s e k i s f i x e d , X u = 0 , X I B = k0 and X i s s c a l e d s o t h a t t h e c o l u m n s a r e o f u n i t l e n g t h . So X = 1 X! 1 1 Xoffii X i e ^ iS: iFP: 1 X r , j X jr> 1 0/X* k 0 / X e 1 X e i * / X i X a a * / X g 1 X t", j K / X j . 5C v-» in-: "** ^ ^ i! where x± i * v a r i a b l e s and i = 2 ( l ) n a r e t h e u n s e a l e d p r e d i c t o r X i c = 0™ + T X i , . * , S : i — 1 X B : e = k^G 6 8 + Z X i t * ® i •-• i C o n s i d e r c"1" = ( 0 , 1 , - 1 ) . Now IXc! = ( 0 / X i - k 0 / A e ) ' a + £ < X i i * / X i - x i e » / X a ) a Now f o r l a r g e 0, X t s : * 0,S: and Xsa** * k i s0' a :. Hence t h e r i g h t hand s i d e t e n d s t o 0. So f o r any & 1 0 t h e r e i s a v a l u e o f 0 s u c h t h a t |Xc| l 6|c|. I n o t h e r words t h e o u t l i e r , w i t h 0 s u f f i c i e n t l y l a r g e , c o r r e s p o n d s t o a c o l l i n e a r i t y a s i n D e f i n i t i o n 1.2. 1.3 E f f e c t s o f C o l l i n e a r i t y T h i s s e c t i o n i l l u s t r a t e s , i n t h e manner o f CGUNST,831, t h e i n t u i t i v e e f f e c t s o f c o l l i n e a r i t y u s i n g D e f i n i t i o n 1.2. 1.3.1 111 c o n d i t i o n i n g o f X I t h as been known f o r a l o n g t i m e t h a t c o l l i n e a r i t y i s i n t i m a t e l y r e l a t e d t o t h e c o n d i t i o n i n g o f X. [KENDALL,571, [SILVEY,691 s t a t e t h a t c o l l i n e a r i t i e s a r e a s s o c i a t e d w i t h s m a l l e i g e n v a l u e s o f X r X , w h i c h i s n u m e r i c a l l y e q u i v a l e n t t o s m a l l s i n g u l a r v a l u e s o f X . T h i s , w h e n q u a n t i f i e d m o r e p r e c i s e l y , p r o v i d e s a n a l t e r n a t i v e d e f i n i t i o n o f c o l l i n e a r i t y , w h i c h [ B E L S E Y , K U H , W E L S C H , 8 0 ] a d v o c a t e . S p e c i f i c a l l y , s u p p o s e a g a i n t h a t t h e c o l u m n s o f X a r e c e n t r e d a n d s c a l e d t o u n i t l e n g t h . T h e n b y s i n g u l a r v a l u e d e c o m p o s i t i o n X = U D V T w h e r e U = lu± . . . . . u r : > ] ! n x p o r t h o g o n a l V = t v i , . . . , v p ] : p x p e i g e n v e c t o r s D = d i a g ( d t , . . . , d F 3 ) s i n g u l a r v a l u e s S o X = Z d , u , V j T [ b i l i n e a r f o r m ] .1 — i Now c o n s i d e r c = v f X c = d j U j [V o r t h o g o n a l ] | X c | = d , | c | [ U , V o r t h o g o n a l ] S o f o r a n y 6 i 0 , i f d j £ 6 t h e n t h e r e i s a c o l l i n e a r i t y a c c o r d i n g t o D e f i n i t i o n 1 . 2 . 1 . 3 . 2 E s t i m a t e E f f e c t s T h e i n i t i a l r e a s o n f o r H o e r l a n d K e n n a r d d e v e l o p i n g t h e t e c h n i q u e o f r i d g e r e g r e s s i o n w a s t h e i r o b s e r v a t i o n t h a t t h e e s t i m a t e s o f R w e r e " t o o l o n g " i n t h e p r e s e n c e o f c o l l i n e a r i t y . T h i s o b s e r v a t i o n i s s e e n by c o n s i d e r i n g t h e norm o f t h e o r d i n a r y l e a s t s q u a r e s e s t i m a t e B . E [ B T f l J = E C ( B - B + f i ) T ( f i - B + B ) ] = B" rB + E I t r ( B - B> (fl - B ) " r ] A = B x f i + t r < V a r ( B ) ) = B T f l + t r ( ( X T X ) *)o- a = B T B + c r ^ t r (VD"- aV T) [ s i n g u l a r v a l u e d e c o m p o s t i o n ] So t h e m a g n i t u d e o f B i n a c o l l i n e a r s y s t e m i s o v e r e s t i m a t e d r e l a t i v e t o an o r t h o g o n a l d a t a m a t r i x X. F u r t h e r , t h e m a g n i t u d e i s i n v e r s e l y p r o p o r t i o n a l t o t h e s i n g u l a r v a l u e s o f X. S i m i l a r l y w i t h t h e v a r i a n c e e s t i m a t e s , s i n c e t h e c o v a r i a n c e m a t r i x i s a d i r e c t f u n c t i o n o f t h e r e c i p r o c a l s o f t h e s i n g u l a r v a l u e s , i e . A V a r ( B ) = t r f f i V D ~ c V T I n p a r t i c u l a r t h e v a r i a n c e o f B c a n be decomposed i n t o p c o m p o n e n t s . Each one i s u n i q u e l y a s s o c i a t e d w i t h j u s t one o f t h e s i n g u l a r v a l u e s a s f o l l o w s V a r ( B w > = a 6* Z -Hence i f two c o e f f i c i e n t s have h i g h p r o p o r t i o n s o f t h e i r v a r i a n c e s e x p l a i n e d by t h e same r e l a t e d s i n g u l a r v a l u e , t h e n t h e r e i s an a p p r o x i m a t e l i n e a r d e p e n d e n c y between them. T h i s l e a d s t o t h e f o l l o w i n g d e f i n i t i o n o f t h e v a r i a n c e d e c o m p o s i t i o n p r o p o r t i o n , T \J •.«-., b e i n g t h e amount o f t h e v a r i a n c e o f B K e x p l a i n e d by t h e d e p e n d e n c y a s s o c i a t e d w i t h t h e j t h s i n g u l a r v a l u e . D e f i n i t i o n 1.2 m a t r i x X:nxp Variance Decomposition Proportions of the Wju = | - J k , j = l ( l ) p V , , ( W : . P» where 0 U J = -^r^-- 0 K = T 0 K , T h e s e p r o p o r t i o n s f o r m t h e b a s i s o f t h e i d e n t i f i c a t i o n a l g o r i t h m d e v e l o p e d by [BELSEY,KUH,WELSCH,801 f o r t h e d e t e c t i o n o f m u l t i p l e c o l l i n e a r i t i e s . T h i s a l g o r i t h m i s shown t o e x t e n d t o t h e g e n e r a l c a s e i n s e c t i o n 3.2. 1.3.3 I n f e r e n c e E f f e c t s A l m o s t a l l s t a t i s t i c s a s s o c i a t e d w i t h l i n e a r r e g r e s s i o n c a n p o t e n t i a l l y be " d e g r a d e d " by t h e e f f e c t s o f c o l l i n e a r i t y . F o r i n s t a n c e , i n t e s t i n g t h e h y p o t h e s i s H.r_ flj = 0 H„ fij * 0 t h e t e s t s t a t i s t i c i s B u t r e c a l l i n g t h e r e l a t i o n s h i p between t h e c o e f f i c i e n t o f d e t e r m i n a t i o n and t h e s i n g u l a r v a l u e s , i t i s s e e n t h a t t h e t s t a t i s t i c c a n be e f f e c t e d by c o l l i n e a r i t y . In p r a c t i c e t h e t s t a t i s t i c i s o f t e n s m a l l e r i n t h e p r e s e n c e o f c o l l i n e a r i t y , b u t t h i s c a n be c o m p e n s a t e d f o r by t h e " o v e r l a r g e " m a g n i t u d e o f R. As CGUNST,83] n o t e s , no e x p l i c i t s t a t e m e n t s c a n be made a b o u t t h e m a g n i t u d e s o f t h e t s t a t i s t i c . However he n o t e s t h e non c e n t r a l i t y p a r a m e t e r o f t h e t s t a t i s t i c i s X = (1 - R , » ) ^ 5 J 2CT So f o r f i x e d B, and a, t h e s t r o n g e r t h e o f f e n d i n g c o l l i n e a r i t y i n v o l v i n g X j , t h e s m a l l e r i s t h e non c e n t r a l i t y p a r a m e t e r o f t h e t e s t s t a t i s t i c and c o n s e q u e n t l y t h e l o w e r t h e power o f t h e t e s t . As i s n o t e d by CBELSEY,KUH,WELSCH,80], t h e e f f e c t s o f c o l l i n e a r i t y i n v o l v i n g a s u b s e t o f p r e d i c t o r v a r i a b l e s need n o t e f f e c t t h e e s t i m a t e s o f t h e r e m a i n i n g c o e f f i c i e n t s , i f t h e two s u b s e t s a r e o r t h o g o n a l . T h i s i s t o be e x p e c t e d by c o m p a r i n g i t w i t h t h e i n t r o d u c t i o n o f o r t h o g o n a l v a r i a t e s i n a s t e p w i s e p r o c e d u r e . 1.3.4 P r e d i c t o r E f f e c t s The e f f e c t s o f c o l l i n e a r i t y a r e o f t e n w i d e l y d i f f e r e n t i n p r e d i c t i o n s u s i n g " i n s a m p l e " and " o u t o f s a m p l e " d a t a . I f t h e p r e d i c t o r v a r i a b l e s l i e i n t h e s u b s p a c e o f t h e o r i g i n a l data, then r e l a t i v e l y precise estimation i s possible. However "out of sample" predictions may be severely effected by c o l l i n e a r i t y . Consider for example the ordinary least squares predictions given by the expression y = x Tfi Now Var(y) = tr^Ml/n + x* T (XTX> ~ 1 x 1 > = o-'-Ml/n + X i T T A - ' T ^ 1 ) where T = (tj. t F , l eigenvectors of XTX A = (Xi Xpi) eigenvalues of X TX Now i f X i T i s a l i n e a r combination of the eigenvectors associated with the "large" eigenvalues, ie X i T = a r T T where a,. = 1 Xt "large" 0 else then Var(y) = CT'«d/n + Z ( l / X i E ) l where the summation i s over "large" eigenvalues. So in t h i s case r e l a t i v e l y precise estimation i s possible. 16 2.0 C o l l i n a a r i t y in Generalized Linear Model* 2.1 D e f i n i t i o n o f a G e n e r a l i z e d L i n e a r Model The g e n e r a l i z e d l i n e a r model a s d e v e l o p e d by CNELDER,WEDDERBURN,723 i s an u m b r e l l a t h a t encompasses amongst o t h e r s , l i n e a r m odels w i t h n o r m a l e r r o r s , l o g i t and p r o b i t m o d e ls f o r q u a n t a l r e s p o n s e s , and l o g l i n e a r m odels f o r c o u n t s . The u n d e r l y i n g f e a t u r e s o f t h e s e m o d e ls a r e : - t h e o b s e r v a t i o n s a r e i n d e p e n d e n t - t h e r e s p o n s e v a r i a n c e i s e q u a l t o , o r d i r e c t l y p r o p o r t i o n a l t o , some f u n c t i o n o f i t s e x p e c t a t i o n - t h e r e s p o n s e i s m o d e l l e d a d d i t i v e l y a s a f u n c t i o n o f a l i n e a r c o m b i n a t i o n o f t h e p r e d i c t o r v a r i a b l e s and e r r o r [WEDDERBURN,743 d e v e l o p s a t h e o r y b a s e d on t h e s e c o n d f e a t u r e a b o v e , t h e r e l a t i o n o f t h e v a r i a n c e t o t h e mean. He t e r m s t h i s q u a s i - l i k e l i h o o d , b e c a u s e ; a l t h o u g h no d i s t r i b u t i o n i s e x p l i c i t l y assumed, i t i s s u f f i c i e n t f o r e s t i m a t i o n p u r p o s e s and y i e l d s a n a l a g o u s r e s u l t s t o F i s h e r s c l a s s i c a l l i k e l i h o o d . [WEDDERBURN,743 shows t h a t f o r a one p a r a m e t e r e x p o n e n t i a l l i k e l i h o o d , t h e l o g l i k e l i h o o d i s t h e same a s t h e q u a s i l i k e l i h o o d . C o n s e q u e n t l y t h e w e akest s o r t o f d i s t r i b u t i o n t h a t c a n be assumed f o r t h i s c l a s s o f models c a n be shown t o be t h e g e n e r a l i z e d one p a r a m e t e r e x p o n e n t i a l f a m i l y . T h i s l e a d s t o t h e f o l l o w i n g d e f i n i t i o n , w h i c h i s b a s e d on t h e n o t a t i o n o f [McCULLAGH,NELDER,83]. D e f i n i t i o n 2.1 Generalized Linear Model y = g-'(XB) + € i s c o m p l e t e l y d e f i n e d by t h e f o l l o w i n g t h r e e components. <i> a random component, y , i n d e p e n d e n t l y d i s t r i b u t e d w i t h an n x l mean v a l u e v e c t o r u = E ( y ) and an n x l e r r o r v e c t o r €. The p r o b a b i l t y d e n s i t y o f y f o l l o w s a g e n e r a l i z e d e x p o n e n t i a l d e n s i t y f ( y ; 9 , 0 ) = e x p ( [ y 9 - b(8)]/a<0> + c ( y , 0 ) ) where 8 i s c a l l e d t h e c a n o n i c a l p a r a m e t e r and 0 i s c a l l e d t h e d i s p e r s i o n p a r a m e t e r . ( i i ) a s y s t e m a t i c component, X b e i n g an nxp m a t r i x o f p r e d i c t o r v a r i a b l e s , and an p x l c o e f f i c i e n t v e c t o r B, g i v i n g an n x l l i n e a r p r e d i c t o r n = Xfl ( i i i ) a l i n k f u n c t i o n , g ( . ) : IR->-]R, c o n n e c t i n g t h e random and s y s t e m a t i c components a s f o l l o w s n = g ( p ) o r p = g _ 1 ( X B ) • I t c a n be shown t h a t s u f f i c i e n t s t a t i s t i c s e x i s t f o r t h e p a r a m e t e r s i n t h e l i n e a r p r e d i c t o r n = XB when XB i s e q u a l t o t h e c a n o n i c a l p a r a m e t e r 8. In t h i s c a s e t h e l i n k f u n c t i o n g ( . ) i s c a l l e d t h e c a n o n i c a l l i n k o f t h e d i s t r i b u t i o n . A l t h o u g h i t i s n o t n e c e s s a r y t o u s e c a n o n i c a l l i n k s , f o r t h e p u r p o s e s o f t h i s t h e s i s t h e y w i l l be assumed, a s c o n s i d e r a b l e a n a l y t i c s i m p l i f i c a t i o n c a n be made ( s e e A p p e n d i x 2 A ) . T a b l e s 2.1 t o 2.3 g i v e d i s t r i b u t i o n d e t a i l s f o r s e l e c t e d members o f t h e g e n e r a l i z e d e x p o n e n t i a l f a m i l y . Note t h a t t h e n o n - c e n t r a l h y p e r g e o m e t r i c and t h e m u l t i n o m i a l d i s t r i b u t i o n s were e x c l u d e d b e c a u s e t h e i r means, v a r i a n c e s and l i n k f u n c t i o n s a r e c o m p l i c a t e d e x p r e s s i o n s . T a b l e 2.1 l i s t s t h e d e n s i t y f u n c t i o n s f o r t h e s e l e c t e d members, w h i l e T a b l e 2.2 g i v e s t h e c o r r e s p o n d i n g d e n s i t y p a r a m e t e r s a, b, c f o r t h e g e n e r a l e x p o n e n t i a l d i s t r i b u t i o n . The f i r s t c o l u m n o f T a b l e 2.3 g i v e s t h e c a n o n i c a l l i n k 9 = n, w h i l e t h e s e c o n d g i v e s t h e r e l a t i o n s h i p between t h e mean v e c t o r \i and t h e c a n o n i c a l p a r a m e t e r 9 ( s e e A p p e n d i x 1A f o r d e t a i l s ) . The t h i r d and f o u r t h c o l u m n s o f t a b l e 2.3 g i v e t h e v a r i a n c e o f y ( i e . v) a s a f u n c t i o n o f t h e mean and t h e c a n o n i c a l p a r a m e t e r r e s p e c t i v e l y . I t i s t o be n o t e d t h a t t h e r e l a t i o n s h i p o f v and p i n column t h r e e u n i q u e l y c h a r a c t e r i z e s t h e member. Table 2.1 Densities of the general exponential family 19 normal f(y;u,0) = r — T [21T0 J 1 -[% ,-(y-M) E , i exp {—L-~ } ^ 20 poisson f(y;p) • * ' H ' y=o,i, gamma f<y;u,0> T ( 0 ) L p J y yiO inverse f(y;u,0) Gaussian [2TT0y; aJ 2p l B :y pth order* f(y;u,0) = exp ( H \ - —-—1 + h( y , 0 ) ) L 1-p 2-p J binomial f(y;u,n) proportion y=0/n,1/n, negative f(y;p,k) binomial ^ y - - i ; L k+M J L k+P J 7 ' ' 1. gamma with 0=1 i s the exponential 2. binomial proportion with n=l i s l o g i s t i c 3. # denotes the "p s e r i e s " Table 2 . 2 Parameters of the general exponential family a(0) b(8) normal 0 KB™ c<0,y) -54[y'5:/0 +log(2ir0>] poisson 1 log(y!) gamma 0 -log(-9) l o g l ( 0 ) + 0 1 o g ( 0 y ) inverse 0 Gaussian (-28) - 1 / ( 2 0 y ) - ( l / 3 ) l o g ( 2 T T 0 y - ' 1 ) # - 1 r - - i i pth order 0 — [ (1-p) 8D U>--iaJ (p-2> h(y,0> binomial 1/n proportion log(l+e") l o g ( " > 3 >">.V negative 1 binomial -klog(l-e<») 1 . # denotes the "p s e r i e s " Table 2.3 Densities of the general exponential family g<H> P<6> V(H> v(8) normal u 8 1 1 poiss o n log(y> e° \i e° # -1 -1 1 gamma — — H i n v e r s e — [--] H Ue] G a u s s i a n # - i r - i V <p~1> f -1 l p ^ < p -b i n o m i a l p r o p o r t i o n r H 1 ke« k e M l + e M l - k ) ) n e g a t i v e l o g r 1 — r~« M + c"/k — — : — - — -J? , , Lk + MJ 1 - ke° r r ( l - k e 0 ) 6 5 b i n o m i a l 1. # denotes the "p s e r i e s " F u r t h e r d e t a i l s o f t h e s e d i s t r i b u t i o n s and t h e i r a p p l i c a t i o n s a r e t o be f o u n d i n [McCULLAGH,NELDER,831 and CMcCULLAGH,821. In t h e e n s u i n g d e v e l o p m e n t , much r e f e r e n c e i s made t o t h e maximum l i k e l i h o o d e s t i m a t i o n p r o c e d u r e s . The i m p o r t a n t r e s u l t s and b r i e f d e t a i l s o f t h e i r d e r i v a t i o n a r e g i v e n i n A p p e n d i x 2A. As i s shown i n t h i s a p p e n d i x F i s h e r ' s s c o r i n g t e c h n i q u e i s e q u i v a l e n t t o i t e r a t i v e l y r e w e i g h t e d l e a s t s q u a r e s , and an a l g o r i t h m f o r t h i s p r o c e d u r e i s g i v e n i n A p p e n d i x 2B. 2.2 D e f i n i t i o n o f C o l l i n e a r i t y i n G e n e r a l i z e d L i n e a r M o d e l s C o l l i n e a r i t y i n a g e n e r a l i z e d l i n e a r model i s more d i f f i c u l t t o d e f i n e t h a n i t s c o u n t e r p a r t i n t h e s t a n d a r d l i n e a r c a s e . I n t u i t i v e l y i t i s a g a i n s a i d t o be p r e s e n t when: - two o r more p r e d i c t o r v a r i a b l e s a r e h i g h l y c o r r e l a t e d - c o e f f i c i e n t e s t i m a t e s a r e d i f f e r e n t i n m a g n i t u d e o r s i g n f r o m t h o s e h y p o t h e s i s e d - c o e f f i c i e n t e s t i m a t e s a r e h i g h l y v a r i a b l e - d a t a c o l l e c t i o n i s d e f i c i e n t - m a t r i c e s u s e d i n t h e c o m p u t a t i o n o f t h e c o e f f i c i e n t e s t i m a t e s a r e i l l c o n d i t i o n e d . T h e r e a r e s e v e r a l a p p r o a c h e s t o d e f i n i n g i t . One a p p r o a c h b a s e d on [BELSEY,KUH,WELSCH,80] u s e s l i n e a r i s a t i o n o f t h e mode l . A s e c o n d a p p r o a c h i s s u g g e s t e d f r o m t h e i t e r a t i v e c a l c u l a t i o n s f o r g e n e r a l i z e d l i n e a r m o d e ls ( s e e A p p e n d i x 2 A ) . 2.2.1 L i n e a r i s a t i o n o f the Link F u n c t i o n [BELSEY,KUH,WELSCH,80] when commenting on t h e non l i n e a r c a s e i n t h e i r d i r e c t i o n s f o r f u t u r e r e s e a r c h s e c t i o n , s u g g e s t l i n e a r i s i n g t h e model y = g - 1 ( X f l ) + € t o t h e f o r m y = Xfi + € I n t u i t i v e l y , by a n a l o g y w i t h t h e s t a n d a r d l i n e a r c a s e , i t i s p l a u s i b l e t o d e f i n e c o l l i n e a r i t y i n t h e g e n e r a l s y s t e m a s b e i n g an a p p r o x i m a t e l i n e a r d e p e n d e n c y amongst t h e c o l u m n s o f X. T h i s l i n e a r i s a t i o n i s e x p r e s s e d f o r m a l l y i n t h e f o l l o w i n g t h e o r e m . Theorem 2.1 L i n e a r i s a t i o n of y = g - M X f l ) + € y = g — * ( X f i ) + € c a n be e x p r e s s e d i n t h e f o r m y = XG + € where X i = V t X i / a ( 0 ) w i t h X = [ x i F x V t] r 5C —* C x & * • • • f x v i D V = d i a g ( v ; L ) 24 T h i s c a n be p r o v e d a s f o l l o w s . F i r s t t h e r e s i d u a l due t o e r r o r f r o m a non l i n e a r p r o c e d u r e i s e x panded a b o u t t h e e s t i m a t e fi, g i v i n g 0(B) = 0(B) + J ( B) (B - B) + 0 e ( f l - B) •» 0(B) - J ( B ) B + J ( B ) B where J ( B ) i s t h e g r a d i e n t m a t r i x o f 0 w i t h r e s p e c t t o B. Now by l e t t i n g y = 0(B) - J ( B ) B ~ A X = - J ( B ) € = -€. t h e f i r s t e q u a t i o n f o l l o w s . The s e c o n d r e s u l t a p p e a l s t o t h e t h e o r y o f A p p e n d i x 2A a s f o l l o w s . X u = - J ( B ) 1 J By d e f i n i t i o n o f t h e g e n e r a l i s e d l i n e a r model t h i s i s j u s t LfiniJ L0B.J ("jL ©Hi] v [©"ii Now u s i n g t h e s i m p l i f i c a t i o n o f t h e c a n o n i c a l l i n k , a s i n A p p e n d i x 2A, and t h e d e f i n i t i o n o f t h e l i n e a r p r e d i c t o r n, t h i s s i m p l i f i e s t o I t i s t o be a d m i t t e d t h a t i n t h e p r o o f a bove, t h e u n w e i g h t e d r e s i d u a l d o e s n o t g i v e s an e f f i c i e n t e s t i m a t e a s t h a t due t o t h e w e i g h t e d r e s i d u a l s , W >,€. However a s a f i r s t a p p r o a c h , i t i s u s e f u l i n s u g g e s t i n g a s u i t a b l e s t a r t i n g p o i n t f o r d e f i n i n g c o l l i n e a r i t y i n t h e g e n e r a l c a s e . 2 . 2 . 2 I t a r a t i v a l y Rawaightad Laast Squaraa Approach The c o m p u t a t i o n o f t h e e s t i m a t e s c a n be c a s t i n a w e i g h t e d l e a s t s q u a r e s f o r m u l a t i o n a s i n A p p e n d i x 2a i e . fl _ (xrW<+: > X) - 4X TW <* > z <* > = (X rX>~*X xz<*> T h i s c a n be v i e w e d a s o r d i n a r y l e a s t s q u a r e s w i t h t h e t r a n s f o r m a t i o n a p p l i e d t o t h e d a t a . So i t would a g a i n be r e a s o n a b l e t o d e f i n e c o l l i n e a r i t y i n t h e g e n e r a l i z e d l i n e a r model a s an a p p r o x i m a t e l i n e a r d e p e n d e n c y amongst t h e c o l u m n s o f W X. Now u s i n g t h e d e f i n i t i o n o f t h e w e i g h t a s i n A p p e n d i x 2A, i e . = v j x ± a(0> X = W 2X z = W 2z v 1 3^(0) g i v e s -J V i 26 a(0) 2.2.3 C h o i c e o f A p p r o a c h The l i n k l i n e a r i s a t i o n and i t e r a t i v e l y r e w e i g h t e d l e a s t s q u a r e s p r o d u c e s i m i l a r t r a n s f o r m a t i o n s , namely X* ' = aT0T- a n d x* ' = r e s p e c t i v e l y . However t h e i n t u i t i v e n o t i o n s o f c o l l i n e a r i t y d i s c u s s e d e a r l i e r a r e s e e n more e a s i l y f r o m t h e i t e r a t i v e l y r e w e i g h t e d l e a s t s q u a r e s a p p r o a c h . T h i s w i l l be i l l u s t r a t e d i n t h e s e c t i o n s 2.4 and 2.5 on s o u r c e s and e f f e c t s o f c o l l i n e a r i t y i n t h e g e n e r a l i z e d l i n e a r m odel. A l s o CSCHAEFER,83] f o r s i m i l a r r e a s o n s i n t h e c a s e o f l o g i s t i c r e g r e s s i o n s e l e c t s t h e same a p p r o a c h . Hence, t h e f o l l o w i n g d e f i n i t i o n i s a d o p t e d . D e f i n i t i o n 2.2 C o l l i n e a r i t y i n a G e n e r a l i z e d L i n e a r Model y = g - 1 < X B ) + € i s s a i d t o e x i s t when F o r some s u i t a b l y c h o s e n 6 2 0 t h e r e e x i s t s a p x l v e c t o r c * O s u c h t h a t |Xc| £ 6|c| where X = W^X 2.3 Relationship of the Standard Linear Model and the Generalized Model C o l l i n e a r i t y D e f i n i t i o n s Now V i i s a f u n c t i o n o f t h e c a n o n i c a l p a r a m e t e r flt (=XG) as i n T a b l e 2.3. So s u b s t i t u t i n g f o r t h i s i n D e f i n i t i o n 2.2 g i v e s a d i r e c t e x p r e s s i o n s f o r X. The f i r s t c olumn o f T a b l e 2.4 l i s t s t h e i j t h e l e m e n t o f X f o r t h e s e l e c t e d members o f t h e g e n e r a l e x p o n e n t i a l f a m i l y . I t i s t o be n o t e d f r o m t h e s e c o n d column o f T a b l e 2.4 t h a t c o l l i n e a r i t y i n t h e s t a n d a r d l i n e a r model c a r r i e s o v e r t o t h e g e n e r a l i z e d l i n e a r model f o r a l l members o f t h e f a m i l y e x c e p t t h e so c a l l e d p s e r i e s . S i n c e , i n f o r m a l l y , when t h e r e I s a ft s u c h t h a t |Xft| i s " s m a l l " , t h e n X —*• X f o r a l l e x c e p t t h e p s e r i e s ( i e e x c e p t gamma ( p = l ) , i n v e r s e G a u s s i a n (p=2) and p t h o r d e r ) . Hence f o r t h e o t h e r members c o n s i d e r e d o f t h e f a m i l y ( i e . P o i s s o n , b i n o m i a l p r o p o r t i o n and n e g a t i v e b i n o m i a l ) c o l l i n e a r i t y i n t h e s t a n d a r d l i n e a r model i m p l i e s c o l l i n e a r i t y i n t h e g e n e r a l i z e d l i n e a r m odel. 28 T a b l e 2 . 4 L i n e a r i s e d P r e d i c t o r E l e m e n t s f o r t h e g e n e r a l e x p o n e n t i a l f a m i l y 1 i m x i j e-o n o r m a l - x , , - X ± J p o i s s o n e 6 / e X i j x , . , # 1 gamma X i J # l r - l l 3 " " * 1 ^ i l ~  r . 0 L 2 e J X : G a u s s i a n i n v e r s e -, — , X ± J p t h o r d e r b i n o m i a l p r o p o r t i o n 1 n Cl+e*) J 2n . . l"k<e«fl + < l - k ) e " ) ] ^ [ k ( 2 - k ) l n e g a t i v e (T-k> X l J b i n o m i a l 1. # d e n o t e s t h e "p s e r i e s ' However i t would be i n c o r r e c t t o d e f i n e c o l l i n e a r i t y i n t h e g e n e r a l l i n e a r model c a s e a s c o l l i n e a r i t y i n X. T h i s i s b e c a u s e i t i s p o s s i b l e f o r t h e X s y s t e m t o be c o l l i n e a r , b u t t h e p r e m u l t i p l i c a t i o n by W may remove t h e c o l l i n e a r i t y i n t h e g e n e r a l l i n e a r c a s e f o r t h e p s e r i e s . F o r example c o n s i d e r a gamma model w i t h 0=1 and X = IXi 1 X i e l _ T f i i 1 Now X = W^X where wt So x 4 6 X j. s B i X e i + BeXee A i X a i +' B^X^c Xcl X c l ( C I X H + C e X i e ) E + (C,.Xe). + CftXae)* f c t X i i * C f f i X i e l + [ d X a t •+ C a X c a 1 Now X may be c o l l i n e a r ( e g . X i ± * X i a * ytt&± * X a a and Ct = 1, clS» = -1) w i t h |Xc| < 6, b u t f o r B:l , B e s u f f i c i e n t l y s m a l l I X c l > 6. 2.4 S o u r c e s o f C o l l i n e a r i t y i n a G e n e r a l i z e d L i n e a r Model In many p r a c t i c a l a p p l i c a t i o n s t h e s c a l i n g m a t r i x W** r e l a t i n g t h e X and X s y s t e m s w i l l be a p p r o x i m a t e l y a c o n s t a n t s c a l a r m u l t i p l e o f t h e i d e n t i t y m a t r i x . In p a r t i c u l a r t h i s o c c u r s when t h e l i n e a r c o m b i n a t i o n s o f t h e rows and ft a r e a p p r o x i m a t e l y e q u a l , a s i s shown i n t h e f o l l o w i n g t h e o r e m . Theorem 2.1 C o n d i t i o n f o r a C o n s t a n t S c a l i n g M a t r i x W"" The s c a l i n g m a t r i x W*^  w i l l be an a p p r o x i m a t e s c a l a r m u l t i p l e o f t h e i d e n t i t y m a t r i x i f x*B a r e a p p r o x i m a t e l y e q u a l , where x± a r e t h e rows o f X. • T h i s i s p r o v e d a s f o l l o w s . I f W i e q u a l s a c o n s t a n t t h e n so d o e s v 4 , s i n c e t h e y a r e e q u a l up t o t h e d i s p e r s i o n p a r a m e t e r when t h e c a n o n i c a l l i n k i s u s e d . But t h e r e i s a one t o one c o r r e s p o n d e n c e between v and p and 8 a s i s l i s t e d i n T a b l e 2.3. Hence w± c o n s t a n t i m p l i e s Q± i s c o n s t a n t . F o r example c o n s i d e r t h e b i n o m i a l p r o p o r t i o n s c a s e where H = 1 + <1 - 4 v ) ^ = C s a y But e^ H 1 + e® s o 8 = l o g ( c / ( l - c ) ) = C Hence i n t h i s c a s e i f w± i s c o n s t a n t t h e n ao i s 8* (=x±fl>. So i n t h e s e c a s e s , t h e same s o u r c e s o f c o l l i n e a r i t y w i l l a l s o a r i s e i n t h e g e n e r a l l i n e a r c a s e , namely : - l a r g e p a i r w i s e c o r r e l a t i o n s - d a t a c o l l e c t i o n - model s p e c i f i c a t i o n - o v e r d e f i n e d model - o u t l i e r s However i n t h e g e n e r a l l i n e a r c a s e , i t i s b e t t e r t o t h i n k t h e o r e t i c a l l y i n t e r m s o f t h e s c a l e d d a t a m a t r i x X. So t h e a b o v e n o t i o n s a r e c a p t u r e d by D e f i n i t i o n 2.2 w i t h t h e a p p r o p r i a t e s u b s t i t u t i o n . F o r example t h e l a r g e p a i r w i s e c o r r e l a t i o n o f x± and X j g i v e s r i s e t o c o l l i n e a r i t y a s i n D e f i n i t i o n 2.2. 2.5 E f f e c t s o f C o l l i n e a r i t y i n a G e n e r a l i z e d L i n e a r Model 2.5.1 E s t i m a t i o n E f f e c t s I n a n a l o g o u s manner t o t h e s t a n d a r d l i n e a r c a s e , b o t h t h e e s t i m a t e s and t h e i r v a r i a n c e s a r e a f f e c t e d by c o l l i n e a r i t y . The e s t i m a t e s o f R may be e x p r e s s e d i t e r a t i v e l y a s i n r e s u l t ( s e e A p p e n d i x 2A) B = fi<*> + (XrW<* > X) -1X""r (y - u<*M O m i t t i n g s u p e r s c r i p t s f o r c l a r i t y , XrWX c a n be decomposed a s f o l l o w s . (X^WX)" 1 =' ( X ^ X ) " 1 where X = W^X = <VDU" rUDV T>- 1 [ s i n g u l a r v a l u e d e c o m p o s i t i o n ] = ( V D " G V T ) - ' = V D _ e V T F> 1 = Z — — — I * , - , , [V o r t h o n o r m a l ] So a t any i t e r a t i o n ( t ) , i n c l u d i n g t h e f i n a l one, c o l l i n e a r i t y ( i e . s m a l l s i n g u l a r v a l u e s o f X > i s s e e n t o d i r e c t l y a f f e c t t h e p a r a m e t e r e s t i m a t e s t h r o u g h t h e s m a l l s i n g u l a r v a l u e s . Hence c o l l i n e a r i t y i n t h e g e n e r a l i z e d l i n e a r model c a n d e v e l o p t h r o u g h o u t t h e i t e r a t i v e e s t i m a t i o n p r o c e d u r e and t h i s s u g g e s t s t h a t i t may be w o r t h w h i l e f o r any 33 e s t i m a t i o n p r o c e d u r e t o a d j u s t f o r c o l l i n e a r i t y a t e a c h i t e r a t i o n . The v a r i a n c e e s t i m a t e s a t c o n v e r g e n c e c a n be a p p r o x i m a t e d by t h e a s y m p t o t i c r e s u l t V a r ( B ) =• <XTWX)-1o-fiS So a g a i n c o l l i n e a r i t y d i r e c t l y a f f e c t s t h e e s t i m a t i o n , by i n f l a t i n g t h e v a r i a n c e e s t i m a t e s a c c o r d i n g t o t h e s m a l l s i n g u l a r v a l u e s o f X ( s i n c e X TVX i s p o s i t i v e d e f i n i t e and h e n c e has p o s i t i v e s i n g u l a r v a l u e s ) . 2.4.2 In f e r e n c e and P r e d i c t o r E f f e c t s I n a s i m i l a r manner t o t h e s t a n d a r d l i n e a r c a s e t h e e q u i v a l e n t s t a t i s t i c s a r e a f f e c t e d by c o l l i n e a r i t y . However t h e e f f e c t i s n o t s o d i r e c t . F o r example t h e a n a l o g u e o f t h e t s t a t i s t i c i s t h e d e v i a n c e d i f f e r e n c e where t h e d e v i a n c e i s D ( y , H ) = 2Z w t [ y ( 8 i - 8,.) - b ( 8 J ) + b ( 8 a ) ] where 8 f = 8(£) 8 i = 8 (y j. ) A i s a complex f o r m o f B and W w h i c h a r e a f f e c t e d by c o l l i n e a r i t y . S i m i l a r l y w i t h t h e g e n e r a l i z e d P e a r s o n s t a t i s t i c i v ( H L ) w h i c h i s u s e d t o c a l c u l a t e t h e d i s p e r s i o n p a r a m e t e r 35 2.6 A p p e n d i x 2A Maximum L i k e l i h o o d and t h e G e n e r a l i z e d L i n e a r Model The f o l l o w i n g d e r i v a t i o n o f t h e s t a n d a r d r e s u l t s f o r t h e maximum l i k e l i h o o d i t e r a t i v e e s t i m a t e s i s t a k e n f r o m [McCULLAGH,NELDER,83]. S i n c e t h e o b s e r v a t i o n s a r e i n d e p e n d e n t , c o n s i d e r i n i t i a l l y t h e l o g l i k e l i h o o d c o n t r i b u t i o n o f t h e i t h o b s e r v a t i o n i n c a n o n i c a l f o r m 1±, w i t h c a n o n i c a l p a r a m e t e r 8j. and d i s p e r s i o n p a r a m e t e r 0. U s i n g t h e n o t a t i o n o f D e f i n i t i o n 2.1, and a p p l y i n g F i s h e r ' s s c o r i n g t e c h n i q u e ( s e e [KENDALL,STUART,67.2] f o r d e t a i l s ) g i v e s t h e f o l l o w i n g r e s u l t s f o r t h e i t e r a t i v e maximum l i k e l i h o o d e s t i m a t e s . 11 = I—aT0i J + c ( y * ' 0 ) @11 = [~yi - b ( 8 i ) 1 _ yj. - p t Hence t h e mean v e c t o r u c a n be r e p r e s e n t e d i n t e r m s o f t h e c a n o n i c a l p a r a m e t e r 8 a s E ( y i ) = Hi = b O i ) by u s i n g t h e s t a n d a r d r e s u l t Now Q a l ± = - b ( 8 i ) 0 8 i a a ( 0 ) ao t h e v a r i a n c e f u n c t i o n v, c a n be s i m i l a r l y r e p r e s e n t e d i n t e r m s o f t h e c a n o n i c a l p a r a m e t e r 8 a s V a r ( y t ) = v± = b ( 9 1 ) a C 0 ) by u s i n g t h e s t a n d a r d r e s u l t EP] * E[H]* •° • The maximum l i k e l i h o o d e q u a t i o n s a r e o b t a i n e d i n t e r m s o f H and v i n t h e f o l l o w i n g s t e p s . Now t h e t o t a l c o n t r i b u t i o n t o any l i n e a r component s u c h as t h e l o g l i k e l i h o o d o r t h e s c o r e i s j u s t t h e sum o f t h e i n d i v i d u a l i n d e p e n d e n t c o n t r i b u t i o n s . Hence t h e s c o r e v e c t o r s , w h i c h i s t h e f i r s t d e r i v a t i v e o f t h e l o g l i k e l i h o o d , c a n be e x p r e s s e d a s - - [lib] • ? [ & ] ' ! [ & ] F o r c a n o n i c a l l i n k s t h e f o l l o w i n g s i m p l i f i c a t i o n i s u s e d r e p e a t e d l y . [SKiJ v t L©9 j.J V i V i a<0> So t h e s c o r e component s i m p l i f i e s t o Hence t h e maximum l i k e l i h o o d e q u a t i o n s i n component f o r m a r e w h i c h f o r c a n o n i c a l l i n k s s i m p l i f i e s t o ? L^^aTsKh- = ° r = 1 < 1 > P o r i n m a t r i x f o r m * r<y - H> _ 0 a<0) The i n f o r m a t i o n m a t r i x , w h i c h was d e f i n e d by F i s h e r a s minus t h e e x p e c t e d v a l u e o f t h e s e c o n d d e r i v a t i v e o f t h e l o g l i k e l i h o o d , i s d e r i v e d i n t h e f o l l o w i n g s t e p s . 0 a l ! 0 where t h e s e c o n d t e r m i s z e r o s i n c e i t c o n t a i n s t h e d e r i v a t i v e o f a c o n s t a n t due t o t h e c a n o n i c a l l i n k s i m p l i f i c a t i o n . Hence t h e e l e m e n t s a r e - E r g f f iii I _ r®iii'| r ? H i i L@fiv.@fisJ 1®"*J L @ f i s J L @ n , J V, x j >,x 1 S r@(^ i | v t L®n,J D e f i n e w e i g h t m a t r i x W = diagCw* w,,,) as 38 a , S :(0> So t h e n e g a t i v e e x p e c t e d v a l u e e x p r e s s i o n s i m p l i f i e s t o f 0 * 1 1 lOBv..@Bs W i X i t , X i s So summing t o g e t t h e t o t a l c o n t r i b u t i o n , t h e r s t h e l e m e n t o f t h e i n f o r m a t i o n m a t r i x i s I v > s = £ Wj X j v » X t S o r i n m a t r i x f o r m I = X rWX So u s i n g F i s h e r ' s s c o r i n g scheme, t h e maximum l i k e l i h o o d e s t i m a t e s o f ft a r e g i v e n by ft<t-Hi> = B < * > + I -• * s F o r f u r t h e r d e t a i l s o f t h i s s e e [KENDALL,STUART,67.2]. Now b e c a u s e o f t h e c a n o n i c a l l i n k s i m p l i f i c a t i o n t h e i n f o r m a t i o n m a t r i x i s t h e same a s t h e H e s s i a n , and s o t h e above s c o r i n g scheme i s e q u i v a l e n t t o t h e Newton Rhapson method ( e g . tBARD,741) t y p i c a l l y e x p r e s s e d as oft = H--*s o r H6 ft = s So A A H f i < * - i > = H B < t ; > + s o r = ft<*> + ( X T W X ) ~ 1 X r ( y - p'* * ) But (Hft) £ Hi.,B, ..j A Z Z w,,.xklxk|Bj Z W k X k i f i k So 39 Hfi ' Z W k ' ^ X k i Ldfiuj B u t t h i s i s j u s t t h e f o r m o f w e i g h t e d l e a s t s q u a r e s A (X TW< t >X)fl< t> = X T z ( t ) w i t h pseudo d e p e n d e n t ( w o r k i n g ) v a r i a b l e Z k - riu  + * T h i s g e n e r a l l y c o n v e r g e s , s o t h e f i n a l e s t i m a t e c a n be r e p r e s e n t e d a s A ft = (X TWX) 1X'rWz 2.7 A p p e n d i x 2B I t e r a t i v e l y R e w e i g h t e d L e a s t S q u a r e s A l g o r i t h m The f o l l o w i n g a l g o r i t h m f o r c a l c u l a t i n g B i s t a k e n f r o m [McCULLAGH,NELDER,83]. A l g o r i t h m 2.1 I t e r a t i v e l y r e w e i g h t e d l e a s t s q u a r e s STEP 0 O b t a i n i n i t i a l e s t i m a t e s o f t h e l i n e a r p r e d i c t o r A A n < o > = X f i < 0 > f r o m t h e d a t a A jj < O > = y * A < o > = g ( p < ° > > ' x <o> r | LdnJ LdfiJ A C O , STEP 1 REPEAT t = 0 ( 1 ) UNTIL c o n v e r g e n c e S e t up w o r k i n g d e p e n d e n t v a r i a b l e S e t up w e i g h t W W<*> = d i a g ( W i ( t ) w „ < t > > A < t: > 2 [ i i ] R e g r e s s z <* :> on x t xf:, w i t h w e i g h t W t o g e t 41 A < * -i- 1 > (X rW< t>X>- lX" rW< t >z < t ;> A Xfi <*•*•'•> A m - [is] END REPEAT 42 3. D i a g n o s t i c s f o r C o l l i n e a r i t y I n G e n e r a l i z e d L i n e a r  M o d e l s 3.1 D e s i r a b l e P r o p e r t i e s f o r D i a g n o s t i c M e a s u r e s A d i a g n o s t i c measure s h o u l d n o t be b a s e d on methods t h a t t r y t o s e n s e c o l l i n e a r i t y f r o m i t s p o s s i b l e e f f e c t s s u c h as i n c o r r e c t l y h y p o t h e s i s e d s i g n s o r s e n s i t i v i t y i n row d e l e t i o n s t a t i s t i c s . T h i s i s b e c a u s e most o f t h e s e a r e n o t n e c e s s a r y and s u f f i c i e n t c o n d i t i o n s f o r c o l l i n e a r i t y ( e g . [MULLET,761 i l l u s t r a t e s t h e s i g n s c a s e ) . R a t h e r , t h e d i a g n o s t i c s h o u l d be d i r e c t l y l i n k e d t o t h e d e f i n i t i o n o f a p p r o x i m a t e l i n e a r d e p e n d e n c y amongst t h e c o l u m n s o f t h e p r e d i c t o r v a r i a b l e s , i e . |Xc| £ 6 | c | T h i s g i v e s two d e s i r a b l e p r o p e r t i e s o f a d i a g n o s t i c measure. F i r s t , i t s h o u l d be c a p a b l e o f i d e n t i f y i n g t h e l i n e a r c o m b i n a t i o n s o f o f f e n d i n g p r e d i c t o r v a r i a b l e s . S e c o n d , i t s h o u l d be a b l e t o q u a n t i f y t h e e f f e c t s o f c o l l i n e a r i t y . However t h e u s e o f t h e q u a n t i t a t i v e a s p e c t o f t h e d i a g n o s t i c measure must be s u b j e c t i v e , s i n c e t h e r e a r e no d i s t r i b u t i o n a s s u m p t i o n s a b o u t t h e n a t u r e o f t h e p r e d i c t o r v a r i a b l e s X. Now w h i l e c o l l i n e a r i t y i a p r i m a r i l y a d a t a p r o b l e m , i t haa c o n s e q u e n c e s f o r t h e e s t i m a t i o n o f t h e p a r a m e t e r s , and i n any s u b s e q u e n t h y p o t h e s i s t e s t i n g o r p r e d i c t i o n . In t h e s t a n d a r d l i n e a r c a s e , t h e measure o f d e g r a d a t i o n due t o c o l l i n e a r i t y ( s a y t h e s i n g u l a r v a l u e s o f X) d i r e c t l y e n t e r s i n t o t h e e x p r e s s i o n s f o r t h e e s t i m a t e s , t e s t s and p r e d i c t i o n s . In a s i m i l a r manner, t h i s n a t u r a l embedding s h o u l d be p r e s e n t i n t h e g e n e r a l c a s e . 3.2 M e a s u r e s o f C o l l i n e a r i t y Now w h i l e D e f i n i t i o n 2.2 e x a c t l y e x p r e s s e s t h e i d e a o f c o l l i n e a r i t y i n t h e g e n e r a l i z e d l i n e a r model s e t t i n g , i t i s an i m p r a c t i c a l measure, a s t h e r e a r e no e a s y ways o f f i n d i n g t h e l i n e a r c o m b i n a t i o n s . However, t h e r e i s a one t o one c o r r e s p o n d e n c e between c o l l i n e a r i t y a s i n D e f i n i t i o n 1.2 and t h e i n s t a b i l i t y o f t h e X s y s t e m a s c h a r a c t e r i s e d by s m a l l s i n g u l a r v a l u e s . T h i s i s p r o v e d i n t h e f o l l o w i n g t h e o r e m . Theorem 3.1 E q u i v a l e n c e o f C o l l i n e a r i t y and a S m a l l S i n g u l a r V a l u e o f X f o r t h e S t a n d a r d L i n e a r Model y = XB + € F o r some s u i t a b l y c h o s e n 6 1 0 t h e r e e x i s t s a p x l v e c t o r c*0 s u c h t h a t |Xc| i 6 | c | i f and o n l y i f d i 1 6 where d 4 i a t h e s m a l l e s t s i n g u l a r v a l u e o f X T h i s i s p r o v e d a s f o l l o w s . The f a c t t h a t s m a l l s i n g u l a r v a l u e s g i v e r i s e t o c o l l i n e a r i t y was e s t a b l i s h e d i n s e c t i o n 1.3.1. So i t r e m a i n s t o show t h a t c o l l i n e a r i t y a c c o r d i n g t o D e f i n i t i o n 1.2 i m p l i e s a s m a l l s i n g u l a r v a l u e . W i t h o u t l o s s o f g e n e r a l i t y , s u p p o s e t h a t t h e c o f D e f i n i t i o n 1.2 has u n i t norm ( t h i s i s e q u i v a l e n t t o s c a l i n g ) . C o n s i d e r \Xc\ eS: = c T X T X c i min u"rX"rXu where u T u = 1 u = d i e [ e x t r e m a o f r a t i o o f q u a d r a t i c f o r m s 1 Hence i f | Xc | i. 6 t h e n d. 1 6. So d, i s s m a l l . A l t h o u g h n o t e m p h a s i z e d i n c h a p t e r one, c o l l i n e a r i t y i s a m u l t i v a r i a t e p r o b l e m , i n t h a t t h e r e may be k l i n e a r l y i n d e p e n d e n t c± s u c h t h a t | X c i | < Now t h e above t h e o r e m can be e x t e n d e d t o t h e c a s e o f m u l t i p l e c o l l i n e a r i t i e s as f o l l o w s . C o r o l l a r y 3.1 E q u i v a l e n c e o f M u l t i p l e C o l l i n e a r i t i e s and S m a l l S i n g u l a r V a l u e s o f X f o r t h e S t a n d a r d L i n e a r Model y = XB + € k c o l l i n e a r i t i e s a r e p r e s e n t i n t h e s t a n d a r d l i n e a r model i f and o n l y i f t h e k t h s m a l l e s t s i n g u l a r v a l u e s o f X a r e " s m a l l " • The " o n l y i f " argument was e s t a b l i s h e d i n s e c t i o n 1.3.1 w h i l e t h e " i f " argument f o l l o w s d i r e c t l y f r o m t h e above t h e o r e m by t h e a n a l o g u e w i t h t h e e s t i m a t i o n o f t h e v a r i a n c e p r i n c i p a l c omponents. • F u r t h e r , t h i s c a r r i e s t h r o u g h t o t h e g e n e r a l l i n e a r s e t t i n g s i n c e i t i s e q u i v a l e n t t o a l i n e a r t r a n s f o r m a t i o n o f t h e v a r i a b l e s . Hence t h e f o l l o w i n g d e f i n i t i o n o f a measure D e f i n i t i o n 3.1 C o l l i n e a r i t y M e asure i n a G e n e r a l i z e d L i n e a r Model y = g- 1<XB) + € i s g i v e n by t h e " s m a l l " s i n g u l a r v a l u e s o f X = W'X • The above d e f i n i t i o n i s r e a s o n a b l e a s i t comes n e a r t o h a v i n g a l l t h e d e s i r a b l e p r o p e r t i e s d i s c u s s e d e a r l i e r . F i r s t i t i s p o s s i b l e t o a p p l y t h e i d e n t i f i c a t i o n a l g o r i t h m o f CBELSEY,KUH,WELSCH,80] t o d e d u c e t h e o f f e n d i n g c o m b i n a t i o n s o f p s e u d o p r e d i c t o r v a r i a b l e s x.j c o r r e s p o n d i n g t o t h e s i n g u l a r v a l u e s . Now i t would be h i g h l y d e s i r a b l e t o r e l a t e t h e s i n g u l a r v a l u e s d i r e c t l y t o t h e o r i g i n a l p r e d i c t o r v a r i a b l e s X j . However t h i s i s i n g e n e r a l n o t p o s s i b l e b e c a u s e t h e n a t u r a l v a r i a b l e s i n t h e g e n e r a l s e t t i n g a r e t h e p s e u d o o n e s x j . N e v e r t h e l e s s i n many p r a c t i c a l s i t u a t i o n s t h e s c a l i n g m a t r i x W i s an a p p r o x i m a t e s c a l a r m u l t i p l e o f t h e i d e n t i t y m a t r i x and has l i t t l e e f f e c t on t h e c o l l i n e a r s t r u c t u r e . S e c o n d , t h e m a g n i t u d e s o f t h e c o l 1 i n e a r i t i e s c a n be measured by t h o s e o f t h e c o r r e s p o n d i n g s i n g u l a r i t i e s . To do t h i s a c o l l i n e a r i t y i n d e x i s d e f i n e d a s f o l l o w s . D e f i n i t i o n 3.2 C o l l i n e a r i t y Index f o r G e n e r a l i z e d L i n e a r Model y = g"l(Xfl) • € i s where ( d t , . . . , d F n) a r e t h e s i n g u l a r v a l u e s o f X • W h i l e i t i s s u b j e c t i v e t h e r e i s a l a r g e body o f e x p e r i e n c e , s u c h a s r e f e r r e d t o i n [MONTGOMERY,PECK,82], t h a t s u g g e s t s a c o l l i n e a r i t y i s p r e s e n t i f t h e c o r r e s p o n d i n g c o l l i n e a r i t y i n d e x i s g r e a t e r t h a n 10.0. F i n a l l y t h e e f f e c t s o f c o l l i n e a r i t y a r e c l e a r l y m a n i f e s t e d i n t h e e s t i m a t i o n , i n f e r e n c e and p r e d i c t i o n c a l c u l a t i o n s a s was d i s c u s s e d i n s e c t i o n 2.4. 3.3 Model Dependency i n C o l l i n e a r S y s t e m s From t h e d e f i n i t i o n i s i s s e e n t h a t t h e c o l l i n e a r i t y m easure f o r a g e n e r a l i s e d l i n e a r model d e p e n d s s p e c i f i c a l l y upon t h e d i s t r i b u t i o n c h o s e n f o r t h e r e s p o n s e y ( e g . b i n o m i a l p r o p o r t i o n , P o i s s o n , e t c ) . I t i s d e s i r a b l e t o be a b l e t o q u a n t i f y t h i s i n d e t e r m i n a n c y i n t h e c o l l i n e a r i t y . The f o l l o w i n g t h e o r e m c o n j e c t u r e d by t h e a u t h o r and p r o v e d by CM0YLS,85] comes p a r t way i n a n s w e r i n g t h i s . Theorem 3.1 C o l l i n e a r i t y Index R e l a t i o n s h i p s a £ a i. a Wr^mx W m i v , where a, a a r e t h e c o l l i n e a r i t y i n d i c e s o f X, X and w m i „ , w,•,,„,« a r e t h e m i n i m a l and maximal d i a g o n a l e l e m e n t s o f t h e w e i g h t m a t r i x W • T h i s i s p r o v e d a s f o l l o w s . L e t t h e s u b s c r i p t s ,,, and M s t a n d f o r m±>*. and ,.,*»>«. Suppose A , T i s t h e e i g e n s y s t e m o f X XX, and A , T t h e e i g e n s y s t e m o f X rWX. Then ( X r X ) t M = X M t M t M T ( X r X ) t M = X M t M r t M = A M tT o r t h o g o n a l ] L e t z = X t M s o X M = z" rz C o n s i d e r u TX TWXu. I t c a n be t h o u g h t o f a s t h e r a t i o o f t h e e x t r e m a o f q u a d r a t i c f o r m s , s u b j e c t t o t h e s c a l i n g r e s t r i c t i o n u T u = 1 and i s s o m a x i m i s e d by u = t M . So max(u rX TWXu) = t M r ( X r W X ) t M = X M > t M x ( X T W X ) t M u Hence X M i z TWz = Z « i Z i T Z i i w r a Z z i T Z i = wri1XM Now by t h e r a t i o o f t h e e x t r e m a o f q u a d r a t i c f o r m a X M = t M rX Twxt M = z TWz where z = X t M = I W i Z i T Z , 1 w „ Z z i T Z i = w M t M T X T X t „ £ w M t M T X T X t M [ e x t r e m a o f r a t i o ] ,= W M X M Thus £ X M £ wMX S i m i l a r l y WrtlXm £ X m i. wMX D i v i d i n g t h e s e a p p r o p r i a t e l y y i e l d s t h e r e s u l t , s i n c e t h e c o v a r i a n c e and w e i g h t m a t r i c e s a r e p o s i t i v e semi d e f i n i t e . • So t h i s t h e o r e m s a y s t h a t t h e c o l l i n e a r i t y o f t h e g e n e r a l s y s t e m i s bounded by t h a t o f t h e s t a n d a r d l i n e a r s y s t e m w i t h f a c t o r s b e i n g t h e r a t i o s o f t h e minimum and maximum s c a l i n g w e i g h t s . T h i s i s a u s e f u l r e s u l t f o r two r e a s o n s . F i r s t , g i v e n a c o l l i n e a r i t y a n a l y s i s w i t h a s t a n d a r d l i n e a r model, t h e e f f e c t s o f c o l l i n e a r i t y w i t h o t h e r g e n e r a l m o d e l s c a n be c a l c u l a t e d w i t h o u t d o i n g a n o t h e r e ' i g e n a n a l y s i s . S e c o n d , i t shows t h a t i£ t h e s c a l i n g m a t r i x W i s n e a r l y a s c a l a r m u l t i p l e o f t h e i d e n t i t y m a t r i x , t h e n t h e c o l l i n e a r s t r u c t u r e o f t h e t r a n s f o r m e d p r e d i c t o r i s a p p r o x i m a t e l y t h a t o f t h e o r i g i n a l p r e d i c t o r m a t r i x . S i n c e t h i s i s o f t e n t h e c a s e w i t h many p r a c t i c a l a p p l i c a t i o n s , t h i s f a c t i s o f c o n s i d e r a b l e use i n t h e f o l l o w i n g c h a p t e r on e s t i m a t i o n . By c o n s i d e r i n g t h e e x p r e s s i o n v ( 0 ) i n t h e f o u r t h column o f T a b l e 2.3 i t i s p o s s i b l e t o compare t h e m a g n i t u d e s o f t h e s e bounds f o r t h e members o f t h e f a m i l y . In p a r t i c u l a r f o r t h e P o i s s o n , X i ft X j f l min e ~ max e _ _ a £ a £ — a X i B , X i B max e min e w h i l e f o r t h e gamma t h e bounds a r e max ( X i f i ) ^ ~ min ( x ( R ) a — . . ,.,a £ a £ ——-or min ( X i f l ) " max ( x i f l ) 1 * I t i s s e e n i m m e d i a t e l y f r o m t h e above l o w e r bound, t h a t c o l l i n e a r i t y i n t h e s t a n d a r d l i n e a r model becomes more o f a p r o b l e m when a gamma model i s u s e d . T h i s i s a n o t h e r example o f t h e p r o b l e m s c a u s e d by t h e so c a l l e d p s e r i e s . 4.0 E s t i m a t i o n f o r G e n e r a l i z e d L i n e a r M o d e l s i n t h e P r e s e n c e  o f C o l l i n e a r i t y H a v i n g d e t e c t e d c o l l i n e a r i t y by t h e methods o f t h e p r e v i o u s c h a p t e r , t h e p r o b l e m o f e s t i m a t i o n i n i t s p r e s e n c e r e m a i n s . The s o l u t i o n t o t h i s d e p e n d s on t h e c a u s e s o f t h e c o l l i n e a r i t y . However a c a u t i o n a r y word i s n e c e s s a r y . W h i l e p r e c i s e e s t i m a t i o n i s p o s s i b l e f o r c e r t a i n l i n e a r c o m b i n a t i o n s o f t h e c o e f f i c i e n t s a s was shown i n s e c t i o n 1.3.4, [ALLEN,77] p96 shows w i t h an example "We s h o u l d r e c o g n i s e t h e e x i s t e n c e o f s i t u a t i o n s where no e s t i m a t i o n by any method i s w a r r a n t e d " B e f o r e p r o c e e d i n g t o examine t h e v a r i o u s p o s s i b l e r e m e d i a l s o l u t i o n s i n d e t a i l , a g e n e r a l o v e r v i e w i s g i v e n w i t h r e s p e c t t o t h e s o u r c e s and e f f e c t s d i s c u s s e d i n c h a p t e r one. 4.1 Remedies f o r C o l l i n e a r S o u r c e s From a p r a g m a t i c v i e w p o i n t , o f t e n t h e e f f e c t o f t h e s c a l i n g m a t r i x W i s n e g l i g i b l e , s o some p r o g r e s s c a n be made by t r e a t i n g t h e model as a s t a n d a r d l i n e a r one. T h i s i s t h e a p p r o a c h t a k e n i n f o l l o w i n g s e c t i o n s , e x c e p t where n o t e d o t h e r w i s e . 4.1.1 L a r g e P a l r w i a e C o r r e l a t i o n s i n X W h i l e t h i s c o u l d be a l l e v i a t e d by a B a y e s i a n a p p r o a c h , i t i s u n l i k e l y s u f f i c i e n t i n f o r m a t i o n e x i s t s t o f o r m u l a t e a p r i o r d i s t r i b u t i o n o r l i k e l i h o o d . I f t h e p a t t e r n i s v e r y s i m p l e i t may be p o s s i b l e t o remedy t h i s by v a r i a b l e d e l e t i o n . 4.1.2 Data C o l l e c t i o n I f t h e d e p e n d e n c y i s s u s p e c t e d t o be due t o e x t r i n s i c f a c t o r s , t h e n c o l l e c t i o n o f e x t r a d a t a a c c o r d i n g t o a p r o p e r s a m p l i n g p l a n i s t h e o b v i o u s s o l u t i o n . However, t h i s may o f t e n be i n f e a s i b l e due t o c o s t , i n c o n s i s t e n c y w i t h t h e p r e v i o u s d a t a , o r J u s t l a c k o f a v a i l a b l e d a t a . In c o n t r a s t , i f t h e d e p e n d e n c y i s i n t r i n s i c , t h e n a B a y e s i a n a p p r o a c h may be u s e d t o i n c o r p o r a t e t h e i n f o r m a t i o n on t h e u n d e r l y i n g d e p e n d e n c y mechanism. 4.1.3 Model S p e c i f i c a t i o n The common s o l u t i o n t o t h e c a s e o f e x a c t d e p e n d e n c i e s , i s t o e l i m i n a t e t h e r e d u n d a n t p r e d i c t o r v a r i a b l e s ( e g . one o f t h e l e v e l s o f a c a t e g o r i c a l v a r i a b l e ) . W i t h a p p r o x i m a t e d e p e n d e n c i e s , s i n c e t h e s o u r c e i s c l e a r l y s p e c i f i e d , t h e r e i s good j u s t i f i c a t i o n f o r e l i m i n a t i n g p r e d i c t o r v a r i a b l e s . However as CGUNST,83] n o t e s , one must be c a r e f u l t o make s u r e t h a t t h i s i s t h e s o l e c a u s e o f t h e c o l l i n e a r i t y i n t h e d a t a . 4.1.4 O v e r d e f i n e d Model One method t o h a n d l e t h i s i s p r i n c i p a l components r e g r e s s i o n , where t h e e i g e n v a l u e s o f X T X a r e u s e d t o e l i m i n a t e some p r e d i c t o r v a r i a b l e s . T h i s has t h e a d v a n t a g e t h a t e x a c t d i s t r i b u t i o n t h e o r y e x i s t s f o r t h e d e c i s i o n s t a t i s t i c s , u n l i k e o t h e r v a r i a b l e s e l e c t i o n p r o c e d u r e s , w h i c h have t h e i r s t a t i s t i c s i n f l u e n c e d by t h e c o l l i n e a r i t y . However tGUNST,831 p2237 n o t e s t h e r e a r e s i t u a t i o n s where " T h e r e a p p e a r s t o be no c r e d i b l e s u b s t i t u t i o n f o r c o l l e c t i n g a d d i t i o n a l d a t a o r l i m i t i n g t h e g o a l s o f t h e s t u d y when m u l t i c o l l i n e a r i t y i s a t t r i b u t e d t o o v e r d e f i n e d m o d e l s . " 4.1.5 O u t l i e r s [PREGIBON,81] shows t h a t o u t l i e r s i n a g e n e r a l i z e d l i n e a r model may be d e t e c t e d by e x a m i n i n g t h e d i a g o n a l e l e m e n t s o f t h e g e n e r a l i z e d p r o j e c t i o n m a t r i x M = I - W" 1X(X TWX>- 1X' TW So u s i n g s i m i l a r c r i t e r i a t o t h o s e i n t h e s t a n d a r d l i n e a r c a s e , t h e o f f e n d i n g d a t a c a n be d r o p p e d , s o a l l e v i a t i n g t h e c o l l i n e a r i t y . 4.2 Remedies f o r C o l l i n e a r E f f e c t s The m a j o r o b s e r v a b l e e f f e c t o f c o l l i n e a r i t y i s t h e i l l c o n d i t i o n i n g o f t h e X m a t r i x , and i n moat p r a c t i c a l c a s e s t h e r e may be no o b v i o u s r e a s o n s f o r t h e c o l l i n e a r i t y o t h e r t h a n complex i n t e r r e l a t i o n s h i p s i n t h e p r e d i c t o r v a r i a b l e s . In t h i s c a s e a t r a d e o f f between i n c r e a s e d b i a s and r e d u c e d v a r i a n c e ( i e . r i d g e e s t i m a t i o n ) may be w o r t h w h i l e . 4.3 E s t i m a t i o n Methods 4.3.1 R i d g e E s t i m a t i o n H i s t o r i c a l l y r i d g e a n a l y s i s stemmed f r o m e s t i m a t i o n methods d e v e l o p e d t o h a n d l e t h e i n s t a b i l i t y o f r e s p o n s e s u r f a c e c o e f f i c i e n t s l y i n g a l o n g t h e r i d g e o f a c a n o n i c a l s u r f a c e tHOERL.,62]. The s i m p l e s t e s t i m a t o r i s t h e b i a s e d r i d g e e s t i m a t o r ftR = <X rX + k I ) - J X r y a s p r o p o s e d i n t h e p i o n e e r i n g p a p e r CHOERL,KENNARD,703. T h a t p a p e r shows t h e r e e x i s t s a k<:, s u c h t h a t t h e mean s q u a r e e r r o r o f t h e r i d g e e s t i m a t o r i s l e s s t h a n t h a t o f t h e o r d i n a r y l e a s t s q u a r e s e s t i m a t o r , i e . an i n t r o d u c t i o n o f b i a s r e s u l t s i n a r e d u c e d mean s q u a r e e r r o r . Numerous e m p i r i c a l methods have been d e v e l o p e d f o r c h o o s i n g k, and s e v e r a l v a r i a n t s o f t h e s i m p l e e s t i m a t o r have been s u b s e q u e n t l y p r o p o s e d . T h e r e has been much c r i t i c i s m o f r i d g e e s t i m a t i o n . [SMITH,CAMPBELL,80] p75 i s a s p i r i t e d a t t a c k a g a i n s t t h e use o f r i d g e r e g r e s s i o n a s " m e c h a n i c a l d a t a m a n i p u l a t i o n t h a t i s i n s e n s i t i v e t o t h e p a r t i c u l a r phenomena b e i n g modeled and t o i n f o r m a t i o n a b o u t c o e f f i c i e n t s " They a r g u e t h a t r i d g e e s t i m a t o r s a r e b a s e d on "adhoc p s e u d o i n f o r m a t i o n " , and i n s t e a d a d v o c a t e a semi B a y e s i a n a p p r o a c h . However t h e c r i t i c s o f t h i s p a p e r p o i n t o u t t h a t , w h i l e t h e p r i o r i n f o r m a t i o n h e l d i s t o o weak t o be f o r m u l a t e d i n t e r m s o f a p r i o r d i s t r i b u t i o n ; i t i s n e v e r t h e l e s s e s s e n t i a l and r i d g e r e g r e s s i o n a p p r o x i m a t e l y i n c o r p o r a t e s t h i s i m p r e c i s e i n f o r m a t i o n . F u r t h e r m o r e , a s [GUNST,80] plOO n o t e s " r i d g e r e g r e s s i o n has been s u c c e s s f u l l y a p p l i e d t o o f r e q u e n t l y , when l i t t l e p r i o r i n f o r m a t i o n i s a v a i l a b l e , f o r i t s use t o be r e s t r i c t e d o n l y t o d a t a f o r w h i c h f o r m a l B a y e s i a n p r i o r s a r e known" 4.3.1.1 S t a n d a r d L i n e a r C a s e In t h e i r d e r i v a t i o n o f a r i d g e e s t i m a t o r CHOERL,KENNARD,70] r e a s o n a s f o l l o w s " t h e worse t h e c o n d i t i o n i n g o f X TX, t h e more R c a n be e x p e c t e d t o be t o o l o n g . On t h e o t h e r hand, t h e worse t h e c o n d i t i o n i n g , t h e f u r t h e r one c a n move from R w i t h o u t an a p p r e c i a b l e i n c r e a s e i n t h e r e s i d u a l sum o f s q u a r e s . In v i e w o f E [ B T 6 ] = B T B + o e t r ( X r X ) - 1 i t seems r e a s o n a b l e t h a t i f one moves away f r o m t h e minimum sum o f s q u a r e s p o i n t , t h e movement s h o u l d be i n a d i r e c t i o n w h i c h w i l l s h o r t e n t h e l e n g t h o f t h e r e g r e s s i o n v e c t o r . " So t h e r i d g e e s t i m a t o r i s t h e s h o r t e s t fi whose sum o f s q u a r e s due t o e r r o r i s c o n s t r a i n e d t o be w i t h i n d'~ s a y o f t h a t o f t h e o r d i n a r y l e a s t s q u a r e s e s t i m a t o r B0u.a i e . d6* = SSE(fi R) - SSECrW.;.) A A A A = ( y - y > T ( y - y ) - ( y - ycii....ia > 1 < y - y«.i !>> A A (y - X B « ) T ( y - X1W - (y - X B o L . m > T ( y - X f l D L B ) = ( B R - Ba,_a> TX TX<BH - BO L . S ) «• 2 ( f l B - B o L B ) T X T ( y - B O L B ) A A = <B R - BOL ) "r X T X ( BR - fioLs) [ n o r m a l e q u a t i o n s ] Thus t h e r i d g e t r a c e i s t h e s o l u t i o n o f m i n i m i s e B R T f l R s u b j e c t t o (fi„ - Boi ,-,) T X r X ( B R - B(:;„ ra> < d's Now t h e L a g r a n g i a n e x p r e s s i o n w i t h m u l t i p l i e r 1/k i s L = fiRTBR + l / k ( B B - fia,..E,)TXTX(fl« - fir„_„> w h i c h g i v e s h as s o l u t i o n o f t h e f o r m B R = ( X r X + k I ) - * B o t . a t h e s i m p l e r i d g e e s t i m a t o r . 4.3.1.2 G e n e r a l L i n e a r C a s e T h i s e x t e n d s t o t h e g e n e r a l s e t t i n g a s f o l l o w s w i t h t h e f o l l o w i n g d e f i n i t i o n o f a r i d g e e s t i m a t o r Bp, d e f i n e d i n t e r m s o f t h e u s u a l i t e r a t i v e l y r e w i g h t e d l e a s t s q u a r e s s o l u t i o n A fli ni...ca • D e f i n i t i o n 4.1 R i d g e E s t i m a t i o n f o r t h e g e n e r a l i z e d l i n e a r model y = g _ 1 ( X B ) + € i s g i v e n b y t h e s o l u t i o n m i n i m i s e B R T B R S u b j e c t t o (GR — GIRL.)=i) r X T W X ( B R - B T h i s h as s o l u t i o n 56 A A ft R = (X TWX + k I ) - 1 X T W X B I R L e The p r o o f o f t h i s f o l l o w s by e x t e n d i n g CSCHAEFER,79] t o t h e A g e n e r a l s e t t i o n g a s f o l l o w s . Brm.™ i s t h e i t e r a t i v e l y r e w e i g h t e d l e a s t s q u a r e s e s t i m a t o r , w i t h t r a n s f o r m a t i o n X = W^X , i n t h e s e n s e t h a t i t m i n i m i s e s t h e w e i g h t e d sums o f s q u a r e s due t o e r r o r WSSE = (y - g 1 ( Xft) ) rW - 1 (y - g ' M X B ) ) A g a i n t h e sum o f s q u a r e s i s c o n s t r a i n e d t o w i t h i n s a y d , ; s o f t h e u s u a l e s t i m a t o r ftxm_ia- Hence A d'a = WSSE ( f l R ) - WSSE (ft t u t * , ) (y - g 1 ( X f W > 1W 1 (y - g 1 ( X 6 R ) > + (y - g- 1 (Xft, „,,,) ) 'W M y - g - 1 ( Xft TR1_.s> ) (y - p (fti=») ) T W - 1 (y - H ( f t , » > ) -(y - | i ( S , H L B ) ) T t t - l ( y - p ( ft T m in ) ) ( p ( B « ) - H ( ft :i R L B ) T'W""1 ( p (BR) - p ( B Tm .„> + A A 2 ( p ( B « ) - H ( B T ) TW M y - p ( B x R L 1 3 ) Now >Ji(Bn) and p (B J: r<i_.!=>) c a n be a p p r o x i m a t e d by f i r s t o r d e r v e c t o r T a y l o r s e r i e s a s p ( B R ) = p ( f i ) + WX(fln - B) A p (fi p ( f t ) + WX(ft ft) so t h e above s i m p l i f i e s t o A A d e = (ft« - fi1RLo)TXTWX(flR - fttm _B> + <B R - ft,: «,...=>) T X r ( y - W X B t m „> and t h e g e n e r a l i z e d l i n e a r model l i k e l i h o o d e q u a t i o n s i m p l i f i e s i t f u r t h e r t o A A ( B R - fii„LB)TXTWX(BR - ftTBUB> So f o l l o w i n g tHOERL,KENNARD,701 i n an a n a l o g o u s manner g i v e s t h e m i n i m i s a t i o n r e s u l t and u s i n g a L a g r a n g e m u l t i p l i e r t h e r e s u l t f o l l o w s . Now [HOERL,KENNARD,703 g i v e a j u s t i f i c a t i o n f o r t h e i r method i n an e x i s t e n c e - c o n s t r u c t i o n t h e o r e m . The e x i s t e n c e r e s u l t shows t h e r e i s a non z e r o v a l u e o f k, s u c h t h a t t h e mean s q u a r e e r r o r o f t h i s e s t i m a t o r i s l e s s t h a n t h a t o f t h e o r d i n a r y l e a s t s q u a r e s e s t i m a t o r . T h i s r e s u l t e x t e n d s t o t h e g e n e r a l s e t t i n g a s i s shown i n t h e f o l l o w i n g t h e o r e m . A A Theorem 4.2 E x i s t e n c e o f k Q : MSE ( B R (k r a) ) £. M S E ( f i i R L f i l ) f o r g e n e r a l i z e d l i n e a r model y = g _ 1 ( X f l ) + € I f ( i ) X i j a r e u n i f o r m l y bounded ( i i ) ( X T V X ) i s o f o r d e r 0 ( l / n a ) ( i i i ) V (u) V = ^- "v bounded dp ( i v ) 3 r d and 4 t h c e n t r a l moments o f € e x i s t Then f o r n s u f f i c i e n t l y l a r g e t h e r e e x i s t s A A k.=, 2 0 : M S E ( f i R ( k c , ) ) <L MSE(fi I R L..e) T h i s t h e o r e m was o r i g i n a l l y p r o v e d by [SCHAEFER,791 f o r t h e c a s e o f l o g i s t i c r e g r e s s i o n . The e x t e n s i o n o f t h e p r o o f t o t h e g e n e r a l c a s e a bove, i s s t r a i g h t f o r w a r d i f c o n d i t i o n ( i i i ) i s i n c l u d e d and t h e a n a l o g u e between t h e 1 s t and 2nd moments i n l o g i s t i c r e g r e s s i o n e s t i m a t i o n and t h o s e i n t h e g e n e r a l s e t t i n g i s r e c o g n i s e d . T h i s p r o o f w i l l be o m i t t e d a s t h e m u l t i v a r i a t e a l g e b r a i s l o n g and t e d i o u s . As w i t h t h e s t a n d a r d l i n e a r c a s e , i t must be n o t e d t h a t w h i l e t h e t h e o r e m i s a p p e a l i n g t h e o r e t i c a l l y i t d o e s n o t g i v e any a s s u r a n c e t h a t k can be c h o o s n o p t i m a l l y f r o m t h e d a t a . F u r t h e r i t i s o f l i m i t e d v a l u e a s no bounds have been f o u n d f o r k a s y e t . A l s o s i n c e t h e d e g r e e o f c o l l i n e a r i t y d e p e n d s s o l e l y upon t h e s i n g u l a r v a l u e s o f X, i t i s d i s t u r b i n g t o have t h e s i d e c o n d i t i o n f o r s u f f i c i e n t l y l a r g e n p r e s e n t . So one i s l e d back t o t h e a l t e r n a t i v e e m p i r i c a l argument o f [GUNST,83] r e f e r e n c e d e a r l i e r . [HOERL,KENNARD,701 g i v e a c o n s t r u c t i o n s e c t i o n i n t h e i r t h e o r e m , where t h e y d e d u c e bounds f o r k,::,. However [SCHAEFER,791 p r o d u c e d a c o u n t e r e x a m p l e t h a t shows, i n t h e l o g i s t i c r e g r e s s i o n c a s e , a c o n s t r u c t i o n r e s u l t i s n o t a c h i e v a b l e . So i t i s n o t p o s s i b l e t o g e t any g u i d a n c e on t h e c h o i c e o f k i n t h e g e n e r a l s e t t i n g . However by a n a l o g y w i t h t h e s t a n d a r d l i n e a r c a s e , s e v e r a l c a n d i d a t e s a r e s u g g e s t e d . [MONTGOMERY,PECK,821 p340 i n r e v i e w i n g v a r i o u s r u l e s s t a t e t h a t "no s i n g l e p r o c e d u r e emerges f r o m t h e s e s t u d i e s a s b e s t o v e r a l l . . . Our own p r e f e r e n c e i n p r a c t i c e i s f o r o r d i n a r y r i d g e r e g r e s s i o n w i t h k s e l e c t e d by i n s p e c t i o n o f t h e r i d g e t r a c e ... I t i s a l s o o c c a s i o n a l l y u s e f u l t o f i n d t h e "optimum" v a l u e o f k s u g g e s t e d by H o e r l , K e n n a r d and B a l d w i n [1975] and t h e i t e r a t i v e l y e s t i m a t e d "optimum" o f H o e r l and K e n n a r d [1976] and compare t h e r e s u l t i n g models w i t h t h e one o b t a i n e d v i a t h e r i d g e t r a c e . " Hence t h e f o l l o w i n g e x t e n d e d d e f i n i t i o n D e f i n i t i o n 4.IB S e l e c t i o n o f k i n R i d g e E s t i m a t i o n f o r t h e g e n e r a l l i n e a r model y = g — 1 ( X B ) + € R u l e I : k i s t h a t v a l u e w h i c h c a u s e s t h e e s t i m a t e fi^OO t o s t a b i l i s e , and i s f o u n d by i n s p e c t i o n o f t h e r i d g e t r a c e i e . B R ( k ) p l o t t e d a g a i n s t k. R u l e I I : [HOERL,KENNARD,BALDWIN,75] k where 0 i s t h e d i s p e r s i o n p a r a m e t e r e s t i m a t e o f t h e g e n e r a l i z e d l i n e a r m odel. R u l e I I I : [HOERL,KENNARD,76] A l g o r i t h m [ R u l e I I ] REPEAT i = l ( l ) UNTIL c o n v e r g e n c e ft<k 1 1> ) T f i ( k <* > ) R ( k < * - 1 > ) = <X'rWX + k < 1 + 1 > I) - 1 X'Wy END REPEAT Th e r e i a a l e g i o n o f o t h e r methods t o c h o o s e f r o m , i n c l u d i n g t h e r i c h e r method o f g e n e r a l i z e d r i d g e e s t i m a t i o n , b u t t h e above t h r e e methods s e r v e t o i l l u s t r a t e t h e g e n e r a l i d e a . 4.3.2 B a y e s i a n E s t i m a t i o n 4.3.2.1 S t a n d a r d L i n e a r C a s e tLINDLEY,SMITH,72] show, i n t h e s t a n d a r d l i n e a r c a s e , i f y has t h e n o r m a l d i s t r i b u t i o n N P ( X B , u i RI> and ft has t h e m u l t i v a r i a t e n o r m a l p r i o r d i s t r i b u t i o n NCO, a, a e> t h e n t h e p o s t e r i o r mean o f ft i s ft = [ X r X + k l l - ' X ' ^ where k = o^/Ufa** So by a d d i n g more i n f o r m a t i o n i n t e r m s o f a p r i o r d i s t r i b u t i o n f o r ft a r i d g e t y p e e s t i m a t o r i s o b t a i n e d . The r e s u l t s o f t h i s p a p e r a r e m o t i v a t e d by a s t r i c t l y B a y e s i a n p h i l o s o p h y , p i v o t i n g a r o u n d t h e c o n c e p t o f e x c h a n g e a b l e p r i o r s . U n f o r t u n a t e l y t h i s c o n c e p t i s a l i t t l e u n p a l a t i b l e f o r a p p l i e d work, s i n c e i t means t h e ft must come from a s t a n d a r d m u l t i v a r i a t e n o r m a l d i s t r i b u t i o n . T h i s i m p l i e s a s t a n d a r d i s e d model must a l w a y s be u s e d , w h i c h d e t r a c t s f r o m t h e method. However CNELDER,72], i n r e v i e w i n g [LINDLEY,SMITH,721, p o i n t s o u t t h a t t h e B a y e s i a n f o r m u l a t i o n i s e q u i v a l e n t t o t h e c o n c e p t o f p r i o r l i k e l i h o o d [EDWARDS,69] p o s t e r i o r l i k e l i h o o d = p r i o r l i k e l i h o o d * l i k e l i h o o d T h i s i s a s i m p l e r a p p r o a c h and f i t s i n b e t t e r w i t h t h e l i k e l i h o o d framework o f g e n e r a l i z e d l i n e a r m o d e l s . 4.3.2.2 G e n e r a l L i n e a r C a s e In t h e a b s e n c e o f e v i d e n c e f r o m o t h e r e x p e r i m e n t s , t h e c h o i c e o f a n o r m a l p r i o r f o r p r i o r l i k e l i h o o d i s a n a t u r a l one s i n c e a s i s known t h a t t h e s a m p l i n g e s t i m a t e s fi a r e a s y m p t o t i c a l l y n o r m a l . So u s i n g t h e p h i l o s o p h y o f [EDWARDS,691 t h e p o s t e r i o r l o g l i k e l i h o o d o f fi i s 1F, = ( S f y i X i f l - b(x : LB)"| . _. , a70> J + c<y : L,0) ) • where t h e f i r s t t e r m i s t h e l o g l i k e l i h o o d o f t h e g e n e r a l i z e d e x p o n e n t i a l d i s t r i b u t i o n a s i n d e f i n i t i o n 1.3, and t h e s e c o n d t e r m i s t h e p r i o r l o g l i k e l i h o o d u s i n g a n o r m a l p r i o r w i t h p a r a m e t e r s (O.dra^). The maximum l i k e l i h o o d e s t i m a t e s a r e o b t a i n e d f r o m t h e p a r t i a l d e r i v a t i v e s and a r e p r e s e n t e d i n A p p e n d i x 4A. Hence t h e f o l l o w i n g d e f i n i t i o n o f a p o s t e r i o r mode e s t i m a t o r . D e f i n i t i o n 4.2 P o s t e r i o r Mode E s t i m a t o r f o r t h e g e n e r a l i z e d l i n e a r model y = g _ 1 ( X f i ) + € i s t h e v a l u e fif-., t h a t m a x i m i s e s t h e d e n s i t y e x p r e s s i o n = ( £ I" y * « , B - b < x ± i » 1 + C ( Y I , 0 > , • i - i [ a ( 0 ) J C J « . S : " 2 ^logo,;,1* } 4.3.3 P r i n c i p a l Component E s t i m a t i o n 4.3.3.1 S t a n d a r d L i n e a r C a s e P r i n c i p a l components, i n t h e s t a n d a r d l i n e a r c a s e , p r o d u c e s . b i a s e d e s t i m a t e s by u s i n g a r e d u c e d s e t o f p r e d i c t o r v a r i a b l e s , where t h e v a r i a b l e s removed a r e t h o s e c o r r e s p o n d i n g t o s m a l l s i n g u l a r v a l u e s i n t h e d e s i g n m a t r i x . The r e s u l t s a r e e a s i e s t d e r i v e d by c o n s i d e r i n g t h e c a n o n i c a l f o r m o f t h e s t a n d a r d l i n e a r model, i e . y~ = X - f l * + € where X" = XT X rX •= T"rT T ' X 1 XT = A A = ['Xi. , . . . , X p ] e i g e n v a l u e s o f X T X T = [ t i , . . . , t r;,l e i g e n v e c t o r s o f X'1 X = T'JWra Then t h e p r i n c i p a l components e s t i m a t o r i s d e f i n e d a s A O P C = TBB~ where B = d l a g ( b i bP,} bi 0 \ : l " s m a l l " 1 e l s e So s u b s t i t u t i n g f o r B" f r o m above A TBT rB ( TBT"1" ( X 1 X ) - 1 X 1 XBo....m TBT"' TA 1 T T X T X f l n u a T BA ~" 1 T r X r X B o i... TA- •••1T"rX"rXB,: [*,/ X'X 1 t d i a g o n a l i s a t i o n ( X ^ X ) ' 1 ] [T o r t h o g o n a l 3 [ A - 1 = BA — 1 3 (XTx>-xrxrw.ra 1=1—<« ^ where (X TX)"" = T - - t i t t T i s an a p p r o x i m a t e i. "•• .1 A j. i n v e r s e Now V a r ( B p c ) = V a r (TBG •*) = T B V a r ( B " ) B T T T [ V a r ( A X ) = A V a r ( X ) A T ] TB ( X» * X * ) - 1 B r T rCT^- [by c o n s t r u c t i o n ] TBA - 'B"' T 1 "a™ [ d i a g o n a l i s a t i o n 3 w h i c h i s t o be compared w i t h t h e v a r i a n c e o f t h e o r d i n a r y l e a s t s q a u r e s e s t i m a t o r V a r ( f l m . a ) = TA-^T^tr* Hence p r i n c i p a l components a l l e v i a t e s t h e c o n s e q u e n c e s o f c o l l i n e a r i t y i n t h e v a r i a n c e s o f t h e e f f e c t s , by r e m o v i n g t h e l a r g e t e r m s due t o t h o s e v a r i a b l e s c o r r e s p o n d i n g t o s m a l l s i n g u l a r v a l u e s . 4.3.3.2 G e n e r a l L i n e a r C a s e In a s i m i l a r manner i t i s p o s s i b l e t o e x t e n d t h i s d e f i n i t i o n t o t h e g e n e r a l s e t t i n g a s f o l l o w s . v D e f i n i t i o n 4.3 P r i n c i p a l Components E s t i m a t o r f o r t h e g e n e r a l l i n e a r model y = g _ 1 ( X f l ) + € i s g i v e n by A A f W = (X TWX)-X TWXB 1 R L. B where (X TWX)'" = TA — 1 T 'r A = [ \ i X r J e i g e n v a l u e s o f XrWX T = t t i , . . . , t,,] e i g e n v e c t o r s o f XrWX B = d i ag (b , b F )) 0 X t " s m a l l " 1 e l s e Hence by d i r e c t a n a l o g y t h e e f f e c t s o f c o l l i n e a r i t y on t h e v a r i a n c e s o f t h e e s t i m a t e s i n t h e g e n e r a l i z e d l i n e a r model a r e s i m i l a r l y r e d u c e d . 4.4 A p p e n d i x 4A Maximum L i k e l i h o o d E s t i m a t e s o f t h e P o s t e r i o r D i s t r i b u t i o n The maximum l i k e l i h o o d e q u a t i o n s a r e o b t a i n e d i n t h e u s u a l manner by d i f f e r e n t i a t i n g w i t h r e s p e c t t o t h e p a r a m e t e r s . However t h e p r o b l e m i s t h e n s l i g h t l y non s t a n d a r d b e c a u s e t h e r e i s no a s s o c i a t e d d i s t r i b u t i o n t o u s e f o r t h e e x p e c t e d v a l u e s i n o b t a i n i n g t h e i n f o r m a t i o n m a t r i x . However [PATEFIELD,771 shows t h a t when n u i s a n c e p a r a m e t e r s l i k e t h e v a r i a n c e o f B a r e i n v o l v e d , t h e t h e f o r m a t i o n m a t r i x o f [EDWARDS,69a] i s more a p p r o p r i a t e . He f u r t h e r shows t h a t i t i s a c c e p t a b l e t o remove t h e n u i s a n c e p a r a m e t e r by s u b s t i t u t i n g i t o u t , u s i n g t h e maximum l i k e l i h o o d e q u a t i o n s . Hence t h e f o l l o w i n g c a l c u l a t i o n s 1 P = I £ f V x X i B - b ( X i f i ) ] _ A , i . L — a T c 5 ) J + c < V - 0 ) > ( —-s- - | l o g a * ~ ) So t h e f o l l o w i n g p a r t i a l d i f f e r e n t i a l s a r e o b t a i n e d Q l t , _ K y i - H i > X i _ flr,_ ©ft,., 1 8 a,3iz 0 1 R = B r B _ _p @am fii 2 ( a r , . ) , s 2arae 01, 00 [" £ y 1 x 1 f l - b(x : L f l ) + @c(y:l. ,0)1 ~ L 0 a + 00 J X i , ~ X i s [Op JH V : l L0n A J = z i i r J L a f t i - [ 6 „ s K r o n e c k e r d e l t a ] 0fl v.0B s v l | A crB M 66 S e l P S a l p Z < y i • - p i > x i - Z ( y t - H i ) x : By • B T B ( C T r a a ) : a ( CT , A E > E l i m i n a t i n g t h e n u i s a n c e p a r a m e t e r af3* e q u a t i o n s g i v e s B J B P u s i n g t h e maximum The s c o r e v e c t o r i s ©fi ©0 and t h e f o r m a t i o n m a t r i x i s 0B0B f l p 0B00 ©Jiie 0B00 0 S :1 F, @0 C w i t h t h e u s u a l s c o r i n g e q u a t i o n b e i n g fi 0 fi 0 + F - ^ s 5.0 I l l u a t . r a t . i v e Example 67 5.1 I n t r o d u c t i o n T h e r e a r e two a p p r o a c h e s t o v a l i d a t i n g t h e methods d e v e l o p e d i n t h e p r e v i o u s c h a p t e r s . One a p p r o a c h would be t o o b t a i n a l a r g e d a t a s e t w h i c h has c o l l i n e a r symptoms. The d i a g n o s t i c m e a sures o f c h a p t e r two c o u l d t h e n be us e d t o v e r i f y t h i s , and t h e methods o f c h a p t e r t h r e e a p p l i e d t o o b t a i n e s t i m a t o r s . However, t h i s a p p r o a c h has t h e d i s a d v a n t a g e , t h a t i t i s n o t p o s s i b l e t o d i r e c t l y c a l c u l a t e t h e mean s q u a r e e r r o r , s i n c e t h e " t r u e " c o e f f i c i e n t s a r e unknown. B e c a u s e o f t h i s , a s i m u l a t i o n h as been c h o o s e n . 5.2 S c o p e o f t h e S i m u l a t i o n O t h e r s i m u l a t i o n s s u c h a s [DEMPSTER,SCHATZOFF,WERMOUTH,771 , [McDONALD,GALARNEAU,751 and [SCHAEFER,82] have s t u d i e d t h e e f f e c t s o f s e v e r a l p a r a m e t e r s - sample s i z e <n), number o f p r e d i c t o r v a r i a b l e s <P>, d e g r e e o f c o l l i n e a r i t y ( d ) , a l i g n m e n t o f fi ( a ) , e t c . B u t t h i s i s beyond t h e r e s o u r c e s o f t h i s t h e s i s , and s o a r e s t r i c t e d s i m u l a t i o n , t h a t s e r v e s t o i l l u s t r a t e some o f t h e methods o f c h a p t e r f o u r , has been a d o p t e d . In t h i s s i m u l a t i o n p a r a m e t e r s n, p, d a r e r e s t r i c t e d t o one l e v e l and a t o two l e v e l s . 5.3 G e n e r a t i o n o f C o l l i n e a r D a t a 5.3.1 S t a n d a r d L i n e a r C a s e The p r o c e d u r e t h a t has been f o l l o w e d i n many s i m u l a t i o n s ( e g . [McDONALD,GALARNEAU,751, [HEMMERLE,BRANTLE,78], [WICHERN,CHURCHILL,78), [GIBBONS,81]) has been t o g e n e r a t e a c o l l i n e a r d e s i g n m a t r i x f r o m a t h e o r e t i c a l l i n e a r d e p e n d e n c y . Two " t r u e " c o e f f i c i e n t v e c t o r s have t h e n been s e l e c t e d , a s t h e e i g e n v e c t o r s c o r r e s p o n d i n g t o t h e s m a l l e s t and l a r g e s t e i g e n v a l u e s o f t h e X T X s y s t e m . I t i s known t h a t t h e mean s q u a r e e r r o r i s m a x i m i s e d and m i n i m i s e d r e s p e c t i v e l y , w i t h t h e s e c h o i c e s . [GIBBONS,81] c i t e s [THISTED,76] p74 a s j u s t i f i c a t i o n f o r t h e s e c h o i c e s " " e x t r e m e - c a s e s i m u l a t i o n " e x p e r i m e n t s a p p e a r t o be t h e most e c o n o m i c a l and i n f o r m a t i v e , e s p e c i a l l y f o r p r e l i m i n a r y s t u d i e s o f new r e s u l t s " F i n a l l y , t h e r e s p o n s e v e c t o r i s g e n e r a t e d a s t h e sum o f t h e s y s t e m a t i c p a r t due t o Xfl and a random n o r m a l e r r o r . Now much c r i t i c i s m has been l e v e l l e d a g a i n s t t h i s d e s i g n e g . [DRAPER,VAN NOSTRAND,79] and much work nee d s t o be done t o c o n s t r u c t a f a i r b a s i s f o r c o m p a r i s o n . However, t h i s i s beyond t h e r e s o u r c e s o f t h i s t h e s i s . 5.3.2 G e n e r a l L i n e a r C a s e The p r o b l e m o f g e n e r a t i n g a c o l l i n e a r d a t a s e t from a g e n e r a l i z e d l i n e a r model i s n o t a s s i m p l e a s i n t h e s t a n d a r d l i n e a r c a s e , s i n c e f o r t h e p s e r i e s t h e X r X s y s t e m must be u s e d . A s o l u t i o n , i n a s i m i l a r manner t o t h e s t a n d a r d l i n e a r c a s e m i g h t a p p e a r t o be t o g e n e r a t e X a s c o l l i n e a r , s e l e c t i n g ft a s t h e e i g e n v e c t o r c o r r e s p o n d i n g t o t h e s m a l l e s t e i g e n v a l u e o f X T X . Then t h e r e l a t i o n X = w'^(X,ft)X c o u l d be u s e d t o r e c o v e r t h e o r i g i n a l X m a t r i x . However t a k i n g ft a s t h e e i g e n v e c t o r s o f X rX, w h i c h i s i t s e l f a f u n c t i o n o f ft l e a d s t o an o v e r c o n s t r a i n e d s y s t e m w i t h no non t r i v i a l s o l u t i o n . Hence i t i s n o t p o s s i b l e t o c h o o s e t h e a l i g n m e n t o f ft as recommended by tTHISTED,761. The a l t e r n a t i v e i s t o s t a r t w i t h X c o l l i n e a r and s e l e c t ft s o a s t o m a i n t a i n c o l l i n e a r i t y i n X by r e a l i s i n g t h a t p r e m u l t i p l i c a t i o n by i s e f f e c t i v e l y row s c a l i n g o f X. F o r c o n v e n i e n c e t h e c a s e o f t h e gamma d e n s i t y w i t h u n i t d i s p e r s i o n p a r a m e t e r ( i e . 0=1) i s c h o s e n . The method o f [GIBBONS,811 i s u s e d t o g e n e r a t e X w i t h a c o n t r o l l a b l e d e g r e e o f c o l l i n e a r i t y . T h i s method i s as f o l l o w s . A l g o r i t h m 5.1 M o d i f i e d [GIBBONS,81] C o l l i n e a r G e n e r a t o r o f X STEP 1 G e n e r a t e z t . j a s i n d e p e n d e n t N<0,1> paeudo random v a r i a b l e s i = l ( l ) n i = l ( l ) p + l STEP 2 S e l e c t ct where a l S i s t h e c o r r e l a t i o n between any two v a r i a b l e s Compute x t j = (1 - o ( e ) Z i j + o z i , t;,,. j i = l ( l ) n j = l ( l ) p STEP 3 Compute t h e X m a t r i x a s X = W^X where w:L^ = l / x t ' r B I n t h e g e n e r a l i z e d l i n e a r model y = g 1 ( X B ) + € t h e r e s p o n s e y i s t h e sum o f t h e s y s t e m a t i c component g- 1 ( X f i ) and t h e e r r o r term €, t h a t h as t h e one p a r a m e t e r e x p o n e n t i a l d i s t r i b u t i o n . So i t i s n o t m e a n i n g f u l t o add a s p e c i f i c e r r o r t e r m t o t h e s y s t e m a t i c component t o g e n e r a t e y. R a t h e r y c a n be t h o u g h t o f as h a v i n g come f r o m a one p a r a m e t e r e x p o n e n t i a l d i s t r i b u t i o n w i t h mean v e c t o r p = g " 1 ( X f i ) . So f o r t h e gamma d i s t r i b u t i o n c o n s i d e r e d , t h e mean v e c t o r i s H ± = l / X i T B 5.4 S i m u l a t i o n 5.4.1 S i m u l a t i o n S e t u p The model t o be c o n s i d e r e d i n t h e s i m u l a t i o n i s y = g 1 ( X f t ) + € where y : n x l gamma v a r i a t e d i s p e r s i o n p a r a m e t e r 0=1 X : nxp n : sample s i z e = 30 p : number p r e d i c t o r v a r i a b l e s ( i n c l u d i n g i n t e r c e p t ) = 4 fi : p x l 2 l e v e l s »<•--> fi<«a> fi < i-. > = O.Ol O.Ol 0.01 O.Ol fi < ™ -0.001 0.01 •0.01 O.Ol Two l e v e l s were c h o s e n f o r t h e d e g r e e o f c o l l i n e a r i t y d a s measured by a'"'-, t h e c o r r e l a t i o n c o e f f i c i e n t between any two p r e d i c t o r v a r i a b l e s d<*> = 0 . 8 0 d " s > = 0 . 9 5 fi<L>, ft<is>> were c h o o s e n a s above f o r t h e f o l l o w i n g r e a s o n . G i v e n X g e n e r a t e d by A l g o r i t h m 5.1 and fi<lchoosen as above t h e e i g e n v e c t o r s c o r r e s p o n d i n g t o t h e s m a l l e s t and l a r g e s t e i g e n v a l u e s ( t * 1 - * , t < r a > ) o f X w i t h d a t t h e two l e v e l s a r e t <u> = 0.80 £ < la > t<' - > = 0.95 -0.63 0.28 -0.48 -0.49 -0.43 0.65 -0.44 -0.50 -0.75 0.05 -0.39 -0.34 -0.36 0.78 -0.37 -0.52 Now aa d i s c u s s e d above i t i s n o t p o s s i b l e t o a l i g n B w i t h t h e e i g e n v e c t o r s o f X ( B > a n a l y t i c a l l y , b u t by t h e s u i t a b l e c h o i c e o f B <'- > i t i s s e e n t h a t &<>•-> and B < E " a r e a p p r o x i m a t e l y a l i g n e d i n t h e d i r e c t i o n s o f t h e r e s p e c t i v e e i g e n v e c t o r s o f X ( B < L > ) f o r d = 0.95. As l i t t l e improvement i s e x p e c t e d w i t h t h e B < i s " a l i g n m e n t o n l y t h e 1 s e t t i n g ( B <»> , d <•=> > i s us e d . A p i l o t s i m u l a t i o n o f t e n r u n s was made t o e s t i m a t e a r e a s o n a b l e r u n sample s i z e . The c r i t e r i o n was t h a t o f b e i n g a b l e t o d e t e c t a 10 p e r c e n t c hange i n t h e e s t i m a t e s B a t t h e 5 p e r c e n t l e v e l . T h i s gave a r u n s i z e (m) o f 20. B e c a u s e o f r e s o u r c e c o n s t r a i n t s , j u s t t h r e e e s t i m a t o r s were c o n s i d e r e d B I R L O : i t e r a t i v e l y r e w e i g h t e d l e a s t s q u a r e s e s t i m a t o r B R I 1 : [HOERL,KENNARD,BALDWIN,751 r i d g e e s t i m a t o r B,=.c: : p r i n c i p a l components e s t i m a t o r To a s s e s s t h e e f f e c t i v e n e s s o f t h e e s t i m a t o r s , t h e u s u a l c r i t e r i o n o f t h e s o c a l l e d a v e r a g e s q u a r e e r r o r ASE(fi) = (fi - fi)T(fi - fi) i s u s e d . Now i t i s p o s s i b l e t o o b t a i n a p p r o x i m a t i o n s f o r t h e v a r i a n c e o f t h e r i d g e e s t i m a t o r s t o o r d e r OCl/n-"} ( s e e [SCHAEFER,79] c h a p t e r I I ) i n t e r m s o f t h e t h i r d and f o u r t h moments o f t h e d i s t r i b u t i o n o f y. However t h i s i s o m i t t e d h e r e , b u t s h o u l d be c o n s i d e r e d i n any f u t u r e work. 5.4.2 S i m u l a t i o n I m p l e m e n t a t i o n and D i f f i c u l t i e s The s i m u l a t i o n was d i v i d e d i n t o t h r e e s t a g e s . The f i r s t s t a g e u s e d t h e SAS p r o c e d u r e PROC MATRIX t o g e n e r a t e t h e c o l l i n e a r d a t a s e t s u s i n g A l g o r i t h m 5.1. The s e c o n d s t a g e u s e d t h e g e n e r a l i z e d l i n e a r m o d e ls p a c k a g e GLIM t o g e t t h e i t e r a t i v e l y r e w e i g h t e d l e a s t s q u a r e s e s t i m a t e s . The t h i r d s t a g e a g a i n u s e d PROC MATRIX and c a l c u l a t e d t h e b i a s e d e s t i m a t o r s and s t a t i s t i c s . W h i l e t h e SAS PROC MATRIX r o u t i n e i s p l e a s a n t t o use b e c a u s e o f i t s m a t r i x s t y l e s y n t a x and i s e f f e c t i v e due t o machine c o d i n g o f t h e m a t r i x r o u t i n e s ; i t s i n p u t / o u t p u t a b i l i t y l e a v e s much t o be d e s i r e d . The r e s u l t o f t h i s d e f i c i e n y i s t h a t v e r y l a r g e m a t r i c e s must be s t o r e d ( w i t h numerous c o n s e q u e n t p h y s i c a l i / o p r o b l e m s ) i f e f f i c i e n t use i s t o be made o f t h e i n t e r p r e t e r c o d e . D u r i n g t h e r u n n i n g o f t h e s e c o n d s t a g e , a c o m p u t a t i o n a l f a u l t was f o u n d i n GLIM. T h i s e f f e c t e d a p p r o x i m a t e l y f i v e p r e c e n t o f t h e d a t a s e t s . I t would a p p e a r t h a t when a gamma model i s u s e d w i t h s e v e r e c o l l i n e a r i t y ( a s was h e r e ) , GLIM has no a d j u s t e m e n t f o r n e g a t i v e f i t t e d v a l u e s and s e n s i b l y a b o r t s w i t h t h e w a r n i n g message #****FAULT 59 * * » « » * * * * * » To overcome t h i s e x t r a d a t a s e t s were s u b s t i t u t e d . 5.4.3 S i m u l a t i o n R e s u l t s and C o n c l u s i o n s The r e s u l t s o f t h e s i m u l a t i o n s a r e d i s p l a y e d i n T a b l e s 5.1 and 5.2 and F i g u r e 5.1. T a b l e 5.1 l i s t s i n e a c h c e l l t h e a v e r a g e e s t i m a t e d p a r a m e t e r v a l u e s fii, f i , s , fl3, fi-+ f r o m t h e m r u n s w i t h t h e e s t i m a t o r t y p e by column and t h e d e g r e e o f c o l l i n e a r i t y d c > n e s t e d w i t h i n a l i g n m e n t fi<> by row. T a b l e 5.2 i n a s i m i l a r l a y o u t g i v e s t h e v a l u e s o f t h e a c t u a l s q u a r e d e r r o r and i t s a s s o c i a t e d s t a n d a r d e r r o r i n p a r e n t h e s e s . T h i s t a b l e a l s o d i s p l a y s t h e p e r c e n t a g e r e d u c t i o n r a t i o i n ASE by u s i n g t h e b i a s e d e s t i m a t o r s fi«T. A A and flpc a s o p p o s e d t o t h e u s u a l e s t i m a t o r fiiRL.i=) ( i e . B / B i RI....E)* 100) . In a c c o r d a n c e w i t h t h e sample s i z e c r i t e r i a t h e o n l y c o m p a r i s o n s c o n s i d e r e d were t h e m u l t i p l e c o m p a r i s o n o f t h e t h r e e e s t i m a t o r s w i t h i n e a c h s e t t i n g o f ft< > and d < J . T u k e y ' a p a i r w i s e c o m p a r i s o n t e s t a t t h e f i v e p r e c e n t s i g n f i c a n c e l e v e l i s d i s p l a y e d p i c t o r i a l l y i n T a b l e 5.2 where t h e b a r s r e p r e s e n t homogeneous g r o u p s . F i g u r e 5.1 d i s p l a y s g r a p h i c a l l y t h e s i m u l a t i o n e s t i m a t e s o f ft f o r t h e t h r e e s e t t i n g s (ft <'--> , d < 1 •> ) , ( B < L > , d < e > ) and ( f t , d < l S : > ) . Note w i t h ft<«> t h e s c a l i n g h as been m o d i f i e d t o t h a t o f ft"-' t o make t h e g r a p h e a s i e r t o comprehend. 76 T a b l e 5.1 E s t i m a t o r s f o r t h e Gamma Model fi < L : fi < s > A fi i: Ft i~ra A A fi r~- in: 0 .0128 0. 0107 0. 00940 0 .00941 0. 00958 0 . 0101 0 .0101 0. 00914 0. 0106 0 .0128 0. 0105 0. 0103 0 .0127 0. 0110 0. 00995 0 .00804 0. 0107 0. 0101 0 .00749 0. 00735 0. 0104 0 .0163 0. 0123 0 . O l O l 0 .00763 0. 00413 0. 00175 o .0100 0. 00672 0. 00229 o .0101 0. 00314 -0 . 00279 0 .0105 0. 00548 0 . 00262 1. fi<L>n" = ( O . O l , O . O l , O . O l , O.Ol) B < S > T = (-o.OOl, O.Ol, -0.01, O.Ol) 2. d < A > = 0.80 d < f i : > = 0.95 A A A A 3. E a c h c e l l c o n t a i n s ( f i t , fis, fi3, B ^ ) T T a b l e 5.2 A c t u a l S q u a r e d E r r o r o f E s t i m a t o r s f o r t h e Gamma Model A A I R i... m A B H I T 0.000450 [0.000984 ] 26% A ft (.;-.: ft <L > d < t > 0.00171 C0.00209 ] 0.000167 [0.00315 )3 9% d < a: > d < a > 0.0139 [0.000177 ] 0.00268 [0.00722 )] 19% 0.0000942 [0.000142 3 1% ft < s > 0.000281 [0.000454 3 0.000197 [0.000193 1 55% 0.000286 [O.0000465 3 100% 1. B < L > r = ( O . O l , O . O l , O . O l , O.Ol) ft<S>r = (-o.OOl, 0 . 0 1 , - 0 . 0 1 , O.Ol) 2. d < % > = 0.80 d < c i > = 0.95 3. Each c e l l c o n t a i n s ASE a v e r a g e f o r m r u n s [ s t a n d a r d e r r o r ! r e d u c t i o n r a t i o % u s i n g a b i a s e d e s t i m a t o r 4. Bars represent homogeneous groups with Tukey's pairwise comparison at the 5% l e v e l Figure 5.1 Mean ± Standard Deviation Intervals of Estimators for the Gamma Model 78 (a) ,d"') L E G E N D fii CZ2 1 E 3 3 • 4 IRLS Rll E S T I M A T E S PC2 (b) (fi ,d<a> ) o.eo o.o« 0.07 0.0* 0.00 0.04 C O S u o.ot o 99 + o.oo -0.01 2 •o.ot -0.00 -0.04 - o . o i -o.oa -o.or -0.8* - i IRL8 Rll E S T I M A T E S P C 2 B<u>x = (O.Ol, O.Ol, O.Ol, O.Ol) A < S > T = ( - 0 . 0 0 1 , O.Ol, -O.Ol, O.Ol) d < 4 > = 0 . 8 0 dtts:> = 0 . 9 5 ( (c) 0.0* o.oo o.or 0.01 0.01 0.04 O.Ol > •1 o o.oa i 0.01 1 + 0.00 1 -0.01 •o.ot -0.00 -0.04 -0.01 -0.00 •o.or -0.00 (fi <s> ,d < , = : > ) E 2 L E G E N D &l t t 4 CS3 CD IRLS Rll EST1MATE8 PCS The m a j o r c o n c l u s i o n s t h a t c a n be drawn f r o m t h e r e s u l t s a r e a s f o l l o w s . When t h e a l i g n m e n t i s f a v o u r a b l e ( i e . fi = ft*1-*), t h e n t h e r e i s c o n s i d e r a b l e r e d u c t i o n i n v a r i a n c e f r o m u s i n g a b i a s e d e s t i m a t o r . A l s o , t h e amount o f r e d u c t i o n i s p r o p o r t i o n a l t o t h e d e g r e e o f c o l l i n e a r i t y p r e s e n t . In t h i s c a s e p r i n c i p a l components seems b e t t e r t h a n t h e r i d g e e s t i m a t o r (Duncan's m u l t i p l e r a n g e t e s t f i n d s a d i f f e r e n c e a t t h e f i v e p e r c e n t ' l e v e l ) . In t h e c a s e when t h e a l i g n m e n t i s u n f a v o u r a b l e ( i e . B = fi<ra>) t h e r e a p p e a r s t o be some g a i n f r o m u s i n g ridge estimation , p r o v i d e d t h e b i a s i n d u c e d c o u l d be c o n s i d e r e d t o l e r a b l e . The e f f e c t o f u s i n g b i a s e d e s t i m a t o r s i s w e l l shown i n F i g u r e 5 . 1 ( c ) . In p a r t i c u l a r t h e i d e n t i t y mean s q u a r e e r r o r = v a r i a n c e + b i a a a c a n be s e e n . The IRLS and PC2 e s t i m a t o r s have t h e same ASE, b e c a u s e t h e v a r i a n c e r e d u c t i o n i n PC2 i s o f f e s t by i t s b i a s . I t i s e n c o u r a g i n g t o n o t e t h a t t h e c o n c l u s i o n s t h a t c a n be drawn f r o m T a b l e 5.2 a r e i n r e a s o n a b l e a g r e e m e n t w i t h t h e m a j o r summary c o n c l u s i o n s o f [GIBBONS,81] p l 3 7 - 8 . 1. A l l e s t i m a t o r s a r e b e t t e r t h a n t h e LS e s t i m a t o r s when t h e u n d e r l y i n g c o e f f i c i e n t v e c t o r i s f a v o r a b l e ; t h a t i s B = B,„. 2. No e s t i m a t o r i a a l w a y s b e t t e r t h a n t h e LS when t h e u n d e r l y i n g c o e f f i c e n t v e c t o r i s u n f a v o r a b l e ; t h a t i s fi = B.n,. 3. E s t i m a t e s HKB, GHW and RIDGM p e r f o r m e d w e l l o v e r a l l , where HKB r e f e r s t o [HOERL,KENNARD,BALDWIN,75] So i n c o n c l u s i o n , t h e r e i s some e v i d e n c e t o s u g g e s t t h e r e a r e d e f i n i t e g a i n s t o be made by u s i n g some f o r m o f b i a s e d e s t i m a t o r when c o l l i n e a r i t y i s p r e s e n t i n a g e n e r a l i z e d l i n e a r model w i t h a gamma d i s t r i b u t i o n and an a r t i f i c i a l c o l l i n e a r s t r u c t u r e a s was g e n e r a t e d . However f a r more e x t e n s i v e t e s t i n g , u s i n g b o t h s i m u l a t i o n and r e a l d a t a , must be c a r r i e d o u t b e f o r e any b r o a d e r s t a t e m e n t s c a n be made. 81 6.0 Summary and C o n c l u s i o n s T h i s t h e s i s has i n v e s t i g a t e d s u i t a b l e d e f i n i t i o n s f o r c o l l i n e a r i t y i n t h e g e n e r a l i s e d l i n e a r model, d i a g n o s t i c and i d e n t i f i c a t i o n p r o c e d u r e s and e s t i m a t i o n methods. The d e f i n i t i o n o f c o l l i n e a r i t y i n a s t a n d a r d l i n e a r model a s an a p p r o x i m a t e l i n e a r d e p e n d e n c y amongst t h e c o l u m n s o f t h e p r e d i c t o r m a t r i x X [GUNST,83], i s shown t o e x t e n d n a t u r a l l y t o t h e g e n e r a l s e t t i n g w i t h t h e s c a l i n g t r a n s f o r m a t i o n X = W X where W i s t h e w e i g h t m a t r i x a s s o c i a t e d w i t h t h e g e n e r a l l i n e a r model. U s i n g t h i s , i t was shown t h a t c o l l i n e a r i t y i n t h e X s y s t e m c a r r i e s o v e r t o t h e g e n e r a l s y s t e m f o r a l l m o d els e x c e p t t h e s o c a l l e d p s e r i e s . T h a t i s c o l l i n e a r i t y i n t h e g e n e r a l s e t t i n g i s model d e p e n d e n t . The CGUNST,831 e x t e n d e d d e f i n i t i o n was shown t o be m a t h e m a t i c a l l y e q u i v a l e n t t o s m a l l s i n g u l a r v a l u e s i n t h e t r a n s f o r m e d p r e d i c t o r m a t r i x X. T h i s p r o p e r t y was u s e d t o e x t e n d t h e [BELSEY,KUH,WELSCH,80] d i a g n o s t i c and i d e n t i f i c a t i o n scheme. To q u a n t i f y t h e model d e p e n d e n c y , bounds were d e r i v e d f o r t h e c o l l i n e a r i t y i n d e x o f t h e g e n e r a l i z e d l i n e a r model i n t e r m s o f t h a t o f t h e s t a n d a r d l i n e a r model c o l l i n e a r i t y i n d e x «*. A l t h o u g h r i d g e r e g r e s s i o n i n t u i t i v e l y seemed t h e e a s i e s t e s t i m a t i o n method t o c o n s i d e r , d i f f i c u l t i e s were e n c o u n t e r e d i n t r y i n g t o implement i t by e x t e n d i n g t h e [HOERL,KENNARD,70] bound r e s u l t f o r t h e p a r a m e t e r k. An a p p e a l i n g a l t e r n a t i v e , b r i e f l y d i s c u s s e d , was t h e p o s t e r i o r l i k l e h o o d a p p r o a c h , w h i c h i n c o r p o r a t e s an " e q u i v a l e n t amount o f i n f o r m a t i o n " . T h i s method i s an a r e a f o r f u t u r e r e s e a r c h . G i v e n t h e p r o b l e m s f o u n d i n t r y i n g t o s e t up a Monte C a r l o s i m u l a t i o n f o r t h e e s t i m a t i o n o f p a r a m e t e r s i n t h e g e n e r a l i z e d l i n e a r model, and t h e c r i t i c i s m d i r e c t e d a g a i n s t p r e v i o u s s i m u l a t i o n s t u d i e s , t h e r e i s a need f o r more work i n t h i s a r e a t o o . W h i l e t h e s i m u l a t i o n was e x t r e m e l y r e s t r i c t e d i n i t s a i m s and c o n c l u s i o n s , i t d i d s u c c e e d i n d e m o n s t r a t i n g t h a t p r o b l e m s e x i s t w i t h e s t i m a t i o n i n t h e p r e s e n c e o f c o l l i n e a r i t y and c a n be a l l e v i a t e d by t h e methods d e s c r i b e d . However much more e x p e r i e n c e i s needed b o t h p r a c t i c a l l y and t h e o r e t i c a l l y b e f o r e some f o r m o f b i a s e d e s t i m a t i o n methods c o u l d be a d v o c a t e d f o r r o u t i n e u s e . B i b l i o q r a p h y [ALLEN,77] A l l e n , D M "Comment on S i m u l a t i o n o f A l t e r n a t i v e s t o L e a s t S q u a r e s " J o u r n a l o f t h e A m e r i c a n S t a t i s t i c a l A s s o c i a t i o n 72 357 95-96. [BARD, 74] B a r d , Y o n a t h o n N o n l i n e a r P a r a m e t e r E s t i m a t i o n A c a d e m i c P r e s s New Y o r k . [BELSEY,KUH,WELSCH,80] B e l s e y , D A Kuh,E Welsch.RE R e g r e s s i o n  D i a g n o s t i c s : I d e n t i f y i n g I n f l u e n t i a l D a t a and S o u r c e s o f  C o l 1 i n e a r i t y J o h n W i l e y & Sons New Y o r k . [BRADLEY,SVRIVASTAVA,79] B r a d l e y , R A S v r i v a s t a v a , S S " C o r r e l a t i o n i n P o l y n o m i a l R e g r e s s i o n " A m e r i c a n S t a t i s t i c i a n 33 11-14. [DEMPSTER,SCHATZOFF,WERMUTH,77] Dempster,AP S c h a t z o f f , M Wermuth,N "A S i m u l a t i o n S t u d y o f A l t e r n a t i v e s t o O r d i n a r y L e a s t S q u a r e s " J o u r n a l o f t h e A m e r i c a n S t a t i s t i c a l A s s o c i a t i o n 72 77-106. [DRAPER,VAN NOSTRAND,79] Draper,NR Van N o s t r a n d , R C " R i d g e r e g r e s s i o n and James S t e i n E s t i m a t o r s : Review and Comments" T e c h n o m e t r i c s 21 451-466. [EDWARDS,69] Edwards,AWF " S t a t i s t i c a l Methods i n S c i e n t i f i c I n f e r e n c e " N a t u r e 222 1233-1237. [EDWARDS,69a] Edwards,AWF L i k e l i h o o d C a m b r i d g e U n i v e r s i t y P r e s s London [GIBBONS,81] Gibbons,DG "A S i m u l a t i o n S t u d y o f Some R i d g e E s t i m a t o r s " J o u r n a l o f t h e A m e r i c a n S t a t i s t i c a l A s s o c i a t i o n 76 373 131-139 [GUNST,80] G u n s t , R F "Comment on A C r i t i q u e o f Some R i d g e R e g r e s s i o n M e t h o d s " J o u r n a l o f t h e A m e r i c a n S t a t i s t i c a l A s s o c i a t i o n 75 369 98-100. [GUNST,83] Gunat,RF " R e g r e s s i o n A n a l y s i s w i t h M u l t i c o l 1 i n e a r P r e d i c t o r V a r i a b l e s : D e f i n i t i o n , D e t e c t i o n and E f f e c t s " C o m m u n i c a t i o n s i n S t a t i s t i c s - T h e o r y and Methods 12(19) 2217-2260. CHEMMERLE,BRANTLE,78] Hemmerle,WJ B r a n t l e , T F " E x p l i c i t and C o n s t r a i n e d G e n e r a l i z e d R i d g e R e g r e s s i o n " T e c h n o m e t r i c s 20 109-120. [H0ERL,62] Hoer1,AE " A p p l i c a t i o n o f R i d g e A n a l y s i s t o R e g r e s s i o n P r o b l e m s " C h e m i c a l E n g i n e e r i n g P r o g r e s s 58 54-59. [HOERL,KENNARD,70] Hoer1,AE Kennard,RW " R i d g e r e g r e s s i o n : B i a s e d E s t i m a t i o n f o r N o n - o r t h o g o n a l P r o b l e m s " T e c h n o m e t r i c s 12 56-67. [H0ERL,KENNARD,76] Hoer1,AE Kennard,RW " R i d g e R e g r e s s i o n : I t e r a t i v e E s t i m a t i o n o f t h e B i a s i n g P a r a m e t e r " C o m m u n i c a t i o n s i n S t a t i s t i c s A5 77-88. [H0ERL,KENNARD,BALWIN,75] Hoer1,AE Kennard,RW B a l d w i n , K F " R i d g e R e g r e s s i o n : Some S i m u l a t i o n s " C o m m u n i c a t i o n s i n S t a t i s t i c s 4 105-123. [KENDALL,57] K e n d a l l , M G A C o u r s e i n M u l t i v a r i a t e A n a l y s i s G r i f f i n L ondon. [KENDALL,STUART,67.2] K e n d a l l , M G S t a u r t , A The Advanced T h e o r y o f S t a t i s t i c s Volume 2 I n f e r e n c e and R e l a t i o n s h i p G r i f f i n L o n don. [LINDLEY,SMITH,72] L i n d l e y , D V Smith,AFM "Bayes e s t i m a t e s f o r t h e L i n e a r M o d e l " J o u r n a l o f t h e R o y a l S t a t i s t i c a l S o c i e t y B 42 31-34. [McCULLAGH,82] M c C u l l a g h , P " C a t e g o r i a l D a t a A n a l y s i s " L e c t u r e N o t e s U n i v e r s i t y o f B r i t i s h C o l u m b i a . [McCULLAGH,NELDER,83] M c C u l l a g h , P N e l d e r , J A G e n e r a l i z e d  L i n e a r M o d e l s Chapman and H a l l London. [McDONALD,GALARNEAU,75] McDonald,GC G a l a r n e a u , D I "A Monte C a r l o E v a l u a t i o n o f Some R i d g e Type E s t i m a t o r s " J o u r n a l o f  t h e A m e r i c a n S t a t i s t i c a l A s s o c i a t i o n 70 407-416. [MONTGOMERY,PECK,82] Montgomery,DC Peck,EA I n t r o d u c t i o n t o L i n e a r R e g r e s s i o n A n a l y s i s J W i l e y & Sons New York [M0YLS,851 M o y l s , B P e r s o n a l c o m m u n i c a t i o n [MULLET,76] M u l l e t , G M "Why R e g r e s s i o n C o e f f i c i e n t s have t h e Wrong S i g n " J o u r n a l o f Q u a l i t y T e c h n o l o g y 8 121-126. [NELDER,77] N e l d e r , J A " D i s c u s s i o n on Ba y e s E s t i m a t e s f o r t h e L i n e a r M o d e l " J o u r n a l o f t h e R o y a l S t a t i s t i c a l S o c i e t y S e r i e s B 34 18-20. [NELDER,WEDDERBURN,72] N e l d e r , J A Wedderburn,RWM " G e n e r a l L i n e a r M o d e l s " J o u r n a l o f t h e R o y a l S t a t i s t i c a l S o c i e t y S e r i e s A 135 370-384. [PATEFIELD,77] P a t e f i e l d , W M "On t h e M a x i m i z e d L i k e l i h o o d F u n c t i o n " Sankhya S e r i e s B 39 92-96. [PREGIBON,81] P r e g i b o n , D " L o g i s t i c R e g r e s s i o n D i a g n o s t i c s " The A n n a l s o f S t a t i s t i c s 9 4 705-724. [SCHAEFER,79] S c h a e f e r , R L M u l t i c o l l i n e a r i t y i n L o g i s t i c R e g r e s s i o n PhD T h e s i s U n i v e r s i t y o f M i c h i g a n #792522D [SCHAEFER,82] S c h a e f e r , R F " A l t e r n a t i v e E s t i m a t o r s i n L o g i s t i c R e g r e s s i o n when t h e D a t a a r e C o l l i n e a r " P r o c e e d i n g s o f t h e A m e r i c a n S t a t i s t i c a l A s s o c i a t i o n S t a t i s t i c s and  Com p u t i n g S e c t i o n 159-164. [SILVEY,69] S i l v e y , S D " M u l t i c o l l i n e a r i t y and I m p r e c i s e E s t i m a t i o n " J o u r n a l o f t h e R o y a l S t a t i s t i c a l S o c i e t y S e r i e s B 31 539-552. [SMITH,CAMPBELL,80] Smith,G C a m p b e l l , F "A C r i t i q u e o f R i d g e R e g r e s s i o n " J o u r n a l o f t h e A m e r i c a n S t a t i s t i c a l A s s o c i a t i o n 75 369 74-81. CTHISTED,76] T h i s t e d , R A " R i d g e R e g r e s s i o n , H i n i m a x E s t i m a t i o n and E m p i r i c a l B a y e s M e t h o d s " T e c h n i c a l R e p o r t No 28 S t a n d f o r d U n i v e r s i t y D i v i s i o n o f B i o s t a t i s t i c s CWEDDERBURN,74] Wedderburn,RWM " Q u a s i L i k e l i h o o d F u n c t i o n s , G e n e r a l i z e d L i n e a r M o d e l s and t h e Gauss-Newton Method" B i o m e t r i k a 61 3 439-447. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0096659/manifest

Comment

Related Items