UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

An analysis of multidimensional contingency talbes Mast, Lilian G. (Feuerverger) 1973

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
831-UBC_1973_A6_7 M38.pdf [ 5.83MB ]
Metadata
JSON: 831-1.0080461.json
JSON-LD: 831-1.0080461-ld.json
RDF/XML (Pretty): 831-1.0080461-rdf.xml
RDF/JSON: 831-1.0080461-rdf.json
Turtle: 831-1.0080461-turtle.txt
N-Triples: 831-1.0080461-rdf-ntriples.txt
Original Record: 831-1.0080461-source.json
Full Text
831-1.0080461-fulltext.txt
Citation
831-1.0080461.ris

Full Text

AN ANALYSIS OF MULTIDIMENSIONAL CONTINGENCY TABLES by L i l i a n Mast (nee Feuerverger) B. Sc. Hons., M c G i l l U n i v e r s i t y , 1969 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE i n the Department of Mathematics We accept t h i s thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA APRIL, 1973 In presenting t h i s t h e s i s in p a r t i a l f u l f i l m e n t of the requirements f o r an advanced degree at the U n i v e r s i t y of B r i t i s h Columbia, I agree that the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r reference and study. I f u r t h e r agree that permission f o r extensive copying o f t h i s t h e s i s f o r s c h o l a r l y purposes may be granted by the Head of my Department or by h i s r e p r e s e n t a t i v e s . I t i s understood that copying or p u b l i c a t i o n of t h i s t h e s i s f o r f i n a n c i a l gain s h a l l not be allowed without my wr i t t e n pe rm i ss ion. Department of M a t h e m a t i c s The U n i v e r s i t y of B r i t i s h Columbia Vancouver 8, Canada D a t e February 1, 1973 ( i i ) Abstract In t h i s thesis we consider the following model for a three-dimensional r x s x t contingency table: E ( f . j k ) = np. j k = n E p ^ ^ ^ ^ ^ ^ ^ a ^ ^ ^ ^ . ^ Y y - M y , , ] > 0 i = l , 2 , . . , r ; j=l,2,..,s; k=l ,2,...,t with s t r t r- s .1. a.. = . E. a.- = 0, . E- g,. = . Sn B., = 0, .E, y... = .E. y.. = 0 j = l j k k=l j k i = l l k k=l l k ' i = l ' i j j = l i j r s t and .En S... = .E, 5... = ,-E. 6"...T=0. A dot indi c a t e s summation over the 1=1 l j k j = l l j k k=l l j k replaced s u b s c r i p t . The f ^ j ^ ' s represent the frequencies and the p^ k ' s represent the proportions. The problem we are concerned with i s tes t i n g the hypothesis H^: = ^ ^o r 3' ^ " i, e* n o second order i n t e r a c t i o n i s present. We then seek to extend the model and problem to a w-way t a b l e . We use the method of the l i k e l i h o o d r a t i o . To a s s i s t us i n determining the numerator of the l i k e l i h o o d r a t i o we reformulate a theorem about constrained extrema and Lagrange m u l t i p l i e r s and prove t h i s reformulation. Some general conclusions we draw are: there are two extensions to our 3-way model; results'we obtain using our model and methods are i n close agreement;with r e s u l t s obtained using the models and methods of other s t a t i s t i c i a n s . Ciii) TABLE OF CONTENTS „ Page Chapter I Introduction 1 Chapter II Presentation of Our Model 9 Chapter I I I Testing the Hypothesis of No Second Order Int e r a c t i o n Using the Likelihood Ratio Technique A. The Denominator of the Li k e l i h o o d Ratio 14 B. A Theorem About Lagrange M u l t i p l i e r s 15 C. Preview of the Argument to Come 20 D. S a t i s f y i n g the Conditions of the Theorem 22 E. Solving f o r One of the Lagrange M u l t i p l i e r s 31 F. The System of Equations Have a F i n i t e Solution ... 32 G. The Numerator of the Li k e l i h o o d Ratio 38 H. Summary of Chapter I I I 38 Chapter IV Contingency Tables for which the Hypothesis of No Second Order Interaction i s Not Meaningful 40 Chapter V Extension to Higher Way Tables A. Two Extensions of the Three-Dimensional Model to the Four Dimensional Case 47 B. Lancaster's D e f i n i t i o n of Interactions Compared to Ourc. 50 C. The General Case f or the F i r s t Extension of Our Three-Way Model 53 (iv) D. The General Case for the Second Extension of Our Page Three-Way Model 70 Chapter VI Numerical I l l u s t r a t i o n Using Our Model and Methods / and Other Models and Methods A. The Likelihood-Ratio with our Model 85 B. The Chi-Squared S t a t i s t i c With Our Model 87 C. Bishop's Model and Methods 89 D. Comparison of Ad d i t i v e and M u l t i p l i c a t i v e (Log l i n e a r ) Models for a 'Three-Way Contingency /.Table 97 Bibliography 100 (v) TABLE I TABLE II TABLE I I I TABLE IV TABLE V TABLE VI - X TABLE XI TABLE XI.I TABLE XIII TABLES COEFFICIENTS IN THE EXPRESSION FOR THE HIGHEST ORDER INTERACTION 74 DATA REPRESENTING NUMBER OF ITEMS PASSING OR FAILING TWO TESTS ON CERTAIN MANUFACTURED PRODUCTS 84 VALUES OF f . . . log f . .- 85 i j k & l j k VALUES OF f . . T log R. where R. .. = l j k " l j k l j k ^fl . .f. j k + f. j .fi . k+ f. . kfi j . > -2 fi . .f. J .f. . k I 8 6 CALCULATION OF y 2 USING EXPECTED VALUES = R . , 2 88 i j k / n CALCULATION OF EXPECTED VALUES USING ITERATIVE METHOD 91-93 CALCULATION OF x" USING EXPECTED VALUES DERIVED FROM ITERATIVE METHOD 94 VALUES OF f . j k log f . j k 96 ADDITIVE MODEL V.S. MULTIPLICATIVE (LOG-LINEAR) MODEL 98 (vi) ACKNOWLEDGMENT The author wishes to express her sincere appreciation to Dr. Stanley W. Nash of the Department of Mathematics, U n i v e r s i t y of B r i t i s h Columbia for suggesting the topic of t h i s thesis and for the many hours of guidance and comments during i t s preparation. She would also l i k e to thank Dr. N. G l i c k of the Department of Mathematics f o r reading the f i n a l manuscript. She i s also pleased to acknowledge the support of the Un i v e r s i t y of B r i t i s h Columbia f o r i t s f i n a n c i a l a s s i s t a n c e . F i n a l l y , the author wishes to thank Eleanor Lannon and Mary Daisley f o r t h e i r assistance i n the typing of t h i s t h e s i s . 1 CHAPTER I INTRODUCTION The cross c l a s s i f i c a t i o n of a s e r i e s of observations according to w c h a r a c t e r i s t i c s , or a t t r i b u t e s , i s re f e r r e d to as a multidimensional contingency t a b l e . The ent r i e s i n the c e l l s of a contingency table are frequencies. The usual hypotheses investigated i n contingency tables are about kinds of s t o c h a s t i c independence or lack of i t . The de s c r i p t i o n s of st o c h a s t i c independence f o r several p a r t i t i o n s of the sample space are those given by Kolmogqrov (and by F e l l e r f o r several events). Similar d e s c r i p t i o n s hold f o r density functions and cumulative d i s t r i b u t i o n func-tions of several random v a r i a b l e s . In t h i s t h e s i s we in v e s t i g a t e how to handle and analyze m u l t i -dimensional contingency t a b l e s . Quite a b i t has been done i n recent y e a r s , but a l o t s t i l l remains to be done. We concern ourselves b a s i c a l l y with d e f i n i t i o n s of i n t e r a c t i o n . Before we present our model and d e f i n i t i o n of i n t e r a c t i o n we w i l l mention, very b r i e f l y , the models and d e f i n i t i o n s of i n t e r a c t i o n of some of the s t a -t i s t i c i a n s who were and may s t i l l be concerned with the a n a l y s i s of c o n t i n -gency t a b l e s . H. H. Ku and S. Kullback investigated the problem of i n t e r a c t i o n i n multidimensional contingency tables from the viewpoint of information theory as developed by Kullback. The hypothesis of no rth-order i n t e r a c t i o n i s defined i n the sense of an hypothesis of "generalized" independence of c l a s s i f i c a t i o n s with f i x e d r - t h order marginal r e s t r a i n t s . For a three-way t a b l e , with given c e l l p r o b a b i l i t i e s P-y^ » t n e minimum d i s c r i m i n a t i o n information f o r a contingency table with marginals p.. , p , p. , i s i j . • j k i • lc given by the set of c e l l p r o b a b i l i t i e s p... = a,.b.. c . p . ,. where a.., b., , 13 k i j j k l k l j k i j j k and c ^ k are functions of the given marginal p r o b a b i l i t i e s . ^ ( P - ^ J ^ / P - L J ^ ) = £n a.. + in b., + £n c M , represents no 2nd-order i n t e r a c t i o n . The minimum i j j k l k 2 d i s c r i m i n a t i o n information s t a t i s t i c , asymptotically d i s t r i b u t e d as x with appropriate degrees of freedom, i s r s t r s t ^ 2 1 I E f . £n f . - 2 E E E f . , , l n f . ., > O.where f . „ are i - l j - l k-1 1 3 k 1 3 k i - l J - l k-1 l j k ^ k " ^ k the observed c e l l frequencies and f . . . are the "no i n t e r a c t i o n " c e l l f r e -l j k quencies uniquely determined by a simple convergent i t e r a t i o n process of the marginals on p... . i j k B a r t l e t t defined no second order i n t e r a c t i o n i n a 2x2x2 table as P111P122P212P221 = P112P121P211P222 ™h e r e t h e P'S a r e t h e C e l 1 Pr o b a b i l l t i e s• Roy and Kastenbaum extended B a r t l e t t ' s d e f i n i t i o n and a r r i v e d a set of "no i n t e r a c t i o n c o n s t r a i n t s " i n an rxsxt table i n the form of pi s tpr j t pi s kpr j k k = 1, 2, (t-1) The "mechanism" used by Roy and Kastenbaum i s based on the f a c t that the two hypotheses V p i . k = p i . . ? . . k H 2 : p i j . = P i . . P . j . w i l l not imply H: p.., = p. p i n a 3-way contingency t a b l e . A dot i n -i j k i . . • j k dicates summation over the replaced s u b s c r i p t . The "no i n t e r a c t i o n " hypothesis 3 i s required to generate the set of constraints such that these c o n s t r a i n t s , when superimposed on A should imply H . Simpson required the d e f i n i t i o n of "no-second-order i n t e r a c t i o n " to be symmetrical with respect to the three a t t r i b u t e s of a 2x2x2 t a b l e . I f some function f ( p ^ , P121' p211' p221^ * S C n ° s e n t o m e a s u r e the a s s o c i a -t i o n of c l a s s i f i c a t i o n s A, B and C , then the function must be such that the equation Hv±lV P m , P 2 U , P 2 2 1> - * ( p u 2 , P 1 2 2 . P 2 1 2., P 2 2 2 ) <=> * ( P m , P 2 U , P n 2 , P 2 1 2).= * ( p m , P 2 n , P 1 2 2 , P 2 2 2 ) <=> U P 1 1 V P 1 2 1 , P 1 1 2 , P 1 2 2 ) = i K p 2 1 1 , P 2 2 1 , P 2 1 2 , P 2 2 2 ) • He showed that the function ty = P121 P211 or the cross product r a t i o used by B a r t l e t t , P111 P221 s a t i s f i e s t h i s requirement. Lancaster defined the second order i n t e r a c t i o n by the p a r t i t i o n of the 2 chi-square s t a t i s t i c x »i«e. i t i s defined as the d i f f e r e n c e between the 2 t o t a l x f °r t e s t i n g complete independence of the three c l a s s i f i c a t i o n s , and the sum of the 3 components corresponding to tes t s for independence i n each of the 3 marginal t a b l e s . Lancaster's d e f i n i t i o n does not always s a t i s f y Simpson's condition of symmetry. In Chapter V of t h i s t h e s i s we w i l l mention the model he gives i n h i s book, which i s much l i k e our own, though we have used d i f f e r e n t approaches. We are i n a way defending Lancaster against the c r i t i c i s m s of the preponents of the l o g l i n e a r model (to be mentioned soon), while what he has done reinforces, what we propose. Darroch made an e x p l i c i t comparison of the d e f i n i t i o n s of i n t e r a c t i o n i n multiway contingency tables and i n the analysis of variance. The main 4 point he made was that a natu r a l symmetrical d e f i n i t i o n of "no second-order i n t e r a c t i o n " P.., = p .,p. ,p.. n e c e s s a r i l y implies constraints on the marginal Pi . .P. j .P. . k (1) p r o b a b i l i t i e s p , p fe, p ± k , t t i'e- z P i i k = P±i = P±i E P± kP i k Pi p i p k .1. • • • J • • • fx t or E p p = p p ^ i , j and the l i k e , k_J^ X . ft. . J K. X . . . J . P..k This i s undesirable since the condition for"no second-order i n t e r a c t i o n " ^should r e l a t e p.., to any given set of marginal p r o b a b i l i t i e s and should ijK. not place r e s t r i c t i o n s on the l a t t e r . Consequently Darroch defined a "perfect three-way t a b l e " as one for which condition (1) and the r e s u l t i n g constraints on the marginal p r o b a b i l i t i e s are s a t i s f i e d e x a c t l y . He con-cluded f u r t h e r that " i n imperfect tables i t i s not po s s i b l e to express p , i n terms of simple functions of p.. , p. p ., when there i s no 2nd order i n t e r a c t i o n . " The existence and uniqueness of the set P-jj k a s the s o l u t i o n of Pr s tPi i t Pr s kPi i k i = 1, 2, (r-1) r S t 1 J t = r S" 1 3 k f o r j = 1, 2, .. . . (s-1) Pi s tPr j t Pi s kPr j k k = 1, 2, (t-1) fo r any given set of mutually consistent marginal p r o b a b i l i t i e s was con-jectured f o r rxsxt tables and proved f o r the 2x2x2 case. The search fo r a simple formulation i n terms of parameters which are i m p l i c i t l y de-5 fined by the marginal p r o b a b i l i t i e s led Darroch to define p.., = uA.,B, . C . where l j k j k k i i j r s Z A = Z B = Z C. . = 1 and u Z E Z A B C = 1 ; and k-1 J k 1-1 k l j - l . 1 J 1-1 j - l k-1 J k k l 1 3 to show that u = 1 , A.k = p ^ , B k. = p ^ and C. . = p _ _ • P i P k P i Since there i s no s o l u t i o n i n closed form to the maximum l i k e l i h o o d equations f o r the parameters under hypothesis of no second-order i n t e r a c t i o n , unless the observed table happens to be p e r f e c t , Darroch suggested an i n t e r -a t i v e s o l u t i o n and gave a numerical i l l u s t r a t i o n using the example given by Kastenbaum & Lamphierar, [1959] . It i s of i n t e r e s t to note that Darroch suggested the l i k e l i h o o d r a t i o test r s t £ based on Z = 2 Z E Z f. In i j k which i s asymptotically ABC i j k „ „ 1 J nuA., B .C. . j k k i i j 2 d i s t r i b u t e d as x with (r-1) x (s-1) x (t-1) degrees of freedom. Good proposed to use the p r i n c i p l e of maximum entropy as a h e u r i s t i c p r i n c i p l e f o r the generation of n u l l hypotheses, with main a p p l i c a t i o n to w-dimensional contingency t a b l e s . By using h i s p r i n c i p l e , i t i s shown that f o r a w-dimensional 2x2x ... x 2 contingency table with p. = p. . . , . . = 0, 1 and with a l l marginal p r o b a b i l i t i e s down l i _ i _ . . l 1- i . 1 2 w 1 w |i|even . |i|odd to (w-l)-way assigned, the n u l l hypothesis to be tested i s Hp. = Hp. i 1 i 1 6 where | i | = + 1^ + ... + 3-w • This expression reduces to P1P4P6P7 = P2P3P5P8 ( P111P122P212P221 = P112P121P211P222) W h e n m = 3 ' Good also generalized the d e f i n i t i o n to that of no rth-order and a l l higher-order i n t e r a c t i o n s i n a w-dimensional contingency table with a complete set of rth-order r e s t r a i n t s by means of d i s c r e t e Fourier transforms of the l o g -arithms of p r o b a b i l i t i e s . However, the i n t e r a c t i o n s so defined are usu a l l y complex valued unless the number of categories within each c l a s s i f i c a t i o n i s equal to two. L. Goodman followed the d e f i n i t i o n by Good but proposed a t e s t that y i e l d s r e a l valued i n t e r a c t i o n s . Goodman has published many papers on contingency t a b l e s , some of which are l i s t e d i n the bibliography of t h i s t h e s i s . Another method of an a l y s i s has i m p l i c i t l y been given by B. Woolf i n the case of a 2x2xt t a b l e . Let the frequencies i n the kth 2x2 table be denoted by f l k , , f 3 k , f ^ , where f l k , f 2 k occupy the f i r s t row and the f i r s t column. Compute = ^nf-^ - ^n^2k ~ ^n^3k + ^n^4k A * 1 1 . 1 . 1 . 1 and e, from — = — h -z— + —— + k ek fl k f2k f3k f4k 2 t 2 t 2 t I f there i s zero second-order i n t e r a c t i o n , then x = E ev^L- - ( 2 e, Z, ) / E e, k-1 k=l k k k=l k i s asymptotically d i s t r i b u t e d as chi-squared with (t-1) degrees of freedom. With u n r e s t r i c t e d sampling c o n d i t i o n s , M.W. Birch states that Roy and Kastenbaum's condition may be rewritten as £nf... = Zn f + tn f . , + X j fcC X"J rC 1SK. Zn f . . . - Zn f .-r zn f - Zn f . + Zn f , i j t i s t r j t rsk r s t (1 _< i _< r-1 ; 1 <_ j <_ s-1 ; 1 £ k £ t-1) where = n p i j k " This 7 condition i s s a t i s f i e d i f , and only i f , £n f.. , can be wr i t t e n i n the ilk form An f . . k = u + + u,,. + + u ^ . . + + u 2 3 j k ( l<i<r; l<j<s; l<k<t), where u ^ = = = 0 ; = = 0 , each i ; ^ . ^  = u ^ . = 0 , r each j ; k = k = 0 , each k ; (where = E ) ; but otherwise i = l u , u2 3 j k a r e completely a r b i t r a r y . In general, we can write An f . ., i n the form; i j k in f = u + u.. . + u + u_. + u, _. . + u- - M + u „ „ + u _ i j k l i 2j 3k 12ij 13ik 23jk 123ijk where ^ - 0 , each ( i , j ) ; ^231.k = 0 ' e a c h k^ 5 a n d U123 j k ~ ® * e a c n • k) . The uj _ 2 3 » s a^ e t h e n .the second .order i n t e r -a c t i o n s . Y.M.M. Bishop adopts Birch's model. She u t i l i z e s h i s r e s u l t that appropriate sums of the observed c e l l frequencies are s u f f i c i e n t s t a t i s t i c s f o r maximum l i k e l i h o o d estimation of t h e ^ c e l l frequencies under a s p e c i f i e d model. B i r c h does not give a computing method but r e f e r s to i t e r a t i v e computing methods of Norton [1945] , Kastenbaum and Lamphier [1959], and Darroch [1962]. Bishop uses a d i f f e r e n t computing method, an i t e r a t i v e proportional method which she adapted from Deming and Stephan [1940]. This i s i l l u s t r a t e d i n Chapter VI of t h i s t h e s i s . The work of S. Fienberg and F. Mosteller i s c l o s e l y a l l i e d with that of Bishop and therefore we do not give a separate disc u s s i o n of t h e i r work. Some of t h e i r papers are l i s t e d i n the bibliography. 8 The above i s by no means an exhaustive l i s t of s t a t i s t i c i a n s who have worked with contingency t a b l e s . I t was our purpose j u s t to present to the reader an idea of some of the a l t e r n a t i v e s . This being accomplished, we proceed to present our model. CHAPTER II PRESENTATION OF OUR MODEL Consider the following model for a three-dimensional rxsxt c o n t i n -gency t a b l e : E ( fi j k} = n pi j k = n t pi . .P. j . P..k + Pi . .aj k + P. j Ak + P . . k Y i j + 6 i j k ] i — 1 j 2, • • • } IT i 2 *~ 1 * 2, • • • y s j k. — 1, 2 5 with Z a - Z a = 0 , Z B = Z 3±k = 0 , Z Y ; L i = 2 Y ± 1 = 0 j = l J k k-1 J k i = l l k k=l l R i-1 1 J j-1 1 J and Z 6.., = Z <5. = Z 6, = 0 •. A dot in d i c a t e s as before summa-i - 1 l j k J-1 l j k k-1 ± l k t i o n over the replaced s u b s c r i p t . The ^ - j ^ ' s represent the frequencies and the P ^ j ^ '3 represent the proportions. E ( f. j k > = n p. j k = n t p. j .p. . k + aj k] E( fi.k> = n p 1 > k = n [ p i > > P > > k + e i k ] E(f ) = np = n[p p + Y..] ^ 13 • i-3 • l . . • j . IJ V = P . j k " P . j . P . . k <ei k= Pi . k -pi . .P. . k ( 1 ) (Jij P i j . " P i . . P . j 6ijk = Pijk " p i . . p . j . p . .k ~ p i . . a jk ~ p . j A k " p..k^ 10 = pi j k - pi . .p. j k - p. j .pi . k - p. . kpi j . + 2 pi . .p. j .p. . k a f t e r s u b s t i t u t i n g the expressions for aj k ' ^ i j * We explain now why the model presented seems reasonable. Consider f i r s t the case of a 2x2x2 contingency t a b l e . Let the p a r t i t i o n s of the sample space be {A, A} , {B, B} , and {C, C} . Consider the two events A and B . Everyone agrees that P(AB)- P(A)P(B) i s a proper way of mea-suring d e v i a t i o n from pairwise independence. Suppose now that the three events A, B, C are pairwise independent so that P(AB) = P(A)P(B) P(AC) = P(A)P(C) (2) P(BC) = P(B)P(C) We show now by an example from F e l l e r that pairwise independence does not n e c e s s a r i l y imply that P(ABC) = P(A)P(B)P(C) i . e . that the two events AB and C are independent (<=> P(ABC) = P(AB)P(C)) or that BC and A are independent (<=> P(ABC) = P(A)P(BC)) or that AC and B are independent (<=> P(ABC) = P(B)P(AB)) . Example: Consider the s i x permutations of the l e t t e r s a, b, c as w e l l as the three t r i p l e s (a, a, a) , (b, b, b) and ( c , c,c) . We take these nine t r i p l e s as points of a sample space and a t t r i b u t e p r o b a b i l i t y 1/9 to each. Denote by A^ the event that the kth place i s occupied by the l e t t e r a . Obviously each of these three events has p r o b a b i l i t y 1/3 while P C A ^ ) = P C A ^ ) = P(A 2A 3) = 1/9 . The three events are therefore pairwise independent, but they are not mutually independent because also P(A^A2A-j) = 1/9 . (The occurence of A^ and A 2 implies the occurence of A^ and so A^ i s not independent of A^A2) . 11 We reserve therefore the term independence f o r the case where not only (2) ho l d s , but i n addi t i o n P(ABC) = P(A)P(B)P(C) . This equation ensures that A and BC are independent and also that the same i s true of B and AC and of C and AB . Consider a 2x2x2 t a b l e , i = 1, 2; j = l , 2 ; k=l, 2 . L e t t i n g Pn = P(A) , P = P(B) , P , = P(C) , P_ = P(A), p = P(B) , P _ = P(C) , P u > = P(AB) , P 1 J L = P(AC) , P > u = P(BC) P12. = P ( A B ) ' e t C* ' we see that the conditions (1) are equivalent to ^al l = P ( B C ) ~ p(B)p^C^ a 1 2 = P(BC) - P(B)P(C) a 2 1 = P(BC) - P(B)P(C) ^ a 2 2 = P(BC) - P(B)P(C) as w e l l as four s i m i l a r equations f or the g , and four s i m i l a r equa-XK. S tions f or the Y - J J » s • Hence the condi t i o n that B and C are independent corresponds to = u > t a e c o n d i t i o n that A and C are independent corresponds to g = 0 ; and the condition that A and B are independent corresponds to y^j = 0 . 12 We have seen, by the example considered, that pairwise independence does not n e c e s s a r i l y imply mutual independence, hence the presence of the term 6 If 6 • were not present, then pairwise independence i j k i j k would automatically imply p. = p. p . p i . e . P(ABC) = P(A)P(B)P(C) i j k i . . . j . . .k (or P(ABC) = P(A)P(B)P(G) etc.) . <5... , we have seen, i s equal to i j k pi j k - pi . .p. j .p. . k " pi . .aj k " p . j A k " p . . k Y i j • 8 0 t h a t ' i f t h e r e i s no f i r s t order i n t e r a c t i o n , 6... becomes p... - P. P . P , • i j k i j k i . . . j . ..k In t h i s thesis we are mainly concerned with rxsxt contingency t a b l e s . Suppose therefore {A^ A 2 A^} , {B.^ B 2, Bg} , {C^, . . . , Cfc} are three p a r t i t i o n s of the sample space. The A and B p a r t i t i o n s are s t o c h a s t i c a l l y independent i f P^B..) = P(A±)P(Bj) , i = l to r-1 , j = l to s-1 . These ( r - l ) ( s - l ) equations are l i n e a r l y independent. Similar statements hold f o r the other two p a i r s of p a r t i t i o n s . There are further conditions f o r complete independence beyond those for pairwise independence. For three p a r t i t i o n s the other conditions are P(A.B.C,) = P(A.)P(B.)P(C, ) , I J K i j k 1 to r-1 ; j = l to s-1 ; k=l to t-1 . These (r-1)(s-1)(t-1) equations are l i n e a r l y independent. A. N. Kolmogorov i n h i s text "Foundations of P r o b a b i l i t y " and W. F e l l e r i n h i s text "An Introduction to P r o b a b i l i t y Theory and i t s A p p l i c a t i o n s " deal with the theory of independence. A l l t h i s information i s contained i n our model p.., = p. p . p , + ijk l . . .j . . .k Pi . .aj k+ P. j .Pi k+ P. . k Y i j + 6i j k * Let P, = P(A n) , P = P(A ) 1.. 1 r . . r p = P(B ) , ...,.p = P(B ) 13 P < > 1 = p ( C l ) , . . . , P „ t - P < c t ) P l l . = P ( A i B i ) e t c * aj k " - ^ v ^ v B.k = P(A iC k) - P(A.)P(C k) \ are the v.. = P(A.B.) - P(A.)P(B.) 1 3 measures of f i r s t order i n t e r a c t i o n or deviations from pairwise independence. In our model, i f we have pairwise independence, that i s , i f the a's, 3's, and y's a r e aH zero we get p. = p. P . p v + 5. . In t h i s case 12lc x.« * J . •. K. 12lc 6i j k = P i 3 k - p i . . P . j . P . . k ° r « 1 J k " PCA±B j C k) - PCA^PCB^PCC^) . If P(A.B.C.) = P(A.)P(B.)P(C, ) for a l l i , j , k then 6. , = 0 f o r a l l i j K 1 j lc iji c i , j , k and since there i s no f i r s t and no second order i n t e r a c t i o n we have complete independence. 14. CHAPTER I I I TESTING THE HYPOTHESIS OF NO SECOND ORDER INTERACTION USING THE LIKELIHOOD RATIO TECHNIQUE A. THE DENOMINATOR OF THE LIKELIHOOD RATIO In our model second order i n t e r a c t i o n i s present i f f 6... ^  0 f o r l j k some ( i , j , k) , i . e . i f f p. j k * P-^P j k + P . j . P ^ + P.. u p i ; j. " ^ . . P . j . f o r some ( i , j , k) . Let q. j k = p ± > > P > j k + P . ^ . k + P.. kP ± J. " 2p. P . p , (p.., = q. ., +6... whether or not 6... i s zero) . Con-i . . . j . . .k ri j k nxjk rjk ljks i d e r i n g the given model we may want to test the following hypothesis: H : 6.., = 0 for a l l ( i , j , k) . i . e . no second order i n t e r a c t i o n i s o l j k \ t J> ' present. Under H q the model y i e l d s : Pi j k = Pi . .P. j k + P. j .Pi . k + P. . kPi j . " 2 pi .P. j .P. . k = qi j k i = 1 to r ; j = 1 to s ; k = 1 to t . We use the method of l i k e l i h o o d r a t i o . The most general assumption i s that the density or p r o b a b i l i t y function ^ ' ^ i i k r s t p .J r s t P = n! n n n with o < P < i , E E E P = 1 , i-1 j = l k=l r i j k ? 1 J t c i-1 j = l k=l 1 J l c where the f are non-negative integers and E E E f . . . = n ± 3 K i-1 j-1 k=l l j l C The given c o n s t r a i n t s are E 6.-., = E S . ., = E 6". ., = 0 as w e l l as i-1 ^k j-1 1Jk k-1 l j k E a., = E a = 0 , E g = E g = 0 , E Y , , = ^ y j-1 J k k-1 J k i-1 x k k-1 i k i-1 1 J j = l 1 J " ° * 15 s These c o n s t r a i n t s are equivalent to £ p. ., = p ., , E P.-t, = P- v » ^ P - - L -x3k , J K x3k 1 , K ^=1 ^-Jk p. . . To maximize P under the most general set of assumptions we use the 3 r s t con s t r a i n t E Z E p _ k = 1 e x p l i c i t l y . The , 8.ik , y^, 6 _ k do not have to brought i n e x p l i c i t l y i n dealing with the general case. The co n s t r a i n t s we placed on the a's , g's , y's a nd 5's were chosen so that the contingency table would add up. We j u s t keep i n mind what Pi . . ' P- j . ' P..k a n d Pi j . ' Pi . k ' P. j k s t a n d f o r i n t e r m s o f t h e P i j K 's S: t P i = Z 1 Piik P ik = 1 Piik 1 " j - l k=l 1 J K , J k i=l 1 J s = 2 1 Pi i k pi k = Z Pi i k i = l k=l 1 J R X , I C j = l 1 J l c p . = E E p . . . _ v ' '•k i - l j - l ^ k p i j . - ^ ' U K Let p... denote the maximum l i k e l i h o o d estimator (MLE) of p... i n the 13 K r i j k iik general case. It i s w e l l known that p.., = ——- . Hence the denominator i j k n r s t f . ^ i j k of our l i k e l i h o o d r a t i o i s n! JI II II (—^) / i - l j - l k-1 n f . j k ! B. A THEOREM ABOUT LAGRANGE MULTIPLIERS The usual Lagrange m u l t i p l i e r procedure i s to have l i n e a r l y independent c o n s t r a i n t s on the v a r i a b l e s , and the number of such co n s t r a i n t s must be fewer than the number of v a r i a b l e s . For reasons of symmetry i t i s use f u l to have a form of the procedure i n which the constrain t s no longer need be l i n e a r l y independent and may even exceed i n number the number of v a r i -1 6 a b l e s , though the number of l i n e a r l y independent constraints must s t i l l be fewer than the number of v a r i a b l e s . The following i s a theorem about the extended procedure. The notation of t h i s theorem and i t s proof i s i n -dependent of that i n the rest of the t h e s i s . The proof of t h i s theorem c l o s e l y p a r a l l e l s Apostol's proof of the o r i g i n a l Lagrange m u l t i p l i e r theorem. (See Tom M. Apo s t o l , "Mathematical A n a l y s i s , A Modern Approach to Advanced C a l c u l u s " . Reading, Mass.,1957. pp. 1 5 2 - 1 5 6 ) . Theorem:, Let f be a r e a l valued function having continuous f i r s t - o r d e r p a r t i a l d e r i v a t i v e s on an open set S i n n-dimensional Euclidean space E^ . Let g^ , • • • , gp be p real-valued functions also having continuous f i r s t - o r d e r p a r t i a l d e r i v a t i v e s on S , and assume that the number of f u n c t i o n a l l y independent members of {g , . . . , g } on S i s r , with i P r < n . Let X be that subset of S on which g, , g a l l vanish, o 1 P that i s , i n s e t , vector n o t a t i o n , X q = {_|_eS , _ ( x) = 0 } . Assume that X Q E X q and assume that there e x i s t s a neighborhood nXi^) such that f(x) <_ f ( X q ) f o r a l l x i n X0nn(2E0) o r that f (x) >^  ^ O ^ ) f ° r a ^ H i n X An(x ) • Assume also that the matrix [D.g.(x )] = [-— g (x) o —o l j —o 3x -j — i J 3 x=x has rank r , with r < n . Then there e x i s t p r e a l numbers A,, A R 1 p such that the following n equations are s a t i s f i e d : D.f(x )•+ Z A.D.g.(x ) = 0 (j = 1 , 2 , n) .v ' I f p > r , then j -o i = 1 i j i -no • v ' then (p-r) of the m u l t i p l i e r s A,, . . . , A can be assigned a r b i t r a r y 1 p values, provided only that the r c o n s t r a i n t s associated with the other r m u l t i p l i e r s form a set of r f u n c t i o n a l l y independent c o n s t r a i n t s . 17 We now give an adaptation of the argument i n Apostol: Proof: Assume, without loss of g e n e r a l i t y , that the (rxr) minor i n the upper left-hand corner of the (nxp) matrix [D.g.(x)l has rank r . i j —o Consider the following system of r l i n e a r equations i n the p unknowns A,, X : Z X^D.g. (x ) = -D,f(x ) (h = 1, 2, ..., r) • 1 P , , k h k —o h —o > • » » » • • k=l Once X r +^, Xp have been assigned a r b i t r a r y values, there i s a unique s o l u t i o n f o r X-^ , . .., X . It remains to show that the other (n-r) l i n e a r equations are also s a t i s f i e d f o r t h i s set of values for V p To do t h i s we apply the i m p l i c i t function theorem.(See Apostol (1957) , pp. 147-148) . Since r < n , every point x i n S can be w r i t t e n i n the form x = (u ; v) ... say, where u e E and v e E . In the — — — — r — n-r remainder of t h i s proof we w i l l write u f o r ( x n , .... x ) and v — 1 r — f o r (x x ) , so that v„ = x ,„ . In terms of the vector r+1' n ' SL r + £ valued f u n c t i o n j* = (g^, g ) , we can now write £ (u ; v) = 0_ when-ever x = (u_ v) belongs to ; X q . Since . ^ has continuous f i r s t - o r d e r p a r t i a l d e r i v a t i v e s on S , and since the (rxr) minor [D.g.(x )] men-i J —o tioned above has rank r , a l l the conditions of the i m p l i c i t function theorem are s a t i s f i e d . Therefore, there e x i s t s an ( n - r ) - dimensional neighborhood V q of y^ and a unique vector-valued function H = (H. , . . . , H ) , defined on V and having values i n E , such that — 1 r o . r H has continuous f i r s t - o r d e r p a r t i a l d e r i v a t i v e s on V , H(v ) = u , — o — o —o 18 and f o r every v i n V q , we have &(H(v) ; v) = 0_ . This amounts to saying that the system of r equations g^(x^, xn )= 0 , . . . , g (x.,..., x ) = 0 can be "solved" for x, , . . . . x i n terms of &r 1' n 1' ' r xr + l » . . . , x^ , giving the solutions i n the form x^ = H^(x x^) , & = 1, 2, . . . , r . We s h a l l now " s u b s t i t u t e " these expressions for x l 5 x i n t o the expression f ( x , , . . . , x ) and also i n t o each ex-1 r I n pression g ( x 1 , . . . , x ) . That i s to say, we define a new function F k n as follows: F^xr + 1 » • • •» xn ) = f f H ! ^ x r + l » * • *» x n ^ •»•••» H r ^ x r + i » • • •» x n J » x r + l " ' ' x n ^ and we define r new functions G as follows: 1 r Gk( Xr + l ' " * ' V = Sk[ BT( xr + l ' ' " ' Xn} "•'> V X r * l ' ' V ' X r + l ' • • • X n ] k = 1, 2, . . . , r . More b r i e f l y , we can write F(v) = f(R(v)) and G^(v) = gk(R(y_)) , where R(v) = (H(v), v) . Here v i s r e s t r i c t e d to l i e i n the set V o Each function G^ so defined i s i d e n t i c a l l y zero on the set V q by t h e i m p l i c i t function theorem. Therefore, each d e r i v a t i v e D, G, i s . h k also i d e n t i c a l l y zero on V , and, i n p a r t i c u l a r , D, G. (v ) = 0 . But o • h k —o by the chain r u l e (Apostol (1957) , pp. 112-114) D.G, (v ) = £ k —o n hE1 Dh8k(Wh(-V = ° (£ = 1> 2> ••*» (n"r)) * B u t = \(y) i f 1 < k < r and ^ ( v ) = ^ i f (r+1) < k < n . Therefore, when 19 (r+1) <_ k <_ n , we have D^R^v) = f 1 i f r + £ = k [o i f r+i / k . Hence the above set of equations becomes (2) J V k ^ V W + Dr+A(^o) = ° k - 1, 2, .... r 4 = 1 , 2, ..., (n-r) . By c o n t i n u i t y of H , there w i l l be a neighborhood nO^) ^ ^ G such that v e n(v ) implies (H(v) ; v) e n(x ) where n(x ) i s the neighbor-— —o — — — —o —o hood i n Apostol's statement of the theorem. Hence v e n(v Q) implies (_(_) ; v) e{Xoon(2^)y a n d the r e f o r e , by hypothesis, we have e i t h e r F(v) <_ F(v ) for a l l v i n n(v ) or el s e we have F(v) > F(v ) f o r — — —o — —o — — —o a l l v i n n(v ) . That i s , F has a l o c a l maximum or a l o c a l minimum — —o at the i n t e r i o r point v . Each p a r t i a l d e r i v a t i v e D„F(v ) must there--o £ —o fore be zero. I f we use the chain r u l e to compute these d e r i v a t i v e s , we f i n d D ^ ) = J V ( ^ o ) D A ( V = ° h=l (£ = 1, 2, (n-r)) , and hence we can write r E V ( ^ o ) D £ W + D r + £ f ( V = 0 a = 1, 2 (n-r)) (3) h=l I f we now mul t i p l y (2) by , sum on k , and add the r e s u l t to (3) , we f i n d 20 I [D,f(x ) + £ X.D.g. (x )]D H , (v ) + D f ( x ) , , h —o , , k h&k -o J £ h —tv r + £ —o' h-1 k=l + * *kWk<^> k=l = 0 for H = 1, 2, (n-r) . In the sum over h , the expression i n the P square brackets becomes [- E \ D g (x )] because of the way =r+l A,,..., A were defined. Thus we are l e f t with [D , f ( x ) + 1 r r+£ o p r <4> WA (^o ) ] + f" * Ak * V k ( x , ) D A ( v o ) ] « 0 for I = k=l k=r+l h=l 1, 2, . .., (n-r) . But, r e f e r r i n g back to equation (2), the second square P bracket of the l a s t equation (4) becomes E \, D g, (x ) . - i * k r+£ k —o k=r+l {1=1, 2, (n-r)) . Su b s t i t u t i n g t h i s i n t o (4) one gets P ^r+j^QSo) + ^ X k 8 k ^ ^ = ° = 1> 2> •••> ( n - r ) ) , which i s equivalent k=l to equation (1), and these are exactly the equations required to complete the proof. C. PREVIEW OF THE ARGUMENT TO COME We w i l l be considering the maximum l i k e l i h o o d estimation of the q.., i j k the h y p o t h e t i c a l P ^ ^ with s e t equal to zero. Consider the 9 r s t equations = 0 where q = log(n!) - E E E l o g ( f . . , T ) ' + 9 pabc i = l j = l k=l X 2 k' r s t r s t r s t 1 = l j = l k = 1 i = l j = l k = lP^k M j - 1 W « k P « k 21 a = 1 to r ; b = 1 to s ; c = 1 to t . f , r s t 9g. | 5 l _ = _ b c - {Q + £ I E n . } 3 pabc pabc i = l j = l k=l 1 3 k 9 pabc a = 1 to r ; b = 1 to s ; c = 1 to t , where g.. k - p. j k - q.. k These give us r s t non-homogeneous l i n e a r equations i n the unknowns 6 and the n. ., . The matrix of c o e f f i c i e n t s of the unknowns i s [- guCE.)] • b = 0, 1, ..., r s t ; a = 1 to r ; b = 1 to s ; c = 1 to t . 8 pabc h The f i r s t step i n applying the Lagrange m u l t i p l i e r theorem i s to f i n d the rank of t h i s matrix. t ^g It i s shown that [Q f j a Y S Z ^ 9 p h ^ ^ ' a n d h e n c e t"f • 8 h ^ - ' h a s r a n k — i j k pi j k t (r-1)(s-1)(t-1) + 1 , where [ J f Q Y Q Z ] i s an ( ( r s t + 1) x ( r s t + 1)) nonsingular matrix to be described l a t e r . Upon el i m i n a t i n g 6 from the equations ^ = 0 , the r e s u l t i n g matrix of c o e f f i c i e n t s of the 8 pabc n... has rank (r-1)(s-1)(t-1) . IJK. O r d i n a r i l y the next step would be to solve f o r the Lagrange m u l t i p l i e r s n . a n d , having found them, to su b s t i t u t e these i n t o the i j k equations —-* = 0 and to solve these equations f or q... , the re-3 pabc X J k s t r i c t e d maximum l i k e l i h o o d estimators of the P-y^ > given that the 22 = 0 . In the present case t h i s i s not necessary. It w i l l be shown that the equations are consistent and have a s o l u t i o n ( a c t u a l l y systems of s o l u t i o n s ) for the p.., when one sets p,., equal to i j k ri j k f . f • + f . f . . + f . f . . f . f . f , _ l . . • j k . j . l .k . .k i j . _ l . . . j . . .k "i j k 2 3 J n n Thus the q.., are the r e s t r i c t e d maximum l i k e l i h o o d estimators of the i j k p.., when 6... = 0 . Generally the equations are inconsistent for i j k i j k other sets o f values of the p... . ij-k D. SATISFYING THE CONDITIONS OF THE THEOREM r s t Define f( P 1 1 1 > P 1 1 2 ', , ,'pl l t Pr s t ^ = l oS n I " E z z l og (fi i k ^ i = l j = l k=l J r s t + Z E E f . . , ^ P i i k > P i i k > 0. i - l j = l k=l X 3 k *- ± j • -l j k Let S =" { ( X l , . . . , x r s ( ; ) | x £ > 0 f o r £ - 1 to r s t } . | | = - - ^ , Pi j k pi j k which i s continuous o n S , i = l , . . . , r j ~ 1»• • •, s k — 1,. . . , t . Consider the r s t + 1 c o n s t r a i n t s e, (p ,...,p ) = 0 : h 111 ' * r s t ' h - 0,1,...,rst: go(^ ' • X\ } . } , Pi j k - 1 i = l j = l k=l J 8i j k ^ - P i j k - ' U K - • i = l to r ; j = 1 to s ; k = 1 to t , where £ = [p^^^,...>P r s tl • 23 3g ° = 1 and, since constant functions are continuous, we have that g Q has continuous f i r s t order p a r t i a l d e r i v a t i v e s on S . E s s e n t i a l l y , the S^jkCpJ a r e polynomials i n the P a^ c » hence a l l of the p a r t i a l d e r i v a t i v e s are continuous. Now ft = {(x,.,...,x )|x,+.. .+x = 1 , x , > 0 f o r 1 = 1 to r s t } 1 r s t 1 1 r s t l i s the subset of S where g (x) = £ £ £ x , - 1 = 0 . ft i s ° i = l j = l k=l i 2 k the X Q to use i n the e a r l i e r case f o r the general multinomial case. When second order i n t e r a c t i o n s are zero, then X = {(x l 9...,x ^)|g, (x) = 0 ' o 1 r s t | G h — x > 0 for SL = 1 to r s t 1 h = 0,1,...,rst} Th'is X C f t C S i s what we need here, o Let us now f i n d how many 8n(p_) a r e f u n c t i o n a l l y independent on the set S of the Lagrange m u l t i p l i e r theorem. The number of S^ Cp.) that are f u n c t i o n a l l y independent on the set S equals the rank of the matrix [- e ( P ) l 8-B-8p , bh.^- , where p_ e S . If we mu l t i p l y the above matrix [ — ] by a non-singular matrix, the rank remains unchanged. Let us use 1 0 ' [ Q XQYQZ^ ' which we describe below. Note that Q stands f o r the d i r e c t product of the matrices. Let {X^u^} , u = 0 , . . . , ( r - l ) , denote a set of functions orthonormal with respect to {p- ,...,p } ; {Y^v^} , v = 0,...,(s-l) , a set X • * r . . orthonormal with respect to {p p } ; {Z^w^} , w = 0 , . . . , ( t - l ) , • J- • • s * 24 a set orthonormal with respect to {p , ...,p } ; with = 1 , • • -L • • t Y « » E l . ^ ) E l . Let , - [ X < > " ] . i - l to j = 1 to s ; Z = [ Z ^ ] , k = 1 to t . The use of X0Y0Z , where the superscript indexes the rows, and the subscript indexes the columns, and x ( u ) y ( v ) z ( w ) i g e l e m e n t o f xfiYQZ i n the (u,v,w)th row and ( i , j , k ) t h 1 J k column should reduce [- &U(E)] t o m u C n simpler form. 9 p i j k h 9 g o 9 p a b c 9 g a b c = 1 9p , 1 " [ p . b c + P a . c + Pab. + P a . . + P.b. + P . . c " 2 ( p . b . P . . c + abc 3 • • • • C 3. • » • b • <k*0 9 g a b k 8 p a b c ~ t p.bk + Pa.k + P..k ~ 2 ( p . b . P . . k + P a . . P . . k ) ] 0 * 0 a g ^ 9 p a b c " f P ' J C + ? a J - + P ' J ' " 2 ( P - J . P . . c + P a . . P . j . ) ] ( i ^ a ) 9 g i b c 8 p a b c ~ [ p i . c + P i b . + P i . . ~ 2 ( p i . . P . . c + P i . . P . b . ) ] M 9 g a i k M 9 g i b k k^cJ 7 - ^ = - [p. , ~ 2(p. p , )] * ' 9 P a b c F i . k V t i . . * \ . k / J frM 3g.. M aT^" - [ p i i " 2 ( p i p i ) ] 9 p a b c 1 J ' -J-25 3Pabc Z Z Z X fU¥v )Z .( w ) 9 gi j k _ y ( u ) v ( v ) 7 ( v ) i-1 j - 1 k-1 1 J k 8 p a b c ' Xa Yb c ( X( u ) [ Z Z Y<v )Z<w )P ., ] + Y<V ) [ Z I x f u ) Z < w ) P . .]+ [ a l. = 1 k = 1 3 k p . 3 k b i = 1 k = 1 i k lx.k Z ( W ) [ Z Z xf UM v )p.. ] + Y< v )Z ( w ) [ Z X? u )p. ] C i-1 j = l 1 2 2' b C i-1 1 1 " + x ( u ) z ( w ) [ Z Y f v ) P . ] + X ( U ) Y < v ) [ Z z < w ) P J a c . , j . j . a b k ..k J — J - K—1 + 2 [ x ( u ) [ Z Y<v)p . ] [ Z Z<w)p J + Y< v ) [ Z X< u )p. 3 [ Z Z<w)p 1 |_a 3 v. 3.. k m l k ..k b . = 1 i x.. k = 1 k ..k + Z We now make use of the orthogonality r e l a t i o n s and also introduce Lancaster's c o r r e l a t i o n notation: p = Z Z Z X ^ Y ^ t v ) • Since X < ° > H i , Y ( 0 ) U W . i-1 j-1 k-1 1 • 3 k l j k 1 , z £ ^ = 1 , t h i s gives p _ J I ( v ) (w) ovw . •, , , T k .i k u v o j = l k-1J J * ^ (u) (v) Z Z X. Y. p. . X - l J-1 J J ^A „ - v v v(uK(wK and p - Z E X. Z. p. . uow . . . . . . x k rx.k. x=l k=l P = 2 (w) O O W k=lZk P..k 1 zk zk p..k k-1 . r 1 i f w = 0 0 i f w 4 0 f1 i f u = 0 [0 i f u t 0 S i m i l a r l y p = [ and p = 1 i f v = 0 Puoo \ „ ,n ovo o i f v ^ o . Note that there i s no second order i n t e r a c t i o n i n our model i . e . 6... l j k r-1 s-1 t-1 (ijk) i f and only i f Z Z Z p = 0 . *J ' J , . , uvw u=l v=l w=l (Refer to Lancaster's "The Chi-Squared D i s t r i b u t i o n " f o r proof) Let An = f l i f 4 = m  M \0 i f A f m •*i A X i U ) Y J ( V ) z k W > ^ - X< U>Y< VV W> - { x W p + Y ( v ) p + i = l j = l k=l J pabc a b- c a Kovw b Huow '+' Z( W )P + Y<v )Z( w )A + X( U )Z( W )A + X( U )Y<V )A } + 2{X( u )A A + c uvo b c uo a c vo a b wo a vo wo Y^V )A A + Z( W )A A } .. b uo wo c uo vo Let us now consider the following cases: ( u = 0 , v = 0 , w = 0 ) . R e c a l l p = 1 . r s fc 3g i-i j-l k=i " a i r f " 1 -6 +2<3> = 1 < 1 e^ a t i o n ) J rabc (u = 0, v = 0, w ^ 0) , p = 0 , p = 1 . oow ooo j j ) 4^ Z ( W ) - 3Z ( W ) + 2Z ( W ) = 0 ((t-l)eq'ns) i = l 1=1 k=l 3p , c c c J *abc (u = 0, v / 0 , w = 0) , p = 0 , p = 1 • » l » / » ^ QVO ' ooo r s t , \ „ Z £ Z Y> ' "6i,1k _ (v) (v) (v) _ - (. n . e x i-1 j=l k-1 j 9p ~ Yb " 3 Y b + 2 Y b ° « s- 1) e <l n s ) rabc (u y 0, v = 0, w = 0) , p = 0 , p = 1 Kuoo Kooo r s t Z Z Z X< u ) ^ L j k Y(u) - 3X ( U ) + 2X ( U ) " ° ((r-Deq'ns) • _ i . _ i i -i i „ — x a a 1-1 j-1 k-1 8p a b c a (u = 0, v -jt 0, w f- 0) r s t z z i Y ^ ^ i j k v ( v ) Z W - ^ o w + Y b V ) z ^ i - i j - i k-i 3 ap a b c = Yb c = - P o w ((s-D(t-l)eq'ns) (u + 0, v = 0, w / 0) z z z x<u)z<w) ^ i j k = (u) (w) _ v<UMW>> i - 1 j-1 k=l 1 k 9 p a b c X a Z c {puow + X a Z c } " Puov (Cr-lXt-Deq'ns) (u 4 0, v 4 0, w = 0) Z Z Z X f U M v ) ^ i j k = X ( u )Y< v ) - { p + X ( U )Y U ( V )} ±=1 j - i k-i 1 J " i p T " ' ' 1 15 u v o a b rabc = - P U V Q ((r-lXs-l)eq'ns) (u / 0, v O , w H ) Z Z Z X<U )Y<V )Z<W ) ^ i j k - X ( u )Y< v )Z ( w ) - { X ( u ) p + Y<V> i - 1 j-1 k-1 2 k toTr a b • c a pouw b f rabc , . + Z( W )P . c Kuvo} ((r-l)(s-l)(t-l)eq'ns) 28 Suppose we l e t h index the rows of [— Sv(_ 1 an<^ (a> b, c) 8 p a b c h the columns. We can put h = 0 f i r s t and then h = ( i , j , k) afterwards i n some convenient order, say lexicographic order. S i m i l a r l y we order the (a, b, c) . The matrix i s ( r s t + 1 ) x r s t . Let X = [ x f U ^ ] , ( r * r ) , with u indexing rows and i columns. S i m i l a r l y f o r Y = [Y? V^] , (sxs) and Z = [ Z ^ ] , (txt) . Then J k XSYfiZ , ( r s t x r s t ) , has rows indexed by (u, v, w) l e x i c o g r a p h i c a l l y and columns by ( i , j , k) l e x i c o g r a p h i c a l l y . Consider r l _ i r — 9 — , »i , + 1) x r s t . The f i r s t row of L0 XSYSZJ 9p . . „ ShK2-Ji x r s t . — * i j k the product matrix consists of l ' s . The row corresponding to u=0, v=0, w=0, cons i s t s of l ' s . The (r-1) rows corresponding to u^O, v=0, w=0 co n s i s t of 0' The (s-1) rows corresponding t o u = 0 , v ^ O , w = 0 co n s i s t of 0's . The (t-1) rows corresponding t o u = 0 , v = 0 , w / 0 consist of 0's. The (s-1)(t-1) rows corresponding to u = 0 , v ^ 0 , w ^ 0 co n s i s t of - P t ovw s. The (r-1)(t - 1 ) rows corresponding to u ^ 0 , v = 0 , w ^ 0 consist of - P i uow s The (r-1) (s-1) rows corresponding to u ^ 0 , v ? * 0 , w = 0 consist of ~*vp I UVO S. Obviously the rank of the submatrix c o n s i s t i n g of the 1 + 1 + (r-1) + (s-1) + (t-1) + (s-1) (t-1) + (r-1) (t-1) + (r-1) (s-1) rows l i s t e d above i s one. In general the submatrix c o n s i s t i n g of the 29 remaining (r-1)(s-1)(t-1) rows w i l l have rank (r-1)(s-1)(t-1) because of the l i n e a r independence of the orthonormal functions. We now prove t h i s statement. The matrix V ( ( r - 1 ) ( s - 1 ) ( t - 1 ) * r s t ) with elements x ( u ) y ( v ) z ( w ) u ^ Q j Q ^ (u,v,w) indexing rows, (a,b,c) a b c indexing columns, has rank ( r - 1 ) ( s - 1 ) ( t - 1 ) , since i t s rows are among those of the non-singular matrix X8Y8Z . The l a s t (r-1)(s-1)(t-1) rows of 1 _ "1 f u & o 1 have elements 0 X8Y8Z. "1 ' 3 g o ] 3p_ • 3£ 1 £ "I X ^ Y ^ Z ^ ^ i - l J - l k-1 1 3 k 3 p a b c = X ( UMV)Z(W) - {X ( u )p + Y^ V ) P + Z ( W ) P } . a b c a ovw b uow c uvo (u,v,w) indexes rows (u = 1 to (r - 1 ) , v = 1 to (s-1), w = 1 to (t-1)) (a,b,c) indexes columns ( a = l to r , b = l to s , c = l to t ) . Let us denote t h i s ((r-1) (s-1) (t-1) x r s t ) matrix by U . Let ir be an (r s t x r s t ) diagonal matrix with diagonal elements p p , p > 0 . a.. .•. ..c l°ne rank of ftnv' i s <_ rank of Q . If we can show that ^ T I V has f u l l rank (r-1)(s-1)(t-1) , then Q must have rank _ (r-1)(s-1)(t-1) . But i t cannot have rank > (r-1)(s-1)(t-1) . The element i n the (u,v,w)th row and ( u 1 , v', w')th column of nirV i s ; 30 I I E p p , p [ X ( U M V ) Z ( W ) - { X ( U ) P + Y < v ) p + Z ( W ) P }] , , , , a . . . b . . . c a b e a ovw b uow c uvo a=l b=l c=l L a b c J - I E E P P , p x ^ > Y ^ Z ^ U , V V , > Z ( w * > a=l b=l c=l a" -b' ..c a b c a b c - E £ E p p , p { X ( u ) p + Y < v ) p + Z ( W ) P } X ( U , ) Y U ( V , ) Z ( W , ) , , , , • a. . .b. . .c a ovw b uow c uvo a b c a=l b=l c=l The f i r s t term on the r i g h t equals < *. pa..Xa U , Xi U , > )< \ p „ Y W v ' > > < I P Z < » V » ' > ) -a=l b=l .b. b b , . .c c c c=l In the second term I E E p p , p X ^ P x C u ' ) Y ( V ) Z ( W ) , , a . . . b . . . c a ovw a b c a=l b=l c=l 1 i f (u,v,w)=(u' 0 i f (u,v,w)^(u! - p - ( E p X( U )X( U' J ( £ p" Y K( V , )) ( E p Z( W , )) = 0 ovw 1ra . . a a \ , .b. b .. ..c c a=l b=l c=l Since E p , Y^V ^ = 0 f o r v1 ? 0 and E p Z^W'^ = 0 f o r , , .b. b . . . c c b=l c=l w' = 0 t s i m i l a r c a l c u l a t i o n s f o r the other two parts of the second term show that each part equals 0 , hence the whole second term equals 0 . The diagonal elements of fttrv1 are l ' s and the o f f diagonal elements are 0's . Hence fiirv' has f u l l rank and hence 0, has rank (r-1) (s-1) (t-1) This means that (rs + r t + st - r - s - t + 1) of the Lagrange m u l t i p l i e r s can be assigned convenient a r b i t r a r y values, and then we must solve for the other 1 + (r-lXs-1)(t-1) . 31 E. SOLVING FOR ONE OF THE LAGRANGE MULTIPLIERS r s t r s t Consider q = log(n!) - E £ £ l o g ( f !) + E E E f.., log p. i = l j = l k=l l j k 1=1 j = l k=l l j k 1 3 k r s t r s t -{Qlz £ £ P. j k - 1] + E E E n i j k t P i j k - q i j k ] . 1=1 j = l k=l J i = l j = l k=l J .29 = _§bc_ _ { 0 + E z z ^ 8j-Jk> a = i t o r ; b = l to s ; c = 1 to t 9 pabc Pabc 1-1 j - l k - i l j k 8 pabc What we have a c t u a l l y found i s the rank of the c o e f f i c i e n t matrix of the homo-geneous part of these r s t equations. f r s t 3g. ., J g _ . _ { e + S E E n.., ^ } 9 pabc pabc i - l j - l k=l l j k 8 pabc f , s t r t = *aVc~ " V c j - l k-1 a j k P ' 3 k l - l k-1 i b k P ± ' k r s r s t + E E n . . P.. + E n., p. + E n . p . + E n , , p , i = l -j=i X3C i = i l b c x " j = i a J c • J • k = i abk r..k s t r t r s - 2{ E E n p p + E S n ± b k P P . + I £ ^ i i c P i P i } j = l k=l J J 1=1 k=l 1=1 j = l J S ± n c e Pabc = ( pa..P.bc + P. b .Pa . c+ P..cPab." 2 pa..P.b.P..c? b y hy Po t h e s i s» r s t r s t r s t Z 1 1 Pabc l2 : = n-6 - E E E n q + E E E n p p a=l b - l c=l a b C 9 pabc a=l b=l c-1 a b° a b C a=l j - l k=l a j k a* * *j k r s t r s t r s t + .2, , 2, ^ ibk P. b .Pi . k + 2 .2 2 ni j c P..c Pi j . + 2 2, 2 \ b c Pi . . P.bc i = l b - l k=l i = l j = l c=l J J i - l b=l c=l r + E s E t E a=l j = l c-1 n • P • ajc r. j P + a.c r E s E t E a=l b - l k=l abk p..k Pab. 32 " 2{\ \ }. \ jk Pa . . P . J . P..k + }. }. W i . . P . b . P . . k a=l 3 = 1 k=l J J i=l b=l k=l r s t + i £ 2 n . . p . p . p } . 1=1 3 = 1 c=l J J r s t ^ r s t *L Pabc 8p , n 0 + E £ E nabc^ Pa.. P.bc + P.b. Pa.c + P..c Pab. a=i b=l c=l *abc a=l b=l c=l ( 5 ) ~ 4 p a . . p . b . P . . c 3 Set t h i s equation equal to 0 r s t Then 0 = n + ^  ^  ^  n . ^ t p . , > P > j f c + P . ^ . k + P..KP±;J. ~ 4 p i . , p . j , p . .k 3 The equations ^ = 0 a = 1 to r ; b = 1 to s ; c = 1 to t 9 p a b c represent r s t equations i n r s t + 1 unknowns; the n ' s and 0 . We found that there are ( r - 1 ) ( s - 1 ) ( t - 1 ) + 1 l i n e a r l y independent equations among these r s t equations. To a r r i v e at the equation ( 5 ) i n 6 we multiply the f i r s t of the ( r - 1 ) ( s - 1 ) ( t - 1 ) + 1 independent equations by a s u i t a b l e nonzero constant and add to t h i s multiples of the other r e -maining independent equations. We can then eliminate 0 from the ( r - 1 ) ( s - 1 ) ( t - 1 ) equations. It i s cl e a r that these ( r - 1 ) ( s - 1 ) ( t - 1 ) equa-tions are l i n e a r l y independent. p. THE SYSTEM OF EQUATIONS HAVE A FINITE SOLUTION • Now l e t us go back and look at the o r i g i n a l equations i n the form p , = 0 . If 0 has been replaced by the expression i n the n , , aoc oP i abc s. abc ' 33 then there are r s t non-homogeneous l i n e a r equations i n r s t unknowns. The rank of the c o e f f i c i e n t matrix now i s (r-1)(s-1)(t-1) . In order f o r the system of equations to have a f i n i t e s o l u t i o n , the rank of the augmented matrix must be the same as the rank of the c o e f f i c i e n t matrix. The equations p , = 0 a f t e r the expression f o r 9 has a b c 8Pabc been su b s t i t u t e d become P a b c i i j i i i + P i - - p ^ k + p - j - P i - k + p - - k P ± j - " 4 p - - p - j - p . ^ ] = (f , - np , ) (6) . Now the term of p . 4^ = 0 not i n v o l v i n g abc *abc N ' *abc 9p , " rabc the n , , a f t e r 0 has been elimi n a t e d , i s (f -np ) . In order f o r &DC 3. DC 3.DC the augmented matrix to have rank (r-1)(s-1)(t-1) , i t i s necessary that £ £ E X ^ Y ^ Z ^ C f , -np ) = 0 whenever one or more of the , , , i a b c abc abc a=l b=l c=l i n d i c e s u,v,w are zero. The above requirement for the equations p , -jr^ = 0 to be consistent automatically r u l e s out many otherwise abc 8p , abc p o s s i b l e sets of values f o r the P a^ c • Let X Cu) f.) f„> represent orthonormal functions with respect to the d i s t r i b u t i o n s (p. } , {p . } , {p ,} r e s p e c t i v e l y , as before. X • • • J * • • K. Let X , Y , Z represent orthonormal functions with respect to f . f . f , the e m p i r i c a l d i s t r i b u t i o n s { n * *} > { ^J *} , { ^* } r e s p e c t i v e l y . 34 Let p - Z Z Z X ( U ) Y ( V ) Z ( W ) P , uvw i t ! i a b e abc a=l b=l c=l ; - Z I Z i(u) Y(v>2(v) fabc 'uvw i , i i a b c n a=l b=l c=l p = Z Z I X ( U ) Y U ( V ) Z ( W ) ^ U W a -1 b - l c - 1 A B C N , Then p = 1 , p = p = p = 0 for u ^ O , v ^ 0 , w ^ 0 ooo uoo ovo oow S i m i l a r l y f o r p but not for p , though P 0 0 0 = 1 • f f , + f . f + f f , 2f f . f T T , a., .be .b. a.c ..c ab. a., .b. ..c> W r i t e qabc = ( 2 3 ) n n r s t Z Z Z X ( u ) Y ^ v ) Z ( w ) q , = A p + A p + A p - 2A A A „ i i . i i a b c abc uo ovw vo uow wo uvo uo vo wo a=i b=l c=i where A 1 for g = h r w l l W n s f since Z X ( u ) X ( 0 ) - ^ [0 for g 4 h , a=l a a n 1 f o r u = 0 0 f o r u 0 Thus Z Z Z X ( u ) Y , ( v ) Z ( w ) q K a-1 b=l c=l a b c a b c 1 for u = 0 , v = 0 , w = 0 0 for any two of u, v, w = 0 p f o r u = 0 , but v ^ 0 , w 5^  0 ovw ' Puow f o r V = °» b u t u * °» w * 0 Puvo f o r W = °» b u t u * °» v ^ 0 0 for u ? 0 , 0 , w ^ 0 S i m i l a r l y f o r Z Z Z X ( u ) Y ^ v ) Z ( w ) p , , where the p ^ represent i , i i a b c abc abc a=l b=l c=± t h e i r hypothetical values, except that p i s without ^ . 35 r S t ;(u)^(v);(w) fabc _ z z a=l b=l c=l Z XN 'YZ"Z' a b c n uvw 1 for u = 0, v = 0, w = 0 0 for any two of u, v, w = 0 p for u = 0, but v 4 0, w ± 0 ovw p for v = 0, but u 4 0, w 4 0 uow p for w = 0, but u 4 0, v i 0 uvo p f o r u / 0 , v ? * 0 , w / 0 uvw Thus Z I \ X( u>Y<V>Z( w ) v „ u , , , , a b c abc a=l b=l c=l ^ o V i r - n^abc^ 0 when at le a s t one of the u, v, w i s zero np when a l l indices are not zeros On the other hand Z Z Z x ( u ) Y , ( v ) Z ( w ) = p , which usually a=l b=l c=l f n uvw 5^  p where p 4 ' 111JT.T 1 '1 uvw , e t c . Also p , p , p can not be ex-^uoo *ovo Koow pected to be zero when p 4 e t c . Thus Z Z Z X ^ Y ^ Z ^ ( f , a . . n , , - - a b c abc a=l b=l c=l np ) = n(p - P ) , where p . has i t s hypothetical value. For abc uvw uvw rabc J t r u = 0, v = 0, w = 0, one has n(p - p ) = 0 . In the other cases, ooo ooo one has n(p - p ) . uvw 'uvw We conclude that i f the equations (6) are m u l t i p l i e d by x ^ Y ^ Z ^ a b c and then summed with respect to a, b, c, then, i n general, the r e s u l t i n g r s t equations w i l l have on the r i g h t side n(o - p ) , of which uvw uvw ( r s t - 1) may be expected to be d i f f e r e n t from zero. However, the rank of the matrix of c o e f f i c i e n t s of n ^ c on the l e f t side i s (r-1)(s-1)(t-1), so that the equations w i l l be inconsistent except f or s p e c i a l sets 36 {p } , {p }, {p } which leave an appropriate set of not more than a. •« • D • • • c (r-1)(s-1)(t-1) terms on the r i g h t of the equations non-zero. We want f f . f to v e r i f y that {p } = {-^} , {p , } = {-^} , {p } = {-^} 3. • * - II • D • n • • C II w i l l accomplish t h i s . A set of non-homogeneous l i n e a r equations are c o n s i s t e n t , that i s , have a s o l u t i o n <=> rank of c o e f f i c i e n t matrix = rank of augmented matrix ... <=> column space of c o e f f i c i e n t matrix = column space of the augmented matrix. We have shown that the vector cj^  l i e s i n the column space of the vectors X^ u) e y/ v) 8 , where u ^ O , v ^ 0 , w ^ 0 , but t h a t , i n general, the same can not be said of other p o s s i b l e p_ . If we can show that the column space of 3 gi i k [8 P^ b7 + p i . . p . j k + p . j . p i . k + p . . k p i j . - 4 p i . . p . j . p . . k ] e q u a l s t h e column space of {X( u ) 8 Y( v ) 8 Z_(w) |u f 0, v 4 0, w 4 0} then the set of equations w i l l be consistent when p_ = q_ . Note that here (a, b, c) indexes rows, ( i , j , k) and (u, v, w) columns. The column space of the c o e f f i c i e n t matrix above i s a subspace of rst-dimensional Euclidean space, E . The matrix was shown to have v r s t r-^ .nk (r-1) (s-1) (t-1) , hence i t s column space i s an (r-1) (s-1) (t-1) sub-space of E r s t Now the set of vectors X( U ) 0 Y( V ) 8 Z( W ) u = 0 to (r-1) v = 0 to (s-1) w = 0 to (t-1) forms a basis f o r E ^ . We w i l l show that the column space of the r s t r 37 c o e f f i c i e n t matrix i s orthogonal to the subspace spanned by the set of vectors {X^u^ 8 y_^v^ 0 | one or more of u , v, w i s zero} . This i s a 1 + (r-1) + (s-1) + (t-1) + (s-1) (t-1) + (r-1) (t-1) + (r-1) (s-1) = r s t - (r-1)(s-1)(t-1) dimensional subspace. Hence i t s complement, the (r-1)(s-1)(t-1) -dimensional subspace spanned by the set of vectors { X ^ © Y ^ § TSW^ | u ^ 0 , v ^ 0 , w ^ 0 } , i s the same as the column space of the c o e f f i c i e n t matrix. We have shown that the vector (f - nci) l i e s i n t h i s subspace. Hence the set of equations would be consistent when (f_ - ng_) i s the non-homogeneous part of the equations. To show that the column space of the c o e f f i c i e n t matrix i s orthogonal to the subspace spanned by the set of vectors {X^u^ Q © z S W ^ \ one or more of u, v, w i s zero} , r e c a l l Z I Z X< U )Y< V )Z< W ) = 1=1 j = l k=l 1 J k 9 pabc r 1 for u = v = w = 0 0 i f two of u, v, w are 0, one not 0 p i f one of u, v, w i s 0, two not 0 uvw f x ( u ) y ( v ) z ( w ) _ { x ( u ) + (V) + a b c a Kovw b "uow r, (w) \.r s: Z p )xf none of u, v, w xs . c uvo R e c a l l also that Z p. = 0 for u i 0 Z Z Y fv )Z l( w )p . o p , e t c . . i , i J k .jk ovw j = l k=l J J • c -Hence Z Z E X Y Z [ J + p p + p p 1=1 j = l k=l 1 2 k 9 pabc x" -J k O- i-k + P. . kPi j . " 4 pi . .P. j .P. . k] 38 r 0 i f at l e a s t one of u, v, w i s 0 x ( u ) (v) z(w) _ { x ( u ) • + Y ( v ) + z(w) } ± f ^ o f v w . s a b c a ow b uow c uvo This shows that the column space of the matrix 9 gi i k t ^ b 7 + Pi . -P. j k + P. j .Pi . k + P. . kPi j . " 4 pi . .P. j .P. . k] ^ orthogonal to a l l the vectors X^U^ 8 8 for which at l e a s t one of u, v, w i s 0 G. THE NUMERATOR OF THE LIKELIHOOD RATIO The numerator of our l i k e l i h o o d r a t i o we now see to be r s t If. f + f . f + f f . . - 2f. f . f n j n n n I 1 • * '-1 • J * 1 • • 1 J • i = l j = l k = l \ n2 H. SUMMARY OF CHAPTER III In t h i s chapter we discussed t e s t i n g the hypothesis of no second order i n t e r a c t i o n . We chose the method of the l i k e l i h o o d r a t i o . There are no d i f f i c u l t i e s i n determining the denominator of the r a t i o . To a s s i s t us i n determining the numerator of the l i k e l i h o o d r a t i o we introduced a theorem about Lagrange m u l t i p l i e r s , modified i t , and proved the modified v e r s i o n . We then proceeded to show that our problem does s a t i s f y the conditions of the modified theorem and hence our use of the modified theorem was v a l i d . We f i n a l l y a r r i v e d at an expression f or the numerator of the l i k e l i h o o d r a t i o . One could a l t e r n a t e l y think of t h i s chapter as f i n d i n g the maximum l i k e l i h o o d estimators of the p..,» f i r s t under the most general assumptions l j k. 39 then under the hypothesis of no second order i n t e r a c t i o n , that i s , 6.., = 0 l j k f o r every i , j , k . We could then decide to t e s t the hypothesis of no second order i n t e r a c t i o n using the l i k e l i h o o d r a t i o c r i t e r i o n or -2 log ( l i k e l i h o o d r a t i o ) , or e l s e we could use 2 r S t f i i k " n q i i k 2 X = E E £ (— J J~) , where t h i s q.. v i s the r e s t r i c t e d 1=1 j = l k=l nq. maximum l i k e l i h o o d estimator of p.., under the hypothesis that second * i j k order i n t e r a c t i o n i s zero. 40 CHAPTER IV CONTINGENCY TABLES FOR WHICH THE HYPOTHESIS OF NO SECOND ORDER INTERACTION IS NOT MEANINGFUL O r d i n a r i l y the p.., can vary over a f i n i t e r e g i o n . When the marginal i j k t o t a l s are s p e c i f i e d , that leaves an (r-1)(s-1)(t-1) subspace for the p. i j k to vary over. We expect that the maximum l i k e l i h o o d estimators of the q. , i j k w i l l u sually l i e i n the i n t e r i o r of t h i s region. Occasionally they may have to l i e on the boundary, on one of the faces of the region or where two or more of the faces i n t e r s e c t . Now P(neither A nor B, but C) = P(C) - T{(AUB)C} = P(C) -P{(AC) U (BC)} = P(C) - P(AC) - P(BC) + P(ABC) _> 0 . Hence P(ABC) _> P(AC) + P(BC) - P(C) . Thus i t follows that max{0, P ± > k + p . k - P > < k , P i j > + P . > k - p . ^ , P ± j > + P > j k - p . } <^  p... _< min{p , p. , p.. } . We have not r e a l l y used these i j k . j k 1 . k i j . r e s t r i c t i o n s i n de r i v i n g the maximum l i k e l i h o o d s o l u t i o n . In a three-way contingency table i t i s sometimes pos s i b l e f or one or more q... to be i j k < max{0, p . k + P ± j > - p . ^ , p . ^ + p . k - p . , P > j k + p i k - P > k} or > min{p , p. , p,. } , which i t i s impossible for p... to s a t i s f y . • j ic i • tc i j * i j R. S i m i l a r l y f o r 3^^. a n c* t n e corresponding r e l a t i v e frequencies. Let ft' denote the set of r e a l non-negative numbers {p ,. . . , p } 111 r s t which are possible p r o b a b i l i t i e s i n an r x s x t contingency table with 41 given marginal probabilities. In the cases indicated above with the P.yk set equal to q ^ or P j j k s e t e c l u a - ' - t o q i j k ' t b e P°i n t does n o t ^ e in ft' . There i s no solution (p 1. 1,...,p ) lying within ft' under 111 r r s t J ° the conditions imposed. We do not want to tamper with the marginal totals or the two-way table interactions, so what can we do? Since p.. k = P . > < P > j k + P . jA.k + P . . k P i j . " 2 p i . . P . j . P . . k + 6 i j k ' expression q. j k < max{0, p . < k + p..# - P ± < > > p..> + p _ j k - p p < j k+ P 1 > k - P „ k > P., > implies that ^ - ^ j k > ^ » while expression ^ > min{p .jk' P i . k ' ^ i j . implies that < ^ • This suggests that when ( P m * • • • > p r s t) l i e s outside of ft' we not hypothesize that a l l of the ^ - j j k a r e z e r o > but ..rather,allow certain contrasts of the &..., to be.non^zero while hypothe-i j k sizing that a l l linear combinations of the 6.., orthogonal to the given xjk contrasts be zero. Possibly one could estimate the direction of this vector (the vector of coefficients of the 6.., in the non-zero contrast), l j k but i t i s much simpler to prescribe the direction to begin with and just worry about the length of the vector. Consider the following marginal totals of a 3x2x2 contingency table: C 1 U C 2 B l B2 Sum V B 2 C l C2 Sum A 1 U A 2 U A 3 V C2 Sum A l 1/12 3/12 1/3 A l 1/24 7/24 1/3 B l 1/12 5/12 1/2 A 2 2/12 2/12 1/3 A2 2/24 6/24 1/3 B2 . 2/12 4/12 1/2 A 3 3/12 1/12 1/3 A 3 3/24 5/24 1/3 Sum 1/2 1/2 1 Sum 1/4 3/4 1 Sum 1/4 3/4 1 Note that p^ P i .2. P..] = 1 / / 4 ' p. 1/3 1/2 3/4 Consider the following table: 42 ( i j k ) Pi • +P. .k" "Pi Pi l ^ O k "P, i -Pi •k+ P i k "P> .k po s s i b l e values of Pi j k 111 1 12 -8 24 -5 24 1 12 + 1 12 6 12 -8 24 1 24 + 1 12 6 24 = -3 24 0 to 1 24 112 1 12 -8 24 = 1 24 1 12 + 5 12 • 6 12 0 7 24 + 5 12 18 24 -1 24 1 24 to 2 24 121 3 12 8 24 -1 24 3 12 2 12 6 12 -2 24 1 24 2 12 6 24 -1 24 0 to 1 24 122 3 12 -8 24 = 5 24 3 12 + 4 12 6 12 2 24 7 24 + 4 -12 18 24 = -3 24 5 24 to 6 24 211 2 12 8 24 -2 24 2 12 4-1 12 6 12 -6 24 2 24 1 12 6 24 -2 24 0 to 2 24 212 2 12 • i f - 8 24 = 2 24 2 12 + 5 12 6 12 2 24 6 24 + 5 12 18 24 = -2 24 2 24 to 4 24 221 2 12 -8 24 = -2 24 2 12 + 2 12 6 12 -4 24 2 24 + 2 12 6 24 = 0 0 to 2 24 222 2 12 — 8 24 = 2 24 2 "12 + 4 12 6 12 0 6 24 4- 4 -12 18 24 = -4 24 ' 2 24 to 4 24 311 3 12 -8 24 = 1 24 3 12 + 1 12 6 12 -4 24 3 24 4-1 12 6 24 = -1 24 1 24 to 2 24 312 3 12 -8 24 = 3 24 3 12 + 5 12 6 12 4 24 5 24 4-5 12 18 24 = -3 24 4 24 to 5 24 321 1 12 -8 24 -3 24 1 12 4-2 12 6 12 -6 24 3 24 4-2 12 6 24 = 1 24 1 24 to 2 24 322 1 12 8 24 -1 24 1 12 + 4 12 6 12 -2 24 5 24 4-4 12 18 24 -5 24 0 to 2 24 TABLE XIV UPPER AND LOWER BOUNDS OF p.., i j k 43 Let us evaluate q m . q m = ? l t ? m l i + p . l . p l . l + P..1 P11. " 2 p l . . P . 1 . P . .1 .1 1 x •. ,1 1 \ . / l I N 9 / l 1  = ( 3 X l 2 ) + (2 X 24} + (4 X l 2 } " 2 ( - 3 x 2 x i - =| . S i m i l a r l y q ^ ^ . We now enter the q.., i n the c e l l s of the t a b l e . For ( i , j , k) = (1, 1, 1) the i j k entry i s -1/72 ; for ( i , j , k) =(2, 1, 1) i t i s 2/72. cl Bl B2 Sum C2 B l B2 Sum Cia C2 Bl B2 Sum A f—1 j C4 J 3 A (l\ / l 4 j 21 A 6 18 24 Al \72/ 72 Al \7_2y: 72 Al 72 72 72 A 2 4 6 A 10 8 18 A 12 12 24 A 2 72 72 72 A2 72 72 72 A2 72 72 72 A 5 4 9 A 13 2 15 A 18 6 24 A 3 72 72 72 A 3 72 72 72 A 3 72 72 72 Sum 6 12 18 Sum 30 24 54 Sum 36 36 1 72 72 72 72 72 72 72 72 X A ]UA 2UA 3 Cl C2 Sum B1U B2 Cl C2 Sum Bl . 6 30 36 A 3 21 24 72 72 72 Al 72 72 72 B2 12 24 36 A 6 18 24 72 72 72 A2 72 72 72 Sum 18 72 54 72 1 A3 9 72 15 72 24 72 Sum 18 72 54 72 1 The c i r c l e d entries are impossible. The i n t u i t i v e way to apportion the 6 contributions of the layers f o r and, A i s to make the 6 contributions proportional to p and p (or to f and f ) r e s p e c t i v e l y . In the example p at l e a s t f . = f p = 1/3 (or = 24) . L e t qi j k= f . f + f . f + f f . . " 2f. f . f 1« » «J rC * J * X»fC « • rC 1J « — X<» * J * « • rC n n In our example the q i j k g i v e u s i m p o s s i b l e estimates: 44 q^^^ = - 1/72 < 0 , e t c . Suppose that 6 -( £2 . .+ f3 . . J * n 111 " n "U " n ) X u211 n f 3"A "311 n Then &m + ^211 + ^311 = ^ * l h~e ^n t*i e o t n e r columns are automatically s p e c i f i e d , since 6 j J 1 + 6 ± j 2 - 0 , 6 ± l k + 6±2k - 0 . The problem then i s to estimate \ , which i s proportional to the length of the c o e f f i c i e n t vector of the 6 co n t r a s t . ( f 9 + f , ) . ( f , + f ) Pl l l= ql l l n : P121 = q121 + n ~X f2 - f2 P211 " q211 + ~n~~X p221 = q221 " 1T~ X f3 F 3 P311 - P311 + ~n~X P321 = q321 " IT"1* ~ ~ £ 2 ^ _ ~ f2.. P212 ~ q212 n X p222 " q222 + ~1T~~A ~ ~ f3 ~ f3 P312 " q312 ~a~X P322 = q322 + "IT^ (f + f )X . (f + f )X P112 q112 n P122 " ql 2 2 " n 3 2 2 p. i j k P = n; n n n i = l j = l k=l 1i j k -3 2 2 3 2 2 Q = log n!.+ Z Z Z f . . . l o g p - Z Z Z f ' i = l j = ! H1 ] k l j k i - l j = l k=l l j k 45 "2. . 211 n f 2 + f U " - f + f " . ql l l " ( 2" n 3" )X q2 1 1 + n •311 n f 2 - f q311 + T~A f2 + f3 f2 f3 r i 0 1 v ~ / + 121 v n 221 n 321 n f 2 + f 3 f 2 . f q121 + ( n U q221 " T~X q321 - ~~n~ A f „ + f „ f. f2 f3 _ <• 2. . 3.. N 212 n 312 n + r n o v. ; -112 v n f f u. / 2 . . *3.. x, q212 " ~n~~ A q312 " ~n~X ql l 2 + ( n ) X f2 + f3 f2 f3 "112 v n "' "222 n 322 n f + f f2 ^3 q112 ~ (-^ -= — ) X q222 + 1T~ A q322 + ~n~^~A n In C R . Rao's text "Advanced S t a t i s t i c a l Methods i n Biometric Research" we f i n d the method of scoring for the estimation of parameters. This method works w e l l with maximum l i k e l i h o o d equations which are too complicated to solve d i r e c t l y . The above equation i s such a one. The general method i n such cases i s to assume a t r i a l s o l u t i o n and derive l i n e a r equations for small a d d i t i v e c o r r e c t i o n s . The process can be repeated t i l l the corrections become n e g l i g i b l e . 30 The quantity -77- i s defined as the e f f i c i e n t score f o r X . The maximum OA l i k e l i h o o d estimate of X i s that value of A for which the e f f i c i e n t 46 score vanishes. If A i s the t r i a l value of the estimate, then expand-o ing and r e t a i n i n g only the f i r s t power of d0 = 6 - 0 , O A O dx - dx + d A _ 2 O d A O 90 ~ -r-f dX I (A ) where I (A ) , the information at the value X = X , — dA o o o o 2 i s the expected value of 9 Q . In large samples the d i f f e r e n c e between dX 2 2 - I(A ) and 9 Q w i l l be of 0(l/n) , where n i s the number of observa-0 2 9A z o t i o n s , so that the above approximation holds to the f i r s t order of small q u a n t i t i e s . The c o r r e c t i o n dA i s obtained from the equation dAI(A ) = | £ o 9A o o The f i r s t approximation i s (A + dA) , and the above process can be re-peated with t h i s as the new t r i a l value. This chapter has by no means exhausted the possible methods of t r e a t i n g three-way contingency tables for which the hypothesis of no second order i n t e r a c t i o n i s not meaningful. It does however propose an idea of how these s i t u a t i o n s can be dealt with. 47. CHAPTER V EXTENSION TO HIGHER DIMENSIONAL CONTINGENCY TABLES. A. TWO EXTENSIONS OF THE THREE DIMENSIONAL MODEL TO THE FOUR DIMENSIONAL CASE. Consider a q x r x s x t contingency t a b l e . Consider as wel l the following two poss i b l e extensions of our model: (34) E ( fh l j k) = n ph i j k = n [ ph . . . P. i . . P. . j . P...k + Ph . . . P. i . . aj k (.24) (23) (14) (13) * Ph . . . P. . j . ai k Ph . . . P...k 01 i j + P. i . . P. . j . ahk + P. i . . P...k ah j (12) (234) (134) (124) (123) + P. . j . P...k ah i + Ph . . . ^ i j k + p. i . . ^hjk + P. . j . ^hik P...k ^ h i j + 9 h i j k ] > h = 1 to q, i = 1 to r , j = 1 to s, k = 1 to t with (34) (34) (24) (24) (23) (23) (14) (14) a , = a. = 0; a . = a. = 0| a , = a. - 0; a, , = a, =0: .k j . .k i . . j i . .k h. (13) (13) (12) (12) (234) (234) (234) a. j = V = 0 ; a. i = V = °5 6. j k = 6i . k = 6i j . = °; (134) (134) (134) (124) (124) (124) (123) (123) • j k h.k h j . . i k h.k h i . .13 h.3 (123) = 6, . =0: 6 " = 6, = 0, . , = 0, . . =0 where the dot indi c a t e s h i . . 13 k h. 3 k h i . k I113. summation over the replaced s u b s c r i p t . 1 1 ' E(fh-HlP = nPh^T, = nfPv, P -r P A P 1 + P ,(34) h i j k ' " "Fh i j k " " ^ h . . . F. i . . H. . j . ^ . . . k ^ P h i . . uj k ,(14) t(23) ,(14) .,(13) ,(12) + p, . a., + p, , a. . + p . . a,. + p . , a, . + p ., a, . ^ h . j . i k *h..k i j . i j . hk . i . k hj . .jk h i ,(234) .,(134) ,(124) ,(123) , + p, • '6... + p , 6. + p . 6... +'p , 6 . . . + 8 . . , . ] , ^h... i j k . i . . hjk ..3. h i k . ..k hi3 hi3k 48. h = 1 to q, i = 1 to r , j = 1 to s, k =-1 to t with ,(34) ,(34) ,(24) ,(24) ,(23) ,(23) ,(14) ,(14) a , = a . = 0; a , = a. = 0 ; a . = a. = 0 : a . = a, - 0 ; .k j . .k I . .3 1. .k h. .,(13) ,(13) ,(12) ,(12) ,(234) ' ,(234) ,(234) a . = C L = 0; a . = a, = 0; <5 = <$.., = 6.. =0; .3 h. .1 h. .3k x.k x j . ,(134) ,(134) ,(134) ,(124) £ 1 2 4 ) 0 2 4 ) ,(123) ,(123) <5 .. = 6. . = <S, . = 0; <5 = 6. . = <S . = 0; 6 . . = 6, . .3k h.k hj . ,xk h.k .,k .13 - h.3 0-23) 1 i • , ' 1 = 6, . = 0: 8 .... = 9. ., = 6, . . = 9 ; . = 0 where the dot ind i c a t e s hx.. .X3k h o k hx.k hx3. summation over the replaced s u b s c r i p t . Note that i f we sum model I over h we obtain (34) (24) E C f. i j k) = n p. i j k = n [ p. i . . p. . j . P...k + p. i . . ° j k + P. . j . " i k . (23) (234) + p , a . . +6.., ] which we note i s the model for the three ...k xj X3k dimensional case. The same r e s u l t holds true i f we sum over h i n the second extension of our model. Let us now determine 9, ... . E(f .. ) = np ., hX3 k ..3k • • 3 k (34) = n [ p . . 3 . P . . . k + a j k ] (34) j k • • J rC • • j • •••lc C24) (23) a4) (13) (12) S i m i l a r l y f or , a.. , a h k , c ^ . , ^ E(f ... ) = np ... = nip . p . p , + p . a f3 4 )+ p . aC 2 4 )+ p a( 2 3 )-f5 .X3k .xjk i r .x. .*\ .3 . .k *\x.. 3k . . j . . i k . . .k i j i j k J (234) 49, (234) (34) (24) (23) > 6i j k . P. i j k P. i . .P. . j . p...k P. i . . aj k P. - j . " i k P. . . kai j • x j lc • x • * • • j • 4 • • lc • i • • »«jlc •»j • • x • lc •••lc • ij • (134) (124) (123) S i m i l a r l y f o r 6 h j k , 6 ^ , 6 ^ . . (34) 8h i j k = ph i j k ~ Ph . . . P. i . . p. . j . P...k ~ ph . . . p. i . . aj k (24) (23) (14) (13) Ph . . . P. . j . ai k Ph . . . P...k " i j P. i . . P. . j . "hk P. i . . P. . , kah j (12) (234) (134) (124) - P . p i a, . - p, 6... - P . <5,., - p . 6, . . j . ...k h i rh . . . l j k . 1 . . hjk • «J • h i k (123) - p ^ ^ h i j * A f t e r s u b s t i t u t i n g the expressions j u s t derived for (34) (234) a., and the other a's and f o r 6.., and the other <5's int o the j k l j k formula f o r ^ ^ ^ j ^ ' a n^ combining terms we obtain: 6h i j k = Ph i j k ~ 3 ph . . . P. i . . P. . j . P...k ~ Ph . . . P. i j k ~ P. i . . Ph . j k _ p. . j . ph ± . k ~ p. . . kph i j . + ph . . . p. i ; . p. . j k + ph . . . p. . j . p. i . k h««• • • . k » i j • • i • • • • j • h •«k • i •« • • • k h * j • • • j * • • • k h i • • t Let us now determine 0 h i j k . From the second extension of our ,(34) ,(34) model fitf ) - n P > t j k - n [ p > # p > > 4 k + a j k ] = > a j k - P.. j k-P..j.P...k 50. ,(24) ,(23) ,(14) ((13) ,(12) ,(234) S i m i l a r l y f or a.. , a.. , a, , , a. . , a,... . 6.., = p . J l k I J h k T i j ' h i x j k . i j k • X • * • • J « • • • lc • X • • »• j lc • • j • * x • lc •••lc « X J, ,(134) ,(124) ,(123) , S i m i l a r l y f o r 6, ... » 5, M »<$,.. . 8 . . M = p, . M -1 hjk ' hik ' hxj hxjk hxjk ,(34) ,(24) ,(23) Ph . . . P. i . . P. . j . P..k Ph i . . aj k ph . j . ai k ph..k ai j ,(14) ,(13) ,(12) ,(234) ,(134) P. i j . ahk P. i . k ah j P. . j k a'hi Ph . . . ^ i j k P. i . . hjk ,(124) ,(123) - p . - p . 6\ . . . A f t e r s u b s t i t u t i n g the expressions j u s t • • j • rixfcc • • • K nxj ,(34) , ,(234) derived f o r a., and the other a ' s and for <5.., and the othei c5' s i n t o the formula f o r 8h i j k ' a n^ combining terms we obtain: 6h i j k Ph i j k ^Ph . . . P. i . . P. . j . P...k + 3 ph i . . P. j . . P...k + 3 ph . j . • x • * • • • lc Ix • • lc • x • • • • j • • x j • h*»» • • • lc • x • lc h • • • • • « + 3 p. . j k ph . . . p. i . . - 2 PM . . p . . j k - 2 p h . j . p . i . k - 2 p . i j . ph..k " ph . . . p. i j k - p. i . . ph . j k ~ p. . j . ph i . k - p...k ph i j . B. LANCASTER'S DEFINITION OF INTERACTIONS COMPARED TO OURS Consider f o r a moment the following d e f i n i t i o n of i n t e r a c t i o n s given by Lancaster i n h i s text "The Chi-Squared D i s t r i b u t i o n " (1969) pp. 254-256: Let F. , F. , F. . . denote the one-, two-, three-, 3 1 1 X 1 1 1 1 2 1 2 3 ... dimensional d i s t r i b u t i o n f u n c t i o n s , where i ^ < i^ < i ^ < 51. i Let F denote the (n-1)-dimensional d i s t r i b u t i o n function of the set II complementary to the p a r t i c u l a r random v a r i a b l e chosen, F as the (n-2) dimensional d i s t r i b u t i o n function of the (n-2) v a r i a b l e s complementary to the p a r t i c u l a r p a i r ( i ^ , i 2 ) chosen, and so on. With t h i s convention, tlx i n t e r a c t i o n s of the (n-1) order are said to e x i s t i f , and only i f , F cannot be displayed as a sum F = I F . F ' - £ F F . F " + Z F . F . F F'" - ( ~ l ) n E F F . . F F ^ 1 " Xl 11 x2 Xl X2 X3 H X2"' (n-1) + ( - l )n 1 F. F. F. Xl X2"' \ where the summation i s over a l l combinations of indices ( i - i „ . i , ) , (n—1) k = 1, 2,..., (n-1). We have w r i t t e n F to mean F with (n-1) primes as s u p e r s c r i p t s . The l a s t two terms are of the same form and may be consolidated as ( - l )n (n-1) F. F. F. , n > 2. X- X 0• • • • X 1 2 n For three v a r i a b l e s , X, Y, Z second order i n t e r a c t i o n s e x i s t i f F cannot be written as the following sum: F = F X(X) F 2 3(YZ) + F 2(Y) F 1 3(XZ) + F 3(Z) F 1 2(XY) - 2F X(X) F 2(Y) F ^ Z ) . Now l e t X, Y, Z be code random v a r i a b l e s : X = i when an observation f a l l s i n the category of the f i r s t c l a s s i f i c a t i o n , i = 1 to r , 3 and so on. The d i s t r i b u t i o n s are then d i s c r e t e . Then p.., = A F Y ( i , j , k ) 3 3 or A F ^ 2 3 ( i , j , k ) , where A denotes a f i r s t d i f f e r e n c e with respect to three, v a r i a b l e s . (See Samuel S. Wilks (1962) Mathematical S t a t i s t i c s pp 39-41, 49-50 about diff e r e n c e s . ) 52. If one has F(x, x ) g& x ) , then A n(Fg) = (A) ( A n ~ m g ) . l , . . . , m m+l,..., n Thus Lancaster's expression i n terms of cumulative d i s t r i b u t i o n s i s equivalent to ours. Our expression i s the three v a r i a b l e f i r s t d i f f e r e n c e of Lancaster's expression. This would be so even i f the d i s t r i b u t i o n s were not d i s c r e t e . Consider now the case of four v a r i a b l e s W,X,Y,Z. Third order i n t e r a c t i o n s e x i s t i f F cannot be w r i t t e n as the following sum: F = FXCW) F 2 3 4(X,Y,Z) + F 2(X) F^CW.Y.Z) + F ^ Y ) F ^ W . X ^ ) + F 4CZ) F 1 2 3(X,Y,Z) - F^W) F 2(X) F^CYZ) - F^W) F 3(Y) F ^ X . Z ) -FXCW) F 4(Z) F 2 3(X,Y) - F 2CX) T^Y) F14(W,Z) - F 2(X) F 4(Z) F^CW.Y) -F 3(Y) F 4(Z) F 1 2(X,Y) + 3F1CW) F 2(X) F ^ Y ) F 4CZ) . Similar to the case of three v a r i a b l e s , l e t W, X, Y, Z be code random v a r i a b l e s which take on the values 1 to q, 1 to r , 1 to s, 1 to t r e s p e c t i v e l y . It i s easy to see that t h i r d order i n t e r a c t i o n i s present i f f Ph i j k + Ph . . . P. i j k + P. i . . Ph . j k+ P. . j . Ph i . k + P..k Ph i j . Ii • • • • x • • •«j lc l i • • • * * 3 * • x • lc l i • • • •••lc • x j • • x • • • • j * Q • • • lc • x • • . • • •lc l i » j • • • j • •••lc lix • • l i • • • • zL • • p . p , f o r some h, i , j , k. (This i s the four v a r i a b l e f i r s t • • J • • • • IC d i f f e r e n c e of Lancaster's expression.) i . e . i f f 9, .., of our f i r s t h i j k extension of our 3-way model =j= 0 f o r some h, i , j , k. Hence for the 53. 4-dimensional case, only our d e f i n i t i o n of the presence of 3rd order i n t e r a c t i o n i n our f i r s t extension of our o r i g i n a l model coincides with that of Lancaster's. C. THE GENERAL CASE FOR THE FIRST EXTENSION OF OUR 3-WAY MODEL. There are w p a r t i t i o n s or c l a s s i f i c a t i o n s of the sample space. We d i v i d e the set {1, 2,..., w} in t o two d i s j o i n t p a r t s , { r ^ , r 2 , . . . »ru ^ and {s., , s O J...,s }, with u + v = w. For de f i n i t e n e s s we take J. l. v r , < r „ < ... < r and s, < s« < ... < s . Let T denote the ordered 1 2 u 1 2 v u u-tuple ( r , , r „ , . . . , r ) and ib the ordered v-tuple (s. s„,..,s ) . 1 2 u v 1, 2 v Let i , = 1 to m be the subscript f o r the r, - th p a r t i t i o n and n r. n h j , = 1 to m the subscript f o r the s, - t h p a r t i t i o n . Let a denote k the ordered u-tuple (i., , i ) . Let a denote the ordered v-tuple 1 2 u v Cj-i, Jo» • • • »J )• We w i l l write pCr,; i,) to denote p . * x z v n n * • • * * • • • * (r, 1 ) d o t s (w-r,)dots. h-1 n The v-f a c t o r i n t e r a c t i o n s i n v o l v i n g the p a r t i t i o n s s^, s 2 > . . , s v can be w r i t t e n AOJ^; a^) or sometimes ACs^s^,.. j S ^ ; j^>J 2>.. . j ) . ^ constraint on such a v-f a c t o r i n t e r a c t i o n i s m Sk £ A C s 1 , s 2 , . . . , s ; J 1,J 2,..,J v> = 0 where k = 1,2,..., or v where_ jk= 1 54. The general case for the f i r s t extension of our 3-way model, can be w r i t t e n E C f > • = n { J i p^»v + \ i t v ] a ( V a v ) } 1 2 w 1 2 w v=2 lb h=l h Tv where i t i s understood that u = w-v, and that 1, - Z . The summation h rh h i s a summation over a l l p o s s i b l e v-tuples with 1 < s, < s„ < ... < s < w. V = 1 2 v = v It i s a v - f o l d summation, and there are (^) terms al t o g e t h e r . ( When 0 u = 0, II p ( r , , Z ) = 1, since empty products are conventionally always h=l " rh set = 1.) The terms of the sum for Vp p p c&n be separated i n t o two 1 2 w u c l a s s e s : (1) those i n which the wth subscript appears i n the II p(r, ; Z ) h=l rh part as the fac t o r p(w; Z ) , and (2) those i n which the wth subscript w appears i n the A Q : a ) f a c t o r , which i s then A(s 1 s -, , w; Z Z Z ) . The two classes of terms are mutually exclusive and s 1 s - , w J 1,..., v-1 take i n a l l p o s s i b l e cases. Now we sum the Vp p p term by term with 1 2 w respect to Z^ (the other £ ' s held f i x e d ) . The terms of the f i r s t c l a s s m w have a f a c t o r £ pCw; Z ) = 1 and those of the second c l a s s a f a c t o r z ^ m w F ACs w=l s w; £ £ / ) = 0. What i s l e f t i s the marginal -L,..» V-X, . . . , S v_^> w 55. model f o r p. « . We could have done the same sort of thing f o r 1 w-1 any other subscript and i t s p a r t i t i o n . One can repeat the argument and get the marginal model f o r Cw—2) p a r t i t i o n s , e t c . To avoid ambiguity with l o t s of dots instead of s u b s c r i p t s , one might write p ( l j 2 , . . tMj\t^,t^,. .Z^) f o r p„ p „, e t c . The f i r s t set of l e t t e r s or numbers with i n the 1 2 w parenthesis i n d i c a t e s the p a r t i t i o n s involved, the second set i n d i c a t e s the i n d i c e s or categories considered of the corresponding p a r t i t i o n s i n the f i r s t s e t . E.g. Suppose there are 5 p a r t i t i o n s . Then p(l,2,4; h,i,k) = p ^ ^ . Here the o l d notation i s good enough, but, when there are, say u subscripts and v dots, things can become pr e t t y confusing. Using the information of the previous paragraph we can a r r i v e at an expression f o r AQ-,2,.. ,w; l-,tn,. .t ) or (w-l)th order i n t e r a c t i o n . X Z W We prove the following by induction: A(l,2,..,w; £^,^2> • • >^w) = £ £ + 1 2 " ' w w-2 u I C-1)U I [ n p ( r h ; £ r )] pC v^; + . u=l i> h=l h rv . W ' ' (-1) Cw-1) n p(k; L). Here k=l k V>U>'> a ) = pCs.. , s 9 , . . ,s ; I ,1 ) , and u = 1 to Cw-2) i s 1 2 v e q u i v a l e n t to v = Cw-1) to 2 o r , turned around, v = 2 to (w-1), s i n c e 56. U + V = w. Thus the (w-l)th order i n t e r a c t i o n of our f i r s t model i s the same as that of Lancaster's model. Proof: Lancaster's model can be written as w-2 . • . w P/ / / = I l C-D11"1 I n p(r ; IJ] p(* ; a) + C-1) W (w-1) n p(h; nri'"\ U=M b=i 11 r h v v h=i v + a(l,2,...,w; Z.^,Z.^t. . >Z^) w-2 u aCl,2,...,w; l v l 2 , . . , l w ) = P £ £ t + I I (-D U [ n p ( r ; I )] x 1 2''* w u=l ty h=l h v P C ^ ; O + ( - i ) w - 1 (w-1) n p(h; JL ) h=l • From our f i r s t extension of our three-way model w w-2 u P/ / / = n P ( n 5 V + I I t n PCr. ; £ )] Ato ; a ) V?*" w h=l tt u=l ty h=l h r h v v v + A ( l , 2 , . . ,w; JL^jZ^r •' »"^w^ w w-2 u A(l,2 w; l v l v . . l ^ t L ^ x - n P C h ; l£ - \ \ [ II p C r h ; £ r h ) ] 1 2*'* w h=l u=l ty h=l v Atty ,o ) V V 57. It has been shown that the two models are equivalent f or w = 2, 3. To show by induction that they are equivalent f or any whole number w > 2, suppose our kQ> ;a ) equals Lancaster's a Op ; a ) for w = 2, 3,...,k. W W W W We now want to show that our . A (1,2,..., k+1, ^ i * ^ ' * ' ' * ^ k + l ^ = Lancaster's a (1,2,..., k+1, ^-j_?^2' ''^k-t-1^ " Our A(1,2,..., k+1 j Z-1!L~,... 1 ) — p» » » 1 2 k+1 k+1 k-1 u v-2 - n P ( h ; L) - I I [ n P ( r ; I )]" { Pc* v; o_) + E I (-D m * h-1 n u=l h=l n rh V V m=l 4» v n [ n pCr.; I )] p O t ; a ) + C - 1 ) V _ 1 (v-1) ft P ( h ; I ')} (*) h=l a rh n n h=l sh w-2 w-1 k-1 k Here u + v = w = k + l £ <=> £ and ][ <=> £ u-1 v=2 u=l v=2 m + n = v ' v-2 v-1 I <=> I i m=l n=2 The l a s t term on the r i g h t of equation (*) i s k-1 j I [ n p ( r . ; I )] (-1) V (v-1) n p(h; I ) u=l 4> h=l 11 h h=l sh v k k+1 k+1 k = I I C-D v Cv-D I n PCh; IA] = I n P ( h ; £.)] £ ( - i ) v Cv-i) I l v=2 h=l n h-1 n v-2 4, v But . I 1 - - C y ) - v , ( k _ v + 1 ) . 58. (v - 1 ) I 1 - Oc+l)! ( k + i ) : Cv-11! (k-v+1)!' v! (k-v+1)! ( k + 1 ) . ( v - l ) ~ ( k v 1 ) I C - l ) v . Cv-1) I 1 « v=2 ^ v - (k+i) I C - i ) ^ 1 C v k x) - ! ( - i ) v ( k ^ ) v=2 V 1 v=2 v k -1 . k - - {(k+D I C-D u 0 + I ( - l ) v ( k + 1 ) } u=l v=2 Now ( 1 - D S = I (-if Cl) = 0 for s a whole number. r=0 r k -1 Thus i C - D U A - - (d) + c - D K ( ? ) } u=l ^ u k k ,k, 0 .if k i s .odd - 2 i f k xs even k and | W ) " C ^1) - - - C*1) + C - l )t + 1 <"*>> - k + •<-!,* v=2 (k-1) i f k i s odd (k+1) i f k i s even I C-D V (v-1) £ 1 v=2 v - {(k+1) 0 + (k-1)} = - (k-1) i f k is odd [ - {(k+1) ( - 2 ) + (k+1)} = (k+1) i f k i s even = (-I) k + 1 k -1 Thus I u=l u v I I n P ( r ; I )] C-D v Cv-l) . n. PCh, I ) * h=i 11 rh h = 1 sh 59. k+1 = (C-D k+l} h n 1 PCh; £ h ) . The next to last term on the right of equation (*) is m k-1 u v-2 I I I n P C r ; I )] I I C - l ) m I n p ( r ; I )] V(ty ; a ) . (**) u = l ty h=l a r h m=l ty h=l n r ' n n n F i r s t the w = k + 1 letters are separated into two disjoint sets T = {r n,r_,...,r } and ty - {s,,s„,...,s } , then the latter i s u 1 2 u v 1 2 v separated into two disjoint sets T and ty . Let T = T u T . m n u+m u n It i s the complement of ty^. Let us look at the terms of the sum above involving a particular ty . Here n i s some whole number "such -that 2 <n < v-1. The partial sum involving a particular ty^ consists of several terms, each of the form vm-l u+m (-1) [ n p ( r ; I )] pO|i; a ) Cu+m = w-n = k-n+1). n=i h The problem i s to find how many such terms there are. ^ u line u+m = k-n+1 1 2 3 k-n-1 k-n J L y = k-n+1 k-n k-n-1 k-n-2 2 1 k k-1 k-2 n+2 n+l 60. Out of the lc - n + 1 l e t t e r s not i n ty , there are ( k n +^) = A n + x \ rn ' u m ways to assign u l e t t e r s to T and m l e t t e r s to T . J u m The sum of the c o e f f i c i e n t s of terms i n v o l v i n g k-n+1 I n | I ^ PCt^; )] pOl^S o"n)» including the factor (-1) , i s h k-n V / , .m-1 /k-n+1 N ,-k-n+l. . / .,. k-n+1 /k-n+1. - , / ...k-n+1 I (-1) C ) - ( 0 ) + C- l ) ( k _ n + 1 ) - l + C - i ) m=l 0 i f (k-n+1) i s odd, (k-n) i s even 2 i f (k-n+1) i s even, (k-n) i s odd Hence the sum (**) i s k-1 k-n+1 k-n+1 jh £ {1 + (-1) } t n P ( r , ; t )] p(i|) ; a ) . One can replace n=2 ty h=l *• r h n n n the dummy v a r i a b l e n by v. The middle term ( t h i r d term) on the r i g h t of equation (*) i s k-1 u k k-v+1 - I I [ n P ( r ; I )] P ( t y v ; a ) = - £ £ [ n P C r h ; I )] P(ty v; u=l ty h=l n rh V V v=2 ty h=l n r h V v rv Now we su b s t i t u t e the above r e s u l t s into (*). Our k+1 AC1.2,..., k+l; ^ ,1^... - vtl p - h n P ( b ; V 1 2 k+l k" k-v+1 -I I I n P ( r • I )] p(.ty ; a ) + v=2 ty h=l 11 r h V V v 61. k-1 , . k-v+1 . I I " U + (-D V X} f n P C r , ; I )] PGJ> : a ) +' U+C-D k} * v=2 1(1 h=l n r h v v v k+1 n p(h; L). h=l k+1 Our A(.l,2,..., k+1; • • • = p £ ^ ^ » + ( - l )k k II pCh,^) 1 2* * * K+1 h=l k 1,- 4.1 k'-v+l + • I I C-D k V + i [ n p(r ; I )] pGj, ; a ) v=2 ^ h=l n rh v v . v k-1 u P* , , +11 C-D U I n p ( r ; I )] pC* ; a ) ^2"'\+i u = l if h=l h r h v v v k+1 + ( - l )k k H P ( h ; L) h=l = Lancaster's a.(l>2,..., k+1; Z.^,t^,... » ^ + ^ ) with w = k + 1. Thus the induction i s complete. In t e s t i n g the hypothesis A(l,2,..,w; , ) = 0 1 Z w for a l l (JL-,Z„,..,£ ), using a s i m i l a r method as was used to test the J. Z W hypothesis 5... = 0 f o r a l l ( i , j , k ) , our f i r s t concern i s whether 9 g H (PJ 1 any problems a r i s e i n determining the rank of [-—jz—7. s — » n—ri dp(.±,z,.. ,w; JL-^JL^, ,. ,JL^) where H ranges through 0, 111...1, 111...2,..., m1m0x,..xm L Z w and 1 wf B0 2^.) = I I • PC1,2,..,W; l l t l 2 , . . . , l ) - 1 and g^ ^  ^ (£) Z.=l £ =1 1 2 *' * w 1 w 6 2 . A C l , 2 , . . . , w ; l v l 2 , . . . , i ^ . L e t A.. = A ( . i , j ) = [ J ^ * ~ j • 9 P C l,2 , . . , w ; a _ , a 9 . . , a ) = kJl A C £ k > a k > + E t - l ) U ? J- I w u=l ty V { n pCr ; I ) n A(£ a ) + . J A(£ , a ) [ n p ( r ) ] p(ty ; a )} ' h=l n r h k = l S k S k y = l r y r y h V V hfy w w - C - D W (w-1) I A ( £ , a ) [ n p ( k , t ) ] z = l k = l We m u l t i p l y t h e - m a t r i x ••[-8 g L l e f t b y [o xi x 9 e... 9 x ] w h e r e <*f ± - ) } > U i = °. • •'> <Vl). denote a s e t of f u n c t i o n s orthonormal w i t h r e s p e c t to { p ( i ; l ) , p ( i ; 2 ) , . . . . p ( i , m )] Cu) and X± = [ X ± , A ] j = 1 to m . . 1 2 w (u.,) G O G O 1 2 w -Then £ Y V x / X./ X . w „ r 1=11=1 1=1  X l 2V"' w * w 3pCl,2 - i . . ,w; a 1 , a 2 , . . , a w ) 1 2 w C ^ ) Cu 2 ) Cu w ) w _ 2 m r = X l a l X 2 * 2 • ' • • X v a w + 1 C - D U • F { S I J h X C u r , £ ) x u=l. ty h = l / = 1 r h r h ^ r h 63. v P ( - V £ )] n X(u ,a ) h rh k-1 s k s k m u u rh + • I XCu ,a ) n [ J X(u ) pCr ; I )] y=l r y X y h-1 £ » 1 r h r h h r h hty r h tn m m m 1 2 s3 sv i 1 ^ i i i i 1 - 1 - - / - 1 r i i x c v v ] p(v %)} S l S2 s3 s v w W - C-DW (w-1) I X(u a ) n [ I X(u j O p C k ; / ) ] z=l " * k=l L , K fc k where X(a,b) stands f o r X with a as superscript and b as su b s c r i p t . T h u s J I X X. 1 X„ 2 .... X, W 1 2 w £1= 1 V1 ' V1 1 1 2 2 w£w 3pCL,2,..,w;.a1,a2,..,aw) w (u, ) w-2 u v V + ^ C~1 ) U 1 { I 11 A C ur >°>J 11 XK > k-1 k u-1 tyv h-1 r h k-1 s k s k u u + I X(u , a ) T H ACu 0)] p (s s ,...,s ; u ,u ,...,u )} y=l r y r y h-1 r h 1 2 v s l s2 V h?*y kw * w - C-l) (w-1) I XCu ,a ) f n ACu 1 }0)], since k-1 k^z 2=1 Z 2 k-1 k 64. I X C v \ ) pOc,\) = A(uk,0) = { 1 L f uk " ° ^k=1 0 i f u^ ^  0 p(s ,s„«...,s : u ,u , ...,u ) i s the v-f a c t o r c o r r e l a t i o n with sub-1 2 v s ' s ' s 1 2 v s c r i p t u at the s, r-th subscript p o s i t i o n . E.g. When w = 3, k s t CD C2) p012 = p(2,3; 1,2) = E \ Y. Z, p ., . By examining the above equation j = l k=l 3 ' c a r e f u l l y i t becomes evident that the rank of the submatrix of co n s i s t i n g of the f i r s t i 1 0 o x.. e x„ e . . .§ x .— 1 2 w 3g H (P) Sp(l,2,..,w; ' * w^  J row and the rows where one or more of u-,..,u i s zero, i s one. That 1 w the submatrix c o n s i s t i n g of the remaining rows has rank (m^-1)(n^-l)...(m w~l) follows as a d i r e c t extension f o r the proof where w = 3, which we have already done. n. m„ m 1 2 w Consider Q = log(n!) - £ £ ... £ log ( f « « I) £.=1 £„=1 £ =1 1 2 w w m, m„ m m,. m 1 2 w 1 w + I I I £p o log p„ » - (e[ I I P/ / - i ] £ H = l ' £ =i . £ =1 • l " * w ^V" w £,=1 1=1^1"' w 1 2 . • - w 1 w ra, m 1 w Ifi £w=i l V % £r--V 9Q _ fV- aw , > > . .9gV..£ 8 pa * " P - { 0 + i — - E V . r i SL } al - "aw V " - a w V 1 V 1 1 w ' V v - * w 1 2 w 65 = 1 to ; 1 = 1 to w. The rank of the homogeneneous part of these mi _m2 • • , mw equations i s (m^-1) (m^-l) • • • (m -1). + 1. If we m u l t i p l i e d the above equations by hypo t h e t i c a l p and then summed with, respect to a..,..., a a.. • •.. a j. w 1 w we would obtain an equation of the following form: • n. m m, m 1 w 8Q 1 w ^ , * " * ^ - pa.....a 3p n 0 + ^ *'.* ^ - ^a-...a ba . a 1 =1 a =1 1 w a.,...a a, =1 a =1 1 w 1 1 w 1 w 1 w from which we could solve for 6 i n terms of the n's. 30 The equations — — - = 0 , a^ = 1 to ( i = ! , . . , « a,- « • * a. 1 w represent .. .m equations i n m-.. .m + 1 unknowns, the n's and 1 w 1 w G. We found that there are (m.,-1) ... (m -1) + 1 l i n e a r l y independent J- w equations among these m^ .-.m^  equations. A f t e r we eliminate 6 we have (m..-l)...(m -1) l i n e a r l y independent equations. This follows by i w . s i m i l a r reasoning to that used i n the case of three v a r i a b l e s . The term of p - — ^ = 0 not i n v o l v i n g the ra,...a 3p 1 w ca....a 1 w n „ a f t e r 0 has been eliminated i s (f - np ) . In a, ••••a a, ..a ra,...a 1 w 1 w 1. . w order f o r the augmented matrix to have rank (m,-1)...(m -1) i t i s 1 w 66. m l m2 v v ^ l1 C ul * necessary that 2 2 •••• L X. ...X Cf - np ) „ H T T wa a.,•«.a a~•..a a =1 a „ - l a =1 1 w 1 w 1 w 1 2 w = 0 whenever one or more of the i n d i c e s u1,..,u are zero. In t h i s 1 w ^ C i O V Cu 2 ) , CuwX case l e t > X2 >•«» .xw. represent orthonormal functions *! ^ • • • 2 * * * with respect to {— }, { },...., { f * * * w n n n } r e s p e c t i v e l y . w-2 / - . S u+1 u W r i t e v....a •-• l.-^^r- ? 1 " f ( v V ] f C V V + 1 w u=l n ill h=l h w u (w-1) [ n f ( r ; I ) ] , where f C r , , I ) = f . . . . I .. .. _ w - i i n r, n r, r , n h=l h h V , > ( r ^ - l ) d o t s (w-r^)dots a n d . f C * v ; o v ) = f C s 1 , s 2 , . . . , s v ; j 1 , j 2 , . . . , j v ) ral m 2 m w I I I X a,=1 a =1 a =1 1 2 w w-2 l a , . *wa V . . . a - I w 1 w u=l u+1 .11 A(u ,0) x rt h=l h w p(s ,s ,...,s ; u ,u - u ) + C-l) WCw-l) n AXu, ,0) . x z v s1 s 2,..., s v h=l Thus m 1 £j • • • - l . l a i X VT" V wa . ^ a , . 1 .a /" 1 f o r u =0,1=1 to w CD 0 f o r any (w-1) of the u.=0 C2) pr'(s s ;u ,u u )' cl » '-• •» cp cl t2, , , M p for u u ^0, p>2 1»**'»- P C3) p(1,2,...,i-l,i+l,...,w;u 1,u 2,...,u i -ui=0, a l l others nonzero O . a l l u =0 ui + l ' " - »Uw) (4) 15) 6 7 . Cl) follows from the equality £ 0-1)^"^ = 1 . I I C-Uk+1 (") = - I C-Uk ( 3 x k I + i = - CC1-X)W~D - -C k=l V k / k=0 K ' x=i 1 = -C i) = l ] (21, C3), (A), C5) follow by cl o s e examination of above equation. S i m i l a r l y for m w (u^) '(u ) •I I X, W P_ a where the p a,=l a =1 i a l w a w a l * * , a w ' al'-- aw x. w represent t h e i r hypothetical values, except that p i s without m. 1 I • a r l m , s w (u ) I Aa a -1 1 w ; ( U W ) ar-- aw -X = p wa n u,.... ..u w 1 w r 1 f o r u.=0,1=1 to w 0 f o r any (w-1) of the u^=0 P Cs. ,s . . . , S t ' U t . , U t ,...,u ) for u ,u u ^ 0 p>: P X • % Cl t 2 p p>2 P Cl,2,.. , i - l , i + l , . . ,w; u^,u2>. .u^_^u^ +^ ,... ,u^) j U ^ O j a l l others nonzero pCl,2,..,w; u 1,u 2,..,u w) a l l u^O. m, m • 1 "w .Cu,! - ^ u w i Thus £ ]T X l a ... X t 7 a ' (f a =1 a =1 1 w - n p ) = wa a,...a ra~...a w 1 w 1 w 0 when at least one of the u.=0 n pCl,2,..,w; U ,U ,...,U ) when a l l indices are not zero's . j. z w 68, 1 w (Uj)_ Cu ) a^..a On the other hand - J- .... h X, ... X = p i i xa- wa n u,...u , a,-! a =1 1 w 1 w 1 w which u s u a l l y ^ • p /- i \ JL 1 . i u,,...u when p(l , a - ) f , etc. Also 1 w ' 1 n pQ-Ju..), p C 2 , u „ , p ( w , u ) cannot be expected to be zero x z w . f ( l , a ^ ) 1 2 w Ca ) when p C l;a 7) ^ — : etc. Thus £ £ x i • • • .X W x x, n -i i • •» xa- wa a = l a =1 a =1 1 w 1 2 w Cf ) = n(p -p ). For u .=0, i = l to w one has a, ...a -np u, .. .u u-.. .u i ' 1 w a, .. .a 1 - w 1 w 1 w n ( p n n _ - p.... _) = 0 . In the other cases, one has n(p -p ). 0 0 . . . 0 0 0 . . . 0 u,...u ru n...u 1 w 1 w 80 Consider the equations p - r - * = 0 a f t e r 6 has a,...a 3p ' 1 w a....a 1 w been replaced by i t s expression i n : the n's. Suppose now that these equations (a L) (u w) are m u l t i p l i e d by X i .... X and then summed with respect to xa., wa 1 w a^,...,a w. We conclude that, i n general, the r e s u l t i n g m^ ...m^  equations w i l l have on the r i g h t s i d e n(p ; . _ p . ) } Q f w h i c h C m , . . . m -1) U l " " U w V U w 1 w may be expected to be d i f f e r e n t from zero depending on the values the u's take on. However, the rank of the matrix of c o e f f i c i e n t s of the n a- • • • a 1 w 69. on the l e f t s ide i s (m.,-1]. ..(m -1) so that the equations w i l l be w i n c o n s i s t e n t except f o r s p e c i a l s e t s { p C l , a 1 ) } , {p(2,a 0)},...,{pCw,a )} x 2 w which, l e a v e an a p p r o p r i a t e set of not more than (m.. - 1 ) (m 0-l) . . . (m - 1 ) x / w terms on the r i g h t of the equations non-zero. We want to v e r i f y that . .f C l ,a,] f C2,a„) f (w;a ) { P C l , a )} = { - A . } ,- ( P C 2 , a „ ) } = { -^-} , { P(w,a )} = { — x n c. n w n w i l l accomplish t h i s . We have shown that the vector cj[ l i e s i n the column space of C u i } C V the vectors X.. 0...8X . , where u. ^  0: i = 1 to w, but that, i n —1 —w ' i ' ' ' general the same can not be said of the other p o s s i b l e p_. I f we can show ^1^2 * * ^  that the column space of I~ + h^ ^  ^ ] equals the column Pa-a_..a 1 2* *' w 1 2 w C O Cu ) (u ) space of {X^ x ^  8 ... Q | u ± 4 .0; i = 1 to w} then the set of equations w i l l be consistent when p_ = c^ . Note that here (a^,..,a w) indexes rows, (£^, . . ,1^) and Cu^,..,uw) columns. The column space of the c o e f f i c i e n t matrix above i s a subspace of m,m-....m - dimensional Euclidean space. The matrix was shown to have 1 2 w r rank (m-,-11 Cm0--1). • • • Cm - * i ) , hence i t s column space i s an (m, -1) (m0-l) • • • Cm -1) X Z W X 2. W subspace of E m,m0.. .m 1 2 w Cu,) Cu ) C O Now the set of vectors '{X. 0 X„ 0 ... 0 X —1 ~2 —w u,=0 to C m-1) : > u =0 to (m -1) w w 70. forms a ba s i s f o r E . We w i l l show that the column space of the 1 2 w c o e f f i c i e n t matrix i s orthogonal to the subspace spanned by the set of 0^1 ( u 2 ) C O vectors {X, 8 X„ 8 ... 8 X one or more of the u . i s zero}. —1 —2 ~w 1 r This i s a m1m„...m - Gn.,-1) (m -1) ... (m -1) dimensional subspace. 1 2 w 1 2 w Hence i t s complement, the (m^-1) C ^ - l ) . .. On -1) dimensional subspace C O C O (u ) spanned by the set of vectors {X. 8 X 8...8 X w lu.^0; i = l to w} l L w i i s the same as the column space of the c o e f f i c i e n t matrix. We have shown that Cf-ng.) l i e s i n t h i s subspace. Hence the set of equations are consistent when (f-no) i s the non-homogeneous part of the equations. That the column space of .the c o e f f i c i e n t matrix i s orthogonal C O (u ) (u ) to the subspace spanned by the set of vectors {X^ ® —'2 ^ '*" ^ ~w one or more of u,,u , . . . , u i s zero } follows by d i r e c t extension 1 1' 2 w from the proof f o r w=3 and so i s l e f t to the reader. D. THE GENERAL CASE FOR THE SECOND EXTENSION OF OUR 3-WAY MODEL. The general case for the second extension of our 3-way model can be w r i t t e n 1 I w. 1 I w i = l v=2 ill v where i t i s understood that u = w-v. CWhen u = 0.p(T ; a ) = 1 ) . ->r u u As before the terms of the sum V o o p c a n be separated 1 2 w into two classes: Cl) those i n which the wth subscript appears in the 71. p(T u;a u) p a r t , and (21 those i n which the wth subscript appears i n the BCP ;c 2 f a c t o r . The two class e s of terms are mutually exclusive and take i n a l l p o s s i b l e cases. Now we sum the J?p p p term by term 1 2 w with respect to £ (the other £ ' s held f i x e d ) . The terms of the f i r s t m w c l a s s have a f a c t o r F p(T ) and those of the second c l a s s have £ =1 U U w m w a f a c t o r J B(s..,...,s . ,w; £ £ £ ) = 0. What i s l e f t i s r - i 1 w _ 1 s l V l ' v w the marginal model f o r p» » .We could have done the same sort 1 w-1 of thing f o r any other subscript and i t s p a r t i t i o n . One can repeat the argument and get the marginal model f o r (w-2) p a r t i t i o n s , e t c . We would l i k e now to e s t a b l i s h that the sum of the c o e f f i c i e n t s of the terms of B(l,2,..,w; l ^ t ^ . . . ,1^) i s zero. For the 3-way case we got 5iak. = P i j k - P t , . P . j t - P . 4 . p i . k - P . . k . P i j . + 2 P i . . P . J . P.. k • Note that the sum of the c o e f f i c i e n t s i s zero. For the four v a r i a b l e case, r e c a l l we obtained 0 hijk = A i j k " Ph... P.ijk ~ P . i . . P h . j k ~ P . . j . Phi.k P...k Ph±j. " 2 p h l . . P . . j k ~ 2 p h . j . P . i . k ~ 2 p . i j . Ph..k Tix • * 3 • ••• lc ^lx* j • »x • • ••• lc lx» • lc • x.« • • • i j • n.» • • • lc • x • lc • * ^» • j • ^««jlc • • 72. Again, note the sum of the c o e f f i c i e n t s i s zero. Let us examine the i 4-way case somewhat more c a r e f u l l y by w r i t i n g the terms of 0 kijk. a s follows: ,(34) ,(24) 6 h i j k = ph i j k " ph . . . P. i . . P. . j . P...k " ph i . . a j k ~ph . j .a i k , (23) , C14) , (13) , (12) , (234) Ph..k a i j ~ P. i j . a hk p. i . k a hj ~ P. . j k a h i ph . . . 5 i j k ,C134) ,(124) ,(123) P. i . . ^ hjk ^ . . j * ^ hik P...k ^ h i j Note that the c o e f f i c i e n t s of the f i r s t two terms cancel out. Since each a c o n s i s t s of two terms, one with p o s i t i v e 1 c o e f f i c i e n t and one with negative 1 c o e f f i c i e n t , the c o e f f i c i e n t s of a l l the terms i n v o l v i n g an a cancel out. Since the sum of the c o e f f i c i e n t s of the terms that make up any 6 i s zero the c o e f f i c i e n t s of the terms i n v o l v i n g the S's t cancel out. Hence the sum of the c o e f f i c i e n t s of the terms of 0 hxjk i s zero. We prove by induction that the r e s u l t holds true f o r B ( l , 2 , . . ,w; Z,, ) . Suppose now the r e s u l t holds true f or i 2. w BCs 1 , s 2 , . . , s k ; £g- ts^ l s ) B C s ^ ; l s ^ t l B ^ • Consider B(l,2,...,k, k+1; Z^Z^ \ > \ + i ^ = p £ I P 1 2 k+1 k - II pCi.-^l - • J" - F P C T , ^ ) B 0 P ; C ) . The c o e f f i c i e n t s of the , / ' i Ln L. r u' u rv v x=l v=2 \b V f i r s t two terms cancel out. By our i n d u c t i v e hypothesis the c o e f f i c i e n t s of the remainding terms cancel out. Hence the sum of the c o e f f i c i e n t s 73. of the terms of B(l,2,...,w; Z^Z^,...,Z} i s zero. In t e s t i n g the hypothesis BQ.,2,...,w; Z^,^* • • • »^w^ = P fo r using a s i m i l a r method as was used to te s t the 1 2. w hypothesis <5.., = 0 for a l l ( i , j , k ) , again our f i r s t concern i s whether r sg H CP_) any problems a r i s e i n determining the rank of 1 -—-p.—7. n—s — 5 — r I 9pCl>2,... ,w; £ , £ 2 , . . . , £ w ; where H ranges through 8, 111...1, 111...2, , m^ra.^'•'mw and m, m , 1 w So (E) = I ••• I P(l»2,...,w; £ £ . . , , £ ) - 1 and U Z =1 £ =1 1 / w 1 w I 6 £ £ £ P^-^  = B(l,2,...,w; £^,£2,... ,£ w) . 1 2* '' w t I t i s not immediately obvious what 8 £ £ £ l s' 1 2' " w *JLJU...l m*JLl0..:l ~ . n r p ( i ^ i ) " X, ? p ( W B < V ° v > 1 2 w 1 2 w x=l v=2 ty v Consider the following table: 74. 2-way' table. 3-way t a b l e 4-way t a b l e i\ coef. x C25 l l 1 ah i - i - i 2 0 (31 a,2i a3) § coef. jx 1 1 1 3 1 5 -1 -3 2 2 0 C4) (1,3) (2,2) ( l 2 , 2 ) • ( I 4 ) # c o e f . 1 1 4 3 6 1 15 -1 -2 3 -9 1 -4 -6 18 -9 0 5-way t a b l e 6-way t a b l e # coef. X # c o e f . X C5)' 1 1 1 C6) 1 1 1 Cl,4) 5 -1 -5 (1,5) 6 -1 -6 (2,3) 10 -2 -20 (2,4) 15 -2 -30 C l 2,3) 10 3 30 (3^) 10 -2 -20 d , 2 2 ) 15 4 60 d 2 , 4 ) 15 3 45 (1 3,2) 10 -11 -110 (1,2,3) 60 4 240 ( I 5 ) 1 44^ 44 ( 2 3 ) 15 ; 6 90 P ' ~ C1J,3) ' 20 - l l ; - •"' -220 -2 2 (1 ,2^) 45 -14 -630 • Cl4,2) 15 53 795 C l6) . 1 -265 : -265 203 0 TABLE I COEFFICIENTS IN THE EXPRESSION FOR THE HIGHEST ORDER INTERACTION 75. 2 The f i r s t column i n the table represents the p's, for example, (1,2 ) stands f o r any product of three p's, one of which has one s u b s c r i p t , the other two each two s u b s c r i p t s , the subscripts h , i , j , k , £ a l l being represented. The second column gives the number of d i f f e r e n t terms of ()(3) the form GL,2 2). There are l ^ i l L i = 1 5 terms of the form 0-,2 2). The t h i r d column gives the c o e f f i c i e n t of the term i n the f i r s t column and the en t r i e s i n the l a s t column are the products of the e n t r i e s i n the second and t h i r d column. Suppose one i s dealing with the w-way case. The subscripts are 1^,1^,...,!^. Consider the following grouping of these subscripts dl d2 dm in t o say n groups (c^ , »•• . »cm )• Here d^+d^"^'•'+am = n and the c , d. are a l l p o s i t i v e i n t e g e r s . Also c d +c_d„+...+c d = w, j ' j ^ 6 1 1 2 2 m m ' and m < n. The number of p a r t i t i o n s of t h i s type are w! . d,l...d ! Cc.t) ...Cc |) m 1 m 1« m! For example^consider w=6 and the 2 2 type of p a r t i t i o n Q- ,2 ) . Here h=2,k=4. The number of d i s t i n c t 2 2 6! " 720 p a r t i t i o n s ^ q f t h i s type (1 ,2 ) i s 212' Cll} C2!) 2 = 16 ~ ' el e2 6w One can also write the p a r t i t i o n as Cl »2 ,... ,w ) , where e^ . may be 0. I f e^ i s 0, then there are no groupings of j l e t t e r s . Then e,+e0+...+e = n, the number of groups. le.+2e_+..,+we = w 1 2 w ' o r 1 2 w The number of d i s t i n c t p a r t i t i o n s of t h i s type i s 76. w' This i s a type of symmetric fu n c t i o n e e e e'e„!...e ! (1!) 1(2!) 2...(w!) W JL Z W and can be found i n tables of symmetric functions. For example, n2 02. . 2 .2 0 ,0 -0 .0, 6! :  Cl ,2 ) = Cl ,2 ,3 ,4 ,5 ,6 ). 5 = r r pj zr- 45. 2!2!0!0!0!0!(1!) (2!) (3!) (4!) (5!) ( 6 ! ) u While i t i s not immediately obvious what the remaining e n t r i e s f o r the w-way table would be, we can however s t i l l make a few observations. I f we denote by c^, k = 2,3,..., the c o e f f i c i e n t of the l a s t term f o r each t a b l e , that i s , o.^= -1, = 2, = -9 etc . we note the following r e l a t i o n s h i p : c k + i = -Ck+1) c^ -1 (and c^=0). We have proved before that the sum of the c o e f f i c i e n t s of the terms of B(l,2 w; t^t^,... ,£) i s zero. Hence the sum of the e n t r i e s under the heading X must always be zero. Note also that the sign of a c o e f f i c i e n t = (-l)n°* °^ S r o u P s !• Note that a c o e f f i c i e n t , say, i n the 6-way table can be derived from the c o e f f i c i e n t s of the previous tables of lower dimension. Consider the term (1,2,3)'. We obtain from p(l;g) B(2,3,4,5,6; h,i,j,k,£) and the table of c o e f f i c i e n t s f o r the five-way table a c o e f f i c i e n t of 2, the negative of the c o e f f i c i e n t entry i n the (2,3) row. From p(2,3;h,i)-B(l,4,5,6; g,j,k,-£) we get a c o e f f i c i e n t of 1, the negative of the c o e f f i c i e n t entry i n the (1,3) row of the table for the four-way case. From p(4,5,6; j,k,£) B(l,2,3; g,h,i) we get a c o e f f i c i e n t of 1, the negative of the c o e f f i c i e n t entry i n the (1,2) row of the table f o r the three-way case. Since (1,2,3) has three groups the c o e f f i c i e n t of (1,2,3) i s + (2+1+1) or 4. F i n a l l y note that we can obtain the sum of the c o e f f i c i e n t s o f , 9 gi i k £ say, - —J by r e f e r r i n g to the four-way t a b l e , m u l t i p l y i n g each number 9 pi j k £ i n the X column by the number of groups i n the entry i n the f i r s t column. 9 gi i k £ To i l l u s t r a t e , the sum of the c o e f f i c i e n t s of the terms i n - —J— i s 9 pi j k t 1 x l + 2 x C-4) + 2 x (-6) + 3 x 18 + 4x(-9) = -1. We complete t h i s section by r e f e r r i n g back to a four-way t a b l e . The work f o r the w-way table w i l l simply involve more terms. We want to f i n d the rank of [- grr(p)J where H takes on the values 9 ph i j k H~ " 0, 1111, 1112,...,qrst. t Again we consider the product [L — ] £— SjjCp_)j # 0 W 0 X 0 Y @ Z 9 P h i j k i A f t e r c a l c u l a t i n g a l l the d e r i v a t i v e s J , m u l t i p l y i n g by 3 pdabc Cm) Cu) Cv) Cw) W^  X^ Y^ and summing we a r r i v e at q r S ' (m) Cu) (v) (w) 9g* (m) (u) (v) (w) • I I I I K X Y."-- Z. , " 1 J K = W. X Y, Z u i * i • i i i n 1 J k 9p, , d a b c h=l i = l j = l k=l J "dabc (w) (v) Cu) Cm) (m) Cu) (v) - Z p n - Y, p - Xo p - W, p - W. X Y, A c muvo b muow a movw d ouvw d a b wo Cm) (u) (w) (m) (v) (w) (u) (v) (w) - W, X Z A - W, Y, Z A - X Y, Z A d a c vo d b c uo a b c mo 78. Cw) Cv) Cw) Cw) + 3Z A p + 3Y- A p + 3Z A p + 3Z A p c mo ouvo b mo ouow c uo movo c vo muoo (v) (v) Cu) (u) + 3Y, A p + 3Y, A p + 3 X A p + 3 X A p b uo moow b wo. muoo a mo oovw a vo moow Cu) (m) (m) Cm) + 3X A p + 3W, A .- p + 3W, A p + 3W ' A p a wo movo d uo oovw d vo ouow d wo ouvo Cv) Cw) Cu) (w) Cm) (w) + 3Y. Z A A + 3X Z A A + 3W, Z A A b c mo uo a c mo vo d c uo vo Cu) Cv) (m) Cv) (ro) (u) (m) + 3X Y. A A + 3W, Y. A A + 3W, X A A - 9W, A A„ A a b mo wo d b uo wo d a vo wo d uo vo wo (u) (v) (w) - 9 X A A A - 9 Y , A A A - 9 Z A A A a mo vo wo b mo uo wo c mo uo vo Cv) Cw) Cu) Cw) (m) Cw) - 2Y, Z p - 2X Z p - 2W, Z p b c muoo a c movo d c ouvo (m) Cv) Cm) Cu) Cu) (v) - 2W, Y, p - 2W, X p - 2X Y, p d b ouow d a oovw a b moow . q r S t (m) (u) (v) (w) 3g I T l I \ X. Y z k -h = l i = l j = l k=l h 1 3 k d pdabc -1 for m=0, u=0, v=0, w=0 (1 equation) 0 for m=0, u=0, v=0, w?*0 ((t-1) equations.) 0 for m=0, u=0, v^O, w=0 ((s-1) equations) 0 for m=0, u^O, v=0, w=0 ((r-1) equations) j 0 for m/0, u=0, v=0, w=0 ((q-1) equations) 79. 2p f o r m=0, u=0, v^O. w^ O ((s-1)(t-1) equations) oovw 2p for m=0, u^O, v=0. w^ O ((r-1) (t-1) equations) OUOW v v / v / i 2Pmoow f ° r m ^ ° ' U = ° ' V = 0 ' C ( q - l ) ( t - l ) equations) 2p f o r m=0, u^O, v^O, w=0 ((r-1)(s-1) equations) ouvo ^ 2p f o r m^ O, u=0, v?*0, w=0 ((q-1) (s-1) equations) movo 2P f o r u^O, v=0, w=0 ((q-1)(r-1) equations) muoo ^ ^ ~Pmuvo f o r m ^ 0 ' U ^ ° ' V ^ ° ' 1 7 = 0 C(q-1) (r-1) (s-1) equations) ~Pmuow f ° r m ^ ° ' U ^ ° ' V =' 0' W ? f 0 ( ( I - 1 ) (r-1) (t-1) equations) ~Pmovw f o r m ^ 0 ' U = 0 ' w 4 ° ( ( q - 1 ) ( s - 1 ) ( t-1) equations) - Pouvw f o r m=0, u^O, v?*0, w^ O ((r-1) (s-1) Ct-1) equations) (m) (u) (v) (w) (w) (v) (u) W, X Y, Z - Z p - Y, P - X P a a b c c muvo b muow a movw (m) (v) (w) -W, p -2Y, Z p d ouvw b c muoo a c movo (u) (w) -2X Z P (m) (w) (m) (v) (m) (u) -2W, Z p -2W, Y, P -2W, X P d c ouvo d b ouow d a oovw (u) (v) -2X a Y f e P m o Q w for m/=0, u£0, v£0, w/=0,^(q-1) (r-1) (s-1) (t-1) equations Consider ri o 0 w 8 X 6 Y 8 Z f 3 g H(£) 9p. h i j k (qrst + 1) x qrst The submatrix c o n s i s t i n g of the f i r s t row of the product matrix and the 80. qrst - (q-1) (r-1) (s-1) (t-1) rows where one or more of the m,u,v,w i s zero has rank one. The submatrix c o n s i s t i n g of the rows where m^O, u^O, v^O, w^ O has rank (q-1)(r-1)(s-1)(t-1) because of the l i n e a r independence of the orthonormal f u n c t i o n s . The proof of t h i s , though not a d i r e c t extension of the proof f o r the 3-way case, i s very s i m i l a r . q r s t Note that terms of the form I £ I I P J P P K P d l H 1 -1 1 Cl • • • • 3. • • • • D a • • • C ^ =1 a=l b=l c=l (v) (w) (m') (u') (v') (w') ' q On') Y, Z p W, . X Y, Z = p I p , W, X b c muoo d a b c inuoo,11, rd . . . d d=l r (u') S (v) (v') t (w) (w') I P a X I p Y Y I p Z Z = 0 since m * 0. 1 » d • • Cl 1-1 • « U * U V ~m • • • I— b=l c=l Hence we conclude that " 8§ H < £ > 8 pdabc J has rank (q-1)(r-1)(s-1)(t-1) + 1. q r s t Consider q 4 = log(nt) - I I I I l o g ( f j) h=l i = l j = l k=l J q r s t q r s t + I I I I f h l j k ^ g p - {Qll J J J p - 1] + h=l i = l j = l k=l J J h=l i = l j = l k=l J q r s t h=l 1=1 j = l k=l h x 2 k h l J 1 C We found that the rank of the c o e f f i c i e n t matrix of the homogeneous part of these qrst equations i s (q-1)(r-1)(s-1)(t-1) + 1 . If we obtain the d e r i v a t i v e , mu l t i p l y i t by hypoth e t i c a l p, d p i i Qa.DC dabc 81. and then sum with respect to d, a, b, c we obtain the following equation: q r s t ^ q r s t £ I I Pdabc ^ T T - = n " 6 + I I J. I V i k I p h . . . P . i j k d=l a=l b=l c=l rdabc h=l i=l j=l k=l + P. i . . Ph . j k + P. .j . Ph i . k + p...k Ph i j . 6 ph . . . P. i j . P. . .k 6 p. i . . Ph . j . P...k ~ 6p . . j . Ph i . . P...k 6 ph . . . P. i . k P. . j . ^ • 1 • • ^ti ••k ^ • • j • ^ l i • • » ^ # • j lc • x • • • »• • x • • • • j • , «• lc + 2 p. i . k ph . j . + 2 p. . j k v.. + 2 p h..k p . i j . ] I f we equate t h i s equation to zero we obtain an expression f o r 6 i n terms of the n's. There are (q-1)(r-1)(s-1)(t-1) l i n e a r l y independent equations among 9q 4 9 pdabc = 0 d=l to q, a = 1 to r , b=l to s, c=l to t a f t e r 8 has been elim i n a t e d . Let us now look at the o r i g i n a l equations i n the form P£ja-[3C x 9 q4 -T-^  = 0 . If 0 has been replaced by the expression i n the n, , s, 3p. , r J L 'dabc rdabc then there are qrst non-homogeneous l i n e a r equations i n qrst unknowns. The rank of the c o e f f i c i e n t matrix now i s ( q - 1 ) ( r - 1 ) ( s - 1 ) ( t - 1 ) . In order f o r the system of equations to have a f i n i t e s o l u t i o n , the rank of the augmented matrix must be the same as the rank of the c o e f f i c i e n t matrix. Now the term of p, , = 0 not i n v o l v i n g the n, , , rdabc 3p , , dabc' rdabc 82. a f t e r 0 has been eliminated, i s ( f , , - n p , , ) • In order f or the dabc dabc augmented matrix to have rank (q-1)(r-1)(s-1)(t-1) i t i s necessary q r s t On) Cu) (v) Cw) that I I I I W. X Y, Z ( f , . - np, , ) = 0 L L L L £ a b c v. dabc rdabc d=l a=l b=l c=l .On) whenever one or more of the indices m, u, v, w are zero. Let W .(u) „(v) .(w) X , Y , Z be as before. Let q r f , . f , + f f , , + f , f , + f f , , L d. .. .abc .a. . d.be . .b. da.c . . .c dab. dabc n 2 f , f . + 2 f , . f + 2f , f , da.. ..be d.b. .a.c .ab. d..c n 3 f , f , f + 3 f , , f f + 3 f , f f + da.. . ,b. ...c d.b. .a c d..c .a.. ..b. 3f . , f , f + 3f f , f , + 3 f , f , f • a D • Q • • • • • • c • a. • c d • • • • • D » »• DC u • #» • a« 9 f . f f . f . .. , d... .a.. ..b. . . .c-, + 4 3 n By s i m i l a r method as before we a r r i v e at q r s t „(m) M .Cv) .Cw) y y y y w, x Y, Z (f,. -nq,, ) = L L L L d a b c dabc ndabc d=l a=l b=l c=l 0 when at l e a s t one of m,u,v,w = 0 n p when a l l muvw indi c e s are not zero. 83. From here the remainder follows s i m i l a r l y to the three-way case and so i s l e f t to the reader. c 84. CHAPTER VI NUMERICAL ILLUSTRATION USING OUR MODEL AND METHODS AND OTHER MODELS AND METHODS. A. THE LIKELIHOOD RATIO WITH OUR MODEL We w i l l now consider a numerical example to i l l u s t r a t e our three-way model. Consider the data i n t a b l e l l below. This data was taken from Kullback's text "Information Theory and S t a t i s t i c s " (p. 180). It represents the number of items passing, P, or f a i l i n g , F, two t e s t s T^, T 2 on c e r t a i n manufactured products from manufacturers A, B, C, D. Tl P F T o t a l T 2 P F T o t a l A 112 32 144 A 84 24 108 B 76 20 96 B 86 10 96 C 87 9 96 C 58 14 72 D 41 7 48 D 40 8 48 T o t a l 316 68 384 T o t a l 268 56 324 TABLE IT: DATA REPRESENTING NUMBER OF ITEMS PASSING OR FAILING TOO TESTS ON CERTAIN MANUFACTURED PRODUCTS. This i s a 4x2x2 t a b l e . The denominator of the l i k e l i h o o d r a t i o i s f . . , f 4 2 2 n n JI i = l j = l k=l n! II II I ( ^ k ) " ^ k / j j' The numerator of the l i k e l i h o o d r a t i o i s f i j k; 4 2 2 n! n n n i = l j = l k=l f . f + f . f . . + f , f . . i . . .jk . j . l.k . .k i j . (n) 2 f- f 1 f k (n) i j k :i j k ! 85. The l i k e l i h o o d r a t i o becomes L = 4 2 2 / n ( f . f .,+f . f. ,+f , f . . ) n n , L i ^ _ v 3 k »,l. x.k ..k 1 3 . x=l j = l k - i 2f. f . f . "\ f i j k 1 • • • 3 • • • k j 4 2 2 ^ 1 = J l Jl { n ( f i . . £ O k + £ . j . f i . . c + f . . k f i j . > " 2 f i . . £ . J . £ . . k ) 2 x n log n - £ I I i = l j = l k=l f . log f . xjk b i j k 112 log 112 528.4718735951 84 log 84 = 372.1886111028 32 log - 32 110.9035488896 24 log 24 = 76.2732919284 76 log 76 = 329.1357338618 86 log 86 • 383.0738674778 20 log 20 = 59.9146454711 10 log 10 = 23.025809299 87 log 87 = 388.5340063229 58 log 58 = 235.5056946117 9 log 9 = 19.7750211960 14 log 14 = 36.9468026146 41 log 41 = 152.2564547349 40 log 40 = 147.5551781646 7 log 7 13.6213710434 8 log 8 = 16.6355323334 1602.6126551048 1291.2048291632 TABLE III VALUES OF f. ., l og f. xjk & xjk 2 x 708 log 708 = 9292.4208366704 The l a s t • two terms of Log L <= -e 12186.2383 Now f l . • 252 f , =584 f • .1 - 3 8 4 f . l l = 316 f 2 . 192 f «, =124 f 2 = 3 2 4 f.21 68 V . = 168 f.12 268 86. f4.. 96 f .22 56 fi r = 196 £12. " 56 f l . l = 1 4 4 f1.2 " 108 f21. 162 f22. 30 f2.1= 9 6 f2.2 = 96 f31. = 145 f32. " 23 f3.1= 9 6 f3.2 = 72 f41. 81 f42. 15 f4.1 = 4 8 f4.2 " 48 R i j k = [n(f, . f . J k + f . ^ . . k ^ . ) " 2 f l f • f Let R Consider the following table ( i j k ) R i . i k l o * R i i k f i 1 k l o ^ R i i k 111 56181312 17.8441 1998.5392 211 40578048 17.5188 1331.4288 311 •41351040 17.5376 - 1525.7712 411 20289024 16.8256 689.8496 121 16000704 16.5881 530.8192 221 7543296 15.8361, 316.7220 321 6770304 15.7280 . 141.5520 421 3771648 15.1431 106.0017 112 42066432 17.5548 1474.6032 212 40626720 17.5200 1506.7200 312 31332240 17.2601 1001.0858 412 20313360 16.8266 :,7 673.0640 122 12070080 16.3060 391.3440 222 7494624 15.8298 158.2980 322 0 4758768 15.3755 215.2570 422 3747312 15.1365 121.0920 12182.1477 TABLE IV . VALUES OF f i j k log R i j k where R..k = [ n ^ ^ f < J k + f # j < £ ^ k « ^ ^) - 2 f i . . f o . f . k 3 87. 1=1 j = i k = i fi j k l 0 g Ri j k 4 2 2 = E E . E f , . , l o g [ n ( f , f ,,+f , f , ,+f , f , , ) i = l j = l k=l " i j k i . . .jk . j . i.k . .k i j .' 2f f . f ] X • • • J • * • = 12182.1477 Hence Log^L = - 4.0906 and U = - 2 Log gL = 8.1812. The dimension of Q, the parameter space, i s r s t - 1 . The dimension of OJ, the subset of the parameter space such that 2nd order i n t e r a c t i o n i s zero , i s (r-1) + Cs-1) + (t-1) + (r-1) (s-1)' + (r-1) (t-1) + (s-1) (t-1) . In t h i s case dimension of fi i s 15 and dimension of w i s 12. U i s asymptotically 2 d i s t r i b u t e d as a x with 3 (= 15-12) degrees of freedom. Since 23 23 X. OI- = 7.82 and x o n =6.25 and since 8.1812 > 7.82 we r e j e c t . y J . y u HQ! "S^jk = 0 f o r a l l i , j , k at the f i v e percent l e v e l . B. THE CHI-SQUARED STATISTIC WITH OUR MODEL 2 (0—E) We now c a l c u l a t e the chi-square s t a t i s t i c x = ^ — 5 where the summation extends over a l l c e l l s . Consider table V 88. E - K±ik, 2 *ijk " 'C708r I l l 112 112.079287 211 76 80.951450 311 87 82.493536 411 41 40.475725 121 32 31.920712 221 20 15.048549 321 9 13.506463 421 7 7.524274 112 84 83.920712 212 86 81.048549 312 58 62.506463 412 40 40.524274 122 24 24.079287 222 10 14.951450 322 14 9.493536 422 8 7.475725 Sum 708 707.999992 <vv>2 , CO. ., -E..,) , 1 i!k l j k | CO. .,-E. ) 2 i j k i j k ' i j k 111 .079287 .006286 .000056 211 4.951450 24.516857 .302858 311 4.506464 20.308217 .246179 411 .524275 .274864 .003331 121 .079288 .006286 .000196 221 4.951451 24.516867 1.629184 321 4.506463 20.308208 1.503591 421 .524274 .274863 .036530 112 .079288 .006286 .000074 212 4.951451 24.516867 .302496 312 4.506463 20.308208 .324897 412 .524274 .274863 .006782 122 .079287 .006286 .000261 222 4.951450 24.516857 1.639764 322 4.506464 20.308217 2.139162 422 .524275 .274864 .036767 Sum 8.172128 2 xik/ o TABLE V CALCULATION OF x USING EXPECTED VALUES = '(708) 89. o x = 8.172128. As before we r e j e c t H Q : 6 fc = 0 f o r a l l i , j , k at the f i v e percent l e v e l . C. BISHOP'S MODEL AND METHODS We w i l l now examine the contingency table using Yvonne M.M. Bishop's model and methods and compare the r e s u l t s we obtain to those from our model. She defines nP i j k ~ ^ ^ i j k ^ a n c* e xPr e s s e s t h i s expected value i n the logarithmic scale as log n p . j k = u + u U i ) + u 2 ( j ) + u 3 ( k ) + u 1 2 ( ± j ) + u 2 3 ( , k ) + u ^ c . k ) + u 1 2 3 ( . j k ) CD where u i s the o v e r a l l mean value and the subscripted u-terms are the main and m u l t i p l e - f a c t o r e f f e c t s . The numerical subscripts denote the va r i a b l e s involved and the alphabetic subscripts the categories for these v a r i a b l e s i n the same order. Thus u,„ ,. . N i s the two-factor e f f e c t 12 U J ) between v a r i a b l e s 1 and 2 at l e v e l s i and j , r e s p e c t i v e l y . The subscripted terms are d e v i a t i o n s , as i n the l i n e a r models f a m i l i a r i n ana l y s i s of variance of qu a n t i t a t i v e data. Thus, r r s £ u 1 / j X = E u, = £ u, 0 ,.,».==' 0 and, i n general, each u-term sums 1=1 i = i 1 2 ( i j ) j = 1 1 2d j ) to zero over any of i t s v a r i a b l e s . Using t h i s n o t ation, B i r c h has shown that models corresponding to d i f f e r e n t hypotheses are defined by omitting one or more terms from expression Cl) i n order of descending hierarchy. For instance, i f we wish to postulate that there i s no three-factor e f f e c t then ui 2 3 ( i j k ) ~ ® 90. f o r a l l i , j , k , or more b r i e f l y , ui 2 3 = ^ a n c* t*i e i a s t t e r m ° f expression (1) disappears. I t i s not p o s s i b l e to write down the estimates for the elementary c e l l s as d i r e c t products of the configuration c e l l s f o r a l l models. When i t i s not p o s s i b l e , the estimates can be obtained i t e r a t i v e l y . In three dimensions the only model that requires i t e r a t i o n i s the one mentioned above, that of no three f a c t o r e f f e c t . We now describe the i t e r a t i v e procedure. Preliminary values CO) a r e Pu t in every elementary c e l l of the matrix; i n p r a c t i c e we use the value 1 for every c e l l and i t i s apparent that as log 1 = 0 we have not introduced unwanted m u l t i p l e - f a c t o r e f f e c t s . Any constant could be used, or any set of numbers that do not e x h i b i t higher-order (0) e f f e c t s than those we wish to estimate. The preliminary v a l u e s , Yi j k » are then adjusted to y i e l d estimates Y.f?^ i n each c e l l where i j k (0) C 1 ) Yi i k fi 1 Y. „ = —~* , n S ^* . The new estimates are again adjusted to y i e l d xjk y ( 0 ) i j -(1) ( 2 ) Yi i k fi k Y. ., = — — T T T ^ — . The c y c l e i s completed when these values are i j k Y v l ) i.k (2) ( 3 ) Yi i k f i k adjusted to y i e l d Y. ., = — / o i — • The cycle i s repeated u n t i l no i j k y*- ^  O k 91. C3r) ( 3 r - l ) d i f f e r e n c e i s d i s c e r n i b l e between Y... and Y.., . In p r a c t i c e i j k 13k we proceed u n t i l no c e l l estimate d i f f e r s from the preceding estimate for t h i s c e l l by more than 0.01. Applying the i t e r a t i v e procedure to our considered contingency ,table we obtain the following sequence of tabl e s : CO) Yi j k Sum Sum 1 1 2 1 1 2 1 1 2 1 1 2 1 1 2 1 1 2 1 1 2 1 1 2 Sum CD Yi j k 4 4 8 Sum TABLE VI Sum 4 4 8 Sum 98.0 28.0 126.0 98.0 28.0 126.0 81.0 15.0 96.0 81.0 15.0 96.0 72.5 11.5 84.0 72.5 11.5 84.0 40.5 7.5 48.0 . 40.5 7.5 48.0 Sum 292.0 62.0 354.0 Sum 292.0 62.0 354.0 TABtE..vfi Y i j k Sum 112.0000 0000 32.0000 0000 144.0000 0000 81.0000 0000 15.0000 0000 96.0000 0000 82.8571 4286 13.1428 5714 96.0000 0000 40.5000 0000 7.5000 0000 48.0000 0000 Sum (2) Y. i j k 316.3571 4286 67.6428 5714 384.0000 0000 Sum 84.0000 0000 24.0000 0000 108.0000 0000 81.0000 0000 15.0000 0000 96.0000 0000 62.1428 5714 9.8571 4286 72.0000 0000 40.5000 0000 7.5000 0000 48.0000 0000 Sum 267.6428 5714 56.3571 4286 324.0000 0000 TABLE VIII C3) Y i j k Sum 111.8735 6061 32.1689 5460 144.0425 1521 80.9085 5724 15.0791 9747 95.9877 5471 82.7636 0354 13.2122 4920 95.9758 5274 40.4542 7860 7.5395 9874 47.9938 7734 Sum (3) Y i j k 315.9999 9999 68.0000 0001 384.0000 0000 Sum 84.1120 8968 23.8479 0874 107.9599 9842 81.1080 8647 14.9049 4296 96.0130 2943 62.2257 8060 9.7946 7681 72.0204 5741 40.5540 4325 7.4524 7148 48.0065 1473 268.0000 0000 55.9999 9999 323.9999 9999 TABLE IX 93. We continue t h i s c y c l e three more times and a r r i v e at the following table a f t e r rounding o f f to the f i f t h decimal place: ( 1 2 ) Sum 111.84777 32.15223 144.00000 80.90964 15.09036 96.00000 82.78777 13.21223 96.00000 40.45482 7.54518 48.00000 Sum C 1 2 ) i j k 316.00000 68.00000 384.00000 Sum 84.15223 23.84777 .... 108.00000 ... 81.09036 14.90964 96.00000 62.21223 9.78777 72.00000 40.54518 7.45482 48.00000 Sum 268.00000 56.00000 TABLE X 324.00000 TABLES VI- X CALCULATION OF EXPECTED VALUES USING ITERATIVE METHOD We now c a l c u l a t e x = s — j j where the summation extends over a l l the c e l l s . We use the values i n the l a s t table as the expected c e l l v a l u e s . 94. Consider the following table : (i j k ) ° i j k E. ., i j k . . . l ° i j k -Ei j k l c w ) 2 C° i j k -Ei j / Ei . i k 111 112 111.84777 .15223 .02317 .00021 211 76 80.90964 4.90964 24.10456 .29792 311 87 82.78777 4.21223 17.74288 .21432 411 41 40.45482 .54518 .29722 .00735 121 32 32.15223 .15223 .02317 .00072 221 20 15.09036 4.90964 24.10456 1.59735 321 9 13.21223 4.21223 17.74288 1.34291 421 7 7.54518 .54518 .29722 .03939 112 84 84.15223 .15223 .02317 .00028 212 86 81.09036 4.90964 24.10456 .29726 312 58 62.21223 4.21223 17.74288 .28520 412 •40 -40.-5-4518 .54518 .29722 .00733 122 24 23.84777 .15223 .02317 .00097 222 10 14.90964 4.90964 24.10456 1.61671 322 14 9.78777 4.21223 17.74288 1.81276 422 8 7.45482 .54518 .29722 .03987 7 RfiflSS TABLE XI CALCULATION OF x USING EXPECTED VALUES DERIVED FROM ITERATIVE METHOD We note that the expected values i n Table V and i n Table XI 2 d i f f e r by very l i t t l e . * = 7.56055. Hence we would r e j e c t Un: u-, 0 0, , 0 12J(ijk) fo r a l l i , j , k at the ten percent l e v e l but not at the f i v e percent l e v e l . F i n a l l y l e t us c a l c u l a t e U = - 2 l o g g L using Bishop's expected = 0 val u e s . 95. n! . TL . I L , TL i = l j = l k=l 13 k ' i j k 11! J L . I L , I L 1=1 .3=1 k=l ' i j k fi j k! 4 2 2 / f. . \  Xi k iSi'j=i fcSi [ f"^"*"' I » where f ^ j k a r e Bishop's expected values, l j ^ / 4 '2 2 „ -4 "2 '2 L = E S E fi i k l o g fi i k " E E E fi i k l 0 S fi i k 1=1 j = l k=l 1 J 1 C 1 : ) k i = l j = l k=l l j l C l j k j i j i k i f U k l 0 ^ i j k - 2893.81748 \ 1 96. ( f i v e d i g i t s ) l o g f . fi j k f . ., log f , .. xjk ° i j k 111.85 4.71716 112 528.32192 80.910 4.39333 76 333.89308 82.788 4.41628 87 384.21636 40.455 3.70019 41 151.70779 32.152 3.47047 32 111.05504 15.090 2.71403 20 54.28060 13.212 2.58112 9 23.23008 7.5452 2.02091 7 14.14637 84.152 4.43262 84 372.34008 81.090 4.39555 86 378.01730 62.212 4.13054 58 239.57132 40.545 3.702410 40 148.09640 23.848 3.17170 24 76.12080 14.910 2.702030 10 27.02030 9.7878 2.28114 14 31.93596 7.4548 2.00886 8 16.07088 2890.02428 TABLE XI VALUES A. OF f . M log i j k f . M WHERE xjk f . . , ARE BISHOP'S EXPECTED VALUES xjk Log e L = 2890.02428 - 2893.81748 = - 3.79320. U = - 2Log £ L = 7.58640, hence we r e j e c t H^: u1 2 3 ( i j k ) = ^ ' f o r a 1 1 i' 3 ' k a t t h e t e n Pe r c e n t l e v e l but not at the 5% l e v e l . These r e s u l t s are i n close agreement with our previous r e s u l t s . 9 7 . D. COMPARISON OF ADDITIVE AND MULTIPLICATIVE (LOG LINEAR) MODELS FOR A THREE-WAY CONTINGENCY TABLE. For the additive model of an r x s x t contingency table: pijk = p ± . . P . J . p . . k + pi.. ajk + P . J Ak + p . . k Y i j + 6ijk' , * e r e a j k = p . j k " p . j . p..k' 6 i k = p i . k ~ p i . . p . . k ' Yu = p i j . " p i . . p . j . - a n d  5 ± j k = p i j k ' p i . . p . j k ~ p . j . p i . k ~ p . . k p i j . + 2 p i . . p . j . p . . k • The most i n t e r e s t i n g hypotheses f o r t h i s model are I 6 i j k = 0, f o r a l l i , j , k , p i j k = P ± . . P . j f e + P.j . p i . k ^ . . k P i j . ~ 2 p i . . P . j . P I I 6 i j k = 0 and Y ± j = 0 f o r a l l i . J . k . p i j k = P i > . P . J ^ . J . P i . k ^ i . . P . j . P . .k I I I 6 i j k = 0, Y ± j - 0, e ± k - 0 f o r a l l i . j . k , P ± J K = P ± . P < J K IV 6 i j k = 0, Y i j - 0, B ± k = 0, a j k = 0 f o r a l l l . J . k , ' P L J K - P ± . . P . J . P . . K The maximum l i k e l i h o o d estimators f o r these hypotheses can be obtained by Lagrangian methods simlar to those presented e a r l i e r i n t h i s t h e s i s . 98. Parameter Constraints 2 P .^(MLE) f o r L i k e l i h o o d Ratio .or x Tests Additive Model Log-Linear Model No Constraints on model parameters • V n f ., Observed c e l l —^— prop o r t i o n s . Zero 2nd-order i n t e r a c t i o n s ; I no constra i n t s on 1st order (AB, AC,BC) i n t e r a c t i o n . - 0 ( f . £ M + f . f . ,+f , f . . . ) 2 i . . .jk . j . i.k . .k i j . n - -M. f . f .) n3 x" -J- - *k no closed-form expression; requires i t e r a t i v e numerical estimation procedure. Zero 2nd-order i n t e r a c t i o n s and zero AB 1st II order i n t e r -a c t i o n . - . ( f . f .,+f . f . ) n2 i . . .jk , j . i.k " V fi f j f k> n .jk ^ i . . + ^ i . k ^ . j . " f..k n f..k n fi . .f. . j f..k n n n fi . k f. j k n f. . k i»lc • j lc • • lc • • lc • • lc v a r i a b l e s A and B c o n d i t i o n a l l y independent given l e v e l of C. Zero 2nd-order, I I I AB, AC i n t e r a c t i o n . f . j kfi . . ^ . j k ^ i . . A independent 2 n 2 of B and C n . . . j o i n t l y , Independence IV zero 2nd order AB, BC, and AC i n t e r a c t i o n . f . f . f . 1 • • • "1 • • * iC f . f . f . 3 n 3 n TABLE XIII ADDITIVE MODEL V.S. MULTIPLICATIVE (LOG LINEAR) MODEL. 99. The two models are equivalent in cases of f u l l independence or one set of first-order interactions. In case II, with two sets of first-order interactions, the log-linear model has a concrete, useful interpretation: i f this hypothesis is accepted, then the overall 2-way table of A cross classified with B is regarded as a pooling (over levels of G) of several independent B x C tables. Case 31 for the additive model does not have an obvious interpretation. One virtue of the additive model is that maximum likelihood estimates under a l l its natural hypotheses are intuitive, are easy to verify theoretically, and can be calculated in closed form, whereas the log-linear model's hypothesis of zero second order interactions requires an iterative numeric procedure to compute maximum likelihood estimates. B I B L I O G R A P H Y 100. [1] Apostol M.T. (1957) Mathematical Analysis, A Modern Approach to Advanced Calculus. Reading, Mass.: Addison-Wesley Co., Inc. pp.152-156. [2] Bartlett M.S. (1935) "Contingency Table Interactions." J. Roy. Statist. Soc, Suppl. 2. pp.248-252. [3] Birch M.W. (1963) "Maximum Likelihood in Three-way Contingency Tables." J. Roy. Statist. Soc, Ser. B, Vol. 25. pp.220-233. 14] Bishop Y.M.M. (1969) "Full Contingency Tables, Logits, and Split Contingency Tables." Biometrics, Vol. 25. pp. 383-399. 15] Bishop Y.M.M. and Fienberg S.E. (1969) "Incomplete Two- Dimensional Contingency Tables." Biometrics, Vol. 25. pp. 119.-128. n 16] Bishop Y.M.M. and Mosteller F. (1969) "Smoothed Contingency Table Analysis." Chapt. IV - 3 of the National Halothane Study, U.S. Gov. Printing Office, pp. 238-272. 17] Darroch J.N. (1962) "Interactions in Multifactor Contingency Tables." J . Roy. Statist. Soc, Ser B, Vol 25. pp251-263. [8] Denting E. W. and Stephan F.F. (1940) "On a Least Squares Adjustment of a Sampled Frequency Table When the Marginal Totals A^e Known." Ann. Math. Statist., Vol. 11. pp. 427-444. [9] Feller W. (1968) An Introduction to Probability Theory and Its Applications. Vol. I, 3rd. Edition. New York: John Wiley & Sons, Inc. pp.125-128. 101. [10] Flenberg S.E. (1970) "The Analysis of Multidimensional Contingency Tables." Ecology, Vol. 51, No. 3. pp .-419-433. £11] Fienberg S.E. (1970) "An Iterative Procedure for Estimation in Contingency Tables." Ann. Math. Statist.,Vol. 41. pp.907-917. [12] Good I.J. (1963) "Maximum Entropy for Hypothesis Formulation, Especially for Multidimensional Contingency Tables." Ann. Math. Statist., Vol. 34. pp.911-934. [13] Good I.J. (1965) "The Estimation of Probabilities: An Essay on Modern Bayesian Methods." Research Monographs, No. 30. Cambridge, Mass. The M.I.T. Press. 114] Good I.J. (1966) "How to Estimate Probabilities." J. Inst. Math. Appl., Vol. 2. pp.364-383. [15] Goodman L.A. (1964a) "Interactions in Multidimensional Contingency Tables." Ann. Math. Statist., Vol. 35. pp.632-646. [16] Goodman L.A. (1964b) "Simple Method for Analyzing Three-Factor Interaction in Contingency Tables." J. Amer. Statist. Assoc., Vol. 59. pp.319-352. [17] Goodman L.A. (1968) "The Analysis of Cross-Classified Data." J. Amer. Statist. Assoc., Vol. 63 pp.1091-1131. [18] Goodman L.A. Q.969) "On Partitioning Chi-Square and Detecting Partial Association in Three-Way Contingency Tables." J. Roy. Statist. Soc, Ser B, Vol 31.pp.486-498. 102. [19] Kastenbaum M.A. and Lamphier D.E. (1959) "C a l c u l a t i o n of Chi-Square to Test the No Three-Factor I n t e r a c t i o n Hypothesis." Biometrics, V o l . 15. pp.107-115. £20] Kolmogorov A.N. (1956) Foundations of the Theory of P r o b a b i l i t y . New York: Chelesa Pub. Co. J21] Ku H.H. and Kullback S. (1968) "Interaction i n Multidimensional Contingency Tables: An Information Theoretic Approach." J . Res. Nat. Bur. Standards, Sect. B. V o l . 72, No. 3. pp.159-199. 122] Kullback S. (1959) Information Theory and S t a t i s t i c s . New York: John Wiley & Sons, Inc. pp.155-188. 123] Lancaster H.o. (1951) "Complex Contingency Tables Treated by the o P a r t i t i o n of x •" J« R° y - S t a t i s t . S o c , Ser. B, V o l . 13. pp.242-249. [24] Lancaster H.O. (1969) The Chi-Squared D i s t r i b u t i o n . New York: John Wiley & Sons, Inc. [25] Mardia K.V. (1970) Families of B i v a r i a t e D i s t r i b u t i o n s . London: Charles G r i f f i n & Co., L t d . pp.30-33. 126] Maxwell A.E. (1961) Analyzing Q u a l i t a t i v e Data. London: Methuen & Co., L t d . [27] M o s t e l l e r F. (1968) "Association and Estimation i n Contingency Tables" J . Amer. S t a t i s t . A s s o c , V o l . 63. pp.1-28. 103. [28] Norton H.W. (1945) " C a l c u l a t i o n of Chi-Square for Complex Contingency Tables" J . Amer. S t a t i s t . Assoc., V o l . 40. pp.251-258. [29] Rao C R . (1952) Advanced S t a t i s t i c a l Methods i n Biometric Research. New York: John Wiley & Sons, Inc. pp.165-166. [30] Roy S.N. and Kastenbaum M.A. (1956) "On the Hypothesis of No I n t e r a c t i o n i n a Multiway Contingency Table." Ann. Math. S t a t i s t . , V o l . 27. pp.749-757. [31] Simpson E.H. (1951) "The Int e r p r e t a t i o n of Int e r a c t i o n i n Contingency Tables." J . Roy. S t a t i s t . S o c , Ser. B, V o l . 13. pp.238-241. [32] Sisam C H . (1946) College Mathematics. New York: Henry Holt & Co. pp.382-383. [33] Wilks S.S. (1962). Mathematical S t a t i s t i c s . New York: John Wiley & Sons, Inc. pp.39-41, 49-50. [34] Woolf B. (1955) "On Estimating the Rela t i o n Between Blood Group and Disease." Ann. Hum. Gen., V o l . 19. pp.251-253. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            data-media="{[{embed.selectedMedia}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0080461/manifest

Comment

Related Items