UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Likelihood ratios in asymptotic statistical theory Leroux, Brian Gilbert 1985

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1985_A6_7 L47.pdf [ 2.38MB ]
Metadata
JSON: 831-1.0096109.json
JSON-LD: 831-1.0096109-ld.json
RDF/XML (Pretty): 831-1.0096109-rdf.xml
RDF/JSON: 831-1.0096109-rdf.json
Turtle: 831-1.0096109-turtle.txt
N-Triples: 831-1.0096109-rdf-ntriples.txt
Original Record: 831-1.0096109-source.json
Full Text
831-1.0096109-fulltext.txt
Citation
831-1.0096109.ris

Full Text

LIKELIHOOD  RATIOS IN ASYMPTOTIC S T A T I S T I C A L  THEORY  By BRIAN GILBERT LEROUX B.Sc,  Carleton  U n i v e r s i t y , 1982  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE  in THE FACULTY OF GRADUATE STUDIES Department of S t a t i s t i c s  We accept  t h i s t h e s i s as conforming  to the r e q u i r e d  standard  THE UNIVERSITY OF BRITISH COLUMBIA A p r i l 1985  ©Brian G i l b e r t Leroux, 1985  In p r e s e n t i n g  t h i s t h e s i s i n p a r t i a l f u l f i l m e n t of  requirements f o r an advanced degree a t the  the  University  o f B r i t i s h Columbia, I agree that the L i b r a r y s h a l l make it  f r e e l y a v a i l a b l e f o r reference  and  study.  I further  agree t h a t p e r m i s s i o n f o r e x t e n s i v e copying of t h i s t h e s i s f o r s c h o l a r l y purposes may department or by h i s or her  be granted by the head o f representatives.  my  It i s  understood t h a t copying or p u b l i c a t i o n of t h i s t h e s i s f o r f i n a n c i a l gain  s h a l l not be  allowed without my  permission.  Department of The U n i v e r s i t y of B r i t i s h Columbia 1956 Main Mall Vancouver, Canada V6T 1Y3 Date  March  If,  written  ABSTRACT  This  t h e s i s d e a l s w i t h two  topics  concept of asymptotic o p t i m a l i t y hypotheses i s i n t r o d u c e d .  properties.  properties  Secondly, the  c h i - s q u a r e t e s t f o r goodness of The  for sequential  t e s t s of  asymptotic power of i n a new  likelihood  r a t i o of two  likelihood  r a t i o based on a sample of s i z e n has  as n ->•  00  and  the  limit  In s i t u a t i o n s  can  be made u s i n g the  way. tests i s  examined here  the  the  a limiting distribution  i s also a l i k e l i h o o d r a t i o .  l i m i t i n g v a l u e s of v a r i o u s performance c r i t e r i a calculations  Pearson's  asymptotic performance of  hypotheses.  shown  c o r r e s p o n d i n g to t h e i r u s u a l  f i t i s derived  main t o o l f o r e v a l u a t i n g  A  statistical  S e q u e n t i a l P r o b a b i l i t y R a t i o T e s t s are  to have asymptotic o p t i m a l i t y optimality  i n asymptotic s t a t i s t i c s .  To  calculate  of s t a t i s t i c a l  limiting likelihood ratio.  tests  the  -  iii  T A B L E OF CONTENTS  Page Abstract Table  i i  of Contents  i i i  List  of Tables  iv  List  of F i g u r e s  v  Acknowledgement  vi  INTRODUCTION  1  CHAPTER 1 - THE THEORY OF LIKELIHOOD RATIOS  3  1.1 1.2 1.3 1.4 1.5  L i k e l i h o o d R a t i o s and Hypothesis T e s t i n g S e q u e n t i a l T e s t s o f Hypotheses Weak Convergence of L i k e l i h o o d R a t i o s F u n c t i o n a l Convergence of L i k e l i h o o d R a t i o s C o n t i g u i t y and Convergence of Experiments  CHAPTER 2 - ASYMPTOTIC OPTIMALITY OF SEQUENTIAL TESTS  3 8 12 19 23  26  2.1  Wald's C r i t e r i o n  26  2.2  Bayes R i s k C r i t e r i o n  32  CHAPTER 3 - POWER OF CHI-SQUARE TESTS  40  BIBLIOGRAPHY  51  • APPENDIX A.l  A.2  A.3  Uniform I n t e g r a b i l i t y of a Sequence of Stopping Rules  53  The L i k e l i h o o d R a t i o of S i n g u l a r M u l t i v a r i a t e Normal D i s t r i b u t i o n s  55  Two Lemmas on Weak Convergence  59  -  LIST  iv  OF  -  TABLES  Page Table  I.  Asymptotic on Z of  n  for  power values  freedom k -  1 of  1 of  $(Z size the  a  a,  v2[)  of  power  chi-square  the 3 and test  test  based  degrees 50  - vL I S T OF FIGURES  Page Fig.  1.  Graph of h which determines A, B of o p t i m a l SPRT  stopping  boundaries 36  - vi -  ACKNOWLEDGEMENT  The  author, being one who t h r i v e s on encouragement, wishes  P r o f e s s o r s Cindy Greenwood and John Petkau gave.  to thank  f o r the constant supply they  -  1 -  INTRODUCTION  The  motivation  concerning  behind  some of t h i s work l i e s  a s e q u e n t i a l procedure f o r t e s t i n g  i n a problem  the mean of a normal  The f o l l o w i n g d i s c u s s i o n of t h i s problem f o l l o w s [ 3 ] .  distribution.  There a r e observed independent i d e n t i c a l l y d i s t r i b u t e d o b s e r v a t i o n s 2  required  to f i n d  positive  or n e g a t i v e  2  as N(y, a ) f o r a known a .  assumed to be d i s t r i b u t e d  X2,.«.  X\,  It i s  a s e q u e n t i a l procedure f o r t e s t i n g whether u i s ( s e q u e n t i a l procedures are d i s c u s s e d  i n Section  1.1).  The c r i t e r i o n by which procedures are to be judged i s the Bayes  Risk.  This i s defined  i n terms of a cost f u n c t i o n having  components, one due to r e a c h i n g  an i n c o r r e c t c o n c l u s i o n and a second  depending on the number of o b s e r v a t i o n s based.  on which the c o n c l u s i o n i s  The proposed c o s t s a r e | | f o r making an e r r o r (K i s a K  constant) The  two  u  and a cost of c per o b s e r v a t i o n . average cost f o r a g i v e n procedure w i l l  problems i n v o l v e d w i t h  depend on u.  To a v o i d  t h i s i t i s assumed that u i s a random v a r i a b l e ,  a l s o w i t h a normal d i s t r i b u t i o n .  I f i t s mean and v a r i a n c e  are s p e c i f i e d  the average cost can be averaged f u r t h e r a g a i n s t t h i s d i s t r i b u t i o n f o r y.  The r e s u l t  i s the Bayes  Risk.  In the development of a s e q u e n t i a l procedure which minimizes the Bayes R i s k  the p a r t i a l  Brownian motion. observations the  sums of the o b s e r v a t i o n s  T h i s i s a reasonable  are r e p l a c e d by a  approximation i f the number of  can be expected to be l a r g e , and t h i s can be expected when  cost c i s s m a l l .  A procedure which i s o p t i m a l  (minimizes  Bayes  -  R i s k ) i n the small  2 -  continuous time s e t t i n g i s d e r i v e d  and  adjustment) to the d i s c r e t e time s e t t i n g .  result  It i s desired  s t a t i n g t h a t t h i s procedure i s a s y m p t o t i c a l l y  sense which can  be made p r e c i s e .  approaching z e r o .  Results  along  optimal  these l i n e s can be  i n some  s i t u a t i o n of s e q u e n t i a l  of weak convergence of l i k e l i h o o d r a t i o s which w i l l Success was  where there  are only two  presented d i s c u s s i o n s  met  only  where  medical  (see  author attempted to e s t a b l i s h s i m i l a r r e s u l t s u s i n g  Chapter 1.  a  to have a  found i n [13]  i n which f u r t h e r components of cost are c o n s i d e r e d  This  (with  Asymptotic here r e f e r s to c  the s e t t i n g i s the more complicated trials  then a p p l i e d  [4]). the  be d i s c u s s e d  theory in  i n simple h y p o t h e s i s t e s t i n g s e t t i n g s  p o s s i b l e s t a t e s of n a t u r e .  of a s y m p t o t i c a l l y  which are based on  the l i k e l i h o o d r a t i o .  methods used there  could  be a p p l i e d  optimal  In Chapter 2  sequential  It i s believed  procedures  that  s u c c e s s f u l l y i n more  are  the  complicated  situations. Another area  f o r a p p l i c a t i o n of l i k e l i h o o d r a t i o  c a l c u l a t i o n of asymptotic performance of other based on  the  chi-square indicated  likelihood ratio.  that  the  chi-square  compared to a t e s t based on the  t e s t s not  lies  in  the  necessarily  In Chapter 3 the asymptotic power of  t e s t s i s s t u d i e d v i a the theory there  theory  of Chapter 1.  test i s asymptotically  likelihood ratio.  It is inefficient  -  3  -  CHAPTER 1 THE  1.1  THEORY OF LIKELIHOOD RATIOS  Likelihood Ratios and Hypothesis Testing We d e s c r i b e the g e n e r a l h y p o t h e s i s  distinguishing  two p r o b a b i l i t y measures.  p r o b a b i l i t y measures PQ and or  t e s t i n g problem of  a  n  d i» p  the q u e s t i o n i s asked: the d i s t r i b u t i o n P j ?  On a s e t ft l e t t h e r e be  A random element X of n i s chosen  was X chosen based on the d i s t r i b u t i o n Pg  A decision  rule  f o r answering the q u e s t i o n i s  a subset D o f ft; i f X belongs to D then i t i s d e c i d e d t r u e d i s t r i b u t i o n , o t h e r w i s e PQ« the  that  In common language D i s a t e s t of  simple h y p o t h e s i s HQ:PO v e r s u s the simple h y p o t h e s i s  D i s also  ?\ i s the  H^:Pj.  c a l l e d the r e j e c t i o n r e g i o n because the o c c u r r e n c e of the  event D leads t o the r e j e c t i o n of the n u l l h y p o t h e s i s HQ i n f a v o r of the  alternative  .  Each d e c i s i o n ct(D) = PQ(D)  B(D)  r u l e has a s s o c i a t e d = probability  w i t h i t two e r r o r  of r e j e c t i n g HQ when i t i s t r u e , and  = P ^ D ) = p r o b a b i l i t y of a c c e p t i n g ^ 0  and type I I e r r o r  when i t i s f a l s e ,  called  the type I e r r o r  called  the l e v e l and 1 - $ ( D ) the power of the t e s t D.  Because i t i s g e n e r a l l y  f o r comparing d e c i s i o n  In many cases the best r u l e s  r a t i o which we w i l l  now  respectively.  a(D)  i s also  i m p o s s i b l e t o minimize both types of e r r o r  simultaneously, various c r i t e r i a employed.  probabilities:  define.  r u l e s have been  are based on the l i k e l i h o o d  -  4  -  Given two p r o b a b i l i t y measures PQ and P^ a b s o l u t e l y continuous w i t h r e s p e c t ratio  to P  Q  such that Pj i s  (F^ < P ) , n  i s the Radon-Nikodym d e r i v a t i v e d P j / d P . the l i k e l i h o o d  likelihood  T h i s can be g e n e r a l i z e d  n  by d e f i n i n g  their  r a t i o of any two p r o b a b i l i t y measures P^,  P^  on a measure space (ft, F) by  dP,/dp z  dVdii  =  "-d  where p i s any measure on (ft, F) such that P^ < p and PQ € p (such as P = P In the  + Pj).  Q  The conventions 1/0 = °° and 0/0 = 0 are used i n ( 1 . 1 ) .  o r d e r to show that Z does not depend on the p a r t i c u l a r c h o i c e of p following result  Lebesque  i s needed.  Decomposition.  For any  J  dp Proof:  A  z  dP  A  Q  =  and X  / Z dP  An{z<»} since  {z = °°} = Now  {X  =  .  0  =  / X .Z dp =  = 0} and  n  = P,(A)  AeF  X Z = X^^ on Q  1  dp = Pj(An{z <  »})  An{z<°°} {z <  because P Q ( ^ Q = 0 ) = 0, Z i s a f i n i t e 0  Z dP  / X  Q  p r o b a b i l i t y space (ft, F, P ) . /»  Then f o r any  dp An{z<~}  Q  {z < -}).  dpj  0  L e t XQ =  Z dP  = PJ(An  0  dp /  AeF,  °°}. random v a r i a b l e on the  For any Ac{z < ~} the i n t e g r a l  i s determined and so Z i s u n i q u e l y determined  on  -  (ft,  F,  hence  is  Pg)« Z is  used  to  from here  with  symmetry  1/Z  also  uniquely  d e t e r m i n e d on  this  on d P ^ / d P g  extension  will  Example.  When Pg  densities  fg and  uniquely  are  the  the  test  likelihood  support  of  Since  Z is  of  Hg v s .  ratio  fg i s  Pg  expected to  ?±.  in  the  H]^ u s e s  be  larger  the d e c i s i o n  D*  where C i s has  the  a constant  following  Neyman-Pearson <_ a ( D * )  then  Proof:  By  which  nice  Lemma.  6(D)  >  =  P^)«  defined  the  in  derivative  dP^/dP^  and  (1.1).  distributions  ratio  and  The n o t a t i o n  of  on R  densities  It  need not  be assumed  support  of  f±.  when  is  true  a  that  reasonable  rule  {z >  determines  C}  the  level  of  the  test.  of  Hg v s  Hj  satisfying  This  property.  If  D is  any  test  3(D*).  the Lebesque  0(D)  F,  F,  V f g  and  contained  (ft,  probability  respectively  of  d e t e r m i n e d on (ft,  the Radon-Nikodym  d e n o t e Z as  and P^ fj  of  Z -  is  -  By  denote  is  5  Decomposition  = P^D ) 0  = /  c  Z dP  + P ( D n {Z c  Q  1  = »}).  a(D)  rule  -  6  -  Since D * f l {z = °°} = 0, c  P^D ^! {z = »}) > P ( D * n {z = »}). 0  c  1  Also  / D  Z dP  -  Q  c  D  J Z dP *c  D  *  s i n c e Z > C on D —  = / (I * D D  Q  D  - I D  ) Z dP  D  c  D  -  I D  J  +  D  D  and I  Q  *c  (I - I *c D D  D  * < 0 on D *c —  ) Z dP  Q  D  .  Therefore / D  Z dP  Q  -  c  / D  =  Z dP  Q  > C/(I  *c  c  -  D  C[P (D*) - PQ(D)] N  This r e s u l t  I * ) c  dP  0  =  C  Q  >  C  0  0.  says that among a l l r u l e s having  type I e r r o r at most  E q u i v a l e n t l y , among  r u l e s with type I I e r r o r at most P i ( D * ) , C  type I e r r o r .  P (D)* )]  D  P Q ( D * ) , D* has the s m a l l e s t type I I e r r o r . all  C[P (D ) -  D* has the s m a l l e s t  A simple and symmetric f o r m u l a t i o n i s :  no r u l e can s i m u l t a n e o u s l y have both a s m a l l e r type I e r r o r and a s m a l l e r type I I e r r o r than D .  I f P Q ( D * ) = a then D* i s c a l l e d  an o p t i m a l l e v e l - a t e s t .  g i v e n number a i t may be i m p o s s i b l e to f i n d  PQ(Z  j> C) = a.  There w i l l  For a  a number C such  always be a randomized  that  d e c i s i o n r u l e which  -  achieves  7  -  t h i s but these w i l l not be c o n s i d e r e d here.  d i s c u s s i o n of randomized  See [15] f o r a  rules.  D e c i s i o n r u l e s having  the form of D  sense of minimal Bayes R i s k .  are o p t i m a l a l s o i n the  The Bayes Risk f o r a r u l e D i s  IT a(D) + (1 -  TT) 3(D)  (1.2)  where IT i s the p r i o r p r o b a b i l i t y of the d i s t r i b u t i o n being Pg*  This  assumes the 0-1 cost ( o r l o s s ) f u n c t i o n whereby a c o s t of 1 i s i n c u r r e d when an e r r o r of e i t h e r type i s made. probability when t h i s  (under  Now the expected  the a p p r o p r i a t e h y p o t h e s i s )  i s averaged  over  c o s t i s the  of making an e r r o r and  the two hypotheses a c c o r d i n g to the p r i o r  p r o b a b i l i t y TT, (1.2) r e s u l t s . For f i x e d TT the Bayes R i s k i s minimized  by D  w i t h C having the  v a l u e TT/(1 - TT) , i . e .  inf[TT ct(D) + (1 - TT) 3(D)] = / [ T T A ( 1 - TT) Z] dP 0 D (  and  the infimum i s achieved  a t D„ = {z J> i r / ( l - TT)}.  e a s i l y u s i n g the Lebesque Decomposition  TT  aCD^) + (1  -  TT)  3(0^) =  -  and  f o r any D  as f o l l o w s .  /[TTA (1 - TT) Z] dP,  T h i s i s proved First,  -  TT cx(D) + (1-TT) g ( D ) = / ir d P  8 -  Q  + J  (1-TT) Z d P  + (1-TT) P ( D f t {Z = » } )  Q  + / ( l - TT) Z dP  c  0  D  1.2  dP  >  / TT  >  / [ TT A  c  (  (1 - TT) Z] dPQ.  S e q u e n t i a l Tests of Hypotheses Wald's S e q u e n t i a l P r o b a b i l i t y  R a t i o Test  (SPRT) i s a procedure f o r  testing  H :P Q  0  v s . H :P 1  1  based on a sequence X^ , X 2 , . . . of independent i d e n t i c a l l y ( i . i . d . ) random v a r i a b l e s h a v i n g d i s t r i b u t i o n  either  distributed  PQ or P^.  If  Xj,...X^ are observed the SPRT uses the s t a t i s t i c  k \  = k  where Z i s the l i k e l i h o o d Z^ i s the l i k e l i h o o d F\ w i t h r e s p e c t  IT  i-1  Z(X.)  (1.3)  1  r a t i o dPi/dPrj. T h i s i s r e a s o n a b l e because  r a t i o of the d i s t r i b u t i o n of Xj,...,X^ under  to the d i s t r i b u t i o n under Pg.  The SPRT proceeds as  follows:  if  Z^ _> A then  i s accepted  if  Z^ <^ B then HQ i s accepted  if  B < Z^ < A then another o b s e r v a t i o n i s taken,  -  i n terras of the s t o p p i n g  T = i n f {k: Z  and  -  B satisfying 0 < B < ^ 1 < ^ A < ° ° .  w i t h A and  expressed  9  can  be  rule  > A or Z  k  T h i s procedure  k  <  (1.4)  B}  the d e c i s i o n r u l e  D =  {Z  where Z-p denotes the v a l u e of Z An  k  > A},  T  when T = k.  immediate q u e s t i o n a r i s e s :  the boundaries  determined  by A and  probability distributions,  can i t happen t h a t B?  negative  and  .  The  to Pg and  never question  Pj, for  These are the i n f i n i t e  questions  crosses  infinite product  above i s answered i n the  by  Q (B  <  Z  k  <  A,  k  =  1,2,...) =  0.  Q^B  <  Z  k  <  A,  k  =  1,2,...) =  0.  Q  These statements  A proof  To answer t h i s  corresponding  sequences X} , X 2 , . . . must be used. measures denoted Qg  (1.5)  are i m p l i e d by  the s t r o n g e r  Z  k  -*• 0 a.s. under OQ  Z  k  •*•  00  a.s. under  of these uses the Strong Law  sequence {-a V  l o g Z(X  )}"  = 1 >  results  .  of Large  Numbers a p p l i e d to  the  where X V Y i s d e f i n e d to be the l a r g e r of  -  X and Y. in  place  Note that the SPRT could of Z^.  10 -  e q u a l l y w e l l be d e f i n e d  with  logZ^  The Strong Law of Large Numbers y i e l d s  1 1 1 m - E (-a V l o g Z(X )) - E (-a V l o g Z(X.)) a.s. (Q ). ^.oo i I u i u n  n  where E Q denotes expected value provided  PQ and V\  are d i s t i n c t  under QQ.  Inequality,  i n the sense that P Q ( Z = 1) < 1,  E ( l o g Z ) = JlogZ 0  For  By Jensen's  dP  < log / Z dP  Q  0  < 0.  l a r g e enough a then  E ( - a V l o g Z C x p ) = E ( - a V logZ) < 0 0  Q  and k lim E (-a V l o g Z ( X . ) ) = -°° a.s. ( Q ) k-x» i = l n  lim k+»  E l o g Z(X,) = - » a.s. i=l  l i m Z, = 0 a.s. k->-°°  By  n  (Q ). Q  symmetry, l i m - i — = 0 a.s. ( Q i ) and so l i m Z, = °° a.s. k->-°° k k->-<*>  We have j u s t met  (Q )  eventually,  other  sequential  seen that  the c o n d i t i o n s  i . e . , T i s a.s. f i n i t e . t e s t s of HQ v s .  .  f o r stopping  (Q.)  i n (1.4) w i l l be  Now l e t us compare the SPRT t o  A sequential  test  i n general  -  consists  of  a stopping  random v a r i a b l e set  {T = k}  sequential  test  until  by  Xi,...,Xfc.  the  is  for  the Average  stopping  tests  that  eventually.  T,  the  comparing  for  (ASN)  A stopping integers  The d e c i s i o n only  on  e a c h k,  sequential  Number  rule.  positive  depends  i.e.,  tests  which  finite  ASN  will  conditions  for  stopping  The e r r o r  The  the  is  the  the  error  (T',  D')  of  is  that  the  observations is  error  X^  determined  probabilities  the  the  probabilities any  are  almost  defined  other  a(T,  D) =  QQ(D)  0(T,  D) =  0^(0°).  following  P r o p e r t y of  is  considered worthwhile; will  surely  exactly  as  of  optimality  SPRT. the  Let  of  the  this  be met for  property.  a = O Q ( D ) and  SPRT d e f i n e d  sequential  test  of  in  HQ:PQ  (1.4)  Q (D') 0  SPRT h a s  smaller  ASN  <  a;  0  under  E (T') n  Q^D' ) <  >  both  E (T) n  B = Qi(D ) C  and  V S . H^:P^  6  hypotheses,  a  a  expected value  probabilities,  the  such  rule  D n {T = k}  include  rule  tests,  SPRT has  Optimality  be  probabilities  and  then  in  D which  time  with  the  non-sequential  error  a decision  rule.  Only implies  -  on X i , . . . , X ^ .  a set  Sample  and  values  only  random  Criteria and  T taking  depends  up  rule  11  (1.5). with  be If smaller  -  and  12  Ej(T') >  (As  f o r E Q , E ^ denotes e x p e c t a t i o n  -  Ej(T).  under Q ^ ) .  There have been f o u r s t r a t e g i e s f o r p r o v i n g original (see  i s due  [ 1 5 ] or  to Wald and  [ 1 1 ] ) and  Bayes p r o c e d u r e s . as a s e p a r a t e  two  Wolfowitz, others  The  result.  [ 2 3 ] , another i s due  ( [ 3 ] , [20]) first  This l a t t e r r e s u l t  result.  this  i s important  The  to Lehmann  prove that SPRTs are enough to be  stated  Bayes R i s k of a s e q u e n t i a l t e s t of HQ:PO  v s . H ^ r P j which uses s t o p p i n g  r u l e T and  decision rule D is  p ( T , D ; r r ) = TT(Q (D) + c E ( T ) ) + ( 1 - T T ) ( Q ( D ) + c E ^ T ) )  (1.6)  where TT i s the p r i o r p r o b a b i l i t y of the t r u e d i s t r i b u t i o n being Po  and  C  0  is  the cost per  0-1  Q  observation.  c o s t f u n c t i o n i s employed  1  J u s t as  There e x i s t  depend on TT such t h a t the Bayes R i s k stopping  This property how  ratio  testing  simple  Likelihood  For  B which  the SPRT  B. will  to the continuous time  demonstrate setting.  Ratios  hypotheses, r e s u l t s  t e s t i n g a simple  A and  ( 1 . 6 ) i s minimized by  there  are good t e s t s as measured by seen.  constants  i s proved i n [ 2 0 ] ; i n S e c t i o n 2.2 we  Weak C o n v e r g e n c e o f  For  just  boundaries A and  to adapt the argument g i v e n  1.3  i n ( 1 . 2 ) the  here.  Bayes O p t i m a l i t y of the SPRT;  which has  f o r the Bayes R i s k  c  based on  the  likelihood  the o p t i m a l i t y p r o p e r t i e s we  n u l l hypothesis  against  have  a composite  -  alternative the  a r e a s o n a b l e approach c o n s i s t s  alternative,  Xi,...,X  thus forming a new  be a random sample  n  consider testing likelihood 0.  13 -  simple a l t e r n a t i v e .  from the N ( 6 , l )  HQ:9 = 0 v s .  : 9 > 0.  r a t i o f o r HQ:9 = 0 v s .  In t h i s  of choosing one element of  case the l i k e l i h o o d  "(x  For example l e t  distribution  One  test  and  i s based on the  : 9 = 9Q f o r some f i x e d  Bq >  ratio is  2.  - 9 ) /2 2  (1.7) z  (  x  )  T7  1  =  and  the l i k e l i h o o d  e  12  x /2r?  =  e  r a t i o based on X j , • . . , X  k Z  k  -  k  k  TT z ( X )  = exp(9  ±  Q  U .  Another example i n v o l v e s the parameter distribution.  The l i k e l i h o o d  (1 + Z(x) =  and the l i k e l i h o o d  9  -(1 ) e y — ^  One  k  way  -k  0  +  9  =  (1 +  -9  (1 +  of comparing  two  9  Q  )  K  Q  is  ) x  , ±  6J/2)  9 i n the e x p ( l + 9)  9  ) e  r a t i o based on a random sample  = TT z ( X )  ,2,  ratio for H : 9 = 0 vs. 1^:9 = 9  k Z  is  x ,  (1.8)  Xj,...X^ i s  k exp(-9  tests  0  S X ). ±  (1.9)  of H Q : 9 = 0 v s . H^:9 > 0 i s to l o o k  -  at  their  performance  alternatives  H  l the  n^c/^  w  desire  approach HQ.  n  e  r  e  n  ^  a  m  Pl  e  size.  statistic  The reason f o r t h i s  When measuring  T h i s enables one to c a l c u l a t e  i t s limiting distribution  of the s t a t i s t i c  tests  based on Z a r e shown to have c e r t a i n  fit  based on the l i k e l i h o o d  the a l t e r n a t i v e ,  can be found from the j o i n t  the n u l l ,  distribution  i s essential.  two uses a r e e x p l o r e d  limiting  tests  to t h i s are  the l i m i t i n g  the performance of t e s t s  These  properties.  c r i t e r i o n that  i n two ways:  The l i m i t i n g d i s t r i b u t i o n , under  under  limiting  to s m a l l d e p a r t u r e s from 9 = 0 .  ratio i s useful  statistics  choice i s  to have a non-degenerate  making asymptotic power c a l c u l a t i o n s  ratio 2.  s  T e s t s which perform w e l l a c c o r d i n g  sensitive  the l i k e l i h o o d 1.  e  and l e t the  A common c h o i c e f o r a l t e r n a t i v e i s  ( o r a s y m p t o t i c ) power and i t i s by t h i s  considered  of  n  HQ V S . simple a l t e r n a t i v e s  the a l t e r n a t i v e .  w i l l be compared.  For  s t  f o r the t e s t  d i s t r i b u t i o n under limiting  for testing  14 -  limiting  of o t h e r distribution,  and the l i k e l i h o o d  i n Chapter 2 and Chapter 3. asymptotic  ratio.  In Chapter 2  optimality  In Chapter 3 weak convergence of Z i s used to f i n d the  d i s t r i b u t i o n of Pearon's  chi-square s t a t i s t i c  f o r goodness of  tests. Let  us examine the asymptotic d i s t r i b u t i o n of the l i k e l i h o o d  i n the e x p o n e n t i a l example i n t r o d u c e d  above.  F o r each n t h e r e i s a  random sample X , X , . . . from the e x p ( l + 9) d i s t r i b u t i o n . N  h y p o t h e s i s H^ ratio  n  N  says that  for H =9 = 0 vs. H j 0  9 = 9Q/Vn f o r t h i s  > n  i s , from  (1.7),  ratio  sample.  The  The l i k e l i h o o d  -  15  0  -  -(6  O  Z ( x ) = (1 +  e  n  and  the  likelihood  ratio  n Z » = IT 1  n Z  statistic  (X^)  expansion  (1  based  on X ? , . . . , X 1 n  6. - V  -9. n e x p ( - ° - E X?). /rT 1  +  of  log(l  + x),  1  log  Z  c a n be  n  n  9 log  Z* = n l o g ( l  + —) /n  9  n  ^ /n  1  9  n  i.e., rate  the  remainder  1//TT)  variance Central  .  Z  Theorem.  Similarly  under  >  n  and  0  +  Q  (  l _  l}  ?  n X  i  1  )  /n"  1  is  Y  vn  _  n  n  deterministic  i  hence  =  j  is  and  a sequence  (l//n)  n Z(X 1  -  1)  converges  to  0  at  of  mean  0,  i.i.d.  d -»-N(0,l)  by  the  1  Therefore  . n - 0 Z^j * N(-yS d  log  n  x  1  , {x^ -  1 random v a r i a b l e s Limit  (  0  " 7=  )>  n  term 0(l//n")  Now u n d e r  9  f  vn  /^T  written  U J  A 0 ^ 1 „ = » ( — - 2TT+ °<-372  1-0  Is  n  /TT  1  n  By a T a y l o r  =  //rT) x 0  6  2 n  2  Bp  under H . Q  (1.10)  - 16 -  {(1 + ~~) /n"  i s a sequence  ,  7  * Zn  -e  =  ( i + e /v^)  0  0  ~  + e //n" (J  -6 (1  + e  1  -  1  Q  expansion,  (1 + 0 //n")  n  1  of i . i . d . mean 0, v a r i a n c e 1 random v a r i a b l e s and thus,  u s i n g the same T a y l o r  log  i )}? . 1 + 9 //n"  <X°  2  /n"  e  -  i  n  E(X,  1  1 + e //n"  1  V'n  )  1 + 8 //n"  Q  2  4Z + o(r--) vn  (1 + 8 0  0 e //n") Q  vn"  n z ( x  1  n  1 1 +  1  d  * /Sn Q  e  - - | + 0 ( — ) * -8  N(0,1) + e  }  0  +  1 +  6  2  Q Q  / ^  2  - -|  2  where N(0,1) stands f o r a random v a r i a b l e having t h a t  distribution.  Therefore  d 9 l o g Z« + N ( / , 2  . e ) under Q  H^.  There i s a c o n n e c t i o n between the l i m i t i n g d i s t r i b u t i o n s of l o g n under  the n u l l and a l t e r n a t i v e hypotheses  with another h y p o t h e s i s  t e s t i n g problem which i s thought of as a " l i m i t i n g problem."  Given  -  X = N(9,l)  the l i k e l i h o o d  ratio  17 -  statistic  H : 9 = 0 vs.  : 6 -  Q  for testing  9  Q  * 0  is  Z(X) = e x p ( 9 X - - | ) . 0  Thus -9  .d „,  2  2  0  l o g Z(X)= N ( - ~ , 9 )  e  Q  (1.11)  under  (1.12)  Q  2  2  log Z(X)= N ( y ^ , 9 ) Q  and  under H ,  hence  log  We w i l l  d * l o g Z(X)  use t h i s  fact  under HQ and under Hj »  f o r evaluating  the asymptotic p r o p e r t i e s of  t e s t s based on Z^ as the sample s i z e n gets l a r g e . i n t e r p r e t a t i o n of (1.13) i s that f a m i l y of d i s t r i b u t i o n s p l a y s N(9,l)  family.  For  We pursue t h i s  (1.13)  n  the parameter  The broadest  9 i n the e x p ( l + 9/ /vi)  the same r o l e ' a s y m p t o t i c a l l y as 9 i n the idea i n Section  1.5.  e v a l u a t i n g the performance of t e s t s of the form { z > K } o r ° n — n  equivalently  n  1  J  {log z " _> C } l e t p " be the e x p ( l + 9 //n) d i s t r i b u t i o n  and PQ be the e x p ( l )  n  distribution.  0  In the n o t a t i o n  introduced i n  -  1.1,  Section  = dP /dP . n  18  -  Now i f the t e s t mentioned p r e v i o u s l y i s t o  n  have asymptotic l e v e l a, i . e . ,  zJJ >_ C ) * a  P (log n  then ( 1 . 1 1 )  and ( 1 . 1 3 )  imply  C  n  that  J'  1  +  7  g-  - Z  9  C  where Z  a  n  as n + »  +  e  Q  Z  a  a  o  - -  as n * -  i s the 100(1 - a) p e r c e n t i l e of the standard normal  distribution.  Note that from ( 1 . 1 1 )  and ( 1 . 1 3 )  l o g Z™ + 0^/2 d —•—g N(0,1)  Now by ( 1 . 1 2 )  and ( 1 . 1 3 )  lim P ( l o g n-»-°° n  i t follows  under HQ (under p ) . n  the asymptotic power of t h i s  z " > C ) = l i m P"( n->-°°  where 0 i s the d i s t r i b u t i o n  that  log  Z - 6 /2 \ — 0 n  C  2  >  "  test i s  -  9 /2 2  Q  " 0  f u n c t i o n of the standard normal  )  (1.14)  - 19  distribution. tests  How  of HQ V S . %  -  does t h i s compare to the asymptotic power of other >  which have asymptotic  n  l e v e l a?  answered by c o n s i d e r i n g the power of l e v e l a t e s t s H  l  :  is  9  =  9  0  b  a  s  e  d  o  n  having d i s t r i b u t i o n N ( 9 , l ) .  x  the power of the t e s t  {log  Z(X) > 9 Z Q  - 9/}  of HQ'9  Now  be  0 vs.  =  1 - <3>(Z a  9) Q  9 /2 2  >^ Z  Q  - 9^}  or  and by the Neyman-Pearson lemma t h i s t e s t  2  a  l o g Z(X) { g  T h i s can  2  has  the g r e a t e s t p o s s i b l e power. From t h i s i t can be shown (see [10]) that the {log  Z^ _> ^ Z n  a  - 9Q/2}  tests  having asymptotic  [10];  the p a r t i c u l a r  will  produce  Chapter  2.  convergence take up t h i s  1.4  has  the g r e a t e s t asymptotic power among a l l  l e v e l a.  result  For t h i s purpose  ratios  i n general i n  i s not i m p o r t a n t .  i n the s e q u e n t i a l t e s t i n g  r a t i o viewed  We  situation in  i t i s necessary to study the  of the l i k e l i h o o d topic  T h i s i s demonstrated  form of the l i k e l i h o o d  a similar  test  functional  as a s t o c h a s t i c p r o c e s s .  We  next.  Functional Convergence of Likelihood Ratios In  S e c t i o n 1.2  were examined and  s e q u e n t i a l procedures  the SPRT was  for testing  simple  seen to be o p t i m a l i n c e r t a i n  q u e s t i o n of asymptotic power a g a i n s t a l t e r n a t i v e s leads to the study of the l i m i t i n g d i s t r i b u t i o n  hypotheses ways.  The  t e n d i n g to the n u l l  of the l i k e l i h o o d  ratio  c o n s i d e r e d as a process w i t h time measured by o b s e r v a t i o n s of the data points.  The  data Xi»X2»... are i . i . d .  o b s e r v a t i o n s from  - 20 -  d i s t r i b u t i o n e i t h e r Pg  o  ?i•  r  The l i k e l i h o o d  ratio  process  {z :k=l,2,...} i s d e f i n e d i n ( 1 . 3 ) . k  In the n o n - s e q u e n t i a l case of the p r e v i o u s s e c t i o n a sequence of a l t e r n a t i v e hypotheses was indexed  by the sample s i z e and the  a l t e r n a t i v e grew c l o s e r to the n u l l as the sample s i z e i n c r e a s e d . a l a r g e r number of o b s e r v a t i o n s s m a l l e r departures h y p o t h e s i s can be d e t e c t e d w i t h equal power.  from  With  the n u l l  I n the s e q u e n t i a l  s i t u a t i o n , to d e t e c t nearby a l t e r n a t i v e s many o b s e r v a t i o n s a r e r e q u i r e d on average and so i t i s r e a s o n a b l e t o approximate the l i k e l i h o o d process by a continuous Based on an i . i . d . or P  n  one continuous  time  ratio  process.  sequence X ,X ,... w i t h d i s t r i b u t i o n e i t h e r P^ n  n  time v e r s i o n of the l i k e l i h o o d  r a t i o process i s  [nt] z (t) n  =  n  z (x ) n  (1.15)  n  i=l  where Z  n  = dP^/dP  p a r t of n t .  n  as d e f i n e d i n (1.1) and [nt] denotes the i n t e g e r  S y m b o l i c a l l y , the o b s e r v a t i o n s X  n  are a s s o c i a t e d with the  time p o i n t s i / n .  Example.  I f X ,X~,... are independent  specifies  9=0  n  while H  l,n  specifies  N(6,1) random v a r i a b l e s and H  0  9 = 9 / VTI then  0*  (1.16)  - 21 -  I t i s known t h a t p r o c e s s e s of t h i s form converge weakly t o a Brownian motion ( e . g . , see C o r o l l a r y 6 of [ 1 6 ] ) ; i n t h i s  w  0  log Z ( t ) + 6 n  0  and  2  B(t) - —  w  t  9  case we have  under H  (1.17)  Q  2  l o g Z ( t ) •»• 9„ B ( t ) . + - ~ t U I n  under H. 1 ,n  (1.18) w  where { B ( t ) : t > 0} i s a standard Brownian motion. takes p l a c e i n the space D([0,°°)) of r i g h t left  The convergence -»•  continuous f u n c t i o n s w i t h  l i m i t s with the Skorohod m e t r i c (see [ 1 ] ) . However, because B ( t )  has continuous sample  paths we can use the a l t e r n a t i v e  formulation  d 9 Z ( - ) ) - f ( 0 B ( . ) - -°r(-)) 2  f(log  n  Q  f o r f u n c t i o n a l s f continuous w i t h r e s p e c t to the sup-norm (uniform) metric  ([1]).  As i n the n o n - f u n c t i o n a l case the l i m i t i n g d i s t r i b u t i o n s of the log-likelihood  under the n u l l and a l t e r n a t i v e hypotheses are the  d i s t r i b u t i o n s of the l o g - l i k e l i h o o d process f o r a " l i m i t i n g h y p o t h e s i s testing  problem."  T h i s f a c t w i l l be used f o r computation of asymptotic  power and the d e r i v a t i o n o f a s y m p t o t i c a l l y o p t i m a l s e q u e n t i a l procedures i n Chapter 2. C o n d i t i o n s which guarantee the weak convergence i n (1.17) and (1.18) f o r g e n e r a l l i k e l i h o o d  ratios  are e x p l o r e d i n [ 1 2 ] .  One r e s u l t  - 22 -  states  that  w Z ( t ) + B(t) - j  log  under H  w , l o g Z ( t ) •*• B ( t ) + j At  and  if  Xt  n  under Hj  n  (1.20)  and o n l y i f  n /(/f (x) - /f (x)) n  and  n J  n  2  dx  (/f (x) - /f (x)) n  n  (1.21)  2  dx •»• 0  (1.22)  as n * » where A ( e ) = { x : | / f ( x ) / f ( x ) - l | > e}. n  n  n  is  (1.19)  0  a Brownian motion with v a r i a n c e X per u n i t  Here  { B ( t ) : t >_ o}  time ( i . e . , Var ( B ( t ) ) =  Xt.) More g e n e r a l processes can a r i s e as the l i m i t ratio  p r o c e s s , i n c l u d i n g processes with independent  increments.  I f we have independent  o b s e r v a t i o n s X , X ,... such that n  limiting clear;  n  i f X ° , X ,... are i . i . d . n  increments  random v a r i a b l e s . independent,  normally  distributed  distributed  (1.21) and (1.22) h o l d then the  process can o n l y be a Brownian motion.  independent  motion.  and i d e n t i c a l l y  of a l o g - l i k e l i h o o d  The reason f o r t h i s i s  then l o g Z ( t ) has s t a t i o n a r y n  because i t i s formed from p a r t i a l sums of i . i . d .  I f the l i m i t i n g p r o c e s s l o g Z ( t ) say, has s t a t i o n a r y ,  normally d i s t r i b u t e d  increments  i t must be a Brownian  T h e l i m i t i n g processes i n (1.19) and (1.20) are Brownian  - 23 -  motions both w i t h v a r i a n c e A p e r u n i t per  1.5  unit  time  time and w i t h d r i f t s -A/2 and A/2  respectively.  Contguity and Convergence of Experiments The  families  concept o f nearness of n u l l and a l t e r n a t i v e hypotheses or of of p r o b a b i l i t y measures i s made p r e c i s e  contiguity  by the n o t i o n s of  and convergence of experiments.  A sequence another sequence  {plj } of p r o b a b i l i t y measures i s s a i d t o be contiguous to 1  {PQ} ( w r i t t e n  lira P Q U  n-n»  has a c l o s e  likelihood  ratio Z  n  1  ) = 0 implies  l i m P ^ ( A ) = 0. n  ->-eo  n  D i s c u s s i o n of c o n t i g u i t y Contiguity  1 1  plj < PQ) i f f o r any sequence of events  and i t s uses can be found i n [12] and [ 2 1 ] .  r e l a t i o n s h i p w i t h weak convergence of the  = dP^/dP^.  In the case that  d i s t r i b u t i o n under the n u l l h y p o t h e s i s c o n t i g u i t y e x i s t e n c e of a l i m i t i n g d i s t r i b u t i o n f o r Z hypothesis  n  Z  n  has a l i m i t i n g  i s equivalent  to the  under the a l t e r n a t i v e  ([12]).  In order t h a t  asymptotic power be non-degenerate  the sequence o f  a l t e r n a t i v e s must be contiguous to the sequence of n u l l hypotheses. Typically  i n the absence of c o n t i g u i t y  arbitrarily  there w i l l  exist tests with  small error p r o b a b i l i t i e s f o r s u f f i c i e n t l y large  sample  - 24 -  sizes.  T h i s was  the case i n S e c t i o n 1.2 where the l i k e l i h o o d r a t i o had  a degenerate l i m i t  because both the n u l l and a l t e r n a t i v e  When a composite h y p o t h e s i s i s s p e c i f i e d be d e s c r i b e d  i n terms of c o n t i g u i t y  or the l i k e l i h o o d r a t i o of two  of p r o b a b i l i t y measures.  sequences  of p r o b a b i l i t i e s at one time i s needed.  A means of comparing more than two  experiments d e s c r i b e s the nearness of f a m i l i e s  probability  An experiment r e f e r s  distributions.  to converge to E ( w r i t t e n  •+• E ) i f f o r every f i n i t e J  of  probability  of experiments E  n the  of  Convergence  to a f a m i l y E = {Pq}  A sequence E  change.  the t e s t i n g problem cannot  sequences  distributions.  d i d not  N  of = {Pg}  i s said  s e t (9, ,...,9 } 1' ' m'  v e c t o r (dPg /dy ,... ,dPg /dy ) converges i n d i s t r i b u t i o n , under 1 m 11  to ( d P  /dy  Q  dP  1 In  11  /dy) under y, where y  Q  m  n  11  m m = E Pg and y = E P . 1 i 1 i  the case of b i n a r y experiments (those that c o n t a i n  distributions)  u ,  Q  two  convergence of experiments c o i n c i d e s w i t h weak  convergence of the l i k e l i h o o d r a t i o . Convergence  of experiments i s the e s s e n t i a l h y p o t h e s i s of the  Hajek-LeCam minimax theorem application  ([17]).  to composite h y p o t h e s i s  T h i s i s one example of i t s testing.  An example of convergence of experiments i s g i v e n by the f a m i l y E  N  = {exp(l + 9//n"):9eR} which has l i m i t i n g experiment E = {N( 9,1) : 9eR}.  T h i s f a c t i s suggested (but not proven) by the one-dimensional weak convergence i n ( 1 . 1 3 ) . An i n t e r e s t i n g way  of t h i n k i n g about convergence of experiments i s  - 25 -  as an e x t e n s i o n (see  o f the l i k e l i h o o d  [6]) says that a l l i n f e r e n c e  based on the l i k e l i h o o d  principle.  The l i k e l i h o o d  about the f a m i l y  principle  {Pg} should  be  function  dP L( 9) = — du fi  9  when there  i s a measure u such t h a t Pg « p  of t h i s might inference  f o r a l l 9.  An  extension  say t h a t when {p^} ->- {Pg} ( i n the sense d e f i n e d  about  when n i s l a r g e .  {PQ} should  be based on the l i k e l i h o o d  above) a l l  f u n c t i o n f o r {Pg}  - 26  -  CHAPTER 2 ASYMPTOTIC OPTIMALITY OF  2.1  Wald's  Criterion  Let X^, either given  P^  SEQUENTIAL PROCEDURES  X^,... be  or P^  and  i.i.d.  random v a r i a b l e s with  l e t the l i k e l i h o o d  common d i s t r i b u t i o n  r a t i o process  {z (t):t n  >^ o}  be  by  [  n  t  ]  d  n  z (t) = n  P  n  ?  — - (x?) l  i-i  dP«  i  dp^  1  with  Z  = — -  d e f i n e d by  (1.1).  As sume t h a t the process  Z  has  the  K asymptotic  behaviour d i s c u s s e d  i n S e c t i o n 1.4,  X  w  log Z (t) •> B(t) ~ J  l o g Z ( t ) •*• B ( t ) ~ j  and  where { B ( t ) : t >_ o} ( i . e . , Var  be  o»  I'! 2  under P ,  t  i s a Brownian Motion with v a r i a n c e  be d e f i n e d u s i n g  H-.tPj? v s . H, :P?. 0 0 1,n 1 of the  the  l i m i t i n g process  T h i s i s an e x t e n s i o n  similar  1  [2.2]  n  shown to be a s y m p t o t i c a l l y o p t i m a l  situation  P  A per u n i t  time  B(t) = At).  A test w i l l will  u n d e r  t  X  w  n  namely  result discussed  and  this  test  when a p p l i e d to  testing  to the s e q u e n t i a l  testing  i n Section  1.3.  - 27 -  As a f i r s t  step we r e c o g n i z e  (2.2) as l i k e l i h o o d r a t i o s . C([0,«0) of the processes respectively  the l i m i t i n g process  L e t PQ and Pi be the d i s t r i b u t i o n s  { B ( t ) : t _> o} and  d  is  t  {B(t) + X t : t 2  n  on  }  and l e t  Z(t)  where PQ  i n (2.1) and  and P^  t  =  P  d P  l t — ^  o,t  are the r e s t r i c t i o n s of PQ and P^ to C ( [ 0 , t ] ) .  It  shown i n [10] that  If HQ r e p r e s e n t s  log Z(t)= B(t) - | t  under P  log Z(t)= B(t) + | t  under P .  (2.3)  Q  (2.4)  PQ and PQ and H^ r e p r e s e n t s P^ and P^ then we have the 1  weak convergence of the  processes  n Z •*• Z w  under H  Q  and under  The S e q u e n t i a l P r o b a b i l i t y R a t i o Test uses the s t o p p i n g  (2.5)  (SPRT) f o r t e s t i n g HQ v s . H  1  rule  T* = i n f {t: Z ( t ) < B or Z ( t ) >_ A}  and d e c i s i o n  .  (2.6)  rule  D* = {Z(T*) > A}.  (2.7)  - 28  It  -  can be shown ( [ 8 ] ) t h a t T* i s f i n i t e under  both hypotheses;  thus  when the event D* does not occur Z(T*) _< B and Hg i s a c c e p t e d . The  SPRT has  in discrete consists  the same o p t i m a l i t y p r o p e r t y i n continuous time as i t does time.  A s e q u e n t i a l procedure f o r t e s t i n g HQ v s .  i n g e n e r a l of a s t o p p i n g r u l e T which  such that  the event  {T <_ t} i s determined  by  takes v a l u e s i n [O, ] 00  {B(S):0 <. s <^ t} and  d e c i s i o n r u l e D which must be such that DflJT _< t} i s determined  a  by  { B ( S ) : 0 < s <_ t } , f o r each te[0,°°].  O p t i m a l i t y P r o p e r t y of Continuous n,Z  n  of  HQ:PO  has a continuous d i s t r i b u t i o n under P . V S . H ^ : P J has  V then (T,D)  D )  smaller error p r o b a b i l i t i e s  (D  }  a n d  must have h i g h e r average  E (T) > E (T*) Q  We  now  Q  prove a r e s u l t  to the d i s c r e t e  (T,D)  than  p  i ° (  C )  V*)  <  D  C  sample numbers  and E ^ T )  (ASN),  !> E ^ T * ) .  ( s t a t e d more p r e c i s e l y below) which  t h i s o p t i m a l i t y p r o p e r t y i s p r e s e r v e d i n the l i m i t  applied  given  will  f o r each  (2.7), i . e . ,  io * p  Assume that  If a sequential test  n  (T*,D*) d e f i n e d by (2.6) and  that  Time SPRT ( [ 8 ] ) :  time s e t t i n g .  T* = i n f { t :  Z ( t ) > A or Z ( t ) < B } . n  when the SPRT i s  C o n s i d e r the procedure  by n  says  (T^, D )  -  and  = {z (T  D  n  n  n  1  29  -  ) > A}. — '  To study the asymptotic p r o p e r t i e s of (T » D ) the f o l l o w i n g results  a r e used  £ d ^ T^ -»• T  •n  ic  under H  ^  Q  &  Z (T ) •*• Z(T ) n  and under H  1  (2.8)  n  under H„ and under H, U 1 ,n  (2.9)  ft ft These  f o l l o w from the f a c t  { z ( t ) : t _> 0 } r e l a t i v e  of  ( 2 . 5 ) holds with respect it  follows  . n  to the sup-norm m e t r i c and the weak convergence to t h i s m e t r i c  immediately that  *  (T ,  that T and Z(T ) a r e continuous f u n c t i o n a l s  (see Section 1 . 4 ) .  the asymptotic e r r o r p r o b a b i l i t i e s of  ft.  ft  D ) are equal to the e r r o r p r o b a b i l i t i e s lim  From ( 2 . 9 )  Pjj (D*)  *  of the SPRT (T , D ) , i . e .  = P (D*)  (2.10)  Q  n>°°  and  lim P  n  (D* ) c  = P.(D* ).  The  same r e s u l t  integrability  lim  n>°° and  1  f o r the average sample  of {T^}; t h i s  numbers r e q u i r e s the u n i f o r m  i s demonstrated  E (T n  U  n  ) = E  U  (T  )  lira E " ( T * ) = E . ( T * ) .  n->-°°  I n  (2.11)  c  I n  1  i n Appendix  1, thus  (2.12)  (2.13)  - 30  The  asymptotic o p t i m a l i t y r e s u l t  Asymptotic O p t i m a l i t y (T^, D^)  i s any  Property  sequential  and  can now  n  l i m P™  be  stated.  of ( T , D ) : n  Assume that P^D)  n  t e s t of  lim P n+°°  -  vs.  > 0.  If  satisfying  (D ) < P (D )  (2.14)  (D°) _< PjCD*)  (2.15)  then lim E ( T n  ) > E (T*)  (2.16)  ) > E.(T*).  (2.17)  0  n-H»  and  lim E (T n  ^ A proof  which has the  from  n  -  1  of t h i s r e s u l t w i l l now  be g i v e n .  the same e r r o r p r o b a b i l i t i e s  as  First  (T , D ). n  n  the e x i s t e n c e  of the  required  SPRT.  We  of Z  we This  f i n d a SPRT i s where  i s needed; i t  n  s t a t e the needed  result  [24]:  Lemma.  B  n  assumption of c o n t i n u i t y of the d i s t r i b u t i o n  implies  and  1  Assume that Z  n  = dP^/dP^ has  are non-negative numbers with such that  probabilities  the a  n  SPRT w i t h s t o p p i n g and  a. .  a continuous d i s t r i b u t i o n . +  <^ 1 there  boundaries A  fl  and  exist B  n  has  A  n  I f <x^ and  error  - 31 -  In must  o r d e r t h a t the lemma a p p l i e s the e r r o r p r o b a b i l i t i e s  satisfy  the c o n s t r a i n t  + a • _< 1.  Since we have assumed  P ( D * ) + P ^ D * ) < 1, (2.14) and (2.15) imply N  c  sequence a f f e c t s generality  Since o n l y the t a i l  (2.16) and 2.17) we can assume without  that t h i s  of the  loss of  i n e q u a l i t y holds f o r a l l n.  Now l e t ( T , D ) be the SPRT determined n  that  that f o r l a r g e n we w i l l  0  Q  have P ( D ) + P " ( D ) < 1 as r e q u i r e d . U n 1 n  of (T » D )  by the Lemma, t h a t i s  N  T' = i n f i t :  Z (t) > A n  or Z ( t ) < B }, n  D» = { z ( T ' ) > A }. n n — n' n  1  By the o p t i m a l i t y p r o p e r t y of ( T ^ , D^) i t must have lower Tn , 'D n '); thus  i t will  p l a c e of (T » ) « n  n  (T^, was  suffice  to show (2.16) and (2.17) w i t h ( Tn*' , Dn ) i n v / v / 1  Because of the i n e q u a l i t i e s  D ^ ) , the sequences {A^} and {B^} must not bounded  above then n>°°  hold  n  = A', l i m B  n  = B'.  be bounded.  P^D' ) 0  0  I f , say,  c o n t r a d i c t s (2.15).  N  A' and B' would be k  (T , D ) i . e . , P Q ( D ' ) _< g ( p  E (T') < E ( T * ) and E (T') < E ( T * ) w i t h Q  {A^}  Now i f one of (2.16), (2.17) d i d not  than the o p t i m a l procedure < P^D* ),  for  assume { A } and {B^} converge,  the SPRT ( T ' , D') w i t h s t o p p i n g boundaries k k  better  (2.14) and (2.15)  P ? ( D ' ) = 0 and t h i s I n  By c o n s i d e r i n g a subsequence i f n e c e s s a r y say l i m A  ASN than  D  )»  strict  - 32 -  i n e q u a l i t y i n one of the l a s t  two i n e q u a l i t i e s .  T h i s c o n t r a d i c t s the  ft ft optionality  2.2  property  of (T , D ) .  Bayes Risk C r i t e r i o n In t h i s  testing  section a different  criterion  f o r comparing  procedures i s used, the Bayes r i s k .  described  i n the f i r s t  sequential  We begin w i t h  paragraph of S e c t i o n 2.1.  the s e t up  For a sequential  of HQ:PQ V S . H^:P^, say ( T , D ) , we d e f i n e the Bayes R i s k , j u s t  test  as i n  ( 1 . 6 ) , by  P ( T , D ; i r ) = ir(P (D) + c E ( T ) ) n  n  n  + (1 - T T ) ( P * ( D ) + cE™(T)), C  where TT i s the p r i o r p r o b a b i l i t y of the d i s t r i b u t i o n being the  (2.22)  PQ and c i s  c o s t per o b s e r v a t i o n . For  a sequential  are as i n the p r e v i o u s  p  r a t i o process We w i l l  +  (1 -  ^(P^D  0  ) + cE^T)),  (2.23)  the l i k e l i h o o d  Z(t). now s o l v e the problem of m i n i m i z i n g  p over a l l c o n t i n u o u s  Our d e r i v a t i o n w i l l mimic the s t r a t e g y used i n  f o r d e r i v i n g the same r e s u l t  theory  where PQ and P^  s e c t i o n , the Bayes R i s k i s  the c o s t per u n i t time of o b s e r v i n g  time s e q u e n t i a l t e s t s .  given  ( T , D ) of HQ:PQ V S . H ^ : P J ,  (T,D;rr) = TT(PQ(D) + C E Q ( T ) )  Here c r e p r e s e n t s  [20]  test  i n d i s c r e t e time; the a p p r o p r i a t e  f o r the continuous time case c o r r e s p o n d i n g i n [20] and a l s o i n [ 9 ] .  The s o l u t i o n w i l l  to S n e l l ' s envelope i s be a p a r t i c u l a r  A l t h o u g h the s o l u t i o n i s d e r i v e d here o n l y f o r the s p e c i a l  SPRT.  case that  - 33 -  PQ and P^ are d i s t r i b u t i o n s will  of Brownian motions the same argument  work f o r more g e n e r a l s i t u a t i o n s .  the g e n e r a l previous  In p a r t i c u l a r  c o n d i t i o n s of [8] which are used  o p t i m a l i t y property  of continuous  i t w i l l work under  there f o r o b t a i n i n g the  time SPRTs g i v e n  i n Section  2.1. To  begin  i twill  be necessary  to c o n s i d e r the e q u i v a l e n t problem of  minimizing  p  (T,D;IT) = T T ( P ( D ) + c E ( T ) ) + r ( l - T T ) ( P ( D ) +  ( r )  allowing  0  1  the new parameter r to v a r y .  a stopping  Lemma.  r u l e and f i n d i n g  The f i r s t  step c o n s i s t s of f i x i n g  the best d e c i s i o n r u l e  0  Q  the minimum  Proof: for By  First  q  i s achieved  at D  A  r ( l - TT) Z ( T ) )  = {Z(T) > T r / ( l - TT) }. r  we note that D O { T <_ t} i s determined by {B(S):0 <_ s £ t } ,  a l l t , and Z(T) equals  dP]/dPQ on the o - f i e l d  of such  events.  the Lebesque Decomposition,  TT P ( D ) Q  It  to go w i t h i t .  I f T i s fixed minfir P ( D ) + r( 1 - TT) P ^ D ) ] = E ( T T A  and  cE^T)),  C  q  +  r(l -  TT) Vl(D c)  =  /  >  / [ TT  D  TT d P  Q  +  /  c  r(l -  TT) Z(T)  dP  Q  A r ( l - IT) Z ( T ) ] d P .  i s s t r a i g h t f o r w a r d to check t h a t t h e r e  Q  i s e q u a l i t y here f o r D = D*.  - 34  The  problem i s now  TT c E Q ( T ) +  = E (TT C T +  E (Y  ( r )  Q  ( r )  {Y  (t)  According  TT) c E ^ T )  TT) C T Z(T)  ( r )  = TT  (t):t  c  >_ 0}  the l a r g e s t  dominated b y { Y ^ ^ ( t ) : t _> o} r  process  v( ) r  q  TT A  +  r(l -  r(l -  TT)  i s given V  ( r )  i s defined  TT)  Z(T))  Z(T))  by  i n [20]  and  ( r )  then forming  (t) = V  ( r )  the s t o p p i n g  ( r )  ( r )  be  (t):t  >  0}  rule  (2.24)  ( T ) | B ( S ) : 0 < s < t)  (2.25)  over a l l s t o p p i n g r u l e s T which s a t i s f y T _> t . r  ( r )  {V  (t)}.  S i n c e Z(0) = 1, the i n i t i a l v a l u e V ^ ) ( 0 )  V  say  can  by  ( t ) = essinf E ( Y  where the e s s i n f i s taken  or Theorem 4 i n [9] t h i s  p o s i t i v e sub-martingale,  T* = i n f { t : Y The  + E (TT A  t + r ( l - TT) c t Z ( t ) + TT A r ( l - TT) Z ( t ) .  to Theorem 7.3  done by f i n d i n g  to m i n i m i z i n g  (T))  where the process  Y  r(l -  r(l -  0  =  reduced  -  ( 0 ) = inf E ( Y  ( r )  (T))  i s d e t e r m i n i s t i c and  =h(r).  (2.26)  Note t h a t h i s an i n c r e a s i n g concave f u n c t i o n because i t i s the infimura of such nature  functions. of the  For any  This fact w i l l  be important  solution. stopping r u l e T s a t i s f y i n g T ^  t  f o r determining  the  -  E(Y  ( R )  (T)|B(s):0  < s <  35  t) = E ( T T ( T -  -  t) + r ( ( l - i r ) Z ( t )  c  c(T -  t)  + TT A r ( 1 — TT) Z ( t ) | ^ - | B ( S ) : 0 < s < t ) +  where we  have used the f a c t  TT t  + r ( l - TT) c t Z ( t )  c  (2.27)  t h a t E ( Z ( T ) | B ( S ) :0 <, s _< t ) = Z ( t ) ( i . e . , B(t)-|t  is a martingale).  Using  the r e p r e s e n t a t i o n Z ( t ) = e  zlul  z(t)  and  s i n c e J B ( t ) } has  [|ftT  :u  B ( u )  expectation  B ( t )  "  T  ( u  "  t }  s t a t i o n a r y independent increments,  >_ t} i s independent of  distribution  "  e  as the process  (B(S)  :0 < s <_ t} and  {Z(u) :u >^ 0  }.  Therefore  i n (2.27) i s minimized e x a c t l y as f o r the  the  has  process  the same  the c o n d i t i o n a l case t = 0  in  (2.26) but with r r e p l a c e d by r Z ( t ) , i . e . ,  V  ( r )  (t)  = essinf E ( Y  ( r )  (T) | B ( s ) :s < t ) ) = h ( r Z ( t ) ) + Tr t c  + r ( l - TT) c t Z ( t ) .  ft Now  the s t o p p i n g  T* = i n f { t : Y  In order i s now  ( r )  (t) = V  rule T  ( r )  i s given  ( t ) } = i n f {t : h ( r Z ( t ) ) = TT A r ( l - TT) Z ( t ) }.  f o r the Bayes R i s k g i v e n  set to 1.  Thus  by  i n (2.23) to be minimized by T*,  r  Z  = i n f { t : h ( Z ( t ) ) = TT A (1 — IT) z ( t ) } ,  T  Since h i s i n c r e a s i n g and concave, T* has the form  T* = i n f { t : Z ( t ) > A or Z ( t ) < B } .  f o r constants A and B i l l u s t r a t e d  below.  TTA(1-H)X  /  j/hCx)  !  x B Fig.  If T r  and  *  TT/( 1-IT)  1.  A  Graph of h which determines of o p t i m a l SPRT.  s t o p p i n g boundaries  i s the stopping r u l e employed, the d e c i s i o n r u l e s  A, B  {Z(T ) > A}  IT 1  |Z(T- ) ^> y-—J- ( r e c a l l the lemma, pg. 33) are e q u i v a l e n t  i n e q u a l i t y B <_  ^ <_ A.  due to the  A l s o , the cases B >^ 1 and A <^ l c o r r e s p o n d t o  T* = 0 i n which the i n i t i a l  d e c i s i o n based only on the p r i o r  p r o b a b i l i t y i s o p t i m a l , having Bayes R i s k TTAO -  •  - 37 -  As i n the p r e v i o u s s e c t i o n the o p t i m a l procedure time problem w i l l be a p p l i e d t o the d i s c r e t e time which minimizes  the asymptotic  Bayes Risk r e s u l t s .  f o r the continuous  setting; a  procedure  D e f i n e the s t o p p i n g  rule  T* = i n f { t : Z ( t ) > A o r Z ( t ) <^ B} n  n  where A and B are the s t o p p i n g boundaries  of the SPRT which  minimizes  the Bayes R i s k (2.23) and the d e c i s i o n r u l e  D* = {z (T*) > A}. n  n  *  n  —  '  *  Thus (T > D^) has the asymptotic (2.17).  1  Here i t w i l l  o p t i m a l i t y p r o p e r t y g i v e n by (2.14) -  be shown to have the f o l l o w i n g p r o p e r t y .  ft ft Asymptotic  Bayes O p t i m a l i t y P r o p e r t y of ( T , D ):The asymptotic n  n  Bayes  R i s k of ( T * D*) i s n' n  l i m p (T*. D*; TT) = p(T*, D*; TT) . n n n n*°°  If  (T^, D ) n  (2.28)  i s any s e q u e n t i a l t e s t of HQ V S . H^ then  lim i n f p ( T , D; n) > p(T*, D*; n ) . n+<» (T,D)  (2.29)  n  ft ft This w i l l  say t h a t ( T > D ) has the s m a l l e s t p o s s i b l e  asymptotic  Bayes R i s k and the v a l u e i s the Bayes Risk of the procedure  (T , D ) .  n  ft ft  - 38 -  The proof of (2.28) i s achieved by the a p p l i c a t i o n of (2.10), (2.11), (2.12) and (2.13) which s t a t e t h a t a l l of the components * * Bayes R i s k P ( T , D^; TT) converge to the c o r r e s p o n d i n g n  components  n  of the of  ft ft p(T  , D ; TT). The f i r s t  step i n p r o v i n g (2.29) i s to compute the minimum v a l u e of  ft p .  From the d i s c u s s i o n p r e c e d i n g  that  p  n  n  the d e r i v a t i o n of T  i t i s known  i s minimized by a SPRT with some s t o p p i n g boundaries,  say A  n  and B , that i s n  inf ~ T,D m  p  n  (T, D; TT) =  p  n  (T , D ; TT) n ' n' '  where  T  = inf{t:Z (t) > A n  n  n  or Z ( t ) < B } n  n  and  D  = {zn(T ) > A }. n — ' n  n  Assume now that along a subsequence of the i n t e g e r s {n'} the l i m i t l i m P , ( T , , D ,; ir) n' n  e x i s t s and i s l e s s  than p(T , D ; Tr) •  a f u r t h e r subsequence lim A n'  n'  n  (also called  n  W i t h i n t h i s subsequence there i s  {n' }) such that the l i m i t s  n' = A' and l i m B = B' e x i s t , p o s s i b l y i n f i n i t e . n'  F i n a l l y we  can  - 39 -  repeat  the argument at the end of the p r e v i o u s s e c t i o n , to show t h a t the  continuous  time  SPRT with s t o p p i n g boundaries  R i s k than the SPRT which uses A and B. fact  A' and B' has lower  T h i s of course  Bayes  c o n t r a d i c t s the  t h a t A and B were d e r i v e d to minimize the Bayes R i s k  (2.23).  - 40  -  CHAPTER 3 POWER OF CHI-SQUARE TESTS  The  focus  chi-square  of t h i s s e c t i o n  statistic  of a l t e r n a t i v e s . hypothesis We  for testing  i n the sense d e f i n e d  will  power of Pearson's  goodness of f i t a g a i n s t a c e r t a i n  These a l t e r n a t i v e s  are contiguous  i n Section  reproduce a d e r i v a t i o n  Pearson's c h i - s q u a r e In  i s the asymptotic  to the  of the l i m i t i n g d i s t r i b u t i o n  s t a t i s t i c under the n u l l h y p o t h e s i s  the  likelihood  ratio.  alternative  parameter-must be A l s o we and that  This highlights  distribution  the u s e f u l n e s s testing  compare the asymptotic  the t e s t based on the l i k e l i h o o d  In [18]  the  where a  power of the c h i - s q u a r e  ratio.  We  test  know from S e c t i o n  r a t i o must win;  the extent  of  interest.  L e t N = (Ni,...,Nk) be a m u l t i n o m i a l records  problems.  likelihood  estimated.  will  i s of  w i l l give a  of the  i s found f o r s i t u a t i o n s  of a t e s t based on the l i k e l i h o o d  difference  We  is  r e s u l t which uses the weak convergence of  r a t i o as a t o o l f o r s t u d y i n g h y p o t h e s i s limiting  of  ( [ 7 ] , [19]).  of a l t e r n a t i v e s  computed, whereby the asympotic power can be computed. development of t h i s  null  (1.4).  [5] the l i m i t i n g d i s t r i b u t i o n under a c l a s s  different  clas  random v e c t o r which  the numbers of data p o i n t s which f a l l  classifications.  into  each of k  Let the t o t a l number of data p o i n t s be n and  probability  of any  one  falling  probability  f u n c t i o n of _N i s  into  the i t h c a t e g o r y  be P^.  the The  1.3 the  - 41 -  , = n. !. .n. ! 1 k  P(N =n ,...,N =n ) 1  1  k  k  k n. HP^ 1  (  n  k e Z , I n =n) 1  A common q u e s t i o n asks whether the p r o b a b i l i t y P_ = ( P j , . . . , P )  belongs to a p a r a m e t r i c  k  question usually  family  vector  (P( 9): 9 E H } .  This  r e f l e c t s on the d i s t r i b u t i o n of the u n d e r l y i n g data which i s the source  sample X^,...,X  n  a  of i n t e r e s t .  For example when t e s t i n g whether a  came from the Normal d i s t r i b u t i o n , c a t e g o r i e s  E^ = (a^_^> i _ l - ' . (^  l , . . . , k ) c o u l d be formed and the numbers  =  N^ = #{Xj e E^} of o b s e r v a t i o n s  falling  into  under the normal d i s t r i b u t i o n the p r o b a b i l i t y  P  i =  L]\ 1  -  .The h y p o t h e s i s particular  of n o r m a l i t y  parametric  recorded.  v e c t o r P would be g i v e n by  d  x  a. , - u  for Xi,...,X  i s also  n  specified  by the  form f o r P^. hypothesis  :_P = _P( 9) f o r some 6 e H  0  e s t i m a t i o n of 9.  c o n s i d e r o n l y the simple  v  intervals  /2T7O"  In g e n e r a l a t e s t of the composite  H  these  J L - e - ^ W 1  a. - u  requires  (3.1)  +  p  This s i t u a t i o n hypothesis  = p  (  v  i s treated i n [18].  We  will  - 42 -  for  some s p e c i f i e d  6Q e H.  H  0  :  i  P  =  T h i s i s a l s o w r i t t e n as  l  P  (i=l,-..,k)  (3.2)  where P ( 8 ) = ( p j ,... ,P°_) . Q  Pearson's c h i - s q u a r e s t a t i s t i c  X (n) Z  f o r t e s t i n g HQ i s  k (N - n P ? ) = E — i 1-1 nP.  2  (3.3)  l  It  will  be shown that X ( n ) has a l i m i t i n g 2  d i s t r i b u t i o n w i t h k-1 degrees of freedom. computing  critical  v a l u e s of the t e s t .  (n •*• °°) c h i - s q u a r e This fact  The proof i s based  m u l t i v a r i a t e C e n t r a l L i m i t Theorem ( [ 7 ] ) a p p l i e d random v e c t o r s V  n  i s used f o r on a s i m p l e  to the sequence of  g i v e n by  V  n,i  N - nP° = - i i .  (3.4)  —?r  /nP°  A simple computation produces  the c o v a r i a n c e m a t r i x of  CovO^) =  where _£ = (/P^ ,...,Vp^)'. identically  distributed  I  k  - i i '  Because the  are sums of independent  ( B e r n o u l l i ) random v a r i a b l e s  the m u l t i v a r i a t e  - 43  CLT  can be a p p l i e d  -  to y i e l d  d V  as n •*•  00  where A =  n  \(0,  A) under H  Ik ~ R 3.' •  ^  n  v  X (n)  Proposition  ^  e w  r e s u l t i s needed ( [ 7 ] ) ,  1.  I f _Y = N ( 0 ,  An a p p l i c a t i o n  A) and  of t h i s to Vn  l  °f  t  n  e  relation  = V V —n —n  2  the f o l l o w i n g  (3.5)  0  (  [19]).  A i s idempotent  (recall  w i t h rank r then  (3.5)) g i v e s  (3.6)  k-l  since  the c o v a r i a n c e m a t r i x A = 1^ - q q' of V_n i s idempotent  k-1.  Here we have used The  s t a t i s t i c X (n)  alternatives  2  the c o n t i n u i t y  rank  X.  i s not designed with any s p e c i f i c  to HQ i n mind.  the sequence of  of the mapping X •*• X_'  with  The  asymptotic  power of X ( n ) 2  against  alternatives  H.  (3.7)  - 44 -  k where E C 1  =0,  can be c a l c u l a t e d .  The l i m i t i n g d i s t r i b u t i o n of X (n)  under the sequence of a l t e r n a t i v e s H I stated  i n the f o l l o w i n g  distribution  of ^  + Z  2  , i s non-central  A The n o t i o n  result.  + v £ )  n  2  2 x' (^0 r  c h i - s q u a r e as  represents  the  2 + .. . + Z where Z . . . , Z are i . i . d . r 1' r 1  t  s t a n d a r d normal random v a r i a b l e s .  Theorem d  2  X (n) Z  *  xl K  _  1  ,2 k C , ( I -4) as n 1 P.  (3.8)  -v  l  One p o s s i b l e proof (as i n [18]) uses a m u l t i v a r i a t e t r i a n g u l a r arrays  which  CLT f o r  establishes  d  "*" K ^ » N  A  )  under  as n •»-  0 0  n  (3.9)  w i t h A as i n (3.5) and l j5 = ( — C  /p° 1  , •. •,  °k  )  /P? k  T h i s must be combined w i t h the f o l l o w i n g normal d i s t r i b u t i o n which g e n e r a l i z e s  f a c t about the m u l t i v a r i a t e  Proposition  1.  - 45  P r o p o s i t i o n 2.  ([17])  -  I f Y = N (j5, A) and k  A i s idempotent w i t h rank r  and _6 i s i n the range (column space) of A then  I ' I = xj. (± ±) . f  2  Using (3.9).  this result  A different  on the l i k e l i h o o d v s . Hi_ .  i t i s immediate that the Theorem f o l l o w s from  proof of (3.9) w i l l now  r a t i o f o r the simple h y p o t h e s i s  This l i k e l i h o o d  n  probabilities  be g i v e n ; i t w i l l be  d e f i n e d by  testing  r a t i o i s simply a r a t i o of  based  problem HQ  multinomial  ( 3 . 1 ) , namely  k  N  TT (P° + C /Sn)  ±  ±  Z  l  l  =  k — I T  i  (3.10)  i In order t h a t Z V  n  n  can be used to f i n d the l i m i t i n g d i s t r i b u t i o n  t h e r e must be e s t a b l i s h e d a r e l a t i o n s h i p  between the two.  done by t a k i n g l o g a r i t h m s and u s i n g a T a y l o r expansion  log  Z  n  k = E N. 1  log(l +  1  C -~--) /rT 2  p  0  c  / n  2(P ) 4  n  c  2  — Z - i N, - - E * , N. + 0_,(—) /— „ 0 i n .,0.2 l P/ /n P 2(? ) /n -  i  of  This i s  as f o l l o w s :  - 46 -  /nP ; 1  /n  i = I —  P  0  n,i  v  1  C  C  v  n  i  - f l-f- + 0 ( 1 )  V 1  1  ,  1  Pj  2  ?  (3.11)  where 0 ( — ) i s a term which converges to 0 i n p r o b a b i l i t y /n p  and 0^(1) converges to 0 i n p r o b a b i l i t y . N /n ±  - PJ -  at r a t e l / / n ,  The l a t t e r term a r i s e s  because  0 (l). p  Now we use the f o l l o w i n g s t r a t e g y .  There i s a " l i m i t i n g " s i m p l e  h y p o t h e s i s t e s t i n g problem which approximates the m u l t i n o m i a l t e s t i n g problem HQ v s . Hj  n  i n such a way that  assumes the r o l e of and under a l t e r n a t i v e ,  there i s a q u a n t i t y which  The d i s t r i b u t i o n of t h i s q u a n t i t y , under n u l l i s the l i m i t i n g d i s t r i b u t i o n of  and under H^ , r e s p e c t i v e l y . n  under HQ  The l i m i t i n g problem i s  H : N ( 0 , A) v s . Hj :\(_6, A). Q  k  The l i k e l i h o o d r a t i o based on a s i n g l e o b s e r v a t i o n X i s (see Appendix A.2)  Z(X) = exp(^' X - - i _ 6 ' _S) .  Comparing  the e q u a t i o n  - 47 -  Z(X) = _6' X - j  log  w i t h (3.11) i t i s seen that  (Z ,  j5« j5  (3.12)  are r e l a t e d by the same  n  e q u a t i o n as ( Z ( x ) , x) > except f o r the e r r o r term O p ( l ) .  _ d Z •*• logZ  log  as n > ».  under H  The l i m i t i n g d i s t r i b u t i o n  of l o g Z  Also  (3.13)  Q  n  i s calculated  from  (3.11) and (3.5) as  log Z  n  d -»• 6' N ( 0 , A) - - i _6' _6  under  k  (3.14)  ( s i n c e the v a r i a n c e of 6/ N (Cj, A) i s _6' A _6 = _6' ( l k  the  Q  N(- I _6' j5, _6' _6)  i  The d i s t r i b u t i o n  H  f c  - ^ q^' ) _6 = _6' _6.)  of l o g Z under HQ i s e a s i l y seen from (3.12) t o be  same. F i n a l l y , we show that  Two lemmas are r e q u i r e d ;  Lemma 1. likelihood  (3.9) f o l l o w s from ( 3 . 5 ) , (3.11) and ( 3 . 1 2 ) .  t h e i r p r o o f s are found i n the Appendix A.3.  L e t Z be a l i k e l i h o o d ratios.  r a t i o and {z } a sequence of n  I f a sequence of s t a t i s t i c s X  d  (x , z ) + (X, Z) n  n  n  satisfies  - 48 -  under the n u l l h y p o t h e s i s then  X  under the a l t e r n a t i v e . alternative  n  d * X  Note that the d i s t r i b u t i o n of X under t h e  i s not the same as the d i s t r i b u t i o n of X under the n u l l  hypothesis.  Lemma 2.  Let X  n  and Y  n  be sequences of random q u a n t i t i e s which  converge i n d i s t r i b u t i o n say  X  n  d •> X, Y  n  d * Y.  I f t h e r e i s a continuous f u n c t i o n H and random q u a n t i t i e s  e  n  such that  Y = H(X), Y and  = H(X ) + e  n  e  n  n  •*• 0 i n p r o b a b i l i t y  n  then  (X , n  d Y ) + (X, Y ) . n  Lemma 1 reduces the proof of (3.9) to showing j o i n t convergence of and Z  n  or e q u i v a l e n t l y of  and ( 3 . 1 1 ) .  Since V  and l o g Z ; t h i s f o l l o w s from Lemma 2 n  > X where X = N, (0, A) under the n u l l and  -  X = N^—' limiting  A) under the a l t e r n a t i v e  49 -  i t f o l l o w s that the l a t t e r i s the  d i s t r i b u t i o n of V J J .  alternative  F i n a l l y we t u r n to a comparison of the t e s t the  test The  n  comparison w i l l  under H  log  Z  n  be done v i a the l i m i t i n g d i s t r i b u t i o n s  Denoting _6' _6 by A the l i m i t i n g d i s t r i b u t i o n s 2  and x^.  of the  of X ( n ) are 2  (A) under H , and the l i m i t i n g d i s t r i b u t i o n s  are N(- —A , A.)  under  and N(yA  power of the t e s t based on l o g Z by A  2  based on Z .  statistics. 2 X,  based on X ( n ) w i t h  n  ,A ) under  .  The asymptotic  i s g i v e n i n (1.14); r e p l a c i n g  9  2  (3.15)  a  where Z  a  i s the l O O ( l - a ) p e r c e n t i l e  the standard normal Now  for  there  the asymptotic power i s  1 - $(Z -  of  of  for levels  and $ i s the d i s t r i b u t i o n  function  distribution.  a = .05 and a = .01 we f i n d the v a l u e s of  the asymptotic power of the X ( n ) t e s t  required  to be .85, .90 and .95.  2  These values of A s o l v e the equations  P  2 where X,  (  X  k - l  (  A  )  y  Xfc-l.a*  =  i s the 100(l-a) p e r c e n t i l e  *  8 5  '  , 9 0  '  *  2 of the X,  9 5  -  d i s t r i b u t i o n and  they can be read from T a b l e 25 of [ 2 ] , From A the power i n (3.15) i s computed and t h i s i s then compared to the r e l e v a n t power f o r the  - 50  chi-square t e s t . v a l u e s of k-1  Table  1.  -  In Table 2 below the r e s u l t s are d i s p l a y e d f o r v a r i o u s  the degrees of freedom f o r the c h i - s q u a r e  Asymptotic  power 1  - Hz  -  ^20  of the t e s t based  a f o r v a l u e s of s i z e a, power (3 and of the c h i --square t e s t .  a =  statistic.  on  z  n  degrees of freedom k-1  a =  .05  .01  k-l\S  .85  .90  .95  .85  .90  .95  1 2 3 4 5 6 7 8 9 10 15  .911 .952 .969 .978 .984 .988 .991 .993 .994 .995 .998  .945 .972 .983 .989 .992 .994 .996 .997 .997 .998 .999  .975 .989 .994 .996 .997 .998 .999 .999 .999 1.000 1.000  .901 .945 .968 .976 .982 .987 .990 .992 .994 .995 .998  .937 .968 .980 .987 .991 .994 .995 .996 .997 .998 .999  .971 .987 .993 .995 .997 .998 .999 .999 .999 .999 1.000  Note that the power seems to converge to 1 as k gets l a r g e ; t h i s i s a l s o expressed  by the f a c t  t h a t the n o n - c e n t r a l i t y parameter &  must  increase with  the degrees of freedom i n order that the c h i - s q u a r e  have constant  power.  s t a t i s t i c X (n) 2  has  For a l a r g e number of c e l l s k the greater d i f f i c u l t y  test  chi-square  in detecting a particular  a l t e r n a t i v e because i t attempts to d e t e c t a l t e r n a t i v e s i n many d i r e c tions.  I t should be mentioned that under a l t e r n a t i v e s other than that  specified  by  ( i . e . , P^ + C^/Zn), Z  n  may  have s m a l l e r asymptotic  power  2 than X ( n ) .  Thus a t r a d e - o f f e x i s t s between i n c r e a s e d power from u s i n g  the l i k e l i h o o d  r a t i o and  the r i s k of u s i n g the wrong l i k e l i h o o d  ratio.  - 51  -  BIBLIOGRAPHY  1.  B i l l i n g s l e y , P. (1968). W i l e y , New York.  Convergence  2.  B i o m e t r i k a T a b l e s f o r S t a t i s t i c i a n s , V o l . I I . (1972). Pearson and H.0. H a r t l e y . Cambridge U n i v e r s i t y P r e s s .  3.  C h e r n o f f , H. (1972). S e q u e n t i a l A n a l y s i s and Optimal D e s i g n . S.I.A.M., P h i l a d e l p h i a .  4.  C h e r n o f f , H. and Petkau, A . J . (1981). " S e q u e n t i a l m e d i c a l i n v o l v i n g p a i r e d d a t a . " B i o m e t r i k a , 68, 1, 119-132.  5.  Cochran, W.G. (1952). "The Math. S t a t . , 23, 315-345.  X  6.  Cox, D.R. and H i n k l e y , D.V. Chapman and H a l l , London.  (1974).  7.  Cramer, H. (1946). University Press.  8.  D v o r e t z k y , A., K i e f e r , J . and W o l f o w i t z , J . (1953). d e c i s i o n problems f o r p r o c e s s e s w i t h continuous time T e s t i n g hypotheses." Ann. Math. S t a t . , 24, 254-264.  9.  Fakeev, A.G. (1970). "Optimal s t o p p i n g r u l e s f o r s t o c h a s t i c p r o c e s s e s w i t h continuous parameter." Thy. Prob. A p p l . V o l . 15, No. 1, 324-331.  2  of P r o b a b i l i t y  Measures.  Eds.:  t e s t of goodness of f i t . "  Theoretical  E.S.  trials  Ann.  Statistics.  Mathematical Methods of S t a t i s t i c s .  Princeton  "Sequential parameter.  10.  Freedman, D. (1971). San F r a n c i s c o .  11.  Ghosh, B.K. (1970). S e q u e n t i a l T e s t s of S t a t i s t i c a l Addison-Wesley, Reading, Mass.  12.  Greenwood, P.E. and S h i r y a y e v , A.N. (1985). C o n t i g u i t y and the S t a t i s t i c a l Invariance P r i n c i p l e . Gordon and Breach, New York.  13.  L a i , T.L., Siegmund, D. and Robbins, H. (1983). " S e q u e n t i a l d e s i g n of comparative c l i n i c a l t r i a l s . " In Recent Advances i n S t a t i s t i c s , Eds.: M.H. R i s v i , J.S. R u s t a g i , D. Siegmund, 51-68. Academic P r e s s , New York.  14.  Lamperti, J .  15.  Lehmann, E.L. York.  (1966). (1959).  Brownian Motion and D i f f u s i o n .  Probability.  Holden-Day,  Hypotheses.  Benjamin/Cummings,  T e s t i n g S t a t i s t i c a l Hypotheses.  Reading. Wiley,  New  - 52  16.  L i p c e r , R. and S h i r y a y e v , A.N. (1980). "A f u n c t i o n a l c e n t r a l l i m i t theorem f o r s e m i m a r t i n g a l e s . " Thy. Prob. A p p l . V o l . 25, No. 4, 667-688.  17.  M i l l a r , P.W. (1983). The Minimax P r i n c i p l e S t a t i s t i c a l Theory. Unpublished n o t e s .  18.  M i t r a , S.K. (1958). "On the l i m i t i n g power f u n c t i o n of the frequency c h i - s q u a r e t e s t . " Ann. Math. S t a t . , 29, 1221-1233.  19.  Moore, D.S. (1983). "Chi-square t e s t s . " S t u d i e s i n Mathematics, V o l . 19: S t u d i e s i n S t a t i s t i c s , 66-106, Ed.: R.V. Hogg, Mathematical A s s o c i a t i o n of America.  20.  Neveu, J .  21.  Roussas, G. (1972). C o n t i g u i t y of P r o b a b i l i t y Measures: Applications i n S t a t i s t i c s . Cambridge U n i v e r s i t y P r e s s .  22.  Thompson, M.E. (1971). "Continuous parameter o p t i m a l s t o p p i n g problems." Z. W a h r s c h e i n l i c h k e i t s t h e o r i e , 19, 302-318.  23.  Wald, A. and W o l f o w i t z , J . (1948). sequential probability ratio test."  24.  Wijsman, R.A. (1963). " E x i s t e n c e , uniqueness and m o n o t o n i c i t y o f sequential probability ratio tests." Ann. Math. S t a t . , 34, 1541-1548.  (1974).  i n Asymptotic  D i s c r e t e Parameter M a r t i n g a l e s . HoMcn-Pay, Sa«Francisco. Some  "Optimum c h a r a c t e r of the Ann. Math. S t a t . , 19, 326-339.  - 53  -  APPENDIX  A.l  Uniform I n t e g r a b i l l t y of a Sequence of Stopping Times The  used  convergence of Average Sample Numbers ( ( 2 . 1 2 ) , (2.13)) which i s  i n Sections  2.1  and  2.2  sequence of s t o p p i n g times  {T }  will  paragraph The of  e s t a b l i s h now  of Chapter  will  distribution  Z ( t ) >_ A or Z ( t ) n  n  <  and  {P^}.  In doing  this  B}.  first  f u n c t i o n of T  t be an i n t e g e r .  Let F  * n under P , n  (t) = P (T* < t ) . n — n  n  Then  1 - F ( t ) = P ( T * > t) n n n  -  P (B N  < Z (s) < A for a l l s < t) n  = P ( l o g B < l o g Z ( s ) < logA f o r a l l s < t ) n  =  P (logB < n  sequences  f o r both sequences at  denote e i t h e r of the sequences.  N  the  2.  {Pg}  let P  of  by  u s i n g the set-up d e s c r i b e d i n the  F  Let  integrability  u n i f o r m i n t e g r a b i l i t y must be e s t a b l i s h e d under both  probabilities  once we  given  n  T* = i n f {t:  T h i s we  r e q u i r e s the uniform  n  logz£ < logA f o r a l l k <^ tn)  n  be  the  - 54 -  < P (logB  < logZ  n  logB < l o g Z  < logA,  n  n n  n  where C = JlogAJ +  |logB .  logZ - logZ? ° tn (t-l)n  logZ  n n  <  logA,...,  < logA)  < P n ( | l o g z £ | < C, | l o g Z  |logZj -  logB < l o g Z  n t  _  1 ) n  |  n n  -  logZ | n  < C,...,  < C)  Since l o g Z ^ , i  o  g  z"  n  - log",...,  are i . i . d .  n  &  1 - F ( t )< [P(n)] n —  fc  with 'P(n) = P ( | l o g Z ^ | n  < C).  Now s i n c e logZ^j = l o g Z ( l ) 3- l o g Z ( l ) we have n  p(n)  and  •»• P( | l o g Z ( l ) | < C) < 1  thus we can assume without  l o s s of g e n e r a l i t y that  P(n) < Y < 1  f o r every n  T h e r e f o r e 1 - F ( t ) <. Y*" f o r i n t e g e r s t , so f o r any t n  1 - F (t) < 1 - F ([t]) < Y n  n  [ t ]  < Y  t _ 1  (Al.l)  h o l d s f o r every n.  Now by i n t e g r a t i o n by p a r t s  2 / Q C I - F ( t ) ) t d t = (1 - F ( t ) ) t / ^ + J Q t 2  n  "  using  the i n e q u a l i t y  2  n  (Al.l).  E (T*) n n  'O  t  2  d F  dF (t) n  n^>  Therefore  2  = 2/" (1 - F ( t ) ) t d t 0 n  < 2 J ^ t  Y  t _ 1  dt < ~ .  I t now f o l l o w s t h a t {^ ) i s u n i f o r m l y i n t e g r a b l e . n  A.2  The L i k e l i h o o d R a t i o  It  of  i s r e q u i r e d to f i n d  N (_6, A) and N (Cj, A ) . k  k  Singular  Multivariate  the l i k e l i h o o d  Normal  Distributions  r a t i o of the d i s t r i b u t i o n s  Consider the r e p r e s e n t a t i o n ( [ 1 9 ] )  A  1 / 2  Z  +  _6  f o r the N (j5, A) d i s t r i b u t i o n , where Z i s a v e c t o r of i . i . d . k  N(0,1)  1/2 v a r i a b l e s and A  i s the square-root  1/2 satisfying A  matrix  1/2 A  = A.  1/2 so t h a t A  of A, a symmetric  = A; a l s o  I n our s i t u a t i o n  A i s idempotent w i t h rank r  -  A = B  P |o.  56 -  B' = B D B'  (A2.1)  w i t h B an o r t h o g o n a l m a t r i x , a r e p r e s e n t a t i o n which w i l l Now  be used below.  i f J> i s i n the range (column space) of A the v e c t o r A Z_ + _6 w i l l  remain i n the range of A and the two d i s t r i b u t i o n s N^(_6, A) and N (0_, A) k  will  have the same support.  meaningful. so  the  In Chapter 3 we had A = 1^  t h a t A _6 = _6 Now  l e t QQ,  respectively.  k  k  corresponding to With D as i n (A2.1)  (Cj, D) => BX = N ( 0 , A)  (A2.2)  k  l e t P Q , P^ be the p r o b a b i l i t y measures c o r r e s p o n d i n g to N (0_, D) , k  N (_u_, D). k  ratio  <L <J.' and _6 was o r t h o g o n a l to q  be the p r o b a b i l i t y measures on R  N ( 0 , A ) , N (_6, A) d i s t r i b u t i o n s k  -  likelihood  thus e n s u r i n g t h a t _6 i s i n the range of A.  X = N  Now  T h i s w i l l make t h e i r  From (A2.2) and (A2.3), Q  Q  = PQB"  1  and Q  = P^"  1  where the  n o t a t i o n means  Q (A) 0  for  =  PQ(B  _ 1  A),  Q (A) X  =  P^B  L  A)  Borel sets A i n R . The l i k e l i h o o d  lemma shows how  r a t i o dP^/dPg  i s simple to f i n d  and the f o l l o w i n g  i t r e l a t e s to dQ^/dQ^, the d e s i r e d l i k e l i h o o d  ratio.  - 57  Lemma. and  L e t Pg, Pj be p r o b a b i l i t y measures on a measure space  f:(X,F)  •»• (Y,G)  f : ( Y , G ) •»• ( X , F ) . - 1  Q (A) Q  Then i f P  P  1  Q  be measureable and Define Q  = P (f  air  y>  (X,F)  w i t h measurable i n v e r s e by  - PjCf"  A).  1  (A e G),  then  Q  dP. (  1-1  (Y,G)  X  « Q  1  on  A ) , Q (A)  _ 1  0  and Q  Q  dQ,  =  lor-  0  Proof:  -  , (  (y  <y»  f  Y  e  >-  0  Let A e G. dP,  dP,  _,  U Hr< <*» V?> " Uw< ™ d  £  f  dP,  /-i F  =  f  (by the change of v a r i a b l e s formula  A  ( x ) dp  0  ,  o  f  o  (x)  dp  with y = f ( x ) . )  = P ^ r ^ A ) ) = Q^A),  The  use  of t h i s lemma r e q u i r e s dPj/dPg.  d i s t r i b u t i o n of a v e c t o r ( X j , . . . , X v a r i a b l e s and  r  x  =  (x^ > • • • *  0, ••.0)  n  i s the  , 0,...,0)' of r i . i . d .  Pj i s the d i s t r i b u t i o n  mean v e c t o r j£ = ( P j , . . . , M , 0  But P  N(0,1)  of t h i s v e c t o r w i t h the added  0)'.  Therefore  f o r any  - 58 -  exp{- j I ( x  dP. •jp-Cx)  - u ) } 2  ±  ±  I V I exp{- y I x } 1  0  1  = exp t E 1  p  i  x  2  i" 1  ^  1  =  = exp{- I(x - _y)' (x -  ^ ' 2. - { 2' J±l  e x p  v)  +  I x' x j .  T h e r e f o r e , by the lemma, s i n c e the l i n e a r map B i s 1-1 from the range o f D to the range of A, f o r each y i n the range of A  dQ.  _ i  dP. ( z )  ._ i  ,  dP.  (B ' > - ^ Z  = exp(u_' B' x  -  (»•  j)  "J Ji' i l l  = exp{(B'_6)' > B'y - i-(B'_6)'(B'_S)}  = exp{6/  X  -  ^V±}.  and t h i s was the formula used to o b t a i n ( 3 . 1 2 ) .  Note:  A f u r t h e r use of t h i s c a l c u l a t i o n i s made f o r the a p p l i c a t i o n o f  the SPRT to the problem of t e s t i n g normal d i s t r i b u t i o n .  the mean v e c t o r of a m u l t i v a r i a t e  Note that o n l y a l t e r n a t i v e s which s p e c i f y a mean  v e c t o r i n the range of the c o v a r i a n c e m a t r i x can be t e s t e d .  - 59 -  A.3  Two Lemmas on Weak Convergence In  this section  1 is first  Lemma 1.  p r o o f s of Lemma 1 and Lemma 2 a r e p r o v i d e d .  r e s t a t e d more  precisely.  L e t PQ and P^ be p r o b a b i l i t y  dP^dPQ be t h e i r l i k e l i h o o d probability  ratio.  measures w i t h P^  random elements X, X  n  Lemma  measures w i t h P^  PQ and Z =  F o r each n=l,2,... l e t PQ and  PQ and Z  = dP^/dP™.  n  be  I f there a r e  such that  d (X , Z ) •»• (X, Z) under P Q , P n  n  Q  then  X  Proof:  I f f i s bounded  d •*• X under P , P ^  n  n  and continuous on the space where X  n  / f ( x ) dP^ - / f ( x ) z d?l n  n  n  = /h(X , Z ) d P j n  n  * jh(X, Z) d P (since  Q  h(x, z) = f(X) z i s continuous on the product space.)  = Jf(X) Z dP = Jf(X) d P  r  Q  and X l i e  - 60 -  Lemma 2. X, Y, X  n  Let P^ and PQ(n=l,2,...) and Y  be p r o b a b i l i t y measures  and l e t  be random elements such that  n  and  X  n  Y  n  ^ X  under P^,  £ Y  under P , n  P  Q  F. Q  I f t h e r e i s a continuous f u n c t i o n H and random elements e  n  such that  Y = H(X) Y e  n  = H(X ) + n  e  n  ->- 0 i n p r o b a b i l i t y under  n  P , n  then Y ) i  (X , n  Proof:  Since e  n  n  •> 0 i n p r o b a b i l i t y  (X , n  (see  (X, Y)  VQ.  under P , n  i t s u f f i c e s to prove that  d H ( X ) ) - (X, H ( X ) ) . n  [ 1 ] ) . For t h i s l e t f be continuous and bounded  space where (X,Y) l i v e s .  on the product  Then  J f ( X , H ( X ) ) dPjJ = / g ( X ) dpJJ n  n  n  *  JR(X)  dP  0  = Jf(X, H(X)) d P Since g(x) = f(x, H(x))  i s continuous.  Q  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0096109/manifest

Comment

Related Items