OVERDISPERSION IN POISSON REGRESSIONByW. Brad McNeneyB. Sc. (Mathematics) University of British ColumbiaA THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF SCIENCEinTHE FACULTY OF GRADUATE STUDIESDEPARTMENT OF STATISTICSWe accept this thesis as conformingto the required standardTHE UNIVERSITY OF BRITISH COLUMBIA1992© W. Brad McNeney, 1992In presenting this thesis in partial fulfilment of the requirements for an advanceddegree at the University of British Columbia, I agree that the Library shall make itfreely available for reference and study. I further agree that permission for extensivecopying of this thesis for scholarly purposes may be granted by the head of mydepartment or by his or her representatives. It is understood that copying orpublication of this thesis for financial gain shall not be allowed without my writtenpermission.(Signature) Department of ST I111 Si C SThe University of British ColumbiaVancouver, CanadaDate /1 It. '2 DE-6 (2/88)AbstractInvestigation of a possible relationship between air quality and human health in the communityof Prince George, British Columbia was undertaken after a public opinion poll in 1972 discov-ered that poor air quality was the number one concern of the residents of Prince George. Ananalysis which attempted to identify such relationships using a data set including air qualitymeasurements and hospital admissions for the period April 1, 1984 to March 31, 1986 is dis-cussed in Knight, Leroux, Millar, and Petkau (1988). A similar analysis using emergency roomvisits during the same period rather than hospital admissions is described in Knight, Leroux,Millar, and Petkau (1989). The data set described here was collected to carry out a follow-upstudy to the emergency room visits analysis.The main part of the analyses carried out involved the use of Poisson regression modelswith a minor extension to account for over-dispersion in the data. The results of the analysiswere not consistent with either the earlier study or with the expectations of the investigators.For example, higher levels of one of the air quality variables was found to be associated with adecrease in the number of emergency room visits for respiratory disease in the winter, but anincrease in emergency room visits for respiratory disease in the fall. A mechanism to explainsuch effects is difficult to imagine. These counter-intuitive results motivated a simulation studyto assess the methods used in the analysis and to compare these to other possible estimatorsand test statistics that can be employed in the analysis of over-dispersed Poisson data.iiTable of ContentsAbstract^ iiList of Tables^ vList of Figures^ viiAcknowledgement^ viii1^The Prince George Study1.1^Data Set Description ^121.1.1 Emergency Room Visits Data ^ 21.1.2 Air Pollution Data ^ 21.1.3 Meteorological Data 31.2 Methodology for the Prince George Study ^ 31.2.1 Poisson Regression ^ 31.2.2 Accounting for Over-Dispersion ^ 81.3 Results of the Data Analysis ^ 101.3.1 Temporal and Meteorological Effects ^ 111.3.2 Respiratory Visits and TRS ^ 121.3.3 Other Analyses of Pollution Covariates ^ 15iii1.4 Questions Raised by the Analysis ^ 152 Methodology^ 172.1 Quasi-Likelihood in Generalized Linear Models ^ 172.1.1 The Estimating Equations ^ 192.1.2 Test Statistics for Hypotheses Regarding Regression Parameters ^ 242.2 Test Statistics for Hypotheses in Over-Dispersed Poisson Regression ^ 273 A Simulation Study Related to the Prince George Study^ 313.1 The General Simulation Procedure ^ 343.1.1 Generating the Data ^ 343.1.2 Fitting the Full Model 353.1.3 Fitting the Reduced Model ^ 363.2 The Specifics of the Simulation Study 364 Results of the Simulation Study^ 424.1 Estimating the Dispersion Parameter 0 ^ 424.2 The Simple Case with a Single Series 504.3 The Simple Case with Three Series ^ 604.4 The More Complicated Case with a Single Series ^ 714.5 The More Complicated Case with Three Series 925 Discussion^ 106Bibliography^ 111ivList of TablesTable 1. Mean Values of Estimates (± Standard Deviation) in 1000 Simulated Data Sets. .43Table 2. Mean Values of Estimates of 0. 50Table 3. Standard Deviations of Estimates of /3 and Mean Values of Estimated Standard Errors(±s.d.). ^ 51Table 4. Observed Rejection Probabilities Under Reduced Model. ^ 54Table 5. Mean Values of Estimated Parameters (+ s.d.) Under Full Model. ^ 57Table 6. Observed Rejection Probabilities Under Full Model. ^ 59Table 7. Mean Values of Parameter Estimates. ^ 60Table 8. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors(±s.d ) ^ 62Table 9. Observed Rejection Probabilities: Common Pollution to No Pollution Model. ^ 63Table 10. Mean values of Parameter Estimates. ^ 64Table 11. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors( +s .d ) ^ 65Table 12. Observed Rejection Probabilities: Separate to Common. ^ 66Table 13. Observed Rejection Probabilities: Common to Null. 67Table 14. Mean Values of Parameter Estimates. ^ 68Table 15. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors(±s.d.). ^ 69vTable 16. Observed Rejection Probabilities: Separate to Common. ^ 70Table 17. Mean Values of Estimated Parameters Under Reduced Model. ^ 71Table 18. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors(±s.d.). ^ 75Table 19. Observed Rejection Probabilities Under Null Hypothesis. ^ 78Table 20. Mean Values of Estimated Parameters Under Alternative Hypothesis. ^ 80Table 21. Standard Deviations of Estimates Under Alternative Hypothesis. ^ 83Table 22. Observed Rejection Probabilities Under Alternative Hypothesis. ^ 89Table 23. Mean Values of Parameter Estimates. ^ 92Table 24. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors(±s.d.). ^ 94Table 25. Observed Rejection Probabilities: Common to Null Model. ^ 95Table 26. Mean Values of Parameter Estimates. ^ 97Table 27. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors(±s.d.) ^ 98Table 28. Observed Rejection Probabilities: Separate to Common. ^ 100Table 29. Observed Rejection Probabilities: Common to Null. 100Table 30. Mean Values of Parameter Estimates. ^ 101Table 31. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors(±s.d.). ^ 103Table 32. Observed Rejection Probabilities: Separate to Common. ^ 104viList of FiguresFigure 1. Comparison of Estimators of Phi: Correct Variance Simulations(Phi=1 or 1.4) ^ 45Figure 2. Comparison of Estimators of Phi: Correct Variance Simulations(Phi=2 or 3) ^ 46Figure 3. Comparison of Estimators of Phi: Misspecified Variance Simulations(Phi=1 or 1.4) ^ 47Figure 4. Comparison of Estimators of Phi: Misspecified Variance Simulations(Phi=2 or 3) ^ 48viiAcknowledgementI would like to express my gratitude to my advisor, Professor A. John Petkau, for his support,advice and patience throughout the writting of this thesis. I would also like to thank ProfessorNancy E. Heckman for reading the manuscript and providing many helpful comments.viiiChapter 1The Prince George StudyMotivation for this work was provided by the analysis of a data set collected in Prince George,British Columbia, to study the possibility of associations between the ambient levels of airpollution and human health as measured by the rates of emergency room visits for respiratoryillnesses. Investigation of such a possible relationship was undertaken after a public opinionpoll in 1972 discovered that poor air quality was the number one concern of the residents ofPrince George, ahead of such issues as crime, alcohol abuse and recreation facilities. Althoughmonitoring of air quality parameters has been carried out at several monitoring stations inPrince George since 1980, most attempts to study this issue have compared hospital admissionsrates and/or mortality rates in Prince George to those in other communities in British Columbiarather than attempt to identify an association between ambient levels of air pollution in PrinceGeorge and human health.An analysis which did attempt to identify such relationships using a data set including airquality measurements and hospital admissions for respiratory illnesses for the period April 1,1984 to March 31, 1986 is discussed in Knight, Leroux, Millar, and Petkau (1988). A similaranalysis for the same period based on emergency room visits rather than hospital admissionsis described in Knight, Leroux, Millar, and Petkau (1989). The data set described here werecollected to carry out a follow-up study to the emergency room visits analysis.1Chapter 1. The Prince George Study^ 21.1 Data Set DescriptionThree measures of air quality were monitored and the daily numbers of visits to the emergencyroom (ER) for respiratory related illnesses were recorded during the period April 1, 1986 toMarch 31, 1988. In addition, meteorological data in the form of temperature and maximumrelative humidity was collected.Description of the relationship between the pollution variables and the ER visits was thegoal of the study. However, meteorological and temporal information was also included aspotential explanatory factors so that any relationships discovered with pollution variables couldreasonably be attributed to the pollution itself and not to some other factor that might underlyboth pollution and ER visits.1.1.1 Emergency Room Visits DataEach visit to the emergency room at Prince George Regional Hospital by a resident of PrinceGeorge over the two year period from April 1, 1986 to March 31, 1988 was originally classifiedaccording to 35 diagnostic categories potentially related to respiratory illness. These 35 cate-gories were grouped into four broad categories which we will refer to as Asthma, Bronchitis+,Ear, and Others. Only the classification according to these four broad categories was availablein the data set provided to us. A total of 8,079 visits were included in the study and there wasone visit in at least one of the four respiratory categories on every day except January 15, 1987.1.1.2 Air Pollution DataData on three air quality parameters were available:Chapter 1. The Prince George Study^ 3TRS — measured via a continuous monitor with readings subsequently converted tohourly averages. The average of these hourly averages becomes the dailyaverage and this was the summary used.TSP — measured via a 24 hour vacuum sample once every six days.SO2 — measured via a continuous monitor in the same fashion as TRS. The dailyaverage was the summary used.1.1.3 Meteorological DataThe meteorological data consisted of daily summaries for each of the following parameters:Temperature — the average of 24 hourly readings in degrees Celsius.Maximum Relative Humidity — the largest of 24 hourly readings.1.2 Methodology for the Prince George StudyThe main part of the analyses carried out involved the use of Poisson regression models (orlog-linear models). We give a brief description of these models and then describe an extensionof Poisson regression models, which was required for the modelling of emergency room visits.1.2.1 Poisson RegressionThe Poisson regression model is a special case of a larger class of models referred to as gener-alized linear models (GLMs), the theory of which is derived largely from the properties of theexponential family of distributions. Thus before we describe the Poisson regression model inChapter 1. The Prince George Study^ 4detail, a brief discussion of the exponential family and GLMs seems appropriate (see McCullaghand Nelder, 1989, for a more detailed description).For a random variable Y in the exponential family with canonical parameter 9 and (known)nuisance parameter 0, we may write its density function as f (y; 9) = exp{(y9 — b(0))/ a(0)c(y; q)}. Denote the mean value of Y as ,u, which is related to the parameter 9 by the function9 = g(p). We will refer to the function g(.) as the link function.For making inference about 9 we introduce the log-likelihood function, the log of the abovedensity function, which we denote 1(9). In the case of n independent observations Yi (i =1, n), with possibly different canonical parameters, the joint log-likelihood is given by 1(9) =li(0i). In GLMs a linear model is specified for the parameters 0i of the form 0i = g(pi) =Xifi (where /3 is a vector of p<n parameters and Xi is the i th row of an n x p design matrix).It is then possible to estimate /3 via maximum likelihood by taking the derivative of the jointlog-likelihood with respect to each of the parameters, equating the resulting p equations to zeroand solving for the parameters of interest. These estimators have several nice characteristicsincluding a limiting multivariate normal distribution with mean vector equal to /3w, the truevalue of the vector of parameters, and covariance matrix equal to the inverse of the FisherInformation matrix corresponding to the joint distribution of the Yi (i = 1, ..., n).Now consider a Poisson random variable Y. We write the density function of Y as f(y; it) =(e —PitY)/y! or equivalently as f(y; = exp{ylog — ,u — logy!}. The latter form allows us toidentify the Poisson distribution as a member of the exponential family with 8 = log /2 (andtherefore link function given by g(•)=log•)), b(9) = fit , a(0) = 1, and c(y; 0) = —logy!.The motivation behind using the Poisson regression model for the Prince George study liesChapter 1. The Prince George Study^ 5in the intuitively reasonable assumption that the daily numbers of emergency room visits followPoisson distributions with possibly different means. The mean number of visits on a given daywill be described by a function of the available covariate information up to and including thatday, which includes temporal variables such as day of the week and month, the meteorologicalvariables, daily temperature and maximum humidity, and the pollution variables, TSP, TRSand S0 2 . The use of Poisson regression depends on the following basic assumptions:(a) the expected number of visits on a given day depends explicitly only on time, meteorologyand pollution and not on the number of visits on past days.(b) daily visits are conditionally independent given the temporal, meteorological and pollutioncovariates.The assumption that the expected number of visits does not depend explicitly on past visitsseems to be a reasonable starting point in this situation. However, more general models inwhich the mean depends explicitly on past visits could be considered as well.Suppose now that ti is a vector of temporal covariates, mi is a vector of meteorologicalcovariates and si is a vector of pollution covariates for day i; si and mi can include values ofpollution or meteorology prior to day i as well as those values on day i. If yi is the number ofemergency room visits on day i (i =1,2,...,731) for a particular diagnostic class, the log-linearmodel assumes that yi has a Poisson distribution with mean p, wherelog pi = a + ti er rni'6 si'rywith a, r, 6 and y unknown parameters (r, 6 and y are vectors having the same length as ti, miand si respectively). The components of the parameter 7 give the relative changes in expectedChapter 1. The Prince George Study^ 6visits on a given day (that is, with temporal and meteorological covariates fixed) associated withchanges in the pollution variables. In particular, for small changes in the pollution variables,from si to si + Ai, exp(A',0) — 1 A'7 gives the percentage change in the correspondingexpected visits.In order to account for possibly different patterns of emergency room visits for the differentdiagnostic categories, the basic model is generalized by allowing different parameters a, T,and y for the different diagnostic categories, and thus describing the expected number of visitsfor category j (j =1,2,3,4) on day i by the equationlog fGij = cri^t'i rjTo fit the above model into the framework described for GLMs let p = (pt, /4,4 /4) t and= (a t , rt, St, ,.yt)t where here a = (0 1 , 02 ,03 , 04 )t, T = 4, 71, Tv, and so on. Then, if Xis the appropriate design matrix containing all of the covariate information, our model can beexpressed as logy = X /3.In addition to assuming the observations to be conditionally independent given the co-variates, we assume that data from different diagnostic categories is independent, so that thejoint likelihood is simply the product of the marginal Poisson distributions. Thus the jointlog-likelihood is:4 7311(0) = E E [yii log pij — pij — log yij!] .j=1 1=1For ease of notation, it will be more convenient to write the above log-likelihood using onesubscript. We order the yii's as yl , ..., y2924, with the pi's numbered correspondingly. NowChapter 1. The Prince George Study^ 7express the joint log-likelihood as:2924l(0) = E [yi log — — log yi!) .i=iTo obtain maximum likelihood estimating equations for the parameters in /3 (suppose thereare p parameters in /3), we differentiate the expression in (1.1) with respect to each of the pcomponents of and equate the resulting derivatives to zero to arrive at:al^2924Yi — Pi ) ON =apa = 0; j = 1, ...,p.fr_1(^' (1.2)Note that our model assumes pi = exp{XiM, or log pi = X0. Thus, (1.2) can be rewritten as81^2924ni3 =^(yi —^Xii^0; j = 1,...,p.^ (1.3)"Pi^1=1The resulting estimators, Q , are asymptotically normal, with mean vector given by /3*, the truevalue of the vector of parameters, and covariance matrix equal toL— E ( ija212 )]In this case^021^2924^821^— apt = —E ()(902^LT., (902 .Given the solution to (1.3), the fitted values, it, = exp{Xi;j}, may be substituted in theabove expression to obtain an estimate of the asymptotic covariance matrix. Using these fittedvalues, the deviance is then defined to be —2 times the log-likelihood, or2924dev. = —2 E [yi log - - log NI .i=iFor a fitted model with p parameters and a joint likelihood from n independent observations,the degrees of freedom associated with the deviance is n — p. By taking the difference in theChapter 1. The Prince George Study^ 8deviances of two fitted models, one of which is nested within the other, we may form a likelihoodratio statistic that can be compared to its limiting chi-square distribution to examine if thesmaller model constitutes a significant deterioration in fit compared to the larger model.The Pearson residuals are defined to be(y, — r, — .These quantities are often used for diagnostic purposes in the same way the residuals from anordinary regression (assuming normality) are used. We may also use these residuals to forma goodness-of-fit statistic, the familiar Pearson X 2 , by summing the squares of these Pearsonresiduals. Another potential use of the Pearson X 2 emerges in the following section dealingwith over-dispersion.1.2.2 Accounting for Over -DispersionThe extension of the Poisson regression model to be described concerns accounting for over-dispersion. Suppose that the variance of the daily visits given the temporal, meteorological andpollution variables is greater than the mean; in this situation the emergency room visits data aresaid to be "over-dispersed". Over-dispersion represents a violation of the Poisson assumptionunderlying the Poisson regression model, which requires that the variance and mean be equal.One way to handle over-dispersed data is to introduce a dispersion parameter 0 > 1 and assumethat the variance of the visits on day i is We now describe the procedure for statisticalinference in this new model.The parameter vector which appears in the specification of the daily mean visits ratesis estimated exactly as for the model without a dispersion parameter, i.e. by maximizing theChapter 1. The Prince George Study^ 9Poisson log-likelihood function of (1.1). However, when assessing the statistical significance ofestimated parameters it is necessary to take into account the dispersion parameter. To describehow this is done in the context of model selection (the dispersion parameter must also be takeninto account when evaluating the standard errors of estimated parameters), assume that webegin with a model containing a number of parameters (the full model) and wish to test theviability of another model which is nested within the full model (the reduced model). Under ourindependence assumptions, the difference in the deviances for the two models is approximatelydistributed as 0 times a )(2 random variable with degrees of freedom given by the differencein the degrees of freedom associated with the two deviances. Thus the difference in deviancesmust be divided by an estimate of the dispersion parameter before it can be compared to the x2distribution to assess whether reduction from the full model to the reduced model is permissible.One choice of an estimate of the dispersion parameter is the ratio of the sum of squaredPearson residuals, the Pearson X2 statistic, to its degrees of freedom (McCullagh and Nelder,1989, p.200), both determined under the full model. Another estimate, which is often very closeto the previous one, is the ratio of the deviance to its degrees of freedom, also under the fullmodel. For this data the estimate based on the deviance was used; this leads to the followingstatistic for testing whether reduction from the full model to the reduced model is permissible:deviance (reduced) — deviance (full)A'dev. =deviance (full)/df (full)^;this statistic is compared to the x2 distribution with degrees of freedom given by Adf =df (reduced) — df (full).The Poisson regression model with unspecified dispersion parameter is a particular case of aChapter 1. The Prince George Study^ 10class of models called "quasi-likelihood" models, so-named because the likelihood which is usedto obtain estimates, the Poisson likelihood in our case, is not the true likelihood for the databut plays the role of a likelihood and so is called a quasi-likelihood. Quasi-likelihood modelsare discussed in more detail in Chapter 2.We now describe the results of the analysis of the Prince George data using the methodsdescribed above.1.3 Results of the Data AnalysisThe possibility of over-dispersion in a series of counts can be judged by the index of dispersion:X 2 = (n — 1)variancemeanwhere n is the sample size, 731 in this case. Under sampling from a Poisson distribution, X 2has an approximate chi-squared distribution with n-1 degrees of freedom and can be convertedto an approximate standard normal variable by z = VT/Y 2 — ^2(n — 1) — 1.The results for the emergency room visits data are presented the following table:category mean variance X 2 zAsthma 0.76 0.80 763 0.87Bronchitis+ 1.54 3.36 1593 18.2Ear 3.62 6.92 1395 14.6Other 5.13 14.2 2027 25.5Except for the Asthma category, these exhibit a large degree of over-dispersion relative tothe Poisson distribution; the values of z given in the above table clearly reflect extreme over-dispersion for the categories Bronchitis+, Ear and Other. In fact, these distributions exhibitthe pattern of a mixture of Poisson distributions, having higher than expected numbers of veryChapter I. The Prince George Study^ 11small and very large counts, at the expense of moderate counts. This over-dispersion must betaken into account in the data analysis.As mentioned previously, we must allow for the possibility that emergency room visits de-pend on factors other than just pollution, such as temporal and meteorological effects. Thesecovariates were considered first, and model reductions were attempted to find a model whichparsimoniously describes these effects. The pollution covariates were then added to the modeland model reductions were attempted to identify the pollution covariates which had a substan-tial effect on emergency room visits for respiratory illness.1.3.1 Temporal and Meteorological EffectsThe first step of the analysis involved the incorporation of adjustments for temporal effects.The full model for the log of the visits rate included separate effects for each of the 12 monthsand each of the 7 days of the week for each diagnostic category. Attempted model reductions tocommon month or common day-of-the-week effects, that is common across all four diagnosticcategories, resulted in unacceptably large changes in adjusted deviance and therefore the fullcollection of separate effects for each diagnosis was retained as the temporal model.Next, the meteorological covariates (temperature and maximum relative humidity) wereadded to the temporal model at lags of 0 to 3 days; that is for any i = 4, ..., 731 we allowfor the possibility that meteorology at times i, i — 1, i — 2 and i — 3 might affect the rate ofemergency room visits at time i. We first allow for separate effects at each lag for each of thefour seasons, winter (December to February), spring (March to May), summer (June to August)and fall (September to November), but with each of these effects common to all four diagnosticChapter I. The Prince George Study^ 12categories. Based on our model reduction criterion, these separate season effects were found tobe unimportant compared to common effects across seasons for each lag.When separate meteorological effects at each lag for each diagnostic category were consid-ered, we found unacceptable increases in adjusted deviance when attempting to simplify theseparate temperature effects to common effects. On the other hand, only common maximumhumidity effects were needed. Thus our model which adjusts for possible temporal and meteoro-logical effects included separate diagnosis effects for month, day-of-the-week and temperature,and common maximum humidity effects.1.3.2 Respiratory Visits and TRSWe next investigated a model which expresses the daily emergency room visits rates for anyparticular diagnosis category aslog visits rate on day i = temporal effects + meteorological effects +7o log TRSi + 71 log TRSi—i + 72 log TRSi_2 7310g TRSi_ 3 .Here the effects of each of the TRS values are common across diagnosis categories; we alsoconsidered models involving separate effects for the four diagnosis categories or separate effectsfor the four seasons. For all models fitted, the temporal and meteorological effects had thesame structure as previously identified. In what follows, we will abreviate this structure as thetemimet model.We started with separate effects for each diagnostic category but found the reduction tocommon effects acceptable and furthermore removal of the TRS effects altogether resulted inChapter 1. The Prince George Study^ 13only a negligible deterioration in the fit. The following table examines separate TRS effects foreach season, where as before the separate season effects are common to all diagnostic categories:model terms deviance d.f. A' dev. A d.f.tem/met effects + separate season TRS effectstem/met effects + common TRS effectstem/met effects only38273852385528042816282018.41.7124Reduction to common effects leads to an increase in adjusted deviance of 18.4 versus adecrease of 12 in the number of parameters which is not so easily dismissed (p=0.11). Unfor-tunately the estimated effects from the full model present a confusing picture as we see in thefollowing table which presents those estimates which are greater than one standard error inmagnitude:effect coefficient s.e.Winter Lag 0 -0.026 0.020Spring Lag 0 -0.027 0.026Summer Lag 0 0.043 0.028Winter Lag 2 -0.035 0.023Summer Lag 2 -0.035 0.029Looking at the effects at each lag we get the suggestion there may be some TRS effect at lagsof 0 and 2 days. Unfortunately the effects from one lag to the other are not consistent andfurthermore, it is difficult to understand why higher levels of TRS should be associated withhigher numbers of emergency room visits at some times of the year and lower numbers at othertimes of the year. These results may not be particularly surprising since the overall reduction inadjusted deviance for going from the full model to one with the tem/met effects only (p=0.20)indicates that the separate effects in the above table could be large simply by chance.However, we could also examine the effects grouped by season as in the following table:Chapter 1. The Prince George Study^ 14Lag Winter Spring Summer^Fall0123-0.026-0.035-0.027 0.043-0.035where the dashes represent estimated effects which are less than their standard error. Lookingdown the columns of this table we see the possibility of a negative cumulative effect in Winter.Accordingly we now present results for a model which includes a single cumulative effect (anaverage of the logarithm of the lag 0 to lag 3 pollution measurements) for each of the fourseasons:model terms deviance d.f. A' dev. A d.f.tem/met effects + separate season TRS effectstem/met effects + cumulative TRS effectstem/met effects only3827383738552804281628206.713.4124While there is little difference in the fit of the separate season effects model and the cumu-lative effects model, the cumulative TRS effects constitute a substantial improvement in the fitover the tem/met only model (0 idev..13.4, Ad.f.=4, p=0.001). The estimates and standarderrors of these cumulative effects are presented in the following table:effect coefficient s.e.Winter -0.060 0.023Spring -0.002 0.033Summer 0.009 0.034Fall 0.075 0.030These effects suggest that an increase in the level of TRS is associated with a decrease in thenumber of emergency room visits for respiratory disease in the winter, but an increase in thenumber of emergency room visits for respiratory disease in the fall. This model provides a moreChapter 1. The Prince George Study^ 15parsimonious description of the effect of TRS on emergency room visits for respiratory diseasethan the season effects model, yet a mechanism to explain such effects is difficult to imagine.1.3.3 Other Analyses of Pollution CovariatesSimilar analyses were carried out for SO 2 (as well as TRS and SO 2 simultaneously) and forTSP. Details of the model reductions and results can be found in McNeney and Petkau (1991).These analyses proceeded in the same fashion as the TRS analysis described above, and similarproblems were encountered; that is, the parameter estimates obtained in the final models weresuch that no plausible explanation for their structure came to mind.1.4 Questions Raised by the AnalysisThe results of the data analysis described above make it difficult to come up with a reasonableand coherent explanation of the effects of air pollution on human health based on this data set.There are many possible reasons for our results, such as:(1) the air pollution data do not accurately represent the exposure of the residents of PrinceGeorge to pollutants,(2) emergency room visits for respiratory illnesses are poor indicators of the effects of airpollution on human health,(3) some other important factor was not considered in the analysis, or(4) the method of analysis is potentially misleading.With respect to the last item, the following questions come to mind:Chapter 1. The Prince George Study^ 16(a) Is the estimate of the scale parameter, 0, used in this analysis, namely deviance(full) d.f(full) ^agood estimate, or should we have used Pearson X 2 (full) ?d.f.(full)^•(b) Is the change in adjusted deviance criterion appropriate for model reduction in this situ-ation?(c) Are statistics other than the adjusted deviance better for testing hypotheses concerningthe regression parameters when over-dispersion is present?(d) Given the amounts of over-dispersion in the emergency room visits series, are the estimatesof the model parameters and standard errors accurate?These questions motivated the simulation study we describe in Chapters 3 and 4. Be-fore proceeding to the simulation study, Chapter 2 presents a more detailed description ofquasi-likelihood theory and the estimating equations obtained from this theory for estimatingregression and dispersion parameters. We also discuss some of the test statistics available fortesting the significance of regression parameters. Finally, over-dispersed Poisson regression isdiscussed as a special case of quasi-likelihood.Chapter 2MethodologyIn this section we describe the aspects of maximum likelihood theory (in the context of GLMs)which motivate the theory of quasi-likelihood first proposed by Wedderburn (1974) and alsodiscussed by McCullagh and Nelder (1989, Chapter 9). In particular, we are interested in theestimating equations and test statistics derived from this theory. We also discuss the applicationof quasi-likelihood to the analysis of the Prince George data.2.1 Quasi-Likelihood in Generalized Linear ModelsRecall the log-likelihood for a single random variable in the exponential family1(9) = ( ye — b(0))1 a(0) + c(y; 0).Maximum likelihood theory assumes sufficient regularity conditions to ensure that the followingrelations hold^E [ 019^0l ] = 0.' and E [ °21 4- E {( 81 ) 21 =0.2^09In this discussion we will assume these conditions hold. The derivative of the log-likelihood is011 00 = (y — b'(9))1 a(0), therefore E(011 00) = 0 implies that E(Y) = b'(9). We also have021 0211^b"(9)— b"(0) so that — E ^=^ao2^a(0)^092 a(0) •17Chapter 2. Methodology^ 18FinallyE [(M 21 , E [CY a—( 1)0;(0))) 2]^1 — a2(0) Var Y,which leads to the expression Var(Y) = b"(0)0(0).Rewriting the log-likelihood in terms of the mean it, we have 1(0) =1(g(p)) andal_ 01 00Op = ae ap'so that01 iy — b'(0)1 00^y — V(0) ap —^a(q5) ) aµ — 40)4^•Sinceay(^opp = b'(0) and b"(0) = 00e) . ,,the score function S(p) = -g- can be rewritten as S(IL) = (y — E(Y))/Var(Y). Note the follow-ing properties of the score function:1. E(S) = 0,2. Var(S) = E(S 2 ) = ( a- .17- 1-12.E(17 — 11)2 — Vari- (Y) ,3. -E(e)=ai&ly; -Now consider a random variable Y for which we do not specify a distribution but onlysuppose it has mean value p and a variance that is related to p by some function V(p; 0) where(kis an unknown parameter. Then if V(µ; q5) correctly specifies the variance, the functionU* ( 17 ; Ft, 0) — Of — li)17(ti; 0)has similar properties to a score function resulting from a log-likelihood in that:Chapter 2. Methodology^ 191. E(U*) = 0,2. Var(U*) = E(U* 2 ) = v2(1,, ;0E(17 — /1 ) 2 =1V (A; ek)EiV(A;c5)1•3. —E( aµ = E [V( 1t-47) (17^V2(µ;0 1 - ve—'mSince most asymptotic likelihood theory is based on the above three properties, we mightexpect the integral of^to behave like a log-likelihood. We refer to the integralfy^tY (y — t) dtQ(Y;^= j U* (Y;t, Odt = j V^ (t;if it exists, as the quasi-log-likelihood or simply the quasi-likelihood. For n independent obser-vations having the same unknown parameter 0 we have the quasi-likelihoodQn(Y;^= E^0)i=iwhere now y and p are vectors of length n. As in GLMs, the mean vector p is assumed to berelated to the regression parameters by the link function g(p) = X/3.2.1.1 The Estimating EquationsAs in the case where we have a true likelihood, the approach with a quasi-likelihood is to differ-entiate Q T, with respect to each of the components of and equate the resulting p derivatives tozero to obtain maximum quasi-likelihood estimators for the parameters. If 0 were known, thenunder the assumption that pi = exp{XiP}, we would solve the following estimating equations:0C2n^n (Yi^ ujo, 0) =^= E = 0 ' ^1, p.u1-1.7^v(2.4)We will refer to the p-dimensional vector (U 1 , ..., Up )' as U0 .Chapter 2. Methodology^ 20Because 0 is unknown, we must introduce an additional estimating equation. This equationis based on the obvious notion that the variance function V(pi; 0) should reflect the true vari-ances of the observed yi (denote these an. Thus, 0 should be estimated so that V(tti; 0), themodel-based variance, agrees well, in some sense, with the true variance q. Only estimates ofthe true variances are available from the data, so a sensible estimate of 0 is provided by thesolution to the "moment" equationOP, =^q)i=12 11 = 0.^ (2.5)To correct for the bias resulting from the simultaneous estimation of the p regression parameters,Breslow (1984) suggests modifying this equation slightly toWV , =^14)2 (ni=1 V(tti;^n^j= 0^ (2.6)in place of (2.5).Given an initial estimate of 0 we may use (2.4) to obtain an estimate of /3. This estimate of/3 can then be used in (2.6) to estimate 0. This procedure is iterated to obtain the joint solution($, ) to the estimating equations (2.4) and (2.6). Subject to regularity conditions (Inagaki,1973; White, 1982) this joint solution converges in probability, as the number of observationsincreases, to (0*, 0*) where d* is the true value of 0, and 0* is the value of 0 which satisfiesthe limit equationn^cr?71^v(ili; 0) = 1.To motivate the limiting distribution of the estimator (), ik) consider the following. Let= (/3t, 0)t and let U(9) = (U0 ', PIT, where U0 is the vector form of the equations givenby (2.4) and U 4' is given in (2.6). While (2.6) is not a maximum quasi-likelihood equation,=^ l nn auiBm^n i1 n aui(r)^au(e*)ncoE E aot = limnoo n^aotBm^i=1aun a0tChapter 2. Methodology^ 21we may consider (2.4) and (2.6) as moment-type estimating equations. The derivation ofthe asymptotic properties of ([3,i4) will proceed along the same lines as the derivation of theproperties of maximum likelihood estimators, although some modifications are required.The fact that 8 is the solution to U(a) = 0 along with the Mean Value Theorem allows usto write the Taylor series expansion of U(a) about the point 0* as0 = U(a) = U(8*) + au00t - r),em where each of the p 1 elements of Om are between the corresponding elements of 0 and 0*.Re-arrangement allows us to write this as‘77-20 —13*) ( _ 1 OU■ n aotyiU (r).em(2.7)Under mild regularity conditions, 0 will converge in probability to 8* (see Moore 1986,Theorem 1, p.586). Then, since Oln is between a and 8* and au loot is a sum of n independentcomponents, we havewhere here the subscript i refers to the contribution of the ith observation; i = 1,^n and theexpectation is taken with respect to the true value 0* under the assumption that p = exp{X/3}correctly specifies the mean. Now —E[au(r)laet] is a (p 1) x (p + 1) matrix of the form[ An bnctn dowhereauo^a r n (yi _= —E^E apt^V(kti; O s') ad 0 .ao tChapter 2. Methodology^ 22= E ^1 ON ONVGai; 0*) 00 00t+ E^(Yi –^°pi av °p io•^v2(4; 04') 013 api aot E^1 ^atLi 0,L,=ii v(iti; 0*) 00 00ta n (yi^)^rt (17 — ^0,4 av] _ o— -bn = –E— = E(Pi; 0* ) " 84) 9*00 i=1 v (pi; 0) 00 0*a n (Y iti)2 (n _ p)cn =^[E^i*, ^^u P 1=1^VI"^0•andn (Y pi)2 (n P)11 = V"`ri (Y /102 aydo =^{E^ Eji=i n^cb*^V2(14;0) 00Now consider the other random vector entering (2.4):n^ 2^29VGri cb* =2=1• V2 (Pii 0) 001 \–y.,1^ L-4.1 (,;;c•) 8/3 0*U(r) –\--T.^[v;;;Li 2^( p,i;0 )^nAssuming the regression model correctly specifies the mean, we have1^ 0E 77 U(0*) =lz_i7n)^•^L.-n=1 v(,e )^nBy the definition of 0* and provided regularity conditions hold (see Moore 1986, Lemma 1,p.585), it follows that this random vector has a limiting multivariate normal distributionN (0,[ eSt fe I)where B = limn_,,,,, ;7_13n. is the normed limit of71^f n^f^\^tBn = E(IP3 P3t )= E [E [ i 14)1 ^yP" affil Ivoli; o*) as 1=1 V(aii 0* ) 0/0o* ;0*Chapter 2. Methodology^ 23Because the observations are assumed to be independent, this reduces to= E^(Y, gi )2^°Pi]^it ^a?^ attiapi^Bi.iv - (4;0*)^adt e*.^a$ aotFor our purposes, the forms of e and f are of no consequence.Combining these results with Slutsky's theorem we have^Vii(o e* ) N (0,[^e fA ° 1 [ B et ^({ A 0 I ly)ct d ctwhere A =^c = lim^le and d = lim 00 in dn . Asn n[ A 0 1 - 1^[ A -1^0^ct d d- 1the asymptotic variance of VT.t(ë — r) is[A- 1^0 I r .13 e 1 I A'^0—^d-1 [ et^— CA- 1The asymptotic covariance matrix for ViT(S — /3*) is the upper left-hand p x p sub-matrix ofthe above matrix, namely A -1 BA -1 . Note that this is exactly the covariance matrix one wouldobtain for the estimation of Q based on the estimating equations (2.4), with 0 fixed at 0*rather than estimated; the asymptotic covariance for S is unaffected by the estimation of 0.This occurs despite the fact that Q and are, in general, correlated (the entries of the upperright-hand pxl vector of the above matrix need not be zero); it is a result of the fact that thepxl vector bn has all p elements identically equal to 0, that is, E[ 4 U 13] = 0. Note further thatif the function V(pi; 0) correctly specifies the variances cr?, then A = B and VTz(S — /13*) hasasymptotic covariance matrix given by A -1 .Thus we can consider two estimates of the asymptotic covariance of /3; the first is model-based and assumes that V(tti; 0) correctly specifies the variances. Denote this estimate 2A,/ =9*Chapter 2. Methodology^ 24A,7 1 where An is An with pi replaced by the fitted values^= pi(Xi; Si) and 0* replaced by-(4. The second estimate we refer to as the empirical estimate. Denote this EE = An ^An-1wheref3n^(yi — /10 2 i=1 v2 (4; ii)) aij aptThe idea of using an empirical covaiance matrix in connection with misspecified models hasbeen developed by numerous authors, including White (1982), Royall (1986), Liang and Zeger(1986), and Carroll and Rupert (1988, Section 4.3.2).We now have estimating equations for 8 and 0 which yield consistent estimators and wehave an expression for the asymptotic covariance of fr-t(T3 — /3*). In the next sub-section, weuse this information to form statistics to test the significance of the regression parameters.2.1.2 Test Statistics for Hypotheses Regarding Regression ParametersConsider the p-dimensional vector /3 partitioned into /31 and /32 of dimensions pl and p2 re-spectively. Here the point is to think of /32 as parameters which have fixed values /32 (usuallyzero) under some null hypothesis H o . We will refer to the model where all of the parameters= (#1, 13V are estimated as the full model, and the model where 131 is estimated when 02 isset to its hypothesized value /l as the reduced model.Under Ho , the random vector 642 — /32) has a limiting multivariate normal distribution withmean 0 and covariance E22 which is estimated by 2 22 , the lower right-hand p2 x /32 sub-matrixof either 2A/ or 2E. We therefore have model-based and empirical versions of a test statisticwe refer to as the Wald test where we compare the test statistic (th /36Dt (th /32 ) to the42) distribution.in 00, ik)^ow, 0. ) + aou; 1 OUOVT1 (13 — n 00 — 0*), (2.8)frii 7 4,m13m1 4,mChapter 2. Methodology^ 25Another possible type of test, one that does not require fitting of the full model, is thescore test. Recall from Section 2.1.1 that E[ i] = 0; j^1, p With sufficient regularityconditions, this implies that for any sequence of estimates^converging to 3* the statisticsU0 (/3, ii)) and U43 ($, cb*) are asymptotically equivalent. This can be seen by comparing thefollowing two asymptotic expansions:andU1 0 13Cb* ) =^OW, 0*) —VT/^VT/ n 0/3 V-17( - 0*)fmz(2.9) for some Om. (i=1,2) between p- and /3* and similarly for some om• The right-hand side ofthese equations are of the same form except for the final term in (2.8). But, as n gets large,— 0*) converges in distribution while -,10UP/00, which is an average of independentcomponents, converges to its expectation which is 0. Also, recall from Section 2.1.1 that theasymptotic covariance matrix of VT,(13 — /3*) does not depend on how, or even if, 0 is estimatedso that the distributions of the random vectors fii(/ — /3*) are the same in both (2.8) and(2.9). This shows that the random vectors defined by the left-hand sides of (2.8) and (2.9)have the same asymptotic mean and variance. Therefore, in what follows we may suppose thatthe estimator of /3 is based on the estimating equations (2.4) alone, with 0 fixed at its limitingvalue 0*.Now we consider the maximum quasi-likelihood estimates under the reduced model. WithUQ partitioned into components U01 and U432 of dimension p1 and /32 respectively, let /3reddenote this estimator, namely the solution to U01[( /31,4)t ,01. 0. We wish to determine theChapter 2. Methodology^ 26asymptotic distribution under the null hypothesis of the score statistic for testing Ho : 132 =To accomplish this, first examine the expansionauoi0 = uoiO^O P *(tired) =^°pi (141 —(371 (2.10) and the corresponding expression for the score statistic(Sred) = u132 613*) + aau: (gi — 01' )or2 (2.11)where Or' (i=1,2) are between 13 1 and 01. Rearranging (2.10) we getau al— 01) =OW),Oi alwhich, substituted into (2.11), allows us to writeou#(Sred) =^[02 (M)012 [au oii -1(9131 /3r i0;n2 Ual (0*) .As n oo we have02 (04') d, N(o, B22) \F-uoi (0*) L, N(0, Bii),and1 au ,32m201p An,auoi 1+' A ll .n ath n Othwhere the matrices All , A21, B11, and B22 are the appropriate submatrices of the matrices Aand B. It follows that the asymptotic distribution of this score statistic is multivariate normalwith mean 0 and covariance matrix1Nrn AVary ^= B22 - A21 A a B12 -B21 Aril Al2+ ^A21 Aril B11 A111 Al2 •(Sr ed) (2.12)Chapter 2. Methodology^ 27If the variance function correctly specifies the variances, then A = B and (2.12) reduces to theusual expressionVary[17.70 -^1—u 2 Pred).1 = A22 — A21111711 Al2.V77,(2.13)Therefore we can obtain a model-based estimate of the asymptotic covariance matrix bysubstituting An for A in (2.13) or an empirical estimate by substituting An for A and fin for Bin (2.12). The score test for testing Ho : 02 = QZ is given by comparing PI (Sred)2 -10.2 (Sred)to the xp2) distribution where 2 is our particular choice of the covariance matrix estimate.( The final type of test statistic we will consider is a deviance or likelihood ratio statistic.Recall the quasi-log-likelihood Q n (y; ,u, = 1 Q(yi; pi, 0). Define the deviance under thefull and reduced models as -2Q n (y; 1.1(X,:j full), (7)) and -2Qn(Y; it(X, Sred), ) respectively, wherethe estimate c6 is obtained under the full model in both cases. The deviance statistic is thenAdev. = 2 [Qn(Y; A(X ijred),^- n(Y; it(X f ull) , (k)]which should behave like a log-likelihood ratio and thus have an approximate 4 2) distribution(McCullagh and Nelder, 1989, p. 471).In the next section, we will examine the specifics of applying these tests for the significanceof regression parameters to the case where we have over-dispersed Poisson data.2.2 Test Statistics for Hypotheses in Over -Dispersed Poisson RegressionIn Section 1.2 the estimating equations and scaled deviance test statistic for over-dispersedPoisson regression were presented as an extension of Poisson regression. This extension wasChapter 2. Methodology^ 28necessary because all inference which requires estimates of variability, such as tests of hypothe-ses, will be in error if the over-dispersion is not taken into account, even though reasonableamounts of over-dispersion have very little effect on estimation of the regression parameters byordinary Poisson regression (Cox, 1983).We saw that an easy way to accomplish this was to introduce a dispersion parameter 0 tomodel this extra variability. In quasi-likelihood this is equivalent to specifying the variance ofthe observed data to be described by V(yi, 0) = 0iti for the i th observation. In this case, forthe link function g(y) = logy equations (2.4) and (2.6) of Section 2.1.1 becomeUM, = E Pi ) Xij = 0; for j = 1, ...,p, (2.14)i=10(0,0) =^[ (Yi Pi)2 n P 1 _ 0,i.1 n(2.15)where pi = exp{X1 18}. Note that 0 may be factored out of (2.14) and that the estimate of 0obtained from (2.15) is Pearson's X 2 statistic (using the Pearson residuals defined in Section1.2) divided by its degrees of freedom. An alternate estimate of 0 which is thought to be verysimilar is to replace X 2 by the deviance given by the joint Poisson likelihood under the fullmodel, which for this discussion we denote G 2 . For this formulation, it is immediate that theestimates of do not depend on the estimated value of 0; in other words, we may use the usualPoisson estimating equations for Q and estimate 0 afterwards with either of the two methodsdescribed above. Clearly this approach has considerable appeal over an approach involvingspecification of a variance function in which 0 cannot be factored out.Chapter 2. Methodology^ 29With the choices V(iti; 0)_^and it = exp{Xi i3}, then from Section 2.1.1 the model-based estimate of the covariance matrix of /3 is given by 2m A,7 1 wheren ^1 ^a/1i^nAn = E aStj1 v 01„nxiexts = (-4-1 EJ=1where the Xi are the row vectors corresponding to the ith row of X. Note that 0 may befactored out of An and the resulting covariance matrix is exactly times that obtained fromPoisson regression.Alternately, the empirical estimate of the covariance matrix of j is given by EE =whereV2 Can iA') as 0/3t i=i cog^i.1Note that the factors 0 appearing in An and En cancel out in the expression for EE.Considering these two estimates of the covariance matrix and using either 0 = X 2 /d.Lfutior = G 2full jd.f.full leads to three possible versions of the Wald test; two model-based and oneempirical.We can similarly examine three versions of the score test. The two model-based versions usethe expression given in equation (2.13) for the variance with An appropriately partitioned andthe score vector as given by the p 2 equations of the form (2.14) which correspond to parameterswith some hypothesized value under H0 . The resulting statistic could use either of the estimatesof 0 described above. The empirical score test, on the other hand, uses the empirical covariancematrix of the form described by (2.12) with An and En appropriately partitioned. Examinationof the expressions involved in this statistic reveals that the 0 terms disappear here also due tocancellation.^( yi pi)2^_^(yi - ^x texi4xiem =^(yi — ) 2AIXi •Bn = ^Chapter 2. Methodology^ 30Finally, we may consider two versions of the deviance test. The expression for the quasi-likelihood is simply the Poisson log-likelihood divided by an estimate of 0. Therefore ourdeviance tests, often referred to as scaled deviance tests, are given byn1Adev. = E (yi log(—) — — 140))where Ili are the fitted values under the full model and^are the fitted values under thereduced model. Note that the best estimate of 0 (using either X 2 /d.f. or G2 /d.f.) will be theone obtained under the full model since this model has the best estimates of pi.Now that we have discussed some of the methods available for modelling and making infer-ence about parameters in over-dispersed Poisson data, we wish to address the questions raisedin Section 1.4 and examine the performance of these procedures in a context similar to thatencountered in the Prince George study. A simulation study designed to answer these questionsis described in Chapter 3.Chapter 3A Simulation Study Related to the Prince George StudyRecall the methodological issues which arose in the Prince George study as described in Sec-tion 1.4:(a) Is the estimate of the scale parameter, q5^ deviance(full), used in this analysis, namely ^ ad.f(full)good estimate, or should we have used Pearson X 2 (full) ?d.f.(full)^•(b) Is the change in adjusted deviance criterion appropriate for model reduction in this situ-ation?(c) Are statistics other than the adjusted deviance statistic better for testing hypothesesconcerning the regression parameters when over-dispersion is present?(d) Given the amounts of over-dispersion in the emergency room visits series, are the estimatesof the model parameters and standard errors accurate?In this chapter we outline the purpose and procedures of a simulation study designed toaddress these questions. The primary goal of the simulation study was to examine the behaviorof the model reduction test statistics described in Section 2.2 in contexts similar to the PrinceGeorge study (items (b) and (c)). Secondary goals were to examine the performance of estimatesof over-dispersion parameters as well as of estimates of regression coefficients and their standard31Chapter 3. A Simulation Study Related to the Prince George Study^ 32errors (item (d)). Of particular concern were the effects of varying amounts of over-dispersionand misspecification of the variance function on this performance.In particular, we will:1. Compare the scaled deviance, score and Wald tests for model reduction.2. Compare the parameter estimates to the simulated values.3. Compare empirical and model-based estimated standard errors to the standard deviationsof the estimated parameters.However, as an essential preliminary to the main thrust of our simulation study, we mustfirst consider the question of how the over-dispersion parameter should be estimated (item (a)).As already mentioned, two possibilities are via G 2 /d.f. (as was done in the Prince Georgestudy) and X 2 /d.f. Results presented in Section 4.1 provide a clear indication that the latter ispreferred, and this estimate is therefore employed in all subsequent work. In particular, the onlyversion of the scaled deviance model reduction test considered is the one where 0 is estimatedby X 2 /d.f. Both empirical and model-based versions of the Wald statistic will be evaluated, butonly the empirical score test will be evaluated; the latter test is reported to perform well evenin the presence of over-dispersion and with misspecified variances (see Breslow, 1990). To carryout these comparisons, a variety of data configurations, all intended to be qualitatively similarto the context of the Prince George study, will be simulated. Different data configurationswill correspond to different log-linear models for the mean levels of the observed counts anddifferent amounts of over-dispersion. In all cases, the model fit to the simulated data will bethat of over-dispersed Poisson regression; that is, estimation of the regression coefficients willChapter 3. A Simulation Study Related to the Prince George Study^ 33be based on the estimating equations (2.14) which result from an assumed variance function ofthe form V(iii;^= cbtti.The simulation study will not be concerned with the possible misspecification of the log-linear model for pi (the regression function), but will examine the effects of misspecification ofthe variance function. For this purpose we define the "correct" variance to be V(pi; 0) = themodel to be used in the fitting, and the "misspecified" variance to be V(iii; 0) = + (0— 1)4.The latter variance function arises from thinking of the datum yi as being sampled from adistribution which is, conditional on the value of an unobserved variable Ai, Poisson with meanAi; if Ai is considered to be sampled from a distribution with mean pi and variance (0 — 1)4,then the marginal distribution of yi has mean pi and variance pi + (0 — 1)4.This simulation study differs in several respects from a study carried out in a differentcontext by Breslow (1990). Firstly, Breslow's study only considered simulated data with meanvalues no smaller than 2. In the Prince George study the Asthma series had many zero valuesand a mean value of only about 0.75. In the current study we wish to investigate the possibilitythat such a series could lead to poor performance of the estimators or test statistics. Anotherdifference is that Breslow was concerned with a 3x4 factorial design with n replicates per cell(and a single continuous covariate) whereas our study will be concerned with one or morelong data series similar to the series available for the Prince George study. This differenceis perhaps not as great as it first appears because all the data in Breslow's simulations weregenerated under the hypothesis that the column factor of the 3x4 design had no effect. Theresult is that, in effect, three data series of length 4n are being considered and for n = 144(the largest value Breslow considered) the length of the series is comparable with the lengthChapter 3. A Simulation Study Related to the Prince George Study^ 34we will employ (700). The last key difference is that the current study is only concerned withtest statistics and estimating equations based on an assumed variance function V(µ1; 0) =Opi. We will examine the performance of these test statistics and estimating equations usingdata generated under "correct" and "misspecified" variance functions, both for a single dataseries and for three series with differing amounts of over-dispersion (to represent the separatediagnostic categories), keeping in mind that the methods used will provide only a single overallestimate of 0. In contrast, Breslow generated each of his data sets under the variance functionV(pi; 0) = pi + (0 — 1)4 and then examined the performance of the estimating equations andtest statistics based on this variance function and also based on V(pi; 0) =3.1 The General Simulation ProcedureThe simulation procedure, or sampling experiment, consists of three phases: generating thedata, fitting a full model, and fitting one or more reduced models. All calculations are carriedout in the programming language C.3.1.1 Generating the DataThe data generated are to represent the main qualitative features of the data analyzed in thePrince George study. To accomplish this we will generate time series of 700 independent over-dispersed Poisson observations. We may choose to generate a single series (Sections 4.2 and4.4) or we may generate three separate series (Sections 4.3 and 4.5).Each data set to be generated requires a choice of a log-linear model for the mean pi of theform log pi = XiO, a choice of the variance function to be used in the simulation, and a choiceChapter 3. A Simulation Study Related to the Prince George Study^ 35of the value for the over-dispersion parameter 0.Over-dispersion in the data is introduced through gamma-mixtures of Poisson random vari-ables. To generate the datum yi, it is most convenient to think of our situation as follows.Let log = Xifi and let there be a random variable vi which has a gamma distribution withmean 1 and variance^Then for each i=1,...,700 we first generate a realization of the gammarandom variable IA and then generate a realization of a Poisson random variable with parametervi pi. The resulting observations are negative binomial, with mean pi and variance pi +The choice V = (0 — 1)/pi leads to simulated data with the "correct" variance function Ofti,whereas the choice Vi = (0 — 1) leads to simulated data with the "misspecified" variance func-tion pi + (0 — 1)4. Note that both variance functions reduce to the usual Poisson variancewhen 0 = 1, and in this case the Poisson variates were generated directly.3.1.2 Fitting the Full ModelWe fit the full model to obtain estimates of the parameters Shin using an iterative method forsolving the estimating equations (2.14) of Chapter 2. We then estimate 0 using the PearsonX2 statistic divided by its degrees of freedom. In contrast to Breslow (1990), when we obtainan estimate of 0 which is less than one, we do not adjust that estimate to be equal to one.In our situation the dispersion parameter is only a nuisance parameter used to account fordispersion. It is not necessarily thought to arise from the variance of a mixing distribution(which would require values of 0 > 1); that is to say, under-dispersion is not ruled out. Usingthis estimate, both the empirical and the model-based estimates of the covariance matrix ofSfun are calculated (allowing the empirical and model-based Wald tests to be evaluated), asChapter 3. A Simulation Study Related to the Prince George Study^ 36well as the scaled deviance under the full model.3.1.3 Fitting the Reduced ModelNext, the reduced model is fit to obtain estimates of the parameters fired. The empiricalestimate of the covariance matrix of fired is calculated and used in the calculation of the scorestatistic. We also calculate the scaled deviance using the estimate of 0 from the full model sothat the scaled deviance statistic for testing the model reduction may be computed.The general procedure described above of generating data, fitting full and reduced models,calculating estimates and standard errors, and calculating the model Wald, empirical Wald,empirical score and scaled deviance test statistics is repeated 1000 times for each particularspecification of the log-linear model, choice of V (pi; 0) and value of q4.In the following section the specific log-linear models, and the choices for the parametersin these models, as well as the levels of over-dispersion utilized in this simulation study arediscussed.3.2 The Specifics of the Simulation StudyAs mentioned in Section 3.1.1, we may choose to simulate either a single series of data, orthree separate series to be modelled simultaneously. The single series simulations are smallenough, in terms of the computing time required to generate the data and fit a model, to allowan investigation of the effects on the estimators and test statistics of varying both the level ofdispersion and the simulated mean values. The three series simulations recreate an importantfeature of the data analysed in the Prince George study, namely separate series with possiblyChapter 3. A Simulation Study Related to the Prince George Study^ 37different amounts of dispersion (we chose to use three series here instead of four as in the PrinceGeorge study simply to reduce the computing time).The overall mean values of the series and the levels of over-dispersion to be considered inthese simulations (whether in a single series or in three) was based on the data series from thePrince George study. To recap the approximate mean values of those series and to get a roughidea of how much dispersion was present, we have the following table:mean variance variance/meanAsthma 0.76 0.80 1.05Bronchitis+ 1.54 3.36 2.18Ear 3.62 6.92 1.91Other 5.13 14.2 2.77From the above, the values 0.75, exp{0.7}(Pe, 2.01) and exp{1.6}(::::. 4.95) were chosen aspossible overall mean values for the series to be simulated. In the single series simulations wewill carry out sampling experiments using each of these values, and we will use the three valuestogether as the overall mean levels in the three series simulations.Based on the third column of the above table, the values 1, 2 and 3 seem to be reasonablelevels of over-dispersion to consider, although the estimates of 0 for each series would be smallerthan the ratio of the variance to the mean (sf/) = E (yi )2 is less than variance/meanu-T-1 i= > ; (y= -y)2 /y in general). We also wish to simulate the value 1.4 since this is very close tothe overall estimate of the dispersion parameter from the Prince George study, and a simulatedvalue 0 = 5 is of possible interest as an "extreme" amount of over-dispersion.In the single series simulations, sampling experiments using each of these levels of dispersionare carried out at each possible mean level, with the exception that 0 = 5 is only simulatedwhen the mean level is exp{1.6} (using 0 = 5 with lower mean values would result in series withChapter 3. A Simulation Study Related to the Prince George Study^ 38mostly zeros and only a few large values; this situation is too extreme and is not of particularinterest in this thesis). In the three series simulations we chose two different combinations ofdispersion parameters. The first is the triplet (1,2,3) which we refer to as combination I. Thesevalues seem like a plausible representation of the level of over-dispersion encountered in thePrince George study. We also consider the triplet (1.4,2,5), which we refer to as combinationII, in order to examine the performance of the estimators and test statistics in a situation witha large amount of dispersion (relative to the amounts encountered in the Prince George study).Thus sampling experiments will be carried out using the following choices of overall meanvalues and dispersion parameters:Single Series SimulationsMean Value 1 1.4 2^3 50.75exp{0.7}exp{1.6}^VA/VA/^xxx= no sampling experiment carried outThree Series SimulationsCombination of 0Mean Values^I=(1,2,3) I1=(1.4,2,5)(0.75,exp{0.7},expl1.61) ,/ ^The models used for generating the data should, in some sense, reproduce the main featuresof the data encountered in the Prince George study. We first describe some very simple modelsthat are used to generate single series of data before we attempt to reproduce the more com-plicated features of the Prince George study data. The simplest models to be considered arefor a single series with the mean value of the j th observation given by//2 = exP{O 7x2 },^j^1, ..., 700^(3.16)Chapter 3. A Simulation Study Related to the Prince George Study^ 39where^is the overall effect and -y is the coefficient corresponding to the covariate xi =sin(4irj/700), used to simulate a pollution variable with a regular annual cycle. To simulatedata under the reduced model, one with no pollution, the choices for the simulated coefficientsare = log0.75, 0.7, or 1.6 (with y = 0), corresponding to series with overall mean levels 0.75,exp{0.7}, and exp{1.6} respectively. To simulate data under the full model, one with pollutionincluded, only /3 log 0.75 is used but three different values of -y are considered (7 = 0.05, 0.10or 0.15). The simulations using models of the form (3.16) to generate data are referred to asthe one series simple case.These simple models are also extended to a three series simple case with the mean of thei th observation in the ith series given bypii = exp{/3i -yoxi 7ixi}^i = 1, 2, 3; j = 1, ..., 700.^(3.17)where Oi is the overall effect for the ith diagnostic category, 7 0 is the common pollution effectand yi is the pollution effect specific to the ith diagnosis (with 73 0 for identifiablility).This parameterization with a common and separate pollution effects is utilized to facilitate thescore test by allowing the hypothesis of common pollution effects to be stated as H: -yi = 0for i = 1,2,3. For the alternate parameterization^= 70 + -yi for i = 1,2,3, the samehypothesis would take the more usual form of H: yi = Yz =^(which can be tested bythe Wald or scaled deviance tests). In these simulations the simulated values of the^are= log 0.75, 02 = 0.7 and /33 = 1.6. The null model (no pollution effects) follows from setting7o = 71 = 72 = 73 = 0. We may also simulate data sets with a common pollution effect (weuse 70 = 0.1, 71 = 72 = 73 = 0), or with separate pollution effects (we use 70 = 0.1, 71 = 0.1,Chapter 3. A Simulation Study Related to the Prince George Study^ 4072 = —0.2 and 73 = 0).The j th observation in the more complicated models with a single series has mean describedby4itj = eXp{O^7-ksk + ^yxj}k=1(3.18)where the sk are indicators designed to represent four "seasons" of equal length (±1 day),within each of our two "years" of 350 observations and xi is the sinusoidal pollution covariatejust as in the simple case. The seasonal structure is the only attempt made in this thesis torecreate the temporal structure of the data in the Prince George study and no attempt is madeto recreate the meteorological structure. For the purpose of simulating data under the reduced(or null) model, the values r 1 = 0.25, r2 = 0.25 and r3 = —0.5 (with r4 = 0 for identifiability)were used with one of /3 = log 0.75, 0.7, or 1.6. For the full model (one with pollution included)only /3 = log 0.75 is used with one of 7 = 0.05, 0.10 or 0.15. The simulations based on modelsdescribed by (3.18) include seasonal effects and are referred to as the one series complicatedcase.As with the simple simulations, the one series complicated models are extended to a threeseries complicated case with the mean for the j th observation in the ith series given by4= exp{3i E Tiksk + -yoxi + 7ix.; }k=1(3.19)where /3, is the overall effect for the i th diagnostic category, rik is the effect of "season" k indiagnosis i, -yo is the common pollution effect, and 7i is the pollution effect specific to the ithdiagnostic category. The values of the coefficients used in the simulations are:Chapter 3. A Simulation Study Related to the Prince George Study^ 4101 = log 0.75 = 133 = 1.6= 0.2 T21 = 0.2 T31 = 0.2T12 = 0.2 T22=0.20. 7-32 == 0.3T13 = —0.2T14 =00T23 = —0.4 T33 = —0.4r24 = 0^T34 = 0There were no compelling reasons for the above choices for the season effects other thanto make them separate effects; that is, different for each series. As in the three series simplecase, the null model (no pollution) corresponds to 7o = 7 1 = 72 = -y3 = 0 while the commonpollution effect model uses simulated values 7o = 0.1, 7 = 72 = -y3 = 0, and the separatepollution effects model uses -yo = 0.1, 7 1 = 0.1, 72 = —0.2 and 73 = 0.The results of fitting the models from each of these four cases (with the various possiblechoices of simulated coefficients, variance function, and dispersion parameter .0) are presentedin Chapter 4. We begin Chapter 4 with a comparison of the possible estimators of q5 (as theremainder of the study depends on a choice of estimator of 0). For this comparison, only the oneseries simple case models were needed as these provide a strong indication that the estimatorX2 /d.f. is preferable to G2/d.f.Chapter 4Results of the Simulation StudyIn this chapter we describe the results of the investigation of the performance of the estimatorsand test statistics. We begin with the comparison of possible estimators for 0.4.1 Estimating the Dispersion Parameter 0Of particular interest in this comparison are any shortcommings in the performance of theestimator ii)G = Geld. f. used in the Prince George study and the possibility that use of thealternate estimator 'cbx = X 2/d.f. in the scaled deviance model reduction test statistic mighthave led to different conclusions than those sketched in Chapter 1.For this investigation single series of data were generated according to the simple modelgiven by (3.16) using one of the three possible values of 0. Only the reduced model (set 7 = 0)was used to generate data, so that pi does not depend on i. As we shall see, the results in eventhis most simple of cases indicates the estimator cbx = X 2/d.f. is preferable. Table 1 presentsthe mean values and standard deviations of the estimates of 0 obtained from the two estimatorsbased on the results of fitting the reduced model. Part A of the table contains the results whenthe correct variance function is used to simulate the data, while part B presents the results forthe misspecified variance function, along with the values Os where 03,ui = + (0 — 1)4. Thevalues 08 are the estimated values of 0 we should see if our incorrectly specified model is still42Chapter 4. Results of the Simulation Study^ 43correctly accounting for the extra variability; that is, 0 8 is the amount of dispersion in the datain terms of the variance function V(//i; = 0/ii used in the estimating equations.Table 1. Mean Values of Estimates (± Standard Deviation) in 1000 Simulated Data Sets.A. Correct Variance Function.Simulated 0Simulated Q Estimator 1^1.4^2^3^5log 0.75^OXEGG0.70^4x:(4G1.60(kG1.002 (.055)1.108 (.039)0.998 (.056)1.137 (.059)0.998 (.052)1.046 (.056)1.401 (.096)1.349 (.057)1.400 (.084)1.515 (.074)1.399 (.076)1.447 (.075)2.001 (.174)1.642 (.086)2.002 (.134)2.010 (.097)2.006 (.118)2.035 (.108)2.996 (.337)2.018 (.136)2.983 (.235)2.700 (.139)3.001 (.208) 4.985 (.378)2.938 (.162) 4.537 (.246)B. Misspecified Variance Function.Simulated 0Simulated /3 Estimator 1^1.4^2^3^51^1.3^1.751.000 (.053) 1.300 (.086) 1.749 (.135)1.107 (.038) 1.292 (.051) 1.526 (.072)1^1.806^3.0141.005 (.054) 1.799 (.114) 2.992 (.248)1.143 (.057) 1.854 (.088) 2.705 (.147)1^2.981^5.9531.002 (.054) 2.979 (.201) 5.948 (.503)1.050 (.058) 2.919 (.159) 5.226 (.310)2.52.497 (.255)1.844 (.112)5.0285.041 (.535)3.823 (.244)10.90610.867 (1.14)8.150 (.552)log 0.750.701.60Os93XOGcx(11GOsx20.81220.66 (2.95)12.17 (1.08)In part A of the table the estimator cx provides good estimates of 0, on average, at all levelsof dispersion. The estimator •G appears to slightly overestimate 0 at low levels of dispersionand underestimate 0 in the presence of large amounts of dispersion. The largest differences inthe estimators occur when the level of dispersion is very high relative to the mean value (see,for example, the simulations with /3 = log 0.75 and 0 = 3). In part B the estimator -ciSx yieldsChapter 4. Results of the Simulation Study^ 44good estimates of Os, on average, whereas (0G produces very poor estimates when the level ofover-dispersion is high. Note that in both parts of Table 1, except perhaps when 0=1, theestimator 4r0x generally appears substantially more variable than qG. This suggests that forsome of the simulated data sets, (0x may differ noticably from the simulated value of 0 (andmay be worse than (0G) although on average it is clearly superior to ;M.To answer the key question of how these differences affect the model reduction test statisticswe use the above data sets (generated under the simple model log pi = /3) to evaluate the scaleddeviance statistic using either ii5G, or cbx based on the full model given by (3.16). This modelreduction, a test of the hypothesis H: y = 0, is intended to be an over-simplified representationof the model reductions tested in the Prince George study. The plots in Figures 1, 2, 3 and4 present the values of three test statistics; the scaled deviance using C60G, the scaled devianceusing 4x and the empirical score test (which does not depend on an estimate of 0, see Section2.2), versus the quantiles of the chi-squared distribution with 1 degree of freedom. Resultsare presented only for a simulated mean of 0.75 and for 0 = 1,1.4,2 and 3; the results forthe other combinations of and 0 are qualitatively similar. These plots are intended to judgehow well the test statistics agree with their predicted asymptotic distribution. The score testis included to allow comparison with a test statistic that has been reported (in a somewhatdifferent context) to perform reasonably well even in the presence of over-dispersion (Breslow1990). This statistic is also useful for comparison in this situation because it does not involvean estimate of the dispersion parameter.We see that for data sets generated under either the correct variance function (Figures 1and 2) or the misspecified variance function (Figures 3 and 4), the use of the estimator 4G2 4 10102^4^8quanfilos of chi square(1)Figure 1. Comparison of Estimators of Phi: Correct Variance Simulations (Phi=1 or 1.4)G2 estimator of phi^ X2 estimator of phi^ Score testmean=0.75 and phf=1 mean=0.75 and phi=1 mean=0.75 and phi=1quandlos of chi square(1)Score testmean=0.75 and phi=1.430 2 10 a4^6coandlos of chl square(1)4^6quandoo of chi square(1)quantiles ol chi squaro(1)G2 estimator of phimean=0.75 and phi=1.4wand's, of chi square(1)X2 estimator of phimean=0.75 and phi-1.420 104^6gaaFigure 2. Comparison of Estimators of Phi: Correct Variance Simulations (Phi=2 or 3)G2 estimator of phi^ X2 estimator of phi^ Score testmean=0.75 and phiL-2 mean=0.75 and phi=2 mean=0.75 and phi=22 a4^6wand's' of NI square(1)0 100 2 10 2 10 0 2 4^6quantles of chi scpare(1)Score testmean=0.75 and phi=34quandlos doh! swam(1)G2 estimator of phimean=0.75 and phi=34^6quanolos of chi square(l)X2 estimator of phimean=0.75 and phi=3a 104^6quintiles ol chl aquare(1)Figure 3. Comparison of Estimators of Phi: Misspecified Variance Simulations (Phi=1 or 1.4)G2 estimator of phi^ X2 estimator of phi^ Score testmean=0.75 and phi=1 mean=0.75 and phi=1 mean=0.75 and phi=10 2^4^6quantiles of chl awaraotScore testmean-0.75 and phi=1.40 22 4^6 a 10 0 4 100 4^0cafantlles of chl square(1)4^6quintiles of chl square(1)2 10 2 10finales of chl square(1)X2 estimator of phimean=0.75 and phi=1.4wimples of chi squaro(1)G2 estimator of phimean=0.75 and phi=1.4Oguantles of chl square(1)X2 estimator of phimean=0.75 and phi-3a 10 2 10 2 102 4guanalss of chl eguars(1)4^6giant's' of chl square(1)64guanakts of cfil ' ,wars( I)Figure 4. Comparison of Estimators of Phi: Misspecified Variance Simulations (Phi=2 or 3)G2 estimator of phi^ X2 estimator of phi^ Score testmean=0.75 and phi.2 mean=0.75 and phi=2 mean=0.75 and phi=21084^62 2 4 a 10guandles of chl scpace(t)Score testmean-0.75 and phi=3guantlIss of chi square(I)G2 estimator of phimean=0.75 and phi=3Chapter 4. Results of the Simulation Study^ 49causes the scaled deviance test statistic to reject too often at higher levels of over-dispersion.The scaled deviance test using the estimator ii)x and the empirical score tests agree reasonablywell with their predicted asymptotic distributions, except for obvious departures at the mostextreme values. Similar plots for the other values of Q show the same departures of the scaleddeviance statistic using (1)G from its predicted asymptotic distribution while the scaled devianceusing crSx and the empirical score tests perform well.Recall that the estimator ikx is the most natural estimator of the two, arising as the momentestimate of 0, whereas cbG was used because it was more convenient and thought to generallyagree closely with i4x. The limited results presented above indicate thatG may not agree wellwith ck x and that the latter is preferable, especially when substantial amounts of dispersionare present in the data. Therefore, in the simulations which follow, when an estimate of 0 isrequired we will always use 0x.With reference to the Prince George study, the question that arises is: Would this improvedestimator have led to different conclusions? It appears that use of 0x would not have changedthe nature of the results outlined in Chapter 1. For the pollutant TRS, in the situations wherewe rejected the null hypothesis in favor of a more complex alternative, i•x was evaluated andfound to be smaller than QG (although the results of Table 1 would suggest that qX is generallylarger than cbG), thus the null hypothesis would still be rejected. It appears quite likely themodel reduction process would have led to the same final models and very similar final sets ofestimates.We now begin to examine the performance of the estimators for parameters and their stan-dard errors, and the test statistics outlined in Section 2.2.Chapter 4. Results of the Simulation Study^ 504.2 The Simple Case with a Single SeriesFor a single series we generate data under the reduced model (-y = 0) corresponding to equation(3.16) at the different possible mean levels and amounts of over-dispersion to be considered.Table 2 presents the mean values of the estimated regression coefficient under the reducedmodel.Table 2. Mean Values of Estimates of O.A. Correct Variance Function.Simulated 0Simulated /3 1^1.4^2^3 5log 0.75 1 -0.287^-0.288^-0.288^-0.2890.7 0.699^0.700^0.699^0.6961.6 1.601^1.600^1.600^1.599 1.598B. Misspecified Variance Function.Simulated 0Simulated /3 1^1.4^2^3 5log 0.751 -0.288^-0.289^-0.290^-0.2870.7 0.700^0.698^0.695^0.7021.6 1.600^1.601^1.600^1.598 1.5921 log 0.75;..-, -0.288For the simulations under the correct variance function, the estimates of 0 are very closeto the simulated values, even in the presence of substantial amounts of dispersion; the same istrue in the misspecified variance simulations. This may not be surprising since in this situationthe estimate of it = exp{/3} is just an average of the observed data.Table 3 summarizes the estimates of the variability of the estimates of 0 under the reducedmodel. In this table, "Simulated" standard error refers to the standard deviation of the 1000parameter estimates. The details of parts A and B of Table 3 are very similar. In all cases theempirical and model-based estimates are identical to three decimal places - even for extremeChapter 4. Results of the Simulation Study^ 51amounts of over-dispersion. Nor is any difference in the variability of these estimates apparent.Further, these estimated standard errors accurately reflect the actual standard deviations, evenwhen the variance function is misspecified. The agreement seen here between the simulated,empirical and model-based estimates of variability is somewhat surprising. It was expected thatthe model-based estimators of the covariance matrices would begin to yield poor estimates asthe dispersion increased, but this is not the case. It appears that in this simple situation theover-dispersion in the data is being adequately accounted for, even when the variance functionis misspecified; the quantity Os presented in part B of Table 1 was estimated very well andserves the purpose of accounting for the extra variability.Table 3. Standard Deviations of Estimates of 0and Mean Values of Estimated Standard Errors (±s.d.).A. Correct Variance FunctionSimulated /3 MethodSimulated q51 1.4 2 3 5log 0.75 Simulated .043 .051 .062 .077Model .044 (.002) .052 (.002) .062 (.003) .075 (.004)Empirical .044 (.002) .052 (.002) .062 (.003) .075 (.004)0.70 Simulated .026 .031 .037 .046Model .027 (.001) .032 (.001) .038 (.001) .046 (.002)Empirical .027 (.001) .032 (.001) .038 (.001) .046 (.002)1.60 Simulated .017 .021 .024 .030 .039Model .017 (.001) .020 (.001) .024 (.001) .029 (.001) .038(.001)Empirical .017 (.001) .020 (.001) .024 (.001) .029 (.001) .038(.001)Chapter 4.^Results of theTable 3 B. Misspecified VarianceSimulated /3^MethodSimulation StudyFunctionSimulated q5521 1.4 2 3 5log 0.75 Simulated .042 .050 .059 .069Model .044 (.002) .050 (.002) .058 (.002) .069 (.003)Empirical .044 (.002) .050 (.002) .058 (.002) .069 (.003)0.70 Simulated .026 .036 .048 .057Model .027 (.008) .036 (.001) .046 (.002) .060 (.003)Empirical .027 (.008) .036 (.001) .046 (.002) .060 (.003)1.60 Simulated .017 .031 .041 .056 .081Model .017 (.001) .029 (.001) .041 (.002) .056 (.003) .077 (.005)Empirical .017 (.001) .029 (.001) .041 (.002) .056 (.003) .077 (.005)We now turn to the performance of the test statistics for the hypothesis H: y = 0, summa-rized in Table 4. The observed rejection probabilities presented in this table can be consideredas averages of 1000 Bernoulli trials with success probability p, which we may approximate bythe nominal level. Thus the standard errors of the observed probabilities are approximatelyVp(1 - p)/n; for nominal levels of 0.10, 0.05 and 0.01 we may expect standard errors of approx-imately .009, .007 and .003. For the results in part A of Table 4, keeping the precision of theseobserved rejection probabilities in mind, we find that they agree quite well with the nominallevels at all mean values and even at higher levels of dispersion. We also note the observedrejection probabilities are very similar across the four tests for each choice of mean and dis-persion suggesting that any deviations of the observed rejection probabilities from the nominallevels are a reflection of the particular 1000 data sets generated. The agreement between modeland empirical Wald tests was to be expected given the agreement of the two estimators of thevariances seen in Table 3. In general, the agreement among the four statistics may not beparticularily surprising either. Conventional wisdom would suggest that, over-dispersion aside,Chapter 4. Results of the Simulation Study^ 53each of the tests should perform well with a sample size of 700 (although there was some ques-tion if this was true when the series consist of a large number of zeros such as in the simulationsusing /3 = log 0.75). Thus, in light of how effectively the over-dispersion is accounted for, weshould not expect any of the test statistics to do poorly.The results in the misspecified variance simulations are similar. In these simulations theagreement between the four tests may seem particularly surprising at first; we might expect themodel-based Wald and the scaled deviance tests to do poorly since they rely on the particularchoice of variance function used to fit the data. However, considering how the choice of variancefunction does not seem to matter in this simple case (as noted for Table 3), the above agreementshould not be surprising.Chapter 4. Results of the Simulation StudyTable 4. Observed Rejection Probabilities Under Reduced Model.54A. Correct Variance Function0 = 1 0 = 1.4 0 = 2Simulated 0 Test/Methodlog 0.75^Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled0.70^Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled1.60^Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled0.10 0.05 0.01.105 .041 .010.105 .040 .010.102 .040 .010.106 .041 .010.103 .056 .005.104 .055 .006.102 .054 .005.103 .056 .005.112 .056 .011.110 .057 .012.108 .057 .012.112 .056 .0110.10 0.05 0.01.096 .045 .007.097 .046 .005.095 .045 .005.096 .045 .008.082 .046 .012.083 .046 .011.082 .046 .011.082 .046 .013.095 .043 .004.097 .043 .004.090 .038 .007.087 .039 .0070.10 0.05 0.01.106 .053 .015.114 .056 .016.111 .056 .016.108 .054 .015.110 .062 .017.111 .061 .017.110 .060 .014.110 .062 .017.087 .039 .007.091 .038 .008.090 .038 .007.087 .039 .0070 = 3^0 = 5Simulated )3 Test/Methodlog 0.75^Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled0.70^Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled1.60^Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled0.10 0.05 0.01.119 .056 .009.122 .059 .011.117 .054 .009.121 .056 .011.114 .061 .013.122 .065 .012.119 .063 .011.114 .061 .013.103 .043 .010.096 .048 .013.095 .045 .010.103 .043 .0100.10 0.05 0.01.092 .043 .009.090 .043 .009.088 .043 .008.092 .043 .009Chapter 4. Results of the Simulation Study 55Table 4 B. Misspecified Variance Function0 = 1 Simulated 0 Test/Method^0.10 0.05 0.01log 0.75^Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled.094 .045 .009.095 .047 .009.095 .046 .009.094 .045 .009cb = 1.40.10 0.05 0.01 0.10 0.05 0.01q=22.094 .044 .006.095 .046 .007.094 .046 .006.095 .044 .006.091.092.091.093.045.050.047.046.009.012.010.0090.70 Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled1.60 Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled.091 .050 .016.091 .051 .016.092 .051 .015.091 .050 .017.097 .041 .007.097 .039 .009.096 .039 .007.097 .041 .007.095 .048 .012.091 .047 .012.091 .046 .011.095 .048 .012.103 .052 .009.105 .051 .010.103 .050 .009.103 .052 .010.107 .046 .010.108 .046 .009.108 .043 .007.109 .047 .010.083 .033 .004.081 .034 .005.079 .033 .004.083 .034 .0040 = 3 0 = 5Simulated 0 Test/Methodlog 0.75 Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled0.70 Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled1.60 Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled0.10 0.05 0.01 0.10 0.05 0.01.112 .063 .014.114 .066 .017.110 .062 .015.112 .064 .015^.101 .044 .010^-.102 .048 .011.099 .042 .009 ^-^.101 .045 .010^-.111 .059 .014 .102 .056 .011.114 .066 .016 .104 .053 .012.108 .059 .013 .100 .047 .006.111 .059 .014 .102 .056 .012In summary, for the simulation results when the data is generated under the reduced modelthere is no clear difference between the empirical and model-based estimates of variances. Norare there any differences among the four model reduction test statistics considered. We willlook to the three series case in Section 4.3 to provide more insight into possible differences inthe estimators and test statistics in this simple situation.Chapter 4. Results of the Simulation Study^ 56A limited number of simulations were also carried out using the full model correspondingto (3.16) to generate the data. In particular, 1000 data sets were generated according to thismodel with = log 0.75 and one of y = 0.05, 0.10, or 0.15. The estimation of 0, fl and y were ofinterest as well as power for testing the hypothesis 11:7 = 0. The mean values of the parameterestimates are summarized in Table 5. First consider part A of the table, corresponding tosimulations where the data were generated with the correct variance function. In all cases (thatis, all combinations of simulated 0 and 7) the estimators yield good estimates. The mean valuesof both model and empirical versions of the estimated standard errors (not presented here) arein close agreement with the displayed standard deviations. The standard errors of the estimatesincrease as the dispersion increases as would be expected. Notice that the estimates of 0 (andtheir standard deviations) are almost identical to the corresponding quantities for qx in partA of Table 1 (which were based on data generated using the reduced model).In part B, we see that # and -y are estimated very well at all levels of dispersion considered.To determine what the estimates of 0 should be, first note that in these misspecified variancesimulations, the amount of extra variability varies as the covariate xj, and thus the mean valuespi, vary. In the present situation, the average of the 0 .3 (where pick; = pi + — 1)/2) for eachcell in part B of Table 5 are the same to two decimals as the values 4r from part B of Table 1.It is therefore reasonable that the estimates of 0 are close to the estimates in part B of Table1, and again cix is accounting for the extra variability very well.Chapter 4. Results of the Simulation Study^ 57Table 5. Mean Values of Estimated Parameters (± s.d.) Under Full Model.A. Correct Variance FunctionSimulated 0Simulated 7 Parameter 1 1.4 2 30.05 0 1.00 (.055) 1.40 (.095) 2.00 (.178) 2.97 (.335)/3 -.288 (.044) -.289 (.050) -.292 (.064) -.292 (.076)7 .049 (.061) .051 (.075) .050 (.087) .053 (.109)0.10 0 1.00 (.054) 1.40 (.093) 1.99 (.179) 2.97 (.323)/3 -.288 (.042) -.290 (.051) -.291 (.063) -.293 (.076)7 .100 (.063) .098 (.073) .099 (.086) .102 (.110)0.15 0 1.00 (.054) 1.40 (.096) 2.00 (.171) 2.98 (.323)0 -.288 (.043) -.289 (.051) -.288 (.063) -.293 (.075)7 .151 (.062) .149 (.072) .150 (.085) .153 (.109)B. Misspecified Variance FunctionSimulated 0Simulated 7 Parameter 1 1.4 2 30.05 0 1.00 (.055) 1.30 (.083) 1.75 (.133) 2.49 (.260)/3 -.288 (.044) -.290 (.050) -.292 (.059) -.292 (.069)7 .049 (.061) .051 (.069) .055 (.084) .053 (.098)0.10 0 1.00 (.054) 1.30 (.082) 1.75 (.134) 2.49 (.258)/3 -.288 (.042) -.290 (.050) -.290 (.056) -.294 (.069)7 .100 (.063) .102 (.068) .099 (.083) .104 (.102)0.15 0 1.00 (.054) 1.30 (.084) 1.75 (.137) 2.50 (.255)/3 -.288 (.043) -.291 (.049) -.291 (.058) -.291 (.070)7 .151 (.062) .151 (.071) .150 (.081) .151 (.100)The observed rejection probabilities under the alternative hypothesis are shown in Table 6below. The results presented in part A show no difference in the power of the four statistics.For a given value of y, the observed rejection probabilities decrease as the amount of dispersionincreases (the increased variability in the data makes it more difficult to detect an effect).The results in part B are very similar. In general, for given values of y and 0 > 1, therejection probabilities are slightly higher in the misspecified variance simulations compared tothe correct variance simulations. Note that for values of pj < 1, such as in these simulationsusing = log 0.75 and y no larger than 0.15, the value of pi + (0 - 1)p .1 (the variance underChapter 4. Results of the Simulation Study^ 58the misspecified formulation) is less than the value of cbiti (the variance under the correctformulation). Therefore from our earlier observation that the power decreases as the amount ofvariablility in the data increases, we should expect the rejection probabilities in the misspecifiedvariance to be higher than the correct variance simulations for these values of y and 0.In summary, from the results of the simulations under the full model (Tables 5 and 6) theestimates of 0 appear reasonable and the regression parameter estimates are close, on average, tothe simulated values. There are also no indications of any differences in the power of the fourtest statistics considered. Further, estimated standard errors from the model and empiricalestimators (not presented) were similar, both being very close, on average, to the simulatedvalues.Chapter 4. Results of the Simulation Study^ 59Table 6. Observed Rejection Probabilities Under Full Model.A. Correct Variance Function7 Test0 = 1 0 = 1.4 0 = 2 0 = 3.10 .05 .01 .10 .05 .01 .10 .05 .01 .10 .05 .01.05 Wald/Mod. .205 .117 .033 .194 .114 .041 .152 .090 .025 .152 .089 .025Wald/Emp. .204 .117 .035 .197 .119 .040 .155 .090 .027 .151 .086 .025Score/Emp. .202 .116 .034 .197 .118 .038 .154 .086 .024 .147 .079 .024Dev/Scaled .205 .117 .033 .194 .117 .041 .153 .092 .026 .156 .092 .025.10 Wald/Mod. .490 .374 .164 .348 .247 .116 .295 .205 .070 .253 .165 .059Wald/Emp. .489 .372 .171 .351 .253 .119 .298 .207 .076 .266 .159 .065S core/ Emp. .487 .371 .164 .353 .249 .111 .296 .203 .072 .260 .156 .060Dev/Scaled .490 .374 .164 .349 .247 .117 .296 .207 .074 .253 .166 .060.15 Wald/Mod. .786 .685 .445 .661 .522 .296 .529 .404 .186 .391 .274 .135Wald/Emp. .788 .687 .445 .663 .521 .297 .527 .411 .187 .402 .274 .136Score/Emp. .788 .683 .439 .662 .518 .287 .527 .408 .182 .395 .269 .128Dev/Scaled .786 .685 .448 .663 .523 .300 .531 .405 .190 .391 .277 .136B. Misspecified Variance Function0 = 1 0 = 1.4 0 = 2 0 = 3y Test .10 .05 .01 .10 .05 .01 .10 .05 .01 .10 .05 .01.05 Wald/Mod. .205 .117 .033 .193 .111 .024 .176 .108 .030 .165 .090 .021Wald/Emp. .204 .117 .035 .195 .111 .028 .174 .112 .032 .161 .087 .020Score/Emp. .202 .116 .034 .189 .107 .026 .169 .109 .030 .157 .082 .018Dev/Scaled .205 .117 .033 .194 .111 .025 .177 .110 .032 .165 .093 .021.10 Wald/Mod. .490 .374 .164 .407 .291 .123 .336 .236 .087 .291 .195 .073Wald/Emp. .489 .372 .171 .410 .299 .131 .333 .241 .097 .294 .203 .074S core/ Emp. .487 .371 .164 .409 .300 .122 .328 .233 .088 .291 .192 .066Dev/Scaled .490 .374 .164 .410 .292 .123 .336 .237 .088 .291 .195 .073.15 Wald/Mod. .786 .685 .445 .680 .579 .337 .585 .455 .218 .470 .340 .158Wald/Emp. .788 .687 .445 .683 .574 .342 .587 .455 .222 .481 .342 .162Score/Emp. .788 .683 .439 .681 .565 .333 .584 .450 .209 .473 .333 .148Dev/Scaled .786 .685 .448 .680 .580 .339 .586 .456 .219 .472 .342 .160Chapter 4. Results of the Simulation Study^ 604.3 The Simple Case with Three SeriesThe results discussed in this section are for three separate series of data generated accordingto log-linear models given by (3.17) with one of the two combinations of dispersion parametersdescribed in Section 3.2 (combination I or II) and one of the two possible variance functions.For the null model (70 = -yi = 72 = -y3 = 0) the parameter values /3i = log 0.75 (= —0.288),/32 = 0.7 and /33 = 1.6 are used to generate the data. The resulting data series are then fit usingthis same log-linear model. The mean values of the estimated parameters from 1000 simulateddata sets generated under each of the four possible scenarios (2 combinations of 0 x 2 varianceformulations) are presented in Table 7.Table 7. Mean Values of Parameter Estimates.ParameterVarianceA. CorrectFunctionB. MisspecifiedCombination of (/)I^IICombination of 0I^II0 1.997 2.799 4.964 8.33301 -0.288 -0.289 -0.289 -0.28702 0.699 0.699 0.701 0.700/33 1.599 1.599 1.598 1.594Intuitively the estimates of 0 should be approximately equal to the average of the threedispersion values associated with the three generated data series in each of the 1000 datasets leading to the results in each of the four scenarios. These averages are 2.0 (combinationI, correct variance), 2.8 (combination II, correct variance), 4.97 (combination I, misspecifiedvariance) and 8.37 (combination II, misspecified variance). We can see that in each of the fourscenarios, the estimates of 0 are very close to these corresponding averages. The estimates ofthe regression parameters are also excellent in all four scenarios.Chapter 4. Results of the Simulation Study^ 61Table 8 presents the standard deviations of the parameter estimates and the mean values ofthe estimated standard errors for the regression parameters. In each column of this table theempirical estimates agree very well with the simulated standard deviations but the model-basedestimates are most often quite different. The latter are much too large for the parameter /9 icorresponding to the series where the simulated dispersion is lower than the average of thethree series, and too small for 03 corresponding to the series where the simulated dispersionis larger than the average of the three series. For the regression parameter N2, first considerthe correct variance simulations. We find that with combination I of 0's the standard errorsfor 02 are about right; this appears to be because the simulated 0 for this series is equal tothe average of the 4's for all three series. However, using combination II of cb's the simulated0 for series 2 is smaller than the average of the O's for all three series and the model-basedstandard error is distinctly larger than the standard deviation of the parameter estimates. Inthe misspecified variance simulations, we also find the model-based standard error to be largerthan the standard deviation of the parameter estimates because the simulated 0 for series 2is smaller than the average of the cb's in these cases. Clearly the empirical estimator is betterin each of these cases (for both combinations of 0 and for both variance formulations) andaccurately reflects the true variability of these parameter estimates, even with the misspecifiedvariance function.Chapter 4. Results of the Simulation Study^ 62Table 8. Standard Deviations of Estimatesand Mean Values of Estimated Standard Errors (±s.d.).Parameter MethodVarianceA. CorrectFunctionB. MisspecifiedCombination of .0I^IICombination of 0I^II0 Simulated .084 .144 .399 .97601 Simulated .042 .052 .042 .050Model .062 (.002) .073 (.003) .097 (.004) .126 (.008)Empirical .044 (.002) .052 (.002) .044 (.002) .050 (.002)/32 Simulated .038 .038 .046 .047Model .038 (.001) .045 (.001) .059 (.003) .077 (.005)Empirical .038 (.001) .038 (.001) .046 (.002) .046 (.002)03 Simulated .029 .037 .054 .079Model .024 (.001) .028 (.001) .038 (.001) .049 (.002)Empirical .029 (.001) .038 (.001) .056 (.003) .077 (.004)Using the data generated under the reduced model, the levels achieved by the four teststatistics under study are examined by considering an alternative model that allows for a com-mon "pollution" effect (71 = 72 = -y3 = 0, but 7o possibly not zero). The hypothesis to betested is H: 70=0. Table 9 presents the observed rejection probabilities of each of the four teststatistics. The precision of the observed rejection probabilities, which should be kept in mindwhen interpreting these results, are the same as those reported in Section 4.2. Consider first thecorrect variance simulations. In the model reductions for the correct variance simulations, themodel-based tests (model Wald and scaled deviance) clearly reject too often while the rejectionprobabilities for the empirical tests (empirical score and empirical Wald) agree quite well withthe nominal levels based on the predicted 4 ) critical values. A similar pattern is apparentin the misspecified variance simulations, although the departure of the model-based tests fromChapter 4. Results of the Simulation Study^ 63the nominal levels is considerably more exaggerated than in the correct variance simulations.Table 9. Observed Rejection Probabilities: Common Pollution to No Pollution Model.A. CorrectCombination of 0Variance FunctionB. MisspecifiedCombination of 0TestI0.10 0.05 0.01II0.10 0.05 0.01I0.10 0.05 0.01II0.10 0.05 0.01Wald/Mod .159 .099 .019 .173 .102 .039Wald/Emp .114 .057 .004 .110 .058 .014S core/Emp .114 .056 .004 .109 .058 .014Dev/Scaled .159 .099 .018 .172 .102 .039.180 .104 .045 .226.089 .054 .016 .108.088 .053 .012 .102.180 .104 .044 .227.139 .060.062 .012.057 .011.139 .060Thus in this multiple series situation (without any pollution effects at this time) we find aclear difference between the empirical and model-based estimates of variance, with the empiricalestimator being preferable. Correspondingly we find that the model-based test statistics areinferior to the empirical test statistics. Given the differences in the variance estimates, thefact that the empirical Wald is preferable to the model Wald test is not surprising. Nor is itsurprising that the empirical score test performs well in all cases since it performed well evenunder misspecified variances in the study described in Breslow (1990). What is interesting isthe very close agreement agreement between the model Wald and scaled deviance tests andbetween the empirical score and empirical Wald tests. Conventional wisdom would suggestthe Wald tests might be inferior to the other two tests, but we see that the model Wald givesobserved rejection probabilities that are almost exactly the same as the scaled deviance test andthe empirical Wald gives observed rejection probabilities that are almost exactly the same asthe empirical score test. It seems that the test statistics which rely on a dispersion parameterto account for extra variability perform poorly, while those that estimate the extra variabilityChapter 4. Results of the Simulation Study^ 64empirically perform much better.A set of simulations with data generated according to the model with a common pollutioneffect (71 = 72 = 73 = 0, but 70 non-zero in (3.17)) were also carried out. Recall that thevalue chosen for 7o is 0.10. Table 10 presents the mean values of the parameter estimates forthe data sets generated in this fashion. The estimates of the Oi are very good in each columnof the table as are the estimates of -y o . The estimates of 0 are close to the estimates seen inTable 7 as we would expect given that in each case, the average of the dispersions is close tothe averages encountered in the simulations under the null hypothesis.Table 10. Mean values of Parameter Estimates.ParameterVarianceA. CorrectFunctionB. MisspecifiedCombination of 0I^IICombination of 0I^II1.998 2.799 4.970 8.325Ql -0.289 -0.292 -0.288 -0.292P2 0.699 0.701 0.699 0.69903 1.599 1.599 1.597 1.591yo 0.100 0.101 0.098 0.098The standard deviations of the above estimates and the mean values of the estimated stan-dard errors for the regression parameters are given in Table 11. The model-based estimates ofthe standard errors for the /3 show the same behavior as in Table 8 with the estimate being toolarge when the parameter corresponds to a series with simulated q5 lower than c b and too smallwhen the parameter corresponds to a series with simulated 0 larger than (7,. The empiricalestimator on the other hand, agrees very well with the simulated standard deviations. Noticethat the model-based estimates of the standard error of 7o are too low throughout the table.Recall that the rejection probabilities were too high for the model-based test statistics in TableChapter 4. Results of the Simulation Study^ 659. This suggests that the standard errors of the estimates of y o were lower than expected inthe simulations under the null hypothesis as well.Table 11. Standard Deviations of Estimatesand Mean Values of Estimated Standard Errors (±s.d.).Parameter MethodVarianceA. CorrectFunctionB. MisspecifiedCombination of 0I^IICombination of qI^II0 Simulated .086 .144 .395 .967131 Simulated .043 .053 .040 .052Model .062 (.002) .073 (.003) .097 (.004) .126 (.008)Empirical .044 (.002) .052 (.002) .044 (.002) .050 (.002)132 Simulated .038 .038 .046 .049Model .038 (.001) .045 (.001) .059 (.003) .077 (.005)Empirical .038 (.001) .038 (.001) .046 (.002) .046 (.002)/33 Simulated .029 .037 .058 .079Model .024 (.001) .028 (.001) .038 (.001) .049 (.002)Empirical .029 (.001) .038 (.001) .056 (.003) .077 (.004)7o Simulated .030 .038 .055 .076Model .027 (.001) .032 (.001) .043 (.002) .056 (.003)Empirical .031 (.001) .038 (.002) .054 (.002) .072 (.006)We now examine the results of evaluating the four test statistics based on an alternativehypothesis that includes common and separate "pollution" effects for each series. Thus thesemodel reductions, from the model with separate effects to the one with a common pollutioneffect (which was used to generate the data), represent the separate to common effects reductionsdescribed in the Prince George study. Note that these tests, the results of which are summarizedin Table 12, are the first discussed in this thesis involving reduction by more than one parameter(and are thus the first that involve estimates of covariance in the Wald and score test statistics).Chapter 4. Results of the Simulation Study^ 66In the correct variance simulations the empirical test statistics achieve levels close to thenominal rejection probabilities with combination I of 0 and still quite close with combinationII (keeping in mind the precision of these observed rejection probabilities discussed above).The levels of the model-based test statistics are clearly too low under both combinations of 0.Even with the misspecified variance formulation, the empirical test statistics achieve levels closeto nominal; while the rejection probabilities may be slightly too large at the 0.10 level undercombination II, overall the agreement is good. The levels of the model-based test statistics arefar too low at the 0.10 and 0.05 nominal levels under both combinations of 0, although it is notclear how they do at the 0.01 level.Table 12. Observed Rejection Probabilities: Separate to Common.TestA. CorrectVariance FunctionB. MisspecifiedCombination of 0 Combination of cb0.10I0.05 0.01 0.10II0.05 0.01 0.10I0.05 0.01 0.10II0.05 0.01Wald/Mod .076 .035 .006 .062 .026 .001 .067 .036 .005 .056 .029 .009Wald/Emp .100 .051 .011 .092 .049 .010 .104 .057 .006 .117 .052 .014Score/Emp .099 .049 .008 .088 .049 .008 .100 .055 .006 .113 .051 .010Dev/Scaled .077 .035 .006 .062 .027 .001 .067 .036 .006 .056 .030 .009We also examine the power of the four tests in this multiple series situation in Table 13by considering the reduction from the common pollution effect to no pollution effects (the nullmodel). The power for the two model-based tests are virtually identical throughout the tableand the two empirical tests achieve similar power. However, the rejection probabilities for themodel-based tests are higher than the empirical tests presumably as a result of the standarderror of 7o being underestimated in the former tests (see Table 11).Chapter 4. Results of the Simulation StudyTable 13. Observed Rejection Probabilities: Common to Null.67A. CorrectCombination of 0Variance FunctionB. MisspecifiedCombination of 0TestI0.10 0.05 0.01II0.10 0.05 0.01I0.10 0.05 0.01II0.10 0.05 0.01Wald/Mod .958 .925 .834 .891 .827 .687 .695 .598 .404 .542 .447 .271Wald/Emp .941 .890 .751 .837 .767 .537 .570 .445 .226 .399 .293 .111Score/Emp .941 .891 .751 .837 .765 .535 .567 .436 .213 .394 .283 .100Dev/Scaled .958 .925 .835 .891 .828 .688 .695 .599 .406 .542 .447 .274The final model to be considered for generating data in this three series simple case isthe model with separate pollution effects. Recall the value chosen for 7 0 is again 0.10, while71 =0.10, 72 = -0.20 and -y3 = 0. Results of fitting the full model to these data sets aresummarized in Tables 14, 15 and 16 which follow.The mean values of the parameter estimates (see Table 14 below) are very close to the correctvalues, even in the misspecified case with substantial amounts of dispersion. The estimates of 0are very close to the values seen in Tables 7 and 10, which is still to be expected because, even inthe misspecified variance simulations, the average of the dispersion values over the entire 2100observations is approximately equal to the averages in the corresponding simulations undereither the null or the common pollution effect model.Chapter 4. Results of the Simulation Study^ 68Table 14. Mean Values of Parameter Estimates.VarianceA. CorrectFunctionB. MisspecifiedCombination of 0 Combination of 0Parameter I II I IICb 1.997 2.799 4.962 8.32601 -0.290 -0.289 -0.290 -0.28902 0.699 0.700 0.700 0.69803 1.598 1.597 1.595 1.59370 0.098 0.102 0.102 0.09671 0.101 0.096 0.100 0.10272 -0.195 -0.200 -0.197 -0.195The estimates of the standard errors are summarized in Table 15. In each of the columnsof this table the empirical estimator is very close, on average, to the simulated standard devia-tions. Again the model-based estimator overestimates the standard errors for parameters whichcorrespond to the series with dispersion lower than 0 (most notably th and 7 1 ) and underes-timates the standard errors corresponding to the series with dispersion larger than 0 (mostnotably 03 ). It would seem that the low rejection probabilities in Table 12 for the model-basedtest statistic can be explained by the general overestimation of the standard error of 71.Chapter 4. Results of the Simulation Study^ 69Table 15. Standard Deviations of Estimatesand Mean Values of Estimated Standard Errors (±s.d.).Parameter MethodVarianceA. CorrectFunctionB. MisspecifiedCombination of 0I^IICombination of 0I^IICb Simulated .085 .142 .393 .96601 Simulated .044 .054 .042 .050Model .062 (.002) .073 (.003) .098 (.004) .127 (.008)Empirical .044 (.002) .052 (.002) .044 (.002) .050 (.002)02 Simulated .039 .037 .047 .047Model .038 (.001) .045 (.001) .059 (.003) .077 (.005)Empirical .038 (.002) .038 (.001) .046 (.002) .046 (.002)03 Simulated .029 .039 .056 .076Model .024 (.001) .028 (.001) .038 (.001) .049 (.002)Empirical .029 (.001) .038 (.001) .056 (.003) .077 (.005)70 Simulated .042 .054 .078 .113Model .034 (.001) .040 (.001) .054 (.002) .069 (.007)Empirical .041 (.002) .054 (.003) .079 (.005) .109 (.011)71 Simulated .074 .090 .098 .133Model .094 (.003) .111 (.004) .148 (.006) .191 (.011)Empirical .075 (.003) .090 (.003) .100 (.004) .130 (.009)72 Simulated .071 .076 .102 .131Model .063 (.002) .075 (.002) .100 (.004) .129 (.007)Empirical .067 (.002) .076 (.003) .102 (.005) .127 (.012)Chapter 4. Results of the Simulation Study^ 70Table 16. Observed Rejection Probabilities: Separate to Common.TestVarianceA. Correct Variance FunctionFunctionB. Misspec. Variance FunctionCombination of 0 Combination of 00.10I0.05 0.01 0.10II0.05 0.01 0.10I0.05 0.01 0.10II0.05 0.01Wald/Mod .936 .890 .741 .870 .786 .558 .621 .465 .222 .340 .218 .088Wald/Emp .962 .932 .822 .944 .889 .747 .911 .858 .685 .880 .791 .593Score/Emp .962 .929 .818 .945 .888 .745 .909 .854 .671 .879 .786 .578Dev/Scaled .936 .891 .743 .870 .789 .560 .623 .468 .227 .341 .218 .088In Table 16 we examine the power of the four test statistics under consideration in testingthe reduction from the separate pollution effects simulated in these data sets to a model with acommon pollution effect. The model-based test statistics have lower power than the empiricaltests in all of the above cases. The difference is most noticable in the cases where the standarderrors of -yi were seriously overestimated, such as the misspecified variance simulations.In all of the simulations in this three series simple case, in contrast to the one series sim-ulations, it is clear that the empirical estimate of the covariance matrix for the regressionparameters is superior to the model-based estimate. As for the observed levels of the four teststatistics under consideration, we find that the model-based test statistics reject too often insome instances, and not as often as predicted by theory in other cases, whereas the empiricaltest statistics always achieve levels close to the nominal rates. Throughout we have found veryclose agreement between the model Wald and scaled deviance tests and between the empiricalscore and empirical Wald tests. It appears that, in these simple simulations, the two model-based test statistics that rely on a dispersion parameter to account for the extra variabilityperform poorly, while the empirical test statistics (which do not rely on the correctness of aspecified model to estimate the variance-covariance matrix) perform much better.Chapter 4. Results of the Simulation Study^ 714.4 The More Complicated Case with a Single SeriesWe now wish to study the performance of the estimators and test statistics when we generatedata using the more complicated model for a single series, described by (3.18), that includesseason effects. For the purpose of simulating data under the reduced model, the values r 1 =r2 = 0.25, r3 = -0.5 and r4 = 0 (with y = 0) were used. Table 17 presents the mean valuesof the parameter estimates from fitting the reduced model to each of the 1000 simulated datasets.Table 17. Mean Values of Estimated Parameters Under Reduced Model.A. Correct VarianceSimulated 0Simulated 0 Parameter 1 1.4 2 3 5log 0.75 (i) 1.00 1.39 1.98 2.96 -Q -.291 -.286 -.295 -.292 -Ti. .253 .245 .253 .247 -r2 .252 .238 .250 .253 -r3 -.498 -.513 -.513 -.520 -0.70 0 .997 1.40 1.99 2.98 -,Q .699 .697 .698 .695 -r1 .246 .250 .247 .252 -r2 .250 .253 .250 .25073 -.499 -.501 -.501 -.508 -1.60 4 .997 1.40 2.01 2.98 4.9713 1.60 1.60 1.60 1.60 1.60ri. .251 .249 .247 .251 .246T2 .250 .250 .248 .249 .25173 -.503 -.501 -.502 -.504 -.499Chapter 4. Results of the Simulation Study^ 72Table 17 B. Misspecified VarianceSimulated 4)Simulated /3 Parameter 1 1.4 2 3 5log 0.75 0 1.00 1.31 1.77 2.54 -/3 -.287 -.292 -.297 -.296 -Tl .245 .246 .256 .248 -T2 .246 .251 .258 .256 -T3 -.507 -.503 -.503 -.507 -0.70 0 1.00 1.83 3.07 5.15 -/3 .702 .695 .699 .698 -Ti. .247 .253 .244 .246 -T2 .245 .254 .244 .248T3 -.505 -.501 -.506 -.502 -1.60 (/) 1.00 3.06 6.15 11.22 21.09/3 1.60 1.60 1.60 1.59 1.58Ti .249 .252 .249 .252 .263T2 .248 .249 .248 .255 .2587-3 -.500 -.500 -.505 -.497 -.495In part A of the table, the correct variance simulations, the estimates are very close tothe simulated values in each column of the table. The regression parameter estimates ( 13 andTi) are also quite good (correct to 1 decimal) even when dispersion is high relative to the mean.There are cases where the estimates appear to be a bit off (for example, the simulations with /3= log 0.75 and c > 1.4), but the regression parameter estimators have substantially higher vari-ances than in the simple simulations of the previous sections (compare the standard deviationsof these estimates provided in Table 18 below with those in Table 3), so it is not unreasonablethat the mean value from 1000 simulated data sets differs slightly from the simulated value.We now consider the misspecified variance results in part B of the above table. Here thevalues of q5 are roughly what they were in the one series simple case. This follows because thefour seasons are like four short series with different mean levels and therefore different levels ofdispersion. Results from the three series simple simulations (Section 4.3) suggest the averageChapter 4. Results of the Simulation Study^ 73of those levels of dispersion is what is being estimated estimated by the overall estimator sr4.Because the ri were chosen so that E i r = 0, the average is approximately the same as theamount of dispersion in the one series simple simulations of Section 4.2 (slightly different becausethe mean levels of the four "series" don't quite average out to the overall mean; >, exp{/3does not equal exp{E i (0 -I- TO} = exp{0}). Thus the estimates of 0 in the above table are aswe would expect. More importantly, the mean values of the regression parameter estimates arequite close to the simulated values at all levels of dispersion and for all choices of 0.The standard deviations of the parameter estimates and the mean values of the standarderrors of the regression parameters are summarized in Table 18. In the correct variance simu-lations (part A of the table), the empirical and model-based estimators are very close to eachother in all cases, and agree well with the actual standard deviations of the parameter estimates.There is one instance (0 = log0.75 and 0 = 3) when the mean values of the estimated standarderrors for all the regression parameters are slightly lower than the standard deviations. Thiscould be a suggestion that the estimators underestimate the standard errors when there is alarge amount of dispersion and low overall mean level, but could also just be chance. In anycase, there is no strong evidence that the empirical estimator particularly underestimates thestandard errors in general as expected a priori (based on conventional wisdom), or at leastnot any more so than the model-based estimator. Overall there is no real difference in theseestimated standard errors. Note however that the model-based estimated standard error isuniformly less variable than that based on the empirical estimator (although the difference issmall in this particular situation).Chapter 4. Results of the Simulation Study^ 74In the misspecified variance simulations there may be a problem with the model-basedestimator similar to the problems in the three series simple case simulations of Section 4.3. If wethink of the four seasons as four short series, then under the misspecified variance formulation,the four series, with different mean levels, have different amounts of dispersion. In particular,the simulated dispersion for the data in the third season will be quite a bit lower than for theother seasons. The "overall" estimate ::$. will be larger than the amount of dispersion simulatedfor this season which might explain why the model-based estimates of standard errors for f 3 aretoo high on average. This same argument would suggest that the standard errors for f 1 and 1:2would be too low since these "series" have the highest means and therefore the most dispersionso that q will be lower than the amount of dispersion simulated in these series. There is somesuggestion of this as well in part B of Table 12.As for the empirical estimator, overall it appears to estimate the standard deviations ofthe parameters much better than the model-based estimator. There is no clear evidence thatthis estimator consistently underestimates the standard errors of the regression parametersalthough whenever the average of the empirical estimates differs to any appreciable extent froma simulated standard deviation, this average does appear to be too low.oQlog 0.75TiT2T3Chapter 4. Results of the Simulation Study^ 75Table 18. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.).A. Correct Variance FunctionSimulated ,8 Parameter MethodSimulated 01 1.4 2 3 5Simulated .056 .097 .175 .326SimulatedModelEmpirical.086.088(.004).087(.006).103.103(.006).103(.008).124.123(.009).123(.011).153.150(.012).150(.015)Simulated .118 .135 .165 .206Model .117(.005) .138 (.006) .165 (.009) .201(.013)Empirical .116(.006) .137 (.007) .164 (.010) .200(.014)Simulated .115 .135 .164 .205Model .117(.005) .138 (.006) .165 (.009) .201(.013)Empirical .116(.006) .137 (.007) .164 (.010) .200(.014)Simulated .142 .166 .205 .258Model .143(.007) .168(.009) .202(.012) .247(.019)Empirical .142(.008) .168(.010) .201(.014) .244(.021)Simulated .056 .087 .136 .245Simulated .055 .064 .077 .093Model .053(.002) .063(.003) .075(.004) .092(.005)Empirical .053(.003) .063(.004) .075(.005) .092(.007)Simulated .074 .087 .097 .121Model .071(.002) .084 (.003) .100 (.004) .123(.005)Empirical .071(.003) .084 (.004) .100 (.005) .122(.006)Simulated .072 .085 .101 .123Model .071(.002) .084 (.003) .100 (.004) .123(.006)Empirical .071(.003) .084 (.004) .100 (.005) .122(.006)Simulated .089 .103 .123 .153Model .087(.003) .103(.004) .123(.005) .150(.007)Empirical .086(.004) .102(.005) .122(.006) .149(.009)Simulated .054 .078 .122 .200 .388Simulated .034 .042 .047 .060 .077Model .034(.001) .040(.001) .048(.002) .059(.002) .076(.004)Empirical .034(.002) .040(.002) .048(.003) .059(.004) .076(.005)Simulated .046 .056 .062 .081 .105Model .045(.001) .054 (.002) .064 (.002) .078(.003) .101(.004)Empirical .045(.002) .053 (.002) .064 (.003) .078(.004) .101(.005)Simulated .045 .054 .063 .079 .102Model .045(.001) .054 (.002) .064 (.002) .078(.003) .101(.004)Empirical .045(.002) .053 (.002) .064 (.003) .078(.004) .101(.005)Simulated .057 .067 .078 .100 .126Model .055(.002) .065(.002) .078(.003) .096(.004) .123(.006)Empirical .055(.002) .065(.003) .078(.003) .095(.005) .123(.007)0.70^0#T1T2T31.60^0)3TiT2T3Chapter 4. Results of the Simulation Study^ 76Table 18 B. Misspecified Variance FunctionSimulated 16 Parameter MethodSimulated 01 1.4 2 3 5log 0.75 0 Simulated .052 .087 .140 .253 .■.# SimulatedModelEmpirical.088.087(.005).087(.006).101.100(.006).099(.008).116.117(.008).115(.009).139.140(.011).137(.013)II..■ri Simulated .119 .137 .162 .196 ...■Model .117(.005) .134 (.006) .156 (.008) .186(.011) .....Empirical .116(.006) .135 (.007) .157 (.009) .190(.012) ■..72 Simulated .118 .136 .165 .192 .1.Model .117(.005) .134 (.006) .156 (.008) .186(.011)Empirical .116(.006) .134 (.007) .157 (.009) .189(.012) ■73 Simulated .140 .157 .185 .208 .1.Model .143(.007) .163(.009) .191(.011) .228(.016)Empirical .142(.008) .158(.010) .178(.011) .207(.015)0.70 0 Simulated .054 .120 .261 .52513 Simulated .055 .072 .094 .125Model .053(.002) .072(.003) .093(.005) .121(.009)Empirical .053(.003) .071(.005) .091(.007) .118(.011)Ti. Simulated .073 .100 .132 .172Model .071(.002) .096 (.004) .125 (.006) .161(.009)Empirical .071(.003) .097 (.005) .128 (.007) .165(.010)72 Simulated .071 .099 .130 .165Model .071(.002) .096 (.004) .125 (.006) .161(.009)Empirical .071(.003) .098 (.004) .127 (.007) .165(.011)73 Simulated .090 .111 .146 .177Model .087(.003) .118(.005) .152(.008) .197(.012)Empirical .087(.004) .109(.005) .136(.008) .173(.011)1.60 0 Simulated .057 .209 .525 1.23 2.96# Simulated .034 .059 .084 .118 .155Model .034(.001) .059(.002) .084(.005) .114(.008) .157(.014)Empirical .034(.002) .058(.004) .082(.006) .110(.010) .152(.017)Ti Simulated .047 .083 .118 .165 .217Model .045(.001) .079 (.003) .112 (.005) .152(.008) .210(.015)Empirical .045(.002) .081 (.004) .115 (.006) .156(.010) .215(.017)Ti Simulated .046 .081 .119 .159 .214Model .045(.001) .079 (.003) .112 (.005) .152(.008) .210(.015)Empirical .045(.002) .081 (.004) .115 (.006) .156(.010) .215(.018)T3 Simulated .055 .090 .122 .164 .221Model .055(.002) .097(.004) .137(.007) .186(.013) .256(.020)Empirical .055(.002) .087(.004) .120(.006) .159(.012) .218(.018)Chapter 4. Results of the Simulation Study^ 77In the next table we present the observed rejection probabilities for the four test statisticsunder study in each of the combinations of /3 and 0 considered above. Here the alternatemodel (to be fit to the data sets generated under the reduced model) includes the sinusoidal"pollution" covariate. In the correct variance simulations the model Wald and scaled deviancetests give almost identical results in all simulations. The empirical tests are not as similar(either to each other or to the model-based tests), but overall there appears to be little todistinguish any of the tests. The observed rejection probabilities are reasonably close to thenominal levels in almost all cases. Possible exceptions might be the simulations with /3 = 1.60and either of 0 = 1 or 0 = 3, but these are likely just chance deviations from the nominal levels.In the misspecified variance simulations we also find the model Wald and scaled deviancetests to be almost identical in their observed rejection probabilities. In general the score testappears most conservative and it seems to get more conservative (relative to the others) as theamount of dispersion increases. This makes it look like it is performing better than the otherswhen the others reject too often and it looks worse when the others reject at about the nominalrate.As a side note, based on the model-based standard errors of Table 18, we might look carefullyat the empirical Wald test to see if it rejects too often, since it seemed the empirical estimates ofthe standard errors of some of the T, were too small, which might imply that the standard errorsof the coefficient of the pollution covariate could be underestimated. The empirical Wald testdoes reject more often than the score test in general, but it occasionally rejects less often thanthe model-based tests so that no general conclusion regarding its performance can be reachedfrom these results.Chapter 4. Results of the Simulation Study^ 78Table 19. Observed Rejection Probabilities Under Null Hypothesis.A. Correct Variance Function0 = 1^4 =1.41.^0 = 2Simulated 0 Test/Methodlog 0.75 Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled0.70 Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled1.60 Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled0.10 0.05 0.01 0.10 0.05 0.01 0.10 0.05 0.01.093 .060 .011 .112 .056 .012 .110 .052 .014.098 .063 .013 .114 .057 .010 .115 .053 .012.098 .062 .012 .112 .055 .009 .110 .047 .011.093 .060 .011 .111 .058 .012 .110 .053 .014.100 .052 .014 .099 .045 .008 .098 .052 .013.101 .049 .012 .102 .044 .010 .101 .056 .011.095 .047 .012 .099 .044 .010 .097 .053 .011.100 .052 .014 .099 .044 .008 .098 .052 .013.115 .054 .011 .092 .040 .011 .109 .053 .009.121 .055 .011 .091 .041 .012 .111 .053 .010.119 .053 .010 .091 .041 .010 .109 .053 .008.115 .055 .011 .092 .040 .011 .109 .054 .0090 = 3^0 = 5Simulated /3log 0.75Test/MethodWald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled0.10 0.05 0.01.062 .012.066 .016.059 .011.062 .0120.10 0.05 0.010.70 Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled1.60 Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled.099 .048 .009.107 .048 .013.101 .045 .007.100 .047 .009.121 .062 .014.125 .063 .015.123 .057 .014.121 .062 .014.105 .048 .010.108 .057 .012.106 .051 .009.106 .049 .010Chapter 4. Results of the Simulation Study^ 79Table 19 B. Misspecified Variance Function0 = 1^0 = 1.4^0 = 2Simulated /3 Test/Method^0.10 0.05 0.01 0.10 0.05 0.01 0.10 0.05 0.01log 0.75 Wald/Model .110 .051 .005 .112 .049 .004 .111 .058 .010Wald /Empirical .113 .055 .005 .111 .047 .004 .112 .059 .007Score/Empirical .111 .052 .005 .110 .042 .004 .109 .055 .005Deviance/Scaled .110 .052 .005 .112 .049 .004 .111 .059 .0100.70^Wald/Model^.095 .049 .009 .105 .049 .010 .109 .060 .013Wald/Empirical .098 .051 .013 .103 .048 .008 .099 .055 .015Score/Empirical .095 .051 .011 .100 .046 .007 .094 .057 .013Deviance/Scaled .095 .049 .009 .105 .048 .009 .109 .060 .0131.60^Wald/Model^.101 .051 .010 .102 .053 .013 .105 .052 .010Wald/Empirical .102 .054 .011 .102 .046 .011 .098 .046 .008Score/Empirical .102 .053 .010 .099 .045 .010 .092 .045 .008Deviance/Scaled .101 .051 .010 .103 .053 .013 .104 .053 .0110 = 3 Cb = 5Test/MethodWald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled0.10 0.05 0.01.113 .059 .010.110 .056 .010.102 .052 .007.113 .060 .0110.10 0.05 0.01Simulated /3log 0.750.70 Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled1.60 Wald/ModelWald/EmpiricalScore/EmpiricalDeviance/Scaled.102 .052 .018.093 .049 .014.087 .046 .013.103 .053 .018.108 .056 .016.104 .050 .017.100 .046 .013.108 .057 .016.112 .056 .018.110 .056 .018.103 .049 .015.114 .057 .018We now consider the simulations under the full model (with pollution included). As in thesimple case simulations, we chose to simulate data using only 0 = log 0.75 but with one of-y = 0.05, 0.10 and 0.15. Table 20 presents the mean values of the estimated parameters fromthe simulated data sets generated under the above model.Chapter 4. Results of the Simulation Study^ 80Table 20. Mean Values of Estimated Parameters UnderAlternative Hypothesis.A. Correct Variance FunctionSimulated 4Simulated y Parameter 1 1.4 2 30.05 0 1.00 1.40 1.97 2.94/3 -.293 -.295 -.311 -.303Ti .254 .253 .274 .256r2 .249 .251 .276 .254T3 -.497 -.503 -.503 -.5137 .054 .051 .038 .0550.10 0 1.00 1.39 1.97 2.94/3 -.289 -.287 -.304 -.306sn. .246 .237 .259 .25872 .247 .244 .260 .259T3 -.506 -.511 -.510 -.5157 .103 .104 .097 .0980.15 Cb 1.00 1.40 1.98 2.94/3 -.289 -.304 -.300 -.299Ti .248 .271 .260 .252T2 .247 .272 .259 .259T3 -.498 -.511 -.515 -.5327 .154 .132 .147 .144B. Misspecified Variance FunctionSimulated 0Simulated 7 Parameter 1 1.4 2 30.05 0 1.00 1.31 1.77 2.54/3 -.293 -.295 -.296 -.293Ti .254 .253 .250 .23572 .249 .252 .254 .246r3 -.497 -.504 -.504 -.5157 .054 .050 .050 .0560.10 Cb 1.00 1.31 1.79 2.56/3 -.289 -.294 -.295 -.291Ti .246 .250 .248 .240T2 .247 .251 .255 .241T3 -.506 -.505 -.504 -.5167 .103 .104 .101 .1020.15 95 1.00 1.32 1.79 2.57/3 -.289 -.298 -.294 -.3037-1. .248 .254 .246 .25272 .247 .263 .246 .257T3 -.498 -.501 -.502 -.4967 .154 .145 .149 .155Chapter 4. Results of the Simulation Study^ 81Before interpreting these results, note that there is non-negligible collinearity between thecontinuous pollution covariate and the seasonal indicators. If, in our simulations, two ex-planatory variables have a substantial positive correlation, one of the corresponding regressionparameters could be estimated to be larger than its simulated (or true) value, while the othercould be estimated to be smaller than its simulated value, with little effect on the goodness-of-fit. Similarly, both could be estimated to be larger than the simulated values if negativelycorrelated. Such deviations from the true values, beyond the normal variation we would expect,lead to more variability from one set of fitted parameters to another than if the predictors werenot collinear. The sample correlations are approximately 0.52 between the pollution variableand each of the indicators for the first and second season, and —0.52 between the pollution vari-able and the indicator for the third season. While such correlations are not a major concern,they must be kept in mind when examining the results of these simulations.For the correct variance simulations, the mean values of the regression parameter estimatesare generally close to their simulated values, considering how big the standard deviations of theparameter estimates are (and thus the standard error of the average of the parameter estimatesas an estimate of the true mean, see Table 21). However, in the cases -y = 0.05, 0 = 2 and-y = 0.15, 0 = 1.4 the averages of the estimates of 7-1 and r2 exceed the simulated value by morethan two standard errors (while the estimate of -y is too small). In the case -y = 0.15, 0 = 3the estimate of r3 is too small (a large negative value that differs from the true value by morethan two standard errors) with the estimate of 7 slightly too small, although it differs from thetrue value by less than one standard error. Note how the pattern of over and underestimationagrees with what we might expect when the pollution variable is positively correlated with theChapter 4. Results of the Simulation Study^ 82first two season indicators and negatively correlated with the third. But there is also a hint of apattern in the estimates of r3 which has no such plausible explanation; in general, as 0 increasesacross the table, r3 appears to decrease (become a more negative value). The observations inthe third season have the lowest mean values, and as the amount of dispersion increases, therewill be more and more zero values among these observations. We might infer that for largeenough amounts of over-dispersion, this could be causing the estimator to underestimate theseason effect, but with the low degree of precision in the averages of the parameter estimateswe can not conclude whether this is the case, or if the deviations from the simulated values arejust chance.In the misspecified variance simulations, the regression parameters estimates are very good;in fact, they appear somewhat better than in the correct variance case. The estimates of 0 seenin parts A and B of this table are similar to those in Table 17.The standard deviations of the parameter estimates in the 1000 simulations and the meanvalues of the standard errors of the regression parameters are summarized in Table 21. As inpart A of Table 18, under the correct variance formulation the model and empirical estimatesare very similar. Both estimators are very close to the standard deviations of the parameterestimates when the dispersion is 1 or 1.4. For simulated dispersions of 2 or 3, the averages ofthe estimates are not as close to the simulated values but the differences can be explained bythe variability of the observed standard deviations and the estimated standard errors.Chapter 4. Results of the Simulation Study^ 83Table 21. Standard Deviations of Estimates Under Alternative Hypothesisand Mean Values of Estimated Standard Errors (±s.d.).A. Correct Variance FunctionSimulated y Parameter MethodSimulated 0 1^1.4^2^30.05^o#rir2T37.054 .099 .166 .326.122 .149 .177 .221SimulatedSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpirical.125(.005) .148(.007) .176(.009) .215(.014).124(.007) .147(.010) .175(.015) .214(.024).209 .255 .300 .374.212(.007) .251(.010) .298(.013) .364(.020).211(.012) .249(.017) .296(.026) .362(.041).210 .246 .305 .373.213(.007) .251(.010) .298(.013) .365(.020).211(.012) .249(.018) .296(.026) .364(.042).143 .171 .194 .261.145(.007) .172(.009) .205(.012) .251(.019).144(.008) .171(.011) .203(.014) .247(.021).138 .166 .203 .243.139(.005) .164(.007) .205(.012) .238(.013).138(.007) .163(.009) .203(.014) .238(.021).054 .096 .173 .321.126 .151 .172 .224.126(.005) .148(.007) .177(.009) .216(.014).125(.008) .147(.011) .175(.015) .214(.023).210 .255 .287 .380.212(.008) .250(.010) .298(.013) .364(.019).210(.012) .248(.018) .295(.026) .361(.040).217 .250 .289 .385.212(.008) .250(.010) .299(.013) .364(.019).211(.012) .249(.018) .295(.026) .360(.041).144 .174 .215 .261.147(.007) .174(.009) .209(.013) .255(.020).147(.008) .173(.011) .207(.015) .251(.021).139 .166 .188 .249.138(.005) .163(.006) .195(.009) .237(.012).138(.007) .162(.010) .193(.014) .236(.022).054 .096 .178 .332.128 .150 .174 .223.126(.005) .149(.007) .177(.010) .216(.015).125(.008) .148(.011) .176(.016) .215(.024).220 .251 .284 .372.211(.007) .250(.010) .298(.013) .363(.020).210(.012) .248(.017) .296(.027) .361(.040).216 .248 .292 .374.211(.007) .250(.010) .298(.013) .363(.020).211(.012) .248(.018) .297(.026) .361(.040).152 .175 .222 .266.149(.007) .177(.010) .212(.014) .259(.020).149(.008) .176(.011) .210(.015) .255(.022).141 .161 .186 .243.138(.005) .163(.006) .194(.008) .236(.013).137(.007) .162(.010) .193(.014) .235(.021)0.10^46rIr2T370.15^cbTir2T37Chapter 4. Results of the Simulation Study^ 84Table 21 B. Misspecified Variance FunctionSimulated y Parameter MethodSimulated 0 1.4^21 3.053 .086 .144 .259.122 .145 .169 .197.125(.005) .143(.007) .167(.008) .199(.012).124(.007) .142(.010) .166(.013) .199(.020).209 .245 .292 .351.212(.007) .243(.009) .282(.012) .338(.017).211(.012) .243(.017) .285(.023) .343(.037).210 .253 .290 .345.213(.007) .243(.010) .283(.012) .338(.017).211(.012) .243(.017) .286(.024) .343(.037).143 .163 .189 .212.145(.007) .166(.009) .194(.011) .232(.016).144(.008) .159(.009) .179(.011) .208(.014).138 .167 .191 .227.139(.005) .159(.006) .185(.008) .221(.011).138(.007) .159(.009) .186(.013) .225(.020).126(.005) .144(.007) .168(.009) .201(.012).125(.008) .143(.010) .167(.014) .199(.020).210 .244 .295 .350.212(.008) .243(.009) .283(.012) .339(.016).210(.012) .244(.017) .285(.024) .343(.037).217 .247 .295 .349.212(.008) .243(.009) .283(.012) .339(.017).211(.012) .243(.017) .286(.025) .342(.037).145 .171 .190 .220.147(.007) .169(.009) .197(.012) .237(.017).147(.008) .162(.010) .182(.01) .210(.015).139 .160 .193 .227.138(.005) .159(.006) .184(.008) .221(.010).138(.007) .159(.009) .187(.013) .225(.019).054 .084 .146 .259.120 .148 .170 .207.126(.005) .145(.007) .168(.009) .203(.012).125(.008) .144(.010) .166(.013) .199(.019).202 .252 .289 .348.212(.008) .243(.009) .283(.012) .340(.017).210(.012) .244(.017) .284(.023) .342(.035).204 .251 .296 .346.212(.008) .243(.009) .283(.012) .340(.017).211(.012) .244(.017) .285(.024) .343(.035).148 .168 .183 .226.150(.007) .172(.009) .200(.013) .241(.017).149(.008) .164(.010) .183(.012) .211(.014).136 .165 .194 .224.138(.005) .158(.006) .184(.008) .221(.011).137(.007) .159(.009) .187(.013) .226(.019)0.05^0oTiT2T37SimulatedSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpirical0.10^4.#TiT2T37SimulatedSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpirical0.15^015'71r2T37SimulatedSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpiricalSimulatedModelEmpirical.054 .085 .143 .264.126 .145 .172 .203Chapter 4. Results of the Simulation Study^ 85To estimate the variance of the observed standard deviations, note the following. Because re-gression parameter estimates are asymptotically normal, if B t is a regression parameter estimatefrom the ith simulated data set and B is the average of these estimates, then (7 -2 Ei(Oi - 6) 2 ,where o.2 represents Var(Oi), should have approximately a x 2 _ 1 distribution. It follows that thesample variance of these estimates should have variance roughly equal to 2a 4 /(n — 1). Usingthe delta method we arrive at the approximation Var(s) o-2 /[2(n — 1)].Using this approximation, observed standard deviations of s = 0.3 and s = 0.2 wouldhave standard errors of approximately 0.0067 and 0.0045 respectively. The mean values ofthe estimated standard errors will have approximate standard errors given by their standarddeviations (enclosed in brackets in the table) divided by ..076 -00. When is 2 this can be aslarge as .027/1005 = 0.0009. With these precisions in mind, the observed differences betweenthe variance estimates and the standard deviations are not too large.Another approach to determining whether the model-based and empirical standard errorsapproximate the simulated standard errors can be based on the increased precision provided bygenerating more data sets for a given choice of y and 0. The following table summarizes theresults of the simulations for the case 7 = 0.10 and = 2 as displayed in part A of Table 21,and the results of five repeats of the simulation experiment using the same y and q5.Chapter 4. Results of the Simulation Study^ 86Parameter Method original repeat 1 repeat 2 repeat 3 repeat 4 repeat 5 average/3 Simulated .172 .169 .182 .180 .177 .179 .177Model .177 .177 .177 .177 .177 .177 .177Empirical .175 .175 .176 .176 .176 .175 .176Ti. Simulated .287 .293 .311 .307 .296 .298 .299Model .298 .299 .299 .298 .298 .298 .298Empirical .295 .296 .296 .297 .295 .296 .296T2 Simulated .289 .289 .304 .299 .297 .300 .296Model .299 .299 .299 .299 .298 .299 .299Empirical .295 .296 .297 .297 .297 .296 .296r3 Simulated .215 .206 .209 .215 .207 .215 .211Model .209 .209 .209 .209 .208 .209 .209Empirical .207 .207 .207 .207 .207 .207 .2077 Simulated .188 .196 .196 .197 .194 .192 .194Model .195 .195 .195 .195 .194 .195 .195Empirical .193 .194 .194 .193 .194 .194 .194From the above table we note that in each simulation experiment, the mean values ofthe model and empirical estimates of the standard errors are very close to the average of thesimulated standard deviations across the columns of the table. This confirms that the modeland empirical estimators do a good job of estimating the standard deviations of the parameterestimates.We now examine part B of Table 21 corresponding to the simulations using the misspecifiedvariance formulation to generate the data. Here we seem to be seeing the same pattern as inthe simulations using data generated with the reduced model (part B of Table 18). The model-based estimates appear too large for regression parameters which correspond to "seasons" withlower than average dispersion and too small for parameters which correspond to "seasons" withlarger than average dispersion. In this situation, with a single pollution covariate, there isno clear picture of under or overestimation of the standard errors of y. However, one mightspeculate that if the pollution covariate were associated with one of the seasons (such as aChapter 4. Results of the Simulation Study^ 87season x pollution interaction) the standard errors of the regression parameter might be underor overestimated, depending upon which season the covariate was related to, and this could leadto unexpected results. As for the empirical estimator, the results suggest the possibility that thestandard errors of the regression parameters tend to be underestimated in these simulations.Of course, when making these observations we should keep in mind the possibility that theobserved standard deviations of the parameter estimates may differ somewhat from the truestandard deviations. A similar exercise to that carried out in the correct variance simulationswas carried out for these simulations under the misspecified variance for the same combination= 0.10 and 0 = 2. The final column of the table which follows shows the averages of the 6simulation experiments.Parameter Method original repeat 1 repeat 2 repeat 3 repeat 4 repeat 5 average/3 Simulated .172 .165 .164 .171 .168 .167 .168Model .168 .167 .168 .168 .168 .167 .168Empirical .167 .166 .166 .167 .166 .166 .166ri. Simulated .295 .289 .287 .288 .282 .291 .289Model .283 .282 .283 .282 .282 .282 .282Empirical .285 .285 .284 .286 .285 .285 .28572 Simulated .295 .288 .286 .293 .286 .289 .290Model .283 .283 .283 .283 .283 .283 .283Empirical .286 .285 .285 .286 .285 .285 .28573 Simulated .190 .189 .185 .182 .188 .190 .187Model .197 .196 .197 .197 .197 .197 .197Empirical .182 .180 .181 .181 .181 .181 .1817 Simulated .193 .193 .192 .185 .190 .195 .191Model .185 .184 .184 .184 .184 .184 .184Empirical .187 .187 .187 .187 .187 .187 .187The final column of this table suggests the model-based estimator does underestimate thestandard errors for r1 and r2, and overestimate the standard error for r3 . We may also note thatthe standard error of y is underestimated by the model-based estimator in these simulations.Chapter 4. Results of the Simulation Study^ 88This part of the table is also consistent with the observation that the empirical estimator appearsto slightly underestimate the standard errors. Whether this is true for other combinations of yand 0 remains speculation.In summary, for the correct variance simulations, both the model-based and the empiricalestimators of standard errors perform well. However in the misspecified variance simulations,there is some evidence that the model-based estimator does not perform well and that theempirical estimator may underestimate the standard errors slightly.Chapter 4. Results of the Simulation Study^ 89Table 22. Observed Rejection Probabilities Under Alternative Hypothesis.A. Correct Variance Function7 Test0 = 1 0 = 1.4 0 = 2 0 = 3.10 .05 .01 .10 .05 .01 .10 .05 .01 .10 .05 .01.05 Wald/Mod. .121 .065 .014 .116 .062 .014 .121 .066 .012 .115 .063 .020Wald/Emp. .124 .069 .015 .120 .062 .012 .127 .072 .014 .113 .066 .020Score/Emp. .124 .070 .014 .116 .063 .012 .126 .067 .013 .110 .063 .017Dev/Scaled .121 .065 .014 .116 .062 .014 .122 .067 .013 .115 .064 .022.10 Wald/Mod. .185 .104 .027 .170 .102 .030 .130 .060 .016 .150 .087 .018Wald/Emp. .196 .110 .030 .171 .103 .034 .129 .068 .018 .159 .085 .019Score/Emp. .196 .108 .028 .167 .100 .032 .130 .068 .016 .156 .081 .017Dev/Scaled .185 .108 .027 .176 .104 .030 .131 .061 .017 .151 .089 .020.15 Wald/Mod. .298 .191 .076 .201 .126 .038 .195 .111 .019 .165 .087 .027Wald/Emp. .300 .189 .081 .205 .134 .039 .188 .115 .027 .161 .095 .025Score/Emp. .299 .190 .080 .206 .131 .040 .190 .114 .027 .159 .093 .023Dev/Scaled .299 .192 .076 .206 .128 .040 .196 .113 .019 .167 .092 .028B. Misspecified Variance Function7 Test0 = 1 0 = 1.4 0 = 2 0 = 3.10 .05 .01 .10 .05 .01 .10 .05 .01 .10 .05 .01.05 Wald/Mod. .121 .065 .014 .136 .075 .019 .120 .062 .018 .122 .052 .012Wald/Emp. .124 .069 .015 .129 .079 .023 .117 .062 .018 .118 .047 .011Score/Emp. .124 .070 .014 .129 .076 .020 .116 .058 .016 .111 .042 .007Dev/Scaled .121 .065 .014 .137 .076 .019 .122 .063 .018 .121 .053 .013.10 Wald/Mod. .185 .104 .027 .165 .092 .031 .167 .096 .024 .142 .078 .016Wald/Emp. .196 .110 .030 .159 .090 .034 .165 .099 .020 .135 .074 .012Score/Emp. .196 .108 .028 .159 .089 .032 .161 .097 .019 .129 .068 .012Dev/Scaled .185 .108 .027 .165 .095 .031 .168 .097 .024 .145 .080 .018.15 Wald/Mod. .298 .191 .076 .253 .166 .055 .219 .136 .047 .190 .106 .027Wald/Emp. .300 .189 .081 .253 .166 .055 .212 .137 .048 .184 .105 .024Score/Emp. .299 .190 .080 .250 .163 .052 .207 .133 .046 .179 .104 .022Dev/Scaled .299 .192 .076 .254 .167 .056 .221 .138 .048 .191 .109 .028The power of the four test statistics under study to detect the simulated pollution effectis summarized in Table 22; this table is similar in nature to Table 6 for the one series simplesimulations. The results of the correct variance simulations suggest very low power for all teststatistics in this more complicated case (only marginally higher than the nominal levels for thesmallest simulated value of -y), but this is not surprising given the variability of the estimatesChapter 4. Results of the Simulation Study^ 90of 7 compared to their simulated values. As in Table 6 the power of the tests tends to decreaseas the amount of dispersion increases. The test statistics show similar rejection probabilities inthe misspecified variance simulations. As in Table 6, the rejection probabilities, in general, areslightly higher in the misspecified variance simulations than in the correct variance simulationsbecause for each simulated value of 0, there is less dispersion in the data under the misspecifiedvariance formulation. Thus, except for the fact that the rejection probabilities are lower in Table21 than in Table 6 due to the larger standard errors of y in these more complicated simulations,the test statistics perform similarly in the simple and complicated simulations with one series.We now summarize the findings of these simulations using the more complicated model fora single series of data. Estimates of 0 and of the regression parameters are relatively unaffectedby the presence of over-dispersion although the estimators of these parameters became morevariable as expected. Standard errors of the regression parameters were accurately estimatedwith either the model-based estimator or the empirical estimator when the data was generatedusing the correct variance formulation, but there is some evidence the model-based estimatorperforms poorly when the misspecified variance formulation was used to generate data. Theempirical estimator appeared to do better in the misspecified variance simulations but it seemedto underestimate the standard errors when the level of dispersion was large relative to the overallmean level (this was not so apparent in the simulations under the reduced model reported inTable 19, but was more readily seen in the simulations under the full model reported in Table21). In these simulations the problems with estimating the standard errors did not translate intoproblems with the model-based test statistics. This is in contrast to the simple case simulationswith three series where poor performance of the model-based estimator of variances appearedChapter 4. Results of the Simulation Study^ 91to result in poor performance of the model-based test statistics. There were no detectabledifferences among any of the four test statistics under study.Chapter 4. Results of the Simulation Study^ 924.5 The More Complicated Case with Three SeriesThe last case to be considered is the simulations based on (3.19) describing the more complicatedcase with three series. Recall that the simulated values for the mean and temporal parameterswere:Pi = log 0.75 /32 = .70 /33 = 1.6Tii = 0.2 T21 = 0.2 731 = 0.2T12 = 0.2 T22 = 0.2 T32 = 0.3713 = -0.2 T23 = -0.4 733 = -0.4T41 = 0 T42 = 0 T43 = 0The first simulations to be considered use the null model (ryo = 71 = 72 = 73 = 0) togenerate data. The mean values of the parameter estimates from fitting the null model to thesedata sets are summarized in Table 23. For the correct variance simulations, we see that in bothcombinations, ii. is again estimated to be the average of the dispersions. The mean values ofthe regression parameter estimates are close to the simulated values.Table 23. Mean Values of Parameter Estimates.ParameterVarianceA. CorrectFunctionB. MisspecifiedCombination of 0I^IICombination of 0I^II4) 1.998 2.785 5.157 8.676pi -0.294 -0.292 -0.291 -0.300/32 0.695 0.697 0.692 0.700/33 1.597 1.594 1.595 1.575T11 0.204 0.203 0.197 0.205T12 0.204 0.198 0.197 0.211ro -0.198 -0.199 -0.195 -0.193721 0.205 0.198 0.205 0.203T22 0.204 0.201 0.206 0.200723 -0.397 -0.397 -0.397 -0.407731 0.202 0.199 0.198 0.217732 0.305 0.304 0.302 0.321T33 -0.396 -0.398 -0.395 -0.396Chapter 4. Results of the Simulation Study^ 93In the misspecified variance simulations, the estimates of cb are in the same neighborhood asthe estimates in the three series simple simulations. This is expected because the average of thelevels of dispersion over all seasons in all series in these simulations is very close to the averageof the three levels of dispersion in the three series simple simulations. The mean values of theregression parameter estimates are fairly close to the simulated values. Any deviations fromthe simulated values that we see here are likely random error due to the substantial amount ofvariability in the data sets, especially in the simulations using combination II.We now turn our attention to the standard deviations of the above estimates and the meanvalues of the estimated standard errors povided in Table 24. As in the three series simple case, inthe correct variance simulations, the model-based standard errors for the parameters associatedwith the first series (th and ri j) are too large. This is a result of using the estimate c (estimatedusing data from all three series) to calculate the standard error for parameters corresponding toa series where the simulated dispersion is lower than 4. Analogously, we see that the standarderrors for 03 and T3i are too small. For the regression parameters corresponding to series 2,we also find predictable results. Using combination I, the standard errors for 02 and .7-23 areabout right because the simulated 0 for this series is equal to the average of the 0 for all threeseries. However, using combination II, the simulated 0 for series 2 is smaller than 4 and we seethat the model-based standard errors are larger than the standard deviations of the parameterestimates.Chapter 4. Results of the Simulation Study^ 94Table 24. Standard Deviations of Estimatesand Mean Values of Estimated Standard Errors (±s.d.).Parameter MethodVarianceA. CorrectFunctionB. MisspecifiedCombination of (g)I^IICombination of 0I^II0 Simulated .084 .140 .407 .973th Simulated .085 .105 .081 .103Model .124 (.006) .146 (.009) .198 (.011) .258 (.019)Empirical .087 (.006) .103 (.008) .087 (.006) .100 (.008)132 Simulated .076 .074 .092 .091Model .075 (.003) .089 (.004) .121 (.007) .156 (.011)Empirical .075 (.005) .075 (.005) .092 (.007) .092 (.007)/33 Simulated .058 .074 .111 .157Model .048 (.002) .057 (.002) .077 (.005) .101 (.009)Empirical .058 (.004) .075 (.005) .111 (.010) .153 (.018)rii Simulated .115 .141 .107 .136Model .167 (.006) .197 (.009) .268 (.013) .348 (.023)Empirical .118 (.006) .139 (.007) .118 (.006) .136 (.007)r12 Simulated .111 .141 .113 .140Model .167 (.006) .197 (.009) .268 (.013) .347 (.023)Empirical .118 (.006) .139 (.007) .118 (.006) .135 (.007)r13 Simulated .130 .154 .125 .150Model .145 (.007) .218 (.010) .295 (.015) .384 (.025)Empirical .130 (.006) .154 (.008) .130 (.007) .147 (.008)r21 Simulated .106 .105 .133 .124Model .102 (.003) .120 (.004) .164 (.008) .211 (.014)Empirical .101 (.005) .101 (.005) .128 (.007) .128 (.007)r22 Simulated .101 .102 .130 .125Model .102 (.003) .120 (.004) .164 (.008) .211 (.013)Empirical .101 (.005) .101 (.005) .128 (.007) .128 (.007)723 Simulated .115 .114 .138 .138Model .119 (.004) .141 (.006) .192 (.010) .248 (.016)Empirical .119 (.006) .118 (.006) .135 (.007) .135 (.008)731 Simulated .081 .099 .163 .216Model .065 (.002) .077 (.002) .104 (.005) .136 (.009)Empirical .079 (.004) .102 (.005) .156 (.010) .216 (.018)732 Simulated .077 .098 .158 .212Model .063 (.002) .075 (.002) .102 (.005) .133 (.008)Empirical .077 (.003) .100 (.005) .156 (.010) .216 (.018)r33 Simulated .092 .119 .166 .225Model .076 (.002) .090 (.003) .122 (.006) .160 (.011)Empirical .092 (.004) .119 (.007) .159 (.010) .218 (.018)Chapter 4. Results of the Simulation Study^ 95The empirical estimates, on the other hand, are very good for all parameters (and at bothlevels of dispersion). There are only small differences between the empirical estimates and theobserved standard deviations and there is no indication here that this estimator consistentlyunderestimates the standard errors. Given the precision of the observed standard deviations andthe average of the standard errors (discussed in the previous section), the observed differencescan be explained by chance.In the misspecified variance simulations, it is even more apparent that the model-basedestimator of the standard errors performs poorly while the empirical estimates are quite good.One could not say that the empirical estimates are consistently too low in these simulations.Table 25 summarizes the observed rejection probabilities of hypothesis tests for a commonpollution covariate (H: 70 = 0) using the data generated under the reduced model.Table 25. Observed Rejection Probabilities: Common to Null Model.TestVarianceA. CorrectFunctionB. MisspecifiedCombination of q5 Combination of q50.10I0.05 0.01 0.10II0.05 0.01 0.10I0.05 0.01 0.10II0.05 0.01Wald/Mod .135 .081 .016 .180 .108 .033 .206 .147 .056 .247 .153 .058Wald/Emp .101 .047 .006 .112 .055 .014 .116 .058 .021 .108 .056 .017Score/Emp .099 .045 .006 .112 .054 .010 .112 .057 .018 .102 .053 .016Dev/Scaled .135 .081 .016 .180 .108 .033 .206 .146 .056 .246 .153 .058In the correct variance simulations we find that with both combinations of 0, the empiricaltest statistics do reasonably well, achieving levels close to the nominal rejection probabilities.The model-based tests reject too often in the simulations using the first combination of 0 andeven more so with the second combination. Notice again how very similar the results are for theChapter 4. Results of the Simulation Study^ 96two model-based tests. As in Table 24, this overall pattern is even more clear in the misspecifiedvariance simulations.Thus in this three series complicated case we find that the parameter estimates are adequateas are the empirical estimates of the standard errors. However, as in the three series simplecase, we find that the model-based estimator of standard errors performs poorly, presumably asa result of using a single estimate of 0 when in fact different amounts of dispersion are simulatedin each series. This poor performance of the model-based estimator appears to carry throughto the model-based test statistics. This should be expected in the case of the model-based Waldtest but it is interesting that it is also true of the scaled deviance test.We now consider simulations using a model which includes the pollution covariate commonto all three series; the simulated value is 7 0=0.1. The results summarized in Table 26 indicatethat using either of the variance formulations, the overall effects and season effects are estimatedjust as well as they were in the null simulations. The mean values of the estimates of thepollution parameter are also close to the simulated value of 0.1.Chapter 4. Results of the Simulation Study^ 97Table 26. Mean Values of Parameter Estimates.ParameterVarianceA. CorrectFunctionB. MisspecifiedCombination of 0I^IICombination of 0I^II0 1.997 2.789 5.202 8.686Al -0.293 -0.299 -0.294 -0.29902 0.695 0.695 0.694 0.694/33 1.601 1.597 1.593 1.589Tii 0.203 0.212 0.205 0.202712 0.202 0.208 0.201 0.205ro -0.195 -0.193 -0.198 -0.192721 0.203 0.204 0.201 0.200722 0.200 0.199 0.201 0.200723 -0.397 -0.400 -0.402 -0.402731 0.196 0.198 0.200 0.192T32 0.297 0.302 0.301 0.289733 -0.407 -0.402 -0.406 -0.40070 0.103 0.101 0.098 0.103A summary of the standard deviations of the above parameter estimates as well as the meanvalues of the estimated standard errors of the regression parameters are presented in Table 27.For the correct variance simulations, as in Table 24, the model-based standard errors for piand nj are overestimated while those for 133 and 733 are underestimated. Using combination Iof 0, the model-based standard errors for /32 and rzi are about right (as expected, see above).With combination II of 0, the model-based standard errors for /32 are only slightly too largeand somewhat surprisingly, only that for 723 is too large (recall that in Table 24 721, T222 _723 were all overestimated on average). We also see that the model-based standard errors for-yo are underestimated. This is consistent with the larger than expected rejection probabilitiesin Table 25 for the model-based Wald test. The empirical estimates, on the other hand, arevery good for all regression parameters.Parameter MethodVarianceA. CorrectCombination of 0I^II0 Simulated .083 .140fii Simulated .100 .124Model .133 (.006) .158 (.009)Empirical .100 (.006) .119 (.007)/32 Simulated .087 .097Model .087 (.003) .103 (.004)Empirical .089 (.005) .094 (.005)/33 Simulated .076 .095Model .063 (.002) .074 (.002)Empirical .075 (.005) .095 (.006)rii Simulated .156 .186Model .185 (.006) .218 (.008)Empirical .147 (.005) .176 (.007)r12 Simulated .150 .180Model .185 (.006) .217 (.008)Empirical .147 (.005) .176 (.007)r13 Simulated .130 .159Model .190 (.007) .226 (.010)Empirical .134 (.007) .159 (.009)rzi Simulated .133 .149Model .128 (.003) .151 (.004)Empirical .134 (.005) .148 (.006)r22 Simulated .132 .150Model .129 (.003) .152 (.004)Empirical .134 (.005) .149 (.006)r33 Simulated .119 .120Model .123 (.004) .145 (.006)Empirical .122 (.006) .122 (.006)r31 Simulated .120 .151Model .101 (.002) .120 (.003)Empirical .118 (.007) .148 (.010)r32 Simulated .121 .149Model .101 (.002) .120 (.003)Empirical .118 (.007) .148 (.010)r33 Simulated .094 .120Model .078 (.002) .093 (.003)Empirical .096 (.005) .123 (.007)7o Simulated .070 .086Model .061 (.001) .072 (.002)Empirical .068 (.003) .084 (.004)FunctionB. MisspecifiedCombination of 0I II^.423^.987.119^.152.215 (.012) .279 (.021).121 (.006) .148 (.011).122 .149.140 (.008) .181 (.012).123 (.007) .143 (.011).036 .192.102 (.005) .132 (.008).138 (.012) .188 (.022).200 .266.297 (.014) .385 (.024).201 (.011) .257 (.020).196 .266.298 (.014) .385 (.024).201 (.011) .257 (.020).130 .161.307 (.016) .397 (.027).134 (.007) .150 (.008).206 .256.207 (.009) .267 (.015).207 (.012) .253 (.021).209 .264.208 (.009) .268 (.015).208 (.012) .254 (.021).136 .140.199 (.010) .257 (.017).137 (.007) .137 (.007).219 .311.163 (.006) .212 (.011).224 (.020) .303 (.038).225 .309.163 (.006) 212 (.011).225 (.021) .306 (.040).162 .228.127 (.006) .164 (.011).160 (.010) .217 (.17).127 .179.098 (.004) .127 (.006).127 (.010) .171 (.18)Chapter 4. Results of the Simulation Study^ 98Table 27. Standard Deviations of Estimatesand Mean Values of Estimated Standard Errors (±s.d.).Chapter 4. Results of the Simulation Study^ 99In the misspecified variance simulations, the model-based standard errors for th and theTli are overestimated while those for 03 and the r32 are underestimated. The model-basedstandard errors for 02 are overestimated with both combinations of (1) while the standard errorsfor r21 and r22 appear quite good compared with the standard error of r 23 which is clearlyoverestimated. These results are similar to the correct variance results but are again differentfrom the results of the simulations using the null model as summarized in Table 25. As for yo ,the standard error for this parameter appears to be underestimated on average, just as it wasin the correct variance simulations.The empirical estimates are close to the simulated values with combination I, but there is ahint the estimates are too low on average with combination II (although keeping the precisions ofthe estimates and standard deviations in mind, the differences seen here could be just chance).The empirical estimates seem to approximate the simulated standard deviations better herethan they did in the single series complicated simulations (for data generated with the modelincluding the pollution effect; see Table 21). The simplest explanation for the poorer results inthe one series complicated case would be that in those simulations we considered much higherlevels of dispersion relative to the overall mean value (0 = log 0.75) than in this three seriescase and these higher levels of dispersion contribute to underestimation of the standard errors.The levels of the four test statistics in a model reduction from separate pollution effects foreach series to a common effect using the data generated with a common pollution effect aresummarized in Table 28. Under both variance formulations and for both combinations of cb,the empirical test statistics adhere reasonably well to the nominal levels, but the model-basedtests do not reject as often as they should. For the model-based Wald test this would suggestChapter 4. Results of the Simulation Study^ 100the standard errors for the separate pollution effects are overestimated (for at least one of theparameters). Again the model Wald and scaled deviance tests perform almost identically.Table 28. Observed Rejection Probabilities: Separate to Common.A. CorrectCombination of 0Variance FunctionB. MisspecifiedCombination of 0TestI0.10 0.05 0.01II0.10 0.05 0.01I0.10 0.05 0.01II0.10 0.05 0.01Wald/Mod .Wald/Emp .Score/Emp .Dev/Scaled .074 .032 .008 .056 .027 .004 .054 .028 .010 .055 .027 .004096 .053 .012 .108 .050 .004 .117 .058 .016 .098 .057 .014093 .050 .009 .106 .046 .004 .111 .058 .013 .097 .054 .011074 .032 .008 .058 .028 .004 .053 .028 .010 .056 .028 .004The powers of these tests to detect the common pollution effect represented by the simulatedvalue of 70=0.10 are summarized in Table 29. In both variance formulations the model-basedtests reject more often than the empirical tests (as expected given the underestimation of thestandard errors of -yo and the consistent similarity of performance of the model Wald and scaleddeviance tests). Predictably, the rejection probabilities decrease as 0 increases.Table 29. Observed Rejection Probabilities: Common to Null.Variance FunctionA. Correct^ B. MisspecifiedCombination of 0Combination of 0TestI0.10 0.05 0.01II0.10 0.05 0.01I0.10 0.05 0.01II0.10 0.05 0.01Wald/Mod .522 .417 .224 .433 .334 .168 .335 .233 .102 .295 .219 .115Wald/Emp .451 .340 .147 .338 .233 .083 .205 .112 .043 .181 .118 .047Score/Emp .451 .340 .143 .338 .230 .082 .203 .108 .042 .171 .113 .038Dev/Scaled .522 .418 .225 .434 .334 .168 .336 .234 .103 .298 .219 .115The final model we will consider for the more complicated simulations with three series willbe the most general one which includes separate pollution effects for each series. The simulatedChapter 4. Results of the Simulation Study^ 101values were 70=0.1, 71=0.1, 72=-0.2 and y3=0. We begin as always with a table summarizingthe mean values of the parameter estimates from 1000 simulated data sets generated with theabove model.Table 30. Mean Values of Parameter Estimates.ParameterVarianceA. CorrectFunctionB. MisspecifiedCombination of 0I^IICombination of 0I^II1.994 2.789 5.187 8.66401 -0.289 -0.295 -0.299 -0.28902 0.694 0.695 0.695 0.69803 1.601 1.595 1.595 1.569711 0.198 0.203 0.204 0.188T12 0.203 0.202 0.205 0.193T13 -0.209 -0.201 -0.191 -0.202T21 0.203 0.205 0.196 0.195T22 0.203 0.203 0.203 0.192T23 -0.393 -0.400 -0.392 -0.399T31 0.200 0.200 0.193 0.228T32 0.299 0.302 0.298 0.323733 -0.405 -0.401 -0.407 -0.4027o 0.100 0.101 0.100 0.08771 0.099 0.103 0.106 0.12472 -0.200 -0.202 -0.195 -0.186The estimates of /3 and T 3 are very similar to those in Tables 23 and 26 for both correct andmisspecified variance formulations. In the correct variance simulations, the estimates are verygood for 70 , 71 and 72, as they are in the misspecified variance simulations using combination I of0. However the estimates of these pollution parameters differ noticably from the true values inthe simulations using combination II of 0, although the differences are not too great consideringthe amount of variability in the simulated data which is reflected in the standard deviationsof these parameter estimates as shown in Table 31. Note that the predictor corresponding toChapter 4. Results of the Simulation Study^ 102the overall pollution effect is not orthogonal to the predictors corresponding to the separateeffects. The mean values of the parameter estimates in a reparameterized model with orthogonalpredictors representing three separate pollution effects, as opposed to a main effect and twoseparate effects, would generally be closer to the simulated values (although the standard errorsof those estimates would be smaller). In that case the interpretation of the table would remainthe same as above; that is, the mean values of the estimates differ from the simulated valuesbut are within acceptable limits considering the precisions of these mean values.In the correct variance simulations the mean values of the model-based standard errors for/3i and r 3 provided in Table 31 are very similar to those observed in Table 24 as one wouldexpect. Recall that following the simulations with a common pollution effect (Table 27) therewas some question about the model-based standard errors for 7 21 and r22 (they were noticablyoverestimated for combination II in Table 24 but did not appear to be overestimated in Table27). The results in the above table and Table 24 agree with the conclusion that standard errorsare underestimated (overestimated) on average for parameters that correspond to data serieswith more (less) dispersion than the estimate (7). We note that the estimated model-basedstandard error for the parameter 7 1 appears quite a bit larger than it should be. This wouldexplain the large observed rejection probabilities for the model-based Wald tests (relative tothe empirical) seen in Table 29. The empirical estimates of the standard errors for the pollutionparameters look very good in these correct variance simulations.Parameter MethodA. CorrectCombination of 0I^II4, Simulated .084 .143$1 Simulated .127 .145Model .178 (.006) .211 (.010)Empirical .125 (.008) .148 (.010)$2 Simulated .108 .108Model .108 (.003) .127 (.004)Empirical .107 (.007) .107 (.007)$3 Simulated .084 .103Model .069 (.002) .082 (.003)Empirical .084 (.005) .108 (.007)ni Simulated .213 .244Model .297 (.009) .352 (.013)Empirical .210 (.013) .247 (.017)r12 Simulated .212 .250Model .298 (.009) .353 (.013)Empirical .210 (.012) .248 (.017)ri3 Simulated .134 .165Model .196 (.008) .233 (.013)Empirical .138 (.007) .164 (.010)r21 Simulated .185 .188Model .185 (.005) .218 (.007)Empirical .184 (.012) .183 (.013)r22 Simulated .189 .186Model .186 (.005) .220 (.007)Empirical .184 (.012) .184 (.013)r23 Simulated .109 .115Model .115 (.004) .136 (.005)Empirical .115 (.006) .114 (.006)r31 Simulated .144 .171Model .116 (.003) .137 (.004)Empirical .141 (.009) .182 (.013)r32 Simulated .142 .170Model .116 (.003) .138 (.004)Empirical .141 (.009) .182 (.013)r33 Simulated .095 .124Model .078 (.002) .093 (.003)Empirical .095 (.005) .123 (.007)'Yo Simulated .092 .113Model .075 (.002) .089 (.002)Empirical .092 (.005) .119 (.007)71 Simulated .165 .195Model .207 (.005) .245 (.008)Empirical .164 (.006) .199 (.008)72 Simulated .156 .165Model .143 (.003) .169 (.004)Empirical .152 (.006) .169 (.007)Variance FunctionB. MisspecifiedCombination of ibI^II^.416 1.05.122^.142.287 (.015) .370 (.026).126 (.008) .143 (.010).139 .133.174 (.009) .224 (.015).133 (.010) .134 (.011).165 .232.112 (.005) .145 (.009).164 (.016) .224 (.026).207 .242.480 (.022) .620 (.040).210 (.012) .242 (.016).207 .241.481 (.022) .621 (.040).210 (.012) .242 (.016).139 .154.317 (.016) .409 (.029).139 (.007) .154 (.009).236 .232.298 (.014) .385 (.024).230 (.019) .231 (.020).238 .230.300 (.013) .387 (.025).231 (.019) .232 (.020).140 .136.186 (.010) .240 (.017).134 (.007) .134 (.007).284 .400.188 (.007) .243 (.012).316 (.032) .387 (.052).289 .404.188 (.007) .244 (.012).287 (.031) .392 (.051).159 .231.127 (.006) .165 (.012).160 (.010) .217 (.017).181 .266.122 (.004) .157 (.008).187 (.016) .256 (.029).220 .313.334 (.014) .431 (.026).231 (.014) .301 (.025).235 .303.230 (.009) .298 (.017).240 (.014) .297 (.026)Chapter 4. Results of the Simulation Study^ 103Table 31. Standard Deviations of Estimatesand Mean Values of Estimated Standard Errors (±s.d.).Chapter 4. Results of the Simulation Study^ 104With the misspecified variance simulations we again find the model-based standard errorsfor ij and to be very similar to the corresponding standard errors in Table 24. As in thecorrect variance case, the model-based standard error for 71 is overestimated. The empiricalestimates are also quite good even in the simulations using combination II of 0. There are someparameters for which the empirical estimator produces estimated standard errors which are onaverage too low but there is no clear indication of underestimation in this situation.Finally, in Table 32, we present the observed rejection probabilities to evaluate the powerof the four test statistics in the multiple parameter reduction from separate pollution effects toa common effect.Table 32. Observed Rejection Probabilities: Separate to Common.A. CorrectCombination of cbVariance FunctionB. MisspecifiedCombination ofTestI0.10 0.05 0.01II0.10 0.05 0.01I0.10 0.05 0.01II0.10 0.05 0.01Wald/Mod .358 .236 .102 .274 .168 .045 .120 .064 .012 .086 .047 .014Wald/Emp .441 .319 .144 .396 .290 .121 .350 .234 .076 .336 .226 .080Score/Emp .441 .315 .134 .390 .286 .115 .340 .224 .072 .325 .219 .071Dev/Scaled .359 .238 .102 .275 .168 .045 .120 .064 .012 .086 .047 .014The model-based test statistics have lower power than the empirical test statistics in thistable. Recall that the model-based estimator overestimated the standard errors for -y i (while thestandard errors for 72 were about right). This overestimation would explain why the rejectionprobabilities for the model-based Wald test are lower than for the empirical tests and, as usual,the scaled deviance test gives rejection probabilities that are almost identical to those for themodel-based Wald test.Chapter 4. Results of the Simulation Study^ 105The results of the simulations under the alternative hypotheses in this more complicatedcase with three series are qualitatively very similar to the results of the simple simulations withthree series. We found that the estimator of 0 performed adequately in estimating the averagelevel of dispersion and that the regression parameter estimator did well for all parametersincluding the "pollution" parameters that were of most interest. We also found the empiricalestimator of the variances to perform well in these simulations. The underestimation of standarderrors apparent in the more complicated case with a single series was not so clearly apparentin these three series simulations, possibly because the levels of dispersion considered in thissection were not as large relative to the overall mean levels as they were in the single seriessimulations where, in the simulations using the full model to generate data, only 0 = log 0.75was considered with values of (/) as large as 2 or 3. The model-based estimator of varianceperformed very badly in these three series simulations because the models did not allow for thepossibility that different series may have different amounts of dispersion. Thus in a sense, thevariance was actually misspecified in all the simulations (even those labelled "correct variance").The model-based test statistics considered also seemed to suffer from the misspecification ofthe variance function. When considering the levels of the model-based statistics, they rejectedtoo often in some instances and less often than they should have in others. The empirical tests,even the empirical Wald test, achieved levels much closer to the nominal levels.Chapter 5DiscussionThe analysis of relationships between air pollution and human health data from Prince Georgeprovided the motivation for carrying out a simulation study designed to evaluate the per-formance of possible variance estimators and test statistics available for making inference inover-dispersed Poisson models.Moderate amounts of over-dispersion are reported to have little effect on the estimation ofregression parameters (Cox, 1983), however the extra variability must be taken into accountwhen estimating variances or testing hypotheses. In the Prince George study, over-dispersionwas accounted for via a dispersion parameter 0 (common to all of the data series) which wasestimated by the deviance, G 2 , divided by its degrees of freedom. To adjust for this over-dispersion, standard errors of the regression parameters were multiplied by the square root ofthe estimate ii;G = G 2 /d.f. and the usual likelihood ratio test (based on a Poisson likelihood),or deviance statistic, for testing the viability of a reduced model nested within a larger model,was divided by ,.. Proceeding with this methodology, models were fit to the emergency roomvisits data that included temporal, meteorological and the pollution parameters of primaryinterest. The final models, resulting from the model reduction procedures (for example, the finalmodel for the pollutant TRS), included parameter estimates whose interpretation suggestedthat higher levels of pollution were related to lower numbers of emergency room visits for106Chapter 5. Discussion^ 107respiratory illness. Such counter-intuitive results raised questions about the appropriatenessof the methodology and motivated a simulation study. This study was designed to investigatethe possibility that the amounts of over-dispersion encountered in the Prince George studywere large enough (relative to the mean levels of the data series) to affect the estimation ofregression parameters, and to examine the performance of other possible variance estimatorsand test statistics that could be employed in the analysis of such over-dispersed Poisson data.An alternate estimator of the dispersion parameter involving the Pearson X 2 statistic,.x =X2 /d.f., was discussed and was found to be superior to the estimator (7)G. In additionto the model-based estimator of the covariance matrix of the estimated regression parametersdescribed in the methodology for the Prince George study, an empirical estimator, which doesnot rely on the correct specification of the variance function, was described. The two estimatorsof the covariance matrix give rise to two versions of the Wald test as alternatives to the scaleddeviance test used in the Prince George study for determining if certain parameters (such aspollution effects) contribute to the fit of the model. An empirical score statistic, suggested inBreslow (1990), which uses the estimating equations themselves for inference in over-dispersedPoisson models, was also considered.In the simple simulations described first in this thesis, the regression parameter estimateswere all very close to the simulated values, even when data were simulated with large amountsof over-dispersion and with a misspecified variance function. The simple simulations with onlyone series of data did not show any clear differences between the estimators of the variancesnor between the test statistics. With the three series simple simulations, the variance functionused to fit the data assumed a common 0 whereas a separate 0 applies in the simulation of eachChapter 5. Discussion^ 108series. Thus the variance function was always misspecified, even for the simulations labelled"correct variance". It appears that in this situation, the model-based estimator overestimates(underestimates) the standard errors for regression parameters corresponding to series withsimulated dispersion smaller (larger) than the estimated dispersion (which is the average ofthe simulated values from the three series). We found that, when considering the levels of thetest statistics, the observed rejection probabilities of the model-based test statistics often didnot approximate the nominal rates well whereas the empirical test statistics always achievedlevels close to nominal. We found an interesting agreement between the model Wald and scaleddeviance tests, as well as between the empirical score and empirical Wald tests.Using the more complicated model for a single series of data, estimates of 0 and of theregression parameters were relatively unaffected by the presence of over-dispersion although theestimators of these parameters became more variable and may have been affected by collinearitybetween some of the predictors. Standard errors of the regression parameters were accuratelyestimated with either the model-based estimator or the empirical estimator when the data weregenerated using the correct variance formulation, but there was some evidence the model-basedestimator performs poorly when the misspecified variance formulation was used to generatedata. It was hypothesized that the problem with the model-based estimator was similar tothe problem in the three series simple case. The observations in different seasons of the oneseries complicated case had different mean values and therefore, under the misspecified varianceformulation, had different amounts of dispersion (a situation similar to four separate seriesgenerated with the correct variance function). The empirical estimator appeared to do better inthe misspecified variance simulations but seemed to underestimate the standard errors when theChapter 5. Discussion^ 109level of dispersion was large relative to the overall mean level. In these one series complicatedsimulations, the problems with the model-based standard errors did not translate into poorperformance of the model-based test statistics; this is in contrast to the simple case simulationswith three series. There were no detectable differences among any of the four test statisticsunder study in this one series complicated case.The results of the simulations in the more complicated case with three series were verysimilar to the results of the simple simulations with three series. The estimator of q performedadequately in estimating the average level of dispersion and the regression parameter estimatorsdid well for all parameters including the "pollution" parameters that were of primary interest.The empirical estimator of the variances also performed well in these simulations. The under-estimation of standard errors apparent in the more complicated case with a single series wasnot so clearly apparent in the three series simulations, possibly because the levels of dispersionconsidered were not as large relative to the overall mean levels. The model-based estimatorof variance performed very badly in the three series simulations because, again, the modelsused to fit the data did not allow for the possibility of differing amounts of over-dispersion inthe different series. The model-based test statistics considered also seemed to suffer from themisspecification of the variance function. When considering the levels of the statistics, thesetests did not achieve the nominal rates. The empirical tests, even the empirical Wald test,achieved levels much closer to the nominal levels.Overall, it appears solutions of the score equations produce good estimates of regressionparameters in a situation with levels of over-dispersion similar to those in the Prince Georgestudy. The empirical estimator of the covariance matrix of the regression parameters appearsChapter 5. Discussion^ 110preferable to the model-based estimator which relies on correct specification of the variancefunction. It appears that in situations where the variance function is misspecified, the twomodel-based test statistics which rely on a single estimated over-dispersion parameter mayperform poorly. This is in sharp contrast to the empirical test statistics which always performedwell in this study. An interesting side note to the investigation of the test statistics was a veryclose agreement between the two model-based tests throughout the simulation study.Only brief speculations were made regarding the possibility that different, and perhapsmore appropriate, methods would have led to different conclusions in the Prince George study.It appears that, on average, the estimators of the regression parameters would yield correctestimates of the effects of pollution on emergency room visits, but it is unclear whether empiricalestimates of the standard errors or an empirical test statistic would have led to a different setof final parameters in the models.Bibliography[1] Breslow, N. (1984). Extra -Poisson Variation in Log -Linear Models. Applied Statistics, 33,38-44.[2] Breslow, N. (1990). Tests of Hypotheses in Overdispersed Poisson Regression and OtherQuasi-Likelihood Models. Journal of the American Statistical Association, 85, 565 -571.[3] Carroll, R. J. and Ruppert, D. (1988). Transformation and Weighting in Regression. NewYork: Chapman and Hall.[4] Cox, D. R. (1983). Some Remarks on Over -dispersion. Biometrika, 70, 269-274.[5] Inagaki, N. (1973). Asymptotic Relations Between the Likelihood Estimating Function andthe Maximum Likelihood Estimator. Annals of the Institute of Statistical Mathematics, 265,1-26.[6] Knight, K., Leroux, B., Millar, J., and Petkau, A.J. (1988). Air Pollution and Human Health:A Study Based on Hospital Admissions Data from Prince George, British Columbia. Reportprepared under contract with the Health Protection Branch, Department of National Healthand Welfare. Also issued as SIMS Technical Report 128, Department of Statistics, Universityof British Columbia (February 1989).[7] Knight, K., Leroux, B., Millar, J., and Petkau, A.J. (1989). Air Pollution and Human Health:A Study Based on Emergency Room Visits Data from Prince George, British Columbia.Report prepared under contract with the Health Protection Branch, Department of NationalHealth and Welfare. Also issued as SIMS Technical Report 136, Department of Statistics,University of British Columbia (June 1989).[8] Liang, K. Y. and Zeger, S. L. (1986). Longitudinal Data Analysis Using Generalized LinearModels. Biometrika, 73, 13-22.[9] McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd ed. London: Chap-man and Hall.[10] McNeney, B. and Petkau, A. J. (1991). Air Pollution and Human Health: A Follow - upStudy Based on Emergency Room Visits Data from Prince George, British Columbia. Reportprepared under contract with the Ministry of Health, Province of British Columbia.[11] Moore, D. F. (1986). Asymptotic Properties of Moment Estimators for OverdispersedCounts and Proportions. Biometrika, 73, 583 -588.[12] Royall, R. M. (1986). Model Robust Confidence Intervals. Intrernational Statistical Review,54, 199-214.[13] Wedderburn, R. W. (1974). Quasi-Likelihood Functions, Generalized Linear Models, andthe Gauss-Newton Method. Biometrika, 61, 439 -447.[14] White, H. (1982). Maximum Likelihood Estimation of Misspecified Models. Econometrica,50, 1-25.111
- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Overdispersion in poisson regression
Open Collections
UBC Theses and Dissertations
Featured Collection
UBC Theses and Dissertations
Overdispersion in poisson regression McNeney, W. Brad 1992
pdf
Page Metadata
Item Metadata
Title | Overdispersion in poisson regression |
Creator |
McNeney, W. Brad |
Date Issued | 1992 |
Description | Investigation of a possible relationship between air quality and human health in the community of Prince George, British Columbia was undertaken after a public opinion poll in 1972 discovered that poor air quality was the number one concern of the residents of Prince George. An analysis which attempted to identify such relationships using a data set including air quality measurements and hospital admissions for the period April 1, 1984 to March 31, 1986 is discussed in Knight, Leroux, Millar, and Petkau (1988). A similar analysis using emergency room visits during the same period rather than hospital admissions is described in Knight, Leroux, Millar, and Petkau (1989). The data set described here was collected to carry out a follow-up study to the emergency room visits analysis. The main part of the analyses carried out involved the use of Poisson regression models with a minor extension to account for over-dispersion in the data. The results of the analysis were not consistent with either the earlier study or with the expectations of the investigators. For example, higher levels of one of the air quality variables was found to be associated with a decrease in the number of emergency room visits for respiratory disease in the winter, but an increase in emergency room visits for respiratory disease in the fall. A mechanism to explain such effects is difficult to imagine. These counter-intuitive results motivated a simulation study to assess the methods used in the analysis and to compare these to other possible estimators and test statistics that can be employed in the analysis of over-dispersed Poisson data. |
Extent | 5996256 bytes |
Genre |
Thesis/Dissertation |
Type |
Text |
FileFormat | application/pdf |
Language | eng |
Date Available | 2008-09-17 |
Provider | Vancouver : University of British Columbia Library |
Rights | For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use. |
DOI | 10.14288/1.0086496 |
URI | http://hdl.handle.net/2429/2230 |
Degree |
Master of Science - MSc |
Program |
Statistics |
Affiliation |
Science, Faculty of Statistics, Department of |
Degree Grantor | University of British Columbia |
GraduationDate | 1992-05 |
Campus |
UBCV |
Scholarly Level | Graduate |
AggregatedSourceRepository | DSpace |
Download
- Media
- 831-ubc_1992_spring_mcneney_brad.pdf [ 5.72MB ]
- Metadata
- JSON: 831-1.0086496.json
- JSON-LD: 831-1.0086496-ld.json
- RDF/XML (Pretty): 831-1.0086496-rdf.xml
- RDF/JSON: 831-1.0086496-rdf.json
- Turtle: 831-1.0086496-turtle.txt
- N-Triples: 831-1.0086496-rdf-ntriples.txt
- Original Record: 831-1.0086496-source.json
- Full Text
- 831-1.0086496-fulltext.txt
- Citation
- 831-1.0086496.ris
Full Text
Cite
Citation Scheme:
Usage Statistics
Share
Embed
Customize your widget with the following options, then copy and paste the code below into the HTML
of your page to embed this item in your website.
<div id="ubcOpenCollectionsWidgetDisplay">
<script id="ubcOpenCollectionsWidget"
src="{[{embed.src}]}"
data-item="{[{embed.item}]}"
data-collection="{[{embed.collection}]}"
data-metadata="{[{embed.showMetadata}]}"
data-width="{[{embed.width}]}"
async >
</script>
</div>
Our image viewer uses the IIIF 2.0 standard.
To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0086496/manifest