- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Overdispersion in poisson regression
Open Collections
UBC Theses and Dissertations
Featured Collection
UBC Theses and Dissertations
Overdispersion in poisson regression 1992
pdf
Page Metadata
Item Metadata
Title | Overdispersion in poisson regression |
Creator |
McNeney, W. Brad |
Date Created | 2008-09-17 |
Date Issued | 2008-09-17 |
Date | 1992 |
Description | Investigation of a possible relationship between air quality and human health in the community of Prince George, British Columbia was undertaken after a public opinion poll in 1972 discovered that poor air quality was the number one concern of the residents of Prince George. An analysis which attempted to identify such relationships using a data set including air quality measurements and hospital admissions for the period April 1, 1984 to March 31, 1986 is discussed in Knight, Leroux, Millar, and Petkau (1988). A similar analysis using emergency room visits during the same period rather than hospital admissions is described in Knight, Leroux, Millar, and Petkau (1989). The data set described here was collected to carry out a follow-up study to the emergency room visits analysis. The main part of the analyses carried out involved the use of Poisson regression models with a minor extension to account for over-dispersion in the data. The results of the analysis were not consistent with either the earlier study or with the expectations of the investigators. For example, higher levels of one of the air quality variables was found to be associated with a decrease in the number of emergency room visits for respiratory disease in the winter, but an increase in emergency room visits for respiratory disease in the fall. A mechanism to explain such effects is difficult to imagine. These counter-intuitive results motivated a simulation study to assess the methods used in the analysis and to compare these to other possible estimators and test statistics that can be employed in the analysis of over-dispersed Poisson data. |
Extent | 5996256 bytes |
Genre |
Thesis/Dissertation |
Type |
Text |
File Format | application/pdf |
Language | Eng |
Collection |
Retrospective Theses and Dissertations, 1919-2007 |
Series | UBC Retrospective Theses Digitization Project [http://www.library.ubc.ca/archives/retro_theses/] |
Date Available | 2008-09-17 |
DOI | 10.14288/1.0086496 |
Degree |
Master of Science - MSc |
Program |
Statistics |
Affiliation |
Science, Faculty of |
Degree Grantor | University of British Columbia |
Graduation Date | 1992-05 |
Campus |
UBCV |
Scholarly Level | Graduate |
URI | http://hdl.handle.net/2429/2230 |
Aggregated Source Repository | DSpace |
Digital Resource Original Record | https://open.library.ubc.ca/collections/831/items/1.0086496/source |
Download
- Media
- ubc_1992_spring_mcneney_brad.pdf [ 5.72MB ]
- Metadata
- JSON: 1.0086496.json
- JSON-LD: 1.0086496+ld.json
- RDF/XML (Pretty): 1.0086496.xml
- RDF/JSON: 1.0086496+rdf.json
- Turtle: 1.0086496+rdf-turtle.txt
- N-Triples: 1.0086496+rdf-ntriples.txt
- Citation
- 1.0086496.ris
Full Text
OVERDISPERSION IN POISSON REGRESSION By W. Brad McNeney B. Sc. (Mathematics) University of British Columbia A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in THE FACULTY OF GRADUATE STUDIES DEPARTMENT OF STATISTICS We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA 1992 © W. Brad McNeney, 1992 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. (Signature) Department of ST I111 Si C S The University of British Columbia Vancouver, Canada Date /1 It. '2 DE-6 (2/88) Abstract Investigation of a possible relationship between air quality and human health in the community of Prince George, British Columbia was undertaken after a public opinion poll in 1972 discov- ered that poor air quality was the number one concern of the residents of Prince George. An analysis which attempted to identify such relationships using a data set including air quality measurements and hospital admissions for the period April 1, 1984 to March 31, 1986 is dis- cussed in Knight, Leroux, Millar, and Petkau (1988). A similar analysis using emergency room visits during the same period rather than hospital admissions is described in Knight, Leroux, Millar, and Petkau (1989). The data set described here was collected to carry out a follow-up study to the emergency room visits analysis. The main part of the analyses carried out involved the use of Poisson regression models with a minor extension to account for over-dispersion in the data. The results of the analysis were not consistent with either the earlier study or with the expectations of the investigators. For example, higher levels of one of the air quality variables was found to be associated with a decrease in the number of emergency room visits for respiratory disease in the winter, but an increase in emergency room visits for respiratory disease in the fall. A mechanism to explain such effects is difficult to imagine. These counter-intuitive results motivated a simulation study to assess the methods used in the analysis and to compare these to other possible estimators and test statistics that can be employed in the analysis of over-dispersed Poisson data. ii Table of Contents Abstract^ ii List of Tables^ v List of Figures^ vii Acknowledgement^ viii 1^The Prince George Study 1.1^Data Set Description ^ 1 2 1.1.1 Emergency Room Visits Data ^ 2 1.1.2 Air Pollution Data ^ 2 1.1.3 Meteorological Data 3 1.2 Methodology for the Prince George Study ^ 3 1.2.1 Poisson Regression ^ 3 1.2.2 Accounting for Over-Dispersion ^ 8 1.3 Results of the Data Analysis ^ 10 1.3.1 Temporal and Meteorological Effects ^ 11 1.3.2 Respiratory Visits and TRS ^ 12 1.3.3 Other Analyses of Pollution Covariates ^ 15 iii 1.4 Questions Raised by the Analysis ^ 15 2 Methodology^ 17 2.1 Quasi-Likelihood in Generalized Linear Models ^ 17 2.1.1 The Estimating Equations ^ 19 2.1.2 Test Statistics for Hypotheses Regarding Regression Parameters ^ 24 2.2 Test Statistics for Hypotheses in Over-Dispersed Poisson Regression ^ 27 3 A Simulation Study Related to the Prince George Study^ 31 3.1 The General Simulation Procedure ^ 34 3.1.1 Generating the Data ^ 34 3.1.2 Fitting the Full Model 35 3.1.3 Fitting the Reduced Model ^ 36 3.2 The Specifics of the Simulation Study 36 4 Results of the Simulation Study^ 42 4.1 Estimating the Dispersion Parameter 0 ^ 42 4.2 The Simple Case with a Single Series 50 4.3 The Simple Case with Three Series ^ 60 4.4 The More Complicated Case with a Single Series ^ 71 4.5 The More Complicated Case with Three Series 92 5 Discussion^ 106 Bibliography^ 111 iv List of Tables Table 1. Mean Values of Estimates (± Standard Deviation) in 1000 Simulated Data Sets. .43 Table 2. Mean Values of Estimates of 0. 50 Table 3. Standard Deviations of Estimates of /3 and Mean Values of Estimated Standard Errors (±s.d.). ^ 51 Table 4. Observed Rejection Probabilities Under Reduced Model. ^ 54 Table 5. Mean Values of Estimated Parameters (+ s.d.) Under Full Model. ^ 57 Table 6. Observed Rejection Probabilities Under Full Model. ^ 59 Table 7. Mean Values of Parameter Estimates. ^ 60 Table 8. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d ) ^ 62 Table 9. Observed Rejection Probabilities: Common Pollution to No Pollution Model. ^ 63 Table 10. Mean values of Parameter Estimates. ^ 64 Table 11. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors ( +s .d ) ^ 65 Table 12. Observed Rejection Probabilities: Separate to Common. ^ 66 Table 13. Observed Rejection Probabilities: Common to Null. 67 Table 14. Mean Values of Parameter Estimates. ^ 68 Table 15. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.). ^ 69 v Table 16. Observed Rejection Probabilities: Separate to Common. ^ 70 Table 17. Mean Values of Estimated Parameters Under Reduced Model. ^ 71 Table 18. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.). ^ 75 Table 19. Observed Rejection Probabilities Under Null Hypothesis. ^ 78 Table 20. Mean Values of Estimated Parameters Under Alternative Hypothesis. ^ 80 Table 21. Standard Deviations of Estimates Under Alternative Hypothesis. ^ 83 Table 22. Observed Rejection Probabilities Under Alternative Hypothesis. ^ 89 Table 23. Mean Values of Parameter Estimates. ^ 92 Table 24. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.). ^ 94 Table 25. Observed Rejection Probabilities: Common to Null Model. ^ 95 Table 26. Mean Values of Parameter Estimates. ^ 97 Table 27. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.) ^ 98 Table 28. Observed Rejection Probabilities: Separate to Common. ^ 100 Table 29. Observed Rejection Probabilities: Common to Null. 100 Table 30. Mean Values of Parameter Estimates. ^ 101 Table 31. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.). ^ 103 Table 32. Observed Rejection Probabilities: Separate to Common. ^ 104 vi List of Figures Figure 1. Comparison of Estimators of Phi: Correct Variance Simulations (Phi=1 or 1.4) ^ 45 Figure 2. Comparison of Estimators of Phi: Correct Variance Simulations (Phi=2 or 3) ^ 46 Figure 3. Comparison of Estimators of Phi: Misspecified Variance Simulations (Phi=1 or 1.4) ^ 47 Figure 4. Comparison of Estimators of Phi: Misspecified Variance Simulations (Phi=2 or 3) ^ 48 vii Acknowledgement I would like to express my gratitude to my advisor, Professor A. John Petkau, for his support, advice and patience throughout the writting of this thesis. I would also like to thank Professor Nancy E. Heckman for reading the manuscript and providing many helpful comments. viii Chapter 1 The Prince George Study Motivation for this work was provided by the analysis of a data set collected in Prince George, British Columbia, to study the possibility of associations between the ambient levels of air pollution and human health as measured by the rates of emergency room visits for respiratory illnesses. Investigation of such a possible relationship was undertaken after a public opinion poll in 1972 discovered that poor air quality was the number one concern of the residents of Prince George, ahead of such issues as crime, alcohol abuse and recreation facilities. Although monitoring of air quality parameters has been carried out at several monitoring stations in Prince George since 1980, most attempts to study this issue have compared hospital admissions rates and/or mortality rates in Prince George to those in other communities in British Columbia rather than attempt to identify an association between ambient levels of air pollution in Prince George and human health. An analysis which did attempt to identify such relationships using a data set including air quality measurements and hospital admissions for respiratory illnesses for the period April 1, 1984 to March 31, 1986 is discussed in Knight, Leroux, Millar, and Petkau (1988). A similar analysis for the same period based on emergency room visits rather than hospital admissions is described in Knight, Leroux, Millar, and Petkau (1989). The data set described here were collected to carry out a follow-up study to the emergency room visits analysis. 1 Chapter 1. The Prince George Study^ 2 1.1 Data Set Description Three measures of air quality were monitored and the daily numbers of visits to the emergency room (ER) for respiratory related illnesses were recorded during the period April 1, 1986 to March 31, 1988. In addition, meteorological data in the form of temperature and maximum relative humidity was collected. Description of the relationship between the pollution variables and the ER visits was the goal of the study. However, meteorological and temporal information was also included as potential explanatory factors so that any relationships discovered with pollution variables could reasonably be attributed to the pollution itself and not to some other factor that might underly both pollution and ER visits. 1.1.1 Emergency Room Visits Data Each visit to the emergency room at Prince George Regional Hospital by a resident of Prince George over the two year period from April 1, 1986 to March 31, 1988 was originally classified according to 35 diagnostic categories potentially related to respiratory illness. These 35 cate- gories were grouped into four broad categories which we will refer to as Asthma, Bronchitis+, Ear, and Others. Only the classification according to these four broad categories was available in the data set provided to us. A total of 8,079 visits were included in the study and there was one visit in at least one of the four respiratory categories on every day except January 15, 1987. 1.1.2 Air Pollution Data Data on three air quality parameters were available: Chapter 1. The Prince George Study^ 3 TRS — measured via a continuous monitor with readings subsequently converted to hourly averages. The average of these hourly averages becomes the daily average and this was the summary used. TSP — measured via a 24 hour vacuum sample once every six days. SO2 — measured via a continuous monitor in the same fashion as TRS. The daily average was the summary used. 1.1.3 Meteorological Data The meteorological data consisted of daily summaries for each of the following parameters: Temperature — the average of 24 hourly readings in degrees Celsius. Maximum Relative Humidity — the largest of 24 hourly readings. 1.2 Methodology for the Prince George Study The main part of the analyses carried out involved the use of Poisson regression models (or log-linear models). We give a brief description of these models and then describe an extension of Poisson regression models, which was required for the modelling of emergency room visits. 1.2.1 Poisson Regression The Poisson regression model is a special case of a larger class of models referred to as gener- alized linear models (GLMs), the theory of which is derived largely from the properties of the exponential family of distributions. Thus before we describe the Poisson regression model in Chapter 1. The Prince George Study^ 4 detail, a brief discussion of the exponential family and GLMs seems appropriate (see McCullagh and Nelder, 1989, for a more detailed description). For a random variable Y in the exponential family with canonical parameter 9 and (known) nuisance parameter 0, we may write its density function as f (y; 9) = exp{(y9 — b(0))/ a(0) c(y; q)}. Denote the mean value of Y as ,u, which is related to the parameter 9 by the function 9 = g(p). We will refer to the function g(.) as the link function. For making inference about 9 we introduce the log-likelihood function, the log of the above density function, which we denote 1(9). In the case of n independent observations Yi (i = 1, n), with possibly different canonical parameters, the joint log-likelihood is given by 1(9) = li(0i). In GLMs a linear model is specified for the parameters 0i of the form 0i = g(pi) = Xifi (where /3 is a vector of p<n parameters and Xi is the i th row of an n x p design matrix). It is then possible to estimate /3 via maximum likelihood by taking the derivative of the joint log-likelihood with respect to each of the parameters, equating the resulting p equations to zero and solving for the parameters of interest. These estimators have several nice characteristics including a limiting multivariate normal distribution with mean vector equal to /3w, the true value of the vector of parameters, and covariance matrix equal to the inverse of the Fisher Information matrix corresponding to the joint distribution of the Yi (i = 1, ..., n). Now consider a Poisson random variable Y. We write the density function of Y as f(y; it) = (e —PitY)/y! or equivalently as f(y; = exp{ylog — ,u — logy!}. The latter form allows us to identify the Poisson distribution as a member of the exponential family with 8 = log /2 (and therefore link function given by g(•)=log•)), b(9) = fit , a(0) = 1, and c(y; 0) = —logy!. The motivation behind using the Poisson regression model for the Prince George study lies Chapter 1. The Prince George Study^ 5 in the intuitively reasonable assumption that the daily numbers of emergency room visits follow Poisson distributions with possibly different means. The mean number of visits on a given day will be described by a function of the available covariate information up to and including that day, which includes temporal variables such as day of the week and month, the meteorological variables, daily temperature and maximum humidity, and the pollution variables, TSP, TRS and S0 2 . The use of Poisson regression depends on the following basic assumptions: (a) the expected number of visits on a given day depends explicitly only on time, meteorology and pollution and not on the number of visits on past days. (b) daily visits are conditionally independent given the temporal, meteorological and pollution covariates. The assumption that the expected number of visits does not depend explicitly on past visits seems to be a reasonable starting point in this situation. However, more general models in which the mean depends explicitly on past visits could be considered as well. Suppose now that ti is a vector of temporal covariates, mi is a vector of meteorological covariates and si is a vector of pollution covariates for day i; si and mi can include values of pollution or meteorology prior to day i as well as those values on day i. If yi is the number of emergency room visits on day i (i =1,2,...,731) for a particular diagnostic class, the log-linear model assumes that yi has a Poisson distribution with mean p, where log pi = a + ti er rni'6 si'ry with a, r, 6 and y unknown parameters (r, 6 and y are vectors having the same length as ti, mi and si respectively). The components of the parameter 7 give the relative changes in expected Chapter 1. The Prince George Study^ 6 visits on a given day (that is, with temporal and meteorological covariates fixed) associated with changes in the pollution variables. In particular, for small changes in the pollution variables, from si to si + Ai, exp(A',0) — 1 A'7 gives the percentage change in the corresponding expected visits. In order to account for possibly different patterns of emergency room visits for the different diagnostic categories, the basic model is generalized by allowing different parameters a, T, and y for the different diagnostic categories, and thus describing the expected number of visits for category j (j =1,2,3,4) on day i by the equation log fGij = cri^t'i rj To fit the above model into the framework described for GLMs let p = (pt, /4,4 /4) t and = (a t , rt, St, ,.yt)t where here a = (0 1 , 02 ,03 , 04 )t, T = 4, 71, Tv, and so on. Then, if X is the appropriate design matrix containing all of the covariate information, our model can be expressed as logy = X /3. In addition to assuming the observations to be conditionally independent given the co- variates, we assume that data from different diagnostic categories is independent, so that the joint likelihood is simply the product of the marginal Poisson distributions. Thus the joint log-likelihood is: 4 731 1(0) = E E [yii log pij — pij — log yij!] . j=1 1=1 For ease of notation, it will be more convenient to write the above log-likelihood using one subscript. We order the yii's as yl , ..., y2924, with the pi's numbered correspondingly. Now Chapter 1. The Prince George Study^ 7 express the joint log-likelihood as: 2924 l(0) = E [yi log — — log yi!) . i=i To obtain maximum likelihood estimating equations for the parameters in /3 (suppose there are p parameters in /3), we differentiate the expression in (1.1) with respect to each of the p components of and equate the resulting derivatives to zero to arrive at: al^ 2924 Yi — Pi ) ON = apa = 0; j = 1, ...,p.fr_1(^' (1.2) Note that our model assumes pi = exp{XiM, or log pi = X0. Thus, (1.2) can be rewritten as 81^2924 ni3 =^(yi —^Xii^0; j = 1,...,p.^ (1.3) "Pi^1=1 The resulting estimators, Q , are asymptotically normal, with mean vector given by /3*, the true value of the vector of parameters, and covariance matrix equal to L — E ( ija212 )] In this case ^021^2924^ 821 ^ — apt = —E ()(902^LT., (902 . Given the solution to (1.3), the fitted values, it, = exp{Xi;j}, may be substituted in the above expression to obtain an estimate of the asymptotic covariance matrix. Using these fitted values, the deviance is then defined to be —2 times the log-likelihood, or 2924 dev. = —2 E [yi log - - log NI . i=i For a fitted model with p parameters and a joint likelihood from n independent observations, the degrees of freedom associated with the deviance is n — p. By taking the difference in the Chapter 1. The Prince George Study^ 8 deviances of two fitted models, one of which is nested within the other, we may form a likelihood ratio statistic that can be compared to its limiting chi-square distribution to examine if the smaller model constitutes a significant deterioration in fit compared to the larger model. The Pearson residuals are defined to be (y, — r, — . These quantities are often used for diagnostic purposes in the same way the residuals from an ordinary regression (assuming normality) are used. We may also use these residuals to form a goodness-of-fit statistic, the familiar Pearson X 2 , by summing the squares of these Pearson residuals. Another potential use of the Pearson X 2 emerges in the following section dealing with over-dispersion. 1.2.2 Accounting for Over -Dispersion The extension of the Poisson regression model to be described concerns accounting for over- dispersion. Suppose that the variance of the daily visits given the temporal, meteorological and pollution variables is greater than the mean; in this situation the emergency room visits data are said to be "over-dispersed". Over-dispersion represents a violation of the Poisson assumption underlying the Poisson regression model, which requires that the variance and mean be equal. One way to handle over-dispersed data is to introduce a dispersion parameter 0 > 1 and assume that the variance of the visits on day i is We now describe the procedure for statistical inference in this new model. The parameter vector which appears in the specification of the daily mean visits rates is estimated exactly as for the model without a dispersion parameter, i.e. by maximizing the Chapter 1. The Prince George Study^ 9 Poisson log-likelihood function of (1.1). However, when assessing the statistical significance of estimated parameters it is necessary to take into account the dispersion parameter. To describe how this is done in the context of model selection (the dispersion parameter must also be taken into account when evaluating the standard errors of estimated parameters), assume that we begin with a model containing a number of parameters (the full model) and wish to test the viability of another model which is nested within the full model (the reduced model). Under our independence assumptions, the difference in the deviances for the two models is approximately distributed as 0 times a )(2 random variable with degrees of freedom given by the difference in the degrees of freedom associated with the two deviances. Thus the difference in deviances must be divided by an estimate of the dispersion parameter before it can be compared to the x2 distribution to assess whether reduction from the full model to the reduced model is permissible. One choice of an estimate of the dispersion parameter is the ratio of the sum of squared Pearson residuals, the Pearson X2 statistic, to its degrees of freedom (McCullagh and Nelder, 1989, p.200), both determined under the full model. Another estimate, which is often very close to the previous one, is the ratio of the deviance to its degrees of freedom, also under the full model. For this data the estimate based on the deviance was used; this leads to the following statistic for testing whether reduction from the full model to the reduced model is permissible: deviance (reduced) — deviance (full) A'dev. = deviance (full)/df (full)^; this statistic is compared to the x2 distribution with degrees of freedom given by Adf = df (reduced) — df (full). The Poisson regression model with unspecified dispersion parameter is a particular case of a Chapter 1. The Prince George Study^ 10 class of models called "quasi-likelihood" models, so-named because the likelihood which is used to obtain estimates, the Poisson likelihood in our case, is not the true likelihood for the data but plays the role of a likelihood and so is called a quasi-likelihood. Quasi-likelihood models are discussed in more detail in Chapter 2. We now describe the results of the analysis of the Prince George data using the methods described above. 1.3 Results of the Data Analysis The possibility of over-dispersion in a series of counts can be judged by the index of dispersion: X 2 = (n — 1)variance mean where n is the sample size, 731 in this case. Under sampling from a Poisson distribution, X 2 has an approximate chi-squared distribution with n-1 degrees of freedom and can be converted to an approximate standard normal variable by z = VT/Y 2 — ^2(n — 1) — 1. The results for the emergency room visits data are presented the following table: category mean variance X 2 z Asthma 0.76 0.80 763 0.87 Bronchitis+ 1.54 3.36 1593 18.2 Ear 3.62 6.92 1395 14.6 Other 5.13 14.2 2027 25.5 Except for the Asthma category, these exhibit a large degree of over-dispersion relative to the Poisson distribution; the values of z given in the above table clearly reflect extreme over- dispersion for the categories Bronchitis+, Ear and Other. In fact, these distributions exhibit the pattern of a mixture of Poisson distributions, having higher than expected numbers of very Chapter I. The Prince George Study^ 11 small and very large counts, at the expense of moderate counts. This over-dispersion must be taken into account in the data analysis. As mentioned previously, we must allow for the possibility that emergency room visits de- pend on factors other than just pollution, such as temporal and meteorological effects. These covariates were considered first, and model reductions were attempted to find a model which parsimoniously describes these effects. The pollution covariates were then added to the model and model reductions were attempted to identify the pollution covariates which had a substan- tial effect on emergency room visits for respiratory illness. 1.3.1 Temporal and Meteorological Effects The first step of the analysis involved the incorporation of adjustments for temporal effects. The full model for the log of the visits rate included separate effects for each of the 12 months and each of the 7 days of the week for each diagnostic category. Attempted model reductions to common month or common day-of-the-week effects, that is common across all four diagnostic categories, resulted in unacceptably large changes in adjusted deviance and therefore the full collection of separate effects for each diagnosis was retained as the temporal model. Next, the meteorological covariates (temperature and maximum relative humidity) were added to the temporal model at lags of 0 to 3 days; that is for any i = 4, ..., 731 we allow for the possibility that meteorology at times i, i — 1, i — 2 and i — 3 might affect the rate of emergency room visits at time i. We first allow for separate effects at each lag for each of the four seasons, winter (December to February), spring (March to May), summer (June to August) and fall (September to November), but with each of these effects common to all four diagnostic Chapter I. The Prince George Study^ 12 categories. Based on our model reduction criterion, these separate season effects were found to be unimportant compared to common effects across seasons for each lag. When separate meteorological effects at each lag for each diagnostic category were consid- ered, we found unacceptable increases in adjusted deviance when attempting to simplify the separate temperature effects to common effects. On the other hand, only common maximum humidity effects were needed. Thus our model which adjusts for possible temporal and meteoro- logical effects included separate diagnosis effects for month, day-of-the-week and temperature, and common maximum humidity effects. 1.3.2 Respiratory Visits and TRS We next investigated a model which expresses the daily emergency room visits rates for any particular diagnosis category as log visits rate on day i = temporal effects + meteorological effects + 7o log TRSi + 71 log TRSi—i + 72 log TRSi_2 7310g TRSi_ 3 . Here the effects of each of the TRS values are common across diagnosis categories; we also considered models involving separate effects for the four diagnosis categories or separate effects for the four seasons. For all models fitted, the temporal and meteorological effects had the same structure as previously identified. In what follows, we will abreviate this structure as the temimet model. We started with separate effects for each diagnostic category but found the reduction to common effects acceptable and furthermore removal of the TRS effects altogether resulted in Chapter 1. The Prince George Study^ 13 only a negligible deterioration in the fit. The following table examines separate TRS effects for each season, where as before the separate season effects are common to all diagnostic categories: model terms deviance d.f. A' dev. A d.f. tem/met effects + separate season TRS effects tem/met effects + common TRS effects tem/met effects only 3827 3852 3855 2804 2816 2820 18.4 1.7 12 4 Reduction to common effects leads to an increase in adjusted deviance of 18.4 versus a decrease of 12 in the number of parameters which is not so easily dismissed (p=0.11). Unfor- tunately the estimated effects from the full model present a confusing picture as we see in the following table which presents those estimates which are greater than one standard error in magnitude: effect coefficient s.e. Winter Lag 0 -0.026 0.020 Spring Lag 0 -0.027 0.026 Summer Lag 0 0.043 0.028 Winter Lag 2 -0.035 0.023 Summer Lag 2 -0.035 0.029 Looking at the effects at each lag we get the suggestion there may be some TRS effect at lags of 0 and 2 days. Unfortunately the effects from one lag to the other are not consistent and furthermore, it is difficult to understand why higher levels of TRS should be associated with higher numbers of emergency room visits at some times of the year and lower numbers at other times of the year. These results may not be particularly surprising since the overall reduction in adjusted deviance for going from the full model to one with the tem/met effects only (p=0.20) indicates that the separate effects in the above table could be large simply by chance. However, we could also examine the effects grouped by season as in the following table: Chapter 1. The Prince George Study^ 14 Lag Winter Spring Summer^Fall 0 1 2 3 -0.026 -0.035 -0.027 0.043 -0.035 where the dashes represent estimated effects which are less than their standard error. Looking down the columns of this table we see the possibility of a negative cumulative effect in Winter. Accordingly we now present results for a model which includes a single cumulative effect (an average of the logarithm of the lag 0 to lag 3 pollution measurements) for each of the four seasons: model terms deviance d.f. A' dev. A d.f. tem/met effects + separate season TRS effects tem/met effects + cumulative TRS effects tem/met effects only 3827 3837 3855 2804 2816 2820 6.7 13.4 12 4 While there is little difference in the fit of the separate season effects model and the cumu- lative effects model, the cumulative TRS effects constitute a substantial improvement in the fit over the tem/met only model (0 idev..13.4, Ad.f.=4, p=0.001). The estimates and standard errors of these cumulative effects are presented in the following table: effect coefficient s.e. Winter -0.060 0.023 Spring -0.002 0.033 Summer 0.009 0.034 Fall 0.075 0.030 These effects suggest that an increase in the level of TRS is associated with a decrease in the number of emergency room visits for respiratory disease in the winter, but an increase in the number of emergency room visits for respiratory disease in the fall. This model provides a more Chapter 1. The Prince George Study^ 15 parsimonious description of the effect of TRS on emergency room visits for respiratory disease than the season effects model, yet a mechanism to explain such effects is difficult to imagine. 1.3.3 Other Analyses of Pollution Covariates Similar analyses were carried out for SO 2 (as well as TRS and SO 2 simultaneously) and for TSP. Details of the model reductions and results can be found in McNeney and Petkau (1991). These analyses proceeded in the same fashion as the TRS analysis described above, and similar problems were encountered; that is, the parameter estimates obtained in the final models were such that no plausible explanation for their structure came to mind. 1.4 Questions Raised by the Analysis The results of the data analysis described above make it difficult to come up with a reasonable and coherent explanation of the effects of air pollution on human health based on this data set. There are many possible reasons for our results, such as: (1) the air pollution data do not accurately represent the exposure of the residents of Prince George to pollutants, (2) emergency room visits for respiratory illnesses are poor indicators of the effects of air pollution on human health, (3) some other important factor was not considered in the analysis, or (4) the method of analysis is potentially misleading. With respect to the last item, the following questions come to mind: Chapter 1. The Prince George Study^ 16 (a) Is the estimate of the scale parameter, 0, used in this analysis, namely deviance(full) d.f(full) ^a good estimate, or should we have used Pearson X 2 (full) ?d.f.(full)^• (b) Is the change in adjusted deviance criterion appropriate for model reduction in this situ- ation? (c) Are statistics other than the adjusted deviance better for testing hypotheses concerning the regression parameters when over-dispersion is present? (d) Given the amounts of over-dispersion in the emergency room visits series, are the estimates of the model parameters and standard errors accurate? These questions motivated the simulation study we describe in Chapters 3 and 4. Be- fore proceeding to the simulation study, Chapter 2 presents a more detailed description of quasi-likelihood theory and the estimating equations obtained from this theory for estimating regression and dispersion parameters. We also discuss some of the test statistics available for testing the significance of regression parameters. Finally, over-dispersed Poisson regression is discussed as a special case of quasi-likelihood. Chapter 2 Methodology In this section we describe the aspects of maximum likelihood theory (in the context of GLMs) which motivate the theory of quasi-likelihood first proposed by Wedderburn (1974) and also discussed by McCullagh and Nelder (1989, Chapter 9). In particular, we are interested in the estimating equations and test statistics derived from this theory. We also discuss the application of quasi-likelihood to the analysis of the Prince George data. 2.1 Quasi-Likelihood in Generalized Linear Models Recall the log-likelihood for a single random variable in the exponential family 1(9) = ( ye — b(0))1 a(0) + c(y; 0). Maximum likelihood theory assumes sufficient regularity conditions to ensure that the following relations hold ^ E [ 019^0 l ] = 0. ' and E [ °21 4- E {( 81 ) 21 =0. 2^09 In this discussion we will assume these conditions hold. The derivative of the log-likelihood is 011 00 = (y — b'(9))1 a(0), therefore E(011 00) = 0 implies that E(Y) = b'(9). We also have 021 0211^b"(9) — b"(0) so that — E ^=^ao2^a(0)^092 a(0) • 17 Chapter 2. Methodology^ 18 Finally E [(M 21 , E [CY a—( 1)0;(0))) 2]^1 — a2(0) Var Y, which leads to the expression Var(Y) = b"(0)0(0). Rewriting the log-likelihood in terms of the mean it, we have 1(0) =1(g(p)) and al_ 01 00 Op = ae ap' so that 01 iy — b'(0)1 00^y — V(0) ap —^a(q5) ) aµ — 40)4^• Since ay(^opp = b'(0) and b"(0) = 00e) . ,, the score function S(p) = -g- can be rewritten as S(IL) = (y — E(Y))/Var(Y). Note the follow- ing properties of the score function: 1. E(S) = 0, 2. Var(S) = E(S 2 ) = ( a- .17- 1-12.E(17 — 11)2 — Vari- (Y) , 3. -E(e)=ai&ly; - Now consider a random variable Y for which we do not specify a distribution but only suppose it has mean value p and a variance that is related to p by some function V(p; 0) where (kis an unknown parameter. Then if V(µ; q5) correctly specifies the variance, the function U* ( 17 ; Ft, 0) — Of — li)17(ti; 0) has similar properties to a score function resulting from a log-likelihood in that: Chapter 2. Methodology^ 19 1. E(U*) = 0, 2. Var(U*) = E(U* 2 ) = v2(1,, ;0E(17 — /1 ) 2 = 1 V (A; ek) EiV(A;c5)1•3. —E( aµ = E [V( 1t-47) (17^V2(µ;0 1 - ve—'m Since most asymptotic likelihood theory is based on the above three properties, we might expect the integral of^to behave like a log-likelihood. We refer to the integral fy^tY (y — t) dtQ(Y;^= j U* (Y;t, Odt = j V̂ (t; if it exists, as the quasi-log-likelihood or simply the quasi-likelihood. For n independent obser- vations having the same unknown parameter 0 we have the quasi-likelihood Qn(Y;^= E^0) i=i where now y and p are vectors of length n. As in GLMs, the mean vector p is assumed to be related to the regression parameters by the link function g(p) = X/3. 2.1.1 The Estimating Equations As in the case where we have a true likelihood, the approach with a quasi-likelihood is to differ- entiate Q T, with respect to each of the components of and equate the resulting p derivatives to zero to obtain maximum quasi-likelihood estimators for the parameters. If 0 were known, then under the assumption that pi = exp{XiP}, we would solve the following estimating equations: 0C2n^n (Yi^ ujo, 0) =^= E = 0 ' ^1, p. u1-1.7^v (2.4) We will refer to the p-dimensional vector (U 1 , ..., Up )' as U0 . Chapter 2. Methodology^ 20 Because 0 is unknown, we must introduce an additional estimating equation. This equation is based on the obvious notion that the variance function V(pi; 0) should reflect the true vari- ances of the observed yi (denote these an. Thus, 0 should be estimated so that V(tti; 0), the model-based variance, agrees well, in some sense, with the true variance q. Only estimates of the true variances are available from the data, so a sensible estimate of 0 is provided by the solution to the "moment" equation OP, =^ q)i=1 2 11 = 0.^ (2.5) To correct for the bias resulting from the simultaneous estimation of the p regression parameters, Breslow (1984) suggests modifying this equation slightly to WV , =^14)2 (n i=1 V(tti;^n^j = 0^ (2.6) in place of (2.5). Given an initial estimate of 0 we may use (2.4) to obtain an estimate of /3. This estimate of /3 can then be used in (2.6) to estimate 0. This procedure is iterated to obtain the joint solution ($, ) to the estimating equations (2.4) and (2.6). Subject to regularity conditions (Inagaki, 1973; White, 1982) this joint solution converges in probability, as the number of observations increases, to (0*, 0*) where d* is the true value of 0, and 0* is the value of 0 which satisfies the limit equation n^cr? 71^v(ili; 0) = 1. To motivate the limiting distribution of the estimator (), ik) consider the following. Let = (/3t, 0)t and let U(9) = (U0 ', PIT, where U0 is the vector form of the equations given by (2.4) and U 4' is given in (2.6). While (2.6) is not a maximum quasi-likelihood equation, =^ l nn aui Bm^n i 1 n aui(r)^au(e*) nco E E aot = limnoo n^aot Bm^i=1 au n a0t Chapter 2. Methodology^ 21 we may consider (2.4) and (2.6) as moment-type estimating equations. The derivation of the asymptotic properties of ([3,i4) will proceed along the same lines as the derivation of the properties of maximum likelihood estimators, although some modifications are required. The fact that 8 is the solution to U(a) = 0 along with the Mean Value Theorem allows us to write the Taylor series expansion of U(a) about the point 0* as 0 = U(a) = U(8*) + au00t - r),em where each of the p 1 elements of Om are between the corresponding elements of 0 and 0*. Re-arrangement allows us to write this as ‘77-20 —13*) ( _ 1 OU ■ n aot yi U (r). em (2.7) Under mild regularity conditions, 0 will converge in probability to 8* (see Moore 1986, Theorem 1, p.586). Then, since Oln is between a and 8* and au loot is a sum of n independent components, we have where here the subscript i refers to the contribution of the ith observation; i = 1,^n and the expectation is taken with respect to the true value 0* under the assumption that p = exp{X/3} correctly specifies the mean. Now —E[au(r)laet] is a (p 1) x (p + 1) matrix of the form [ An bn ctn do where auo^a r n (yi _ = —E^E apt^V(kti; O s') ad 0 .ao t Chapter 2. Methodology^ 22 = E ^1 ON ON VGai; 0*) 00 00t + E^(Yi –^°pi av °p i o•^v2(4; 04') 013 api aot E^ 1 ^atLi 0,L,= ii v(iti; 0*) 00 00t a n (yi^)^rt (17 — ^0,4 av] _ o— -bn = –E— = E (Pi; 0* ) " 84) 9*00 i=1 v (pi; 0) 00 0* a n (Y iti)2 (n _ p) cn =^[E^i*, ^ ^u P 1=1^VI"^0• and n (Y pi)2 (n P)11 = V"`ri (Y /102 aydo =^{E^ Eji=i n^cb*^V2(14;0) 00 Now consider the other random vector entering (2.4): n^ 2^29V Gri cb* = 2=1• V2 (Pii 0) 00 1 \–y., 1^ L-4.1 (,;;c•) 8/3 0* U(r) – \--T.^[v;;;Li 2 ^( p,i;0 )^n Assuming the regression model correctly specifies the mean, we have 1^ 0 E 77 U(0*) =lz_i7n)^• ^ L.-n=1 v(,e )^n By the definition of 0* and provided regularity conditions hold (see Moore 1986, Lemma 1, p.585), it follows that this random vector has a limiting multivariate normal distribution N (0,[ eSt fe I) where B = limn_,,,,, ;7_13n. is the normed limit of 71^f n^f^\^t Bn = E(IP3 P3t )= E [E [ i 14)1 ^ y P" affil Ivoli; o*) as 1=1 V(aii 0* ) 0/0 o* ; 0* Chapter 2. Methodology^ 23 Because the observations are assumed to be independent, this reduces to = E ^(Y, gi )2^°Pi]^it ^a?^ attiapi^B i.iv - (4;0*)^adt e* .^ a$ aot For our purposes, the forms of e and f are of no consequence. Combining these results with Slutsky's theorem we have ^Vii(o e* ) N (0,[^e fA ° 1 [ B et ^({ A 0 I ly)ct d ct where A =^c = lim^le and d = lim 00 in dn . Asn n [ A 0 1 - 1^ [ A -1^0 ^ct d d- 1 the asymptotic variance of VT.t(ë — r) is [ A- 1^0 I r .13 e 1 I A'^0—^d-1 [ et^— CA- 1 The asymptotic covariance matrix for ViT(S — /3*) is the upper left-hand p x p sub-matrix of the above matrix, namely A -1 BA -1 . Note that this is exactly the covariance matrix one would obtain for the estimation of Q based on the estimating equations (2.4), with 0 fixed at 0* rather than estimated; the asymptotic covariance for S is unaffected by the estimation of 0. This occurs despite the fact that Q and are, in general, correlated (the entries of the upper right-hand pxl vector of the above matrix need not be zero); it is a result of the fact that the pxl vector bn has all p elements identically equal to 0, that is, E[ 4 U 13] = 0. Note further that if the function V(pi; 0) correctly specifies the variances cr?, then A = B and VTz(S — /13*) has asymptotic covariance matrix given by A -1 . Thus we can consider two estimates of the asymptotic covariance of /3; the first is model- based and assumes that V(tti; 0) correctly specifies the variances. Denote this estimate 2A,/ = 9* Chapter 2. Methodology^ 24 A,7 1 where An is An with pi replaced by the fitted values^= pi(Xi; Si) and 0* replaced by -(4. The second estimate we refer to as the empirical estimate. Denote this EE = An ^An-1 where f3n^(yi — /10 2 i=1 v2 (4; ii)) aij apt The idea of using an empirical covaiance matrix in connection with misspecified models has been developed by numerous authors, including White (1982), Royall (1986), Liang and Zeger (1986), and Carroll and Rupert (1988, Section 4.3.2). We now have estimating equations for 8 and 0 which yield consistent estimators and we have an expression for the asymptotic covariance of fr-t(T3 — /3*). In the next sub-section, we use this information to form statistics to test the significance of the regression parameters. 2.1.2 Test Statistics for Hypotheses Regarding Regression Parameters Consider the p-dimensional vector /3 partitioned into /31 and /32 of dimensions pl and p2 re- spectively. Here the point is to think of /32 as parameters which have fixed values /32 (usually zero) under some null hypothesis H o . We will refer to the model where all of the parameters = (#1, 13V are estimated as the full model, and the model where 131 is estimated when 02 is set to its hypothesized value /l as the reduced model. Under Ho , the random vector 642 — /32) has a limiting multivariate normal distribution with mean 0 and covariance E22 which is estimated by 2 22 , the lower right-hand p2 x /32 sub-matrix of either 2A/ or 2E. We therefore have model-based and empirical versions of a test statistic we refer to as the Wald test where we compare the test statistic (th /36Dt (th /32 ) to the 42) distribution. in 00, ik)^ow, 0. ) + aou; 1 OUO VT1 (13 — n 00 — 0*), (2.8)frii 7 4,m13m1 4,m Chapter 2. Methodology^ 25 Another possible type of test, one that does not require fitting of the full model, is the score test. Recall from Section 2.1.1 that E[ i] = 0; j^1, p With sufficient regularity conditions, this implies that for any sequence of estimates^converging to 3* the statistics U0 (/3, ii)) and U43 ($, cb*) are asymptotically equivalent. This can be seen by comparing the following two asymptotic expansions: and U1 0 13 Cb* ) =^OW, 0*) —VT/^VT/ n 0/3 V-17( - 0*)fmz (2.9) for some Om. (i=1,2) between p- and /3* and similarly for some om• The right-hand side of these equations are of the same form except for the final term in (2.8). But, as n gets large, — 0*) converges in distribution while -,10UP/00, which is an average of independent components, converges to its expectation which is 0. Also, recall from Section 2.1.1 that the asymptotic covariance matrix of VT,(13 — /3*) does not depend on how, or even if, 0 is estimated so that the distributions of the random vectors fii(/ — /3*) are the same in both (2.8) and (2.9). This shows that the random vectors defined by the left-hand sides of (2.8) and (2.9) have the same asymptotic mean and variance. Therefore, in what follows we may suppose that the estimator of /3 is based on the estimating equations (2.4) alone, with 0 fixed at its limiting value 0*. Now we consider the maximum quasi-likelihood estimates under the reduced model. With UQ partitioned into components U01 and U432 of dimension p1 and /32 respectively, let /3red denote this estimator, namely the solution to U01[( /31,4)t ,01. 0. We wish to determine the Chapter 2. Methodology^ 26 asymptotic distribution under the null hypothesis of the score statistic for testing Ho : 132 = To accomplish this, first examine the expansion auoi0 = uoiO^O P *(tired) =^°pi (141 —(371 (2.10) and the corresponding expression for the score statistic (Sred) = u132 613*) + aau: (gi — 01' )or2 (2.11) where Or' (i=1,2) are between 13 1 and 01. Rearranging (2.10) we get au al— 01) =OW),Oi al which, substituted into (2.11), allows us to write ou# (Sred) =^[02 (M) 01 2 [au oii -1 (9131 /3r i0;n2 Ual (0*) . As n oo we have 02 (04') d, N(o, B22) \F -uoi (0*) L, N(0, Bii), and 1 au ,32 m201 p An, auoi 1+' A ll .n ath n Oth where the matrices All , A21, B11, and B22 are the appropriate submatrices of the matrices A and B. It follows that the asymptotic distribution of this score statistic is multivariate normal with mean 0 and covariance matrix 1 Nrn A Vary ^= B22 - A21 A a B12 -B21 Aril Al2+ ^A21 Aril B11 A111 Al2 •(Sr ed) (2.12) Chapter 2. Methodology^ 27 If the variance function correctly specifies the variances, then A = B and (2.12) reduces to the usual expression Vary[ 17.70 -^1 —u 2 Pred).1 = A22 — A21111711 Al2.V77, (2.13) Therefore we can obtain a model-based estimate of the asymptotic covariance matrix by substituting An for A in (2.13) or an empirical estimate by substituting An for A and fin for B in (2.12). The score test for testing Ho : 02 = QZ is given by comparing PI (Sred)2 -10.2 (Sred) to the x p2) distribution where 2 is our particular choice of the covariance matrix estimate.( The final type of test statistic we will consider is a deviance or likelihood ratio statistic. Recall the quasi-log-likelihood Q n (y; ,u, = 1 Q(yi; pi, 0). Define the deviance under the full and reduced models as -2Q n (y; 1.1(X,:j full), (7)) and -2Qn(Y; it(X, Sred), ) respectively, where the estimate c6 is obtained under the full model in both cases. The deviance statistic is then Adev. = 2 [Qn(Y; A(X ijred),^- n(Y; it(X f ull) , (k)] which should behave like a log-likelihood ratio and thus have an approximate 4 2) distribution (McCullagh and Nelder, 1989, p. 471). In the next section, we will examine the specifics of applying these tests for the significance of regression parameters to the case where we have over-dispersed Poisson data. 2.2 Test Statistics for Hypotheses in Over -Dispersed Poisson Regression In Section 1.2 the estimating equations and scaled deviance test statistic for over-dispersed Poisson regression were presented as an extension of Poisson regression. This extension was Chapter 2. Methodology^ 28 necessary because all inference which requires estimates of variability, such as tests of hypothe- ses, will be in error if the over-dispersion is not taken into account, even though reasonable amounts of over-dispersion have very little effect on estimation of the regression parameters by ordinary Poisson regression (Cox, 1983). We saw that an easy way to accomplish this was to introduce a dispersion parameter 0 to model this extra variability. In quasi-likelihood this is equivalent to specifying the variance of the observed data to be described by V(yi, 0) = 0iti for the i th observation. In this case, for the link function g(y) = logy equations (2.4) and (2.6) of Section 2.1.1 become UM, = E Pi ) Xij = 0; for j = 1, ...,p, (2.14) i=1 0(0,0) =^[ (Yi Pi)2 n P 1 _ 0, i.1 n (2.15) where pi = exp{X1 18}. Note that 0 may be factored out of (2.14) and that the estimate of 0 obtained from (2.15) is Pearson's X 2 statistic (using the Pearson residuals defined in Section 1.2) divided by its degrees of freedom. An alternate estimate of 0 which is thought to be very similar is to replace X 2 by the deviance given by the joint Poisson likelihood under the full model, which for this discussion we denote G 2 . For this formulation, it is immediate that the estimates of do not depend on the estimated value of 0; in other words, we may use the usual Poisson estimating equations for Q and estimate 0 afterwards with either of the two methods described above. Clearly this approach has considerable appeal over an approach involving specification of a variance function in which 0 cannot be factored out. Chapter 2. Methodology^ 29 With the choices V(iti; 0)_^and it = exp{Xi i3}, then from Section 2.1.1 the model- based estimate of the covariance matrix of /3 is given by 2m A,7 1 where n ^1 ^a/1i^n An = E aStj1 v 01„ n xiexts = (-4-1 E J=1 where the Xi are the row vectors corresponding to the ith row of X. Note that 0 may be factored out of An and the resulting covariance matrix is exactly times that obtained from Poisson regression. Alternately, the empirical estimate of the covariance matrix of j is given by EE = where V2 Can iA') as 0/3t i=i cog^i.1 Note that the factors 0 appearing in An and En cancel out in the expression for EE. Considering these two estimates of the covariance matrix and using either 0 = X 2 /d.Lfuti or = G 2full jd.f.full leads to three possible versions of the Wald test; two model-based and one empirical. We can similarly examine three versions of the score test. The two model-based versions use the expression given in equation (2.13) for the variance with An appropriately partitioned and the score vector as given by the p 2 equations of the form (2.14) which correspond to parameters with some hypothesized value under H0 . The resulting statistic could use either of the estimates of 0 described above. The empirical score test, on the other hand, uses the empirical covariance matrix of the form described by (2.12) with An and En appropriately partitioned. Examination of the expressions involved in this statistic reveals that the 0 terms disappear here also due to cancellation. ^( yi pi)2^_^(yi - ^x texi4xiem =^(yi — ) 2AIXi •Bn = ^ Chapter 2. Methodology^ 30 Finally, we may consider two versions of the deviance test. The expression for the quasi- likelihood is simply the Poisson log-likelihood divided by an estimate of 0. Therefore our deviance tests, often referred to as scaled deviance tests, are given by n1 Adev. = E (yi log(—) — — 140)) where Ili are the fitted values under the full model and^are the fitted values under the reduced model. Note that the best estimate of 0 (using either X 2 /d.f. or G2 /d.f.) will be the one obtained under the full model since this model has the best estimates of pi. Now that we have discussed some of the methods available for modelling and making infer- ence about parameters in over-dispersed Poisson data, we wish to address the questions raised in Section 1.4 and examine the performance of these procedures in a context similar to that encountered in the Prince George study. A simulation study designed to answer these questions is described in Chapter 3. Chapter 3 A Simulation Study Related to the Prince George Study Recall the methodological issues which arose in the Prince George study as described in Sec- tion 1.4: (a) Is the estimate of the scale parameter, q5^ deviance(full), used in this analysis, namely ^ ad.f(full) good estimate, or should we have used Pearson X 2 (full) ?d.f.(full)^• (b) Is the change in adjusted deviance criterion appropriate for model reduction in this situ- ation? (c) Are statistics other than the adjusted deviance statistic better for testing hypotheses concerning the regression parameters when over-dispersion is present? (d) Given the amounts of over-dispersion in the emergency room visits series, are the estimates of the model parameters and standard errors accurate? In this chapter we outline the purpose and procedures of a simulation study designed to address these questions. The primary goal of the simulation study was to examine the behavior of the model reduction test statistics described in Section 2.2 in contexts similar to the Prince George study (items (b) and (c)). Secondary goals were to examine the performance of estimates of over-dispersion parameters as well as of estimates of regression coefficients and their standard 31 Chapter 3. A Simulation Study Related to the Prince George Study^ 32 errors (item (d)). Of particular concern were the effects of varying amounts of over-dispersion and misspecification of the variance function on this performance. In particular, we will: 1. Compare the scaled deviance, score and Wald tests for model reduction. 2. Compare the parameter estimates to the simulated values. 3. Compare empirical and model-based estimated standard errors to the standard deviations of the estimated parameters. However, as an essential preliminary to the main thrust of our simulation study, we must first consider the question of how the over-dispersion parameter should be estimated (item (a)). As already mentioned, two possibilities are via G 2 /d.f. (as was done in the Prince George study) and X 2 /d.f. Results presented in Section 4.1 provide a clear indication that the latter is preferred, and this estimate is therefore employed in all subsequent work. In particular, the only version of the scaled deviance model reduction test considered is the one where 0 is estimated by X 2 /d.f. Both empirical and model-based versions of the Wald statistic will be evaluated, but only the empirical score test will be evaluated; the latter test is reported to perform well even in the presence of over-dispersion and with misspecified variances (see Breslow, 1990). To carry out these comparisons, a variety of data configurations, all intended to be qualitatively similar to the context of the Prince George study, will be simulated. Different data configurations will correspond to different log-linear models for the mean levels of the observed counts and different amounts of over-dispersion. In all cases, the model fit to the simulated data will be that of over-dispersed Poisson regression; that is, estimation of the regression coefficients will Chapter 3. A Simulation Study Related to the Prince George Study^ 33 be based on the estimating equations (2.14) which result from an assumed variance function of the form V(iii;^= cbtti. The simulation study will not be concerned with the possible misspecification of the log- linear model for pi (the regression function), but will examine the effects of misspecification of the variance function. For this purpose we define the "correct" variance to be V(pi; 0) = the model to be used in the fitting, and the "misspecified" variance to be V(iii; 0) = + (0— 1)4. The latter variance function arises from thinking of the datum yi as being sampled from a distribution which is, conditional on the value of an unobserved variable Ai, Poisson with mean Ai; if Ai is considered to be sampled from a distribution with mean pi and variance (0 — 1)4, then the marginal distribution of yi has mean pi and variance pi + (0 — 1)4. This simulation study differs in several respects from a study carried out in a different context by Breslow (1990). Firstly, Breslow's study only considered simulated data with mean values no smaller than 2. In the Prince George study the Asthma series had many zero values and a mean value of only about 0.75. In the current study we wish to investigate the possibility that such a series could lead to poor performance of the estimators or test statistics. Another difference is that Breslow was concerned with a 3x4 factorial design with n replicates per cell (and a single continuous covariate) whereas our study will be concerned with one or more long data series similar to the series available for the Prince George study. This difference is perhaps not as great as it first appears because all the data in Breslow's simulations were generated under the hypothesis that the column factor of the 3x4 design had no effect. The result is that, in effect, three data series of length 4n are being considered and for n = 144 (the largest value Breslow considered) the length of the series is comparable with the length Chapter 3. A Simulation Study Related to the Prince George Study^ 34 we will employ (700). The last key difference is that the current study is only concerned with test statistics and estimating equations based on an assumed variance function V(µ1; 0) = Opi. We will examine the performance of these test statistics and estimating equations using data generated under "correct" and "misspecified" variance functions, both for a single data series and for three series with differing amounts of over-dispersion (to represent the separate diagnostic categories), keeping in mind that the methods used will provide only a single overall estimate of 0. In contrast, Breslow generated each of his data sets under the variance function V(pi; 0) = pi + (0 — 1)4 and then examined the performance of the estimating equations and test statistics based on this variance function and also based on V(pi; 0) = 3.1 The General Simulation Procedure The simulation procedure, or sampling experiment, consists of three phases: generating the data, fitting a full model, and fitting one or more reduced models. All calculations are carried out in the programming language C. 3.1.1 Generating the Data The data generated are to represent the main qualitative features of the data analyzed in the Prince George study. To accomplish this we will generate time series of 700 independent over- dispersed Poisson observations. We may choose to generate a single series (Sections 4.2 and 4.4) or we may generate three separate series (Sections 4.3 and 4.5). Each data set to be generated requires a choice of a log-linear model for the mean pi of the form log pi = XiO, a choice of the variance function to be used in the simulation, and a choice Chapter 3. A Simulation Study Related to the Prince George Study^ 35 of the value for the over-dispersion parameter 0. Over-dispersion in the data is introduced through gamma-mixtures of Poisson random vari- ables. To generate the datum yi, it is most convenient to think of our situation as follows. Let log = Xifi and let there be a random variable vi which has a gamma distribution with mean 1 and variance^Then for each i=1,...,700 we first generate a realization of the gamma random variable IA and then generate a realization of a Poisson random variable with parameter vi pi. The resulting observations are negative binomial, with mean pi and variance pi + The choice V = (0 — 1)/pi leads to simulated data with the "correct" variance function Ofti, whereas the choice Vi = (0 — 1) leads to simulated data with the "misspecified" variance func- tion pi + (0 — 1)4. Note that both variance functions reduce to the usual Poisson variance when 0 = 1, and in this case the Poisson variates were generated directly. 3.1.2 Fitting the Full Model We fit the full model to obtain estimates of the parameters Shin using an iterative method for solving the estimating equations (2.14) of Chapter 2. We then estimate 0 using the Pearson X2 statistic divided by its degrees of freedom. In contrast to Breslow (1990), when we obtain an estimate of 0 which is less than one, we do not adjust that estimate to be equal to one. In our situation the dispersion parameter is only a nuisance parameter used to account for dispersion. It is not necessarily thought to arise from the variance of a mixing distribution (which would require values of 0 > 1); that is to say, under-dispersion is not ruled out. Using this estimate, both the empirical and the model-based estimates of the covariance matrix of Sfun are calculated (allowing the empirical and model-based Wald tests to be evaluated), as Chapter 3. A Simulation Study Related to the Prince George Study^ 36 well as the scaled deviance under the full model. 3.1.3 Fitting the Reduced Model Next, the reduced model is fit to obtain estimates of the parameters fired. The empirical estimate of the covariance matrix of fired is calculated and used in the calculation of the score statistic. We also calculate the scaled deviance using the estimate of 0 from the full model so that the scaled deviance statistic for testing the model reduction may be computed. The general procedure described above of generating data, fitting full and reduced models, calculating estimates and standard errors, and calculating the model Wald, empirical Wald, empirical score and scaled deviance test statistics is repeated 1000 times for each particular specification of the log-linear model, choice of V (pi; 0) and value of q4. In the following section the specific log-linear models, and the choices for the parameters in these models, as well as the levels of over-dispersion utilized in this simulation study are discussed. 3.2 The Specifics of the Simulation Study As mentioned in Section 3.1.1, we may choose to simulate either a single series of data, or three separate series to be modelled simultaneously. The single series simulations are small enough, in terms of the computing time required to generate the data and fit a model, to allow an investigation of the effects on the estimators and test statistics of varying both the level of dispersion and the simulated mean values. The three series simulations recreate an important feature of the data analysed in the Prince George study, namely separate series with possibly Chapter 3. A Simulation Study Related to the Prince George Study^ 37 different amounts of dispersion (we chose to use three series here instead of four as in the Prince George study simply to reduce the computing time). The overall mean values of the series and the levels of over-dispersion to be considered in these simulations (whether in a single series or in three) was based on the data series from the Prince George study. To recap the approximate mean values of those series and to get a rough idea of how much dispersion was present, we have the following table: mean variance variance/mean Asthma 0.76 0.80 1.05 Bronchitis+ 1.54 3.36 2.18 Ear 3.62 6.92 1.91 Other 5.13 14.2 2.77 From the above, the values 0.75, exp{0.7}(Pe, 2.01) and exp{1.6}(::::. 4.95) were chosen as possible overall mean values for the series to be simulated. In the single series simulations we will carry out sampling experiments using each of these values, and we will use the three values together as the overall mean levels in the three series simulations. Based on the third column of the above table, the values 1, 2 and 3 seem to be reasonable levels of over-dispersion to consider, although the estimates of 0 for each series would be smaller than the ratio of the variance to the mean (sf/) = E (yi )2 is less than variance/meanu-T-1 i = > ; (y= -y)2 /y in general). We also wish to simulate the value 1.4 since this is very close to the overall estimate of the dispersion parameter from the Prince George study, and a simulated value 0 = 5 is of possible interest as an "extreme" amount of over-dispersion. In the single series simulations, sampling experiments using each of these levels of dispersion are carried out at each possible mean level, with the exception that 0 = 5 is only simulated when the mean level is exp{1.6} (using 0 = 5 with lower mean values would result in series with Chapter 3. A Simulation Study Related to the Prince George Study^ 38 mostly zeros and only a few large values; this situation is too extreme and is not of particular interest in this thesis). In the three series simulations we chose two different combinations of dispersion parameters. The first is the triplet (1,2,3) which we refer to as combination I. These values seem like a plausible representation of the level of over-dispersion encountered in the Prince George study. We also consider the triplet (1.4,2,5), which we refer to as combination II, in order to examine the performance of the estimators and test statistics in a situation with a large amount of dispersion (relative to the amounts encountered in the Prince George study). Thus sampling experiments will be carried out using the following choices of overall mean values and dispersion parameters: Single Series Simulations Mean Value 1 1.4 2^3 5 0.75 exp{0.7} exp{1.6} ^ VA/VA/ ^ x x x= no sampling experiment carried out Three Series Simulations Combination of 0 Mean Values^I=(1,2,3) I1=(1.4,2,5) (0.75,exp{0.7},expl1.61) ,/ ^ The models used for generating the data should, in some sense, reproduce the main features of the data encountered in the Prince George study. We first describe some very simple models that are used to generate single series of data before we attempt to reproduce the more com- plicated features of the Prince George study data. The simplest models to be considered are for a single series with the mean value of the j th observation given by //2 = exP{O 7x2 },^j^1, ..., 700 ^ (3.16) Chapter 3. A Simulation Study Related to the Prince George Study^ 39 where^is the overall effect and -y is the coefficient corresponding to the covariate xi = sin(4irj/700), used to simulate a pollution variable with a regular annual cycle. To simulate data under the reduced model, one with no pollution, the choices for the simulated coefficients are = log0.75, 0.7, or 1.6 (with y = 0), corresponding to series with overall mean levels 0.75, exp{0.7}, and exp{1.6} respectively. To simulate data under the full model, one with pollution included, only /3 log 0.75 is used but three different values of -y are considered (7 = 0.05, 0.10 or 0.15). The simulations using models of the form (3.16) to generate data are referred to as the one series simple case. These simple models are also extended to a three series simple case with the mean of the i th observation in the ith series given by pii = exp{/3i -yoxi 7ixi}^i = 1, 2, 3; j = 1, ..., 700.^(3.17) where Oi is the overall effect for the ith diagnostic category, 7 0 is the common pollution effect and yi is the pollution effect specific to the ith diagnosis (with 73 0 for identifiablility). This parameterization with a common and separate pollution effects is utilized to facilitate the score test by allowing the hypothesis of common pollution effects to be stated as H: -yi = 0 for i = 1,2,3. For the alternate parameterization^= 70 + -yi for i = 1,2,3, the same hypothesis would take the more usual form of H: yi = Yz =^(which can be tested by the Wald or scaled deviance tests). In these simulations the simulated values of the^are = log 0.75, 02 = 0.7 and /33 = 1.6. The null model (no pollution effects) follows from setting 7o = 71 = 72 = 73 = 0. We may also simulate data sets with a common pollution effect (we use 70 = 0.1, 71 = 72 = 73 = 0), or with separate pollution effects (we use 70 = 0.1, 71 = 0.1, Chapter 3. A Simulation Study Related to the Prince George Study^ 40 72 = —0.2 and 73 = 0). The j th observation in the more complicated models with a single series has mean described by 4 itj = eXp{O^7-ksk + ^yxj} k=1 (3.18) where the sk are indicators designed to represent four "seasons" of equal length (±1 day), within each of our two "years" of 350 observations and xi is the sinusoidal pollution covariate just as in the simple case. The seasonal structure is the only attempt made in this thesis to recreate the temporal structure of the data in the Prince George study and no attempt is made to recreate the meteorological structure. For the purpose of simulating data under the reduced (or null) model, the values r 1 = 0.25, r2 = 0.25 and r3 = —0.5 (with r4 = 0 for identifiability) were used with one of /3 = log 0.75, 0.7, or 1.6. For the full model (one with pollution included) only /3 = log 0.75 is used with one of 7 = 0.05, 0.10 or 0.15. The simulations based on models described by (3.18) include seasonal effects and are referred to as the one series complicated case. As with the simple simulations, the one series complicated models are extended to a three series complicated case with the mean for the j th observation in the ith series given by 4 = exp{3i E Tiksk + -yoxi + 7ix.; } k=1 (3.19) where /3, is the overall effect for the i th diagnostic category, rik is the effect of "season" k in diagnosis i, -yo is the common pollution effect, and 7i is the pollution effect specific to the ith diagnostic category. The values of the coefficients used in the simulations are: Chapter 3. A Simulation Study Related to the Prince George Study^ 41 01 = log 0.75 = 133 = 1.6 = 0.2 T21 = 0.2 T31 = 0.2 T12 = 0.2 T22=0.20. 7-32 == 0.3 T13 = —0.2 T14 =00 T23 = —0.4 T33 = —0.4 r24 = 0^T34 = 0 There were no compelling reasons for the above choices for the season effects other than to make them separate effects; that is, different for each series. As in the three series simple case, the null model (no pollution) corresponds to 7o = 7 1 = 72 = -y3 = 0 while the common pollution effect model uses simulated values 7o = 0.1, 7 = 72 = -y3 = 0, and the separate pollution effects model uses -yo = 0.1, 7 1 = 0.1, 72 = —0.2 and 73 = 0. The results of fitting the models from each of these four cases (with the various possible choices of simulated coefficients, variance function, and dispersion parameter .0) are presented in Chapter 4. We begin Chapter 4 with a comparison of the possible estimators of q5 (as the remainder of the study depends on a choice of estimator of 0). For this comparison, only the one series simple case models were needed as these provide a strong indication that the estimator X2 /d.f. is preferable to G2/d.f. Chapter 4 Results of the Simulation Study In this chapter we describe the results of the investigation of the performance of the estimators and test statistics. We begin with the comparison of possible estimators for 0. 4.1 Estimating the Dispersion Parameter 0 Of particular interest in this comparison are any shortcommings in the performance of the estimator ii)G = Geld. f. used in the Prince George study and the possibility that use of the alternate estimator 'cbx = X 2/d.f. in the scaled deviance model reduction test statistic might have led to different conclusions than those sketched in Chapter 1. For this investigation single series of data were generated according to the simple model given by (3.16) using one of the three possible values of 0. Only the reduced model (set 7 = 0) was used to generate data, so that pi does not depend on i. As we shall see, the results in even this most simple of cases indicates the estimator cbx = X 2/d.f. is preferable. Table 1 presents the mean values and standard deviations of the estimates of 0 obtained from the two estimators based on the results of fitting the reduced model. Part A of the table contains the results when the correct variance function is used to simulate the data, while part B presents the results for the misspecified variance function, along with the values Os where 03,ui = + (0 — 1)4. The values 08 are the estimated values of 0 we should see if our incorrectly specified model is still 42 Chapter 4. Results of the Simulation Study^ 43 correctly accounting for the extra variability; that is, 0 8 is the amount of dispersion in the data in terms of the variance function V(//i; = 0/ii used in the estimating equations. Table 1. Mean Values of Estimates (± Standard Deviation) in 1000 Simulated Data Sets. A. Correct Variance Function. Simulated 0 Simulated Q Estimator 1 ^1.4^2^3^5 log 0.75^OX EGG 0.70 ^4x :(4G 1.60 (kG 1.002 (.055) 1.108 (.039) 0.998 (.056) 1.137 (.059) 0.998 (.052) 1.046 (.056) 1.401 (.096) 1.349 (.057) 1.400 (.084) 1.515 (.074) 1.399 (.076) 1.447 (.075) 2.001 (.174) 1.642 (.086) 2.002 (.134) 2.010 (.097) 2.006 (.118) 2.035 (.108) 2.996 (.337) 2.018 (.136) 2.983 (.235) 2.700 (.139) 3.001 (.208) 4.985 (.378) 2.938 (.162) 4.537 (.246) B. Misspecified Variance Function. Simulated 0 Simulated /3 Estimator 1^1.4^2^3^5 1^1.3^1.75 1.000 (.053) 1.300 (.086) 1.749 (.135) 1.107 (.038) 1.292 (.051) 1.526 (.072) 1^1.806^3.014 1.005 (.054) 1.799 (.114) 2.992 (.248) 1.143 (.057) 1.854 (.088) 2.705 (.147) 1^2.981^5.953 1.002 (.054) 2.979 (.201) 5.948 (.503) 1.050 (.058) 2.919 (.159) 5.226 (.310) 2.5 2.497 (.255) 1.844 (.112) 5.028 5.041 (.535) 3.823 (.244) 10.906 10.867 (1.14) 8.150 (.552) log 0.75 0.70 1.60 Os 93X OG cx (11G Os x 20.812 20.66 (2.95) 12.17 (1.08) In part A of the table the estimator cx provides good estimates of 0, on average, at all levels of dispersion. The estimator •G appears to slightly overestimate 0 at low levels of dispersion and underestimate 0 in the presence of large amounts of dispersion. The largest differences in the estimators occur when the level of dispersion is very high relative to the mean value (see, for example, the simulations with /3 = log 0.75 and 0 = 3). In part B the estimator -ciSx yields Chapter 4. Results of the Simulation Study^ 44 good estimates of Os, on average, whereas (0G produces very poor estimates when the level of over-dispersion is high. Note that in both parts of Table 1, except perhaps when 0=1, the estimator 4r0x generally appears substantially more variable than qG. This suggests that for some of the simulated data sets, (0x may differ noticably from the simulated value of 0 (and may be worse than (0G) although on average it is clearly superior to ;M. To answer the key question of how these differences affect the model reduction test statistics we use the above data sets (generated under the simple model log pi = /3) to evaluate the scaled deviance statistic using either ii5G, or cbx based on the full model given by (3.16). This model reduction, a test of the hypothesis H: y = 0, is intended to be an over-simplified representation of the model reductions tested in the Prince George study. The plots in Figures 1, 2, 3 and 4 present the values of three test statistics; the scaled deviance using C60G, the scaled deviance using 4x and the empirical score test (which does not depend on an estimate of 0, see Section 2.2), versus the quantiles of the chi-squared distribution with 1 degree of freedom. Results are presented only for a simulated mean of 0.75 and for 0 = 1,1.4,2 and 3; the results for the other combinations of and 0 are qualitatively similar. These plots are intended to judge how well the test statistics agree with their predicted asymptotic distribution. The score test is included to allow comparison with a test statistic that has been reported (in a somewhat different context) to perform reasonably well even in the presence of over-dispersion (Breslow 1990). This statistic is also useful for comparison in this situation because it does not involve an estimate of the dispersion parameter. We see that for data sets generated under either the correct variance function (Figures 1 and 2) or the misspecified variance function (Figures 3 and 4), the use of the estimator 4G 2 4 10 102 ^ 4^8 quanfilos of chi square(1) Figure 1. Comparison of Estimators of Phi: Correct Variance Simulations (Phi=1 or 1.4) G2 estimator of phi^ X2 estimator of phi^ Score test mean=0.75 and phf=1 mean=0.75 and phi=1 mean=0.75 and phi=1 quandlos of chi square(1) Score test mean=0.75 and phi=1.4 3 0 2 10 a4^6 coandlos of chl square(1) 4^6 quandoo of chi square(1) quantiles ol chi squaro(1) G2 estimator of phi mean=0.75 and phi=1.4 wand's, of chi square(1) X2 estimator of phi mean=0.75 and phi-1.4 20 104^6 g a a Figure 2. Comparison of Estimators of Phi: Correct Variance Simulations (Phi=2 or 3) G2 estimator of phi^ X2 estimator of phi^ Score test mean=0.75 and phiL-2 mean=0.75 and phi=2 mean=0.75 and phi=2 2 a4^6 wand's' of NI square(1) 0 10 0 2 10 2 10 0 2 4^6 quantles of chi scpare(1) Score test mean=0.75 and phi=3 4 quandlos doh! swam(1) G2 estimator of phi mean=0.75 and phi=3 4^6 quanolos of chi square(l) X2 estimator of phi mean=0.75 and phi=3 a 104^6 quintiles ol chl aquare(1) Figure 3. Comparison of Estimators of Phi: Misspecified Variance Simulations (Phi=1 or 1.4) G2 estimator of phi^ X2 estimator of phi^ Score test mean=0.75 and phi=1 mean=0.75 and phi=1 mean=0.75 and phi=1 0 2^4^6 quantiles of chl awaraot Score test mean-0.75 and phi=1.4 0 2 2 4^6 a 10 0 4 10 0 4^0 cafantlles of chl square(1) 4^6 quintiles of chl square(1) 2 10 2 10 finales of chl square(1) X2 estimator of phi mean=0.75 and phi=1.4 wimples of chi squaro(1) G2 estimator of phi mean=0.75 and phi=1.4 O guantles of chl square(1) X2 estimator of phi mean=0.75 and phi-3 a 10 2 10 2 102 4 guanalss of chl eguars(1) 4^6 giant's' of chl square(1) 64 guanakts of cfil ' ,wars( I) Figure 4. Comparison of Estimators of Phi: Misspecified Variance Simulations (Phi=2 or 3) G2 estimator of phi^ X2 estimator of phi^ Score test mean=0.75 and phi.2 mean=0.75 and phi=2 mean=0.75 and phi=2 1084^62 2 4 a 10 guandles of chl scpace(t) Score test mean-0.75 and phi=3 guantlIss of chi square(I) G2 estimator of phi mean=0.75 and phi=3 Chapter 4. Results of the Simulation Study^ 49 causes the scaled deviance test statistic to reject too often at higher levels of over-dispersion. The scaled deviance test using the estimator ii)x and the empirical score tests agree reasonably well with their predicted asymptotic distributions, except for obvious departures at the most extreme values. Similar plots for the other values of Q show the same departures of the scaled deviance statistic using (1)G from its predicted asymptotic distribution while the scaled deviance using crSx and the empirical score tests perform well. Recall that the estimator ikx is the most natural estimator of the two, arising as the moment estimate of 0, whereas cbG was used because it was more convenient and thought to generally agree closely with i4x. The limited results presented above indicate thatG may not agree well with ck x and that the latter is preferable, especially when substantial amounts of dispersion are present in the data. Therefore, in the simulations which follow, when an estimate of 0 is required we will always use 0x. With reference to the Prince George study, the question that arises is: Would this improved estimator have led to different conclusions? It appears that use of 0x would not have changed the nature of the results outlined in Chapter 1. For the pollutant TRS, in the situations where we rejected the null hypothesis in favor of a more complex alternative, i•x was evaluated and found to be smaller than QG (although the results of Table 1 would suggest that qX is generally larger than cbG), thus the null hypothesis would still be rejected. It appears quite likely the model reduction process would have led to the same final models and very similar final sets of estimates. We now begin to examine the performance of the estimators for parameters and their stan- dard errors, and the test statistics outlined in Section 2.2. Chapter 4. Results of the Simulation Study^ 50 4.2 The Simple Case with a Single Series For a single series we generate data under the reduced model (-y = 0) corresponding to equation (3.16) at the different possible mean levels and amounts of over-dispersion to be considered. Table 2 presents the mean values of the estimated regression coefficient under the reduced model. Table 2. Mean Values of Estimates of O. A. Correct Variance Function. Simulated 0 Simulated /3 1^1.4^2^3 5 log 0.75 1 -0.287^-0.288^-0.288^-0.289 0.7 0.699^0.700^0.699^0.696 1.6 1.601^1.600^1.600^1.599 1.598 B. Misspecified Variance Function. Simulated 0 Simulated /3 1^1.4^2^3 5 log 0.751 -0.288^-0.289^-0.290^-0.287 0.7 0.700^0.698^0.695^0.702 1.6 1.600^1.601^1.600^1.598 1.592 1 log 0.75;..-, -0.288 For the simulations under the correct variance function, the estimates of 0 are very close to the simulated values, even in the presence of substantial amounts of dispersion; the same is true in the misspecified variance simulations. This may not be surprising since in this situation the estimate of it = exp{/3} is just an average of the observed data. Table 3 summarizes the estimates of the variability of the estimates of 0 under the reduced model. In this table, "Simulated" standard error refers to the standard deviation of the 1000 parameter estimates. The details of parts A and B of Table 3 are very similar. In all cases the empirical and model-based estimates are identical to three decimal places - even for extreme Chapter 4. Results of the Simulation Study^ 51 amounts of over-dispersion. Nor is any difference in the variability of these estimates apparent. Further, these estimated standard errors accurately reflect the actual standard deviations, even when the variance function is misspecified. The agreement seen here between the simulated, empirical and model-based estimates of variability is somewhat surprising. It was expected that the model-based estimators of the covariance matrices would begin to yield poor estimates as the dispersion increased, but this is not the case. It appears that in this simple situation the over-dispersion in the data is being adequately accounted for, even when the variance function is misspecified; the quantity Os presented in part B of Table 1 was estimated very well and serves the purpose of accounting for the extra variability. Table 3. Standard Deviations of Estimates of 0 and Mean Values of Estimated Standard Errors (±s.d.). A. Correct Variance Function Simulated /3 Method Simulated q5 1 1.4 2 3 5 log 0.75 Simulated .043 .051 .062 .077 Model .044 (.002) .052 (.002) .062 (.003) .075 (.004) Empirical .044 (.002) .052 (.002) .062 (.003) .075 (.004) 0.70 Simulated .026 .031 .037 .046 Model .027 (.001) .032 (.001) .038 (.001) .046 (.002) Empirical .027 (.001) .032 (.001) .038 (.001) .046 (.002) 1.60 Simulated .017 .021 .024 .030 .039 Model .017 (.001) .020 (.001) .024 (.001) .029 (.001) .038(.001) Empirical .017 (.001) .020 (.001) .024 (.001) .029 (.001) .038(.001) Chapter 4.^Results of the Table 3 B. Misspecified Variance Simulated /3^Method Simulation Study Function Simulated q5 52 1 1.4 2 3 5 log 0.75 Simulated .042 .050 .059 .069 Model .044 (.002) .050 (.002) .058 (.002) .069 (.003) Empirical .044 (.002) .050 (.002) .058 (.002) .069 (.003) 0.70 Simulated .026 .036 .048 .057 Model .027 (.008) .036 (.001) .046 (.002) .060 (.003) Empirical .027 (.008) .036 (.001) .046 (.002) .060 (.003) 1.60 Simulated .017 .031 .041 .056 .081 Model .017 (.001) .029 (.001) .041 (.002) .056 (.003) .077 (.005) Empirical .017 (.001) .029 (.001) .041 (.002) .056 (.003) .077 (.005) We now turn to the performance of the test statistics for the hypothesis H: y = 0, summa- rized in Table 4. The observed rejection probabilities presented in this table can be considered as averages of 1000 Bernoulli trials with success probability p, which we may approximate by the nominal level. Thus the standard errors of the observed probabilities are approximately Vp(1 - p)/n; for nominal levels of 0.10, 0.05 and 0.01 we may expect standard errors of approx- imately .009, .007 and .003. For the results in part A of Table 4, keeping the precision of these observed rejection probabilities in mind, we find that they agree quite well with the nominal levels at all mean values and even at higher levels of dispersion. We also note the observed rejection probabilities are very similar across the four tests for each choice of mean and dis- persion suggesting that any deviations of the observed rejection probabilities from the nominal levels are a reflection of the particular 1000 data sets generated. The agreement between model and empirical Wald tests was to be expected given the agreement of the two estimators of the variances seen in Table 3. In general, the agreement among the four statistics may not be particularily surprising either. Conventional wisdom would suggest that, over-dispersion aside, Chapter 4. Results of the Simulation Study^ 53 each of the tests should perform well with a sample size of 700 (although there was some ques- tion if this was true when the series consist of a large number of zeros such as in the simulations using /3 = log 0.75). Thus, in light of how effectively the over-dispersion is accounted for, we should not expect any of the test statistics to do poorly. The results in the misspecified variance simulations are similar. In these simulations the agreement between the four tests may seem particularly surprising at first; we might expect the model-based Wald and the scaled deviance tests to do poorly since they rely on the particular choice of variance function used to fit the data. However, considering how the choice of variance function does not seem to matter in this simple case (as noted for Table 3), the above agreement should not be surprising. Chapter 4. Results of the Simulation Study Table 4. Observed Rejection Probabilities Under Reduced Model. 54 A. Correct Variance Function 0 = 1 0 = 1.4 0 = 2 Simulated 0 Test/Method log 0.75 ^ Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 0.70^Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 1.60^Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 0.10 0.05 0.01 .105 .041 .010 .105 .040 .010 .102 .040 .010 .106 .041 .010 .103 .056 .005 .104 .055 .006 .102 .054 .005 .103 .056 .005 .112 .056 .011 .110 .057 .012 .108 .057 .012 .112 .056 .011 0.10 0.05 0.01 .096 .045 .007 .097 .046 .005 .095 .045 .005 .096 .045 .008 .082 .046 .012 .083 .046 .011 .082 .046 .011 .082 .046 .013 .095 .043 .004 .097 .043 .004 .090 .038 .007 .087 .039 .007 0.10 0.05 0.01 .106 .053 .015 .114 .056 .016 .111 .056 .016 .108 .054 .015 .110 .062 .017 .111 .061 .017 .110 .060 .014 .110 .062 .017 .087 .039 .007 .091 .038 .008 .090 .038 .007 .087 .039 .007 0 = 3^0 = 5 Simulated )3 Test/Method log 0.75 ^ Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 0.70^Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 1.60^Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 0.10 0.05 0.01 .119 .056 .009 .122 .059 .011 .117 .054 .009 .121 .056 .011 .114 .061 .013 .122 .065 .012 .119 .063 .011 .114 .061 .013 .103 .043 .010 .096 .048 .013 .095 .045 .010 .103 .043 .010 0.10 0.05 0.01 .092 .043 .009 .090 .043 .009 .088 .043 .008 .092 .043 .009 Chapter 4. Results of the Simulation Study 55 Table 4 B. Misspecified Variance Function 0 = 1 Simulated 0 Test/Method^0.10 0.05 0.01 log 0.75^Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled .094 .045 .009 .095 .047 .009 .095 .046 .009 .094 .045 .009 cb = 1.4 0.10 0.05 0.01 0.10 0.05 0.01 q=22 .094 .044 .006 .095 .046 .007 .094 .046 .006 .095 .044 .006 .091 .092 .091 .093 .045 .050 .047 .046 .009 .012 .010 .009 0.70 Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 1.60 Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled .091 .050 .016 .091 .051 .016 .092 .051 .015 .091 .050 .017 .097 .041 .007 .097 .039 .009 .096 .039 .007 .097 .041 .007 .095 .048 .012 .091 .047 .012 .091 .046 .011 .095 .048 .012 .103 .052 .009 .105 .051 .010 .103 .050 .009 .103 .052 .010 .107 .046 .010 .108 .046 .009 .108 .043 .007 .109 .047 .010 .083 .033 .004 .081 .034 .005 .079 .033 .004 .083 .034 .004 0 = 3 0 = 5 Simulated 0 Test/Method log 0.75 Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 0.70 Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 1.60 Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 0.10 0.05 0.01 0.10 0.05 0.01 .112 .063 .014 .114 .066 .017 .110 .062 .015 .112 .064 .015 ^ .101 .044 .010^- .102 .048 .011 .099 .042 .009 ^- ^.101 .045 .010^- .111 .059 .014 .102 .056 .011 .114 .066 .016 .104 .053 .012 .108 .059 .013 .100 .047 .006 .111 .059 .014 .102 .056 .012 In summary, for the simulation results when the data is generated under the reduced model there is no clear difference between the empirical and model-based estimates of variances. Nor are there any differences among the four model reduction test statistics considered. We will look to the three series case in Section 4.3 to provide more insight into possible differences in the estimators and test statistics in this simple situation. Chapter 4. Results of the Simulation Study^ 56 A limited number of simulations were also carried out using the full model corresponding to (3.16) to generate the data. In particular, 1000 data sets were generated according to this model with = log 0.75 and one of y = 0.05, 0.10, or 0.15. The estimation of 0, fl and y were of interest as well as power for testing the hypothesis 11:7 = 0. The mean values of the parameter estimates are summarized in Table 5. First consider part A of the table, corresponding to simulations where the data were generated with the correct variance function. In all cases (that is, all combinations of simulated 0 and 7) the estimators yield good estimates. The mean values of both model and empirical versions of the estimated standard errors (not presented here) are in close agreement with the displayed standard deviations. The standard errors of the estimates increase as the dispersion increases as would be expected. Notice that the estimates of 0 (and their standard deviations) are almost identical to the corresponding quantities for qx in part A of Table 1 (which were based on data generated using the reduced model). In part B, we see that # and -y are estimated very well at all levels of dispersion considered. To determine what the estimates of 0 should be, first note that in these misspecified variance simulations, the amount of extra variability varies as the covariate xj, and thus the mean values pi, vary. In the present situation, the average of the 0 .3 (where pick; = pi + — 1)/2) for each cell in part B of Table 5 are the same to two decimals as the values 4r from part B of Table 1. It is therefore reasonable that the estimates of 0 are close to the estimates in part B of Table 1, and again cix is accounting for the extra variability very well. Chapter 4. Results of the Simulation Study^ 57 Table 5. Mean Values of Estimated Parameters (± s.d.) Under Full Model. A. Correct Variance Function Simulated 0 Simulated 7 Parameter 1 1.4 2 3 0.05 0 1.00 (.055) 1.40 (.095) 2.00 (.178) 2.97 (.335) /3 -.288 (.044) -.289 (.050) -.292 (.064) -.292 (.076) 7 .049 (.061) .051 (.075) .050 (.087) .053 (.109) 0.10 0 1.00 (.054) 1.40 (.093) 1.99 (.179) 2.97 (.323) /3 -.288 (.042) -.290 (.051) -.291 (.063) -.293 (.076) 7 .100 (.063) .098 (.073) .099 (.086) .102 (.110) 0.15 0 1.00 (.054) 1.40 (.096) 2.00 (.171) 2.98 (.323) 0 -.288 (.043) -.289 (.051) -.288 (.063) -.293 (.075) 7 .151 (.062) .149 (.072) .150 (.085) .153 (.109) B. Misspecified Variance Function Simulated 0 Simulated 7 Parameter 1 1.4 2 3 0.05 0 1.00 (.055) 1.30 (.083) 1.75 (.133) 2.49 (.260) /3 -.288 (.044) -.290 (.050) -.292 (.059) -.292 (.069) 7 .049 (.061) .051 (.069) .055 (.084) .053 (.098) 0.10 0 1.00 (.054) 1.30 (.082) 1.75 (.134) 2.49 (.258) /3 -.288 (.042) -.290 (.050) -.290 (.056) -.294 (.069) 7 .100 (.063) .102 (.068) .099 (.083) .104 (.102) 0.15 0 1.00 (.054) 1.30 (.084) 1.75 (.137) 2.50 (.255) /3 -.288 (.043) -.291 (.049) -.291 (.058) -.291 (.070) 7 .151 (.062) .151 (.071) .150 (.081) .151 (.100) The observed rejection probabilities under the alternative hypothesis are shown in Table 6 below. The results presented in part A show no difference in the power of the four statistics. For a given value of y, the observed rejection probabilities decrease as the amount of dispersion increases (the increased variability in the data makes it more difficult to detect an effect). The results in part B are very similar. In general, for given values of y and 0 > 1, the rejection probabilities are slightly higher in the misspecified variance simulations compared to the correct variance simulations. Note that for values of pj < 1, such as in these simulations using = log 0.75 and y no larger than 0.15, the value of pi + (0 - 1)p .1 (the variance under Chapter 4. Results of the Simulation Study^ 58 the misspecified formulation) is less than the value of cbiti (the variance under the correct formulation). Therefore from our earlier observation that the power decreases as the amount of variablility in the data increases, we should expect the rejection probabilities in the misspecified variance to be higher than the correct variance simulations for these values of y and 0. In summary, from the results of the simulations under the full model (Tables 5 and 6) the estimates of 0 appear reasonable and the regression parameter estimates are close, on average, to the simulated values. There are also no indications of any differences in the power of the four test statistics considered. Further, estimated standard errors from the model and empirical estimators (not presented) were similar, both being very close, on average, to the simulated values. Chapter 4. Results of the Simulation Study^ 59 Table 6. Observed Rejection Probabilities Under Full Model. A. Correct Variance Function 7 Test 0 = 1 0 = 1.4 0 = 2 0 = 3 .10 .05 .01 .10 .05 .01 .10 .05 .01 .10 .05 .01 .05 Wald/Mod. .205 .117 .033 .194 .114 .041 .152 .090 .025 .152 .089 .025 Wald/Emp. .204 .117 .035 .197 .119 .040 .155 .090 .027 .151 .086 .025 Score/Emp. .202 .116 .034 .197 .118 .038 .154 .086 .024 .147 .079 .024 Dev/Scaled .205 .117 .033 .194 .117 .041 .153 .092 .026 .156 .092 .025 .10 Wald/Mod. .490 .374 .164 .348 .247 .116 .295 .205 .070 .253 .165 .059 Wald/Emp. .489 .372 .171 .351 .253 .119 .298 .207 .076 .266 .159 .065 S core/ Emp. .487 .371 .164 .353 .249 .111 .296 .203 .072 .260 .156 .060 Dev/Scaled .490 .374 .164 .349 .247 .117 .296 .207 .074 .253 .166 .060 .15 Wald/Mod. .786 .685 .445 .661 .522 .296 .529 .404 .186 .391 .274 .135 Wald/Emp. .788 .687 .445 .663 .521 .297 .527 .411 .187 .402 .274 .136 Score/Emp. .788 .683 .439 .662 .518 .287 .527 .408 .182 .395 .269 .128 Dev/Scaled .786 .685 .448 .663 .523 .300 .531 .405 .190 .391 .277 .136 B. Misspecified Variance Function 0 = 1 0 = 1.4 0 = 2 0 = 3 y Test .10 .05 .01 .10 .05 .01 .10 .05 .01 .10 .05 .01 .05 Wald/Mod. .205 .117 .033 .193 .111 .024 .176 .108 .030 .165 .090 .021 Wald/Emp. .204 .117 .035 .195 .111 .028 .174 .112 .032 .161 .087 .020 Score/Emp. .202 .116 .034 .189 .107 .026 .169 .109 .030 .157 .082 .018 Dev/Scaled .205 .117 .033 .194 .111 .025 .177 .110 .032 .165 .093 .021 .10 Wald/Mod. .490 .374 .164 .407 .291 .123 .336 .236 .087 .291 .195 .073 Wald/Emp. .489 .372 .171 .410 .299 .131 .333 .241 .097 .294 .203 .074 S core/ Emp. .487 .371 .164 .409 .300 .122 .328 .233 .088 .291 .192 .066 Dev/Scaled .490 .374 .164 .410 .292 .123 .336 .237 .088 .291 .195 .073 .15 Wald/Mod. .786 .685 .445 .680 .579 .337 .585 .455 .218 .470 .340 .158 Wald/Emp. .788 .687 .445 .683 .574 .342 .587 .455 .222 .481 .342 .162 Score/Emp. .788 .683 .439 .681 .565 .333 .584 .450 .209 .473 .333 .148 Dev/Scaled .786 .685 .448 .680 .580 .339 .586 .456 .219 .472 .342 .160 Chapter 4. Results of the Simulation Study^ 60 4.3 The Simple Case with Three Series The results discussed in this section are for three separate series of data generated according to log-linear models given by (3.17) with one of the two combinations of dispersion parameters described in Section 3.2 (combination I or II) and one of the two possible variance functions. For the null model (70 = -yi = 72 = -y3 = 0) the parameter values /3i = log 0.75 (= —0.288), /32 = 0.7 and /33 = 1.6 are used to generate the data. The resulting data series are then fit using this same log-linear model. The mean values of the estimated parameters from 1000 simulated data sets generated under each of the four possible scenarios (2 combinations of 0 x 2 variance formulations) are presented in Table 7. Table 7. Mean Values of Parameter Estimates. Parameter Variance A. Correct Function B. Misspecified Combination of (/) I^II Combination of 0 I^II 0 1.997 2.799 4.964 8.333 01 -0.288 -0.289 -0.289 -0.287 02 0.699 0.699 0.701 0.700 /33 1.599 1.599 1.598 1.594 Intuitively the estimates of 0 should be approximately equal to the average of the three dispersion values associated with the three generated data series in each of the 1000 data sets leading to the results in each of the four scenarios. These averages are 2.0 (combination I, correct variance), 2.8 (combination II, correct variance), 4.97 (combination I, misspecified variance) and 8.37 (combination II, misspecified variance). We can see that in each of the four scenarios, the estimates of 0 are very close to these corresponding averages. The estimates of the regression parameters are also excellent in all four scenarios. Chapter 4. Results of the Simulation Study^ 61 Table 8 presents the standard deviations of the parameter estimates and the mean values of the estimated standard errors for the regression parameters. In each column of this table the empirical estimates agree very well with the simulated standard deviations but the model-based estimates are most often quite different. The latter are much too large for the parameter /9 i corresponding to the series where the simulated dispersion is lower than the average of the three series, and too small for 03 corresponding to the series where the simulated dispersion is larger than the average of the three series. For the regression parameter N2, first consider the correct variance simulations. We find that with combination I of 0's the standard errors for 02 are about right; this appears to be because the simulated 0 for this series is equal to the average of the 4's for all three series. However, using combination II of cb's the simulated 0 for series 2 is smaller than the average of the O's for all three series and the model-based standard error is distinctly larger than the standard deviation of the parameter estimates. In the misspecified variance simulations, we also find the model-based standard error to be larger than the standard deviation of the parameter estimates because the simulated 0 for series 2 is smaller than the average of the cb's in these cases. Clearly the empirical estimator is better in each of these cases (for both combinations of 0 and for both variance formulations) and accurately reflects the true variability of these parameter estimates, even with the misspecified variance function. Chapter 4. Results of the Simulation Study^ 62 Table 8. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.). Parameter Method Variance A. Correct Function B. Misspecified Combination of .0 I^II Combination of 0 I^II 0 Simulated .084 .144 .399 .976 01 Simulated .042 .052 .042 .050 Model .062 (.002) .073 (.003) .097 (.004) .126 (.008) Empirical .044 (.002) .052 (.002) .044 (.002) .050 (.002) /32 Simulated .038 .038 .046 .047 Model .038 (.001) .045 (.001) .059 (.003) .077 (.005) Empirical .038 (.001) .038 (.001) .046 (.002) .046 (.002) 03 Simulated .029 .037 .054 .079 Model .024 (.001) .028 (.001) .038 (.001) .049 (.002) Empirical .029 (.001) .038 (.001) .056 (.003) .077 (.004) Using the data generated under the reduced model, the levels achieved by the four test statistics under study are examined by considering an alternative model that allows for a com- mon "pollution" effect (71 = 72 = -y3 = 0, but 7o possibly not zero). The hypothesis to be tested is H: 70=0. Table 9 presents the observed rejection probabilities of each of the four test statistics. The precision of the observed rejection probabilities, which should be kept in mind when interpreting these results, are the same as those reported in Section 4.2. Consider first the correct variance simulations. In the model reductions for the correct variance simulations, the model-based tests (model Wald and scaled deviance) clearly reject too often while the rejection probabilities for the empirical tests (empirical score and empirical Wald) agree quite well with the nominal levels based on the predicted 4 ) critical values. A similar pattern is apparent in the misspecified variance simulations, although the departure of the model-based tests from Chapter 4. Results of the Simulation Study^ 63 the nominal levels is considerably more exaggerated than in the correct variance simulations. Table 9. Observed Rejection Probabilities: Common Pollution to No Pollution Model. A. Correct Combination of 0 Variance Function B. Misspecified Combination of 0 Test I 0.10 0.05 0.01 II 0.10 0.05 0.01 I 0.10 0.05 0.01 II 0.10 0.05 0.01 Wald/Mod .159 .099 .019 .173 .102 .039 Wald/Emp .114 .057 .004 .110 .058 .014 S core/Emp .114 .056 .004 .109 .058 .014 Dev/Scaled .159 .099 .018 .172 .102 .039 .180 .104 .045 .226 .089 .054 .016 .108 .088 .053 .012 .102 .180 .104 .044 .227 .139 .060 .062 .012 .057 .011 .139 .060 Thus in this multiple series situation (without any pollution effects at this time) we find a clear difference between the empirical and model-based estimates of variance, with the empirical estimator being preferable. Correspondingly we find that the model-based test statistics are inferior to the empirical test statistics. Given the differences in the variance estimates, the fact that the empirical Wald is preferable to the model Wald test is not surprising. Nor is it surprising that the empirical score test performs well in all cases since it performed well even under misspecified variances in the study described in Breslow (1990). What is interesting is the very close agreement agreement between the model Wald and scaled deviance tests and between the empirical score and empirical Wald tests. Conventional wisdom would suggest the Wald tests might be inferior to the other two tests, but we see that the model Wald gives observed rejection probabilities that are almost exactly the same as the scaled deviance test and the empirical Wald gives observed rejection probabilities that are almost exactly the same as the empirical score test. It seems that the test statistics which rely on a dispersion parameter to account for extra variability perform poorly, while those that estimate the extra variability Chapter 4. Results of the Simulation Study^ 64 empirically perform much better. A set of simulations with data generated according to the model with a common pollution effect (71 = 72 = 73 = 0, but 70 non-zero in (3.17)) were also carried out. Recall that the value chosen for 7o is 0.10. Table 10 presents the mean values of the parameter estimates for the data sets generated in this fashion. The estimates of the Oi are very good in each column of the table as are the estimates of -y o . The estimates of 0 are close to the estimates seen in Table 7 as we would expect given that in each case, the average of the dispersions is close to the averages encountered in the simulations under the null hypothesis. Table 10. Mean values of Parameter Estimates. Parameter Variance A. Correct Function B. Misspecified Combination of 0 I^II Combination of 0 I^II 1.998 2.799 4.970 8.325 Ql -0.289 -0.292 -0.288 -0.292 P2 0.699 0.701 0.699 0.699 03 1.599 1.599 1.597 1.591 yo 0.100 0.101 0.098 0.098 The standard deviations of the above estimates and the mean values of the estimated stan- dard errors for the regression parameters are given in Table 11. The model-based estimates of the standard errors for the /3 show the same behavior as in Table 8 with the estimate being too large when the parameter corresponds to a series with simulated q5 lower than c b and too small when the parameter corresponds to a series with simulated 0 larger than (7,. The empirical estimator on the other hand, agrees very well with the simulated standard deviations. Notice that the model-based estimates of the standard error of 7o are too low throughout the table. Recall that the rejection probabilities were too high for the model-based test statistics in Table Chapter 4. Results of the Simulation Study^ 65 9. This suggests that the standard errors of the estimates of y o were lower than expected in the simulations under the null hypothesis as well. Table 11. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.). Parameter Method Variance A. Correct Function B. Misspecified Combination of 0 I^II Combination of q I^II 0 Simulated .086 .144 .395 .967 131 Simulated .043 .053 .040 .052 Model .062 (.002) .073 (.003) .097 (.004) .126 (.008) Empirical .044 (.002) .052 (.002) .044 (.002) .050 (.002) 132 Simulated .038 .038 .046 .049 Model .038 (.001) .045 (.001) .059 (.003) .077 (.005) Empirical .038 (.001) .038 (.001) .046 (.002) .046 (.002) /33 Simulated .029 .037 .058 .079 Model .024 (.001) .028 (.001) .038 (.001) .049 (.002) Empirical .029 (.001) .038 (.001) .056 (.003) .077 (.004) 7o Simulated .030 .038 .055 .076 Model .027 (.001) .032 (.001) .043 (.002) .056 (.003) Empirical .031 (.001) .038 (.002) .054 (.002) .072 (.006) We now examine the results of evaluating the four test statistics based on an alternative hypothesis that includes common and separate "pollution" effects for each series. Thus these model reductions, from the model with separate effects to the one with a common pollution effect (which was used to generate the data), represent the separate to common effects reductions described in the Prince George study. Note that these tests, the results of which are summarized in Table 12, are the first discussed in this thesis involving reduction by more than one parameter (and are thus the first that involve estimates of covariance in the Wald and score test statistics). Chapter 4. Results of the Simulation Study^ 66 In the correct variance simulations the empirical test statistics achieve levels close to the nominal rejection probabilities with combination I of 0 and still quite close with combination II (keeping in mind the precision of these observed rejection probabilities discussed above). The levels of the model-based test statistics are clearly too low under both combinations of 0. Even with the misspecified variance formulation, the empirical test statistics achieve levels close to nominal; while the rejection probabilities may be slightly too large at the 0.10 level under combination II, overall the agreement is good. The levels of the model-based test statistics are far too low at the 0.10 and 0.05 nominal levels under both combinations of 0, although it is not clear how they do at the 0.01 level. Table 12. Observed Rejection Probabilities: Separate to Common. Test A. Correct Variance Function B. Misspecified Combination of 0 Combination of cb 0.10 I 0.05 0.01 0.10 II 0.05 0.01 0.10 I 0.05 0.01 0.10 II 0.05 0.01 Wald/Mod .076 .035 .006 .062 .026 .001 .067 .036 .005 .056 .029 .009 Wald/Emp .100 .051 .011 .092 .049 .010 .104 .057 .006 .117 .052 .014 Score/Emp .099 .049 .008 .088 .049 .008 .100 .055 .006 .113 .051 .010 Dev/Scaled .077 .035 .006 .062 .027 .001 .067 .036 .006 .056 .030 .009 We also examine the power of the four tests in this multiple series situation in Table 13 by considering the reduction from the common pollution effect to no pollution effects (the null model). The power for the two model-based tests are virtually identical throughout the table and the two empirical tests achieve similar power. However, the rejection probabilities for the model-based tests are higher than the empirical tests presumably as a result of the standard error of 7o being underestimated in the former tests (see Table 11). Chapter 4. Results of the Simulation Study Table 13. Observed Rejection Probabilities: Common to Null. 67 A. Correct Combination of 0 Variance Function B. Misspecified Combination of 0 Test I 0.10 0.05 0.01 II 0.10 0.05 0.01 I 0.10 0.05 0.01 II 0.10 0.05 0.01 Wald/Mod .958 .925 .834 .891 .827 .687 .695 .598 .404 .542 .447 .271 Wald/Emp .941 .890 .751 .837 .767 .537 .570 .445 .226 .399 .293 .111 Score/Emp .941 .891 .751 .837 .765 .535 .567 .436 .213 .394 .283 .100 Dev/Scaled .958 .925 .835 .891 .828 .688 .695 .599 .406 .542 .447 .274 The final model to be considered for generating data in this three series simple case is the model with separate pollution effects. Recall the value chosen for 7 0 is again 0.10, while 71 =0.10, 72 = -0.20 and -y3 = 0. Results of fitting the full model to these data sets are summarized in Tables 14, 15 and 16 which follow. The mean values of the parameter estimates (see Table 14 below) are very close to the correct values, even in the misspecified case with substantial amounts of dispersion. The estimates of 0 are very close to the values seen in Tables 7 and 10, which is still to be expected because, even in the misspecified variance simulations, the average of the dispersion values over the entire 2100 observations is approximately equal to the averages in the corresponding simulations under either the null or the common pollution effect model. Chapter 4. Results of the Simulation Study^ 68 Table 14. Mean Values of Parameter Estimates. Variance A. Correct Function B. Misspecified Combination of 0 Combination of 0 Parameter I II I II Cb 1.997 2.799 4.962 8.326 01 -0.290 -0.289 -0.290 -0.289 02 0.699 0.700 0.700 0.698 03 1.598 1.597 1.595 1.593 70 0.098 0.102 0.102 0.096 71 0.101 0.096 0.100 0.102 72 -0.195 -0.200 -0.197 -0.195 The estimates of the standard errors are summarized in Table 15. In each of the columns of this table the empirical estimator is very close, on average, to the simulated standard devia- tions. Again the model-based estimator overestimates the standard errors for parameters which correspond to the series with dispersion lower than 0 (most notably th and 7 1 ) and underes- timates the standard errors corresponding to the series with dispersion larger than 0 (most notably 03 ). It would seem that the low rejection probabilities in Table 12 for the model-based test statistic can be explained by the general overestimation of the standard error of 71. Chapter 4. Results of the Simulation Study^ 69 Table 15. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.). Parameter Method Variance A. Correct Function B. Misspecified Combination of 0 I^II Combination of 0 I^II Cb Simulated .085 .142 .393 .966 01 Simulated .044 .054 .042 .050 Model .062 (.002) .073 (.003) .098 (.004) .127 (.008) Empirical .044 (.002) .052 (.002) .044 (.002) .050 (.002) 02 Simulated .039 .037 .047 .047 Model .038 (.001) .045 (.001) .059 (.003) .077 (.005) Empirical .038 (.002) .038 (.001) .046 (.002) .046 (.002) 03 Simulated .029 .039 .056 .076 Model .024 (.001) .028 (.001) .038 (.001) .049 (.002) Empirical .029 (.001) .038 (.001) .056 (.003) .077 (.005) 70 Simulated .042 .054 .078 .113 Model .034 (.001) .040 (.001) .054 (.002) .069 (.007) Empirical .041 (.002) .054 (.003) .079 (.005) .109 (.011) 71 Simulated .074 .090 .098 .133 Model .094 (.003) .111 (.004) .148 (.006) .191 (.011) Empirical .075 (.003) .090 (.003) .100 (.004) .130 (.009) 72 Simulated .071 .076 .102 .131 Model .063 (.002) .075 (.002) .100 (.004) .129 (.007) Empirical .067 (.002) .076 (.003) .102 (.005) .127 (.012) Chapter 4. Results of the Simulation Study^ 70 Table 16. Observed Rejection Probabilities: Separate to Common. Test Variance A. Correct Variance Function Function B. Misspec. Variance Function Combination of 0 Combination of 0 0.10 I 0.05 0.01 0.10 II 0.05 0.01 0.10 I 0.05 0.01 0.10 II 0.05 0.01 Wald/Mod .936 .890 .741 .870 .786 .558 .621 .465 .222 .340 .218 .088 Wald/Emp .962 .932 .822 .944 .889 .747 .911 .858 .685 .880 .791 .593 Score/Emp .962 .929 .818 .945 .888 .745 .909 .854 .671 .879 .786 .578 Dev/Scaled .936 .891 .743 .870 .789 .560 .623 .468 .227 .341 .218 .088 In Table 16 we examine the power of the four test statistics under consideration in testing the reduction from the separate pollution effects simulated in these data sets to a model with a common pollution effect. The model-based test statistics have lower power than the empirical tests in all of the above cases. The difference is most noticable in the cases where the standard errors of -yi were seriously overestimated, such as the misspecified variance simulations. In all of the simulations in this three series simple case, in contrast to the one series sim- ulations, it is clear that the empirical estimate of the covariance matrix for the regression parameters is superior to the model-based estimate. As for the observed levels of the four test statistics under consideration, we find that the model-based test statistics reject too often in some instances, and not as often as predicted by theory in other cases, whereas the empirical test statistics always achieve levels close to the nominal rates. Throughout we have found very close agreement between the model Wald and scaled deviance tests and between the empirical score and empirical Wald tests. It appears that, in these simple simulations, the two model- based test statistics that rely on a dispersion parameter to account for the extra variability perform poorly, while the empirical test statistics (which do not rely on the correctness of a specified model to estimate the variance-covariance matrix) perform much better. Chapter 4. Results of the Simulation Study^ 71 4.4 The More Complicated Case with a Single Series We now wish to study the performance of the estimators and test statistics when we generate data using the more complicated model for a single series, described by (3.18), that includes season effects. For the purpose of simulating data under the reduced model, the values r 1 = r2 = 0.25, r3 = -0.5 and r4 = 0 (with y = 0) were used. Table 17 presents the mean values of the parameter estimates from fitting the reduced model to each of the 1000 simulated data sets. Table 17. Mean Values of Estimated Parameters Under Reduced Model. A. Correct Variance Simulated 0 Simulated 0 Parameter 1 1.4 2 3 5 log 0.75 (i) 1.00 1.39 1.98 2.96 - Q -.291 -.286 -.295 -.292 - Ti. .253 .245 .253 .247 - r2 .252 .238 .250 .253 - r3 -.498 -.513 -.513 -.520 - 0.70 0 .997 1.40 1.99 2.98 - ,Q .699 .697 .698 .695 - r1 .246 .250 .247 .252 - r2 .250 .253 .250 .250 73 -.499 -.501 -.501 -.508 - 1.60 4 .997 1.40 2.01 2.98 4.97 13 1.60 1.60 1.60 1.60 1.60 ri. .251 .249 .247 .251 .246 T2 .250 .250 .248 .249 .251 73 -.503 -.501 -.502 -.504 -.499 Chapter 4. Results of the Simulation Study^ 72 Table 17 B. Misspecified Variance Simulated 4) Simulated /3 Parameter 1 1.4 2 3 5 log 0.75 0 1.00 1.31 1.77 2.54 - /3 -.287 -.292 -.297 -.296 - Tl .245 .246 .256 .248 - T2 .246 .251 .258 .256 - T3 -.507 -.503 -.503 -.507 - 0.70 0 1.00 1.83 3.07 5.15 - /3 .702 .695 .699 .698 - Ti. .247 .253 .244 .246 - T2 .245 .254 .244 .248 T3 -.505 -.501 -.506 -.502 - 1.60 (/) 1.00 3.06 6.15 11.22 21.09 /3 1.60 1.60 1.60 1.59 1.58 Ti .249 .252 .249 .252 .263 T2 .248 .249 .248 .255 .258 7-3 -.500 -.500 -.505 -.497 -.495 In part A of the table, the correct variance simulations, the estimates are very close to the simulated values in each column of the table. The regression parameter estimates ( 13 and Ti) are also quite good (correct to 1 decimal) even when dispersion is high relative to the mean. There are cases where the estimates appear to be a bit off (for example, the simulations with /3 = log 0.75 and c > 1.4), but the regression parameter estimators have substantially higher vari- ances than in the simple simulations of the previous sections (compare the standard deviations of these estimates provided in Table 18 below with those in Table 3), so it is not unreasonable that the mean value from 1000 simulated data sets differs slightly from the simulated value. We now consider the misspecified variance results in part B of the above table. Here the values of q5 are roughly what they were in the one series simple case. This follows because the four seasons are like four short series with different mean levels and therefore different levels of dispersion. Results from the three series simple simulations (Section 4.3) suggest the average Chapter 4. Results of the Simulation Study^ 73 of those levels of dispersion is what is being estimated estimated by the overall estimator sr4. Because the ri were chosen so that E i r = 0, the average is approximately the same as the amount of dispersion in the one series simple simulations of Section 4.2 (slightly different because the mean levels of the four "series" don't quite average out to the overall mean; >, exp{/3 does not equal exp{E i (0 -I- TO} = exp{0}). Thus the estimates of 0 in the above table are as we would expect. More importantly, the mean values of the regression parameter estimates are quite close to the simulated values at all levels of dispersion and for all choices of 0. The standard deviations of the parameter estimates and the mean values of the standard errors of the regression parameters are summarized in Table 18. In the correct variance simu- lations (part A of the table), the empirical and model-based estimators are very close to each other in all cases, and agree well with the actual standard deviations of the parameter estimates. There is one instance (0 = log0.75 and 0 = 3) when the mean values of the estimated standard errors for all the regression parameters are slightly lower than the standard deviations. This could be a suggestion that the estimators underestimate the standard errors when there is a large amount of dispersion and low overall mean level, but could also just be chance. In any case, there is no strong evidence that the empirical estimator particularly underestimates the standard errors in general as expected a priori (based on conventional wisdom), or at least not any more so than the model-based estimator. Overall there is no real difference in these estimated standard errors. Note however that the model-based estimated standard error is uniformly less variable than that based on the empirical estimator (although the difference is small in this particular situation). Chapter 4. Results of the Simulation Study^ 74 In the misspecified variance simulations there may be a problem with the model-based estimator similar to the problems in the three series simple case simulations of Section 4.3. If we think of the four seasons as four short series, then under the misspecified variance formulation, the four series, with different mean levels, have different amounts of dispersion. In particular, the simulated dispersion for the data in the third season will be quite a bit lower than for the other seasons. The "overall" estimate ::$. will be larger than the amount of dispersion simulated for this season which might explain why the model-based estimates of standard errors for f 3 are too high on average. This same argument would suggest that the standard errors for f 1 and 1:2 would be too low since these "series" have the highest means and therefore the most dispersion so that q will be lower than the amount of dispersion simulated in these series. There is some suggestion of this as well in part B of Table 12. As for the empirical estimator, overall it appears to estimate the standard deviations of the parameters much better than the model-based estimator. There is no clear evidence that this estimator consistently underestimates the standard errors of the regression parameters although whenever the average of the empirical estimates differs to any appreciable extent from a simulated standard deviation, this average does appear to be too low. o Q log 0.75 Ti T2 T3 Chapter 4. Results of the Simulation Study^ 75 Table 18. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.). A. Correct Variance Function Simulated ,8 Parameter Method Simulated 0 1 1.4 2 3 5 Simulated .056 .097 .175 .326 Simulated Model Empirical .086 .088(.004) .087(.006) .103 .103(.006) .103(.008) .124 .123(.009) .123(.011) .153 .150(.012) .150(.015) Simulated .118 .135 .165 .206 Model .117(.005) .138 (.006) .165 (.009) .201(.013) Empirical .116(.006) .137 (.007) .164 (.010) .200(.014) Simulated .115 .135 .164 .205 Model .117(.005) .138 (.006) .165 (.009) .201(.013) Empirical .116(.006) .137 (.007) .164 (.010) .200(.014) Simulated .142 .166 .205 .258 Model .143(.007) .168(.009) .202(.012) .247(.019) Empirical .142(.008) .168(.010) .201(.014) .244(.021) Simulated .056 .087 .136 .245 Simulated .055 .064 .077 .093 Model .053(.002) .063(.003) .075(.004) .092(.005) Empirical .053(.003) .063(.004) .075(.005) .092(.007) Simulated .074 .087 .097 .121 Model .071(.002) .084 (.003) .100 (.004) .123(.005) Empirical .071(.003) .084 (.004) .100 (.005) .122(.006) Simulated .072 .085 .101 .123 Model .071(.002) .084 (.003) .100 (.004) .123(.006) Empirical .071(.003) .084 (.004) .100 (.005) .122(.006) Simulated .089 .103 .123 .153 Model .087(.003) .103(.004) .123(.005) .150(.007) Empirical .086(.004) .102(.005) .122(.006) .149(.009) Simulated .054 .078 .122 .200 .388 Simulated .034 .042 .047 .060 .077 Model .034(.001) .040(.001) .048(.002) .059(.002) .076(.004) Empirical .034(.002) .040(.002) .048(.003) .059(.004) .076(.005) Simulated .046 .056 .062 .081 .105 Model .045(.001) .054 (.002) .064 (.002) .078(.003) .101(.004) Empirical .045(.002) .053 (.002) .064 (.003) .078(.004) .101(.005) Simulated .045 .054 .063 .079 .102 Model .045(.001) .054 (.002) .064 (.002) .078(.003) .101(.004) Empirical .045(.002) .053 (.002) .064 (.003) .078(.004) .101(.005) Simulated .057 .067 .078 .100 .126 Model .055(.002) .065(.002) .078(.003) .096(.004) .123(.006) Empirical .055(.002) .065(.003) .078(.003) .095(.005) .123(.007) 0.70^0 # T1 T2 T3 1.60^0 )3 Ti T2 T3 Chapter 4. Results of the Simulation Study^ 76 Table 18 B. Misspecified Variance Function Simulated 16 Parameter Method Simulated 0 1 1.4 2 3 5 log 0.75 0 Simulated .052 .087 .140 .253 .■. # Simulated Model Empirical .088 .087(.005) .087(.006) .101 .100(.006) .099(.008) .116 .117(.008) .115(.009) .139 .140(.011) .137(.013) II. .■ ri Simulated .119 .137 .162 .196 ...■ Model .117(.005) .134 (.006) .156 (.008) .186(.011) ..... Empirical .116(.006) .135 (.007) .157 (.009) .190(.012) ■.. 72 Simulated .118 .136 .165 .192 .1. Model .117(.005) .134 (.006) .156 (.008) .186(.011) Empirical .116(.006) .134 (.007) .157 (.009) .189(.012) ■ 73 Simulated .140 .157 .185 .208 .1. Model .143(.007) .163(.009) .191(.011) .228(.016) Empirical .142(.008) .158(.010) .178(.011) .207(.015) 0.70 0 Simulated .054 .120 .261 .525 13 Simulated .055 .072 .094 .125 Model .053(.002) .072(.003) .093(.005) .121(.009) Empirical .053(.003) .071(.005) .091(.007) .118(.011) Ti. Simulated .073 .100 .132 .172 Model .071(.002) .096 (.004) .125 (.006) .161(.009) Empirical .071(.003) .097 (.005) .128 (.007) .165(.010) 72 Simulated .071 .099 .130 .165 Model .071(.002) .096 (.004) .125 (.006) .161(.009) Empirical .071(.003) .098 (.004) .127 (.007) .165(.011) 73 Simulated .090 .111 .146 .177 Model .087(.003) .118(.005) .152(.008) .197(.012) Empirical .087(.004) .109(.005) .136(.008) .173(.011) 1.60 0 Simulated .057 .209 .525 1.23 2.96 # Simulated .034 .059 .084 .118 .155 Model .034(.001) .059(.002) .084(.005) .114(.008) .157(.014) Empirical .034(.002) .058(.004) .082(.006) .110(.010) .152(.017) Ti Simulated .047 .083 .118 .165 .217 Model .045(.001) .079 (.003) .112 (.005) .152(.008) .210(.015) Empirical .045(.002) .081 (.004) .115 (.006) .156(.010) .215(.017) Ti Simulated .046 .081 .119 .159 .214 Model .045(.001) .079 (.003) .112 (.005) .152(.008) .210(.015) Empirical .045(.002) .081 (.004) .115 (.006) .156(.010) .215(.018) T3 Simulated .055 .090 .122 .164 .221 Model .055(.002) .097(.004) .137(.007) .186(.013) .256(.020) Empirical .055(.002) .087(.004) .120(.006) .159(.012) .218(.018) Chapter 4. Results of the Simulation Study^ 77 In the next table we present the observed rejection probabilities for the four test statistics under study in each of the combinations of /3 and 0 considered above. Here the alternate model (to be fit to the data sets generated under the reduced model) includes the sinusoidal "pollution" covariate. In the correct variance simulations the model Wald and scaled deviance tests give almost identical results in all simulations. The empirical tests are not as similar (either to each other or to the model-based tests), but overall there appears to be little to distinguish any of the tests. The observed rejection probabilities are reasonably close to the nominal levels in almost all cases. Possible exceptions might be the simulations with /3 = 1.60 and either of 0 = 1 or 0 = 3, but these are likely just chance deviations from the nominal levels. In the misspecified variance simulations we also find the model Wald and scaled deviance tests to be almost identical in their observed rejection probabilities. In general the score test appears most conservative and it seems to get more conservative (relative to the others) as the amount of dispersion increases. This makes it look like it is performing better than the others when the others reject too often and it looks worse when the others reject at about the nominal rate. As a side note, based on the model-based standard errors of Table 18, we might look carefully at the empirical Wald test to see if it rejects too often, since it seemed the empirical estimates of the standard errors of some of the T, were too small, which might imply that the standard errors of the coefficient of the pollution covariate could be underestimated. The empirical Wald test does reject more often than the score test in general, but it occasionally rejects less often than the model-based tests so that no general conclusion regarding its performance can be reached from these results. Chapter 4. Results of the Simulation Study^ 78 Table 19. Observed Rejection Probabilities Under Null Hypothesis. A. Correct Variance Function 0 = 1^4 =1.41.^0 = 2 Simulated 0 Test/Method log 0.75 Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 0.70 Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 1.60 Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 0.10 0.05 0.01 0.10 0.05 0.01 0.10 0.05 0.01 .093 .060 .011 .112 .056 .012 .110 .052 .014 .098 .063 .013 .114 .057 .010 .115 .053 .012 .098 .062 .012 .112 .055 .009 .110 .047 .011 .093 .060 .011 .111 .058 .012 .110 .053 .014 .100 .052 .014 .099 .045 .008 .098 .052 .013 .101 .049 .012 .102 .044 .010 .101 .056 .011 .095 .047 .012 .099 .044 .010 .097 .053 .011 .100 .052 .014 .099 .044 .008 .098 .052 .013 .115 .054 .011 .092 .040 .011 .109 .053 .009 .121 .055 .011 .091 .041 .012 .111 .053 .010 .119 .053 .010 .091 .041 .010 .109 .053 .008 .115 .055 .011 .092 .040 .011 .109 .054 .009 0 = 3^0 = 5 Simulated /3 log 0.75 Test/Method Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 0.10 0.05 0.01 .062 .012 .066 .016 .059 .011 .062 .012 0.10 0.05 0.01 0.70 Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 1.60 Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled .099 .048 .009 .107 .048 .013 .101 .045 .007 .100 .047 .009 .121 .062 .014 .125 .063 .015 .123 .057 .014 .121 .062 .014 .105 .048 .010 .108 .057 .012 .106 .051 .009 .106 .049 .010 Chapter 4. Results of the Simulation Study^ 79 Table 19 B. Misspecified Variance Function 0 = 1^0 = 1.4^0 = 2 Simulated /3 Test/Method^0.10 0.05 0.01 0.10 0.05 0.01 0.10 0.05 0.01 log 0.75 Wald/Model .110 .051 .005 .112 .049 .004 .111 .058 .010 Wald /Empirical .113 .055 .005 .111 .047 .004 .112 .059 .007 Score/Empirical .111 .052 .005 .110 .042 .004 .109 .055 .005 Deviance/Scaled .110 .052 .005 .112 .049 .004 .111 .059 .010 0.70^Wald/Model^.095 .049 .009 .105 .049 .010 .109 .060 .013 Wald/Empirical .098 .051 .013 .103 .048 .008 .099 .055 .015 Score/Empirical .095 .051 .011 .100 .046 .007 .094 .057 .013 Deviance/Scaled .095 .049 .009 .105 .048 .009 .109 .060 .013 1.60^Wald/Model^.101 .051 .010 .102 .053 .013 .105 .052 .010 Wald/Empirical .102 .054 .011 .102 .046 .011 .098 .046 .008 Score/Empirical .102 .053 .010 .099 .045 .010 .092 .045 .008 Deviance/Scaled .101 .051 .010 .103 .053 .013 .104 .053 .011 0 = 3 Cb = 5 Test/Method Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 0.10 0.05 0.01 .113 .059 .010 .110 .056 .010 .102 .052 .007 .113 .060 .011 0.10 0.05 0.01Simulated /3 log 0.75 0.70 Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled 1.60 Wald/Model Wald/Empirical Score/Empirical Deviance/Scaled .102 .052 .018 .093 .049 .014 .087 .046 .013 .103 .053 .018 .108 .056 .016 .104 .050 .017 .100 .046 .013 .108 .057 .016 .112 .056 .018 .110 .056 .018 .103 .049 .015 .114 .057 .018 We now consider the simulations under the full model (with pollution included). As in the simple case simulations, we chose to simulate data using only 0 = log 0.75 but with one of -y = 0.05, 0.10 and 0.15. Table 20 presents the mean values of the estimated parameters from the simulated data sets generated under the above model. Chapter 4. Results of the Simulation Study^ 80 Table 20. Mean Values of Estimated Parameters Under Alternative Hypothesis. A. Correct Variance Function Simulated 4 Simulated y Parameter 1 1.4 2 3 0.05 0 1.00 1.40 1.97 2.94 /3 -.293 -.295 -.311 -.303 Ti .254 .253 .274 .256 r2 .249 .251 .276 .254 T3 -.497 -.503 -.503 -.513 7 .054 .051 .038 .055 0.10 0 1.00 1.39 1.97 2.94 /3 -.289 -.287 -.304 -.306 sn. .246 .237 .259 .258 72 .247 .244 .260 .259 T3 -.506 -.511 -.510 -.515 7 .103 .104 .097 .098 0.15 Cb 1.00 1.40 1.98 2.94 /3 -.289 -.304 -.300 -.299 Ti .248 .271 .260 .252 T2 .247 .272 .259 .259 T3 -.498 -.511 -.515 -.532 7 .154 .132 .147 .144 B. Misspecified Variance Function Simulated 0 Simulated 7 Parameter 1 1.4 2 3 0.05 0 1.00 1.31 1.77 2.54 /3 -.293 -.295 -.296 -.293 Ti .254 .253 .250 .235 72 .249 .252 .254 .246 r3 -.497 -.504 -.504 -.515 7 .054 .050 .050 .056 0.10 Cb 1.00 1.31 1.79 2.56 /3 -.289 -.294 -.295 -.291 Ti .246 .250 .248 .240 T2 .247 .251 .255 .241 T3 -.506 -.505 -.504 -.516 7 .103 .104 .101 .102 0.15 95 1.00 1.32 1.79 2.57 /3 -.289 -.298 -.294 -.303 7-1. .248 .254 .246 .252 72 .247 .263 .246 .257 T3 -.498 -.501 -.502 -.496 7 .154 .145 .149 .155 Chapter 4. Results of the Simulation Study^ 81 Before interpreting these results, note that there is non-negligible collinearity between the continuous pollution covariate and the seasonal indicators. If, in our simulations, two ex- planatory variables have a substantial positive correlation, one of the corresponding regression parameters could be estimated to be larger than its simulated (or true) value, while the other could be estimated to be smaller than its simulated value, with little effect on the goodness- of-fit. Similarly, both could be estimated to be larger than the simulated values if negatively correlated. Such deviations from the true values, beyond the normal variation we would expect, lead to more variability from one set of fitted parameters to another than if the predictors were not collinear. The sample correlations are approximately 0.52 between the pollution variable and each of the indicators for the first and second season, and —0.52 between the pollution vari- able and the indicator for the third season. While such correlations are not a major concern, they must be kept in mind when examining the results of these simulations. For the correct variance simulations, the mean values of the regression parameter estimates are generally close to their simulated values, considering how big the standard deviations of the parameter estimates are (and thus the standard error of the average of the parameter estimates as an estimate of the true mean, see Table 21). However, in the cases -y = 0.05, 0 = 2 and -y = 0.15, 0 = 1.4 the averages of the estimates of 7-1 and r2 exceed the simulated value by more than two standard errors (while the estimate of -y is too small). In the case -y = 0.15, 0 = 3 the estimate of r3 is too small (a large negative value that differs from the true value by more than two standard errors) with the estimate of 7 slightly too small, although it differs from the true value by less than one standard error. Note how the pattern of over and underestimation agrees with what we might expect when the pollution variable is positively correlated with the Chapter 4. Results of the Simulation Study^ 82 first two season indicators and negatively correlated with the third. But there is also a hint of a pattern in the estimates of r3 which has no such plausible explanation; in general, as 0 increases across the table, r3 appears to decrease (become a more negative value). The observations in the third season have the lowest mean values, and as the amount of dispersion increases, there will be more and more zero values among these observations. We might infer that for large enough amounts of over-dispersion, this could be causing the estimator to underestimate the season effect, but with the low degree of precision in the averages of the parameter estimates we can not conclude whether this is the case, or if the deviations from the simulated values are just chance. In the misspecified variance simulations, the regression parameters estimates are very good; in fact, they appear somewhat better than in the correct variance case. The estimates of 0 seen in parts A and B of this table are similar to those in Table 17. The standard deviations of the parameter estimates in the 1000 simulations and the mean values of the standard errors of the regression parameters are summarized in Table 21. As in part A of Table 18, under the correct variance formulation the model and empirical estimates are very similar. Both estimators are very close to the standard deviations of the parameter estimates when the dispersion is 1 or 1.4. For simulated dispersions of 2 or 3, the averages of the estimates are not as close to the simulated values but the differences can be explained by the variability of the observed standard deviations and the estimated standard errors. Chapter 4. Results of the Simulation Study^ 83 Table 21. Standard Deviations of Estimates Under Alternative Hypothesis and Mean Values of Estimated Standard Errors (±s.d.). A. Correct Variance Function Simulated y Parameter Method Simulated 0 1 ^ 1.4^2 ^ 3 0.05 ^o # ri r2 T3 7 .054 .099 .166 .326 .122 .149 .177 .221 Simulated Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical .125(.005) .148(.007) .176(.009) .215(.014) .124(.007) .147(.010) .175(.015) .214(.024) .209 .255 .300 .374 .212(.007) .251(.010) .298(.013) .364(.020) .211(.012) .249(.017) .296(.026) .362(.041) .210 .246 .305 .373 .213(.007) .251(.010) .298(.013) .365(.020) .211(.012) .249(.018) .296(.026) .364(.042) .143 .171 .194 .261 .145(.007) .172(.009) .205(.012) .251(.019) .144(.008) .171(.011) .203(.014) .247(.021) .138 .166 .203 .243 .139(.005) .164(.007) .205(.012) .238(.013) .138(.007) .163(.009) .203(.014) .238(.021) .054 .096 .173 .321 .126 .151 .172 .224 .126(.005) .148(.007) .177(.009) .216(.014) .125(.008) .147(.011) .175(.015) .214(.023) .210 .255 .287 .380 .212(.008) .250(.010) .298(.013) .364(.019) .210(.012) .248(.018) .295(.026) .361(.040) .217 .250 .289 .385 .212(.008) .250(.010) .299(.013) .364(.019) .211(.012) .249(.018) .295(.026) .360(.041) .144 .174 .215 .261 .147(.007) .174(.009) .209(.013) .255(.020) .147(.008) .173(.011) .207(.015) .251(.021) .139 .166 .188 .249 .138(.005) .163(.006) .195(.009) .237(.012) .138(.007) .162(.010) .193(.014) .236(.022) .054 .096 .178 .332 .128 .150 .174 .223 .126(.005) .149(.007) .177(.010) .216(.015) .125(.008) .148(.011) .176(.016) .215(.024) .220 .251 .284 .372 .211(.007) .250(.010) .298(.013) .363(.020) .210(.012) .248(.017) .296(.027) .361(.040) .216 .248 .292 .374 .211(.007) .250(.010) .298(.013) .363(.020) .211(.012) .248(.018) .297(.026) .361(.040) .152 .175 .222 .266 .149(.007) .177(.010) .212(.014) .259(.020) .149(.008) .176(.011) .210(.015) .255(.022) .141 .161 .186 .243 .138(.005) .163(.006) .194(.008) .236(.013) .137(.007) .162(.010) .193(.014) .235(.021) 0.10^46 rI r2 T3 7 0.15^cb Ti r2 T3 7 Chapter 4. Results of the Simulation Study^ 84 Table 21 B. Misspecified Variance Function Simulated y Parameter Method Simulated 0 1.4^21 3 .053 .086 .144 .259 .122 .145 .169 .197 .125(.005) .143(.007) .167(.008) .199(.012) .124(.007) .142(.010) .166(.013) .199(.020) .209 .245 .292 .351 .212(.007) .243(.009) .282(.012) .338(.017) .211(.012) .243(.017) .285(.023) .343(.037) .210 .253 .290 .345 .213(.007) .243(.010) .283(.012) .338(.017) .211(.012) .243(.017) .286(.024) .343(.037) .143 .163 .189 .212 .145(.007) .166(.009) .194(.011) .232(.016) .144(.008) .159(.009) .179(.011) .208(.014) .138 .167 .191 .227 .139(.005) .159(.006) .185(.008) .221(.011) .138(.007) .159(.009) .186(.013) .225(.020) .126(.005) .144(.007) .168(.009) .201(.012) .125(.008) .143(.010) .167(.014) .199(.020) .210 .244 .295 .350 .212(.008) .243(.009) .283(.012) .339(.016) .210(.012) .244(.017) .285(.024) .343(.037) .217 .247 .295 .349 .212(.008) .243(.009) .283(.012) .339(.017) .211(.012) .243(.017) .286(.025) .342(.037) .145 .171 .190 .220 .147(.007) .169(.009) .197(.012) .237(.017) .147(.008) .162(.010) .182(.01) .210(.015) .139 .160 .193 .227 .138(.005) .159(.006) .184(.008) .221(.010) .138(.007) .159(.009) .187(.013) .225(.019) .054 .084 .146 .259 .120 .148 .170 .207 .126(.005) .145(.007) .168(.009) .203(.012) .125(.008) .144(.010) .166(.013) .199(.019) .202 .252 .289 .348 .212(.008) .243(.009) .283(.012) .340(.017) .210(.012) .244(.017) .284(.023) .342(.035) .204 .251 .296 .346 .212(.008) .243(.009) .283(.012) .340(.017) .211(.012) .244(.017) .285(.024) .343(.035) .148 .168 .183 .226 .150(.007) .172(.009) .200(.013) .241(.017) .149(.008) .164(.010) .183(.012) .211(.014) .136 .165 .194 .224 .138(.005) .158(.006) .184(.008) .221(.011) .137(.007) .159(.009) .187(.013) .226(.019) 0.05 ^0 o Ti T2 T3 7 Simulated Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical 0.10 ^ 4. # Ti T2 T3 7 Simulated Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical 0.15 ^ 0 15' 71 r2 T3 7 Simulated Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical Simulated Model Empirical .054 .085 .143 .264 .126 .145 .172 .203 Chapter 4. Results of the Simulation Study^ 85 To estimate the variance of the observed standard deviations, note the following. Because re- gression parameter estimates are asymptotically normal, if B t is a regression parameter estimate from the ith simulated data set and B is the average of these estimates, then (7 -2 Ei(Oi - 6) 2 , where o.2 represents Var(Oi), should have approximately a x 2 _ 1 distribution. It follows that the sample variance of these estimates should have variance roughly equal to 2a 4 /(n — 1). Using the delta method we arrive at the approximation Var(s) o-2 /[2(n — 1)]. Using this approximation, observed standard deviations of s = 0.3 and s = 0.2 would have standard errors of approximately 0.0067 and 0.0045 respectively. The mean values of the estimated standard errors will have approximate standard errors given by their standard deviations (enclosed in brackets in the table) divided by ..076 -00. When is 2 this can be as large as .027/1005 = 0.0009. With these precisions in mind, the observed differences between the variance estimates and the standard deviations are not too large. Another approach to determining whether the model-based and empirical standard errors approximate the simulated standard errors can be based on the increased precision provided by generating more data sets for a given choice of y and 0. The following table summarizes the results of the simulations for the case 7 = 0.10 and = 2 as displayed in part A of Table 21, and the results of five repeats of the simulation experiment using the same y and q5. Chapter 4. Results of the Simulation Study^ 86 Parameter Method original repeat 1 repeat 2 repeat 3 repeat 4 repeat 5 average /3 Simulated .172 .169 .182 .180 .177 .179 .177 Model .177 .177 .177 .177 .177 .177 .177 Empirical .175 .175 .176 .176 .176 .175 .176 Ti. Simulated .287 .293 .311 .307 .296 .298 .299 Model .298 .299 .299 .298 .298 .298 .298 Empirical .295 .296 .296 .297 .295 .296 .296 T2 Simulated .289 .289 .304 .299 .297 .300 .296 Model .299 .299 .299 .299 .298 .299 .299 Empirical .295 .296 .297 .297 .297 .296 .296 r3 Simulated .215 .206 .209 .215 .207 .215 .211 Model .209 .209 .209 .209 .208 .209 .209 Empirical .207 .207 .207 .207 .207 .207 .207 7 Simulated .188 .196 .196 .197 .194 .192 .194 Model .195 .195 .195 .195 .194 .195 .195 Empirical .193 .194 .194 .193 .194 .194 .194 From the above table we note that in each simulation experiment, the mean values of the model and empirical estimates of the standard errors are very close to the average of the simulated standard deviations across the columns of the table. This confirms that the model and empirical estimators do a good job of estimating the standard deviations of the parameter estimates. We now examine part B of Table 21 corresponding to the simulations using the misspecified variance formulation to generate the data. Here we seem to be seeing the same pattern as in the simulations using data generated with the reduced model (part B of Table 18). The model- based estimates appear too large for regression parameters which correspond to "seasons" with lower than average dispersion and too small for parameters which correspond to "seasons" with larger than average dispersion. In this situation, with a single pollution covariate, there is no clear picture of under or overestimation of the standard errors of y. However, one might speculate that if the pollution covariate were associated with one of the seasons (such as a Chapter 4. Results of the Simulation Study^ 87 season x pollution interaction) the standard errors of the regression parameter might be under or overestimated, depending upon which season the covariate was related to, and this could lead to unexpected results. As for the empirical estimator, the results suggest the possibility that the standard errors of the regression parameters tend to be underestimated in these simulations. Of course, when making these observations we should keep in mind the possibility that the observed standard deviations of the parameter estimates may differ somewhat from the true standard deviations. A similar exercise to that carried out in the correct variance simulations was carried out for these simulations under the misspecified variance for the same combination = 0.10 and 0 = 2. The final column of the table which follows shows the averages of the 6 simulation experiments. Parameter Method original repeat 1 repeat 2 repeat 3 repeat 4 repeat 5 average /3 Simulated .172 .165 .164 .171 .168 .167 .168 Model .168 .167 .168 .168 .168 .167 .168 Empirical .167 .166 .166 .167 .166 .166 .166 ri. Simulated .295 .289 .287 .288 .282 .291 .289 Model .283 .282 .283 .282 .282 .282 .282 Empirical .285 .285 .284 .286 .285 .285 .285 72 Simulated .295 .288 .286 .293 .286 .289 .290 Model .283 .283 .283 .283 .283 .283 .283 Empirical .286 .285 .285 .286 .285 .285 .285 73 Simulated .190 .189 .185 .182 .188 .190 .187 Model .197 .196 .197 .197 .197 .197 .197 Empirical .182 .180 .181 .181 .181 .181 .181 7 Simulated .193 .193 .192 .185 .190 .195 .191 Model .185 .184 .184 .184 .184 .184 .184 Empirical .187 .187 .187 .187 .187 .187 .187 The final column of this table suggests the model-based estimator does underestimate the standard errors for r1 and r2, and overestimate the standard error for r3 . We may also note that the standard error of y is underestimated by the model-based estimator in these simulations. Chapter 4. Results of the Simulation Study^ 88 This part of the table is also consistent with the observation that the empirical estimator appears to slightly underestimate the standard errors. Whether this is true for other combinations of y and 0 remains speculation. In summary, for the correct variance simulations, both the model-based and the empirical estimators of standard errors perform well. However in the misspecified variance simulations, there is some evidence that the model-based estimator does not perform well and that the empirical estimator may underestimate the standard errors slightly. Chapter 4. Results of the Simulation Study^ 89 Table 22. Observed Rejection Probabilities Under Alternative Hypothesis. A. Correct Variance Function 7 Test 0 = 1 0 = 1.4 0 = 2 0 = 3 .10 .05 .01 .10 .05 .01 .10 .05 .01 .10 .05 .01 .05 Wald/Mod. .121 .065 .014 .116 .062 .014 .121 .066 .012 .115 .063 .020 Wald/Emp. .124 .069 .015 .120 .062 .012 .127 .072 .014 .113 .066 .020 Score/Emp. .124 .070 .014 .116 .063 .012 .126 .067 .013 .110 .063 .017 Dev/Scaled .121 .065 .014 .116 .062 .014 .122 .067 .013 .115 .064 .022 .10 Wald/Mod. .185 .104 .027 .170 .102 .030 .130 .060 .016 .150 .087 .018 Wald/Emp. .196 .110 .030 .171 .103 .034 .129 .068 .018 .159 .085 .019 Score/Emp. .196 .108 .028 .167 .100 .032 .130 .068 .016 .156 .081 .017 Dev/Scaled .185 .108 .027 .176 .104 .030 .131 .061 .017 .151 .089 .020 .15 Wald/Mod. .298 .191 .076 .201 .126 .038 .195 .111 .019 .165 .087 .027 Wald/Emp. .300 .189 .081 .205 .134 .039 .188 .115 .027 .161 .095 .025 Score/Emp. .299 .190 .080 .206 .131 .040 .190 .114 .027 .159 .093 .023 Dev/Scaled .299 .192 .076 .206 .128 .040 .196 .113 .019 .167 .092 .028 B. Misspecified Variance Function 7 Test 0 = 1 0 = 1.4 0 = 2 0 = 3 .10 .05 .01 .10 .05 .01 .10 .05 .01 .10 .05 .01 .05 Wald/Mod. .121 .065 .014 .136 .075 .019 .120 .062 .018 .122 .052 .012 Wald/Emp. .124 .069 .015 .129 .079 .023 .117 .062 .018 .118 .047 .011 Score/Emp. .124 .070 .014 .129 .076 .020 .116 .058 .016 .111 .042 .007 Dev/Scaled .121 .065 .014 .137 .076 .019 .122 .063 .018 .121 .053 .013 .10 Wald/Mod. .185 .104 .027 .165 .092 .031 .167 .096 .024 .142 .078 .016 Wald/Emp. .196 .110 .030 .159 .090 .034 .165 .099 .020 .135 .074 .012 Score/Emp. .196 .108 .028 .159 .089 .032 .161 .097 .019 .129 .068 .012 Dev/Scaled .185 .108 .027 .165 .095 .031 .168 .097 .024 .145 .080 .018 .15 Wald/Mod. .298 .191 .076 .253 .166 .055 .219 .136 .047 .190 .106 .027 Wald/Emp. .300 .189 .081 .253 .166 .055 .212 .137 .048 .184 .105 .024 Score/Emp. .299 .190 .080 .250 .163 .052 .207 .133 .046 .179 .104 .022 Dev/Scaled .299 .192 .076 .254 .167 .056 .221 .138 .048 .191 .109 .028 The power of the four test statistics under study to detect the simulated pollution effect is summarized in Table 22; this table is similar in nature to Table 6 for the one series simple simulations. The results of the correct variance simulations suggest very low power for all test statistics in this more complicated case (only marginally higher than the nominal levels for the smallest simulated value of -y), but this is not surprising given the variability of the estimates Chapter 4. Results of the Simulation Study^ 90 of 7 compared to their simulated values. As in Table 6 the power of the tests tends to decrease as the amount of dispersion increases. The test statistics show similar rejection probabilities in the misspecified variance simulations. As in Table 6, the rejection probabilities, in general, are slightly higher in the misspecified variance simulations than in the correct variance simulations because for each simulated value of 0, there is less dispersion in the data under the misspecified variance formulation. Thus, except for the fact that the rejection probabilities are lower in Table 21 than in Table 6 due to the larger standard errors of y in these more complicated simulations, the test statistics perform similarly in the simple and complicated simulations with one series. We now summarize the findings of these simulations using the more complicated model for a single series of data. Estimates of 0 and of the regression parameters are relatively unaffected by the presence of over-dispersion although the estimators of these parameters became more variable as expected. Standard errors of the regression parameters were accurately estimated with either the model-based estimator or the empirical estimator when the data was generated using the correct variance formulation, but there is some evidence the model-based estimator performs poorly when the misspecified variance formulation was used to generate data. The empirical estimator appeared to do better in the misspecified variance simulations but it seemed to underestimate the standard errors when the level of dispersion was large relative to the overall mean level (this was not so apparent in the simulations under the reduced model reported in Table 19, but was more readily seen in the simulations under the full model reported in Table 21). In these simulations the problems with estimating the standard errors did not translate into problems with the model-based test statistics. This is in contrast to the simple case simulations with three series where poor performance of the model-based estimator of variances appeared Chapter 4. Results of the Simulation Study^ 91 to result in poor performance of the model-based test statistics. There were no detectable differences among any of the four test statistics under study. Chapter 4. Results of the Simulation Study^ 92 4.5 The More Complicated Case with Three Series The last case to be considered is the simulations based on (3.19) describing the more complicated case with three series. Recall that the simulated values for the mean and temporal parameters were: Pi = log 0.75 /32 = .70 /33 = 1.6 Tii = 0.2 T21 = 0.2 731 = 0.2 T12 = 0.2 T22 = 0.2 T32 = 0.3 713 = -0.2 T23 = -0.4 733 = -0.4 T41 = 0 T42 = 0 T43 = 0 The first simulations to be considered use the null model (ryo = 71 = 72 = 73 = 0) to generate data. The mean values of the parameter estimates from fitting the null model to these data sets are summarized in Table 23. For the correct variance simulations, we see that in both combinations, ii. is again estimated to be the average of the dispersions. The mean values of the regression parameter estimates are close to the simulated values. Table 23. Mean Values of Parameter Estimates. Parameter Variance A. Correct Function B. Misspecified Combination of 0 I^II Combination of 0 I^II 4) 1.998 2.785 5.157 8.676 pi -0.294 -0.292 -0.291 -0.300 /32 0.695 0.697 0.692 0.700 /33 1.597 1.594 1.595 1.575 T11 0.204 0.203 0.197 0.205 T12 0.204 0.198 0.197 0.211 ro -0.198 -0.199 -0.195 -0.193 721 0.205 0.198 0.205 0.203 T22 0.204 0.201 0.206 0.200 723 -0.397 -0.397 -0.397 -0.407 731 0.202 0.199 0.198 0.217 732 0.305 0.304 0.302 0.321 T33 -0.396 -0.398 -0.395 -0.396 Chapter 4. Results of the Simulation Study^ 93 In the misspecified variance simulations, the estimates of cb are in the same neighborhood as the estimates in the three series simple simulations. This is expected because the average of the levels of dispersion over all seasons in all series in these simulations is very close to the average of the three levels of dispersion in the three series simple simulations. The mean values of the regression parameter estimates are fairly close to the simulated values. Any deviations from the simulated values that we see here are likely random error due to the substantial amount of variability in the data sets, especially in the simulations using combination II. We now turn our attention to the standard deviations of the above estimates and the mean values of the estimated standard errors povided in Table 24. As in the three series simple case, in the correct variance simulations, the model-based standard errors for the parameters associated with the first series (th and ri j) are too large. This is a result of using the estimate c (estimated using data from all three series) to calculate the standard error for parameters corresponding to a series where the simulated dispersion is lower than 4. Analogously, we see that the standard errors for 03 and T3i are too small. For the regression parameters corresponding to series 2, we also find predictable results. Using combination I, the standard errors for 02 and .7-23 are about right because the simulated 0 for this series is equal to the average of the 0 for all three series. However, using combination II, the simulated 0 for series 2 is smaller than 4 and we see that the model-based standard errors are larger than the standard deviations of the parameter estimates. Chapter 4. Results of the Simulation Study^ 94 Table 24. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.). Parameter Method Variance A. Correct Function B. Misspecified Combination of (g) I^II Combination of 0 I^II 0 Simulated .084 .140 .407 .973 th Simulated .085 .105 .081 .103 Model .124 (.006) .146 (.009) .198 (.011) .258 (.019) Empirical .087 (.006) .103 (.008) .087 (.006) .100 (.008) 132 Simulated .076 .074 .092 .091 Model .075 (.003) .089 (.004) .121 (.007) .156 (.011) Empirical .075 (.005) .075 (.005) .092 (.007) .092 (.007) /33 Simulated .058 .074 .111 .157 Model .048 (.002) .057 (.002) .077 (.005) .101 (.009) Empirical .058 (.004) .075 (.005) .111 (.010) .153 (.018) rii Simulated .115 .141 .107 .136 Model .167 (.006) .197 (.009) .268 (.013) .348 (.023) Empirical .118 (.006) .139 (.007) .118 (.006) .136 (.007) r12 Simulated .111 .141 .113 .140 Model .167 (.006) .197 (.009) .268 (.013) .347 (.023) Empirical .118 (.006) .139 (.007) .118 (.006) .135 (.007) r13 Simulated .130 .154 .125 .150 Model .145 (.007) .218 (.010) .295 (.015) .384 (.025) Empirical .130 (.006) .154 (.008) .130 (.007) .147 (.008) r21 Simulated .106 .105 .133 .124 Model .102 (.003) .120 (.004) .164 (.008) .211 (.014) Empirical .101 (.005) .101 (.005) .128 (.007) .128 (.007) r22 Simulated .101 .102 .130 .125 Model .102 (.003) .120 (.004) .164 (.008) .211 (.013) Empirical .101 (.005) .101 (.005) .128 (.007) .128 (.007) 723 Simulated .115 .114 .138 .138 Model .119 (.004) .141 (.006) .192 (.010) .248 (.016) Empirical .119 (.006) .118 (.006) .135 (.007) .135 (.008) 731 Simulated .081 .099 .163 .216 Model .065 (.002) .077 (.002) .104 (.005) .136 (.009) Empirical .079 (.004) .102 (.005) .156 (.010) .216 (.018) 732 Simulated .077 .098 .158 .212 Model .063 (.002) .075 (.002) .102 (.005) .133 (.008) Empirical .077 (.003) .100 (.005) .156 (.010) .216 (.018) r33 Simulated .092 .119 .166 .225 Model .076 (.002) .090 (.003) .122 (.006) .160 (.011) Empirical .092 (.004) .119 (.007) .159 (.010) .218 (.018) Chapter 4. Results of the Simulation Study^ 95 The empirical estimates, on the other hand, are very good for all parameters (and at both levels of dispersion). There are only small differences between the empirical estimates and the observed standard deviations and there is no indication here that this estimator consistently underestimates the standard errors. Given the precision of the observed standard deviations and the average of the standard errors (discussed in the previous section), the observed differences can be explained by chance. In the misspecified variance simulations, it is even more apparent that the model-based estimator of the standard errors performs poorly while the empirical estimates are quite good. One could not say that the empirical estimates are consistently too low in these simulations. Table 25 summarizes the observed rejection probabilities of hypothesis tests for a common pollution covariate (H: 70 = 0) using the data generated under the reduced model. Table 25. Observed Rejection Probabilities: Common to Null Model. Test Variance A. Correct Function B. Misspecified Combination of q5 Combination of q5 0.10 I 0.05 0.01 0.10 II 0.05 0.01 0.10 I 0.05 0.01 0.10 II 0.05 0.01 Wald/Mod .135 .081 .016 .180 .108 .033 .206 .147 .056 .247 .153 .058 Wald/Emp .101 .047 .006 .112 .055 .014 .116 .058 .021 .108 .056 .017 Score/Emp .099 .045 .006 .112 .054 .010 .112 .057 .018 .102 .053 .016 Dev/Scaled .135 .081 .016 .180 .108 .033 .206 .146 .056 .246 .153 .058 In the correct variance simulations we find that with both combinations of 0, the empirical test statistics do reasonably well, achieving levels close to the nominal rejection probabilities. The model-based tests reject too often in the simulations using the first combination of 0 and even more so with the second combination. Notice again how very similar the results are for the Chapter 4. Results of the Simulation Study^ 96 two model-based tests. As in Table 24, this overall pattern is even more clear in the misspecified variance simulations. Thus in this three series complicated case we find that the parameter estimates are adequate as are the empirical estimates of the standard errors. However, as in the three series simple case, we find that the model-based estimator of standard errors performs poorly, presumably as a result of using a single estimate of 0 when in fact different amounts of dispersion are simulated in each series. This poor performance of the model-based estimator appears to carry through to the model-based test statistics. This should be expected in the case of the model-based Wald test but it is interesting that it is also true of the scaled deviance test. We now consider simulations using a model which includes the pollution covariate common to all three series; the simulated value is 7 0=0.1. The results summarized in Table 26 indicate that using either of the variance formulations, the overall effects and season effects are estimated just as well as they were in the null simulations. The mean values of the estimates of the pollution parameter are also close to the simulated value of 0.1. Chapter 4. Results of the Simulation Study^ 97 Table 26. Mean Values of Parameter Estimates. Parameter Variance A. Correct Function B. Misspecified Combination of 0 I^II Combination of 0 I^II 0 1.997 2.789 5.202 8.686 Al -0.293 -0.299 -0.294 -0.299 02 0.695 0.695 0.694 0.694 /33 1.601 1.597 1.593 1.589 Tii 0.203 0.212 0.205 0.202 712 0.202 0.208 0.201 0.205 ro -0.195 -0.193 -0.198 -0.192 721 0.203 0.204 0.201 0.200 722 0.200 0.199 0.201 0.200 723 -0.397 -0.400 -0.402 -0.402 731 0.196 0.198 0.200 0.192 T32 0.297 0.302 0.301 0.289 733 -0.407 -0.402 -0.406 -0.400 70 0.103 0.101 0.098 0.103 A summary of the standard deviations of the above parameter estimates as well as the mean values of the estimated standard errors of the regression parameters are presented in Table 27. For the correct variance simulations, as in Table 24, the model-based standard errors for pi and nj are overestimated while those for 133 and 733 are underestimated. Using combination I of 0, the model-based standard errors for /32 and rzi are about right (as expected, see above). With combination II of 0, the model-based standard errors for /32 are only slightly too large and somewhat surprisingly, only that for 723 is too large (recall that in Table 24 721, T222 _ 723 were all overestimated on average). We also see that the model-based standard errors for -yo are underestimated. This is consistent with the larger than expected rejection probabilities in Table 25 for the model-based Wald test. The empirical estimates, on the other hand, are very good for all regression parameters. Parameter Method Variance A. Correct Combination of 0 I^II 0 Simulated .083 .140 fii Simulated .100 .124 Model .133 (.006) .158 (.009) Empirical .100 (.006) .119 (.007) /32 Simulated .087 .097 Model .087 (.003) .103 (.004) Empirical .089 (.005) .094 (.005) /33 Simulated .076 .095 Model .063 (.002) .074 (.002) Empirical .075 (.005) .095 (.006) rii Simulated .156 .186 Model .185 (.006) .218 (.008) Empirical .147 (.005) .176 (.007) r12 Simulated .150 .180 Model .185 (.006) .217 (.008) Empirical .147 (.005) .176 (.007) r13 Simulated .130 .159 Model .190 (.007) .226 (.010) Empirical .134 (.007) .159 (.009) rzi Simulated .133 .149 Model .128 (.003) .151 (.004) Empirical .134 (.005) .148 (.006) r22 Simulated .132 .150 Model .129 (.003) .152 (.004) Empirical .134 (.005) .149 (.006) r33 Simulated .119 .120 Model .123 (.004) .145 (.006) Empirical .122 (.006) .122 (.006) r31 Simulated .120 .151 Model .101 (.002) .120 (.003) Empirical .118 (.007) .148 (.010) r32 Simulated .121 .149 Model .101 (.002) .120 (.003) Empirical .118 (.007) .148 (.010) r33 Simulated .094 .120 Model .078 (.002) .093 (.003) Empirical .096 (.005) .123 (.007) 7o Simulated .070 .086 Model .061 (.001) .072 (.002) Empirical .068 (.003) .084 (.004) Function B. Misspecified Combination of 0 I II ^ .423^.987 .119^.152 .215 (.012) .279 (.021) .121 (.006) .148 (.011) .122 .149 .140 (.008) .181 (.012) .123 (.007) .143 (.011) .036 .192 .102 (.005) .132 (.008) .138 (.012) .188 (.022) .200 .266 .297 (.014) .385 (.024) .201 (.011) .257 (.020) .196 .266 .298 (.014) .385 (.024) .201 (.011) .257 (.020) .130 .161 .307 (.016) .397 (.027) .134 (.007) .150 (.008) .206 .256 .207 (.009) .267 (.015) .207 (.012) .253 (.021) .209 .264 .208 (.009) .268 (.015) .208 (.012) .254 (.021) .136 .140 .199 (.010) .257 (.017) .137 (.007) .137 (.007) .219 .311 .163 (.006) .212 (.011) .224 (.020) .303 (.038) .225 .309 .163 (.006) 212 (.011) .225 (.021) .306 (.040) .162 .228 .127 (.006) .164 (.011) .160 (.010) .217 (.17) .127 .179 .098 (.004) .127 (.006) .127 (.010) .171 (.18) Chapter 4. Results of the Simulation Study^ 98 Table 27. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.). Chapter 4. Results of the Simulation Study^ 99 In the misspecified variance simulations, the model-based standard errors for th and the Tli are overestimated while those for 03 and the r32 are underestimated. The model-based standard errors for 02 are overestimated with both combinations of (1) while the standard errors for r21 and r22 appear quite good compared with the standard error of r 23 which is clearly overestimated. These results are similar to the correct variance results but are again different from the results of the simulations using the null model as summarized in Table 25. As for yo , the standard error for this parameter appears to be underestimated on average, just as it was in the correct variance simulations. The empirical estimates are close to the simulated values with combination I, but there is a hint the estimates are too low on average with combination II (although keeping the precisions of the estimates and standard deviations in mind, the differences seen here could be just chance). The empirical estimates seem to approximate the simulated standard deviations better here than they did in the single series complicated simulations (for data generated with the model including the pollution effect; see Table 21). The simplest explanation for the poorer results in the one series complicated case would be that in those simulations we considered much higher levels of dispersion relative to the overall mean value (0 = log 0.75) than in this three series case and these higher levels of dispersion contribute to underestimation of the standard errors. The levels of the four test statistics in a model reduction from separate pollution effects for each series to a common effect using the data generated with a common pollution effect are summarized in Table 28. Under both variance formulations and for both combinations of cb, the empirical test statistics adhere reasonably well to the nominal levels, but the model-based tests do not reject as often as they should. For the model-based Wald test this would suggest Chapter 4. Results of the Simulation Study^ 100 the standard errors for the separate pollution effects are overestimated (for at least one of the parameters). Again the model Wald and scaled deviance tests perform almost identically. Table 28. Observed Rejection Probabilities: Separate to Common. A. Correct Combination of 0 Variance Function B. Misspecified Combination of 0 Test I 0.10 0.05 0.01 II 0.10 0.05 0.01 I 0.10 0.05 0.01 II 0.10 0.05 0.01 Wald/Mod . Wald/Emp . Score/Emp . Dev/Scaled . 074 .032 .008 .056 .027 .004 .054 .028 .010 .055 .027 .004 096 .053 .012 .108 .050 .004 .117 .058 .016 .098 .057 .014 093 .050 .009 .106 .046 .004 .111 .058 .013 .097 .054 .011 074 .032 .008 .058 .028 .004 .053 .028 .010 .056 .028 .004 The powers of these tests to detect the common pollution effect represented by the simulated value of 70=0.10 are summarized in Table 29. In both variance formulations the model-based tests reject more often than the empirical tests (as expected given the underestimation of the standard errors of -yo and the consistent similarity of performance of the model Wald and scaled deviance tests). Predictably, the rejection probabilities decrease as 0 increases. Table 29. Observed Rejection Probabilities: Common to Null. Variance Function A. Correct^ B. Misspecified Combination of 0Combination of 0 Test I 0.10 0.05 0.01 II 0.10 0.05 0.01 I 0.10 0.05 0.01 II 0.10 0.05 0.01 Wald/Mod .522 .417 .224 .433 .334 .168 .335 .233 .102 .295 .219 .115 Wald/Emp .451 .340 .147 .338 .233 .083 .205 .112 .043 .181 .118 .047 Score/Emp .451 .340 .143 .338 .230 .082 .203 .108 .042 .171 .113 .038 Dev/Scaled .522 .418 .225 .434 .334 .168 .336 .234 .103 .298 .219 .115 The final model we will consider for the more complicated simulations with three series will be the most general one which includes separate pollution effects for each series. The simulated Chapter 4. Results of the Simulation Study^ 101 values were 70=0.1, 71=0.1, 72=-0.2 and y3=0. We begin as always with a table summarizing the mean values of the parameter estimates from 1000 simulated data sets generated with the above model. Table 30. Mean Values of Parameter Estimates. Parameter Variance A. Correct Function B. Misspecified Combination of 0 I^II Combination of 0 I^II 1.994 2.789 5.187 8.664 01 -0.289 -0.295 -0.299 -0.289 02 0.694 0.695 0.695 0.698 03 1.601 1.595 1.595 1.569 711 0.198 0.203 0.204 0.188 T12 0.203 0.202 0.205 0.193 T13 -0.209 -0.201 -0.191 -0.202 T21 0.203 0.205 0.196 0.195 T22 0.203 0.203 0.203 0.192 T23 -0.393 -0.400 -0.392 -0.399 T31 0.200 0.200 0.193 0.228 T32 0.299 0.302 0.298 0.323 733 -0.405 -0.401 -0.407 -0.402 7o 0.100 0.101 0.100 0.087 71 0.099 0.103 0.106 0.124 72 -0.200 -0.202 -0.195 -0.186 The estimates of /3 and T 3 are very similar to those in Tables 23 and 26 for both correct and misspecified variance formulations. In the correct variance simulations, the estimates are very good for 70 , 71 and 72, as they are in the misspecified variance simulations using combination I of 0. However the estimates of these pollution parameters differ noticably from the true values in the simulations using combination II of 0, although the differences are not too great considering the amount of variability in the simulated data which is reflected in the standard deviations of these parameter estimates as shown in Table 31. Note that the predictor corresponding to Chapter 4. Results of the Simulation Study^ 102 the overall pollution effect is not orthogonal to the predictors corresponding to the separate effects. The mean values of the parameter estimates in a reparameterized model with orthogonal predictors representing three separate pollution effects, as opposed to a main effect and two separate effects, would generally be closer to the simulated values (although the standard errors of those estimates would be smaller). In that case the interpretation of the table would remain the same as above; that is, the mean values of the estimates differ from the simulated values but are within acceptable limits considering the precisions of these mean values. In the correct variance simulations the mean values of the model-based standard errors for /3i and r 3 provided in Table 31 are very similar to those observed in Table 24 as one would expect. Recall that following the simulations with a common pollution effect (Table 27) there was some question about the model-based standard errors for 7 21 and r22 (they were noticably overestimated for combination II in Table 24 but did not appear to be overestimated in Table 27). The results in the above table and Table 24 agree with the conclusion that standard errors are underestimated (overestimated) on average for parameters that correspond to data series with more (less) dispersion than the estimate (7). We note that the estimated model-based standard error for the parameter 7 1 appears quite a bit larger than it should be. This would explain the large observed rejection probabilities for the model-based Wald tests (relative to the empirical) seen in Table 29. The empirical estimates of the standard errors for the pollution parameters look very good in these correct variance simulations. Parameter Method A. Correct Combination of 0 I^II 4, Simulated .084 .143 $1 Simulated .127 .145 Model .178 (.006) .211 (.010) Empirical .125 (.008) .148 (.010) $2 Simulated .108 .108 Model .108 (.003) .127 (.004) Empirical .107 (.007) .107 (.007) $3 Simulated .084 .103 Model .069 (.002) .082 (.003) Empirical .084 (.005) .108 (.007) ni Simulated .213 .244 Model .297 (.009) .352 (.013) Empirical .210 (.013) .247 (.017) r12 Simulated .212 .250 Model .298 (.009) .353 (.013) Empirical .210 (.012) .248 (.017) ri3 Simulated .134 .165 Model .196 (.008) .233 (.013) Empirical .138 (.007) .164 (.010) r21 Simulated .185 .188 Model .185 (.005) .218 (.007) Empirical .184 (.012) .183 (.013) r22 Simulated .189 .186 Model .186 (.005) .220 (.007) Empirical .184 (.012) .184 (.013) r23 Simulated .109 .115 Model .115 (.004) .136 (.005) Empirical .115 (.006) .114 (.006) r31 Simulated .144 .171 Model .116 (.003) .137 (.004) Empirical .141 (.009) .182 (.013) r32 Simulated .142 .170 Model .116 (.003) .138 (.004) Empirical .141 (.009) .182 (.013) r33 Simulated .095 .124 Model .078 (.002) .093 (.003) Empirical .095 (.005) .123 (.007) 'Yo Simulated .092 .113 Model .075 (.002) .089 (.002) Empirical .092 (.005) .119 (.007) 71 Simulated .165 .195 Model .207 (.005) .245 (.008) Empirical .164 (.006) .199 (.008) 72 Simulated .156 .165 Model .143 (.003) .169 (.004) Empirical .152 (.006) .169 (.007) Variance Function B. Misspecified Combination of ib I^II ^ .416 1.05 .122^.142 .287 (.015) .370 (.026) .126 (.008) .143 (.010) .139 .133 .174 (.009) .224 (.015) .133 (.010) .134 (.011) .165 .232 .112 (.005) .145 (.009) .164 (.016) .224 (.026) .207 .242 .480 (.022) .620 (.040) .210 (.012) .242 (.016) .207 .241 .481 (.022) .621 (.040) .210 (.012) .242 (.016) .139 .154 .317 (.016) .409 (.029) .139 (.007) .154 (.009) .236 .232 .298 (.014) .385 (.024) .230 (.019) .231 (.020) .238 .230 .300 (.013) .387 (.025) .231 (.019) .232 (.020) .140 .136 .186 (.010) .240 (.017) .134 (.007) .134 (.007) .284 .400 .188 (.007) .243 (.012) .316 (.032) .387 (.052) .289 .404 .188 (.007) .244 (.012) .287 (.031) .392 (.051) .159 .231 .127 (.006) .165 (.012) .160 (.010) .217 (.017) .181 .266 .122 (.004) .157 (.008) .187 (.016) .256 (.029) .220 .313 .334 (.014) .431 (.026) .231 (.014) .301 (.025) .235 .303 .230 (.009) .298 (.017) .240 (.014) .297 (.026) Chapter 4. Results of the Simulation Study^ 103 Table 31. Standard Deviations of Estimates and Mean Values of Estimated Standard Errors (±s.d.). Chapter 4. Results of the Simulation Study^ 104 With the misspecified variance simulations we again find the model-based standard errors for ij and to be very similar to the corresponding standard errors in Table 24. As in the correct variance case, the model-based standard error for 71 is overestimated. The empirical estimates are also quite good even in the simulations using combination II of 0. There are some parameters for which the empirical estimator produces estimated standard errors which are on average too low but there is no clear indication of underestimation in this situation. Finally, in Table 32, we present the observed rejection probabilities to evaluate the power of the four test statistics in the multiple parameter reduction from separate pollution effects to a common effect. Table 32. Observed Rejection Probabilities: Separate to Common. A. Correct Combination of cb Variance Function B. Misspecified Combination of Test I 0.10 0.05 0.01 II 0.10 0.05 0.01 I 0.10 0.05 0.01 II 0.10 0.05 0.01 Wald/Mod .358 .236 .102 .274 .168 .045 .120 .064 .012 .086 .047 .014 Wald/Emp .441 .319 .144 .396 .290 .121 .350 .234 .076 .336 .226 .080 Score/Emp .441 .315 .134 .390 .286 .115 .340 .224 .072 .325 .219 .071 Dev/Scaled .359 .238 .102 .275 .168 .045 .120 .064 .012 .086 .047 .014 The model-based test statistics have lower power than the empirical test statistics in this table. Recall that the model-based estimator overestimated the standard errors for -y i (while the standard errors for 72 were about right). This overestimation would explain why the rejection probabilities for the model-based Wald test are lower than for the empirical tests and, as usual, the scaled deviance test gives rejection probabilities that are almost identical to those for the model-based Wald test. Chapter 4. Results of the Simulation Study^ 105 The results of the simulations under the alternative hypotheses in this more complicated case with three series are qualitatively very similar to the results of the simple simulations with three series. We found that the estimator of 0 performed adequately in estimating the average level of dispersion and that the regression parameter estimator did well for all parameters including the "pollution" parameters that were of most interest. We also found the empirical estimator of the variances to perform well in these simulations. The underestimation of standard errors apparent in the more complicated case with a single series was not so clearly apparent in these three series simulations, possibly because the levels of dispersion considered in this section were not as large relative to the overall mean levels as they were in the single series simulations where, in the simulations using the full model to generate data, only 0 = log 0.75 was considered with values of (/) as large as 2 or 3. The model-based estimator of variance performed very badly in these three series simulations because the models did not allow for the possibility that different series may have different amounts of dispersion. Thus in a sense, the variance was actually misspecified in all the simulations (even those labelled "correct variance"). The model-based test statistics considered also seemed to suffer from the misspecification of the variance function. When considering the levels of the model-based statistics, they rejected too often in some instances and less often than they should have in others. The empirical tests, even the empirical Wald test, achieved levels much closer to the nominal levels. Chapter 5 Discussion The analysis of relationships between air pollution and human health data from Prince George provided the motivation for carrying out a simulation study designed to evaluate the per- formance of possible variance estimators and test statistics available for making inference in over-dispersed Poisson models. Moderate amounts of over-dispersion are reported to have little effect on the estimation of regression parameters (Cox, 1983), however the extra variability must be taken into account when estimating variances or testing hypotheses. In the Prince George study, over-dispersion was accounted for via a dispersion parameter 0 (common to all of the data series) which was estimated by the deviance, G 2 , divided by its degrees of freedom. To adjust for this over- dispersion, standard errors of the regression parameters were multiplied by the square root of the estimate ii;G = G 2 /d.f. and the usual likelihood ratio test (based on a Poisson likelihood), or deviance statistic, for testing the viability of a reduced model nested within a larger model, was divided by ,.. Proceeding with this methodology, models were fit to the emergency room visits data that included temporal, meteorological and the pollution parameters of primary interest. The final models, resulting from the model reduction procedures (for example, the final model for the pollutant TRS), included parameter estimates whose interpretation suggested that higher levels of pollution were related to lower numbers of emergency room visits for 106 Chapter 5. Discussion^ 107 respiratory illness. Such counter-intuitive results raised questions about the appropriateness of the methodology and motivated a simulation study. This study was designed to investigate the possibility that the amounts of over-dispersion encountered in the Prince George study were large enough (relative to the mean levels of the data series) to affect the estimation of regression parameters, and to examine the performance of other possible variance estimators and test statistics that could be employed in the analysis of such over-dispersed Poisson data. An alternate estimator of the dispersion parameter involving the Pearson X 2 statistic, .x =X2 /d.f., was discussed and was found to be superior to the estimator (7)G. In addition to the model-based estimator of the covariance matrix of the estimated regression parameters described in the methodology for the Prince George study, an empirical estimator, which does not rely on the correct specification of the variance function, was described. The two estimators of the covariance matrix give rise to two versions of the Wald test as alternatives to the scaled deviance test used in the Prince George study for determining if certain parameters (such as pollution effects) contribute to the fit of the model. An empirical score statistic, suggested in Breslow (1990), which uses the estimating equations themselves for inference in over-dispersed Poisson models, was also considered. In the simple simulations described first in this thesis, the regression parameter estimates were all very close to the simulated values, even when data were simulated with large amounts of over-dispersion and with a misspecified variance function. The simple simulations with only one series of data did not show any clear differences between the estimators of the variances nor between the test statistics. With the three series simple simulations, the variance function used to fit the data assumed a common 0 whereas a separate 0 applies in the simulation of each Chapter 5. Discussion^ 108 series. Thus the variance function was always misspecified, even for the simulations labelled "correct variance". It appears that in this situation, the model-based estimator overestimates (underestimates) the standard errors for regression parameters corresponding to series with simulated dispersion smaller (larger) than the estimated dispersion (which is the average of the simulated values from the three series). We found that, when considering the levels of the test statistics, the observed rejection probabilities of the model-based test statistics often did not approximate the nominal rates well whereas the empirical test statistics always achieved levels close to nominal. We found an interesting agreement between the model Wald and scaled deviance tests, as well as between the empirical score and empirical Wald tests. Using the more complicated model for a single series of data, estimates of 0 and of the regression parameters were relatively unaffected by the presence of over-dispersion although the estimators of these parameters became more variable and may have been affected by collinearity between some of the predictors. Standard errors of the regression parameters were accurately estimated with either the model-based estimator or the empirical estimator when the data were generated using the correct variance formulation, but there was some evidence the model-based estimator performs poorly when the misspecified variance formulation was used to generate data. It was hypothesized that the problem with the model-based estimator was similar to the problem in the three series simple case. The observations in different seasons of the one series complicated case had different mean values and therefore, under the misspecified variance formulation, had different amounts of dispersion (a situation similar to four separate series generated with the correct variance function). The empirical estimator appeared to do better in the misspecified variance simulations but seemed to underestimate the standard errors when the Chapter 5. Discussion^ 109 level of dispersion was large relative to the overall mean level. In these one series complicated simulations, the problems with the model-based standard errors did not translate into poor performance of the model-based test statistics; this is in contrast to the simple case simulations with three series. There were no detectable differences among any of the four test statistics under study in this one series complicated case. The results of the simulations in the more complicated case with three series were very similar to the results of the simple simulations with three series. The estimator of q performed adequately in estimating the average level of dispersion and the regression parameter estimators did well for all parameters including the "pollution" parameters that were of primary interest. The empirical estimator of the variances also performed well in these simulations. The under- estimation of standard errors apparent in the more complicated case with a single series was not so clearly apparent in the three series simulations, possibly because the levels of dispersion considered were not as large relative to the overall mean levels. The model-based estimator of variance performed very badly in the three series simulations because, again, the models used to fit the data did not allow for the possibility of differing amounts of over-dispersion in the different series. The model-based test statistics considered also seemed to suffer from the misspecification of the variance function. When considering the levels of the statistics, these tests did not achieve the nominal rates. The empirical tests, even the empirical Wald test, achieved levels much closer to the nominal levels. Overall, it appears solutions of the score equations produce good estimates of regression parameters in a situation with levels of over-dispersion similar to those in the Prince George study. The empirical estimator of the covariance matrix of the regression parameters appears Chapter 5. Discussion^ 110 preferable to the model-based estimator which relies on correct specification of the variance function. It appears that in situations where the variance function is misspecified, the two model-based test statistics which rely on a single estimated over-dispersion parameter may perform poorly. This is in sharp contrast to the empirical test statistics which always performed well in this study. An interesting side note to the investigation of the test statistics was a very close agreement between the two model-based tests throughout the simulation study. Only brief speculations were made regarding the possibility that different, and perhaps more appropriate, methods would have led to different conclusions in the Prince George study. It appears that, on average, the estimators of the regression parameters would yield correct estimates of the effects of pollution on emergency room visits, but it is unclear whether empirical estimates of the standard errors or an empirical test statistic would have led to a different set of final parameters in the models. Bibliography [1] Breslow, N. (1984). Extra -Poisson Variation in Log -Linear Models. Applied Statistics, 33, 38-44. [2] Breslow, N. (1990). Tests of Hypotheses in Overdispersed Poisson Regression and Other Quasi-Likelihood Models. Journal of the American Statistical Association, 85, 565 -571. [3] Carroll, R. J. and Ruppert, D. (1988). Transformation and Weighting in Regression. New York: Chapman and Hall. [4] Cox, D. R. (1983). Some Remarks on Over -dispersion. Biometrika, 70, 269-274. [5] Inagaki, N. (1973). Asymptotic Relations Between the Likelihood Estimating Function and the Maximum Likelihood Estimator. Annals of the Institute of Statistical Mathematics, 265, 1-26. [6] Knight, K., Leroux, B., Millar, J., and Petkau, A.J. (1988). Air Pollution and Human Health: A Study Based on Hospital Admissions Data from Prince George, British Columbia. Report prepared under contract with the Health Protection Branch, Department of National Health and Welfare. Also issued as SIMS Technical Report 128, Department of Statistics, University of British Columbia (February 1989). [7] Knight, K., Leroux, B., Millar, J., and Petkau, A.J. (1989). Air Pollution and Human Health: A Study Based on Emergency Room Visits Data from Prince George, British Columbia. Report prepared under contract with the Health Protection Branch, Department of National Health and Welfare. Also issued as SIMS Technical Report 136, Department of Statistics, University of British Columbia (June 1989). [8] Liang, K. Y. and Zeger, S. L. (1986). Longitudinal Data Analysis Using Generalized Linear Models. Biometrika, 73, 13-22. [9] McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd ed. London: Chap- man and Hall. [10] McNeney, B. and Petkau, A. J. (1991). Air Pollution and Human Health: A Follow - up Study Based on Emergency Room Visits Data from Prince George, British Columbia. Report prepared under contract with the Ministry of Health, Province of British Columbia. [11] Moore, D. F. (1986). Asymptotic Properties of Moment Estimators for Overdispersed Counts and Proportions. Biometrika, 73, 583 -588. [12] Royall, R. M. (1986). Model Robust Confidence Intervals. Intrernational Statistical Review, 54, 199-214. [13] Wedderburn, R. W. (1974). Quasi-Likelihood Functions, Generalized Linear Models, and the Gauss-Newton Method. Biometrika, 61, 439 -447. [14] White, H. (1982). Maximum Likelihood Estimation of Misspecified Models. Econometrica, 50, 1-25. 111
Cite
Citation Scheme:
Usage Statistics
Country | Views | Downloads |
---|---|---|
China | 23 | 1 |
United States | 1 | 0 |
Japan | 1 | 0 |
City | Views | Downloads |
---|---|---|
Beijing | 23 | 0 |
Sunnyvale | 1 | 0 |
Tokyo | 1 | 0 |
{[{ mDataHeader[type] }]} | {[{ month[type] }]} | {[{ tData[type] }]} |
Share
Share to: