Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

On continuous-time generalized AR(1) processes : models, statistical inference, and applications to non-normal.. 2002

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
ubc_2002-732630.pdf
ubc_2002-732630.pdf
ubc_2002-732630.pdf [ 17.83MB ]
ubc_2002-732630.pdf
Metadata
JSON: 1.0090499.json
JSON-LD: 1.0090499+ld.json
RDF/XML (Pretty): 1.0090499.xml
RDF/JSON: 1.0090499+rdf.json
Turtle: 1.0090499+rdf-turtle.txt
N-Triples: 1.0090499+rdf-ntriples.txt
Citation
1.0090499.ris

Full Text

On continuous-time generalized AR(1) processes: models, statistical inference, and applications to non-normal time series. by Rong Zhu B.Sc, USTC 1986 M.Sc, USTC 1988 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF Doc tor of Phi losophy in THE FACULTY OF GRADUATE STUDIES (Department of Statistics) we accept this thesis as conforming to the required standard The University of British Columbia March 2002 © Rong Zhu, 2002 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of The University of British Columbia Vancouver, Canada Date DE-6 (2/88) Abstract This thesis develops the theory of continuous-time generalized AR(1) processes and presents their use for non-normal time series models. The theory is of dual interest in probability and statis- tics. From the probabilistic viewpoint, this study generalizes a type of Markov process which has a similar representation structure to the Ornstein-Uhlenbeck process (or continuous-time Gaussian AR(1) process). However, the stationary distributions can now have support on non-negative in- tegers, or positive reals, or reals; the dependence structures are no longer restricted to be linear. From the statistical viewpoint, this study is dedicated to modelling unequally-spaced or equally- spaced non-normal time series with non-negative integer, or positive, or real-valued observations. The research on both the probabilistic and statistical sides contribute to a complete modelling procedure which consists of model construction, choice and diagnosis. The main contributions in this thesis include the following new concepts: self-generalized distributions, extended-thinning operators, generalized Ornstein-Uhlenbeck stochastic differential equations, continuous-time generalized AR(1) processes, generalized self-decomposability, general- ized discrete self-decomposability, P-P plots and diagonal P-P plots. These concepts play crucial roles in the newly developed theory. We take a dynamic view to construct the continuous-time stochastic processes. Part II is de- voted to the construction of the continuous-time generalized AR(1) process, which is obtained from the generalized Ornstein-Uhlenbeck stochastic differential equation, and the proposed stochastic in- tegral. The resulting continuous-time generalized AR(1) process consists of a dependent term and an innovation term. The dependent term involves an extended-thinning stochastic operation which i i generalizes the commonly used operation of constant multiplier. Such a Markov process can have a simple interpretation in modelling non-normal time series. In addition, the family of continuous- time generalized AR(1) processes is surprisingly rich. Both stationary and non-stationary situations of the process are considered. In Part III, we answer the question of what kind of stationary distributions are obtained from the family of continuous-time generalized AR(1) processes, as well as the converse question of whether a specific distribution can be the stationary distribution of a continuous-time generalized AR(1) process. This leads to the characterization of distributions according to the extended- thinning operations. The characterization results are meaningful in statistical modelling, because under steady state, the marginal distributions of a Markov process are the same as the stationary distribution. They will guide us to choose appropriate processes to model a non-normal time series. The probabilistic study also shows that the autocorrelation function is of exponential form in the time difference, like that of the Ornstein-Uhlenbeck or Ornstein-Uhlenbeck-type process. Part IV deals with statistical inference and modelling. We have studied parameter esti- mation for various situations such as equally-spaced time, unequally-spaced time, finite marginal mean, infinite marginal mean, and so on. The graphical tools, the P-P plot and diagonal P-P plot, are proposed for use in identifying the marginal distribution and serial dependence, and diagnosing the fitted model. Three data examples are given to illustrate the new modelling procedure, and the application capacity of this theory of continuous-time generalized AR(1) processes. These time series are non-negative integer or positive-valued, with equally-spaced or unequally-spaced time observations. iii Contents Abstract ii Contents iv List of Tables ix List of Figures xi Notations, abbreviations and conventions xv Acknowledgements xviii I Introduction 1 1 Overview 2 1.1 Motivation and literature review 2 1.2 Highlights of our new research 6 II Theory for model construction 12 2 Relevant background on characteristic tools, distribution families and stochastic processes 13 2.1 Ornstein-Uhlenbeck processes and Ornstein-Uhlenbeck-type processes 14 iv 2.2 Characterization tools of distributions and examples 16 2.2.1 Probability generating function 17 2.2.2 Laplace transformation, moment generating function and characteristic func- tion 34 2.3 Particular families of distributions 43 2.3.1 Infinitely divisible, self-decomposable and stable distributions 43 2.3.2 Tweedie exponential dispersion family 50 2.3.3 Generalized convolutions 54 2.4 Independent increment processes and examples 64 3 Self-generalized distributions and extended-thinning operations 72 3.1 Self-generalized distributions 73 3.1.1 Non-negative integer case and examples 73 3.1.2 Positive case and examples 76 3.2 Properties of self-generalized distributions 80 3.3 Construction of new self-generalized distributions 88 3.4 Extended-thinning operation 92 4 Generalized Ornstein-Uhlenbeck stochastic differential equations and their pos- sible solutions 103 4.1 Stochastic differentiation and integration 104 4.2 Generalized Ornstein-Uhlenbeck equations 109 4.3 Explanations, innovation types, non-stationary situations and examples 112 4.4 Construction of possible solutions for the generalized Ornstein-Uhlenbeck SDE . . . 116 4.5 Summary and discussion 126 5 Results for continuous-time generalized AR(1) processes 128 5.1 Main results for continuous-time GAR(l) processes 128 5.2 Non-negative integer innovation processes and examples 137 5.3 Positive-valued innovation processes and examples 152 5.4 Real-valued innovation processes and examples 157 5.5 Tweedie innovation processes 162 III Probabilistic and statistical properties 165 6 Stationary distributions, steady states and generalized AR(1) time series 166 6.1 Stationary distributions 167 6.2 Marginal distributions under steady state 169 6.2.1 Non-negative integer margins 170 6.2.2 Positive-valued margins 174 6.2.3 Real-valued margins 176 6.3 Customizing margins 178 6.4 Generalized AR(1) time series 196 7 Characterization of stationary distribution families 202 7.1 Self-decomposable and discrete self-decomposable classes 203 7.2 Generalized self-decomposable, generalized discrete self- decomposable classes and their infinite divisibility property 220 7.3 Relationships among the classes of generalized self-decomposable and discrete self- decomposable distributions 236 8 Transition and sojourn time 244 8.1 Infinitesimal transition analysis 245 8.1.1 Non-negative integer margin 245 8.1.2 Positive-valued margin 255 8.2 Characteristic feature of the P D E of the conditional pgf or L T 257 8.2.1 Non-negative integer margin: P D E of the conditional pgf 258 8.2.2 Positive-valued margin: P D E of the conditional L T 263 8.2.3 Summary: margins, self-generalized distribution and increment of innovation 266 vi 8.3 Distributions of sojourn time 268 9 Conditional and joint distributions 271 9.1 Consistency in process construction: the view from distribution theory 271 9.2 Conditional properties 283 9.3 Joint properties 302 9.3.1 Bivariate distributions 303 9.3.2 Multivariate distributions 314 I V S t a t i s t i c a l in fe rence a n d a p p l i c a t i o n s 320 10 Parameter estimation 321 10.1 Maximum likelihood estimation 322 10.2 Conditional least squares estimation and variations 325 10.3 Empirical characteristic function estimation approach and variations 341 10.4 Other estimation approaches 346 10.5 Numerical solution of optimization 352 11 Asymptotic study of estimators 354 11.1 Random sampling scheme, assumptions and fundamental theorem 355 11.2 Asymptotic properties of M L E 362 11.3 Asymptotic properties of conditional least squares estimator 373 12 Autocorrelation detection, model selection, testing, diagnosis, forecasting and process simulation 385 12.1 Assessing autocorrelation 386 12.2 Model selection 398 12.3 Model diagnostics and hypothesis testing 400 12.3.1 Graphical diagnostic method 400 12.3.2 Parameter testing 404 vii 12.4 Forecasting 405 12.5 Simulation of the continuous-time GAR(l) processes 407 13 Applications and data analyses 415 13.1 Introduction to modelling procedure . . 415 13.2 Manuscript data study 417 13.3 WCB claims data study 428 13.4 Ozone data study 444 V Discussion 456 14 Conclusions and further research topics 457 14.1 Discussion of continuous-time generalized AR(1) processes 458 14.2 Some thoughts on model construction 459 14.3 Future research 460 Appendix A Data sets 462 A . l Manuscripts data ; 462 A.2 WCB claims data 463 A.3 Abbotsford daily maximum ozone concentrations data 466 Bibliography 468 viii List of Tables 2.1 Summary of Tweedie exponential dispersion models (S = support set) 54 3.1 Some results from Theorem 3.3.1 89 6.1 Partial derivative of pgf, H(s), for self-generalized distributions with non-negative integer support 181 6.2 Partial derivative of negative log LT, H(s), for self-generalized distributions with positive support 183 6.3 Conditional pgf G(a)K,g,Xi-i\Xi-i=x{s) when K is a non-negative integer self-generalized random variable 198 6.4 Conditional LT 4>(a)K®xl-i\xl^i-x{s) when K is a positive self-generalized random variable 199 9.1 Mean and variance of non-negative integer and positive self-generalized random vari- able K{a) 287 9.2 Auto-covariance and auto-correlation function of the stationary continuous-time GAR(l) process associated with known self-generalized random variable K (e~M^ 2 _ i l^). Here the variance function is V(t) = V 305 13.1 Summary of the frequencies of the number of manuscripts in refereeing queue. . . . 418 13.2 Summary of the number of pairs by lag for the manuscripts data 419 ix 13.3 Summary of different estimates of parameter n and A in the GAR(1) model for the manuscript data 425 13.4 Summary of the series CS in WCB claims data 431 13.5 Estimated conditional probabilities: Pr[X(t') = y \ X(t) = x]. The highest probability in each column is highlighted with an asterisk 443 13.6 Estimated conditional cdf: Pi[X(t') < y | X(t) = x]. The median in each column is highlighted with an asterisk 443 13.7 One month predictions: ymode> ymedian and yP, 443 13.8 Summary of the series of daily maximum ozone concentration 444 List of Figures 3.1 Illustration of {<//<-(£); — 0 0 < t < 00} in Cases 1 and 2. (a) corresponds to non- negative integer X in Case 1, where dotted vertical lines indicate the discrete time points {0,1,2,...}. (b) corresponds to positive X in Case 2, where t is continuous on [0, 00) 95 3.2 Illustration of {Ji{t);t > 0}, {Ji{t);t > 0} and {Jfc(t); -00 < t < 00} in Case 3. . . 96 4.1 Illustration of increment in the deterministic and stochastic cases, (a) corresponds to deterministic function x(t). (b), (c) and (d) correspond to three different paths of the stochastic process {X(t);t > 0} 105 4.2 Illustration of stochastic integration via infinitesimal partition, (a) and (b) corre- spond to two different paths of the stochastic process {X(t);t > 0} 108 4.3 Illustration of the geometrical explanation of the extended-thinning operation, (a) corresponds to a constant multiplier cX; X can be either real or positive-valued, (b) corresponds to a non-negative integer X. (c) corresponds to a positive X 110 4.4 Illustration of the mechanism of the generalized Ornstein-Uhlenbeck SDE 113 7.1 The relationship of ID, DSD and GDSD{12) 238 7.2 The relationship of ID, SD and GSD(P2) 240 12.1 Sunflower plots of two time series count data. The left one is from the model in (12.1.1), while the right one is from an independent Poisson(5) series 389 xi 12.2 Scatterplots and diagonal P-P plots of positively correlated and negatively correlated bivariate normal data. The left side corresponds to positive correlation, while the right side corresponds to negative one 392 12.3 Diagonal P-P plots of two time series count data. The left one is from the model in (12.1.1), while the right one is from an independent Poisson(5) series 394 12.4 Randomized quantile transformation scatterplots of two time series count data. The left one is from the model in (12.1.1), while the right one is from an independent Poisson(5) series 396 12.5 The Auto-correlation function (ACF) plot of the count time series from the model in (12.1.1) 397 12.6 Simulation of a time series with length 100 from (12.5.2) with A = 2.15 and /J, = 0.43. 409 12.7 Simulation of time series with length 100 from (12.5.3) and (12.5.4). Both processes have Gamma(8.1,0.17) margins 411 12.8 Simulation of a continuous-time path from (12.5.2) with A = 2.15 and /J, — 0.43. . . 413 13.1 The time series plot of refereeing queue length of manuscripts 419 13.2 The histogram of the manuscript data, and its P-P plot against Poisson(2.412). . . 420 13.3 The ACF plot of the manuscript data. The dotted horizontal line indicates the 95% boundary of the estimate of correlation coefficient for 85 pairs of independent Poisson(2Al2) random variables; the boundary is obtained by simulation 421 13.4 The sunflower, randomized quantile transformation, and diagonal P-P plots for the pairs with lag 1, 2 and 3 months from the manuscript data. . : 422 13.5 The plot of function RcLS(ME)(ao) °f the manuscript data 424 1'.].() Model diagnosis for manuscript data: diagonal P-P plots for estimates of the CLS(ME) (top row), MLE (middle row) and CLS (bottom row) 426 13.7 Model diagnosis for manuscript data: diagonal P-P plots for estimates of the CWLS (top row) and diagonal PLS (bottom row) 427 13.8 Model diagnosis for manuscript data: diagonal P-P plots for the estimate (fx, A) = (0.433,1.04) 429 xii 13.9 The histogram of the series CS (left), and 1000 simulated sample variances from Poisson(6.l3); the dotted vertical line is the sample variance of CS (right) 431 13.10 The P-P plots of CS in WCB claims data against Poisson(Q.13) (left), NB(6M, 0.48) (middle) and GP(4.42,0.28) (right) 433 13.11 The time series plot (top) and ACF plot (bottom) of CS in WCB claims data. . . . 434 13.12 Serial dependence: sunflower plots (1st row), randomized quantile transformation plots (2nd row) and diagonal P-P plots (3rd row) of pairs in CS with lag 1, 2 and 3 months 435 13.13 Model diagnosis for WCB claims data: diagonal P-P plots of CS against the fitted (method of moments) NB GAR{1) model 438 13.14 Model diagnosis for WCB claims data: diagonal P-P plots of CS against the fitted (method of moments) GP GAR(1) model 439 13.15 Model diagnosis for WCB claims data: diagonal P-P plots of CS against the fitted (MLE) NB GAR(1) model 440 13.16 Model diagnosis for WCB claims data: diagonal P-P plots of C3 against the fitted (MLE) GP GAR{1) model 441 13.17 The histogram of the daily maximum ozone concentration, and the P-P plot against Gamma(8.03,0.17) 445 13.18 The time series plot and ACF plot for the daily maximum ozone concentration. . . 446 13.19 The scatterplots and diagonal P-P plots of lag one day to three days for the daily maximum ozone concentration 447 13.20 Model diagnosis: diagonal P-P plots of lag one day for 7 = 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7 and 1/1.17 in the ozone data study 450 13.21 One step ahead predictions (dotted line) by the conditional mean for the daily max- imum ozone concentrations (solid line) 451 13.22 The analysis of differences between observations and one step ahead predictions by the conditional mean for the daily maximum ozone concentration 453 xiii 13.23 Model diagnosis: diagonal P-P plot of lag one day for Model (13.4-3) at 8MLE = 8.31, J3MLE = 0.17 and aMLE = 0.51 xiv Notations, abbreviations and conven- tions We follow the widely used conventions throughout the thesis. Latin upper-case letters, often X, Y, Z, usually with subscripts or incorporated with other variables such as t, a, are used for random variables. Bold Latin upper-case letters, often X, Y, Z, are used for random vectors. Greek lower-case letters, say a, 8, 7, with or without subscripts, are used for parameters in models. Bold Greek lower-case letters stand for parameter vectors, e.g., a, /3, 7. For a vector or matrix, the transpose is indicated with a superscript of T. A l l vectors are column vectors; hence transposed vectors such as XT, xT are row vectors. List of notation and abbreviations: •R (—00,00) 5f t 0 [0,00) - K o (-oo,0] SR+ (0,oo) (-oo,0) X {1,2,...} N Q {0,1,2,...} ~ "distributed as" or "has its distribution in" = equality in distribution oc proportional to -<st stochastically smaller than convergence in distribution —> convergence in probability convergence almost surely Gx{s) probability generating function of rv X Mx(s) moment generating function of rv X $x{s) Laplace transformation (LT) of rv X ipx(s) characteristic function of rv X xv h~1(x) functional inverse function of h(x) * binomial thinning operator © extended-thinning a C L S Conditional least squares estimate of a " O W L S Conditional weighted least squares estimate of a a D P L S Diagonal probability least squares estimate of a a E C F Empirical characteristic function estimate of a aM Method of moments estimate of a a M L E MLE of a ACF Autocorrelation function AIC Akaike information criterion A M absolute monotonicity (absolutely monotone) AR(p) autoregressive of order p cdf cumulative distribution function cf characteristic function CLS conditional least square C M complete monotonicity (completely monotone) DSD discrete self-decomposable ECF empirical characteristic function iid independent and identically distributed IIP independent increment process ID infinitely divisible (infinite divisibility) iff if and only if GAR generalized autoregressive GC generalized convolution GDSD generalized discrete self-decomposable G L M generalized linear model GNBC generalized negative binomial convolution GP generalized Poisson xvi GSD generalized self-decomposable LT Laplace transform mgf moment generating function ML maximum likelihood M L E maximum likelihood estimate (estimation) NB negative binomial PDE partial differential equation pdf probability density function Pgf probability generating function pmf probability mass function P-P plot Probability-probability plot Pr probability of rv random variable SD self-decomposable SDE stochastic differential equation SE standard error SG self-generalized (self-generalizability) X\Y rv X conditioned on rv Y [X\Y = y] rv X conditioned on rv Y = y xvii Acknowledgements I'd like to warmest acknowledge my supervisor, Prof. Harry Joe, a tremendous mentor, for his constant inspiration and guidance. Without him, the theory of continuous-time generalized AR(1) processes would never have been developed. He casts a great impact on me in various aspects of statistical research. His sound intuition and experience always guide me on the right track. I have received numerous suggestions and discussions from him; these were of crucial help to me in the development of this theory. I would say that I am standing on the shoulders of a great statistician. I would also thank Prof. John Petkau and Prof. Lang Wu for serving on my supervisory committee. Their comments are very valuable to this thesis. In addition, I would thank Dr. Jian Liu, a former supervisory committee member, for his kind help and encouragement. Many thanks go to Prof. Paul Gustafson, Prof. Martin Puterman, Prof. Martin Barlow, Prof. Nancy Heckman, Prof. James Zidek, Prof. Reg Kulperger for their help and support in many circumstances. Particularly, Prof. Edwin Perkins had carefully corrected mistakes and given valuable suggestions in the probabilistic part of the theory of continuous-time generalized AR(1) processes. Special thanks to Ryan Woods, Mahbub Latif, Lee Shean Er, Yinshan Zhao, Rachel MacKay, Dana Scott Aeschliman, Weiliang Qiu, Jafar Ahmed Khan, Hongbin Zhang, Dr. Steven Xiaogang Wang, Dr. Renjun Ma, Dr. L i Sun, Dr. Huiying Sun, Dr. Kathy Huiqing L i , Dr. Peter Xuekun Song and others for their help, discussion, encouragement and support in diverse ways. No doubt, the support from my wife Yalin Chen and my son Si Chen is unforgettable. Their great patience and constant reminding are one of the key sources to impel me to complete this work. xviii Financial support from Prof. Harry Joe and the Department of Statistics in the past five years are highly appreciated. Finally, I take this chance to appreciate our student services coordinator Christine Graham. She offers far more than professional service to us. One example is that she always register us for the summer term, because many of us forget it for naively assuming that is vacation. Her excellent work and kind help deeply move me, and make my study at UBC very smooth. The Department of Statistics at UBC is an incredibly integrated and cozy study environ- ment. I would say that it is truly a statistician's home. xix Part I Introduction Chapter 1 Overview This thesis is devoted to the development of a theory to construct models for non-normal equally- spaced or unequally-spaced time series. The time series models are based on continuous-time stochastic processes in a class called generalized AR(1) processes, and these are constructed based on classes of random operators or stochastic differential equations. Section 1.1 briefly explains the motivation of this study, and reviews the relevant literature. In Section 1.2, we summarize the key ideas that led to our direction of theoretical development, and highlight the new concepts and main results in subsequent chapters; these may help readers to navigate through the details and obtain an integrated understanding of the theory of continuous- time generalized AR(1) process. 1.1 Motivation and literature review Dynamic phenomena exist in diverse disciplines like chemistry, physics, economics, actuarial science, epidemiology, biology, management science, and so on. It means an event evolving over time. People have been developing various stochastic process models to try to describe or approximate these phenomena. A series of observations of a dynamic process lead to a time series. 2 Traditionally, for a real-valued time series, we use the Gaussian or normal time series model, which has a Gaussian or normal marginal distribution. However, in reality, there are many situa- tions where the observed series are discrete or positive-valued. Such issues arise especially in the longitudinal studies or clinical trials. The marginal distributions are often skewed and have large variations. Hence, the normal marginal distribution is no longer directly suitable for such situa- tions. This has motivated the development of non-normal time series models, where the marginal distributions could be like the Poisson or Gamma distribution, to handle discrete or positive-valued data. Such a transition is similar to the transition from the linear model to the generalized linear model where the response variables are discrete or positive-valued. However, unlike the G L M where distributions for modelling discrete or positive-valued responses are well developed, there has been little past research for stochastic processes for discrete or positive-valued time series. For example, suppose we find the marginal distribution for a count data time series is well modelled by the generalized Poisson distribution, what kind of stochastic processes should we use? Or in other words, is there any simple stochastic process which has the generalized Poisson margin? We believe most people will face a difficulty when encountering such a problem. Therefore, it is important to construct probabilistic models which haven't been considered previously. In addition, the sampling scheme is another serious question. Usually, we take the equally- spaced sampling scheme when we design an experiment study. However, for practical reasons, we may obtain unequally-spaced observations. Many reasons could lead to such phenomena: • subjects can't be observed on the original schedule plan, say the patients can't visit the clinic for the scheduled appointments due to personal matters; • there exist missing values; • or even more extreme, the schedule can't be made equally-spaced, it is random. For a stationary process, an equally-spaced sampling scheme can guarantee the dependence struc- ture between two adjacent margins is always the same. However, this is not true when the sampling scheme is unequally-spaced. Unequally-spaced time series are sometimes called irregular time series. 3 Many methods have been developed toward this issue. One reasonable approach is to construct continuous-time stochastic process models as pointed out by Jones [1993], p. 56, because only the continuous-time underlying process can allow the observations taken at arbitrary time points. In this study, we focus on unequally-spaced count or positive-valued time series. We try to develop the continuous-time stochastic processes for them in a systematic approach. Before we proceed, we take a literature review for both discrete-time and continuous-time stochastic processes with marginal distributions whose support is the non-negative integers or the positive reals. For positive-valued margins, an incomplete list is Gaver and Lewis [1980] Lawrance and Lewis [1980], Wolfe [1982], Sato and Yamazato [1983], Lewis, McKenzie and Hugus [1989], Andel [1988, 1989a, 1989b], Rao and Johnson [1988], Hutton [1990], Sim [1990, 1993, 1994], Adke and Balakrishna [1992], Jayakumar and Pillai [1993], J0rgensen and Song [1998], Barndorff-Nielsen [1998b], etc. These marginal distributions include gamma, exponential, and so on. Most of them are discrete-time processes which can not be extended to continuous-time. For non-negative integer-valued margins, there are: Phatarfod and Mardia [1973], van Harn, Steutel and Vervaat [1981], McKenzie [1985, 1986, 1988], Al-Osh and Alzaid [1987], Al-Osh and Aly [1992], Alzaid and Al-Osh [1993], Aly and Bouzar [1994]. These marginal distributions include Poisson, negative binomial, generalized Poisson, etc. Some of the processes come from the birth- death processes, especially for the linear birth-death processes; one can even trace them to Kendall [1948, 1949]. Joe [1996] proposed a class of discrete-time stochastic processes with infinite divisible mar- gins, which include both count and positive-valued margins. These processes are first-order Markov processes. Some of them can be generalized to higher order Markov processes. Although the constructions of these processes differ from one another, there are three major approaches: constructing the birth-death process by the generating function method, constructing the process by specifying multivariate distributions for adjacent margins, and constructing the process by solving stochastic differential equations. Next we give a brief comments on these three approaches. The approach of constructing the birth-death process by the generating function method 4 was established by Kendall [1948, 1949]. It will yield a continuous-time stochastic process with state space being non-negative integers. This approach is still active in finding models for population processes in biological and cancer research. By sampling on equally-spaced time points, we can obtain the discrete-time processes. Two examples from the resulting linear birth-death processes with Poisson and negative binomial margins respectively, are often cited in the literature to model count data time series. However, the birth-death process approach can not yield the processes with state space being the real numbers. In the area of multivariate non-normal statistics, researchers (see Joe [1997]; Kotz, Bal- akrishnan and Johnson [2000]) have used copulas and other approaches to construct multivariate distributions with given univariate margins and desirable dependence structures. The theory ex- tends to construct discrete-time Markov processes with given non-normal margins by specifying appropriate multivariate distributions for adjacent margins. One famous example is the one de- fined by binomial thinning when the marginal distribution is discrete self-decomposable. However, some of these models, for example, random coefficient models, are quite isolated without a sys- tematic method. We can't extend most of them from discrete-time case to continuous-time case because of the consistency requirement for stochastic processes. Moving from the discrete-time to the continuous-time situation, we will experience the change from finite or countably infinite dimen- sions to uncountably infinite dimensions. This makes it harder to develop theory for continuous-time stochastic processes with given margins. The third approach is to define a type of stochastic differential equation, and find the so- lution which yields a continuous-time stochastic process. The obvious benefit is that it could provide a large family of Markov processes with desired margins. For example, Ornstein-Uhlenbeck and Ornstein-Uhlenbeck-type processes obtained from their corresponding SDE's lead to self- decomposable margins (see Section 7.1), known as the class L in Feller [1966b]. Since the theory of stochastic differential equations is dominated by the Ito integral which is involved in Brownian motion, the stochastic differential equations defined for processes with positive-valued margins was not developed until the early 1980s when the Ornstein-Uhlenbeck-type process evolved. Probably this is the first one appearing in that area. To our knowledge, we have not seen any stochastic 5 differential equation denned for processes with non-negative integer-valued margins. The reason could be that we don't know how to define such kind of stochastic differential equations and how to define their solutions. However, the counterpart of self-decomposable distribution was proposed a little bit earlier than the Ornstein-Uhlenbeck-type process, and it leads to the concept of discrete self-decomposable distribution (see definition in Section 7.1). This discrete self-decomposability property leads to continuous-time Markov processes with a special stochastic representation, which involves the binomial thinning operation. The linear birth-death process with Poisson margins discovered by Kendall [1948] is fortunately a concrete example in this family. 1.2 Highlights of our new research Our study is dedicated to developing continuous-time stochastic processes with count or positive- valued margins which can be used to model equally-spaced or unequally-spaced count or positive- valued time series. To achieve this, for reasons of simplicity, we focus on first-order Markov pro- cesses, rather than on more general classes. We take the dynamic view of building the continuous-time stochastic process with desired margins. Based on the infinitesimal analysis for the stochastic representations of the two linear birth-death processes with Poisson and negative binomial margins, we propose the stochastic dif- ferential equation for a continuous-time process with non-negative integer-valued margins. We introduce the concepts of a self-generalized distribution and the extended-thinning operation to define the stochastic differential equation: dX(t) = [K{l-ndt)®X(t)-X(t)]+de(t) = [{l-ndt)K®X(t)-X{t)]+de(t), (1.2.1) which we call the generalized Ornstein-Uhlenbeck equation. Here K(a) is a self-generalized rv with respect to parameter a, and "©" denotes the extended-thinning operation. The new stochastic integral in our theory is defined by convergence in distribution, rather than in L2 or probability. 6 The solution of the generalized Ornstein-Uhlenbeck equation has a simple stochastic representation, X{t2)= (e-^-'A ®X(h}+ t tl (e""*)K®de(t), (1.2.2) ^ ' K Jo with a dependent term (e -7^*2-*1))^. ®X(ti) and an innovation term / g 2 - * 1 (etit) K ®de(t), quite similar to the structure of first-order auto-regressive process. Hence, we call it the continuous- time generalized AR(1) process. One special case of extended-thinning operations is binomial thinning. In Section 1.1, we mentioned that the binomial thinning operation can lead to continuous- time processes with non-negative integer-valued margins. Such a class is included in the class of continuous-time generalized AR(1) processes. In this way, we can obtain the two linear birth-death processes with Poisson and negative binomial margins again. By the hint of a correspondence between self-decomposability and discrete self-decomposabi- lity, we obtain the positive real counterpart of the discrete self-generalized distribution. This leads to the positive real counterpart of the extended-thinning operation and stochastic differential equation, as well as the solution. Finally, we generalize the extended-thinning operation to the real case; the only known operator is the constant multiplier, and consequently the common AR(1) process obtains. In summary, we unite the cases of non-negative integer, positive-valued and real-valued state space by the self-generalized distribution and extended-thinning. The corresponding general- ized Ornstein-Uhlenbeck equation (1.2.1) leads to the continuous-time generalized AR(1) process (1.2.2). This type of Markov process has a simple stochastic representation, which provides an easy explanation when modelling, and a wide range of stationary infinitely divisible distributions such as Poisson, negative binomial, generalized Poisson, Gamma, exponential, inverse Gaussian etc, to cover diverse problems arising in various disciplines. In addition, the generalized Ornstein-Uhlenbeck equation can allow us to obtain continuous- time process which is not only stationary, but also non-stationary with time-varying parameters. For example, replacing only the constant parameter \x by a time-varying parameter /i(i) in (1.2.1), we obtain the stochastic differential equation X(t + h) = (l-u.{t)h)K®X(t) + Ae{t), (1.2.3) 7 which leads to non-stationary continuous-time generalized AR(1) process X(t2)= (e-ft*®*) ®X(h)+ f2 (e~ & ®de(t). (1.2.4) Such flexibility is very meaningful in developing models for non-negative integer, or positive, or real-valued time series with non-stationarity like trend, or seasonality, or covariate effects. In the context of the continuous-time generalized AR(1) process, the constant multiplier operation leads to self-decomposability and the binomial thinning operation leads to discrete self-decomposability. This is well known in the literature. Now the development of the theory of continuous-time generalized AR(1) processes certainly extends the existing concepts of self- decomposability and discrete self-decomposability to other operators: a self-generalized distribution with non-negative integer support leads to generalized discrete self-decomposability, while a self- generalized distribution with positive real support corresponds to generalized self-decomposability. These concepts of generalized self-decomposability and discrete self-decomposability help us to develop continuous-time generalized AR(1) processes with specific marginal distributions to fit practical needs. This work presents the theory of continuous-time generalized AR(1) processes in the order of model constructions, properties and applications. Now we highlight by chapter the new concepts and key results to help readers gain an overview of the theory of continuous-time generalized AR(1) processes. Chapter 2 defines the basic distribution families and independent increment processes for the subsequent theoretical developments. Some new distributions are discovered; these include four generalized convolution families: GC I, GC II, GC III and GC IV, which will be used in constructing independent increment processes. We propose the concepts of self-generalized distributions and extended-thinning operations in Chapter 3. These generalize the binomial thinning and constant multiplier operators for random variables with support on non-negative integers and positive reals. Besides a general theory, four new pairs of families of self-generalized distributions are discovered; there is a one-one mapping of operators with the two types of support. This theory has its origins from a careful study of the conditional probabilities of the linear birth-death process. The self-generalized operator in Example 8 3.2 is the operator associated with the linear birth-death process whose stationary distribution is negative binomial. In addition, the discovery of extended-thinning operations for positive-valued rv's enlarges our vision on obtaining a positive linear conditional expectation; we need not restrict ourselves on the commonly used constant multiplier operation to achieve this property. These operators also give us more choices in modelling the correlation between two positive random variables. Chapter 4 develops the generalized Ornstein-Uhlenbeck SDE's to include processes with support on non-negative integers, and construction of solutions of these equations (in the sense of convergence in distribution). The solution has a simple representation in terms of an extended thinning operator and an independent increment innovation process. These resulting processes are called continuous-time generalized AR(1) processes to emphase the similarity of their conditional expectation with that of the continuous-time Gaussian AR(1) process. Applying the theory in Chapter 4, we obtain interesting results from the generalized Ornstein- Uhlenbeck equations by choosing different extended-thinning operations and independent increment processes; their state spaces cover the non-negative integers, positive reals and reals. Both station- ary and non-stationary processes are considered. Many special cases are developed and studied in Chapter 5. In Chapter 6, we study the stationary distributions of the continuous-time generalized AR(1) processes. This study is to answer the question whether a specific distribution can be the marginal distribution of a continuous-time generalized AR(1) process. It guides us to choose proper pro- cesses with certain margins when modelling. Time series with diverse marginal distributions from the stationary continuous-time generalized AR(1) processes are also obtained. Key theorems are Theorems 6.1.1 and 6.3.1. The latter theorem is a result on the pgf or LT of the independent increment innovation process based on the pgf or LT of the stationary distribution, with a given extended-thinning operator. Many special cases are developed and studied. Chapter 7 further studies the stationary distributions under different extended-thinning operations. The generalized self-decomposable (GSD) and generalized discrete self-decomposable (GDSD) classes are defined in a similar way to the self-decomposable and discrete self-decomposable 9 classes associated with the constant multiple and binomial thinning operators. Several ways are developed to check if a given distribution is in one of the GSD or GDSD classes. Key theorems are Theorems 7.2.3, 7.2.5 (possibly simpler ways to check if a distribution is GSD or GDSD), and Theorem 7.2.7 (infinite divisibility of the classes). Relations between different GSD and GDSD classes are studied, as well as analog results between the cases of discrete and continuous margins. Chapter 8 investigates infinitesimal transition and duration features of the continuous-time generalized AR(1) processes. A PDE characterization is given for the conditional pgf or LT; a key result is that the pgf or LT of a self-generalized distribution is determined by its partial derivative evaluated at a boundary. For the continuous-time generalized AR(1) process with non-negative integer support, the infinitesimal generator matrix has the downwardly skip-free property. Another key result is that a steady-state continuous-time generalized AR(1) process can be determined based on two of the following three elements: marginal distribution, self-generalized distribution for the operator, increment of the innovation process. In Chapter 9, we present some differences for stochastic process constructions for the discrete-time and continuous-time situations. We also study conditional and multivariate distri- butions associated with some specific cases of the continuous-time generalized AR(1) process. A by-product is a new approach to construct families of multivariate distributions with given univari- ate margins. Interesting stochastic representations are given for some special processes and it is shown that some new discrete-time time series with gamma margins have better properties in the innovation random variable, compared with time series based on self-decomposability. We give a thorough study on parameter estimation methods in Chapter 10; these estimators including MLE, CLS, ECF etc, are desired in different situations and have their own advantages and disadvantages. Chapter 11 looks into the asymptotic properties of the commonly used estimates like M L E and CLS in the unequally-spaced setting. A random sampling scheme is proposed, and results build on the techniques of proof in Billingsley [1961a] and Klimko and Nelson [1978]. Chapter 12 discusses a variety of topics like detection of serial dependence, model diagnosis and selection, hypothesis testing, forecasting and process simulation. The graphical methods, called 10 the P-P plot and diagonal P-P plot, are proposed for assessing autocorrelation and model diagnosis. In Chapter 13, we illustrate the capability of the theory of continuous-time generalized AR(1) processes for real problems with three applications. These time series include non-negative integer and positive-valued observations. Finally in Chapter 14, we summarize the strengths and weakness of the continuous-time generalized AR(1) processes in modelling. Also we briefly discuss some thoughts on construction of stochastic processes. Areas for future research are also mentioned. 11 Part II Theory for model construction 12 Chapter 2 Relevant background on characteristic tools, distribution families and stochastic processes In this chapter, we cover background concepts needed in the development of the new theory of continuous-time generalized AR(1) processes. We try to select a minimum of necessary materials for the subsequent chapters. This chapter is organized in the following way: Section 2.1 briefly introduces the Ornstein- Uhlenbeck processes and Ornstein-Uhlenbeck-type processes; we will generalize these processes to a wider range, leading to the continuous-time generalized AR(1) processes. In Section 2.2, we discuss some characteristic tools for probability distributions and prove some new results. We present some particular distribution families in Section 2.3, and independent increment processes in Section 2.4. These results are used in subsequent chapters for special examples of generalized AR(1) processes. 13 2.1 Ornstein-Uhlenbeck processes and Ornstein-Uhlenbeck-type processes The Ornstein-Uhlenbeck process comes from the Ornstein-Uhlenbeck stochastic differential equa- tion (SDE), which has another name, the Langevin equation (see Ornstein and Uhlenbeck [1930], also Nelson [1967], 0ksendal [1995], Hsu and Park [1988], Schuss [1988]). Let {X(t); t > 0} be a continuous-time process. The Ornstein-Uhlenbeck equation is defined as dX(t) = -pX{t)dt + adW(t), n > 0, a > 0, where {W(t);t > 0} is a standard Brownian motion independent of X(0). The solution of this SDE is well known as X(<) = e-"*X(0) + ff / e-^-^dWir), Jo where J0* e _ / i T dW(r) is the Ito integral, which is the limit of a sequence of rv's in the sense of convergence in L2, and is normally distributed. Hence, the support of the margin X(t) is 5R. Furthermore, X(t) can be represented as X(t) = e - ^ - s ) X ( s ) + a f S e-^-^dWir), s < t. Jo Note that f*~s e~'1( t~fl~7"JdW(T) can be written as $1 e~^'^dW{T), and is independent of X(s) because X(s) is independent of {W(T);T > s}. This feature shows that the process is a Markov process, and a discrete-time AR(1) process can be readily obtained from it. If X(0) is normally distributed, then X(t) is normally distributed for all t > 0. This model serves continuous-time time series very well, and is named as continuous-time AR(1) (CAR(l)). See Brockwell and Davis [1996] and references therein. Because of normal margins under steady state, it is sometimes called a continuous-time AR(1) Gaussian process. The Ornstein-Uhlenbeck process has applications in mathematical finance (see Neftci [1996]). Wolfe [1982] initiated the study of Ornstein-Uhlenbeck-type processes. Almost at the same time, Sato and Yamazato [1982], Jurek and Vervaat [1983] studied this process too. The Ornstein- Uhlenbeck SDE is extended to dX{t) = -iiX{t)dt + dW{t), n>0, 14 where {W(t);t > 0} is a homogeneous Levy process independent of X(t). The solution has the same form as the Ornstein-Uhlenbeck SDE: X(t) = e-^-^Xis) + f S e-^-s-TUw(r), s < t. Jo but f0* e"*'( t" s~ T ' ( i iy(T) is not the Ito integral. This stochastic integral is the limit of a sequence of rv's in the sense of convergence in probability. The existence of such stochastic integral can be found in Lukacs [1968], where the characteristic function of the integral has the form exp S log < p w { l ) ( s e - " ( * - « - T ) ) d T j . Similarly, f^~s e'^-'-^dWir) {= /* e - " ( ' - T ' d i y ( T ) ) is independent of X{s). The support of W(t) can be positive real-valued. Hence, {X(t);t > 0} can be a positive-valued process. Sim- ilar to the Ornstein-Uhlenbeck process, a generalized time series (other than classical Gaussian distributed time series) can be easily obtained if sampling on equally-spaced time points. This fea- ture allows the Ornstein-Uhlenbeck-type processes to model positive-valued observed data. Wolfe [1982] showed two possible applications: the study of radioactive material in stockpile, and bank currency. Later Barndorff-Nielsen et al. [1993, 1998a] applied this kind of process with specific marginal distributions to turbulence and finance. Now we discuss some common features of Ornstein-Uhlenbeck and Ornstein-Uhlenbeck-type processes: • Nice stochastic representation form: the sum of two independent terms. One governs the dependence relation with the previous state, one explains the input (noise or innovation). Note that | e -M*-*)x(s) | < \X{s)\, hence, the term e'^'^Xis) looks "thinner" than X{s). • First order Markov: this Markov property is very helpful in the study of conditional properties, stationary distribution or margin under steady state, transition properties, and joint finite- dimensional distributions. • Simple auto-correlation: the auto-correlation function, if it exists, under steady state has the exponential form Cor [X(s), X(t)} = e - / t l*~ s L This implies that for a bigger time difference, there is less influence on the future state. 15 Although the Levy process t > 0} can have increments which are non-negative inte- ger valued, the term e~^t~s'lX(s) excludes the possibility of non-negative integer valued margins. Such a disadvantage precludes the application to count data time series. In the study of continuous-time generalized AR(1) process, we extend the stochastic oper- ation of a constant multiplier to an extended-thinning operation, and define generalized stochas- tic integrals. Such modifications allow us to obtain a similar representation for processes with non-negative integer state space. The continuous-time generalized AR(1) processes includes the Ornstein-Uhlenbeck process and Ornstein-Uhlenbeck-type process as special cases. 2.2 Characterization tools of distributions and examples In this section, we review the common characterization tools which are heavily used in the theory of continuous-time generalized AR(1) process. Note that this is a simplification of terminology; the processes are AR(l)-like with AR(1) autocorrelation, but not always autoregressive. These tools include the probability generating function (pgf), Laplace transformation (LT), moment generating function (mgf) and characteristic function (cf). This section consists of results that are used in subsequent chapters. It can be skimmed in the first reading. Proposition 2.2.2 is especially important. Any kind of generating function has the property that there is one-to-one mapping between the generating functions and the distributions. Hence, by investigating the generating function, we can know the corresponding distribution. In principle, the cf can be used in all types of random variables because it always exists. However, for specific types of random variables or distribution families, other generating functions may be more convenient. For example, the pgf is often used in non-negative integer-valued rv's, while the LT is adopted for positive real-valued rv's. This is for convenience of theorems for pgf's and LT's that can be applied. In exponential dispersion models, the mgf is used because the definition of that kind of model is related to the cumulant generating function. 16 We also prove some new results concerning pgf s and LT's. These will play certain roles in the theory of continuous-time generalized AR(1) processes. 2.2.1 Probability generating function The probability generating function is used for discrete distributions with non-negative integer support AV Assume X is a non-negative integer-valued rv with probability mass function Pr[X = i] = pi > 0, i = 0,1,2,... . The pgf of X is defined as oo Gx(s) = E{sx) = Y,Pis\ 0 < s < l . i=0 Usually the pgf is defined on [0,1] because the power series on the right hand side always exists when 0 < s < 1. This domain is sufficient for our need. Of course, it can be extended to \s\ < 1. As for |s| > 1, the finiteness of Gx{s) depends on the individual probability mass function. The function Gx(s) is increasing from po to 1 as s increases from 0 to 1. Once we have the pgf, we can obtain the probability masses: P i = Gx\o)/i\, i = 0,1,2,.. . . Also the mean and variance are derived as E ( X ) = G'x(l), Var (X) = E ( X 2 ) - (E(X)) 2 = G"x{\) + G'x(l) - {G'x(l))2. The index of dispersion, D, is defined as D(X) = Var ( X ) / E ( X ) , and is referred as an index of dispersion for distributions for count data. If D(X) > 1, there is overdispersion relative to Poisson. If D(X) < 1, there is underdispersion relative to Poisson. The following theorem characterizes the pgf; it is useful to verify if a function G(s) is a pgf. Refer to Bondesson [1992], p. 9. Theorem 2.2.1 Suppose G(s) is a Taylor series in s. Then, G(s) is a pgf iff 17 1. G(s) is absolutely monotone (AM), i.e., G^(s) >0,i € JVQ, S 6 [0,1). 2. G{s) —• 1, as s -»• 1. This is equivalent to checking G^(0) > 0 for all i and G(l) = 1. It is viewed as the discrete counterpart of Bernstein's theorem (Theorem 2.2.5). See Bondesson [1992], p. 9. Perhaps the simplest distribution is the Bernoulli distribution. It is often used to build other distributions, say Binomial, Poisson, etc. Let X ~ Bernoulli(p). X takes only two values, 0 and 1, with Pr[X = l]=p, Pv[X = 0] = 1 - p; and the pgf is Gx(s) = (1 — p) +ps. Consequently, the mean and variance are B(X)=p and Var(X) = p(l -p). Some distributions with non-negative integer support are listed below; these can be used in mod- elling count data. A l l of them are discussed to some extent in Bondesson [1992]. Also refer to Johnson and Kotz [1969]. (a) Poisson: X ~ Poisson(A). Then Pi = Pr[X = *] = — e~A; i = 0,1,2,...; A > 0. The pgf is Gx(s) = exp{A(s - 1)}, and E (X) = Var (X) = A. Thus, D(X) = 1, which is referred to as equidispersion. (b) generalized Poisson: Let X ~ GP(0,7?). Then P i = Pr[X = i] = 0(6 + rli)i-1e-e-rii/i\, i = 0,1,2,...; 6 > 0, 0 < n < 1. The pgf is Gx{s) = exp | e (^n(kr1)k-le-kr>sk/k\ - 1 The mean, variance and index of dispersion are B(X)=e(l-rj)-\ V a r ( X ) = f l ( l - » 7 ) - 3 , D(X) = (1 - r?)"2 > 1. 18 Note that if r\ = 0, it becomes Poisson(0). A good reference is Consul [1989]; there r\ can be negative to obtain an underdispersed distribution. However, this case has nothing to do with the study of continuous-time generalized AR(1) processes, because the marginal distribution of continuous-time generalized AR(1) process should have probability mass on the whole non- negative integer set, not on a bounded subset. When r? < 0, the generalized Poisson rv has an upper bound of support. ) negative binomial: Let X ~ NB(7 , q). [Note that this is a non-standard parametrization.] Then the pmf is P i = Fv[X = i}= ( 7 + * - 1 ) ( l - g ) V ; » = 0,1,2,...; 7 > 0 , 0 < g < l . The pgf is Gx(s) = ( Y T T ^ ) 7 = e x P - S ) " 1 and E ( X ) = 79/(1-<?), V a r ( X ) = <yq/(l -q)\ D{X) = 1/(1 - q) > 1. When 7 is an integer, the negative binomial distribution can be explained by Bernoulli trials with success probability 1 — q or failure probability q, in which the experiment stops until the 7-th successes, and X is the total number of trials in the experiment. The geometric distribution is the special case in the negative binomial family. It is obtained when 7 = 1 with pmf: P i = Pr[X = i] = (1 - <?)V; •* = 0,1,2,...; 7 > 0 , 0 < g < l , and pgf Gx{s) = jE^- Note that X can take value 0. But sometimes people treat X' = X + l as the geometric distribution which is positive integer-valued and has pgf Gx'(s) = Unless stated otherwise, we will take the former as geometric distribution throughout the thesis. log(l - qs) log(l - q) 19 (d) discrete stable: Steutel and van Harn [1979] proposed this discrete stable distribution. Let X be a rv from discrete stable distribution. Then the pgf is defined as Gx(s) = exp{-A(l - s)e} = exp{A[l - (1 - s)e ' - 1]}, A > 0, 0 < 9 < 1. The pmf can be obtained by expanding the pgf in a power series: n = P l = Afc-*, p , = (-1). g ' - f ^ ^ ^ , , - 2.3 However, since G'x(s) = Gx(s) • A0/(1 — the expectation will be infinite if 0 < 0 < 1. When # = 1, it becomes Poisson(A). (e) logarithmic series distribution: Let X be a rv from logarithm series distribution. Then the pmf is defined as ci+i P i = Pr[X = i] = ^—y0, 1 = 0,1,2,...; c = 1 - e'\ 6 > 0. The pgf is Gx(s) = - s - M o g l l - c S ) / 0 = 5 - 1 l o g ( l - c S ) / l o g ( l - c ) , and c0~^(l — c0~^) E(X) = c 0 - l / ( l - c ) - l s Var(X) = d T ^ l - c O / U - c ) 2 , D(X) = ( 1 _ c ) ( ^ - i + c 1 1 } - Note that this logarithm series distribution is left shifted to 0 compared to the usual defi- nition in Johnson and Kotz [1969], p. 166. Therefore, this logarithm series distribution is sometimes overdispersed, and sometimes underdispersed depending on the parameter 9. Let 9o be the solution of (i^^fl-i+c-i) = Then if 0 > 9Q, it has overdispersion; otherwise, underdispersion. (f) power series distribution: Let X be distributed in power series distribution. Then the pmf is i p0 = 0, Pl=Pr[X = i}=6H(k-9)/+ * = 1,2,...; 0 < 6 < 1. k=l 20 When 9 = 1, X degenerates to 0. The pgf is Gx(s) = s-1[l-(l-s)e]. Note that when 0 < 9 < 1, X has no moments. See Bondesson [1992], p. 128 and p. 132. This is also related to LTC (Laplace transform family C) in Joe [1997], p. 375; there is a left shift. Also the discrete stable distribution is compound Poisson with the distribution of X + 1. See the pgf expression in (d). (g) Zeta (discrete Pareto) distribution: Let X be a rv from Zeta(p). Then the pmf is defined as P i = c • (i + l)-( ' + 1 >, i = 0 , l , 2 , . . . ; p>0 , where c = Y^ili i~^p+1^- Note that this distribution comes from left shifting the Zeta dis- tribution in Johnson and Kotz [1969], p. 240; it is commonly called the Zipf-Estoup law in linguistic studies. Unfortunately, the pgf, expectation and variance of Zeta distribution have no explicit expres- sions. Stochastic operations can lead to new pgf's. Here we summarize some of the facts regarding operations on one rv. Proposition 2.2.2 (1) G(s) pgf =>• (1 — a) + ctG(s), 0 < a < 1, is a pgf [random zero-truncation operation]. (2) G(s) pgf G(as + 1 - a), 0 < a < 1, is a pgf [binomial-thinning operation]. (3) G{s;8) pgf for 3 £ B and F a distribution on B ^ G(s;B)dF(8) is a pgf [mixture oper- JB ation]. (4) G(s) pgf eAfG( s) _ 1] (A > 0) is a pg/[compound Poisson operation]. (5) G(s) pgf (1 - a) + asG(s), 0 < a < 1, is a pgf [zero-modification operation]. 21 Proof: Suppose X has pgf G(s) or G(s;B). (1) Consider Y = I • X, where 7 ~ Bernoulli(a). Then GY(s) = E ( s y ) = B(sLX) = Pr[7 = 0] + Pr[7 = l}E{sx) = (1 - a) + aG(s). (2) Let y = 0 w h e r e 7o = 0, 7i ' - ' Bernoulli(a), i = 1,2,.... Then (3) Suppose y conditioned on /3* = 8 has the same pgf G(s\ B),. and/3* is distributed in F on 73. (4) Let Y = Ezto where X 0 = 0, Xt i - ~ ' with pgf G(s) (t = 1,2,...), and Z ~ Poisson(A). (5) Consider Y = 7 • (X + 1), where 7 ~ Bernoulli^). Then GY(s) = E (a y ) = E (sr<x+V) = Pr[7 = 0] + Pr[7 = l]sE(s x) = (1 - a) + asG{s). Note that random zero-truncation operation is very similar to zero-modification operation. Both involve truncation. The random zero-truncation operation directly applies truncation to a rv X, while the zero-modification operation first shifts X to right as X + 1, then applies truncation. Hence, both primarily keep the shape of the pmf of X with slight differences. However, there does exist a difference between two operations. The random zero-truncation operation increases the probability mass at zero: Pr[7 • X = 0] = (1 - a) + a Pr[X = 0] = (1 - a) - (1 - a) Pr[X = 0] + Pr[X = 0] > Pr[X = 0]. Then Then 22 But the zero-modification operation relocates the probability mass at zero: Pr[I • {X + 1) = 0] = Pr[I = 0] = a, which could be bigger or smaller than Pr[X = 0]. The fraction of zeros in count data is one concern when modelling. Both random zero- truncated distribution and zero-modified distribution of X are alternative choices for data with zero fraction if the original distribution of X does fit the data very well. However, the zero- modified distribution of X is more flexible then the random zero-truncated distribution, because it can be used to either lower or higher zero fraction situation, while the random zero-truncated distribution can only applied in higher zero fraction situation (sometimes called zero-inflated). Example 2.1 Poisson(X) compounded with NB(l,q) will have pgf where A > 0 and 0 < q < 1. This is the basis of GC I introduced in Section 2.3.3. Another example of a compound Poisson distribution leads to the GC II in Section 2.3.3. We claim that is a pgf. This is because 1-73 l-q _ l - q n _ W 1 ^ _ ^ 2 0 2 ^ 3 C 3 1-7 1 - qs 1-7 (1 - 7s)(l + qs + q 2s2 + g 3s 3 + • • •) -[1 + qs + q2s2 + q3s3 + ••• 1-7 -7s - 79s 2 - 7<?2s3 - • • •] l-q , ( l - g ) ( g-7)„ , (1 ~ g)g(9 - 7) „2 , H s -\ : :* T 1-7 1-7 1-7 Let G(s) = 1 + ̂  (T=£ " l ) , ^ere d>j^>0. Then G(s) = 1 + 1 i-q , ( i - g ) ( g-7). , ( i - g ) g ( g-7 ) „2 , -f* S ~r z o ~r~ ' ' * , 1.-7 1-7 1-7 1-7 1 1 g , 7 + ( i - g ) ( g -7) „ 1 (1-9)9(9-7 ) 2 , ~ 1 _ d l - 7 + rf(l-7) S + d ( l - 7 ) 23 Since all coefficients of series expansion of G(s) are non-negative and G(l) = 1, G(s) is a pgf. Hence we can represent 2.2.1 as exp { " - ^ ~ ^ } - exp M l - T)|0(.) - 1]} . This implies that exp j ^^l^T 7 ^ j is the pgf of a compound Poisson distribution. Example 2.2 Following the zero-modification operation, we can show that is a pgf. This is because the following decomposition: Lis) = (X ~ ") + (a ~ ^ 5 ^ (1 - cry) - (1 - a)^s ' (1 -7 ) 0 ( l - 7 ) a _ ( l - 7 ) / ( l - c r y ) l - c r y 1 - (1 - a)7s/(l - «7) 1 — cry Here we know that 0 < {-]^~ < 1 and 0 < 1 - ^ = < i . Let I ~ Bernoulli ) , Z ~ NB ( l , (|l^ 7). TTien 7(Z + 1) /ias i/ie L(s) as its pc//. Mien a = 0, X = 0, iwMe a = I, X = Z + 1. We are noi c/ear z/ SUC/J a distribution has been previously studied. Since Z is Geometric, we call this distribution the zero-modified geometric distribution. Sometimes operations are carried out on more than one random variable. The well known convolution, which is the sum of independent rv's, is an example. For two independent non- negative integer valued rv's X\ and X2, the pgf of convolution X\ + X2 is GXl+X2{s) =E(sx^x>) = E ( S X l ) E ( S * 2 ) = GXl(s)GX2(s). For more than two, say n independent rv's, we have G E ? = i X i (s) = E = f[ E = f[ GXi (s). i=l i=l Furthermore, the convolution concept can be naturally extended to the situation of uncountably many rv's, leading to the generalized convolution. See Section 2.3.3. Next we prove that some functions are probability generating functions; these will be used in the study of continuous-time generalized AR(1) processes in later chapters. 24 Theorem 2.2.3 The following functions L(s) are pgf's. (V Hs) = l-00?> o<e<i. (2) L(s) = c - ^ l - e - ^ - ^ l - cs)Q], 0 < a < 1, c = 1 - e~\ 0 > 0, (3) L(s) = l - a e ( l - 7 ) e [ ( l - a ) 7 + ( l - 7 ) ( l - s ) " 1 / e ] ~ 9 ' 0 < a < 1, 0 < 7 < 1, 0 > 1. (4) L(s) = {s- + 1, where G(s) is the pgf of Zeta(p), p > 0. (5) L(s) = l+ ^ ^ ^ y 0 < a < 1, 0 < B < 1, ° a ) 3 / 2 < / 3 < « , ^ a < / 3 < ^ = 1 + , t  (1"71)(V-1I \, 0 < a < l , 0 < / 3 < l , 0 < 7 < 1 , and either a </? < i ^ ™ , or max (7 , < /? < a and B(B - 7)(1 - a) 3 > ( a - 0 + 7-07)1/*-0) Proof: It is obvious that for all cases, L(l) = 1. Suppose L(s) has series expansion of form 00 L(s) = r0 + ns + r2s2 H = ^ r,s\ i=0 It suffices to show that all coefficients rj > 0 (i = 0,1,2,.. .)• (1) Rewrite L(s) as Since L ( s ) _ s 1 - ( 1 - S ) 0 _ S (1 (1 _ s)-« = 1 + # s + v 2 j V + — ^ V + • • • = 1 + es + ^ j{ QIlizKk + e) j we have j=2 J " 1 - 00 r-n 1 + r D k M ^ - I N Assume the Taylor expansion: 1 , y n i = 1 ( H g ) = 1 - qis - q2S2 - Q3s 3 - - 25 Then L(s) = s~l (1 - [1 - qis - q2S2 - g 3s 3 - c /4« 4 - • • • ] )= gi + gas + q%s2 + g 4s 3 + We now need to prove that q3 > 0 for j = 1,2,.... Because x , ^ n i = i ( f c + g ) , , - ft (i + i ) ' 1 - c/is - g 2s 2 - 93« 3 - c/4S 4 it follows that 9i 92 1 + 9 2 ' (l + 0)(2 + 0) _ l+_0 3! 2 9i, 9j = ni = i ( fc+g) y n i = i ( f c + g ) m (i + i)! Note that f < if 0 < a < b. For j > 2, n u ( f c + g ) y i i a ( f c + f l ) - i+e j-i+e ni=i(fc+g) j - 1 + f l y m ; ' r x ( f c + A . 1+e 7 - 1 + 0 1+0 /j -1+6 l + 0 \ — — 9 , - 1 - — * - i = ( — ^ J 9 , - L > Due to the fact that c/i = ^ > 0, by induction, we obtain qj > 0 (j > 2), which means that L(s) is a pgf. (2) Note that for 0 < a < 1, (1 - cs)a = 1 - acs - ^~^-c2s2 - Q ( 1 ~ Q 3 ) , ( 2 " ^ C 3 S 3 . . It is straightforward to show that all coefficients of series expansion of L(s) are non-negative, which shows that L(s) is a pgf. 26 (3) Let rj = L^{0)/j\. It suffices to show that L^(0) > 0 for j = 0,1,2,.... Check it when j = 0. e L(0) = 1 - a°(l - j)6 [(1 - ah + (1 - T)]-* = 1 " ( a ( 1 ~ 7 ) > \ > 0. 1 — ory Now consider the derivatives. L'(s) = - a 0 ( l - 7 ) e ( - ^ ) f ( l - « ) 7 + ( l - 7 ) ( l - 5 ) _ 1 / 1 ^ f l - ^ W " ^ 1 = a 9 ( l - 7 ) 9 + 1 ( l - a ) 7 + ( l - 7 ) ( l - s ) " -i/e (1 - s ) = a 1 + 0 ^ ( 1 1 - 7 -(«+i) L"(s) 1 - 7 "(0+2) 0 1 - 7 When 0 < s < 1, it follows that L'(s) > 0 and L"{s) > 0. Starting from j = 3, higher order derivatives are non-negative linear combination of products of form (1 - s)1'6-1 i + i l z^2( i - a ) i / » 1 - 7 , f > 1, fc > 2. (2.2.2) Since for 0 < 5 < 1 d ds _d ds ( 1 - , ) ! / « - ' = ^ f l ( l - f l ) V M ' + D > o , 1 + 0 - a h ( 1 _ f l ) i / , 1 - 7 _ 0 + _ (1 - a ) 7 ^ _ s ) i / e - i 0 1 - 7 1 + 1 - 7 >0, (2.2.2) follows by induction, and we can conclude that all higher order derivatives are positive when 0 < s < 1. This leads to that L^(0) > 0.(j > 1). Hence, L(s) is a pgf. (4) Since the pmf of Zeta(p) is oo p i = c-( i + l ) - ( p + 1 ) , i - 0,1,2,...; p>0, c = J2 the pgf of Zeta(p) is oo oo G(i) = x)w«i = E c - ( » + 1 ) " ( p + 1 ) s * - i=l i=0 i=0 27 Because L(s)G(s) = (s - l)G'(a) + G(a), and L(s)G(s) = p 0 r 0 + (p0ri + pir0)s + (p 0r 2 + P\n + p 2 r 0 )s 2 H h I ̂ p ^ i - k ) s% H , \k=o ) (s-l)G'{s) + G(s) = (5 - 1) • [pi + 2p2s + 3p 3s 2 + •••] + [po + pis + p 2 s 2 + P3S 3 + • • •] = (po - Pi) + 2(pi -p 2 ) s + 3(p2 - p 3 ) s 2 + ••• + (» + l)(pi -pi+i)si + •••, we obtain pon> = po - p i , Pon +pi r 0 = 2(pi - p 2 ) , Por2 + p i n +p 2 r 0 = 3(p2 - p 3 ) , < El;=0P* rt-* = (* + l)(Pt -Pi+l) , or more specifically, r 0 = l - 2 - ( " + 1 ) , n + 2-("+1)r0 = 2[2-("+1) - < : n + 2-(P+1)ri_1 + . . . + (» + l)-("+i) r o = (i + i)[(i + i)-(P+i) _ (j + 2)-("+ 1)], Thus r 0 = 1 - 2~^ + 1) > 0, r i = 2 [ 2 _ ( p + 1 ) - 3-^ + 1)] - 2 ~ ( p + 1 ) [ l - 2~ ( p + 1 ) ] = 2~ ( / , + 1 ) + 4~^ + 1) - 2 • 3-^ + 1 ) > 2\/2~iP+l)^P+1) - 2 • 3 ~ ( p + 1 ) = 2[\ /8-^ + 1 ) - v V ^ + 1 ) ] >.0. Assume rj > 0 (i > 2). We show that r j + i > 0. To prove it, we apply the contradiction method, which supposes r j + i < 0. Note that 5 < § < • • • < j+i- This leads to (fc + l ) - ^ 1 ) < fc-^"1) ^ i ± i j , fe = l > 2 , . . . J i > 28 Consider the new equation i + 1 i + 2 (P+1) r n + 2 - ( " + 1 V i _ i + . . . + (» + l ) - ^ + 1 V 0 ' r i + 1 + 2 ' ^ r i + --- + (i + 2 ) - ^ r 0 1 1 (2.2.3) i + l\{p+1) The left hand side is LHS = -n+i + (i + (i + 2)P+ 1 i + l\(p+1) (i + 2) + • • • + + 2 -(p+i) rt + 2~(P+!) (i + 2)P+ 1 (i + 3)P+ L\' i + l\{p+1) i + V _ 3-(p+i) I + 1 N(P+D i + 2 + (̂  + 1) -(P+1) i + l \ ( p + D i + 2 (i + l ) - ( p + 1 ) | n - ( i + 2 ) "^ + 1 ) | r 0 > -ri+i > 0. Denote the right hand side of (2.2.3) as a function of p: 1 (i + l)p+2 Then h'(p) = RHS = h(p) = - \og(i + 2) 1 i + 2 + i + 2 (i + 2)P+1 (i + 2)2"+2 (i + 3)P+ L' (i + 2)2\ (i + l)P+ 2 log(t + 3)- + log i + 2 (i + 2)P i + 3 ~ 6 V * ' " ; (i + 3)P 1 ~ & V * + 1 J (i + 2)2P+2 ' As p increases, (j^Y decreases to 0, thus, ' ° g ^ 2 2 ) JI+Ty ~ f+§ l o § ( * + 3){7+3p" w i l 1 eventually be positive. This means h'(p) will eventually be positive when p increases, although it could be negative at the beginning. Hence, h(p) could be either always increasing, or decreasing first and then increasing. Thus h(p) < max(/i(0), h(oo)). By calculation, we have M0) = - 5 T W T W < 0 , M~) = ,£»*(,) = <>. Thus h(p) < max(/i(0),/i(oo)) = h(oo) = 0. This contradicts to that LHS = RHS, which implies that rj+i > 0. Therefore, L(s) is a pgf. 29 (5) Rewrite and expand L{s) = 1 + = 1 + s - 1 (1 - M ( l - 1 - a I - a 1 1 = 1 + x x a l-Bs a a 1 a 1 + 8s + 82s2 + 83s3 + ••• 1 - a I - a 8 — a\ Q 7 I s 3 + 1 — a 1 a 1 s + - a (1 - a)82 - 8 - a 1 - a 1 + - a 1 oo S 3 + -< (1 - a)/3 1 - a Now we verify if all coefficients are non-negative, i.e. 'B-a "' (1 - a)Bl - This is equivalent to 1 - a a >0, * = 1,2,3,.... ( l - a r + 1 > ( 1 - ^ ) > * = 1,2,3 Ua<8, then 1 - f > 0. Thus, (2.2.4) holds iff 1 - ^ < (1 - <*) ( i + 1 ) / i, * = 1,2,3,... iff 1 - - < mm 1 B - JGAT r a 8 ~ ieM In this situation, we obtain the range of 8: (I-a)', 8< — a (1-cr) 2 2-a' a<8< 1 (2.2.4) 2 - a If a > 8, 1 - | < 0. We only need to consider i being even integers. (2.2.4) can be rewritten as 2j ( l - a ) 2 J + 1 * ( i - l ) 3 ' j ' = 1 , 2 , 3 30 This holds if 8 ~ jeM Hence, the range of B in this situation is 2 - 1< min( ( l - a)(^yM\ = (1 - a) 3 / 2 . a <B<a. 1 + (1 - a) 3 / 2 These imply that for 8 in above ranges, the function L(s) is a pgf. (6) We rewrite and expand ( l - 7 s ) ( s - l ) L(s) = 1 + (1 - M ( l - = l + ( l - 7 S ) l 1 - a 1 1 x x 1 - a 1 — 7s 1 1 — 7s = 1 + X '- X a 1 - 8s a 1 - £zH< l - a / 7 /? - 7 1 = 1 + l^ + ^-r1 X a 8 B I-8s 1 / ( l - a ) 7 | Q 8 - a ) - ( l - a ) 7 y a \ 6 - a 8 - a " a/3 a(/3 - a) a/3 1 - Ba 8 — 7 — a + cry 1 a(/3 - a) = 1 + (1 - « ) 7 _ (1 ~ a)7 + (1 - - 7) x h + ^ + ^ 2 + ^ 3 + ,. .j a/3 a(/3 - a) a/3 /J — 7 — a + a 7 a(/3 — a) 1 - a 1 - a 1 - a = i V [ ( l - a ) ( | 3 - 7 ) ^ - r - (8 - 7 - a + 07)03 - a ^ l - a)''} sl. a f-r' We want (1 - a)(/3 - a ) ^ " 1 - (/3 - 7 - a + a7)(/3 - a)1'1^ - a)" 2 > 0, for i > 1. 31 The inequality can be written as (8 - 7)(1 - a)2 > (B - 7 - a + cry) (Y^) , for t > 1. When a < 8, 0 < < 1. In this case, it simply results in (8—7)(l-a)2 > (/?—7—a+cry). Further calculations lead to a < 8 and 1 + 7 - cry - 8(2 - a) > 0, i.e., a < 8 < 1 ± ™ . When a > 8, < 0. We obtain ( / ? - 7 ) ( l - a ) 2 > ( / 3 - 7 - « + « 7 ) ( - i r 1 1 = ( a - / 3 + 7 - a 7 ) ( - l ) i ( a ( ^ ~ ^ > f o r « > l . Note that a — 8 + •y — cry > 0. First, /3 can't be smaller than 7. Otherwise, the left hand side is always negative while the right hand side alternates in sign, which is a contradiction. Thus, 8 > 7, and we only need to consider those situations where the right hand side is non- negative. Secondly, if a{^_~1 > 1, i.e., 8 < the right hand side can go to infinity. This is impossible because the left hand side is finite. Hence, it must be 5 ^ < P, and consequently, 0 < "(f" 1 < 1. Under such a situation, we can simplify the inequality to be ( / i - 7 ) ( l - a ) 2 > ( 0 - / 3 + 7 - 0 7 ) x 1 — a or equivalently, 8(8 — 7)(1 — a) 3 > (a - 8 + 7 — 07) (a — In summary, we obtain two groups of conditions: (1) a < 8 < (1 + 7 — cry)/(2 — a); (2) max(7, a/(2 - a)) < 8 < a and 8(8 - j)(l - a ) 3 > (a - 8 + 7 - cry)(a - /?). Under these conditions, the function L(s) is a pgf. Remark: More on the compounding operation. The compound Poisson rv can be represented as random summation: Y Z = ^ X ; , Xo = 0, Xi iid, Y is distributed in Poisson. (2.2.5) i=0 32 Here Xi can be not only a non-negative integer-valued rv, but also positive-valued or real-valued rv. The random variable Y can be extended to any other non-negative integer-valued rv. For example, if Y is a Bernoulli rv, the compound operation is the random zero-truncation operation, it is often used in Zero-inflated models for economic applications. See Winkelmann [1997], p. 107-108, and references therein. Now we briefly study a few properties of this extended compounding operation such as the mean, variance and probability mass at zero. E fe*^ = E fE (E X*| y)) = E ( X 1 ) E ( F ) , Y E(Z) Var(Z) vi=0 / \ \i=0 Y \ / / Y s.i=0 D(Z) = V a r {poXi)=E [V a r [ E x « - | y J J + V a r (̂ E ( E x * | y E(Var(X!)y) + Var(E(X!)y) Var(X!)E(y) + (E (Xi)) 2 Var(Y), Var(Z) _ Var(Xi)E(y) + (E [X1))2 Var(y) E(Z) Var(Xi) + E ( X i ) E(Xi)E(y) Var(y) Pr[Z = 0] = Pr E ( X i ,i=0 oo E(Y) ' = Pr[Y = 0] + Pr[Xi = 0] Pr[Y = 1] + (Pr[Xi = 0])2 Pr[Y = 2] + --- + (Pr[Xi = 0 ] ) ^ ^ = !] + ••• £ ( P r [ X 1 = 0]) iPrrr = i] = Gy(Pr[Xi = 0]), where GY(s) is the pgf of Y. If X\ is also a non-negative integer-valued rv, we have Gz(s) = E {sz) = E ( a£r=o*) = E (E ( s £. r =o*|y)) = E ( [G X l ( S ) ] y ) = GY(GXl(s)), where Gxt {s), Gy{s) are the pgf of X\ and Y. With these results, we have the following proposition. Proposition 2.2.4 Suppose X\ is a non-negative integer-valued rv, and (2.2.5) holds. (1) J / E ( X i ) is positive and Var(Xi) exists, then D(Z) > D(X{), 33 (2) If Pv[Z = 0] > 1 - Pr[Y = 1], then Pv[Z = 0] > Pr[Xi = 0]. Proof: Apply the above results. (1) Since B(Xi) > 0 and E(Y) > 0, we have _ V a r ^ ) Var(y) Var (X x ) _ This means that after compounding, the index of dispersion becomes larger. (2) Because GV(Pr[A"i = 0]) = £ ^ 0 ( P r [ ^ i = 0]) JPr[y = i] > Pr[Y = 0]+Pr[Xi = 0]Pr[y = 1], it follows that . PT[Z = 0] > Pr[Y = 0] + Pr[Xi = 0] Pr[y = 1] = Pr[y = 0 ] + P r [ X i = 0 ] ( l - P r [ Y = 0]) = Pr[y = 0](1 - Pr[Xi = 0]) + Pr[A"i = 0] > Pr[Xi=0], where the equality holds only when Pr[Y = 1] = 1 or Pr[Xi =0] = 1, which are extreme cases. Hence, the compound operation results in a larger mass at zero in general. The compound Poisson and random zero-truncation operations result in a larger probability mass at zero. Such a property of a larger probability mass at zero makes the compound distribution an alternative candidate in modelling count data with a higher fraction of zeros. 2.2.2 Laplace transformation, moment generating function and characteristic function The Laplace transformation (LT), moment generating function (mgf) and characteristic function (cf) are widely used in probability and statistics. They are defined as below. Definition 2.1 Let X be a rv. Then 34 (1) the LT of X is 4>x{s) = E ( e - s X ) , s e Si, where Sx = {s : E(e-sX) < oo}. (2) the mgf of X is Mx{s) = E ( e s X ) , s E S2, where S2 = {s : B{esX) < oo}. (3) the cf of X is ipx(s) =V(eisX), se (-00,00). The relationships among the pgf, LT, mgf and cf, over approriate domains, are listed below: Gx(s) = fo-(-loga) = Mx{logs) = (px(-ilogs), <f>x{s) = Gx(e~s) = Mx(-s) = tpx(is), Mx(s) = Gx{es) = <j>x(-s) = <px(-is), <Px(s) = Gx{eis) = Mx(is) = 4>x{-is). For positive-valued rv X, the LT is more convenient than mgf, because its convergence domain of s includes 5Ro = [0, 00), a fixed set, while the domain of s for the mgf depends on the individual distribution. The LT is decreasing while mgf is increasing. The mean and variance can be derived from both LT and mgf. ' ^ ( O ) - ( ^ ( O ) ) 2 , E ( X ) = M'x(0), and Var (X) = E (X 2 ) - (E (X)) 2 = { [ - « ^ ( o ) . Note that the probability mass at zero can be obtained as M'x(0) - (M'X(Q))2, { - ^ ( o ) + ( ^ ( o ) ) 2 . Pr[X = 0] = lim </>x(s) = <j>x(oo) = lim Mx(s) = M(-oo). s—>oo s—> —OO Example 2.3 Gamma distribution: Let X be the rv of Gamma (a, 6), where a is the shape param- eter and B is the rate parameter. Then the pdf is fx(x;a,P) = -^-xa-le-^, r(a) x > 0; a, B > 0. 35 The LT is the mgf is and the cf is <Px(s) = (^g f i g ) > a € ( - 0 0 , 0 0 ) . The expectation and variance are E ( X ) = ad~l and Var (X) = a3~2. The Gamma family contains a couple of special distribution cases: • when a = 1, it is Exponential(/3), • when a = k/2, 3 = 1/2 (k is an integer), it is Chi squared, x | . More examples of LT and mgf can be seen in the successive subsections. Because, the theory of continuous-time generalized AR(1) process heavily involves non- negative rv's, we focus on the LT in the rest of this subsection. The following theorem is important to characterize the LT of a non-negative rv. Refer to Bondesson [1992], p. 8-9. Theorem 2.2.5 (Bernstein's theorem) 0(5) is a LT iff 1. 4>(s) is completely monotone ( C M ) , i.e., (-l)l(p^(s) >0,ie A/"o, s 6 (0,00). 2. <p{s) —> 1, as s -> 0. Following two theorems are very useful to identify new LT. Very nice proofs for the first two can be found in Joe [1997], p. 374. Theorem 2.2.6 Let cf>(s) be a LT. Then </>a(s) is a LT for all a > 0 if and only if -log0(s) is an infinitely differentiable increasing function of [0, 00) onto [0,oo)7 with alternating signs for the derivatives. 36 Theorem 2.2.7 Suppose <f>i(s) and (j>2{s) are LT's. / / - log is an infinitely differentiable in- creasing function of[0, oo) onto [0, oo), with alternating signs for the derivatives, then log</>i(s)) is a LT. An explanation of <j)2{— log </>i(s)) is discussed later in Section 3.4; refer to„Bondesson [1992], p. 17. Note that for a LT <f>(s), 4>a(s) being a LT for all a > 0 means that <j>(s) is the LT of an infinitely divisible distribution. See Section 2.3.1. By Theorem 2.2.6, we can prove that the exponential function <p(s) = exp{</>o(s)} is a LT if Ms) satisfies that <f>o(s) < 0, (-iy<$\s) > 0, i = 1,2,3,..., s € (0,oo), and </>0(s) ->• 0 as s —>• 0. In some exponential form situations, the conditions of Theorem 2.2.7 are easier to be verified than Theorem 2.2.5. Applying these theorems, we can obtain the following results, which are used in the theory of continuous-time generalized AR(1) processes. Theorem 2.2.8 The following functions </>(s) are LT of distribution with support on [0, oo). (1) </>{s) = exp{-A- " ( 1 ^ 7 S ) | , where A,/3>0, 0 < 7 < 1 and 7 < j ^ . (2) ^ ) = e x p { ^ l o 6 ( g E ^ ) } = ( g E ^ ) ^ , ^ e r e « > 0 , 0 < 7 < l . (3) <f>{s) = exp j - ^ 1 + ( e ^ ^ ~*} , where 0 < a < I, Q > 0. (4) ^ ) = exp{- ( 14 (^a ) 7 S}, ^ e r e 0 < a < l , 0 < 7 < l . (5) fa) = exp | - [(1_a)7%"-V̂ ]g} ' « ^ f i e O < a < l , 0 > l a n d 0 < 7 < l . (6) <(>(s) = exp j A • (^^1/2 } , where A, 8 > 0. Proof: (1) When s = 0, (f>(s) — e° = 1. Hence, it suffices to show the complete monotonicity of function (j)(s). Since 7 < ytĵ j, we know that 1 - 7(1 + 8) > 0. Taking the first order derivative, we 37 obtain (1 - 7 + 21S)(B + s) - s(l - 7 + is) / 3 (1 -7 + 7*) 1 *(*) = - A : e x p j - A - B + s j /?(1-7)+ 2/37s + 7s2 / 3 (1 -7 + 73)1 —exprA' ^ + 3 / = - A - (P + s) f s ( l - 7 + 7s)l -\B[l - 7(1 + B)]{B + 3 ) " 2 exp { - A • ̂ - T + T ^ j < 0. By induction, the higher derivatives (f>^(s) (i > 2) are the sum of terms of form (omitting the coefficients) ( - i r (^ + 3 ) - f c e x p | - A - S ( 1 ~ ^ 7 ' ) } , k>0. (2.2.6) This follows because derivatives of (2.2.6) lead to two terms having the same form (ignoring coefficients that don't depend on s). With this property, we can conclude that the derivatives alternate the signs. By Theorem 2.2.5, 4>(s) is a LT. ) First we prove that (0 < a < b) is a LT. This is because 1 + as a t a\ 1 _ a t a\ 1/6 1 + bs = b + \ ~b)l + bs = b + \ ~ b) 1/6 +a ' the LT of the zero truncation of the Exponential(l/6) distribution. Secondly, we show that ~ logy+ff is a r i infinitely differentiable increasing function of [0, oo) onto [0, oo), with alter- nating signs for the derivatives. Since is decreasing, — log is increasing, and - log \ + a S = log(l. + 6s) - l o g ( l + as), 1 + OS 1 + a s \ ' 6 a 6 — a log l + 6s/ l + 6s 1 + as ( l + as)(l + 6s) 1 + O 3 \ ( 0 _ ( - l ) * - 1 ^ (-!)*" V = , (b + absf - (a + a6s)' g l + 6sJ ~~ (1 + 63)* (1 + as)' [ ' ( l + as)*(l + 6s)i ' 38 Because ^ i + ^ i W ^ > 0 for all i > 1, we conclude that the derivatives of - log ± ± f f take alternating signs. Hence, by Theorem 2.2.6, ( i ^ f f ) (c > 0) is a LT. Rewrite y v ; 1 (1 - 7 ) ( « + s) x 1 - 7 1+, " 7 , S \ 1 - 7 - 7 " ( l - 7 ) u 1 1-7 1 I ! - 7 . i + ( l - 7 ) " ' (1-7)" (1 |0" if 1 — 7 — 7 U > 0, if 1 — 7 — 7U < 0. When either I - 7 - 7 U > 0 or I - 7 - 7 U < 0, cj>(s) has the form of ( p ^ f f ) (0 < a < b, c> 0). Therefore, 4>(s) is a LT. It is clear that <f>(s) —> 1 when s ->• 0. Hence, it suffices to show the complete monotonicity of function 4>(s). Taking the first and second order derivatives, we obtain 4>'(s) exp [ i + ( e e - i ) * r - i i v e9 - 1 —a l + (e e -l)s a - i j [l + {e e - l)s]a - 1 exp< - 0"(s) = I - a [ l + [ee - l)s Q - l = ( a ) ( l - a ) l + ( e e - l ) s e* - 1 J ' [ l + ( e ^ - l H a - l l V a - 2 f [ l + ( e « - l ) S ] a - l exp exp < - l + (ee- l)a 2a-2 exp e8 - 1 [ l + ( e * - l ) g ] Q - l ee - 1 Obviously, </>'{s) has negative sign while 0"(s) has positive sign. By induction, the higher order derivatives are the sum of terms of form (omitting the coefficients) [1 + (eP-DsT-l] 1 + (ee - l)s exp < - - ee - 1 1 < m < n. Such a term has a derivative with negative sign, just like (j)"(s) changes the sign of (j)'(s). Hence, the derivatives ^(s) alternate in sign. This shows the C M property of <f>(s). By Theorem 2.2.5, we conclude that (f>(s) is a LT. 39 (4) Rewrite i, \ f a ( l - 7)« ] * " - a p - ( i - T ) H i - . ) T . r B p "(1 - 7) ( l - 7 ) / [ ( l - g ) 7 ] ( l _ 7 ) / [ ( l _ a ) 7 ] + a (1 - a ) 7 Note that (1 (l;7/)[/([1(l;"jj7j.s is the LT of Exponential(1 - 7 ) / [ ( l - a) 7]). Hence, 0(s) is the LT of compound Poisson with the Exponential (1 - 7 ) / [ ( l - 0)7]) distribution. (5) Rewrite 4(s) exp exp a ( l -7) L ( l - a ) 7 + ( l - 7 ) s - 1 / 0 " ( 1-7) exp (1 - 0)7 " ( l - 7 ) l e / . ( l - a ) 7 J \ Let ^ ) = 1-((T̂ )̂  [TJE$ ^o(s) = 1 - 1 1-7 - I (1 - a ) 7 1-7 (1 - 0)7 (1 — 0)7 _ i ( 1-7) - 1 0, + 5 « 1-7 We prove that </>o(s) is a LT. First, when 1. (1 - a ) 7 , Hence, it suffices to show the C M property of <j>o(s). Now check the first and second order derivative of </>0(s). Denote C = (j^E^) • W e obtain <j>'0(s) = -C-(-0) = -C (1 — 0)7 _ i -(6+1) 1 —- s ^ - 3 - 1 (1 — 0)7 I 0 + 1 (1 - a ) 7 0 ( 1-7) (1-7) J U 1 - 7 ) e ) = c (1 - crW 1 -(0+2) -1 (1-7) Note that 0'o'(s) alternates the sign of $,(s). By induction, the higher order derivatives are the sum of terms of form (omitting the coefficients) (1 - 0)7 1 L. LLse + 1 (1-7) -(0+fc) m/6—n k > 2, 1 < m < n. 40 Since (l-<*h A16 , i -(e+fc) and smle~n have derivatives with negative sign, it is straightfor- ward to show that higher order derivatives <^ (s) (i > 3) alternate in sign. Lastly, we check that if 4>o(s) > 0 for s € (0, oo). Since <j>o(s) is decreasing, it follows that Ms) > M°o) = 1 - C{C~l + 0) = 1 - 1 = 0. Thus, <f>o(s) is non-negative. This completes the proof that 4>o(s) is a LT. Therefore, (f)(s) is the LT of the compound Poisson with the distribution characterized by the LT (po(s). (6) The proof is similar to that of (3). Rewrite ^ ) = e x p { A . ( i + - S s ) 1 / 2 } = e x p { Then, the first order derivative is A r 8 ( l + ^ ) - l / 2 _ ( l + ^ ) l / 2 ] j (l + 6s)-^2 + (l + 3s)-^2 \-V2 - (1 + Bs) —s 1/2 exp < A (1 + Bs) 1/2} < 0. By induction, the higher order derivatives ^W(s) (i > 2) are the sum of terms of form (omitting the coefficients) (1 + ^ / 2 e x p { A ' ( i w ) ' + Bs)1/2 By the same reasoning in (3), we know that 4>(s) is a LT. The LT's in (1) and (2) of Theorem 2.2.8 lead to GC IV and GC III in Section 2.3.3. This distribution corresponding to (2) has non-zero probability mass at zero, and the mass is: 1 - 7 The LT's in (3), (4) and (5) of Theorem 2.2.8 will be adopted as positive self-generalized distribu- tions denoted as P4, P2 and P5 in Section 3.1.2. The LT in (3) belongs to Tweedie exponential 41 dispersion family. See Section 2.3.2. Comparing with the LT of Tw^in, o2) there, we find it is the LT with specific parameter d = ̂ —4>2, fi = l/a>0, a2 = (1 - a)(ee - l)/a^ > 0. a — 1 The pdf is given in Section 2.3.2. This distribution does not have probability mass at zero, because 0(oo) = 0. However, the LT's in (4) and (5) are not in the Tweedie exponential dispersion family. This can be verified by comparing their LT forms. But they have non-zero (positive) probability masses at zero, which are respectively. The LT in (6) will serve as the innovation of a stationary continuous-time generalized AR(1) process with inverse Gaussian margins (see Section 6.3). As to their explicit pdf forms, we are not clear at this moment. Perhaps the most enjoyable and popular distribution is the Normal distribution. It has a lot of good properties, such as bell-shaped density and limiting distribution of an average of rv's. To enlarge and modify this family of distributions, the variance mixture of normal distributions was introduced: X = VYZ, Z ~ N(0,1) and Y > 0 is a rv independent of Z. Since X is a real rv, we prefer to calculate its cf. Let 4>Y{S) be the LT of Y. Because <pz(s) = e -" 2/ 2 , we have <px(s) = E (e"x) = E (eis^z) = E ( E (j'^Y)) = E (e^ 2 ) = <M*2/2). See Bondesson [1992], p. 115. An equivalent version is the scale mixture of normal distributions, which is defined as X = Z/Y, Z ~ N(0,1) and Y > 0 is a rv independent of Z. However, its cf can not be explicitly expressed via (f>y(s): <px(s) = E {elsX) = E (eisZlY) = E ( E (eisZ'Y\Y)) = E ( e ^ ' ^ ) . 42 Refer to Joe [1997], p. 132-134 and references therein. Any symmetric Stable distribution is the variance mixture of the normal distribution (see Bondesson [1992], p. 116). Other examples of the variance mixture of the normal distribution are shown in the EGGC family in the Section 2.3.3. 2.3 Particular families of distributions We review and investigate some distribution families which are used either as distribution of inno- vation or as marginal distributions in the theory of continuous-time generalized AR(1) process. 2.3.1 Infinitely divisible, self-decomposable and stable distributions Infinitely divisible (ID), self-decomposable and stable distributions appear quite often in the study of the continuous-time generalized AR(1) process. For ease of reference, we briefly review them here. Good references are Bondesson [1992] and Feller [1966a, 1966b]. Definition 2.2 (Infinite. Divisibility) Suppose X ~ F. IJ for each n > 1, X can be decomposed into the sum of n independent and identically distributed rv's, namely X = XnX + Xn2 -I h Xnn, where XnX, Xn2, • • •, Xnn iid, then the probability distribution F is said to be infinitely divisible (ID). By the definition, it follows that <fix{s) = {<Pxnl {s))n. This leads to that ip][n(s) is a cf for any non-negative integer n. Thus, the ID is equivalent to that tpx(s) 1S a °f f ° r a u a > 0. This class of ID distributions is closed under convolution and weak limits. Some canonical representations of the mgf of the ID distributions are summarized below. Proposition 2.3.1 Suppose X ~ F, an ID distribution. 43 • When the support of F is 5J, the Levy(-Khintchine) representation of mgf is Mx{s) =exp |as + y ' ̂  + Jy if ~ 1 ~ 1+^) ' ^ e s = °> a 6 K> where the measure L satisfies J min(l, y2)L(dy) < oo. • When the support of F is 3ft+, the representation of LT is <j)x{s)=exp\-as+ I (e~sy - l) L{dy) \ , a > 0, I •/(o.oo) J w/iere t/ie Lew?/ measure L is non-negative and satisfies / min(l, y)L(dy) < oo. The J(0,oo) parameter a is called the left-extremity. • When the support of F is Ao, the representation of pgf is Gx(s) = exp { I (Sy - 1) L(dy) \ = exp {\[Q(s) - 1]} , '(0,oo) I where the Levy measure L is non-negative and satisfies / min(l, y)L(dy) < oo. Here J(0,oo) A = JjQ ^ L(dy), the total Levy measure, and the pgf Q(s) = X-1J2skL({k})- k=l Note that the term eas corresponds to the mgf of the constant a. Hence, for the case that the support is 3 ? + , the lower bound is a > 0. Also for this case, there is a nice stochastic explanation. Ignoring a and considering A = Jj 0 ^ L(dy) < oo, we know that L(dy)/\ is a probability measure on (0,oo). Assume Yo = 0,Yj,j = 1,2,..., be iid rv's with probability measure L(dy)/X, and Z ~ Poisson(A). Define the compound Poisson X = 2~Zf=ô j> n a m e r y Poisson compound with a distribution with support on Then the mgf of X is to be Mx(s) = E(esX) = E [e"^UY^ = E (E (es^UYi | z ) ) E f \-lesyL{dy) J(0,oo) = exp < A f A- 1 e^L(dy) - 1 J(0,oo) 44 {A f \-lesyL(dy)- [ \-xL(dy) \ |_./(0,oo) 7(0,oo) {/ ( (7(0,00) = expM (eaV-l)L(dy)\. (7(0,oo) J This shows that X is distributed in the ID distribution represented in the second case of this proposition. When A = oo, the explanation is a little bit complicated. Interested readers can refer to Bondesson [1992], p. 16. For support being yVo, the ID distribution is compound Poisson too, i.e., Poisson compound with another discrete distribution with support on J\f (excluding 0). A nice proof for this case can be found in Feller [1966a], p. 271-272. Non-negative ID rv's are of particular interest in our research. In practice, we may not have the Levy representation of its pgf or LT. However, there is an simple verification approach: check the absolute monotonicity of M'x(s)/Mx(s). See Bondesson [1992], p. 16. This absolute monotonicity is equivalent to the infinite divisibility of a non-negative rv. Note that this verification approach is equivalent to Theorem 2.2.6 given by Joe [1997]. Common examples of infinitely divisible distributions are: Gamma, Negative Binomial, Stable distributions, and so on. Next we turn to self-decomposable distributions. Definition 2.3 (Self-decomposability) Suppose X ~ F. If for each c, 0 < c < 1, there exists a rv e c such that X = cX + e c, where ec is independent of X, then the probability distribution F is said to be self-decomposable (SD). An equivalent definition is that X(s) is SD iff for each c, 0 < c < 1, (j>x{s)/4>x{cs) [or ipx(s)/(fx{cs)] is the LT [or cf] of a probability distribution. In probability, this class is sometimes called the L—class. For example, Gamma and Stable distributions are SD. The property of self- decomposability can be applied to construct the stationary discrete-time or continuous-time first 45 order autoregressive process by setting X(n + 1) = cX(n) + en, n G A/"o, X(n), en are independent, (refer to Vervaat [1979]) and X(t + h)= e-phX(t) + e{h), h > 0, t £ 3ft+, X(t), e(h) are independent. The latter corresponds to Ornstein-Uhlenbeck-type processes; see Section 1.2. However, the support of marginal distributions of these processes can not be MQ. For this reason, the concept of SD is generalized to discrete distributions with support MQ by replacing the constant multiplier with binomial thinning. Definition 2.4 (Discrete Self-decomposability) Suppose X ~ F. If for each c, 0 < c < 1, there exists a rv ec such that where ec is independent of X, then the probability distribution F is said to be discrete self-decomposable (DSD). This is credited to Steutel and van Harn [1979]. In the sense of pgf, this definition is equivalent to that Gx (s)[Gx (cs + l- c) is a pgf for each 0 < c < 1. Similar to SD, the property of DSD leads to applications in construction of stationary discrete-time or continuous-time first order autoregressive processes with non-negative integer-valued margins in the literature. We will show examples of DSD distributions; these are analogues of continuous SD distri- butions and are given in the end of this subsection. Now we consider the Stable distributions. Definition 2.5 (Stability) Suppose X ~ F. If for each n > 1, there exists constants bn and cn such that X can be decomposed as X = bn + cn{Xnl + Xn2 + . • • + Xnn) = ( — + cnXni) , where Xnl,Xn2,..., Xnn X X = c * X + ec = ^ h + ec, IQ = 0, h , I 2 , . . . i.i.d. Bernoulli(c), 46 then the probability distribution F is said to be Stable. If bn = 0 for all n, then F is said to be strictly Stable. This class has the mgf of form See Urbanik [1972]. Finally, we discuss the relationship among ID, SD and Stable. Obviously, Stable is a subset of ID by their definitions. As to others, however, it's not clear by their definitions. Further research has shown that proof of SD C ID can be found in Feller [1966b], p. 553-555, and a brief explanation of Stable C SD can be seen in Bondesson [1992], p. 19. As for the discrete self-decomposability, Steutel and van Harn [1979] (Theorem 2.2) proved that a DSD distribution is ID. For a continuous distribution with positive support, it is of interest to find and study its discrete analogue, because they may share some common features in analysis. Essentially, the discrete analogue is defined in such a way: Definition 2.6 (Discrete analogue) Assume the LT of a continuous distribution with positive support is (/)(s). Then, its discrete analogue with non-negative integer support is defined to have pgf of form G(s) = cj>{\ - s). This definition sometimes can be modified to be G(s) = <j)(d(l — s)) (d > 0) to enlarge the family of discrete analogue (see Example 2.5). Common examples are: Poisson is the discrete analogue of a degenerate rv on a positive point; Negative binomial is the discrete analogue of Gamma; in particular, Geometric is the discrete analogue of Exponential. If cj)(s) is a LT, then G(s) = 0(1 — s) is always a pgf. This follows from Theorem 2.2.1 by checking the [0,1] domain of s, G(l) = 1 and A M feature [follows from C M of <f>\. Thus, for any K is a non-negative measure. Stable C SD C ID. The converses are not true. For example, the Gamma distribution is ID and SD, but not Stable. The 47 positive continuous distribution, we can always obtain its analogue by denning its pgf in terms of the LT. One may wonder what's the explanation of the discrete analogue. Suppose A is a positive rv with the LT cf>{s). Given A = A, Y ~ Poisson(A). Hence Y is a Poisson mixture, and is a non-negative integer rv. The pgf of Y is then G(s) = E (sY) = E (E (sY\A = A)) = E (e^"'1^ = <p{\ - s), 0 < s < 1. This means that the discrete analogue is the Poisson mixture and the positive continuous distri- bution is just the mixing distribution. Therefore, by Poisson mixing, there is one-to-one mapping between the class of positive continuous distributions and the class of discrete Poisson mixtures. For a discrete Poisson mixture distribution, we call the corresponding positive mixing distribution as the continuous analogue of that discrete distribution, and by algebra, it has the LT in terms of the pgf: <t){s) = G( l - a). Note that in general, we can't define a LT by an arbitrary pgf in this way. The big problem is whether G(s) can be extended from domain [0,1] to (-00,1]. It is sure to work for a Poisson mixture, but not certain for a non-Poisson mixture. We end this subsection with two examples of continuous SD distributions and their discrete analogues, DSD distributions. Example 2.4 (Positive stable distribution and discrete stable distribution) The positive stable distribution has LT ci(s) = exp {-As 7} , A > 0, 0 < 7 < 1. The discrete stable distribution was introduced by Steutel and van Ham [1979] to have pgf G{s) = <j>{l -s) = exp {-A(l - s)7} , A > 0, 0 < 7 < 1. The first one is SD, while the latter is DSD (refer to Steutel and van Ham [1979]). Example 2.5 (Mittag-Leffler distribution and discrete Mittag-Leffler distribution) Re- fer to Bondesson [1992], p. 15. Assume X — Y1^ Z, where Y ~ Gamma(/3,1) and Z is distributed 48 in positive stable with LT e s 7 , 0 < 7 < 1. Extending to 7 = 1 so that 0 < 7 < 1, we will have Z = 1 as a special case at the upper bound of 7. Then the LT of X is Ms) = E ( e ^ 1 ^ ) = E (E ( e - ^ \ Z ) ) = E ( e-^) = This LT family is labeled as LTE in Joe [1997], p. 376 where it is a special case of Theorem 2.2.7. Taking 3 = 1, we obtain Ms) = —-—, 0 < 7 < 1 , which is the LT of Mittag-Leffler distribution named by Pillai [1990], because the corresponding cdf is linked to the Mittag-Leffler function. When 7 = 1, it is exponential. Hence, the Mittag-Leffler distribution can be viewed as a generalization of the exponential distribution. It seems that the Mittag-Leffler distribution is unlikely to be a stable distribution since one special case is the exponential distribution, which is in the Gamma family, and the Gamma family is not stable. The discrete Mittag-Leffler distribution was introduced by Pillai and Jayakumar [1995], and has pgf of form G(s) = - - , d > 0, 0 < 7 < 1. w 1 + d( l - s) 7 "~ Pillai and Jayakumar [1995] also gave an explanation for this distribution. Consider an infinite sequence of Bernoulli trials where the k—th trial has success probability y/k, 0 < 7 < 1, k = 1,2,3, Denote Y as the trial number in which the first success happens. Then the pmf and pgf ofY are - l 7 ( 7 - l ) - - - ( 7 - f c + l) J . - 1 2 . 1 ĵ j 5 ft, — 1, Z, O, . . . , GY(S) = i-(i-sy. Hence Y has a power series distribution with lower support point 1, Let Z be from Geometric with pgf Gz{s) = i lwn+d - i ) = i=S' and X = YI Yi> where Y0 = 0, Yt (i > 1) iid from the power series distribution. Then X has pgf of form GX(S) = . - E (B (.EJ.*|Z)) = E ([1 - ( 1 - sVf) - f T ^ - Pk I i \k 49 Similarly, the discrete Mittag-Leffler distribution can be seen as a generalization of Geometric distribution, because it becomes Geometric distribution when 7 = 1. The Mittag-Leffler distribution is SD, and the discrete Mittag-Leffler distribution is DSD. See Section 7.1. 2.3.2 Tweedie exponential dispersion family The Tweedie exponential dispersion family is a major member in the class of exponential dispersion models, which has been systematically studied by Prof. Bent J0rgensen. Important references are J0rgensen [1986, 1987, 1992, 1997]. The following is extracted from J0rgensen [1997] and Song [1996]. This section is referred to in a few places in subsequent chapters. It can be skimmed in the first reading. Suppose X ~ ED*(9,\), the exponential dispersion distribution with probability density (mass) function proportional to c(x; A) exp{0x — XK(9)}, X G 5ft, where c(x; A) is a density with respect to a suitable measure (typically Lebesgue measure or counting measure), and the cumulant generator. Hence, a suitable measure v(dx) is required so that This kind of distribution, ED*(9, A), is called the additive exponential dispersion model with the canonical parameter 9 and the index parameter A. Let 6 = {9 £ Sft : K(9) < oo} be the canonical parameter domain, int0 be the interior of 0. Denote the mean value mapping T : int9 —>• 9?, and the mean domain defined by r{9) = K'(0) and ft = r(intG) 50 respectively. Define the unit variance function V : 0 —• 3?+ as V{n) = r ' (T 1(/X)). By the property of exponential family, the cumulant generating function is - . o g E [ e - , = IoE(e-M.)/E-C(L;A)E%(DL) = log (E-M«)EM»+«)) = A[k(0 + S) - «(&)], s e e - 0. Note that s takes value 0*-0 to guarantee that K(9+S) = n(6*) < oo, where 6* € 9. Differentiating K*(s; 9, A) twice with respect to s and setting s = 0, we find the mean and variance to be E ( X ) = Ar(0) and Var (X) = AV(r(0)) = \T'{9). Let Y = X / A , fi = r(0),cr2 = 1/A. Then by definition, Y ~ ED{fi,a2), with probability density (mass) function c(y]X)exp{X[9y-K(6)}}, ye®, where c(y; A) is a density with respect to a fixed measure, and the cumulant generator is K(0) =log(y 'e e»'c(y;A)dy) , the cumulant generating function is K(s;9,X) = \ogE[esY} = X{K(9 + s/X)-K(9)}, s 6 9 - 9, and the mean and variance are E ( Y ) = E ( X ) / A = /i(=T(0)) and Var (Y) = V a r ( X ) / A 2 = a2V{p){= CTV(0)). This kind of distribution, ED(p, a2), is called the reproductive exponential dispersion model. Here a2 = 1/A is called the dispersion parameter. Concrete examples in exponential dispersion models include the Binomial, Negative bino- mial, Poisson, Gamma, Normal, hyperbolic secant and generalized hyperbolic secant distributions. The Tweedie family is a special member in the class of exponential dispersion models; it was first studied by Tweedie [1947]. Following the reproductive form, it has special form of unit variance function: v(ji) = pen, d e K . • 51 Hence, the ratio of variance and mean is Var(T) _ „_! E ( y ) - " • A model with this reproductive form is denoted as TWd(/i, o 2). Tweedie models are closed with respect to scale transformation, i.e., if Y ~ Twd(/x, cr2), then cY ~ Twd(cfi, c 2~ do 2). Since V{n) = u- d = rd(9) = T'{9), T{9) must be [(1 -d)9]1^1~d\ r(9) = dfl, d = l. For the sake of convenience, let 8 = ffi • Then 1/(1 -d) = B-l, l-d = 1/(8 - 1). This leads to ee, d = l, or jS = oo. K(0) Since T(6>) = «'(#), for Tw^f//, er2), the cumulant generator «(0) has explicit form ( ^(^y, d?l, 2, o r / 3 ^ 0 , oo, log(-0), d = 2, or/3 = 0, d = 1, or /3 = oo. (ignoring the arbitrary constants in the integrations will not affect the final results of cumulant generating function.) Thus, one of the other advantages of Tweedie model we are appreciating is that it has explicit expression for the cumulant generating function and nigf. The cumulant generating function of Twd(/i, a 2) is K(S-0,X)= I _ A l 0 g ( i + J L ) , k Xee [e*/A - 1] , and the mgf of this family has special exponential form: e x p J A ^ ^ / ^ l + ^ - l ] } , d # l , 2 ; ( l + d = 2; [ exp {Xe9 [eslx - l] } , d = 1. d = 2; d = l, M(s;9,X)=B [esX] = \ 52 Further research shows that d £ (-00,0]U[1,00]. Thus, if d £ (-00,0], then 8 £ (1,2]; if d £ [1,00], then 8 £ [—00,1]. This leads to 8 £ (1,2] U [—00,1] = [—00,2]. Now for the future use, we impose the subscript 8 on T(6), K(6),K(S; 6, A), M(s; 0, A), c(x; A) and c(y; A) to indicate that they are linked to the specific parameter 8. Also, the probability density (mass) function of Tweedie model can be obtained, though it is complicated. Recall X = XY has the additive model if Y has the reproductive model. The probability density (mass) function is fx{x;d,X,B) = c0(x; A)exp{0a; - A / ^ f l ) } , where 1, 8<0,x = 0; 2_, xV(-k/3)k< ' p <u,x > V, 00 D X^Xk4(-Vx)*™(-k*P)> 0 < /3 < l , x > 0; k=l £ £ ^ ( l ^ ) ' : ^ ( - ^ ) , K8<2,x£X. Here Kp(0) is just the previously calculated K(9) for Twd(n,o-2). The corresponding reproductive model then has has probability density (mass) function fy(y; 8, X,8) = Xcp(Xy; A) exp{A[% - K0(6)]}. Tweedie family includes distributions with support on 5R, 5ft+,.A/o, corresponding to real- valued, positive-valued and non-negative integer-valued random variables. These are related to different ranges of d, in which we view the endpoints of ranges as boundaries. Table 2.1 summarizes the different types of Tweedie models. From the table, we know that the real support 3ft appears when d < 0 and d = 00; this corresponds to 1 < 8 < 2, while the non-negative support A/o, or Oft0, or 3 f t + appears when 1 < d < 00; it corresponds to 8 < 1. Some boundary cases are well known distributions: normal (d = 0), Poisson (d = 1), Gamma (d = 2), inverse Gaussian (d = 3) and extreme stable (d = 00). When 1 < d < 2 (corresponding to 8 < 0), the compound Poisson with Gamma obtains, that is, i=0 N 53 Table 2.1: Summary of Tweedie exponential dispersion models (S = support set). Distributions d S 0 6 Extreme stable d < () 3ft 3ft+ 3fto Normal d = 0 3ft 3ft 3ft (Do not exist) 0 < d< 1 — — — Poisson d = 1 3ft+ 3ft Compound Poisson 1 < d < 2 3?o 3ft+ 3ft_ Gamma d 2 3ft+ 3ft+ 3ft_ Positive stable 2 • d < 3 3ft+ 3ft+ -3ft0 Inverse Gaussian d 3 3ft+ 3ft+ -3ft0 Positive stable d > 3 3ft+ 5ft+ -3ft0 Extreme stable d oo 5ft 3ft 3ft_ where ZQ = 0,Zi *~ Gamma(0, — 8) and N ~ Poisson(AKrf(0)). This distribution has a positive probability on zero, Pr[F = 0] = Pr[7V = 0] = exp{-A« d(0)}, and density function My; 6,\8) = -J2 'V exp{% - A/*(0)}. y~{ v-T{-tB) 2.3.3 G e n e r a l i z e d c o n v o l u t i o n s The generalized convolution is a natural extension of a finite convolution. It helps to connect those individual distributions which seem to have quite different forms in the pdf or cdf. Fortunately, we find that the generalized convolution provides a huge ammunition for the theory of continuous-time generalized AR(1) process. A good reference on the generalized convolution is Bondesson [1992], The following materials regarding GGC, EGGC, GCMED and GNBC are extracted from that book. First, we review the generalized Gamma convolution, which was introduced by Thorin [1977a, 1977b], to understand the mechanism of construction of generalized convolution. 54 Since the LT of Gamma(uj,/%) is ( J (i = 1,... , n), the LT of the sum of n such Gamma rv's, i.e., finite convolution, is *-w=n(̂ r-̂ {|*̂ (̂ )}- Consider pointwise limits of (j>n{s) and permit a Gamma distribution to be degenerate at a point a > 0 with LT e~as. This leads to the following definition. Definition 2.7 A generalized Gamma convolution (GGC) is defined as a distribution with support on [0, oo) and LT of the form </>{s) = exp \ -as + / log ( —-— ) U(du) > , { 7(0,00) V« + s / J where a > 0 and U(du) is a non-negative measure on (0, oo) satisfying / | logu(C/(du) < oo and / u~lU(du) < 7(0,1] 7(l,oo) oo. Therefore, the GGC is the limiting distribution for a sequence of sums of independent Gamma variables with possibly different rate parameters. Extending this idea to random variables from other families, we obtain the concept of the generalized convolution, which is defined as the limit distribution for a sequence of sums of independent variables from a parametric family. In the sense of limit, we know that there are usually numerous rv's involved in the convolution. In our study, we tentatively call the distribution involved in the sum as the base distribution of the generalized convolution. For a rv X distributed in the GGC class, we have E[X] = -<//(0) = a + [ u~lU{du), Var[X] = 0"(O) - (</>'(0))2 = / u~2U{du). 7(0,oo) 7(0,oo) This class is surprisingly rich. Some of the examples include • (strictly) positive Stable distribution on (0, oo): the LT <f)(s) = exp{—s7} (0 < 7 < 1), and the measure U(du) = ysm^J7I'>u'y~1du which leads to / U(du) = 00. JfO.oo) 55 Pareto distribution: the pdf f(x) = 7 A 7 (a: + A) 7 1 (x,7, A > 0), and the density of the measure U is U'{u) = j^Xyu1~1e~Xu. Generalized inverse Gaussian distribution: the pdf —C\X — c2x 1}, x > 0 (ci,C2 > 0,/3 G 3ft). The [/"-measure has density 0, U < C l , > - c i ) ' , + 1 / o 0 0 / 0 0 0 ( A - c 1 ) - ' , - 1 p - 1 e - 1 / ' ' x exp {-C2(s - l)2p(A - c i ) - 1 } dpc/A , u > c x. Letting cx —> 0 leads to inverse Gamma distribution. A good reference on the inverse Gaussian distribution is Seshadri [1999], where applications can be found in reliability, survival analysis and actuarial science. Generalized Gamma distribution (power of Gamma random variable): the pdf is f(x) = Cxp-1exp{-xa}, x>0 ( 0 < a < l,/3 >0). The density of [/-measure is U{u) = 7r 1 arg 0 0 f-~nfc -ka k=0 Beta distribution of the second kind (Ratio of Gamma variables): the pdf is f{x) = Cxp~l(l + cx)" 7 , x>0 (7 > £ > 0,c > 0). However, the density of the [/-measure does not have a simple expression. Lognormal distributions: the pdf is /(x) = ^ - ^ e x p { - S ^ } , *>0 . Unfortunately, no simple expression for the [/-measure exists. 56 Next we visit the extended generalized Gamma convolutions (EGGC), which have support on the whole real line Oft. This class is needed to cover the limit distributions for sums of independent positive and negative Gamma variables. It was also introduced by O. Thorin. See Thorin (1978). Definition 2.8 An extended generalized Gamma convolution (EGGC) is defined as a distribution with support on Oft and cf of the form v { s ) = e x p { i 6 s _ £ + ( l o g ( _ H _ ) _ J^L.) u(Mj , where b £ Oft, c > 0 and U(du) is a non-negative measure on Oft \ {0} satisfying [ —-—7rU(du) < oo and f | log u21U(du) < oo. JSt\{0} 1+UZ i|u|<! Remark: The term su/(l +u2) is added to guarantee the convergence of the integral. When / \u\~1U(du) < oo, it can be omitted. Hence, GGC is a subclass of EGGC. Further research J\u\>\ shows that the symmetric EGGC is the variance mixture of the Normal distribution, with cf of form = exp j - ^ + 1 ^ log ( ^ ) J7(d«) j , where U is symmetric on Oft \ {0}. Thorin [1978] proved the EGGC is SD. Some examples of the EGGC class are listed below. They are verified by characterizations other than specifying [/-measure. • Stable distribution: the cf of the general Stable distribution of index a (0 < a < 2) is <p(s) = exp {iu-s - C\s\a(l - i/3sign(s)uj(s, a))} , p £ Oft, C > 0, \B\ < 1, where tan(«7r/2), a ^ 1, -27T-1 log [s|, a = l. The Cauchy distribution is within the Stable distribution family with cf u(s,a) = { <p(s) = exp{-C| 5 |} , C > 0 . 57 Generalized Logistic distribution: This rv is derived from two Exponential rv's as X = ]og(Yi/y 2), where Yi ~ Gamma(/3i, 1) (i = 1,2). The pdf of X is o(Pl,P2) When Bx = /32 = 1/2, it is /(a;) = - (e x / 2 + e- s / 2 ) - 1 , s 6 S. While /3i = Bi = 1, it is the logistic distribution with pdf f(x) =e-x/{l+e-x)2, x€3ft; and mgf s °° 1 °° 1 M(S) = r ( i + s ) r ( i - s ) = .™ = T T ^ — 5 7 7 ^ = T T 7 1 — , / M - sm(7rs) £J (1 - s2/k2) ^ (1 - s/k)(l + s/k) A stochastic representation of scale mixture of the Normal distribution for this logistic rv X can be found in Joe [1997], p. 133-134. See also Andrews and Mallows [1974] and Stefanski [1991]. It is X = Z/V, where Z ~ N(0,1), and V is a positive rv with pdf oo / y ( a ;) = 2 5](- l )^ 1 fc 2 ^ 3 exp{-fc 2 / (2x 2 )} . fc=i Hence, X is the rv of a variance mixture of the Normal distribution. » Logarithm of Gamma variable: X = logy, Y ~ Gamma(/3,1). » Other symmetric EGGC distributions with pdf: (i) f(x) — C ( l + e x 2 ) - 7 , t-distribution, essentially (ii) /(x) = C ( l + c |x | ) - 7 , two-sided Pareto distribution (iii) /(aj) = Cexp{—cy/x2 + J}; Hyperbolic distribution • 58 (iv) f(x) = Cexp{-c\x\2/k} (k = 1,2,...) The generalized convolutions of mixtures of Exponential distributions (GCMED) is another extension of GGC (see Bondesson [1992], pl39-140). This class has support on [0, oo). Definition 2.9 A generalized convolution of mixtures of Exponential distribution (GCMED) is defined as a distribution with support on [0, oo) and LT of the form <b(s) = exp \ -as + f (— | U(du) \ = exp \ -as + / —:—^-^U(du) > , [ 7(0,oo) \U + S U) J [ ./(O.oo) «(« + J where a > 0 and U(du) is a non-negative measure on (0, oo) satisfying 1 -U(du) < oo. '(0,oo) «(1 + « ) ' Remark: A mixture of Exponential distributions (MED) is defined as a probability distribution on [0, oo) with pdf f{x) = [ ue~xuU{du), 7(0,oo) / , or cdf F(x)= [ (l-e-xu)U(du). 7(0,00] Here U(du) is the mixing measure (for the inverse of the scale parameter), which is non-negative and satisfies / U(du) = 1. U({oo}) > 0 implies the distribution has an atom at 0. The LT of 7(0,oo] an MED is <f>(s) = U({^}) + / ^-U{du). 7(0,oo) u + s The LT of compound Poisson with Exponential has the form of 1 1 ^ ) = e x p { A ( ^ _ i ) } = e x p { A w G + s u / Hence, the GCMED is a generalized compound Poisson-Exponential convolution following the convention of GGC. Some examples of GCMED are 59 Compound Poisson with Exponential distribution: Let X = YiLo^i, where N ~ Poisson(A) and Yo = 0, Yi ~ Exponential(u) (i = 1,2,...). Then 4>x{s) = exp{A(c6yi(s) - 1)} = expJAu ( —-• J 1 . ^ \u + s u J ) Non-central x2-distribution: Let Z\ ~ N(jUj, 1) (i = 1,2,... ,n) be independent. Then n W where A = YH=I A*?, ~ Poisson(A/2) and Y 0 = 0, Y, ~ Exponential(l/2). Logarithm of Beta variable: X = - log Y', where Y ~ Beta(a, B). The LT of X is i > + /3)r(q + s) ^ l s j ~ r ( a ) r ( a + /3 + s ) ' Inverse Gaussian mixture distribution (introduced by Jorgensen, Seshadri & Whitmore [1991]): the pdf is f{x) =C'(p + q^/cl~j72x)fl{x), q=l-p, 0 < p < 1, where /i(a:) is the pdf of inverse Gaussian distribution fiix) = Cx~3l2 exp{—c\x — c2x~1}, which has LT <t>\is) = exp{c3(l - yjl + C4s)}, C 3 = 2y/CiC2, c 4 = 1/ci. The LT of the inverse Gaussian mixture distribution is cf>{s) = ip + q/y/l + CAs)(j)i($). This family includes the well-known life distribution of Birnbaum & Saunders (1969) when p = l /2 . 60 The last generalized convolution discussed in Bondesson [1992] is the generalized Negative Binomial convolution (GNBC), which has support on non-negative integer {0,1,2,...}. The discrete analogue of the Gamma distribution is the Negative Binomial distribution; hence, the GNBC is the discrete analogue of the GGC. Definition 2.10 A generalized Negative Binomial convolution (GNBC) is defined as a distribution with support on non-negative integer and pgf of the form <K.) = « P { . ( . - i ) + / w i i , ( r i i ; j v w } , where a >0, p = 1 — q and U(du) is a non-negative measure on (0,1) satisfying / qV(dq) < oo and / log(p)V(dq) 7(0,1/2] 7(1/2,1) < oo. Some examples of the GNBC class are Discrete Stable distribution: the pgf is G(s) = exp{-c(l - s)a}, c> 0, 0 < a < 1. (refer to Steutel and van Ham [1979]). Generalized Waring distribution: this family is defined with probability mass W = C ; ! ( a + /3 + 7)br ^ = 0 ' 1 ' 2 ' — a ^ > 0 ' c = r ( a + ^ + 7 ) r(7)' where 8^ = 1, 8^ = 8 • (8 + 1) • • • (8 + j - 1). The pgf is the sum of a Hypergeometric series. Furthermore, the generalized Waring distribution is the Poisson(A)-mixture, where A = Y • X\ IX2 and Y, X\, X2 are independent with Y ~ Gamma(/3,1), X\ ~ Gamma(a, 1), X2 ~'Gamma(7,1). This family leads to several distributions: 61 (i) Waring distribution (see Johnson h Kotz [1969], p. 250): 8 = 1. A special case is the power series distribution when a = l — n, 7 = n (0 < n < 1), which has pmf and pgf: (1 - n)W 1 - (1 - s)71 Pi = T l r ^ L . 3 = 0,1,..., and G(s) = U *J . (ii) Yule distribution: a = 8 = 1 and 7 —>• 0. (iii) NB(/3, g)-distribution: q = a/(a + 7 ) and let a —»• 00, 7 - 4 00. Distributions (i) and (ii) have applications in modelling word size in prose. • Logarithmic series distribution (shifted): the pmf and pgf are ^ = ^ T T (̂  = 0,1,2,...), c = l - e * , 0>O, and G(s) = ^ . ~ ^ 6(1 + 1) slog(l — cj In the study of continuous-time generalized AR(1) process, we have discovered four new generalized convolutions, which we tentatively name as GC I, GC II, GC III and GC IV. These generalized convolutions play an important role in customizing marginal distributions of a steady state Markov process (see Chapter 6). Definition 2.11 A generalized convolution I (GC I) is defined as a distribution with support on non-negative integer and pgf of the form G(s) = exp J -as + [ q[S~l)V(dq) \ , ^ 7(o,i) 1 - 9 s J where a > 0 and V(dq) is a non-negative measure on (0,1) satisfying I qV(dq) < 00 and / (1 - q)~lV{dq) < 00. 7(0,1/2] 7(1/2,1) The base exp j 9 !̂_~^ j was proved to be a pgf in Example 2.1 in Section 2.2.1. Definition 2.12 A generalized convolution II (GC II) is defined as a distribution with support on non-negative integer and pgf of the form G(s)=e*J-as+ [ q { s ~ " ^ V(dg)\, I 7[7,l) 1-qs J 62 where a > 0, 7 > 0 and V(dq) is a non-negative measure on [7,1) satisfying f (l-q)-1V(dq)<^. The base exp { ^ " i l ^ " " 7 ^ }' 0 < 7 < c/ < 1, was also proved to be a pgf in Example 2.1 in Section 2.2.1. Definition 2.13 A generalized convolution III (GC III) is defined as a distribution with support on [0,00) and LT of the form = exp Las+ [ 1 1 - 7 l o g f ( 1 " 7 + 7 ^ ) ^ ) l , [ 7(o,oo) 1 - 7 - iu V U - 7 ) ( « + s) / J where a > 0, 0 < 7 < 1 (7 is fixed) and U(du) is a non-negative measure on (0,00) satisfying / I logu\U(du) < 00 and / u~2U{du) < 00. 7(0,1]) 7(1,00) iVoie that when 7 = 0, GC III will become GGC. Hence, GGC is a special case of GC III. The base exp {log ( { r f ^ g ^ y ) } was proved to be a LT in (2) of Theorem 2.2.8. Definition 2.14 A generalized convolution IV (GC IV) is defined as a distribution with support on [0, 00) and LT of the form Hs) = exp J -as + A f ~ 7 + 7 s ) U(du) I, ^ 7(o , 7 - l - i ] u + s J where a > 0, A > 0, 0 < 7 < 1 ( 7 is fixed) and U(du) is a non-negative measure on ( 0 , 7 - 1 — 1] satisfying / u~lU(du) < 00. 7 (o , 7 - 1 - i ] Note that when 7 = 0, the LT will be 4>{s) = exp \ -as + A / ——U(du) \ = exp I -as + / - r - 1 7(0,oo) u + s J I 7(0,oo) «(« S U'{du) • + a) where U'(du) = XuU(du). Thus, GC IV will become GCMED. This shows that GCMED is a special case of GC IV. 63 The base exp {-A • a ( 1 ^ 7 ' ) ' } w a s P r o v e d t o b e a L T i n ^ o f T n e o r e m 2 - 2 - 8 - Specific distributions which are in the new generalized convolution families are not known at this moment; thus, further investigations are under study. 2.4 Independent increment processes and examples The independent increment process is well studied, and is intimately connected with infinitely divisible distributions. The latter links to the study of Levy process. Refer to Prabhu [1980], p. 69, Feller [1966b], p. 177-179, Bhattacharya & Waymire [1990], p. 349-356, Protter [1990], Section 5 in Chapter 1. We review this process family and will choose some of them to be the noise process or innovation process in the theory of continuous-time generalized AR(1) processes defined in subsequent chapters. Definition 2.15 Stationary independent increment process (IIP): A process {X(t);t > 0} is said to have stationary independent increments if it satisfies the following properties: (i) For 0 < t\ < t2 < • • • < tn(n > 2), the random variables (ii) The distribution of the increment X(tk) — X(tk-i) depends on (tk-\,tk) only through the difference tk-tk-\- Without loss of generality, X(0) is usually taken as 0. This is because that if X(0) ^ 0, we can subtract it from the process which results: Y(t) = X(t) - X(0). The new process {Y(t);t > 0} has stationary independent increments and starts from Y(0) = 0. This means that the starting point is independent of any increment for a stationary independent increment process. Since X(h),X(h) - X(ti),X(t3) - X(t2),X(tn) - X{tn-i) are independent. 64 X(t) can be viewed to be the sum of n independent random variables, all of which have the same distribution as X (t/n). This is true for all n > 1. Hence, it follows that X(t) has an infinitely divisible distribution. Note that it's easy to extend the stationary IIP to non-stationary IIP by loosening condition (ii) , in which case the distribution of X(t) may not be infinitely divisible if the small increments are not identically distributed. With the following additional conditions, the stationary independent increment process becomes the Levy process: (iii) X{t) is continuous in probability, namely, for any e > 0, lim Pr[|X(i)| > el —> 0. This is equivalent to stochastic continuity: lim Prfl-X^) — -X"(*i)| > e] = 0. *i — (iv) There exist left and right limits X(t—) and X(t+). Assume that X(t) is right continuous: X(t+) = X(t). Here the difference X(t+) - X(t-) = X(t) - X(t-) is called the jump of the process at time t. Note that the number of conditions required for a Levy process may appear as three to five in the literature, depending on the author's view. For example, some impose X(0) = 0, some don't. Because the increment Xfa) — X{t\) is infinitely divisible for the Levy process, we can characterize this process by the mgf, LT or pgf of the infinitely divisible distribution discussed in Proposition 2.3.1. Some scholars even define the Levy process in this way, e.g., Bondesson [1992], p. 16. Since we are particularly interested in three kinds of supports of increment: (—oo,+oo), (0, +00) and {0,1,2,...}, we summarize the results in these three cases in the following proposition. Proposition 2.4.1 Suppose {X(t);t > 0} is a Levy process with X(0) = 0. Consider X(t), the margin at time t. • When the support of X(t) is 5ft, the mgf of X(t) is cf>xit)(s) = exp j i as + y • s2 + J - 1 - L(dv) } > »e s = 0, o G », 65 where the measure L satisfies / min(l, y2)L(dy) < oo Jy^O When the support of X(t) is 5?+, the LT of X(t) is <i>x(t)(s) = e x P \ * —as + 7(0,00) -sy 1) L(dy) a > 0, w/iere i/ie Levy measure L is non-negative and satisfies / min(l, y)L(dy) < oo. Now the 7(0,00) parameter a is called the left-extremity. When the support of X(t) is No, the pgf of X(t) is f (Sy-l)L(dy) 7(0,oo) = exp{*A[Q(a)-l]} where the Levy measure L is non-negative and satisfies / min(l, y)L(dy) < oo. Here 7(0,oo) A = f,Q ^ L(dy), the total Levy measure, and the pgf is oo Q(s) = \~lYJskL({k}). k=l When the support of X(t) is Tvo, obviously it is compound Poisson based on another discrete distribution which also has support AV When the support of X(t) is 0?+, it is also compound Poisson. See the explanation in Bondesson [1992], p. 16. When the support of X(t) is Dft, further research shows that the Levy process can be decomposed as a Brownian motion plus drift and a jump process. And the only one in Levy process family, which have a.s. continuous sample paths, is the Brownian motion. See Bhattacharya & Waymire [1990], p. 349-356, and Protter [1990], Section 5 in Chapter 1. The compound Poisson process is a concrete example in Levy process family, which is defined to have compound Poisson increments. Assume X(0) = 0. Then N(t) X(t) = Yh where Y0 = 0, Yi (i > 1) iid and N(t) ~ Poisson(Ai). i=0 66 The margin can be real, positive or non-negative integer valued depending on the support of Yj. The cf, LT or pgf of X{t) is then <Px(t) (s) = e x P {Xt(tpY\ (s) - 1)} , if Yi is a real rv, </>X{t)(s) = e x P {^(^Yi (s) — 1)} , if Yi is a positive rv, [ GX(t)(s) — exp{A2(Gyi(s) - 1)} , if Yi is a non-negative integer rv, where (py1(s), ^ ( s ) or Gyj(s) is the cf, LT or pgf of Y\ respectively. This family contains many processes such as Poisson process, Negative Binomial process, Gamma process, etc. The increment with three kinds of domains: (—00,+00), (0,+00) and {0,1,2,...} are of our special interests in the theory of continuous-time generalized AR(1) process. In the rest of this section, we list some specific stationary IIP {X(t);t > 0} with non-negative integer rv, positive rv and real rv margins respectively for the future use. They are used to construct specific models in the theory of continuous-time generalized AR(1) processes. The non-stationary case can be easily generalized by allowing the the time difference t2 — t\ to be a function of ti and t2, say a function of t2 — t\. A l l starting points are assumed as 0, namely X(0) = 0. Case 1: Non-negative integer rv margins Example 2.6 Poisson IIP. The increment X(t2) - X{t\) ~ Poisson(\(t2 - t\)), with pgf Gx{t2)-x{tl){s) = E = e x p { A ( i 2 - t^s - 1)}, where A > 0. Thus, the margin X(t) ~ Poisson(Xt) with pgf Gx(t)(s) = exp{Ai(s — 1)}. Example 2.7 Compound Poisson IIP. {Y(t);t > 0} is a Poisson IIP defined as in Example 2.6. Z is a non-negative integer rv with pgf Gz{s) = E ( s z ) . The increment of {X(t);t > 0} is defined as Y(t2)-Y(h) X(t2)~X(h)= Z^ i=0 where Z0 = 0 and Zx, Z2,... '~d' Z. Thus, the pgf of X(t2) - X(h) is E • y(t2)-^(*i) E Zi s i = 0 Y(t2)-Y(h) = E{G2 ( t 2 )" y ( t l ) ( s ) } = e x p { A ( i 2 - i 1 ) [ G z ( s ) - l ] } , 67 and the pgf of X(t) is GX(t)(s) =.exp{Ai[G^(s) - 1]}. Gz(s) has a variety of choices. For instance, we can take Gz(s) = Po + Pis + • • • + pnsn, where the pi's are non-negative and sum .to 1. If Gz(s) = s, the Poisson IIP in Example 2.6 obtains. Z can be generalized to a continuous-time process {Z(t);t > 0} {Z(0) = 0) with the property that for ti < t2 < £3, (*2 - h)[Gz{t2_tl)(s) - 1] + (is - t2)[Gz{t3-t2)(s) - 1] = (t3 - h)[Gz{t3-h)(s) - 1]. Define the increment of {X(t);t > 0} as Y(ta)-Y(t!) x(t2) - x(h) = Z ^ - *i)> where Z 0 ( t 2 - * i ) =0 and Zx(t2-ti), Z2(t2-tx),... Z(t2-h). Hence, the pgf ofX(t2) - X(t±) is GX(t2)-x(h){s) =expJA(< 2 - t i ) Gz{t2_tl)(s) -1 J , Checking the pgf of X(t^) — X(t\), we obtain = exp {A(*2 - h) = exp{A(i3-ti) O W - 1 ] } C?x( t 3)-x ( t l)(3) = E ( ^ ^ ) - x ( t 1 ) ) = E ( s ^ 3 ) - x ( < 2 ) + x ( t 2 - x ( t l ) ) = GX(t2yX(ti)(s)Gx{t3)-x(t2)(s) GZ{t2-ti)(s) - 1 + Hh ~ h) Gz(tz_t2 GZ(t3-h){s) - 1 }• Therefore, the pgf of X(t) is exp {A^GZ(t)(s) - 1 }• Example 2.8 Negative Binomial IIP. Let the increment X(t2) - X(t\) ~ NB(6(t2 - ti),j), with P9f ( i _ 7 ^ ( t 2 - t l ) Gx{t2)_x{tl){s) = [ Y Z ^ J , et where 9 > 0 and 0 < 7 < 1. S o X{t) has pgf {^jz^) • 68 Example 2.9 Discrete stable IIP. Let the increment X(t2) — X(t\) be distributed with discrete stable, i.e., the pgf is GX(t2)-x(h)(s) = exp{-A(*2 - - s)a}, where A > 0 and 0 < a < 1. Then X(t) has pgf Gx(t)(s) = exp{—Ai(l — s)a}. Example 2.10 Generalized Negative Binomial convolution (GNBC) IIP. Let the increment X(t2)~ X(ti) be distributed in GNBC with such kind of pgf GX(t2)-X(tl)(s) = exp {(t2 - h) j ]og(^—)V (dq)}. Then X(t) has pgfGx{t)(s) = exp [t J m log^JV (dq)}. Example 2.11 GC I IIP. Let the increment X(t2) — X(ti) be distributed in GC I with pgf of the form Gx{t2)-X(tl)(s) = exp Ut2 - h) f ^^-V(dq)}, Then X(t) has pgf Gx{t)(s)=exp{t f q-^±V(dq)}. ' L 7(o,i) i-qs > Example 2.12 GC II IIP. Let the increment X(t2) - X(t\) be distributed with GC II with pgf of the form GXM-XMW = exp {fe ~ ti) J ^ ~ ^ 7 ' V ( d g ) } , 7 > 0. Then X(t) has pgf Gx(„(») = e x P { * / " < ' - 1 » 1 - ^ V W } . L 7(0,1) l - q s i Case 2: Positive rv margins Example 2.13 Gamma IIP. Let the increment X(t2) - X(ti) ~ Gamma(a(t2 - h),B), with LT where a,B>0. The LT of X(t) is (̂ f̂ )0*, *-c, the LT of Gamma(at,B). 69 Example 2.14 Inverse Gaussian IIP. Inverse Gaussian rv X has pdf fx(x;n,\) = y/\/(2irx3)exp{-\{x- /i)2/(2^x)}, x > 0, where u-, A > 0, and the LT is fo(S) = E[e**] = expj - 1 - 1 + 2u2s^1/2' Now let A = ku2, where k is a constant. Then 4>x(s) = exp \kn 2 ^l'2 For this special form, we can construct Inverse Gaussian IIP {X(t);t > 0}, such that the increment X(t2) - X{h) has LT l - [ 1 + k S 1/2 ^A'(e 2 ) -A' (d)( -5) = exp |A;(t 2 - *i) Hence, the LT of X(t) is <t>x(t)(s) = exp j / c f l - ^ l + | s ^ Example 2.15 GGC IIP. Let the increment X(t2) — X(t\) be distributed in generalized Gamma convolution distributed with LT - X ( t i ) W . » X ( t 2 ) - i ) exp | ( t 2 - *i) / l o S (^7) ̂ d u) #ence, X(t) has LT r*x(i)(5) = e x p [ * y " I o S (^7) ^(RFTI)} • This family is a big class, consisting of many known distributions. Example 2.16 GCMED IIP. Let the increment X(t2) - X(ti) be distributed in GCMED with LT ixM-xwis) = exp [ft , - tl) 1 ^ ^-U(du)} , Then X(t) has LT <l>x(t)(s) = exp H (0,+oo) u + s U(du) } . 70 Example 2.17 GC III IIP. Let the increment X(t2) - X{tx) be distributed in GC III with LT [ 7(0,00) 1 ~ 7 - iu \(1 - 7)(« + 5)) J T/ien /ms LT [ 7(o,oo) 1 - 7 - 7" V C 1 -7>(« + s ) / J Case 3: Real rv margins Example 2.18 Gaussian IIP (Brownian Motion). This is well known. The increment X(t2) — X(h) ~ N(0,t2~h). Example 2.19 Cauchy IIP. A Cauchy(0,\) rv X has pdf fx(x\X) = - T 0 - 7 — 2 , -00 < x < +00, A > 0, 7T A ~r X and cf <px(s) = E[eisX]=e-xW. To obtain a Cauchy IIP, just set the increment X(t2) — X(t\) ~ Cauchy(0,X(t2 — ti)). Example 2.20 Stable Paretian (or stable non-Gaussian) IIP. Consider a special case in stable Paretian family, which has cf of form <p(s) = exp{-A|s| a}, 0 < a < 2. (When a = 2, it's normal distribution.) To obtain a stable Paretian IIP, just let the cf of the increment X(t2) — X(t\) be Px(t2)-*(ti)( s) = exp{-A(t 2 - ti)|5|Q}. Thus, the cf of X(t) is exp{-Ai|s | a }. In summary, all pgf's, LT's or cf's in these examples are of exponential form with (£2 — 1̂) as linear parameter in the exponent. We pick up such a form because we want to change the product form to summation form in obtaining the pgf, LT or cf form of the continuous-time generalized AR(1) process. In all these cases, X(t) is infinitely divisible. 71 Chapter 3 Self-generalized distributions and extended-thinning operations In this chapter, we shall propose a new concept of closure for probability distributions. Families with this closure property are called self-generalized distributions. The support of these families can be non-negative integer or positive real. They induce a class of stochastic operators, which we call extended-thinning operators. These stochastic operators will be applied in generalized Ornstein-Uhlenbeck stochastic differential equations, and the property of self-generalizability plays a crucial role in model construction of continuous-time generalized AR(1) processes (see Chapter 4). In Section 3.1, we shall define the self-generalized distribution in the non-negative integer- valued case and the positive-valued case respectively, and give some examples as well. We discuss the properties of self-generalized distributions in Section 3.2, as well as construction of self-generalized distributions in Section 3.3. Finally, we propose the extended-thinning operations in Section 3.4. 72 3.1 Self-generalized distributions A family of self-generalized distributions has a pgf or LT which is closed under some compound one- parameter operation. The support is non-negative integer or positive real. We give the thorough discussion on both cases in the following subsections. 3.1.1 Non-negative integer case and examples Suppose iv" is a non-negative integer random variable, taking value on {0,1,2,...}. Now we define the self-generalized distribution in non-negative integer case. Definition 3.1 Let A be a subset of reals that is closed under multiplication. Suppose K has cdf F(x; a) depending on a parameter a , a € A . The probability generating function is then the distribution family {F(x; a); a £ A } is said to be self-generalized with respect to parameter a. For brevity and convenience, we say that K is self-generalized with respect to parameter a to refer to the self-generalizablity of the distribution family {F(x;a);a € A } . In non-negative integer case, the self-generalizability is closed under the compound opera- tion for the probability generating function. This closure operation corresponds to an interesting stochastic representation (refer to Property 3.6, which leads us to call it self-generalizability). To illustrate this new family in non-negative integer case, we give five examples in the remainder of this subsection. For the sake of saving space and reducing redundancy in the later study, we label them from II to 15. GK(s;a) =B[sK}= / sxdF(x;a) = £ V Pr[# = *]. If GK{GK(S; a); a') = GK(s; act), 73 Example 3.1 (II): Let K ~ Bernoulli(a) (0 < a < I). The pgf of K is GK(s; a) = (1 - a) + as. Thus GK(GK(s;a);a') = (1 - a') + a'GK(s; a) = (1 - a') + a'[(l - a) + as] = (1 — aa) + aa's = GK(s;aa'). Therefore K is self-generalized with respect to parameter a. Example 3.2 (12): Consider K = ZI, where I ~ Bernoulli(a), Z = Z' + l, Z' ~ NB(l,^h), and a — ^{^a > b = {i-a)i> 0 < a ^ l j 0 ^ 7 < l - Llere the parameter 7 is fixed. Note that Z, Z' have Geometric distributions with positive integer support and non-negative integer support respectively. The pgf of Z is (1 — q)s/(l — qs) where q = (1 + 6 ) - 1 . A straightforward calculation leads to (1 - a) + (a - 7)a GK(s;a) = { 1 _ o n ) _ { 1 _ a h a - It follows that n ( r ( , ,x (1 -a') + (a' -l)GK(s;a) GK(GK(s; a); a ) = ( 1 _ _ ( 1 _ ( a ; a ) = ( i - ^ + ( a - - 7 ) ( r Q ; ; i [ n ) ) ; (1 - a ' 7 ) - (1 - a07(l (ir7{!((?:a7)); (1 - a')[(l - 07 ) - (1 - a)js] + (a1 - 7)[(1 - a) + (a - j)s] (1 — a ; 7)[(l - 07) - (1 - a)7s] - (1 - a ' )7[(l - a) + (a - 7)5] ,[(1 - 00(1 - ay) + (a' - 7 )(1 - a)] + [-(1 - a')(l - 0)7 + K - 7)(« - 7)]« [(1 - 0/7)(1 - 07) - (1 - a')(l - 0)7] - [(1 - a ' 7 ) ( l - a ) 7 + (1 - a'){a - 7 ) 7 ] s _ (1 — 7)(1 — aa') + (1 — 7)(aa' — 7)5 (1 — 7)(1 — aa ' 7 ) — (1 — 7)(1 — aa ' )75 (1 — aa') + (aa' - j)s (1 — aa ' 7 ) — (1 — a a ' ) 7 « = Gxis^aa'). Hence, K is self-generalized with respect to a. When 7 = 0, this becomes Example 3.1. 74 Example 3.3 (13): Let K be a right-shift power series random variable, taking values in {1,2,3,...}. The pgf is GK(s;a) = l-(l-s)a, 0 < a < 1. It follows that GK(GK(s;a);a') = 1 - (1 - GK(s; a)f = 1 - ((1 - s)a)a' = 1 - (1 - s)aa' = GK(s;aa'). This shows that K is self-generalized. Example 3.4 (14): Suppose the non-negative integer random variable K has pgf GK{s;a) = c-l[l-e-^l-a\l-cs)% where 0 < a < 1, c = 1 — e~e, 8 > 0. The parameter 8 is fixed. Then GK(GK(s;a);a') = c^l - e ^ - a ' \ l - cGK(s; a))a'} = c-l[l-e-e^(e-e^-a\l-cs)a)a'] = C-l[l - e-^l-a'+a'-aa')^ _ ^ j a o / j = c - 1 [ l - e - * ( 1 - a a ' > ( l - c s ) Q a ' ] = GK{s;aa'). Thus, K is self-generalized with respect to a. Since lim c _ 1 [ l — e~6(l~a^(l — cs)a] = 1 — a + as, the lower boundary leads to Example 3.1. Example 3.5 (15): Consider the non-negative integer random variable K which has pgf GK(s; a) = l - a 8 ( l - j)0 [(1 - ah + (1 - 7 ) U - s)~1/d} ^ , where 0 < a < l , 0 < 7 < l and 8 > 1. Here the parameter 7 and 8 are fixed. Then, it follows that GK(GK(s;a);a') = 1 - ( « ' ) * ( ! - 1)e \(1 - a'h + (1 - 7)(1 - GK(s;a))"1/*! l - ( « r ( l - 7 ) 6 (1 - a'h + « _ 1 ((1 - «)7 + (1 - 7)(1 - s)~1/8) 75 = l - ( a Y ( i - 7 ) 6 ( l - a a ' ) 7 + ( l - 7 ) ( l - a ) _ 1 / 0 a = 1 - (aa')9(l - j)9 [(1 - aa'h + (1 - 7)(1 - s)'1^9 = Gft-(s;aa'). Hence, K is self-generalized with respect to a. When 6 = 1, the pgf becomes GK(s; a) = l- a(l - 7 ) [(1 - a ) 7 + (1 - 7)(1 - s)"1]~' = ^ ^ Z ^ which is the pgf of Example 3.2. Therefore, Example 3.2 is a special case in this family. We summarize the existing relationship among these classes: I I C 12 C 15 and I I C 14. 3.1.2 P o s i t i v e case a n d e x a m p l e s In this section, we define self-generalizability for positive rv's. De f in i t i on 3.2 Let A be a subset of reals that is closed under multiplication. Suppose K has cdf F(x;a) depending on a parameter a, a € A. The Laplace transformation of K is bK(s;a) = E[e -sKi If $ K {-log <j)K(s; a); a') = <f>K(s;aa'), then the distribution family {F(x;a);a € A} is said to be self-generalized with respect to the pa- rameter a. For convenience, we say that K is self-generalized with respect to the parameter a to refer to the self-generalizablity of the distribution family {F(x; a); a € A}. In positive rv case, self-generalizability is closed under the negative logarithm-compound op- eration for the LT. This seems quite different from non-negative integer case, where self-generalizability 76 is closed under compounding for the pgf. However, recalling that GK(s; a) = E (sK) = E (e ( l o e s ) K ) 4>K{ — logs; a), one can induce from GK{GK(s]a)\a') = GK(s;aa') to </>/f(-log - logs; a); a') = <j>{-log s; aa'). Replacing — log s with s, we see that the non-negative integer self-generalized distribution still satisfy the definition for positive rv case. This implies that both definitions are the same in principle. Of course, we can use the definition regarding LT to unite both cases; however, the pgf is more convenient than the LT for the non-negative integer case. Similarly, this kind of closure of LT with respect to a parameter corresponds to another inter- esting stochastic representation (see property 3.7) leading to the terminology of self-generalizability. The following are five positive rv self-generalizability examples. Similarly, we label them from P I to P5; they form pairs with II to 15. Example 3.6 (PI) : Suppose K is a degenerate rv on point a (a > 0). Then the LT of K is (f>K{s;ot) = e~as. It is easy to check self-generalizability, because where 0 < a < 1, 0<7<1 and 7 is fixed. This is the LT of a compound Poisson distribution with exponential rv's. It follows that 4>K(-log (j)K{s; a); a') = e -a'[-\og(pK(s;a)} _ e~a'[as] _ g - . Example 3.7 (P2): Suppose K is a positive random variable with LT 77 = exp < - ( l - 7 ) = <f>K{s\aa'). aa'(l — 7)s 1 + (1 - aa'^s J Hence, K is self-generalized with respect to a. When 7 = 0, this becomes Example 3.6, namely P I . Example 3.8 (P3): Let K be positive stable with LT ^K(S;O) = exp{—sa}, where 0 < a Then M~ log <t>K(s;a);a') = exp { - [- log <j>K(s; a)f } = exp { - [saf } = exp {-s a a'} = <f>K (s;aa'). Thus K is self-generalized. Example 3.9 (P4): Consider positive random variable K with LT 4>K(s;a) = exp < - - e8 - 1 where O < a < l , 0 > O and 9 is fixed. It follows that f [l + ( e 0 - l ) { - l o g ^ ( g ; a ) } ] a ' - l 4>K{-\og<t>K(s\a)\a ) = exp< exp < a ' ^ - 1 = expj- = 4>K(s;aa') Thus, K is self-generalized with respect to a. e° - 1 [1 + (e* - 1 )*P ' - 1 ee - 1 Since lim exp 0->o |_[1 + ( e^H—_ j = e - Q S , tfie lower boundary leads to Example 3.6. 78 Example 3.10 (P5): Consider positive random variable K with LT A t \ J I" "(1-7) (f>K{s\a)=exp<- — r y [(1 - 0)7 + (1 -7)s « where 0 < a < l , 0>1 and 0 < 7 < 1. The parameters 9 and 7 are fixed. Now we check the self-generalizability. a ' ( l-7) <f>K(-log<i>K(s;a);a') = exp < - (l-a ,)7+(l-7)[-log^(a;a)] * exp < a'(l - 7) exp = exp < - (1- a')7 + ( l - 7 ) a ' ( l - 7 ) a ( l - 7 ) L ( l -a)7+( l -7)s * , ( l - ^ ) 7 + ( l -7 ) ( 1 - a y iV" r a a ' ( l - 7) exp a ( l - a')7 + (1 - «)7 + (1 - 7) s" aa'(l — 7) (1 - aa')7 + (1 - 7)s « J TTiis implies that K is self-generalized. When 9 = 1, the LT will be a(l - 7) 4>K{S;O>) = exp T | =exp[- a ( l — 7)5 a)7s } ' ( l - a ) 7 + ( l - 7 ) * - 1 J " " I ( l - 7 ) + (l lo/iic/i is the LT of the Example 3.7. Hence, Example 3.7 is a special case in this bigger family. The relationships among these classes are: P I C P2 C P5 and P I C P4. 79 3.2 Properties of self-generalized distributions In this section, we shall discuss some properties of the proposed self-generalized distributions in proceeding section. These involve the properties of their means, boundaries, as well as possible stochastic representations for the compounded pgf and LT. For non-triviality we assume that the distributions of a self-generalized family {FK(; a); a G A} are distinct for different a G A. Thus, trivial cases like K being a constant 0 for the entire family are excluded. Theorem 3.2.1 Suppose K is a self-generalized random variable. The expectation of K is: dGK(s,a) j l s=l if K is positive rv. h(a)=B(K) = ds d4>K(s,a) ds , if K is non-negative integer rv, =i s=0 Then h{a)h(a') = h{aa'). (This is the Cauchy functional equation.) Proof: Taking partial derivative with respect to s for both sides in the self-generalizability definitions, by the chain rule, we obtain dGK(Gk(s;a);a') dGk(s;a) _ dGk{s;aa') dGk(s;a) ds ds and d(j)K{-1og^K{s;a);a') ^ d(-log (fa-(s; a)) _ d<j>K{s;aa') d(-log<fa(s;a)) ds ds The latter can be further written as d<fa(--log<fa-(s;a);a') ^ ( 1 d<j>K{s\a)\ = d<j)K(s;aa') d{-log<fa(s;a)) ~ V <J>K{S\O) ds J ds By setting s = 1 or s = 0, we can obtain the related equations regarding to the expectations associated with parameter values a and a' for non-negative integer and positive self-generalized distributions respectively. 80 Since GK(l,a) = E{1K) = E(1) = 1 and <^(0,a)=E(e -QxK )=E(1) = 1, we have dGK(Gk(s;a);a') dGk{s;a) s=l dGK{si;a') = h(a') and d(j>K (-log (PK(S; a);a') d{-\og <j}K(s;a)) s=0 dS2 = -h(a'), where Si = Gk(s;a) and s2 = - log <$>K{S; a) respectively. Thus, for the non-negative integer self-generalized distribution, it is straightforward to obtain which simply leads to h(a)h(a') = h(aa'). To distinguish the self-generalized random variables with different values of parameter a, we adopt X(a) to denote the one corresponding to a. Hence, X(a) and X(a') will be from the same self-generalized distribution family, but with parameter values a and a' respectively. Since the closure property of self-generalized distribution is with respect to the parameter a, i.e., aa' 6 A, the possible domain A for a are the real set, or the intervals [^1,1] and (—oo, —1]U [1, oo) (including or excluding the boundaries), or the positive real set, or the intervals [0,1], [1, oo) (boundaries could be excluded). Note that reparametrizing by taking inverse, there is one-to-one mapping between (0,1] and [l,oo), and such a reparameterization keeps the self-generalizability. This feature can be seen in the following reseasoning. Suppose G*K(s;a) = GK(s;l/a), where a E (0,1]. Then l/a € [l,oo), and G*K(G*K{s;a);a') = GK{GK(s; l/a); l/a') = GK(s;l/{aa')) = G*K(s,aa'). Hence, (0,1] and [1, oo) are equivalent. We only need to consider domain (0,1]. However, we can't find a one-to-one mapping between (0,1) and (0, oo) such that the self-generalizability is kept. h{a)h(a') = h{aa'). For the positive self-generalized distribution, we first have the following equation -h(a') x h(a) = -h(aa'), 81 In the remainder of this section, we are only interested in non-negative set A: [0,1] and [0, +00). In fact, the theory of continuous-time generalized AR(1) processes only needs A = [0,1]. The boundary 0 may be excluded, but the boundary 1 is always included in A through the remainder of this thesis. The inclusion of 1 has been justified from the definition of self-generalizability. This property plays an important roles in the theory of continuous-time generalized AR(1) processes. Property 3.1 Let K be a self-generalized rv. Then, GK(S'A) — s or </>#-(s;l) = e~s, that is, K(a) = 1 for a = 1. Proof: We consider the discrete and positive case respectively. (1) Discrete case. GK{S\O) is increasing in s for any a G A. Hence, from GK(GK(S;1);<X) = GK(s;a), for all s, we conclude that either GK(S;1) = s or GK(S;CX) = 1. However, GK(S\O) = 1 means that K(a) takes value 0 with probability 1 for all a. This contradicts the non-triviality assumption. Therefore, the only choice is GK{S\ 1) = s. (2) Positive case. 4>K{S\ ct) is decreasing in s for any a € A. Since <J>K{-log 0̂ 0; a); 1) = 4>K(S; a), for all s, either — logcj^s; 1) = s or 4>K(S~, a) = 1 holds. However, the latter implies that <[>K(S; 01) = 1, contradicting to the non-triviality assumption. Thus, <J>K{S; 1) = e~s- Both cases imply that K(l) = 1. With extra conditions, we can obtain the functional form of the expectation of a self- generalized rv by Theorem 3.2.1. Property 3.2 Suppose h(a), the expectation h(a) of a self-generalized rv K, is continuous with respect to a. Then h(a) = ar. 82 Here r can be positive or negative (r = 0 is eliminated to avoid triviality). If h(a) is bounded in (0,1), then r > 0. If h(a) is finite but unbounded in (0,1), then r < 0. Proof: Under the continuity assumption, it is straightforward to deduce h(a) = ar by Theorem 3.2.1. Excluding the trivial case, we know that r > 0 or r < 0. h(a) = ar will goes to 0 or oo according to r > 0 or r < 0 respectively. This completes the proof. Note that h(a) may not be finite when a ^ 1. See the cases of (13) and (P3) in the following example. Example 3.11 Checking the non-negative integer and positive self-generalized distributions in last two sections, we find h(a) = a for II, 12, 14, PI, P2 and P4. For 13, the power series distribution, and P3, the positive stable distribution, the expectations are infinity, i.e., h(a) = oo when 0 < a < 1. For 15 and P5, h(a) = oP. Also see the summary for the mean and variance of self-generalized distribution in Table 9.1. Note that if K(a) has finite expectation for all a > 0, namely E [iv"(a)] = ar, r > 0, then K(a) can be reparameterized by allr so that E [K(Q)] = (allr)T = a. This is because that for the reparameterization transformation, allT(a!)llr = (aa')llr, is closed under multiplication. Property 3.3 Let K be a self-generalized rv. (1) If the boundary 0 is included in the domain A of parameter a, then GK(S; 0) = 1 or <fa(s; 0) = 1, that is, K(a) = 0 for a = 0. (2) If the boundary 0 is not in domain A, but the expectation of K is bounded and continuous P with respect to a, then K(a) —> 0 as a —> 0. Proof: (1) The boundary 0 is included in the domain A. Then by self-generalizability, it follows that for a e A, GK(GK(s;a);0)=GK(s;b), for all s, 83 and <fa(-log <fa(s; a); 0) = (f>K{s;0), for all s. Because of the monotonicity of GK{S; a) and 4>K{S\ &) with respect to s, the above equations yield that GK(S',CX) = S or GK(S;0) = 1, and - log (j>K(s; a) = s or <fa(s;0) = 1. But Gftr(s;a) = 5 and (f>K{s;a) = s will lead to the triviality that K(a) = 1, thus, it must hold that GK(s;0) = 1 and <fa(s;0) = 1, namely K(0) = 0. (2) The boundary 0 is not included in the domain A. Then by Property 3.2, lim E[K(a)] = 0. p By non-negativity, we obtain that K(a) ——10 as a —> 0. The support of self-generalized rv is of interests. Below is the feature of support of a non- negative integer self-generalized rv. Property 3.4 Suppose Gx(s;«) = Po(a) +pi(a)s-| \-pn(a)sn, with n > I, and pn(a) > 0 for all a ^ 1 if n is finite. Then the order n is either 1 or oo. This is because that the polynomial degree of GK(GK{S\ a); a') will be n2. Only 1 or +00 are possible choices. Therefore, any distribution with domain in a finite non-negative integer set other than {0,1}, such as Binomial distribution, can not be self-generalized. The pgf GK{S]O) and the LT <fa(s; a) are uniformly continuous in s on their range [0,1] and [0,00] respectively. How about their continuity in a? This leads to the following conclusion. Property 3.5 Let K be a self generalized rv. (1) For K being a non-negative integer-valued rv with pgf G K ( S ; a), if GK{S\ a) is left continuous at a = 1, then GK(S; a) is continuous in a in (0,1]. Furthermore, if GK{S;OI) is right continuous at a = 0 and lim GK(S;(X) = 1, then Gft-(s;a) is uniformly continuous in a in [0,1]. 84 (2) For K being a positive-valued rv with LT 4>K{S; a), if 4>K{S; a) is left continuous at a — 1, then 4>K{S',O>) is continuous in a in (0,1). Furthermore, if (J>K(S;O) is right continuous at a = 0 and lim (f>n(s;a) = 1, then <pK{s;a) is uniformly continuous in a in [0,1]. Proof: (1) Suppose a' < a. It follows that GK(s; a) - GK{s; a') = GK{s; a) - GK{s; Ba) = GK(s; a) - GK{GK(s; 3); a), where B = a'/a. When a' —> a, 3 —> 1. Since lim GK{S\O) = s, thus, GK{S\B) —> s. By the continuity of a pgf in s, we know that Gxisya) — GK(GK(S] 8)\a) —> 0. This implies that GK{S; a) is continuous in a in (0,1). If GK{S\ a) is left and right continuous at its two boundaries of a, then GK{S; a) is continuous in a in the closed interval [0,1], which shows that GK(S;O) is uniformly continuous in a in [0,1]. (2) Applying the same reasoning, we can obtain the similar conclusion for <J>K{S\ «). Remark: For 13, K does not have finite mean, and in fact, the right limit lim GK{S;O) = 0; this is not a pgf. Similarly, for P3, K does not have finite mean too, and the right limit lim <PK(S; a) = e - 1 , which is not a LT. In both cases, the pgf or LT is left continuous at a = 1. As to II , 12, 14, 15 and P I , P2, P4, P5, K has finite mean, and its pgf or LT is continuous at boundaries a = 0 and 1. Stochastic representations of GK{GK{S;a);a') and <J>K{— log<J>K{S\a);a') are of interest. Here we discuss their possible representations. Property 3.6 Suppose K(a) and K(a') are distributed from the same non-negative integer self- generalized distribution family with respective parameter values a and a!. Then Y^i=o ^ Ki(a) has pgf GK(GK(s;a);a'), where KQ(Q) = 0; Ki(a) FK{-\a) and are independent of K(a'). 85 Proof: K(a')l Property 3.7 Suppose the positive self-generalized rv K(a) has the LT (pK(s;a). K(a') is from the same family but with parameter value a'. Let (J^(*);* > 0} be a process with stationary and independent increments, and assume that Also suppose that K(a') is independent of the process {Ji({t)\t > 0}. Then J{K(a')) has the LT <fa(-log 4>K{s;a);a'). Proof: Since the rv of self-generalized distribution can be decomposed as sum of any number of iid rv's from the same distribution (see Property 3.6 and 3.7), it arises an interesting question: is the self-generalized distribution ID? We give a brief conclusion here. Suppose K has the self-generalized distribution. If K is positive-valued, according to Prop- erty 3.7, </>̂ -(s;a) is a LT for any t > 0. Thus, K is ID. If K is non-negative integer-valued, by Property 3.6, G^-(s;a) is a pgf for ra = 1,2, However, it is not clear whether this is true for 0 < n < l o r n > 0 . Hence, it may or may not be ID. For example, K from II is obviously not ID. It is also a boundary case in 12, 14 and 15. Thus, we know that at least some members in 12, 14 and 15 are not ID. There could exist ID members in these classes; their ID features can be verified by Theorem 2.2.6, or the absolute monotonicity of M'K(s; OI)/MK(S; a). A few more properties of the self-generalized distribution are given below. Property 3.8 Let K be a self-generalized rv. Suppose A is (0,1] or (0, oo). <PjK(t)(s)=V{e-sJK{t)} = Ms;<x), *>0- 86 (1) Discrete case. Suppose GK is left differentiate in a at 1. Let H(s) = %j*-(s;a)\ . // a=l H(s) < 0 for 0 < s < 1, then GK(S',CX) is decreasing in a for all 0 < s < 1. Similarly if H(s) > 0 for 0 < s < 1, i/ten G K ( S ; a) is increasing in a for all 0 < s < 1. (#,) Positive case. Suppose 4>K is left differentiate in a at 1. Let H(s) = ~dl2^K (s; a) . If H(s) > 0 for s > 0, then cf>x(s;a) is decreasing in a for all s > 0. Proof: (1) Fix 0 < s < 1. H(s) < 0 implies that GK{s\B) > s for all Bs < 8 < 1 for some 8S > 0. Let a' < a. There exists a positive integer m and Bs < 8 < 1 such that a' = aBm. Note that GK{S; S) is increasing function of s, GK(s;58) = GK(GK(s;B);6) > GK(s;8), 0 < S < 1. Hence by induction GK(S;(X') > Gx(s;a) or GK(S;a) is decreasing in a. (2) Fix s > 0. i f (s) > 0 implies that -\og</>K{s;B) < s for all Bs < 8 < 1 for some & > 0. Note that 4>K(S;88) = $K(-logct>K(s;8);5) > </>K{S;S), 0 < S < 1. The completion of the proof is like case (1). Property 3.9 Let K be a self-generalized rv. (1) Discrete case. Suppose GK{S; a) is decreasing in a £ (0,1] for 0 < s < 1. Then Fi[K(a)] < 1. (2) Positive case. Suppose 4>K{S\OL) is decreasing in a £ (0,1] for s > 0. Then E[iv"(a)] < 1. Proof: (1) Since GK(S; 1) = s, the supposition implies GK{S\OI) > s. Hence GK(l;a) - GK(s;a) < 1 - s _ 1-s ~ 1-s ' Take a limit as s 11 to get the conclusion. 87 (2) The supposition implies <f>K(s;a) >es. Hence 4>K(0;C<) - <t>K{s\ct) < 1 - e~ s • ~ s Take a limit as s | 0 to get E[K{a)] = -ct>'K{^a)<\. 3.3 Construction of new self-generalized distributions Exploring new self-generalized distributions is quite meaningful and challenging. In this section, we summarize some approaches leading to new self-generalized distributions, and conclude with results/conjectures regarding the relationship between non-negative integer self-generalized and positive self-generalized distributions. A function g : 3ft x 3ft —>• 3ft, g(x; y), satisfying g(g(x]y);y') = g{x;yy'), is called a self-generalized function. We can search for non-negative integer self-generalized distri- butions in the family of self-generalized functions. If a self-generalized function G(s; a) is a pgf in s, then it is the pgf of a self-generalized distribution. With this idea, we have the following results. Theorem 3.3.1 Suppose gi{x) is a monotone real-valued function, and its inverse gT1 exists. Let 9i{x;y) be a self-generalized function. Then g(x;y) = g7l (g2 (gi(x);y)) is another self-generalized function. Proof: A direct calculation shows g{g{x;yi)\y2) = g^1 (g2 {gi(g(x; yx)\y2)) = gT1 (92.(51 (gi1 (92 (9i{x);yi));2/2)) 88 Table 3.1: Some results from Theorem 3.3.1. 9i{s) 92(5;") 9i  1 (92 {9i(x);y)) l - 7 1—7s I _ l z i r i 7 7 (1 - a) + as (1—a)+(a-7)s ( l - Q 7 ) - ( l - a ) 7 « -0 - x log[ l - (1 -e-6)s] l-e-O' (1 - a) + as l _ e - A ( l - a ) f 1 _ ( 1 _ e - t f ) s ] a l - e -« l-(l-s)1'0 1 - ( ! - * ) * (l-a)+(a-7)s ! a" (.1-7)" (1—07) — (1—a)7s 1 fd-aW+fl^fl-s)- 1 /"!" = 9'1 (92(92 (gi{x);yi);y2)) = gi1 (92 (910*0; 2/12/2)) = g(x;ym)- Hence, y) is a self-generalized function. Certainly, g\ can be chosen as a pgf, and g2 a self-generalized pgf. Examples given by this approach can be found in Examples 3.2, 3.4, 3.5 illustrated in Table 3.1. Theorem 3.3.2 Suppose g\(x\y) is a self-generalized function. Then (1) g(x;y) = (gi(x~1;y)) 1 is a self-generalized function. (2) g(x;y) = 1 — 31(1 — x;y) is a self-generalized function. Proof: We verify their self-generalizability by direct calculation. (1) g(g(x\y\)\y2) = (91 ( g ( g ^ y i ) ; ! / 2 ) ) = (91 (9i(^ _ 1;yi);y2)) 1 = (91 ( z - 1 ; 2/12/2)_1 = g(x;yiU2). (2) 9(9^; 2/1); 2/2) = l - 91 (1 - g(x\y\)\y2) = 1 - gi (gi(x;yi);y2) = 1 - 91 (z; 2/12/2) = g(x; 2/12/2)- 89 Next we study the analogues between non-negative integer self-generalized and positive self- generalized distributions. This extends an idea of McKenzie [1986]. The following result describes analogous features between these two kinds of self-generalized distributions. Theorem 3.3.3 Suppose GK(S; a) is the pgf of a non-negative integer self-generalized rv K. Sup- pose GK{-\O) can be extended to domain (—oo, 1] with self-generalizability Gfc{Gfc{s;a);a') = GK(s\aa.') for all 0 < a,a' < 1. Let c/)(s; a) = exp {GK(1 - s; a) - 1} , s > 0. If cj)(s; a) is a LT, then it is the LT of a positive self-generalized distribution. Proof: We need to check the self-generalizability of <f>(s;a). By definition, it follows that log^(s; a) = GK(1 - s;a) - 1. Thus, (/>(-log (j>{s; a); a') = <f> ( l - GK(1 - s; a); a') = exp{G* ( l - [ l - G * ( l - a ; a ) ] ; a ' ) - l} = exp{GK (GK{1~ s;a);a')-1} = exp {GK(1 - s; act) - l} = cp(s;aa'). Examples 3.1 to 3.5 are just the analogues of Examples 3.6 to 3.10 respectively. The resulting positive self-generalized rv denoted as K' has expectation and variance: E(i l") = -^'(0;a) = G'K(l;a) = E ( K ) , Var( lC) = 0"(O;a)-f>'(O;a)) 2 = G"K{1;a) + {G'K(l;a)) 2 - {-G'K(1;a)) 2 = G"K{l;a) = V&r{K)+E2{K)-E{K). Furthermore, we have the following open questions. 90 Conjecture 1: If G(s\a) is the pgf of a non-negative integer self-generalized distribution, then (f>(s; a) = exp{G(l — s; a) — 1} is the LT of a positive self-generalized distribution. To show (f)(s; a) is a LT, we need to: (1) extend the range of s in G(l — s; a) from 0 < s < l t o s > 0 , (2) prove the completely monotone property of (/>(s; a). For (1), it is equivalent to extend the domain of s in pgf G(s; a) from 0 < 5 < 1 to —oo < s < 1. This is fine for the interval — 1 < s < 0. As to (2), the completely monotone property holds for 0 < s < 1, but this is not clear for s > 1. However, there is no need for domain extension if we define the pgf for the discrete analogue by the LT of a positive self-generalized distribution. Thus, under minor conditions, the counterpart of Conjecture 1 holds. This leads to the following theorem. Theorem 3.3.4 Let <fa(s;a) be the LT of a positive self-generalized distribution. Define G(s;a) = log (fa (1 - s ; « ) + 1. 0 < s < 1, 0 < a < 1. Suppose (1) <fa(l;a) > e~l, 0 < a < 1, and (2) G(s;a) has a Taylor series expansion in s. Then G(s;a) is the pgf of a non-negative integer self-generalized distribution. Proof: First we check the self-generalized condition: G(G(s;a);a') = log<fa(l - G(s;a);a') + 1 = log<fa(- log<fa(l - s; a); a') + 1 = log (fa(l — s;aa') + 1 = G(s;aa'). Next note that G(s;a) = log <fa(l - s;a) + 1 is increasing in s, G(0;a) = log<fa(1; a) + 1 > -1 + 1 = 0, G(l;a) = 1. On the other hand, G(s; a) = GK'(S; a) = E ( > » ) is a proper pgf iff <fa/(s;a) = GK,(e~s;a), s > 0, 91 is a proper LT. We will show that <pK'{s; a) is completely monotone. Note that . ^ ( s ; a ) = l o g ^ ( l - e - * ; a ) + l . (3.3.1) K infinitely divisible implies that (by Theorem 2.2.6) the derivatives of x(s) = — l°g <t>K {s; a) alternate in sign, that is, (—1)J'-1X , ,'(S) t 0- Then UJ(S) = ui(s;a) = x ( l - e~s;a) has the same property: u'(s) = x ' ( l - e") e-s > 0, U"(s) = x"( l - e~s) e~2s - x ' ( l - e"*) e~s < 0, and the derivatives of each term of the form x ^ ( l _ e _ s ) e _ m s will continue to be opposite in sign to the original term. Hence <f>K'(s;a), given in (3.3.1), is completely monotone. Finally, <j>K>(s;a) is the LT of an nonnegative integer-valued rv, if GK'{S\OI) has a Taylor series expansion. Because of condition (2), we know that (pK'{s; a) is a LT, and consequently G(s; a) is a pgf. This completes the proof. Theorem 3.3.3 and Theorem 3.3.4 disclose the relationship between a self-generalized oper- ator for positive reals and one for non-negative integers. 3.4 Extended-thinning operation In this section, we propose an extended-thinning operation which is one of the essentials to the model construction of continuous-time stochastic processes with given univariate margins. This extends binomial thinning (see (2) in Proposition 2.2.2). In fact, we hinted at this topic in Section 2.3, where we studied the stochastic representations of the compound self-generalized pgf and logarithm-compound self-generalized LT. Now we study the stochastic operation between two independent rv's X and K, which have LT's 4>x(s) and 4>K{S) respectively. We wish to define the operation: K © X , such that its LT has the form (j>K®x{s) = E e-s(K®X) = <j)X (-log0*(s)). 92 We shall give the stochastic representation of this definition in three cases where X is non-negative integer-valued, positive-valued, and real-valued respectively. Case 1: X is a non-negative integer rv. Define a discrete-time process {J^( i ) ; i = 0,1,2,...} independent of X as t JK(t) = Y,Ki, i=0 where K0 = 0, K\,..., Ki,... are iid with LT: <j>Ki (s) = <J>K(S)- Let x K®X = JK(X) = J2Kii i=0 the random summation over the process {J/<(£); t — 0,1,2,...}. Direct calculations show that E(e-°Z?=0Ki\x)]=-E[<f>x-(s)] = <j>x(-log<f>K(s)). 4>K®X{S) = E = E 0-s{K®X) E - ( - l o g ^ ( « ) ) X The illustration can be seen in (a) of Figure 3.1. Case 2: X is a positive rv. Consider a continuous-time process {Ji<-(£);t > 0} independent of X which has stationary and independent increments, such that the LT of J K W is: <t>jK{t){s) = 4>K(*),. *>0- Define Then K®X = JK(X). <I>K®X{S) = E -s(K®X) = E = E [^(s)} =E E (^es(K®X) e-(-log0jc(*))x" X ) ] =E [E (e-8jK{x)\x) = 4>X ("log 4>K (s)) • See (b) in Figure 3.1 for the illustration. One example of the defined process {.//<-(£); i > 0} is that in the family of Levy processes with LT (t>jK{t)(s) = e x p | i -as + jo {e~sy - l)L(dy) j , 93 where L(-) is the Levy measure. Certainly, in this case, the LT of the K is exp[ -as + j™{e~sy - l)L(dy) j . Case 3: X is a real rv. Consider two stationary independent increment processes {Ji(t);t > 0} and {J2(t);t > 0} independent of X with LT <l>Mt)(s) = 4>K(S) a n d ^ J 2 ( t ) ( s ) = respectively. Note that (j>j2(t){s) 1S ^ e reciprocal of ^ (^ ( s ) . Hence, under the requirements of a LT, 4>K{S) can not be arbitrary. Construct a new process over the whole real axis {Jx{t);t G (—oo,+oo)} such that Mi) = { For this new process, the LT of J/c (t) is <t>JK(t)(s) = E M\t\) = < if t > 0; if t < 0. t > 0; t < 0; Define Then K®X = JK(X). <t>K®x{s) = E -s(K®X) E E(e-sJ«W\x) = E [^(s)] = 4>x (— log (J>K(S)) • See the illustration in Figure 3.2. In Case 3, process {J2(t);t > 0} is in fact artificially developed by process {J\(t);t > 0} for (f)j2(t)(s) — [ < / ) J i ( t ) ( s ) ] - 1 - Hence, J\(t) and Jiit) can not both be positive, because (f>j^t){s) and 4>j2(t){s) both be bounded above by 1 and can't satisfy the completely monotone property at the same time. Suppose we restrict the process {J\(t);t > 0} to the Levy process family, with ^ J l ( t ) (a) = exp[t - a s + jo (e~sy-l)L(dy) j 94 Figure 3.1: Illustration of{JK(t); - o o < t < 00} in Cases 1 and 2. (a) corresponds to non-negative integer X in Case 1, where dotted vertical lines indicate the discrete time points {0,1,2,...}. (b) corresponds to positive X in Case 2, where t is continuous on [0,00). 95 (a) J2(t) / — X 0 t (c) gure 3.2: Illustration of {Ji(t);t > 0}, {Ji(t);t > 0} and {Jk(t); -oo < t < oo} in Case 3. 96 /•oo where L(-) is the Levy measure, non-negative and satisfies / min(l, y)L(dy) < oo. Also assume Jo that the process {J2{t);t > 0} is a Levy process. Then <t>Mt)(s) = [ ^ J i ( t ) ( 5 ) ] _ 1 = exp j i Since —L(dy) < 0, the only possible choice is L(dy) = 0. This implies 4>Jl{t)(s) = e-ats, and 4>Mt)(s) = eats, for some constant a, i.e., Ji{t) = at, and J 2 W = —at, degenerate at points at and — at respectively. In summary, we propose the extended-thinning operation as below. Definition 3.3 Suppose {Jx{t);t £ To} is an appropriate stationary independent increment pro- cess constructed via rv K such that ' 0J , r ( t ) ( s ) = ^Kis), where To could be {0,1,2,...}, or [0,oo) or (—00,+00) (refer to cases 1, 2 and 3). The extended- thinning operation is defined as a stochastic operation between JK and X with X independent of {JK(t)}, K®X = JK(X). Such an operation results in a rv with LT (f>K®x(s) = 4>x (-log<Ms)). The notation K® means an independent copy of rv K, which has the same distribution as that of K. Hence, notations K © X and K ® Y do not means the rv K in both is the same, but their distributions are the same. Note that K and X may not be arbitrary random variables, restrictions on them in the different domain cases should be imposed. In other words, <j>x ( _ l o g 0 K ( s ) ) being a LT requires conditions on 4>K and <j>X- as poo / ( ( Jo -sy i)(-L)(dy) 97 We can calculate the cf of K ® X: <PK®x(s) = E [eisK®x] = cj>mx{is) = <l>x ( - l o g ^ ( w ) ) 4>x (— log IPK(S)) , if X is non-negative; = < <px (i log > if X is real. A natural property regarding expectation is given next. Property 3.10 IfE[K(a)] is finite and continuous with respect to a, i.e., ~E[K(a)] = ar, then E [K(a) ® X] = E [K(a)] • E [X] = arB [X]. Proof: This can be readily derived by taking the derivative of the LT. Hence, E[K(a)®X] < E[X] if a is within [0,1] and r > 0. In general, the extended-thinning operation rescales the expectation. Now we look into the examples of extended-thinning operation in statistical practice. Example 3.12 The well-known binomial-thinning is one special case of extended-thinning opera- tion, for x a * X = ^ i f i , K0 = 0, K\, Ki, ... ' Bernoulli(a). One special feature of binomial-thinning is that a*X < X, which means a*X does become "thinner" than X almost surely. However, in general, the extended-thinning may not retain this feature, but the expectation is "thinner" than E[X] if we restrict the domain of parameter a to [0,1]. Example 3.13 A branching process has an operation similar to the extended-thinning operation. This kind of processes {X(n) : n = 0,1, 2,...} is defined as X(n) X ( n + 1) = ^ Zi, Z0 = 0, Zu Zl, ... iid. X(n) is the size of the nth generation. 98 Example 3.14 The product of a constant a with a rv X, aX, is another example. This may not be straightforward at the first glance. However, we can check its LT. In this case, we can view a as a rv degenerate at point a. Hence, it has LT: (f)a(s) = e~as. The LT of aX, then, is <f>aX(s) = E {e~saX) = <f>x{as) = <j>x (- log <f>a(s)). Therefore, aX is an extended-thinning operation. We use the notation '©' for the extended-thinning operation based on the consideration to unite the constant multiplier '• ' and the binomial-thinning operation '*', in a simple expression, namely, (•) U (*) = > ( < § > ) . Following are two properties of extended-thinning operations; these are very important to the construction of the continuous-time generalized AR(1) processes in Chapter 4. Property 3.11 (Distributive law) Suppose K is a self-generalized rv, and let X and Y be independent rv's. Then K®{X + Y) = K®X + K®Y. Proof: Since X and Y are independent, we have <t>K®(X+Y)(s) = <t>X+Y (-log<fa'(s)) = 4>X {-^g (j)K{s)) 4>Y {-\0g(j)K{s)) . x This implies that the distributive law holds. Property 3.12 (Associative law) Let K\, K2 be two different self-generalized rv's acting as operators. Then Kx © (K2 ®X)~ {Ki ® K2) ® X. Proof: Direct calculation shows 4>Ki®(K2®X){s) = (f>K2®X (~ log (j)Kl{s)j = (f>X (-log<fa2 (-logfofi(s))) = 4>x {-^og(j)Kl@K2(s)) = cj)^Kim^@x(s). 99 Hence, the associative law holds. Recalling self-generalizability, we find that it is closed under the extended-thinning opera- tion, i.e., K(a)®K(a')=K{aa'). To keep symbolic consistency with constant multiplier • and binomial-thinning operator *, we rewrite the notation for the self-generalized rv K(a) in extended-thinning operation as ax such that (a)K ®X = K(a)®X. This change makes the extended-thinning operation with a self-generalized rv looks like constant multiplier or binomial-thinning operator. For instance, we can rewrite the closure property of self-generalizability via new notation: {a)K® {a')K = (aoi')K- But remember (O)K is a rv, not a parameter, and it is valid only with the extended-thinning operator ©. The reason we impose the subscript K on a is to try to avoid the misunderstanding of (CX)K, a rv, to the parameter a, and the convention of binomial-thinning operation as well. This new notation will benefit us immediately with the following commutative law. Property 3.13 (Commutative law) (a)K © {a')K = ( « ' ) # © (a)K- Proof: This is simply because (a)K ® (a')K = (aa')K = (a'a)K{a')K © {OL)K- Note that the commutative law only holds for two self-generalized rv's from the same family. Property 3.14 (Weak convergence) Let an -4 a, where an G A for all n, and a £ A. If GK(S;O.) or (f>K{s\a) is continuous with respect to a, then (an)K®X-^(a)K®X. 100 Proof: This follows from the continuity of GK or fa in a. Suppose A, the domain of the parameter a, is the interval [0,1]. Then on boundaries, extended-operation behaviors like the constant multiplier. Here we assume boundary 0 is included in A. Property 3.15 ( 0 ) ^ © X = 0 and. [l)K®X = X. Proof: By Properties 3.1 and 3.3, we have (0)K = K(0)±0 and (1)^ = ^ ( 1 ) ^ 1 . Hence, the resulting process {«/#(*);£ > 0} has margins JR-(O) (t) = 0, and J]<{i)(t) — *> for £ > 0. Thus, This completes the proof. Remark: If the boundary 0 is excluded from A, but the expectation of the self-generalized rv K is bounded and continuous in a, then lima_>0+(Q!)ft' ® X = (0)K ® X = Q. This is because of (2) of Property 3.3. Lastly, we discuss the variable type of K ® X : non-negative integer, positive, or real. This is basically determined by the variable type of K , not the variable type of X . Correspondingly, we study its pgf, or LT, or cf. Recall K ® X = JK(X). We have: (1) if K is non-negative integer, then {J^-(i);i = 0,1,2,....} or {JK{t);t > 0} is a process with non-negative integer increments, so JK{X) is non-negative integer no matter if X is non-negative integer or positive real. And it follows that GX(GK(S)) if X is non-negative integer; <f>x{~ log GK(S)) if X is positive, 101 (2) if K is positive, then {JK{t);t = 0,1,2,....} or {JK(t)\t > 0} is a process with positive increments, so JR(X) is non-negative no matter if X is non-negative integer or positive real. And it follows that ^ J [ fixi— log<J>K{S)) if X is positive, (3) if K is degenerate, namely being a real number a, then K © X = a l , and {</#(£);£ > 0} is a process with real-valued increments, so JK(X) is real no matter if X is positive or real. And it follows that The extended-thinning operation will be discussed again in Section 4.1, where a geometrical explanation is given. if X is non-negative integer; <PK®x(s) = -E{eiaW} = 4>x {ias) if X is positive; (px {OLS) if X is real, 102 Chapter 4 Generalized Ornstein-Uhlenbeck stochastic differential equations and their possible solutions Ornstein-Uhlenbeck stochastic differential equations is a classical topic well discussed in the liter- ature. Its applications can be found in mathematical finance, physics, and so on. Refer to Hsu and Park [1988], Neftci [1996]. In this chapter, we shall propose generalized Ornstein-Uhlenbeck stochastic differential equations, and define the corresponding generalized stochastic integration. These are fundamental techniques and key ideas in the model construction of a class of continuous- time Markov processes given in this chapter and the next chapter. We start with the introduction to stochastic differentiation and integration in Section 4.1. The generalization of Ornstein-Uhlenbeck equations will be given in Section 4.2, and the explanation and examples are shown in Section 4.3. We construct the solutions for the generalized Ornstein- Uhlenbeck equation in Section 4.4, and summarize the resulting processes in Section 4.5. 103 4.1 Stochastic differentiation and integration The dynamic feature of a continuous-time process {X(t);t > 0} is of interest, as it describes the instantaneous behavior of the process. To address this feature, we need the concept of stochastic differentiation and integration. / In the literature, some scholars explain the concept of stochastic differentiation via stochas- tic integration, while others illustrate it in terms of the infinitesimal increment of the process. For the beginner, the former is not a direct approach, and furthermore, the definition of stochastic in- tegration requires the concepts of infinitesimal increment. Hence, we shall take the latter approach. However, it may not be very strict in the mathematical sense. We just focus on main ideas. There are many references on this area, such as: Chung and Williams [1990], Lukacs [1968], 0ksendal [1995], Protter [1990], etc. A good introductory book, which clearly explains the concepts of SDE and stochastic integration without measure theory is Neftci [1996]. To clearly state the idea, let's recall the concept of differentiation in calculus, where it is defined as the infinitesimal increment. For instance, if x(t) is a function of t, denote by Ax{t) =x(t + h) -x(t), the increment of x(t) when the argument changes from t to t + h. When h, the argument increment, is infinitesimal, we denote it as dt. Correspondingly, the infinitesimal increment of x(t) is written as dx(t), which can be, expressed as dx(t) = x(t + dt) - x(t). Thus, the differential is the infinitesimal increment. For a stochastic process {X(t);t > 0}, this well-known idea can be borrowed to define the differentiation of the process. Definition 4.1 The differential dX(t) of a continuous-time stochastic process {X(t);t > 0} is defined as the infinitesimal increment X(t+h)—X(t), where the increment of time h is infinitesimal. 104 Figure 4.1: Illustration of increment in the deterministic and stochastic cases, (a) corresponds to deterministic function x(t). (b), (c) and (d) correspond to three different paths of the stochastic process {X(t);t>0}. Note that the increment of a function x(t) is a number, however, the increment of a process is a rv. Hence, the differential dX(t) is understood as a rv, not a number unless in the degenerate case. Figure 4.1 illustrates the increments in both cases. Note that in (b), (c) and (d) of Figure 4.1, the infinitesimal increment dX(t) are different because they correspond to three different paths of the process {X(t); t > 0}. This clearly shows that dX(t) is a random variable. The Riemann integration of a function x(t) over [ii,^2], the area with sign, is constructed via the infinitesimal partition approach. Here we roughly review its idea. Consider the argument 105 range [£1,̂ 2]- Divide this interval in n equal pieces, i.e., [ti,ti + h), [h + h, tv + 2h), ..., [h + (n- l)h, h + nh] = [ti + (n - l)h, t2), where h = (t2 — ti)/n. When n goes to infinity, each piece will become an infinitesimal interval. Use a finite Riemann sum over these small intervals 71-1 x(ti + ih)h i=0 to approximate the integrated "area". When n goes to 00, the limit is defined as the integration. This method was introduced to stochastic integration over a half century ago. However, the difference between common integration and stochastic integration is how to define the limit. In probability theory, the common modes of convergence include in distribution, in probability, in L 1 , in L2, a.s., etc. Hence, different stochastic integrations arise. For example, the Ito integration is the limit in L2. In our study, the convergence mode that we adopt is "in distribution". Suppose {X(t);t > 0} is a continuous-time process. We now define the stochastic integral ft'2 g(X(t))dX(t). Consider n + l equally spaced points ti,ti + h,ti + 2h,..., ti + (n - l)h,ti + nh = t2 over [£ i ,£ 2 ] , where h = (t2 — ti)/n. Let n - l Sn = 9{x(t + ih))[X(t + + l)h) - X(t + ih)}. i=0 If there exists a rv Y such that Sn Y, as n —> 00, then Y is defined as g(X(t))dX(t). This leads to the following definition. Definition 4.2 Let {X(t);t > 0} be a continuous-time process. Divide [ti,t2] into n equally small intervals. Then / g(X(t))dX(t) = lim V g(X{t + ih))[X{t + (t + l)h) - X{t + ih)}, L n—>oo ^—' t=0 where h = (t2 — ti)/n. The summation on the right hand side converges in distribution. 106 Note that stochastic integral is still a rv, not a number. Figure 4.2 shows the idea of stochastic integration via the infinitesimal partition approach. Note that the "area" in (a) and (b) may not be the same, because they correspond to two different path of the process {X(t);t > 0}. This clearly indicates that the stochastic integration g(X(t))dX(t) is a random variable. Since the increment, AX(t) = X(t + h) - X(t), of a continuous-time process {X(t); t > 0} can be written as for any positive integer n, we can rewrite the increment via a stochastic differential and integral as This kind of expression is used to formally define the stochastic differentiation by many authors. Look back at the extended-thinning where K © X = JK(X). When X is a non-negative integer rv, the extended-thinning K ® X = JR(X) = Yld=Q^i IS a random summation. What will it be if X is a positive or real-valued rv? Note that in these two cases, {Jx(t);t > 0} is a continuous-time process. Thus, a random stochastic integral. Therefore, in principle, the extended-thinning operation is a random summation or a stochastic integration. Now we make up a geometric explanation for the extended-thinning operation. Let's consider aX, a special case of an extended-thinning operation, as the area of random rectangle with length X (in the horizontal direction) and width a (in the vertical direction). Imagine a random rectangle in this way: the length is a rv. However, the width is not a fixed constant or rv. On every slice orthogonal to the length, the cutting width is a rv. Al l these cutting widths are iid rv's. One can use the sliced bread to mimic this random rectangle. Because fX dJx{t) is the limit of a Riemann sum, and in small time intervals the increments of process {«/&:(£); t > 0} are iid, a natural explanation of JK{X) is that it is the limit of sums of areas of rectangles with widths rv X/n and iid heights Kn\,..., Knn (with distribution of Jx(X/n)). This area is random. Thus, it is a new random rv. ) •t+h 107 X(t) (a) t 2 X(t) t i (b) Figure 4.2: Illustration of stochastic integration via infinitesimal partition, (a) and (b) correspond to two different paths of the stochastic process {X(t);t > 0}. 108 Figure 4.3 illustrates the idea of this geometrical explanation of the extended-thinning op- eration. We give the "imagined" random rectangles in three cases corresponding to X being (a) real, (b) non-negative integer, and (c) positive valued. Without loss of generality, we assume the mode of convergence for stochastic integration is convergence in distribution throughout the remainder of this thesis. 4 .2 Generalized Ornstein-Uhlenbeck equations Like a differential equation which expresses the dynamic characteristic of a function, a stochastic differential equation describes the dynamic feature of a continuous-time process. However, because the derivative of a process commonly doesn't exist, we can not include the derivative of a process in the equation. Instead, we include the differential of a process into the equation. Recall the Ornstein-Uhlenbeck process (see Section 2.1), which is defined by the following stochastic differential equation (SDE) for real-valued process rv X(t), dX{t) = -nX(t)dt + adW{t), where {W(t);t > 0} is a Brownian motion independent of X(t). To keep consistency with the literature, we absorb a into the innovation process so that it becomes dX(t) = -nX{t)dt + dW{t). This SDE shows that the infinitesimal increment of X(t) in the near future depends on the present circumstance and the innovation term. Note that X(t) has support on the range of (—oo,+oo). Replacing the innovation term from a Wiener process (Brownian motion) with a more general Levy process (Brownian motion is a special process in Levy process family) leads to the Ornstein- Uhlenbeck-type process (see Barndorff-Nielsen et al. (1998) and references therein), namely, dX(t) = -fiX{t)dt + dL(t), 109 (a) (b) AK (C) Figure 4.3: Illustration of the geometrical explanation of the extended-thinning operation, (a) corresponds to a constant multiplier cX; X can be either real or positive-valued, (b) corresponds to a non-negative integer X. (c) corresponds to a positive X. 110 where {L(t);t > 0} is a Levy process. This offers possible marginal distributions for X(t) with support [0,oo). However, it doesn't provide any marginal distributions with support on the non- negative integers, because /j,X(t)dt is unlikely to be an integer. The extension of innovation processes seems to be ideal. It covers distributions with domain on the non-negative integers. But only extending the innovation processes won't help us to construct models with marginal distributions having the non-negative integer support. Therefore, extending the dependence term from a product to a generalized stochastic operation may lead to a successful approach. This inspiration comes from Joe [1996]. Note that the dependence structure of such kind of processes {X(t);t > 0} is determined by., —fj,X(t)dt, and an independent innovation process is introduced to explain the fluctuation. Hence, the process is simply governed by the dependence mechanism part, —p:X(t)dt, and the independent input part {e(t);t > 0}. Recall that aX is a special operation in the class of extended-thinning operations (see Section 3.4). We can rewrite the dependent mechanism part —fiX(t)dt as -liX(t)dt = -ftdtX{t) = (1 - ndt)X{t) - X{t). Hence, a natural generalization for this dependent mechanism term is K(l - iidt) © X(t) - X(t) = (1 - fidt)K © X(t) - X{t). However, we will restrict K to be within a self-generalized family. Since this new term could be a non-negative integer, or positive, or real rv, we may hopefully obtain marginal distributions with support on non-negative integer, or positive, or real values respectively. We now formally define the generalized Ornstein-Uhlenbeck SDE below. Definition 4.3 Suppose {X(t);t > 0} is a continuous-time process, and {e(t);t > 0} is an inno- vation IIP. The generalized Ornstein-Uhlenbeck SDE is defined as dX(t) = [K(l-iidt).®X{t) -X(t)]+de(t) = [(1 - ndt)K © X(t) -X{t)] +de(t), where K(a) is a self-generalized rv with respect to parameter a. I l l Because the generalized Ornstein-Uhlenbeck SDE involves an extended-thinning operation, we should name the corresponding stochastic integration as generalized stochastic integration. Definition 4.4 Let {X(t);t > 0} be a continuous-time process. Divide [ii,^] into n equally small intervals. Then (g(t))K®dX(t) = t K(g(t))®dX(t) n-1 = lim V K(g(t + ih)) ® [X(t + (i + l)h) - X(t + ih)} n—>oo z—' i=0 n-1 = lim *y (g{t + ih))K®[X(t + (i + l)h) -X(t + ih)], n-4oo ' i=0 where h = (t2 — ti)/n, g(-) is a function with range [0,1], and K(a) is a self-generalized rv with respect to parameter a. This is well defined if the summation on right hand side converges in distribution. This generalized stochastic integration will be applied to solving generalized Ornstein- Uhlenbeck SDE in Section 4.4. 4 .3 Explanations, innovation types, non-stationary situations and examples We may give a further explanation of the generalized Ornstein-Uhlenbeck SDE in this section. In the generalized Ornstein-Uhlenbeck SDE dX(t) = [(1 - iidt)K ® X(t) - X(t)} + de{t), X(t) means present state, while dt, de(t) and dX(t) means the infinitesimal increment in the near future infinitesimal time interval. Hence, we can comprehend the generalized Ornstein-Uhlenbeck SDE as a forward expression, not a backward expression. 112 Ae = e(t+h) - £(t) X(t) X(t+h) A(X(t), -Vdt) Figure 4.4: Illustration of the mechanism of the generalized Ornstein- Uhlenbeck SDE. With such an understanding, we can write down the difference equation from the generalized Ornstein-Uhlenbeck SDE. It is X{t + h)- X{t) = [(1 -nh)K® X(t) - X(t)} + Ae, Ae = e(t + h) - e(t), which can be simplified as X{t + h) = (l-fih)K®X(t) + Ae. (4.3.1) Denote A(X(t), —fih) = (1 — (J.h)K ® X(t), the dependence mechanism. We can further write it as X(t + h) = A(X{t),-nh)+Ae. This uncovers the stochastic representation of the process in an infinitesimal time interval. Figure 4.4 roughly shows the mechanism idea of the process generated from the generalized Ornstein- Uhlenbeck SDE when h is infinitesimal. From the discussion, we know that these type of continuous-time processes are completely governed by the dependence mechanism term and the innovation term. 113 Next we investigate the innovation types: non-negative integer, or positive, or real-valued increment. From the stochastic difference equation (4.3.1), we see that the dependent term and the innovation term are independent. Besides, the dependent term can take value 0 if i i " or X(t) can be 0. Thus, we deduce the following. (1) When K is a non-negative integer self-generalized rv, X(t + h) and (1 — (J.h)x ®X(t) are non- negative integer, thus, Ae is non-negative integer. This implies that the innovation process {e(t);t > 0} has non-negative integer-valued increment. (2) When i f is a positive self-generalized rv, X(t + h) and (1 — fih) K ® X (t) are non-negative real, thus, Ae is positive. This implies that the innovation process {e(t); t > 0} has positive-valued increment. (3) When i f is a positive constant c, X(t + h) and c • X(t) are real or positive; thus, Ae is real or positive respectively. Note that positive case has been included in (2). We only consider real case. Therefore, the innovation process {e(t);t > 0} has real-valued increment. In summary, the type of the increment of the innovation process is the same as the margins of the process {X{t);t > 0}. In reality, we often encounter dynamic phenomena modelled by a process {X(t);t > 0} which could be stationary or non-stationary over time. Stationarity is a simple and natural requirement for a process model. Non-stationarity usually arises from seasonality, increasing or declining trend, heteroscedasticity, etc. Thus, appropriate model settings should be considered. A good process model theory should be able to accommodate both stationary and non-stationary situations. For the stationary case, we may just simply assume that (e(i); t > 0} is stationary, and that in the dependence mechanism term, is a constant. For the non-stationary case, we can modify either the dependence mechanism term or the innovation term to be time-dependent. Hence, the SDE becomes dX(t) = [(1 - v{t)dt)K ® X{t) - X(t)] + de{t), 114 where {e(t);t > 0} may be a non-stationary independent increment process. However, the modifi- cation should correspond to what the non-stationary situation is. Sometimes it is a time-varying marginal mean or variance, sometimes it is a time-varying autocorrelation. Finally, we look at some examples, where the innovation processes have non-negative integer, or positive, or real increments. Also the stochastic operations include binomial-thinning, and other extended-thinning operators. We just mention their SDE's to illustrate the existence of generalized Ornstein-Uhlenbeck SDE. Their solutions will be given in Section 4.4, as well as Chapter 5. Example 4.1 Let {X(t);t > 0} be a process with non-negative integer margins. Consider the binomial-thinning operation. Then the following is the corresponding generalized Ornstein- Uhlenbeck SDE: dX(t) = [(1 - fj,dt)K ® X(t) - X(t)] + de(t) = (1 - fidt) * X(t) - X(t) + de(t), where {e(t); t > 0} is a stationary Poisson process, and the increment Ae = e{t + h) —e(t) has pgf G A e = exp{^A/i(s - 1)}, n > 0, A > 0. Example 4.2 Let {X(t);t > 0} be a process with non-negative integer margins. Still consider binomial-thinning operation. But change the innovation process to be an IIP with an increment whose pgf is G A e = exp IuO-yh^-—— 1, u > 0, 6 > 0, 0 < 7 < 1. I 1 - is J Then the following is another generalized Ornstein- Uhlenbeck SDE: dX{t) = {l-iidt)*X{t)-X{t)+de(t). ' Example 4.3 Let {X(t);t > 0} be a process with non-negative integer margins. Consider gener- alized Ornstein-Uhlenbeck SDE with operator 12 (Example 3.2): dX(t) = [(1 - fidt)K ®X(t) - X(t)} + de{t) where K0 = 0, Ki, ..., Kh ...are iid, with pgf [ih + (1 - \ih - (1 - 7 - jfih) - njhs x(t) Ki - X(t) I +de(t), i=0 (i   (l-fih-7)3 • Q 0 < ^ < 1 115 f {e(t);t > 0} is a stationary Poisson process with such increment Ae = e(t + h) — e(i) that the pgf is GA(L = exp ^^r^h(s - l ) j , 0>O. > Example 4.4 Lei i > 0} 6e a process with positive margins. Suppose {Jftr(i); £ > 0} is a sia- tionary IIP such that (f)jK^(s; a) = ̂ ( s ; a) (a > 0), where <fa(s; a) = exp {- ^ J ^ ^ - a h s } ' 0 < 7 < 1. Choose {e(t);t > 0} to be a stationary IIP with positive increment Ae = e(t + h) — e(t), whose LT is - _ J _ 1 + [1 - 7 ~ A*h + 2M7^](1 ~ 7 ) - 1 g ^ A e _ l + s X 1+A*7(1 - 7 ) - 1 ^ T/ien H > 0. dX(t) = [(1 - /id*)A- © *(*) - X{t)} + de(t) = rX{t) / dJK(s) - X(t) Jo + de{t) is a generalized Ornstein-Uhlenbeck SDE with operator P2 (see Example 3.7). Example 4.5 We now return to constant multiplier operation, but choose the innovation process {e(t);t > 0} to be a stationary IIP with real increment Ae = e(t + h) — e(t) such that its cf ipAe = exp{-Xh\s\a}, A > 0, 0 < a < 2. Then dX{t) = [(1 - iidt)K © X{t) - X{t)] + de{t) = ~nX{t)dt + de(t) is a generalized Ornstein-Uhlenbeck SDE. 4 .4 Construction of possible solutions for the generalized Ornstein- Uhlenbeck SDE We define the generalized Ornstein-Uhlenbeck SDE as dX(t) = [(1 - fidt)K ® X(t) - X(i)] + de(t) for the stationary case, or dX(t) = [(1 - fi(t)dt)K © X{t) - X(t)} + de(t) for the non-stationary case, 116 where {K(a)} is a family of self-generalized rv with respect to parameter a. This shows the differential, or in other words, the infinitesimal increment of the process {X(t);t > 0} can be split into two terms: a dependence term associated with the extended-thinning operation on the current observation, and an innovation term introduced to explain the remaining fluctuation. Assuming that a solution exists, our tasks are (1) what does the solution mean? (2) how to find it? The unknown in the generalized Ornstein-Uhlenbeck SDE is the entire process {X(t); t > 0}, not just X(t) at a single time point. Thus, we need to find such a continuous-time process that satisfies the generalized Ornstein-Uhlenbeck SDE. Such a process is called the solution of the generalized Ornstein-Uhlenbeck SDE. Next we have to figure out a way to obtain the solution. For this purpose, we resort to infinitesimal partition method well known in calculus. The following is the rough idea of how this method works in stochastic calculus. Suppose the continuous-time process is {X(t);t > 0}. We study some kind of feature or behavior of this process between time t\ and t2, namely the time interval [ti, t2]. Divide this interval into n equal pieces, i.e., [ti,h + h), [h + h,h + 2h), [h + {n- l)h,h + nh] = [h + (n - l)h,t2], where h = (t2 — ti)/n. When n goes to infinity, each piece will become an infinitesimal interval. We consider the feature or behavior of the process in each small interval [ti + (k — l)h, ti + kh) (k = 1,2,..., n), and apply an approximation in each small interval. Then, we sum these approximations, and finally let n increase to infinity to obtain the limit. This resulting limit is the desired process on the interval [ti, t2]. In summary, the infinitesimal partition method applied to a continuous-time phenomena on a certain time interval involves the following steps: (1) Discretize the continuous-time phenomena by dividing the time interval into n equal small pieces; 117 (2) Carry out relevant measures such as approximation and summation over these n small intervals; (3) Make it continuous over time again for those n discretized pieces by letting n —> +00. For the generalized Ornstein-Uhlenbeck SDE, the finite difference approximation in a small interval is X{t + h)-X(t) = [(l-iih)K®X(t)-X{t)] + Ae, Ae = e{t + h) - e(t), or simply X(t + h) = (l-iih)K®X{t) + Ae, (4.4.1) for the stationary case; and X{t + h)- X{t) = [(1 - n(t)h)K © X{t) - X{t)] + Ae(t), Ae{t) = e(t + h) - e(t), or simply X(t + h) = (l-n(t)h)K®X(t) + Ae{t), (4.4.2) for the non-stationary case. These will be applied to construct the solution of the generalized Ornstein-Uhlenbeck SDE next. Before proceeding to the solution, we list a useful lemma below. Lemma 4.4.1 If {a^} is a bounded sequence such that lim n _ 1 5Zfc=o ak = a> ^en n-1 J J (1 _ £*) e~a as n -> 00. k=0 n Proof: Expand the product and take a limit term by term. In the rest of this section, we construct the possible solution {X(t);t > 0} for the generalized Ornstein-Uhlenbeck SDE. We are interested in the conditional stochastic representation form of {X(t);t>0}. 118 First consider the generalized Ornstein-Uhlenbeck SDE in the stationary case. Concretely, the setting of generalized Ornstein-Uhlenbeck SDE consists of constant parameter /A, and the in- novation being a stationary independent increment process, namely dX(t) = [(1 - ndt)K © X(t) - X(t)] + de{t). We apply the infinitesimal partition method to obtain an explicit expression for X(t2) given X(t\), where t\ < t2. Let h = (t2 - h)/n, and Ae^ = e(t\ + kh) - e(t\ + (k - l)/i), k = 1,2,... ,n. Then from (4.4.1), we have X(tl + h) = (l-Lih)K®X(t1)+Ae1; X(ti + 2h) = (1 - (J,h)K © X{tx +h) + Ae 2 ; X(t2) = X(h + nh) = (1 - iih)K © X(h + (n - l)h) + Aen. By induction and employing the properties of the extended-thinning operation, X{t1+2h) = (I-Lih)K®[(l-Lxh)K®X(ti) + Aei] + Ae2 = (1 - LLK)K © (1 - Lih)K ® X(ti) + (1 - iih)K © Aei + Ae 2 = (l-Lih)2K®X(h) + {l-Lih)K®Ae1 + Ae2, X(h + 3h) = {l-nh)3K®X{t1) + (l-fxh)2K®Ael + (l-nh)K®Ae2 + Aeri, n - l X(t2) = (l-Lih)nK®X{tl) + Y,(l-Vh) kK®&en-k- Let Yn = (1 - iih) nK ® X(ti), and n-l Z n = £ ( l - / x / i ) k ® A e n _ f c . (4.4.3) fc=0 Note that Y n and Zn are independent. When n goes to +oo, (1 _ = ( l _ ^ ^ i ) " e - " ^ - * 0 . 119 Hence, by Property 3.14, Yn A ( e - ^ * 2 - * 0 ) ®X(t i ) , asn—++00. Assume that {Zn} converges in distribution. Then this limit will be n—1 n—1 lim Zn = lim V ( l - A / / 0 t f ® A e „ - f c = lim W e " " ^ ) © Ae„_ f c ra—>+oo ra—>+oo ^—' ra—>+oo z — ' \ / K k=0 k=0 n . , rt2-h lim V f e - ^ 2 - * 1 " ^ ®Ae3 = (e-^-t,-t)\ @de(i)_ However, in the stationary situation, since Aei, Ae2, Ae n are iid, we can derive a simpler expression, lim Y(e-*khA ©Ae„_ f c = lim V ( V ^ ) © Ae f c = P ^ ( e ~ K ® de{t). k=0 k=0 u Finally, by the independence of Yn and Zn, we obtain X(t2) = (e-rtt*-t^)K®X(t1) + h {e-^)K®de(t). (AAA) Next we turn to the non-stationary case, where we allow p to be a function of £, i.e., n(t), and the innovation process could be a stationary independent increment process or a non-stationary independent increment process. Then the generalized Ornstein-Uhlenbeck SDE becomes dX(t) = [(1 - n(t)dt)K ® X{t) - X{t)] + de{t). 0 We follow the convention that a null product f] aj 1S 1- We have a slightly different version of the approximation of differences in a small interval based on (4.4.2): X(h+2h) = (l - n(ti + h)h) ® (l-n{h)hjK®X(ti) + Ael + Ae 2 = ( l - (i(ti + h)h)K®(1~ © + ( l - /x(ii + h)fi) ® Aei + Ae 2 = ([1 - A*(*I)/I][1 - fi(h + h)h]) ® X(ti) + ( l - /x(ti + /i)/i) © Aei + Ae 2 , 120 X(tx + 3h) = ( [ l - / / ( i i ) / i ] [ l - / x ( i i + /i)/i][l-M(£i + 2/i)/ i])^©X(£ 1 ) + ([1 - n(ti + h)h][l - n(ti + 2/i)/i]) © Aei + ([1 - n(h + 2ft)/t]) © Ae 2 + Ae 3 , x(h) = (Y[[i-^(ti + kh)h])K®x(t1) k=0 n— 1 k Similar to before, let Yn = ( [1 - A*(*l + ^ © *(*l) n-l k=0 and it,— ± rv z" = E(II[1-^i + (N~̂ /L 0̂̂ ®AE"-FC fc=0 j = l n—1 k = z)(ntI-̂ *2_^,I0A:®AEN-*- (4A5) fc=0 "j'=l Then V n and Z n are independent. Note that by Lemma 4.4.1, as n —> +oo, J][l - /i(t! + Jfc/i)/»] = II e - " ( * 1 + f c h ^ = exp [ - ^ + ^4 —>• e " ^ m d t - fc=0 fc=0 I fc=0 J Hence, r„ A (e" "(0<tt) © X(*!), as n —> +oo. Assume that {Zn} converges in distribution. Then n—l k lim Zn = lim V ( TTj1 - n(t2 - jh)h]) ® Ae„_* fc=0 j = l = ni&ooE ne"Mt2_j/l)M ®Ae"-fc ^ / - £ Kt2-jh)h\ = lim > e © Ae n _ f c n—>4-oo ̂ —^ 1 / fc=0 \ i x 121 ( n-k \ - E n{t2-jh)h\ e ' = 1 I I K ®Aek f2 L- St2 ^r)dr\ @ r f e ( t J Jti V / l Finally, by the independence of YN and Z N , we obtain (4.4.6) The stochastic integral J St 2 ^)dr\ ®de(t) can be viewed as a cumulative innovation. The more recent innovation has more influence on the cumulative innovation, because e J* ^T> T is increasing as t approaches t2. For a stationary generalized Ornstein-Uhlenbeck SDE, although the stochastic integrals j*-'1 { e - ^ - t ^ ) K ® de{t) and /0*2 t l {e~'a)K®de{t) have slightly different interpretations, it doesn't matter because they are equal in distribution. The stochastic representations of the solution of the generalized Ornstein-Uhlenbeck SDE are shown in (4.4.4) and (4.4.6), where the current state can be split into two independent terms: the first is the dependent part related to previous state; the second is the cumulative innovation, a • When does the generalized stochastic integral exist? • How to find this stochastic integral? Recall that the generalized stochastic integral is defined as the limit in distribution. Hence the convenient tools to investigate such a generalized stochastic integral are the pgf, or LT, or cf, depending on if the innovation process has non-negative integer, or positive, or real-valued margins. Assume K has pgf GK{S',(X), or LT <fa(s, a), or cf </>#(«, a); and Aei has pgf G&.e(s), or <f>Ae(s), ° r <PAe(s)- Now we study the pgf, or LT, or cf for Z N . For the stationary case, since Aei, Ae2, • • •, Aen are iid, it follows that generalized stochastic integration. Now the natural questions arise: K®Aen-i n-l 122 or n-1 <t>zM E n-1 9 - « E 2 i o ( ( 1 - " h ) * ) « ® A e » - = iW< , - s ( ( l -M/ i ) ' c )K©Ae n - i fc=0 = *[[<l>At(-\og<l>K(s;{l-iih)ky) k=0 or is^Ul-iiti)*) ®Aen-k n-1 = II E \ [ e i s ( { 1 - f l h ) k " > K ® A € n - k fc=0 fc=0 n—1 n ̂  (* i os ( s; (x - v hn) • n-1 fc=0 If as n — > +00, and —>G(s), or <j)Zn{s) —• </>(s), or y>z„(s) —> y>(s), G(s) is continuous at s = 1 with G(l) = 1, or <f)(s) is continuous at s = 0 with <̂ (0) = 1, or ip(s) is continuous at s = 0 with <p(0) = 1, then G(s), or c/>(s) or ip(s) is the pgf, or LT, or cf of j ( e © de(*)- For the non-stationary case, Aei, Ae2, . . . , Ae„ are independent, they may or may not be identically distributed. The corresponding pgf, or LT, or cf of Zn are E I ft [l-/*(*2-.j70/i] ) © A e „ _ * GZn(s) = E l . ' ^ V = ' JK n-1 = ri E fc=0 n-1 Jl [l-v(t2-jh)h] © A e n _ f c 1=! ) K II G ^ n - k \ GK\S; - n(h - jh)h] 123 or 4>zM = E ( n [l-ti(.t2-jh)h]) © A e n _ f c , fc=0 \j=i / K n-1 n-1 = n* -s\^U.[l-ii(t2-jh)h]j © A e n _ f c fc=0 log for ( s ; n [ r - M * 2 - j / i M i=i or and i E ( f l [l-«(*2-j/i)fc] ) © A e n - fc=0 \ j = l / K n-1 n-1 II ^Ae n _ f c fc=0 n-1 log^K s,Y[[l -n(t2 -jh)h] k = n ̂  e n — fc fc=0 Similarly, if •Gz„(s) — o r fon(s) —•> fas), or V Z n ( s ) —> as n G(s) is continuous at s = 1 with G(l) = 1, or <£(s) is continuous at s = 0 with 0(0) = 1, or <p(s) is continuous at s = 0 with cp(0) = 1, +oo, then we can conclude that G(s), or fas), or <p(s) is the pgf, or LT, or cf of I (e ^ ^r)dr) ^®de(t). 124 These analyses show that finding the generalized stochastic integral is equivalent to finding the limit of pgf, or LT, or cf of Zn, where the limit should be a pgf, or LT, or cf. Due to self- generalizability, the dependence part Yn in the approximation always converges. Hence, we obtain the following theorem. Theorem 4.4.2 (Solutions of the generalized Ornstein-Uhlenbeck SDE) (1) For the stationary generalized Ornstein-Uhlenbeck SDE dX(t) = [(1 - ndt)K®X{t) -X(t)] + de{t), Pt2—tl if / ( e _ / i t ) i < - ® de(t) exists, then the solution is Jo X(t2) = ( e - /*( fe-t i )} © X(h) + f2 h [e-^)K © de(t). (2) For the non-stationary generalized Ornstein-Uhlenbeck SDE, where ii(t) is bounded, dX{t) = [(1 - n(t)dt)K ® X{t) - X(t)} + de(t), if (e~^2^T"ldr^j^® de(t) exists, then the solution is X{t2)± (e-tt^)K®X{tl) + (e-tt^)K®de(t). Note that in the sense of convergence in distribution, the classical Ornstein-Uhlenbeck SDE has the same solution as in the sense of convergence in L2. In summary, the stochastic representations of X(t2) conditioned on X(t\) in both the sta- tionary and non-stationary cases show that {X(t);t > 0} (if existing) is a first-order Markov process. Since the classical Ornstein-Uhlenbeck SDE leads to the continuous-time AR(1) Gaussian process, we name the new processes, constructed by the generalized Ornstein-Uhlenbeck SDE, the continuous-time generalized AR(1) processes, or in short, the continuous-time GAR(l) processes. Specifically, they are of the forms given in Theorem 4.4.2. Some comments about the comparison with traditional AR(1) processes are given in the next section. Our next main task is to search for appropriate IIP innovations, which guarantee that the generalized stochastic integral exists. 125 4 .5 Summary and discussion In this section, we compare among the Ornstein-Uhlenbeck processes, the Ornstein-Uhlenbeck-type processes, and the new continuous-time generalized AR(1) processes. We summarize their features only for the stationary situation. The Ornstein-Uhlenbeck process and the Ornstein-Uhlenbeck-type process have the same stochastic representation: X(t2) = e~^-^ • X{h) + f2 h •de{t). Jo If {e(t);t > 0} is a Wiener process, it's the ordinary Ornstein-Uhlenbeck process. If {e(t); t > 0} is a Levy process, it is then the Ornstein-Uhlenbeck-type processes. Extending the constant multiplier operation to extended-thinning operation, we obtain the continuous-time generalized AR(1) process: X(t2) = ( e - " t e - ' i ) ) ®X(t1) + f2 h {e-»t)K®de(t). ^ ' K Jo Now we investigate their representation structure. The expressions of the stochastic repre- sentation show that X{t2) consists of two independent parts: one related to X(t\) only and one related to the innovation process only. When t2 — t\ goes to infinity, e - ^ 2 - * 1 ) goes to zero. Thus, the first part will diminish to zero, which means X(t\) will gradually have less and less influence on X(t2) until the influence reduces to null. Then the influence will exclusively come from the innovation process. This shows us a dynamic process picture: after continuously repeated treat- ment by the dependence mechanism device (i.e., A(X(t), — fj.dt)), the original input X(ti) will be diminishing to nothing. On the other hand, the innovation during this period will be treated by the same mechanism; however, it has accumulated as a stochastic integral and finally accounts for X(t2) solely. What are the restrictions to applying the Ornstein-Uhlenbeck process and the Ornstein- Uhlenbeck-type process in modelling dynamic phenomena? Since the Ornstein-Uhlenbeck process has real-valued margins, it only works for those with real observations over time, otherwise, some transformation for the data, say log-transformation, should be take to fit the model. The Ornstein- Uhlenbeck-type process extends to positive real-valued margins, but can not have non-negative 126 integer-valued margins, because e - / i(* 2 _ i l) • X(ti) is not likely to be an integer. However, the continuous-time generalized AR(1) process offers non-negative integer-valued, positive real-valued and real-valued margins. Therefore, it is quite flexible in modelling different types of margins. In Chapter 5, we give abundant examples. For the Ornstein-Uhlenbeck process and the Ornstein-Uhlenbeck-type process, the depen- dence part is a linear function of X(t\). Hence, conditioned on X(t\), this dependence part is fixed, not random. The conditional variation of Xfo) only comes from the part related to innovation. For the continuous-time generalized AR(1) process, however, this part may not be a linear function of X(tj). Furthermore, conditioned on X(ti), it's no longer fixed, but random, so it looks like a random effect, and also contributes to the conditional variation of Xfo)- The reader may wonder why we name this type process as continuous-time generalized AR(1) process, instead of following the conventional way to name it like the generalized Ornstein- Uhlenbeck-type process. This is because we focus on the statistical point of view. We wish to emphasize its advantage, the auto-regression like property, in statistical modelling. One big con- cern in modelling dynamic phenomena is to capture the dependence structure over time. Since, the processes we study possess the same auto-correlation as the continuous-time AR(1) Gaussian pro- cess, we propose the name continuous-time generalized AR(1) process to clearly show that one can apply this kind of process to model, or in another word, to approximate the real dynamic problems which have obvious dependence structure over time. However, the concept "auto- regression" in the new processes is not strictly autoregression, because X(£2) is not equal in distribution to a linear function of X(t\). This jump in concept resembles the relationship of the generalized linear model to the linear model. If one like, one may call such an autoregression a generalized autoregression to distinguish it with the classical autoregression, which is linear in the previous observation. 127 Chapter 5 Results for continuous-time generalized AR(1) processes In this chapter, we shall deduce some concrete results of continuous-time GAR(l) processes dis- cussed in Chapter 4. We consider special HP's as the innovation processes. These innovation processes are classified as having non-negative integer; positive and real increment. The general conclusion for these special innovation processes is given in Section 5.1. From Sections 5.2 to 5.3, we discuss the non-negative integer, positive and real increment cases and examples respectively. Finally, in Section 5.4 we explore the Tweedie IIP as the innovation process to study or revisit the models from the view of dispersion. These innovation processes lead to the continuous-time GAR(l) processes with non-negative integer, positive and real margins. The abundant resulting processes could be potential models for real phenomena that statisticians seek to explore. 5.1 Main results for continuous-time G A R ( l ) processes In this section, we apply the theory in Chapter 4 to construct concrete examples of the continuous- time GAR(l) processes.̂ 128 First, we choose relevant HP's as innovation processes. They could have non-negative inte- ger, or positive, or real margins. Secondly, we specify the extended-thinning operations. These two steps determine the generalized Ornstein-Uhlenbeck SDE, and thus, the corresponding continuous- time GAR(l) processes. The key point in such procedure is to calculate G(s), or (f>(s), or ip(s) discussed in Section 4.4. If this is a pgf, or LT, or cf, then we obtain the corresponding generalized stochastic integral, as well as the corresponding continuous-time GAR(l) process. Both stationary and non-stationary situations are investigated. However, for the sake of simplicity, we restrict the innovation processes as stationary HP's so that we can easily give ex- plicit results. This idea can be readily extended to the case of the non-stationary HP's being the innovation processes. Now we probe the issue of the support type of margins of continuous-time GAR(l) processes. Recall the stochastic representations of this kind of processes in stationary case: X(t2) = (e-^-^)K © X(h) + 11 {e'^)K © de(t). When t2 - t\ oo, the term ( e - ^ ' 2 - ' 1 ) ) K © X(t\) will converge to zero. Hence, the margins and their support are essentially governed by the rv K and the innovation process {e(t);t > 0}. We may want the marginal support to be the non-negative integer, or positive, or real set. This can be realized by choosing the appropriate self-generalized rv K and IIP {e(t); t > 0}. Recall that the generalized stochastic integral involved in the continuous-time GAR(l) pro- / (E_/II) K ® DE(^> 0 1 / (e~ / e ' 2 fl(T)dT) ® de(t) have pgf, or LT, or cf as the following Jo Jti ^ ' K cess Gzn(s) iti nf[GAe (GK (s; ( 1 - ^ ) ) i=0 I n - l II GAe i=0 GK U n [(I-M*2 -rn for constant /J,; for n(t); or <t>zn(s) = ( n ^ A ^ - i o g ^ ^ a - ^ r ) ) , j Jl <t>Ae f - 4>K \ S \ ft [(1 - M*2 - jh)h] for constant LI; , for n(t); 129 or or n - l <Pzn{s) = < IT fAe {-ilog ipK (s;{l- un)*)) . for constant LI; i=0 Tl1 ^Ae (~i log WK (s; ]Q [(1 - M * 2 - J J , for M*)? where Z n is defined in (4.4.3) and (4.4.5). Since they are all products, we may choose innovation processes in which the pgf, or LT, or cf of increment is of the exponential form. Such a form has the advantage that it can change the product to summation of their exponents so that the limit will be an integral. The following proposition from calculus is essential to our study of results of continuous-time GAR(l) processes. P r o p o s i t i o n 5.1.1 Suppose R(x) is a differentiate real-valued function with bounded first order derivative. Let h = (t2 — t\)/n, where t2 — t\ > 0. Then (1) for constant LI, h V i ? ( ( l - / i h ) ' ) - > / R(e-^)dt, asn->oo, 7=t J o (2) for function //(£), n - l ( i \ r t 2 H E R n i 1 - - -> / R ( E _ H 2 MR)DT) DT> i=o \j=i I J t l as n —>• oo. Proof : The key step is to show that n - l n - l i=0 i=0 i=0 \j=l J i=0 \ Then by the definition of Riemann integration, the conclusions hold. 130 According to Inequality 3.6.2 in Mitrinovic and Vasic [1970] (Section 3.6, p. 266), we have 1' Ct- rl Thus, (1-fihY = (1- and consequently, n-1 M*2 _ * l) n = e -fi(t2-ti) + A n i/n n n fc£fl((l- i=0 n-1 h ^ R i=0 n-1 ,~Hih , til + — + 0 n \n i=o n V Similarly, n[i-M<2-j/oft] = n i = n which leads to n—1 / i 1 - A«(*2 -jh){t2 - h) n n e-Kt2-Jh)(t2-h) + e-Kt2-jh)h + Mj_ + q n \n n-1 n n i=0 \j=i J *=o v = fc £ {i? (e- SJ=I + R> ( e - S j - i Mb-ifc)fc) . | + 0 ( I ) } i=0 n-1 i=0 ^ ' Remark: The technique of proof uses a bounded first order derivative, but it may be possible to prove the result without this condition by a different method. 131 Now we choose a special innovation process, in which the increment, e(t + h) — e(i), has the exponential form of pgf, or LT, or cf. Applying Proposition 5.1.1, we can obtain the following theorem. Theorem 5.1.2 Assume the innovation process {e(t); t > 0} has increment e(t + h) — e(t) such that its pgf, or LT, or cf is of form ehC^s\ depending on the increment being non-negative integer- valued, or positive-valued, or real-valued. C(s) is assumed to be differentiable with bounded first order derivative. (1) For the stationary situation, let Zn be defined in (4-4-3). If GK(S;OS), or log <J>K{S; a), or log <PK{S\ OL) have bounded first order derivative with respect to a in [0,1] (boundaries could be excluded), then it follows that GZn(s) = n °^ (* - )̂)=n e x p i h c (g* (x - PW))) or n-1 <t>zn(s) l\GAe {4>K (s;(l-nhy)) or n-1 <pzAs) And the G(s), fas) and <p(s) are the pgf, LT and cf of the generalized stochastic integral respectively. 132 (2) For the non-stationary situation, let ZN be defined in (4-4-5). IfGK{S\Q), or log4>K(S;a), or log <PK(S; ct) have bounded first order derivative in [0,1] (boundaries could be excluded), then it follows that n-l Gzn(s) = HGA£\GK[s;Yl[l-^t2-jh)h} i=0 \ V 3=1 n-l or or = HexplhC lGK ls;H [1 - n{t2 - jh)h] ^ ° exp {J** C (GK (S; e~^ ^ d r ) ) dt} = G(s), <f>zn (s) = n 4>Ae ( - log <f>K ( s; fl [1 - M*2 - jh)h] e x p [ ^ * 2 C (-log<fa (S;e-̂ 2^dr)) d i } = n-l i=0 \ \ J'=l n—>oo exp [ j ^ C ^ - i log ( s ; e - £ 2 " ( r ) d T ) ) df} = ^(s). The G(s), (f>(s) and cp(s) are the pgf, LT and cf of the generalized stochastic integral "t2 respectively. Proof: W i t h suppression of the dependence on s, let R(a) = C(GK(S; a)), if iv" is a non-negative integer rv, ( C ( - log <fa(s; a)), if iv" a is positive rv, C(-i log ipK(S; a)), if iv" a is real rv. 133 Then the derivative of R(a) is C'(GK(S; O))-^GK(S; a), if A" is a non-negative integer rv, R'(a) = { -C'(-\og(f)K(s;a))£-\og<t>K{s;a), if K a is positive rv, -iC(-Hog ipK(s; a))log <pK(s; a), if K a is real rv. Hence, R'(a) is bounded in [0,1] (boundaries could be excluded) under the conditions of this theorem. To save space, we only verify them for K being a non-negative integer rv and the increment of innovation process being non-negative integer-valued. (1) For the stationary situation, n-1 n-1 GZn(s) = l[GA£(GK{s;(l-fihy)) = lleW{hC(GK{S;(l-^h)i))} i=0 i=0 ( n-1 \ exp / i ^ x u - ^ r ) i . i=o ; under the conditions in this theorem, Proposition 5.1.1 holds. Thus, GzM ^ exp { ^ 2 _ t l C {GK e""*)) dt} = G(s). When s = 1, since C(l) = \og(G Ae{l)) / h = log(l)//i = 0, rti-ti -\ ( cti-t\ G(l) = exp | ^ * 2 ^ C {GK (1; e""*)) dt} = exp C(l)dt j = eu = 1. rti—ti Therefore, G{s) is a pgf. We can conclude that / (e^*)*- © de(t) exists with pgf G{s). Jo (2) For the non-stationary situation, GzM = \lGAAGK(s-,fl[l-^h-jh)h] i=0 \ \ j=l n-1 i=0 = n e x p hC i GK s;Y[[l - fi(t2 - jh)h] J'=I = exp < n-1 i=0 \j=l 134 By Proposition 5.1.1, GZn(s) exp { jf" C (GK (S; e'^ dt} = G(a), and G(l) = 1. Hence ^ ' (e~ & ^ d T ^ ® de(t) exists with pgf G(s). • Recall Examples 3.1 to 3.5 (labeled from II to 15) where all of the self-generalized rv K are non-negative integer-valued. We check the partial derivatives (s - 1), for II, (1 - 7)(a - 1)(1 - 7*)[7(* - 1)« + (1 - 7s)] - 2 , for 12, -(1 - s ) Q l o g ( l - s ) , for 13, -^zr[ee- (ed - l)s}a log [e° - (e9 - l)s] , for 14, -6(1 - 7 ) * [l + ( l - 7 ) ( l - s ) - 1 / ( ? ] x a ^ 1 [ ( l - a ) 7 + ( l - 7 ) ( l - 5 ) - 1 / 0 ] " ( m ) , for 15. Thus, they are all bounded if 0 < s < 1. For Examples 3.6 to 3.10 (labeled from PI to P5) where K is a positive self-generalized rv, because the following relationship corresponding to the II to 15 holds 4>K(S;a) = exp{Gj<-'(i - s;a) - 1}, JL da GK(s;a) = < we obtain 3 0 — log(f)K{s; a) = Q^Gk(1 - s;a). Therefore, the ^ log<fo-(.s; a) is bounded in [0,1] too. Essentially, to apply Theorem 5.1.2, we only need to check the boundedness of C'(s) if we consider K being from II to 15 and PI to P5. The type of the generalized stochastic integral Jo Jti ^ ' K is determined by the self-generalized rv K and the innovation process {e(t); t > 0}. It could be non-negative integer-valued, or positive-valued, or real-valued. Under the cases that K is from II to 15 and PI to P5, we classify them in the following theorem. 135 G(s) = Theorem 5.1.3 Assume the innovation process {e(t); t > 0} has increment Ae(i) = e(t + h) — e(t) whose pgf, or LT, or cf has form ehC^ depending on the increment being non-negative integer- valued, or positive-valued, or real-valued. C(s) is differentiate with bounded first order derivative. We classify the possible type of the generalized stochastic integral. (1) non-negative integer-valued: K could be from I I to 15, the increment of innovation process should be non-negative integer-valued. In this circumstance, if the increment has pgf GAe^(s) = ehC^, then exp [y C (GK (S; dt} , for constant LI, | exp[yt2c(G^(S;e-/*2^dT))^}, for »{t). (2) positive-valued: K could be from P I to P5, the increment of innovation process should be positive-valued. In this circumstance, if the increment has LT 4>Ae(t)(s) = ehC(s\ then exp[y C (— \og(j>K (s;e _ ' i t)) dt} , for constant LI, exp [ jT 2 C ( - log for (a; e" tt ^ d r ) ) di} , for Li(t). (3) real-valued: K is only from P I , the increment of innovation process is only real-valued. In this circumstance, <fiAe(t)(s) = ehC(s\ and ( rt2-ti -\ exp IJ C (se _ / i t) dtj , for constant LI, exp [yf2 C (se' tt dt} , for n{t). Proof: The proof is straightforward. We omit it to save space. <j>(s) = <p(s) Fortunately, there are several distribution families which have exponential form of pgf, or LT, or cf. For instance, the compound Poisson, GNBC, GGC, stable, Tweedie families are well- known examples. We will discuss them in the case of innovation processes with non-negative integer, positive and real increment respectively. 136 5.2 Non-negative integer innovation processes and examples In the following we mainly consider four process families as the innovations: compound Poisson IIP, generalized Negative Binomial convolution IIP, GC I IIP and GC II IIP. These four families lead to a lot of well-known distributions as margins. As to the self-generalized distributions, because the increment of the innovation process {e(t);t > 0} is non-negative integer-valued, K should be a non-negative integer rv, which further leads to the non-negative integer generalized stochastic integral. We pick up those five non-negative integer rv's in Examples 3.1 to 3.5 (labeled from II to 15) for the extended-thinning operations. First, we consider the Compound Poisson IIP as innovation process. By Theorem 5.1.2, we have Theorem 5.2.1 Let {e(t);t > 0} be a Compound Poisson IIP with pgf of Ae = e(t + h) - e(t) GA£(s) = exp{\h[g(s)-l}}, where g(s)(= YlPis*) z s a P9f> a n d differentiate with bounded first order derivative. Suppose K(a) is a non-negative integer self-generalized rv with pgf GK{S\OL), which is differentiate with bounded first order derivative with respect to a. Then, it follows that e x p J A ^ h [g(GK(s;e-'d)) G(s) = | e x p J A ^ [g(GK(s;e-Si2»WT)) _ and G(s) is a pgf. Hence, the generalized stochastic integrals for constant /J,; for n(t); f* h ( E ~ ^ ) K ® D E W A N D ( E ~ / T < 2 " ( T ) D R ) K © D E ( * ) exist and are non-negative integer rv's. For a specific self-generalized rv K, we know the form of its pgf GK{S)- Thus, we can obtain the full expression of G(s) by Theorem 5.2.1. This leads to the following corollary, where the K is chosen from II to 15 (non-negative integer case). 137 For the sake of saving space, we only list the results for the stationary case. The non- stationary case can be straightforward to deduce without any difficulty. C o r o l l a r y 5.2.2 Consider the innovation being the Compound Poisson IIP. In Theorem 5.2.1, by specifying GK{S\Q) or <fa(s;a) for the self-generalized rv K, we can get the further form of g(GK(s;a)) or g(<fa(s;a)). The following are the results for K being from II to 15 under the stationary case. II: G(s) = exp [A [g ([1 - e""*] + e'^s) - l] dt} . oo Furthermore, if g(s) = YlPisl> then r*2—ti i = 0 oo [ l _ e - J > ( t 2 - t 1 ) ] ( s _ l ) J (1 - e~^) + (e-^ - -y)s , _ j 9 ' (1 - - (1 - e - ^ ) 7 s 12: G(s) = expJAy 13: G(s) = exp JÂ2"*1 [<? ( l - (1 - s)^) - l ] dt} dt 14: G(s) = exp 15: G(s) = exp ^ A L t2-h t2 - * i dt 1 - e - W ( l _ 7 ) » V ( l - e - ^ ) 7 + ( l - 7 ) ( l - s ) - For the non-stationary case, just replace the e~̂ * with e~ •/*2 ̂ d T . - 1 dt P r o o f : The second half part of II needs some details. r*2—ti G(s) = exp JA r t l \g([l - e""*] + e""**) - l ]d t} = exp {A ft2~h [ f > ( [ l - e~"*] + e ^ ' s ) ' - l ]d t} i = 0 = e x p { A / ^ ^ [ ( [ l - e - ^ + e-^a) - l ] d t } , ' 0 i=i O O pt^—tl {A5>[/1 ([i + ( ^ - i ) ^ r - i ) d t ] } = exp 138 00  r rt2-h = e x p J A ^ T i=l J 0 j=l V / = exp{A5>[/ W M ( s _ i ) V ^ d i ] } i=i •'° j=i - - {* | * [£G)« - I » I 1 1 ^ ]} , oo 1 OO / . \ = ^ {; E (HE (iWli- '-^'K'-»>')}• ^ j=l J i=j V By choosing an appropriate pgf g(s), we can find the exact form of the pgf G(s) of interest so that we can obtain the corresponding generalized stochastic integral. The following are a few examples. Example 5.1 Consider the compound Poisson IIP innovation. Let g(s) = s and let K be from I I , so that the extended-thinning operator is binomial-thinning. Then G{s) = exp JA J2 - t i [»( 1 - e~^] + e - ^ s ) - l] dt} = exp{A^ - t i [01 - e""*] + e - " *a ) - = exp{Â  - t i ( s — = exp |A(S — 'o - t i . e'^dtj = e x p { - [ l - or G(s) = exp{Ar[ 0 ([ l-e-/ / 2 ^] + e - ^ ^ ) - l ] d t } = exp (A j f* [([1 - e- / / 2 ^ ] + 6 " / i 8 - l] d*} 139 = exv{\([t2e-tt^dt)(s-l)}. These results correspond to the following models: r*2—*i X(t2) = e-^'^ * X{ti) + f~ " e-^ * de(t), Jo where j ' \ e"^ * de(t) ~ Poisson^ [l - e^* 2 " ' 1 )]) , and X(t2) = e~ tt " ( T ) d T *X{t1)+ f2 e~ tt Mr)dr + d e ( i ) ; w/iere e~ tt vWdr ^ d e ^ „ P o i s s o n ^ A E - / / 2 M M * - ^ Example 5.2 Still consider the compound Poisson IIP innovation, and suppose K remains in II. Now we choose g(s) = j^^, i.e., the pgf of NB(\,~j). Then for the stationary case, G(s) = expJA^* 2 11 [g{[l-e-^] + e-^s) - l]dt) = exp | A t2—tl r 1-7 exp exp 1 - 7^[1 - e-^] + e-^s^j - 1 dt} t 2 - t i 7 ( s _ ! ) e - M t (1 - 7) - 7(s - l ) e -^ dt } = exp { - log - ( r A r *1 * 2 - t i i {^log (1 - 7 ) - l{s - l)e^ } I LI I J 0 J ]} ( l - 7 ) - 7 ( s - l ) e - ^ 2 ~ t l ) (1-7) -7 ( ^ - 1 ) (1 _ 7 ) _ 7 ( a - i ) e -M*2- t iK x/n 1 — 7s / ( e -Mt 2 - t i ) + [j _ e - ^ - t O ] ^ ^ ) ^ T/iis Zeads to the model X(t2) = e - ^ ~ t l ) * X ( t i ) + T2 1 e-** * de(t) Jo 140 where P e~^ * de(t) has pgf (e-"(fe-*i) + [1 - e ' ^ ^ } ^ ) Jo For the non-stationary case, it follows that A / u G(s) = exp{A P [ g ( [ l - e - ^ ^ ] + e-^^s)-l]dt} '- 1^1 ; . - l]dt\ - 1 _ 7 ^ [ l _ e - / / 2 M r ) d r ] + e - / / V ( r ) d r ^ J J exp \ A exp t 2  1 ( s - l ) e - t i 2 ^ d l \X P — ^ L Jt! 1 - 7 -7 - 7 ( 5 - l)e-/* 2 '*( r) d dt). T)dr ) Hence, we have model X(t2) = e- ̂  ^ T ) d T * X{h) + P e~ & ̂ d T * de(t), Jh where e" & ̂ r ) d T * de(t) has pgf exp { A j f * — 7 ( s - l ) e - / / 2 ^ d l -dt • 7 - 7 ( s - l ) e - / t 2 M T ) d r Example 5.3 Let g(s) = ± - e^~^/e (#>!)• Consider the compound Poisson IIP innovation, and suppose K remains in II. Then for the stationary case, t2—*1 r G(s) = exp < A = exp < A = exp < A o - e""*] + e-^s) - l] dt} i e- l(\-s)xi 9e-^i° 1 + (s - l)e-^ 1 - (1 - sY^e-^l6 - ( g - l ) e - ^ 9-l{l-s)^ee-^e 1 + (s - l)e-" 1 1 - (1 - sY^e-M0 t2—*1 0 A / u dt dt = exp { - [log [1 + (s - l)e-"*] *2 ^ - log [l - (1 - s)l'ee L u L o L -/zt/0 * 2 - t l - 0 ' l l + ( a - l ) e - M t 2 - t i ) ~ X 1 - (1 - S)l/fl e-M*2-tl)/e /"*2—*1 w/uc/i is t/te pgf of / e_M* * de(t). T/ius, t/te model is Jo X{t2) = e-^t2-^ * X(h) + P 1 1 e""' * de(t). 7o 141 A may take the same value as LI. In this case, G{s) = s _ 1 [ l - (1 - s)1/9] (as t 2 - h ->• oo), which is just the pgf of power series distribution (this is an alternative way to show its DSD property). For the non-stationary case, the pgf becomes G(s) = exp [A j'' \g ([1 - e~ ̂ fi{T)dr) + e~ & " ( r ) d T s ) - l ] } 1 9~l(l -sf/Oe-o-'St'nWT = exp < A = exp < A l + ( s _ l ) e - / / V ( r ) r f r ~ 1 _ (1 _ s J l / A e - O - 1 Si2 Kr)dr -{S- l ) e - J / 2 M ^ - a ) l / g e - 0 J l j/2Mr)dr 1 + (a - l ) e - /» 2 1 - (1 - a)!/^-*"1 / / 2 "Mrfr - 1 dt dt}, the pgf of [ e'tt ^T">dr * de(t), and the model is Jti X(t2) = e~fi »{T)dT * X(h) + f2 e~tt " M " T * de(t). Jti Example 5 . 4 Suppose innovation remains as compound Poisson IIP, and g(s) = s. Now consider r ( g - 7 ) s - ( I — a ) 7 « ' i f / r o m 12 with pgf GK(s; a) = Jlffi? ^ - - Men ^ , v , a ( l - 7 ) ( a - l ) Gif(s;a) - 1 = 7- r 7- r — , v (1 - 07) - (1 - a)-ys and for the stationary case, G{s) = e x p { A ^ = e x p J A ^ •t2-ti (1 - e "^ ) + (e-^f - 7 ) 8 (1 - e-^7) - (1 - e'^-ys t2-h ( ! _ 7 ) ( s _ i ) e - M « ) - i ] * } exp exp exp (1 - ye-^) - (1 - e -«* )7s •fa-ti ( ! _ 7 ) ( s _ i ) e - M t ( A / (1 - 7 s ) + 7 ( 5 - l )e~^ I / i 7 4 log LM 7 d i } ( 1 _ 7 s ) + 7 ( s _ l ) e - ^ ] | o 2 _ 1 } ( l -7s)4-7 (s - l )e - ' i ( t 2 -* 1 ) -IJ 1 - 7 X 1--) p 1 (1 - 7s) + 7(5 - l ^ - ^ - ' 1 ) . x _ 7 \ A( l -7 ) / (^7) l _ - y e - M ( ' 2 _ t l ) _ 7 ( i - e - ^ ( ' 2 - t i ) ) \ A ( l - 7 ) / ( M 7 ) l _ 7 e - ^ ( t 2 - ' l ) _ 7 ( i - e - / . ( t 2 - t i ) ) l _ 7 e - M ( t 2 - ' l ) _ 7 ( i - e - M ( i 2 - ' i ) ) l _ 7 e - f » ( ' 2 - ' t l ) 142 Note that this last form is a NB pgf. Hence the model is X{t2)= (e-^-^)K®X(h) + j'' H {e~'a)K ®de(t) where (e^)K®de(t) ~ NB^^Z-X) Similarly, the model for the non-stationary case is X(t2) = (e-fc ® X(h) + £ 2 (e'fc ® de(t), where P [e~ ^ ^dT) k® de{t) has pgf ow = «P{A r [»(""'"frr'̂ '̂ T.vi') - w = exp T L L v (1 _ e - M W ^ - y ) _ (1 _ e - Jt"' M ( T J d r ) 7 S , * 2 - * i (1 _ 7 ) ( a _ l ) e - / / V ( r ) d r -dt (1 - 7 5 ) + 7 ( s - l)e - / / 2 " ( r ) d T Example 5.5 ifeep the previous stochastic setting, but choose g(s) = jEps (0 < /3 < 1). Then, 1-6 g(GK(s,a))-l I _ a ( l - a ) + ( q - 7 ) s 1 " (1—07) —(1—a)7« ( 1 - / ? ) [ ( !-7*)- 7 ( 1 - * H 1 ( l - / 3 ) ( l - 7 s ) + ( i 8 - 7 ) ( l - s ) a - /3 (1 -7)(!-*)« ( l - j 8 ) ( l - 7 s ) + ( ) 8 - 7)(l-s)a- For the stationary case, rti—h G(s) = e x p { A ^ = e x p { A ^ 1 - (1 - e-"*) + (e~^ - 7 )3 ,(1 - e-^7) - (1 - e - ^ s -3(l-7)(l-s)e-^ (1 - 6)(1 - 1S) + (6 - j)(l - s)e- )-l}dt} It} exp { A / ? ' ' _ ^ l o g [(1 - « ( 1 - 7*) + (0 " 7)(1 - } - ( M/3 - 7) f W - 7 ) (1 -B)(l- 7^) + - 7)(1 - a ) e - ^ -^ ) ^ 6 X P I „(/9 - 7) § (1 -B)(l- 7 f l ) + 03 - 7)(1 -.̂ ) J (1 - B)(l - 7 5 ) + ()9 - 7)(1 - aje-^"* 1 ) \ ( l - / 5 ) ( l - 7 5 ) + ( / 3 - 7 ) ( l - s ) 143 and the model is X(t2) = ( e - M * 2 - t i ) ^ ® + J^2 h (e-^)K ® de(t), where I ( e _ / i t ) K © de(t) has the above G(s) as its pgf. Jo For the non-stationary case, r«2 r , (1 - e ~ St2 Kr)dr^ + (g- / / 2 n{r)dr _ ^ g G{s) = exp {A J2[g( _ (1 _ e - / / 2 ^ ( r ) d r ) y J J = exp (1 _ e - f t 2 ^(r)dr^   e-j;*l*(T)dTfrs. t 2-ti - 7)(1 - a ) e - / t 2 / * W d T -Jt'^\T)aT 1 ; r, (1 -/3)(1 - 7 s ) + ( / 3 - 7 ) ( l - s ) e - / t 2 M r ) r f r J and corresponding model becomes X{t2) = (e-^2^dTyK®X(ti) + J*' ( e - ^ 2 ^ T ) ^ © d e ( t ) , where the second G(s) is the pgf of (e~ ^ 2 ^ T ) d r ^ ^ © rfe(<). Secondly, we consider GNBC IIP as innovation processes. In this case, we have following theorem, from Theorem 5.1.2. Theorem 5.2.3 Let (e(t); t > 0} be a GNBC IIP with pgf of Ae = e(t + h) - e(t) GAe(s) = exp \h ' I l o g ( i^Ts 1 v { d q ) J(o,i) Suppose K(a) is a non-negative integer self-generalized rv with pgfGK{s;a), which is differentiate with bounded first order derivative. Then, it follows that •v r*2—*i exp / (/' 7(o,i) \Jo log P l-qGK (s;e-^) dt V{dq) for constant LI; G(s) = { exp < / if 7(o,i) Wti log dt V(dg)>, /or/z(t), P [ l - g G * (s;e-ft 2̂ )dr^ and G(s) is a pgf. Hence, the generalized stochastic integrals f2 t l ( e " ^ ) K® de(t) and [** (e~ "WT) ©de(t) To 7ti v ' K " ezist and are non-negative integer rv's. 144 Proof: Omitted. Consequently, we have derived the following corollary by direct calculation. Corollary 5.2.4 Here we consider the specific self-generalized rv K given in I I to 15. The inno- vation process is the GNBC IIP. I I : 12: 13: 14: G(s) = I [ e x p { ( * 2 - t i ) / ( 0 ! l ) i o g ( r ^ ) m ) + ^ ^ - /(0>1) &=£v(dq)} for constant M; exp{(i2-i1)/(01)log( I^5) V(dq) + ( j ? [tt M r ) d r ] eft) J ( 0 ) 1 ) *£=£v{dq)} for ( e x p l ^ - t O / ^ l o g ^ ) ^ ) G(s) = { + H(t2-h)2 2 /(0,1) +{it: [it2 dt) /(o,D q (l-s)(l-7s) 1—qs 1—7 / o r constant LI; ' V(dq)} forLi{t). G(s) = { q(l-s)log(l-s) y ( d g ) | / o r constant LI; 2 J( 0 ) i) ^ t ^ - * 0 / ( o , i ) log ( r ^ s ) n<*rt + {tt [tt »(r)alr] dt) » - l F ( ^ } f o r ^ [ e x p { ( t 2 - t 1 ) / ( 0 ] 1 ) l o g ( T ^ )m) G(s) = { M*2-*i) 2 qc~1(l-cs)[e+log(l-cs "r" 2 J(0,1) 1-qs % ( d g ) } / o r constant LI; + (j? [J/ 2  M ( r ) d r ] <tt) / { 0 > 1 ) ^ M i - c s ) ^ o g ( i - c s ) ] y ( d g ) | f Q r 145 15: f e x p j ^ - t O / ^ j l o g^ndg) + 2 J(0,1 1) 1 - 9 S exp {(t 2 - ti) / ( 0 i l ) log ( ^ j ) + [ j? /x(r)dr] dt) 9(l-7) / 3- e(l-s)[^+flTi-(l-S)^] 1 /(0,1) 1—qs -V{dq) for constant LI; for Li(t). However, usually the measure V(-) on (0,1) is not clear. Hence, Corollary 5.2.4 is not helpful in obtaining the pgf or LT of the generalized stochastic integral. For each specific member in the GNBC family, we have to calculate the pgf or LT individually. Following are some examples resulting from the GNBC IIP innovation family. Example 5.6 Consider the NB IIP innovation, in which the increment Ae = e(t + h) — e(t) has pgf where p,q> 0,p + q = 1. This is a special case of the GNBC when V(-) has a mass of 0 at a single value q. When K is from II, we obtain rt2—t\ [ exp j# J log P G(s) = { log p + q{s - l ) e -^J P p + q(s - l)e-ft2 n(r)dT dt > , for constant LI; dt>, for fi(t). which leads to the models X{t2) = e-^~tl) *X(tl)+ [ Jo e **** de(t), and X(t2) = e~ tt M ( r ) d r * X(t i ) + t e~ tt ^)dT * de(t) Ju 146 for the stationary and non-stationary case respectively. When K is from 12, then ( rt-2-ti exp <.6J log G(s) = { p(l - js) +P7{s - l)e - / i t dt} for constant \i; exp ^0 ft l o S dt} , for fj,(t). [p(l- 7 s ) + ( 7 - q){s - l ) e - " * J p(l - 7 5 ) + p 7(s - l)e~ ̂ 2 ^ r ) r f T _p(l - 7 s ) + ( 7 - o)(s - l ) e " -A'2 ^ r ) d T T^e resulting models are X(t2) = (e-^-^) ®X(t1)+ r t l (e^t)K®de{t), v ' K Jo and X{t2) = (e~ ti2 ^ d T ^ k® X{h) + j* (e-fc^dT^K®de{t). Example 5.7 Consider discrete stable IIP innovation, in which the increment e(t + h) — e(t) has pgf GA,(s) = exp{-A/i(l - s)0}, A > 0, 0 < 8 < 1. Case 1: X is /rom II. For </ie stationary situation, n-1 G(s) = l im T T e x p { - A / i [ l - ( l - ( l - s ) ( l - / i / i ) ¥ } n—+00 i=0 n-1 = l i m e j c p{ -AV/» [ ( l - s ) ( l - / i / i ) f } t=0 n-1 = JLim exp{ - A(l - s f Y h e ' ^ i h } n °° i=0 = exp {- A ""'"7'" ' ' > 1 (1-^}, exp w/iic/i Zeads to i/ie model X{t2) = e - ^ t 2 ~ h ) * X ( t i ) + / e-"* * de(t), 7o t 2 - t i 147 rt2—t\ JO where / e_/i* * de(t) has pgf G{s), and is distributed as discrete stable also. For the non-stationary situation, n-1 G(s) = Jim JJ exp { - Xh[l - ( l - (1 - s) - M(*2 - jh)h])f) " _ > 0 ° i=0 j=i = exp { - A(l - sf J'2 e-0fc ^ d t } which leads to model X(t2) = e-fc »{r)dT * X(h) + f2 e~fc ^ d T * de(t). Jti where / e~^2 ^T)dT * de(t) remains in discrete stable family. Ju Dase 2: K is from 12. Then, by straightforward calculation, f exp | - A ( 1 - 7)*(1 -sf £ H p exp(-/0/u£) 7s) - 7(1 -a)exp(-/i t)p /or constant n; G(s) = < exp | -A(1 - 7)^(1 - ^ / ' 2 exp (-Bg*n(T)dT Tdt (l-ys)-j(l-s)exp[-f;^(r)dr) for n(t). Hence, resulting models are X(t2) = e-^t2~tl) ® X(ti) + f2 11 e-i* © de(t), Jo and X(t2) = e-ti>[r)dT ®X{tx)+ P e-fc^)dT ®de{t). Jti corresponding to the stationary and the non-stationary case respectively. In Case 2, the generalized stochastic integrals are not in discrete stable family. 148 In the remainder of this section, we study two generalized convolution families: GC I and GC II. Theorem 5.2.5 Let {e{t);t > 0} be a GC I IIP with pgf of Ae = e(t + h) - e(t) of form GAe(s)=eW{h f ?^lv(dq)}. L 7(0,1) 1 - <l s J Suppose K(a) is a non-negative integer self-generalized rv with pgf GK(S; ot), which is differentiate with bounded first order partial derivative with respect to a. Then, it follows that G(s) = { exp exp < r I r 2 _ t l q[ 7(0,1) l i o 1 q [GK (a;e-^) - 1 qGK (s;e-^) dt V(dq) } , i / ( f- ' l c ' (^; f ; ' " ;" ' )- 1 L , | 1 , w [7(0,1) y t i l-qGK(s;e-ft 2^)drj for constant LI; for Li(t), and G(s) is a pgf. Hence, the generalized stochastic integrals ft2 h [e-^)K®de{t) and {e~ ̂  ^)dr^ @ d e ^ Jo J t\ ^ exist and are non-negative integer rv's. Proof: Omitted. As in the case of GNBC innovation processes case, we can obtain the further expression for a specific K. However, it may not be useful since the measure V(-) is not clear. Nevertheless, we use two specific K in the following. Corollary 5.2.6 Consider the specific self-generalized rv K from II and 12. The innovation is the GC I IIP. Then, it follows that I I : e x p j j / log( L ^ 7(o,i) v i - g - q{s - i)e-"fo-*.i) 1 — qs )v(dq)}, for constant LI; G(s) = { e x p I / \ I ~ L 7(o,i) L7t! 1 *** q(s - l)e~ ft2 q-q(s- l)e~St2^T)d- -dt V(dq)}, forLi(t). 149 12: G(a) = { 9 ( 1 - 7 ) , / (1 - g)(l - 7*) + (Q ~ 7)(1 ~ g ) e - ^ 2 - t l } 6 X P ^ /o,i) -M9 - 7) ^ / i ( Q - 7 ) " °  ( l - 7 ) ( l - ? s ) 6 X P < f /o,i) t / i ( 1 - « ) ( 1 i 2 g ( l - 7 ) ( s - l ) e - ^ 2 ^ T 7s) + ( q - 7 ) ( l - s ) e - / / 2 ^ T ) d -dt )]v(dq)}, for constant LI; V(dq)}, for n(t). Proof: Omitted. Example 5.8 Example 5.2 can be revisited as one member of the GC I family. GAt(s) = exp\h Theorem 5.2.7 Let {e(t);t > 0} be a GC II IIP with pgf of Ae = e(t + h) - e(t) of form -q{l-3){l-is) y '(o,i) J Suppose K(a) is a non-negative integer self-generalized rv with pgf GK(S; a), which is differentiate with bounded first order partial derivatives with respect to a. Then, it follows that -t 2-ti - Q [l _ QK (a;e-^)] [ I - ^ G K (s;e-*)] G(s) = { exp / / l-qGK (s;e-"*) dtj V(dq)j, for constant LI; exp / / J[j,i) \Jti - G K ( S ; e - / / 2 ^ ) ] [l -TGK ( * ; e - / / 2 ^ ) l-qGK (s;e-tt ^ d r ) dt ) V{dq) for Li{t), and G(s) is a pgf. Hence, the generalized stochastic integrals f2 h i^^K ® d e W a n d f2(e-tt^r)dr^ m d £ ^ Jo J ti ^ exist and are non-negative integer rv's. 150 Proof: Omitted. Like previous innovation cases, we can specify the self-generalized rv K to obtain the com- plete expressions of pgf or LT of generalized stochastic integral. However, most of expressions in- volve in integration over time t are not simply expressed. Hence, we only give two self-generalized rv K for GC II innovation in the following corollary. Corollary 5.2.8 Consider the two special self-generalized rv's discussed in II and 12, and the innovation is the GC II IIP. II: exp I £ (1 - e-^- t ! ) ) ( s _ !) f V{dq)+ 1 - 7 , 1 -q-q(s- l)e-^-^ log ' [ 7 , D \ Q V(dq) G(s) = { for constant //; exp q{S - 1) [(1 - 7 ) - 7 (a - l)e~ St 2 ^)dr^- ft 2 ^r)<J7 l-q-q(s- l)e~ /*2 ^ d r dtj V(dq)\, for fi(t). 12: ( OTJl f |Ya- ( (l-<?)(l-7s)+(<7-7)(l-s)e-^2-t i) \ . ( / o r constant fx; G(s) = { e x p { ( l - 7 ) ( l - 7 « ) / [ 7 l l ) [(!-<?)]? ( — 7 « ) + ( 9 - 7 ) ( l - s ) e J* 1 ( l - 9 ) ( l - 7 s ) - 7 ( l - 9 ) ( l - s ) ^ ^ ) r f t ] ^ ( d g ) ) , / o r n(t). Proof: Omitted. 151 5.3 Positive-valued innovation processes and examples For positive innovation processes, we choose the compound Poisson (with a distribution with posi- tive support) IIP, Generalized Gamma Convolution IIP and GCMED IIP. The LT's of the increment in these three kind processes are of exponential form. The families of Generalized Gamma Con- volution and GCMED include many distributions having domain on (0,oo). Hence, these lead to many continuous-time GAR(l) processes with positive-valued margins. The results are summarized by Theorem 5.3.1, 5.3.2 and 5.3.3 in the following. Theorem 5.3.1 Let {e(t); t > 0} be a compound Poisson IIP with pgf of Ae = e(t + h) — e(t) where fa{s) is a LT, and differentiate with bounded first order derivative. Suppose K{a) is a pos- itive self-generalized rv with LT 4>K(s;a), which is differentiate with bounded first order derivative with respect to a. Then it follows that <f>Ae{s) = exp {Xh[(f>0(s) - 1]} , exp { A T 1 ^ ( - l o g ^ s j e - " * ) ) -l]dt}, for constant p,; )) -l]dt], for fi{t), and 4>(s) is a LT. Hence, the generalized stochastic integrals and exist and are positive rv's. Proof: It is straightforwardly derived by Theorem 5.1.2. Example 5.9 Consider the innovation being a compound Poisson with Gamma IIP, in which the increment e(t + h) — e(t) has LT 152 Choose K a positive self-generalized rv. By (2) of Theorem 5.3.1, we have fas) = ^ exp < A *2—ti r e e-\og<t>K(s;e-i*) ) 7 - l dt}, for constant LI; exp{A£2 log for (s;e~fc Mr)dry dt), for fi(t). Let K be from P I , then fas) = { exp < A exp < A t 2 - * l t 2 9 9 + se-v* l\9 + se~ St2 Kr)dr dt> , for constant LI; dt), for Li(t). - 1 dt}, When 7 = 1 and for constant LI, this becomes fas) = 9 + s Let K be from P2 , then rt2 — t l exp < A r ( (1 - 7)ae"'rf fas) = { exp | A £ 2 9(1 - 7) + 07«(1 - e-"*) (1 _ 7 ) s e - / / 2 /x(r)dT - 1 0(1 - 7 ) +^7S (l-e-fc Mr )<2T - 1 for constant dt) , for n(t). When 7 = 1 and for constant LI, ' l + fas) = 1 -7 + e 0(1-7) A 1 - 7 y. 1 —7—#7 1+1 Theorem 5.3.2 Let {e(t);t > 0} be a GGC IIP with LT of Ae = e(t + h) - e(t) Ms) =exp{/i f log (^—)U(du)\, L 7(0,00) u + s } 153 where U(du) is a non-negative measure on (0, oo) satisfying / \logu\U(du) < oo, and / u~xU(du) < oo. 7(0,1] J(l,oo) Suppose K(a) is a positive self-generalized rv with LT 4>K(S\ ct), which is differentiate with bounded first order partial derivative with respect to a. Then it follows that fas) = { exp exp / [/ J(0,oo) UO "t2-tl log u u - log (f>K (s; e dt U(du) } , / f log '(0,oo) \Jti \u-\og4>K (s;e-ti2^dT dt U{du) for constant pt; }, for n{t), and fas) is a LT. Hence, the generalized stochastic integrals ft2-ti rtz rt2-ti rt2 , I {e-"t)K®de{t) and / ( Jo Jh v exist and are positive rv's. e-St 2^T)dr\ @ d e ^ / K Proof: Omitted. Similar to GNBC and other generalized convolutions innovation situations, we have to cal- culate the LT for specific member in GGC family by Theorem 5.1.2 or 5.1.3. Example 5.10 Consider the Gamma IIP innovation, in which the increment e(t + h) — e(t) has LT <])Ae(s) = ( ^ T ^ ) 7 / l = e x P { - 7^1og(l + Bs)}, where a, B > 0. Let K be from P I . By Theorem 5.1.2, we have rt2—ti e x p { - 7 ^ log (l + Bse dt > , for constant LI; fas) = I exp < —7 • log ( l + Bse' fc ^dT) dt} , for Li(t), which seems to be in the GGC family. 154 Example 5.11 Consider the inverse Gaussian IIP innovation, in which the increment e(t+h)-e(t) has LT <t>Ae(s) = exp { 7 h [l - (1 + 2 7 - 1 s ) 1 / 2 ] } , where 7 > 0. Let K be from P I . Then, we have r *2 - t i 4>{s) = { exp<; / 7 lo exp<; / 7 1 - (1 + 2 7 for constant fj-; ( l + 2 7 - 1 S e - ^ ^ ) d T ) 1 / 2 dt}, for fj,(t). Theorem 5.3.3 Let (e(i); i > 0} be a GCMED IIP with LT of Ae = e(t + h) - e(t) (f>At(s) = exp{/j / —^-U(du)\, ' . L 7(0,00) u + s J w/iere U(du) is a non-negative measure on (0,00) satisfying / u~lU(du) < 00. Suppose K(a) 7(0,oo) ?'s a positive self-generalized rv with LT fo(s;a), which is differentiate with bounded first order partial derivative with respect to a. Then it follows that '2-ti log fo (s; e~^) (s) = { exp / 7(0,00) it - logfo (s;e dt C/(du) /or constant JJL; exp < / /" J(0,oo) Jti U ,t2 log for ( s ; e - /* t 2 ^ T ) d T ) -dt [/(du) ^ , /or/i(t), logfo: (s ;e-J i ' 2 ^ T ) d T ) J and (j)(s) is a LT. Hence, the generalized stochastic integrals exist and are positive rv's. Proof : Omitted. 155 For K from P I , we can calculate the LT under the stationary case: exp( / f-log(u + e-"*) * 2 tl}u{du)\ 1 7(0,oo) LM o J > e x p { - / log ( LM7(0,oo) v u + s )[/(du)}. When *2 — *i °°> exp { - / log ( l-M7(o,oo) v u + se"^* 2 - ' 1) ) t / ( ( i u ) } - . e x P { I ^ ) 1 o g ( ^ ) ^ „ ) } , u + s which is the GGC class. For K from P2, we can calculate the LT under the stationary case: -V-l)**-'* ^dt]u(du)} cj)(s) = exp = exp = exp When t 2 - *i °°-> u r \J(0 (0,oo) /(0,oo) u(l - 7 + 7s) + (1 - 7 - wy)se-^ 1-7 / " * 2 - t l J , / u ( l - 7 + 7a) ^ x- / d l o § 7T^ ' — ^ + e . / i ( l - 7 - « 7 ) 7o" \ ( l - 7 - « 7 ) s 17 (du) /( ,oo) 1-7 , u ( l-7 + 7s) + ( l-7-w7 ) s e - M ( t 2 _ t l ) log /z(l - 7 - «7) ( l - 7 ) ( « + s) 17 (du) exp I 7(0 /(0,oo) exp 1-7 , u ( l-7 + 7s) + ( l-7-^7 ) s e ~ / i ( * 2 " i l ) log 7(o,, - 7 - «7) 1 - 7 log ( l - 7 ) ( u + s) u(l - 7 + 7s) U{du) . / x ( l - 7 - « 7 ) 5 ( l - 7 ) ( ^ + s) t/(du) } , / 0 oo) which is the GC III class. Furthermore, we can construct the following example. Example 5.12 Consider the measure U(du) is 9 on point 8, and 0 elsewhere. If K is from P I , it follows that ' exp log ('+'e~;(;a~'l)) } - (e-"te-*i) + [1 - e -" te-«i)] , /or constant w <£(s) = < t2 ( _ s p - / / 2 M r ) d r for n(t). 156 Thus, resulting models are X{t2) = e - ^ - f i ) . + ft2 t l e-nt . d c ( t ) ) Jo and X(t2) = e- fc ^T)dT . X(h) + t e~ fc ^T)dT . de(t), Jh corresponding to the stationary and the non-stationary case respectively. 5.4 Real-valued innovation processes and examples Finally, we consider the real-valued innovation processes to include all possible cases for the theory of continuous-time GAR(l) processes. Since for real rv's, the only choice among extended-thinning operations is the constant multiplier, our task is simply to choose proper innovation processes. Also the cf of K in this case is of form e~tas; hence, its first partial derivative with respect to a is bounded. First, we choose the compound Poisson (with a variance mixture of the normal distribution) IIP as the innovation process. Then, we shall choose the EGGC IIP as the innovation process. In particularly, we will calculate for a special case, the stable non-Gaussian distribution family, which includes Gaussian (when 7 = 2) and Cauchy (when 7 = 1). Hence, the classical continuous- time GAR(l) Gaussian process is included in our theory, but the process is defined in the sense of convergence in distribution, not in L2. Note that Cauchy distribution has no expectation. Therefore, it's impossible to construct a continuous-time GAR(l) Cauchy process in the sense of the Ito integral, but it works in the theory of continuous-time GAR(l) processes where convergence in distribution is used. Note that these processes with stable stationary distributions are already known in the literature, however, the convergence for stochastic integration there is in probability, not in distribution; thus, they can induce the processes with stable stationary distributions in this section. Interested readers can see Samorodnitsky and Taqqu [1994]. Here we just show that they can be unified by the theory of continuous-time generalized AR(1) processes. 157 Theorem 5.4.1 Let {e(t);t > 0} be a IIP of the compound Poisson with the variance mixture of the normal distribution, and the pgf of Ae = e(t + h) — e(t) <pAe(s) = exp {\h[fa(s2/2) - 1]} , where fa(s) is a LT of a positive rv, and differentiate with bounded first order derivative. Then it follows that ( ft2-ti ,- ,52g-2ut. , . exp|A / [fay— J - l J d i J , for constant LI; J 0 <P(s) = { exp s2e-2fp H{r)dr and ip(s) is a cf. Hence, the generalized stochastic integrals f2 ^ e-^ • de(t) and P e~ & ^ r ) d r • de(t) Jo Jti exist and are positive rv's. Proof: This is a direct conclusion from Theorem 5.1.3 since the extended-thinning operation is very simple, just the constant multiplier operation. We can show the rough calculation for the stationary case, i.e., constant \x case in the following. For K from P I , the cf is (PK{S; a) = eias, and - ilog <PK{S; a) = —i(ias) = as. Thus n-1 n-1 i=0 VzM - n^f-ilog^fed-Mr)) = IJ <PA, ((1 - lihfs) ( n-1 = exp { A/i ] P i=0 158 By Theorem 5.1.2, as n oo, this goes to tp{s) =exp{\f *2-*i r 's2e~2^ 0 o ( ^ — ) - l ] ^ } , and ip(0) = exp {\J 2 1 [̂ o(O) - = exp{0} = 1. Therefore, y>(s) is a LT. Note that the compound Poisson distribution has a positive mass at zero. These kind of distributions are useful in modelling zero-inflated data. Theorem 5.4.2 Let {e(t);t > 0} be a EGGC IIP with cf of Ae = e(t + h) - e(t) <PAe{s) =exp{ -h-s + (—00,00) log u ISU u — is I 1 + u1 U(du) } , where U(du) is a non-negative measure on (0, 00) satisfying f ^-^rU(du) < 00 and / | \ogu2\U{du) < 00. M m 1 + u •/iU|<i Then exp { - Is 2 [1 - e - 2 M t 2 - t i ) ] + IUS 1 _ e-2M*2-ti)] u(du)} 1 ir'v J(-00,00) lJo \ u — ise' dt exp { - Is2 t e~ tt 2 ^ d t +[ [ f log (- L Jti J(-00,00) lJh \i J* e-Stt2i*WTdt]u(dv.)}, ise' St 2 Mr)dr for constant LI; dt IUS 1+u2 for Li{t), and (p(s) is a cf. Hence, the generalized stochastic integrals t h e^ • de(t) and t e~ tt ^)dr . de(t) Jo Jti exist and are real rv's. Proof: This is the straightforward result from Theorem 5.1.3. However, we usually are not clear on the form of measure U(-). Thus, for a specific distribu- tion in the EGGC to be the innovation process, we have to calculate the ip(s) based on the specific form of the cf of that distribution. 159 Theorem 5.4.3 Let {e{t);t > 0} be a stable non-Gaussian IIP with cf of Ae = e(t + h) - e(t) <PAe(s) = exp{-Aft|s|7}, where A > 0 and 1 < 7 < 2. Then, it follows that [ exp|- <p(s) = { exp This leads to the model 7/x '}• -or -lit2 ^(-r)dr /or constant LI; for n{t). rt2—t\ X(t2) = e-^t2~tx) • + / e^* • de(i), and X(t2) = e-^i^dT . JTfo) + r e - ^ ^ d T • de(t), corresponding to stationary and non-stationary case respectively. The stochastic integrals P ^ e^ • de{t) and P e~ ^ ^ d r • de{t) Jo J ti remain in the same distribution family as the innovation. Proof: Directly applying (3) of Theorem 5.1.3 can lead to this theorem. However, we can check some calculations by using Theorem 5.1.2. For the stationary case, we have n-1 n—1 (p(s) = lim T]ipAe((l-pihYs] = lim TT exp ( - Xh\(l-/ifc^sl7} n—>oo A - L \ / n—>oo - L - L L J i=0 i=0 n-l s . t 2 _ t l = Jiirn expj - A ^ f c ( l - ^ ) 7 i | s | 7 | = exp{-A | s | 7 y e'^dtj — exp A[l - e"7^*2-*1)] JLl Similarly, for the non-stationary case, n-1 i n-1 * .. <p(s) = t^J\(PAe(s]][l-tJi(t2-jh)h]) = Jiirn J J e x p [ - A / i | s f J [ l - / i ( t 2 - i / i ) ^ | 7 } n 0 0 i=0 j=l i=0 i=i 160 n—1 i = Mm e x p { - A | S p ^ / i ( j ] [ l - u ( t 2 - j 7 l ) / l ] ) } i=o j = i = exp { - A( j ' ' [e-jti2^dT]dt) Since ip(s) is still of the form exp{-/3|s|7}, we conclude that t2 [ t 2 e""* • de(t) and /" * e~ ̂  " ( r ) d r • de(t) with cf </>(s) in the same family as the innovation. Applying the generalized Ornstein-Uhlenbeck SDE theory, we obtain the models in this theorem corresponding to stationary and non-stationary case respectively. Example 5.13 Consider Cauchy IIP. Then the increment e(t + h) - e(t) has cf: (fAe(s) = exp{-/i|s|}. By the above theorem, we have ' e x p l - Z J - ^ l - e - ^ - ^ H s l } , <P(*) = { for constant LI; / x p H r • ft* l*(T)*r dt)\s\\, for n{t). Example 5.14 Consider Brownian motion. Then the increment e(t + h) — e(t) has cf: <P&e(s) = exp{-/is 2/2}. By the above theorem, we have \ exp {-(ALL)'1 [l - e-Mt2-h)j s 2 | ^ jor c o n s t a n t ^ <p(s) = I exp •ft2 »(r)dr d?j S 2 } , for n(t). This means for the stationary case, Jo e'^ • de{t) ~ J V 0, 1 _ e - 2 M ( t 2 - t i ) ' 161 and for the non-stationary case, j*2 e - ft2 Mr)*- . de{t) „ N (o, [e"St2 »W-r]dtj . 5.5 Tweedie innovation processes This is another viewpoint to choose the innovation processes. The Tweedie family includes many of the distributions discussed in the previous sections in this chapter such as compound Poisson, Gamma, inverse Gaussian, stable distribution, and so on. Although this overlaps with the previous discussion, we would like to revisit or summarize this case from the perspective of dispersion. The Tweedie family consists of three types of distributions: non-negative integer support, positive support and real support. Al l the distributions in this family have the mgf of special exponential form: Mx(s;6,X,8)=E[esX] = { ' e x p J A ^ ^ ) ' [(1 + ^ - 1 exp d ^ l , 2 ; d = 2; d= 1, where d = jjrf or 8 = j^j. In specific, the non-negative integer case includes only one distribution,' that is Poisson distribution when 0 = 1. The positive case includes the compound Poisson with Gamma distribution (1 < d < 2), Gamma (d = 2), positive stable (2 < d < 3 or d > 3) and inverse Gaussian (d = 3). The real case includes normal (d = 0) and extreme stable (d < 0 or d = oo). Refer to Section 2.3.2. The innovation and the self-generalized rv should be of the same type. That is, if the increment of innovation is to be Poisson, then the self-generalized rv K should be non-negative integer-valued, which leads to the choice like II, etc. If the increment of innovation is to be a positive rv, we can choose a positive self-generalized rv K like PI. For the real case like normal and extreme stable, K can only be PI. We give the following theorem without proof. 162 Theorem 5.5.1 Let {e{t);t > 0} be a Tweedie IIP with mgf of Ae = e(t + h) - e(t) ' exp j f c A ^ [(1 + ^ - l] } , d+1,2; M A e ( 5 ; 0 , A , / 3 ) = E [ e ^ ] = i (l + fx)~hX, d = 2; ^ exp {hXe9 [eslx - l] } , d = 1, 1. Suppose K(a) is a non-negative integer self-generalized rv with pgf GK{S; a). In this situation, the only non-negative integer distribution is Poisson and d = 1. Then, it follows that r*2—*1 r G(s) = exp exp dt} for constant p,, for Li(t), and G(s) is a pgf. Hence, the generalized stochastic integrals ^ (E~^) K ® D E ^ A N D (E~ /T<2 KT)DT) K ® D E ^ exist and are non-negative integer rv's. 2. Suppose K(a) is a positive self-generalized rv with LT (f>K{s;a>). In this situation, d corre- sponds to (1,2), {2}, (2,3), {3} and (3, oo). Then it follows that for constant p, M(s) = I exp < A 0 rt2-ti ex j dt d#2; d = 2; and /or //(£), M(s) = ^ exp A0-r ( * ) ' f ' l o g ^ ( - s ; e - ^ V ( T ) d T ) 1 H - ^ ex - I dt d + 2; f ft2 ( \og<pK(-s;e-fc^d e x p j - A ^ ^1 + ^ - dt d = 2; 163 and M(s) is a mgf. Hence, the generalized stochastic integrals f2 h (e-"*) K® de(t) and F (e~ ^ ^dT) ®de(t) Jo Jti ^ ' K exist and are positive rv's. 3. Suppose K(a) is from PI with LT <px(s;a) = e~as. In this situation, d corresponds to (—oo,0), {0} and {oo}. Then it follows that M(s) = { i + se - 1 1 + ex s e ~ ft2 Mi")dr ' 9X dt } , for constant LI, l\dt), forn(t), and M(s) is a mgf. Hence, the generalized stochastic integrals j ' 2 ^ (e""*) K ® de(t) and . (e~ & ̂ r)dT) r ® de(t) exist and are real rv's. 164 Part III Probabilistic and statistical properties 165 Chapter 6 Stationary distributions, steady states The continuous-time GAR(l) processes constructed in Chapter 4 are first order Markov processes. Hence, it's possible that the stationary distribution, namely the limiting distribution of the process, could exist. These are discussed in Section 6.1. Also if the stationary distribution exists, then the process will evolve under steady state when starting from the stationary distribution. This means that X(t) is distributed as the stationary distribution for all t. We study three cases of margins in Section 6.2. Such a steady state process offers a reasonable good model for a stationary time series. This motivates us to study the possible margins under steady states. For margins with specific distributions of interest, we propose a general approach to fit such a need in Section 6.3. In other words, we are trying to investigate the continuous-time GAR(l) processes from the perspective of In Section 6.4, we discuss the generalized AR(1) time series obtained from the continuous- time GAR(l) processes via equally-spaced time observations. They cover many of the first order autoregressive non-Gaussian time series existing in the literature. time series state space. 166 6.1 Stationary distributions Assume in this section that {K(a)} has bounded expectation for a G (0,1]. The stationary dis- tribution, if it exists, is the long run result of a stationary or homogeneous process. Hence, it is independent of "time". This means that as the process evolves, the distribution of the margin X(t) will finally reach a fixed or invariant equilibrium. From the view of state space, i.e., the support of X(t), we are interested in that if there is a stationary distribution for the continuous-time GAR(l) process. Now we look into the structure of the continuous-time GAR(l) process. By part (1) of Theorem 4.4.2, X(t2)= (e~'ite-V)K®X(t1) + ^ (e-^)K®de(t), h < t2, where / ( e - ^ ) ©de(t) = E(0,t2 - t i ) , the integrated innovation or cumulative innovation, Jo ^ / K has the following pgf G E { 0 ^ h ) ( s ) , or LT <pE{0M-ti)(s)i o r c f < / > E ( o , t 2 - i i ) ( s ) : n-1 GE{0M-ti)(s) = l i m FI °Ae {GK (S; (1 - , i f t h e support of e(t) is jV, n->oo i;_Q n-1 <l>E(o,t2-ti)(s) = hm n <Me (-log for (s ; ( l - M ) 1 ) ) , if the support of e(t) is &+,• n->oo i = Q n-1 <PE(ot2-ti)(s) = L I M IT VAe (i log <p K {s; {I-phy)), if the support of e(t) is SR. i=0 Here h = (t2 — t i)/n, and {e(i); £ > 0} is a stationary independent increment process of innovation. Ae is the increment with time lag h in the innovation process. First we study the dependence term (e -^* 2 -* 1))^ ®X(t\) to see its limiting behavior. Fix time t i , and let t2 -t oo. Then the time difference t2 - ti —>• oo, which leads e~^t2~tl^ -¥ 0. For the margin being a real rv, the extended-thinning becomes the constant multiplier. Hence, ^e-^* 2 -* 1^ ®A"(ti) = e " ' l ( t 2 - * l ) - X ( t i ) ^ 0 , * 2 — * i —> oo. This means that the dependence term will finally diminish to zero. It leaves us a hint that this could be true for margins being positive or non-negative integer-valued. Revisiting Property 3.14 167 and 3.15, we obtain (e-ri**-^ ®X(ti) —>{0)K®X{t1) = 0, a s * 2 - * i -> oo. Therefore, as time goes to infinity, the margin of a continuous-time GAR(l) process will be Consequently, this integral will have the pgf Goo(s) = lim GE(o,t2-ti)(s), or LT 0oo(s) = lim cpE(0t2_tl)(s), or cf Voof» = lim VB(o,ta-ti)(a). C2—tl—>00 where Goo(s) is continuous at s = 1 with Goo(l) = 1, or <f><x>{s) is continuous at s = 0 with ôo(O) = 1, or <£>oo(s) is continuous at s = 0 with <Poo(0) = 1. It is possible to calculate the explicit form of pgf, or LT, or cf of the stationary distribution. Note that the Goo(s), or </>oo(s) or <poo(s) is of product form of the pgf, or LT, or cf of the increment of the innovation process. Hence, we can choose special innovation process, in which the increment, e(t + h) — e(t), has the exponential form of pgf, or LT, or cf. Of course, the choice of extended- thinning should be appropriate too. This leads to the following theorem. Theorem 6.1.1 Assume the innovation process {e(t); t > 0} has increment e(t+h) — e(t) such that its pgf, or LT, or cf is of form ehC^s\ depending on the increment being non-negative integer-valued, or positive-valued, or real-valued. C(s) is assumed to be differentiate with bounded first order derivative. Also assume GK{S; a) and logGK{S; a), or (f>i((s;a) and log <J>K{S; ct), or log <£K(S; a) have bounded first order derivative with respect to a in [0,1] (boundaries could be excluded). It follows that for the stationary continuous-time GAR(l) process X(t2) = (e~^-^)k®X{h) + j'' h {e^t)K®de(t), h < t2, 168 if the following integrals exist: poo poo poo / C(GK{s;e-i*))dt, / C ( - log for^e-"*) )* , / C {-ilog<pK {s; e""')) dt, Jo Jo Jo then the stationary distribution exists, and has the following pgf, or LT, or cf according to non- negative integer, or positive, or real margins: Goo(s) = exp |y C ( G K (s; e~M*)) dt} , if the support of e(t) is jV, ' (f>oo{s) = exp ij C (— log 4>K {s; e _ / 2 t)) dt} , if the support of e(t) is SR+, Voo(s) = e x P G (—i log (fx (s; e _ , i t )) dt} , if the support of e(t) is SR. poo The corresponding rv is the generalized stochastic integral / (e_A(t)j<- ®de(t). Jo Proof: It is straightforward to derive them by Theorem 5.1.2. By Theorem 5.1.3, we know if K is from II, 12, 14, 15, P I , P2, P4 and P5, then Theorem 6.1.1 holds. This theorem is not valid for 13, P3, because GK(S;0) ^ 1 for 13, and 13, P3 do not have finite expectations. In the next section, we will discuss the situations of steady state where the relevant stationary distributions have support on j\f, or SR+ and SR. 6.2 Marginal distributions under steady state The stationary distribution is a particular feature of a homogeneous Markov process. If the station- ary distribution exists, then this process has steady state when starting just from this stationary distribution. Typically, the margins of a Markov process, do not have the same distributions when the process evolves. However, under steady state, all marginal distributions are the same as the stationary distribution. 169 In this section, we shall investigate the particular types of marginal distributions under steady state; they may have non-negative integer, or positive, or real support. These results mainly correspond to those continuous-time GAR(l) processes constructed in Chapter 5. In statistical practice, often we encounter time series data (observed on equally or unequally spaced time points) which are near stationary. Hence, an assumption of stationarity is reasonable. Of course, obvious non-stationary situations like trend, seasonality, and so on can happen too. This leads to a general principle of modelling for observations over time: define a stationary process for the time series first, then make parameters depend on covariates to define a process with time- varying marginal distribution. For this purpose, constructing a steady state process which has the same distribution on every margin is the first consideration of modelling. Thus, it gains more attention from statisticians. 6.2.1 Non-negative integer margins First, we turn to the continuous-time GAR(l) processes constructed in Chapter 5, which have non- negative integer-valued margins. We shall study the limiting behavior of these processes as t2 — t\ goes to infinity. We only consider the stationary SDE case, namely, constant LI and stationary IIP innovation. These limiting behaviors lead to the stationary distributions, and the resulting processes have the same distribution as the marginal distributions under steady state. To guarantee the margins being non-negative integer-valued, the self-generalized rv K in- volved in extended-thinning operation should be non-negative integer-valued, and the increment of innovation process {e(t);t > 0} should be non-negative integer-valued too. Hence, K could be from II to 15, while the innovation process can be the compound Poisson (with a non-negative integer distribution) IIP, GNBC IIP, GC I IIP and GC II IIP. The results for the general non-negative integer-valued self-generalized rv K are given in the following theorem. Theorem 6.2.1 Suppose K(a) is a non-negative integer self-generalized rv with pgf GK(S;O:), which is differentiate with bounded first order derivative with respect to a. Consider the stationary generalized Ornstein- Uhlenbeck SDE where LI is a constant. 170 (1) Let {e(i);t > 0} be a non-negative integer-valued compound Poisson IIP with pgf of Ae = e(t + h)-e(t) GAe(s)=exV{Xh[g(s)-l]}, oo where g(s)(= J2, Pis%) * s a P9f> and differentiate with bounded first order derivative. Then, the limiting pgf is Goo(S) = exp{A r [g(GK(s;e-^)) -l]dt). J 0 (2) Let {e(i); t > 0 } k a GNBC IIP with pgf of Ae = e(t + h) - e(t) G a < W = e x P { A / ( w ) l o g ( r ^ ) ^ ) } . 1 If / log [l-qGK (s;e-^) Goo(s) = exp^ / ( / I 7(o,i) \7o dt < oo for any q G (0,1), then the limiting pgf is log P dt V(dq) } • l-qGK (s;e-^) (3) Let {e{t);t > 0} be a GC I IIP with pgf of Ae = e(t + h) - e(t) of form q(s - 1) sAe{S) = exp W i I 1 7(0,1) r°° q [GK (s;e-^) -1 ] GA (s)=exv{h q-^-V(dq)}. L (o,i) L - qs J C a \ GK \S' If / q 1 v ' —'-—-^-dt < oo for any q £ (0,1), then the limiting pgf is J0 1 - qGK [s; e-A**) GM = eXPV(o,i) \ h l-qGK{s;e-^) dt V(dq) (4) Let {e{t);t > 0} be a GC II IIP with pgf of Ae = e(t + h) - e(t) of form -q(i-8)(l-is)v{dq)y GAe(s) = exp \ h (0,1) 1 — qs To Goo(s) =exp\ / I 7[7,1) \70 1 - 9 G A - ( s ; e - " 4 ) °°-q[l- GK (s;e-^)] [ l - 7GK ( 8 ; e - * ) ] l-qGK ( s ; e - " * ) dt F(dg) > . 171 rOO All these Goo(s) are the pgf's of the generalized stochastic integral / (e-^*) K © de(t). Jo Proof: Straightforward to derive from Theorem 6.1.1. We now verify the existence of stationary distributions for specific self-generalized rv's K and innovation processes. These stationary distributions are the Poisson, negative binomial, geometric, power series, discrete stable distributions, and the GNBC class. Example 6.1 (Poisson) Consider Example 5.1, where p Thus, as t2 — t i -» oo, the limiting pgf is A -I H which implies that the stationary distribution is Poisson(Xfp). G E t o M - t ^ s ) = exp { £ [ 1 - c - * f a - ' i ) ] ( s - 1)}. Goo{s) = exp j - ( s - 1)}, Example 6.2 (Negative binomial and geometric) First consider Example 5.2. Then G E { o M - h ) ( s ) = (e-"<*-*> + [1 - e-^)]^-fr This leads to the limit ^ ) = ( ^ ) A / " . indicating that the stationary distribution is NB(X/p, 7). When A = p, it's the geometric distribu- tion with parameter 7. Secondly consider Example 5.4, where I _ 7 ( l _ e - M ( t 2 - * i ) ) \ M l - 7 ) / ( « 7 ) ,^ _ j l - 7 e - M ( t 2 - t i ) * bE(o,t2-ti)W - 7 ( 1 - e - M ( t a - ^ r As t2 — ti —> 00, we have which shows that the stationary distribution is NB^1^1^, 7^. When A(l — 7) = /17, it's the geometric distribution with parameter 7. I _ 7 \ A ( l - 7 ) / ( / n ) 172 Example 6.3 (Power series) Consider Example 5.3. We choose A = LI, then l - ( l - 8 ) i / g 1 + (S - l ) e - ^ 2 - t i ) Ti goes to Goo(5) = s _ 1 [l — (1 — s)l/e] as ti — t\ —> oo, namely the stationary distribution is the power series distribution. Example 6.4 (Discrete stable) Consider case 1 in Example 5.7. Then GE{o,t2-h)(s) = exp { - - i — 1(1 - sf}, When t2 — t\ —> oo, we obtain Goo(s) '= exp | — ^ (1 — s)^}, thus, the discrete stable distribution. Example 6.5 ( G N B C ) First, we consider Case I I in Corollary 5.2.6, where when t2 — t\ —&