NUMERICAL PARAMETER ESTIMATION IN DIFFERENTIAL EQUATIONS MAURICE W. BENSON B.Sc, Lakehead University 1971 M.Sc, Lakehead University 1973 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE STUDIES (Department of Computer Science and The Institute for Appli Mathematics and S t a t i s t i c s ) We accept this thesis as conforming to the required standard. THE UNIVERSITY OF December (c) Maurice W. BRITISH COLUMBIA , 1977 Benson, 1977 In presenting th is thes is in pa r t i a l fu l f i lment of the requirements for an advanced degree at the Un ivers i ty of B r i t i s h Columbia, I agree that the L ibrary shal l make it f ree ly ava i lab le for reference and study. I fur ther agree that permission for extensive copying of th is thesis for scholar ly purposes may be granted by the Head of my Department or by h is representat ives. It is understood that copying or pub l i ca t ion of th is thes is for f inanc ia l gain sha l l not be allowed without my wri t ten permission. Department of 7 " The Univers i ty of B r i t i s h Columbia 2075 Wesbrook P l a c e Vancouver, Canada V6T 1W5 Date ^J2*Q/rJL, ,/f77 i i ABSTRACT The problem of numerical least squares parameter estimation in d i f f e r e n t i a l equations i s considered. Several new algorithms that pay pa r t i c u l a r attention to the d i f f e r e n t i a l equation aspect of the problem are presented. These reduce some of the d i f f i c u l t i e s encountered when the problem i s treated solely as a question of nonlinear optimization. The extremely powerful int e r a c t i v e approach i s considered and an interactive package incorporating standard techniques using s e n s i t i v i t y equations along with a selection of our special algorithms i s presented. We consider methods involving the f i t t i n g of integrals and derivatives using piecewise polynomial approximations to the observations. Continuation methods with a quasi multiple shooting technique to bridge the gap between these coarse but well behaved methods and the f u l l least squares problem are explored. Special methods are developed for the important case of two state variables with observations available on only one of them. In p a r t i c u l a r we consider algorithms which use an i n i t i a l guess at the behavior of the unobserved state variable and then i t e r a t i v e l y improve th i s guess. The need for e f f e c t i v e algorithms for f i t t i n g population growth models in ecology i s one motivation for this thesis. We devote a chapter to an important predator-prey model of population dynamics and extensive experiments are presented i i i which demonstrate some of the t y p i c a l d i f f i c u l t i e s which can arise and which i l l u s t r a t e the a b i l i t y of our algorithms to overcome some of these d i f f i c u l t i e s . Some special problems involving jumps from one equilibrium to another (loosely referred to as catastrophes) are examined. This type of model has important applications in ecology. Models involving s t i f f d i f f e r e n t i a l equations are also considered. A short chapter i s devoted to the use of sequential reestimation techniques. Experiments indicate that such methods can be useful for improving a crude i n i t i a l guess at the parameters and this improvement can be c r u c i a l for the successful solution of the problem. F i n a l l y a chapter i s devoted to a selection of "real world" problems. It i s on such problems that the true value of an algorithm i s determined. i v CONTENTS INTRODUCTION 1 CHAPTER 1-NOTATION AND BACKGROUND 4 1.1 INTRODUCTION AND BASIC NOTATION 4 1.2 MOTIVATION 8 1.3 MORE BACKGROUND 10 CHAPTER 2-OPTIMIZATION AND INTEGRATION 21 2.1 NONLINEAR LEAST SQUARES 21 2.2 INTEGRATION OF MODEL AND SENSITIVITY EQUATIONS 26 2.3 WHEN TO INTEGRATE THE SENSITIVITY EQUATIONS 31 2.4 AN EXAMPLE WITH KNOWN DIAGONAL COVARIANCE MATRIX 38 CHAPTER 3-SPECIAL METHODS FOR THE INTERACTIVE APPROACH 42 3.1 THE INTERACTIVE APPROACH 4 2 3.2 DERIVATIVE FITTING (DFIT) 46 3.3 INTEGRAL FITTING (IFIT) 57 3.4 ITERATED INTEGRAL AND DERIVATIVE FITTING 62 3.5 CONTINUATION AND QUASI MULTIPLE SHOOTING 74 3.6 IMPLEMENTATION OF AN INTERACTIVE PACKAGE 83 CHAPTER 4-PARAMETER FITTING IN A PREDATOR-PREY DYNAMIC MODEL .95 4.1 INTRODUCTION 95 4.2 A PREDATOR-PREY DYNAMIC MODEL 95 4.3 IMPROVING STARTING PARAMETERS 102 4.4 GUESSED OBSERVATIONS AND ITERATED METHODS 106 4.5 THE PRESENCE OF NOISE 115 CHAPTER 5-SEQUENTIAL TECHNIQUES 123 5.1 INTRODUCTION 123 5.2 A SEQUENTIAL ALGORITHM 124 5.3 EXPERIMENTAL RESULTS 128 CHAPTER 6-REAL WORLD PROBLEMS 132 6.1 INTRODUCTION 132 6.2 A DYNAMIC MODEL FOR AGGRESSIVE AND DOCILE MICE 133 6.3 A MODEL INVOLVING A CHANGE IN EQUILIBRIUM 147 6.4 A REINDEER POPULATION GROWTH MODEL 155 6.5 AN OCEAN PLANKTON MODEL 163 CHAPTER 7-CONCLUSIONS AND FUTURE WORK 167 7.1 CONCLUSIONS 167 7.2 SUGGESTIONS FOR FUTURE WORK 169 BIBLIOGRAPHY 170 APPENDIX A 176 V APPENDIX B 205 v i FIGURES 2.2.1 D i s c r e t i z a t i o n of s e n s i t i v i t y equations 30 3.2.1 Equilibrium change—no error in observations 55 3.2.2 Equilibrium change—error in observations (<r=l) 56 3.2.3 Equilibrium change—error in observations (<r=2) 57 3.3.1 Equilibrium change—no error in observations 61 3.3.2 Equilibrium change—error («r=l) 61 3.3.3 Equilibrium change—error (<r=2) 62 3.4.1 Simulation results and spline approximation 70 3.4.2 Iterations on the guessed observations 71 3.4.3 Observations and successive integrations of y, 71 3.4.4 Iterations on guessed observations 73 3.4.5 Integrations at successive parameter estimates 73 3.5.1 Data and smoothing for continuation tests 78 3.5.2 Results using break points with Tf = .2 80 3.5.3 Results at optimal parameters 80 3.6.1 Overall strategy 88 3.6.2 Refined parameter f i t t i n g 89 3.6.3 Interactive optimization 90 3.6.4 Derivative and integral f i t t i n g 91 3.6.5 Guessed observations and i t e r a t i v e improvement 92 3.6.6 Continuation and quasi multiple shooting 93 3.6.7 Sequential reestimation (not implemented in PARFIT) ... 94 4.2.1 Phase plot for case (1) 98 4.2.2 Phase plot for case (2) 98 4.2.3 Phase plot for case (3) 99 4.2.4 Observations for Problem 4.2.1 100 4.2.5 Observations for Problem 4.2.2 101 4.2.6 Observations for Problem 4.2.3 101 4.3.1 A l o c a l minimum for Problem 4.2.1 103 4.3.2 A l o c a l minimum for problem 4.2.2 104 4.3.3 A l o c a l minimum for Problem 4.2.3 106 4.4.1 Guessed observations for problem 4.4.1 108 4.4.2 FIT on Problem 4.4.1 109 4.4.3 DFIT+FIT on Problem 4.4.1 using guess (b) 110 4.4.4 Iterated DFIT results 112 4.4.5 Iterated DFIT guessed observation i t e r a t i o n s 113 4.4.6 Iterated IFIT results 114 4.4.7 Iterated IFIT guessed observations 114 4.5.1 Results for Problem 4.5.1 117 4.5.2 Results for Problem 4.5.2 118 4.5.3 DFIT results 120 4.5.4 IFIT results (using subsystem integrations) 120 4.5.5 IFIT results (using (3.4.1)) 121 4.5.6 FIT results (p^. scaled) st a r t i n g from IFIT results — 122 6.2.1 Observations and spline approximation 135 6.2.2 Iterations on guessed observations 137 6.2.3 Integrations at successive parameter approximations .. 138 6.2.4 Optimum star t i n g from iterated IFIT results 139 6.2.5 Integration at parameters found i n t e r a c t i v e l y 141 v i i 6.2.6 Optimum sta r t i n g from parameters found i n t e r a c t i v e l y . 142 6.2.7 Optimum with logarithmic scaling on p^ . and p 7 146 6.3.1 Phytoplankton observations and smoothing function .... 148 6.3.2 Zooplankton densities 150 6.3.3 Integration at IFIT results 151 6.3.4 Integration results at IFIT+FIT results 151 6.3.5 Integration at i n t e r a c t i v e l y obtained optimum 153 6.3.6 Integration near i n t e r a c t i v e l y obtained optimum 154 6.4.1 Observations and integration results at p t o > 157 6.4.2 Integration results for Experiment 1 158 6.4.3 Integration at results of Experiment 3 160 6.4.4 Integration results at optimum of Experiment 4 ,.. 161 6.5.1 Observations and best f i t for y7 165 6.5.2 Observations and best f i t for y a 166 6.5.3 Observations and best f i t for yg 166 A. l A t y p i c a l model d e f i n i t i o n procedure 183 v i i i TABLES 2.3.1 Work rati o s W,/Wa 33 2.3.2 Some work rati o s for a predictor-corrector method 36 2.3.3 Work ra t i o s for a s t i f f method (<*=.3) 38 2.3.4 Work rati o s for a s t i f f method (oi = .6) 38 2.4.1 Observations 39 2.4.2 Optimization results 40 3.2.1 Observations for s t i f f problem 52 3.5.1 A continuation experiment 81 4.3.1 FIT compared with DFIT+FIT and IFIT+FIT 102 5.3.1 Results with sequential approach 130 6.3.1 Roots of (6.3.2) corresponding to Figure 6.3.4 152 6.3.2 Roots of (6.3.2) corresponding to Figure 6.3.5 154 ix ACKNOWLEDGEMENTS I would l i k e to thank my supervisor, Professor J . M. Varah, for his advice and encouragement during the preparation of t h i s thesis. I would also l i k e to thank Dr. C. J. Walters for keeping me aware of the "real world" aspect of the problem. F i n a l l y , I would l i k e to express my appreciation for National Research Council of Canada Scholarship support and Killam Pre-Doctoral Fellowship support during the preparation of this thesis. 1 INTRODUCTION Parameter f i t t i n g in dynamic models occurs in a wide variety of f i e l d s (see for example Section 1 . 2 ) . In many cases, standard procedures (employing the s e n s i t i v i t y equations as outlined in Chapter 2) s u f f i c e . However, in a substantial number of cases, this approach i s extremely sensitive to poor i n i t i a l guesses at the optimal parameters (see for example Chapter 4 ) . The main purpose of this thesis i s to develop algorithms and strategies designed to overcome poor or absent i n i t i a l approximations to the optimal parameters. Our basic philosophy for attaining this goal i s to avoid the f u l l nonlinear optimization problem as much as possible during the early stages of parameter estimation. The interactive approach i s ideal for addressing this problem and the algorithms developed in thi s thesis are designed with user intervention in mind. Indeed there are cases, such as the one in Section 6.3, where an interactive approach appears to be the only way to obtain certain solutions. Chapters 1 and 2 establish our notation and provide a background for the numerical integration and optimization procedures employed throughout the thesis. In Chapter 3, we begin our development of special procedures. We st a r t Chapter 3 with the standard approaches of derivative and integral f i t t i n g and then we expand on these techniques later in the chapter. These techniques, which INTRODUCTION 2 involve the use of smoothed approximations to the observations on the state variables, are obvious candidates for an int e r a c t i v e approach. These methods are compared on a s t i f f problem and on a problem involving a change in equilibrium. In Section 3.4, we develop techniques (which employ guessed observations and i t e r a t i v e improvement of guessed observations) for extending the methods of Sections 3.2 and 3.3 to the important case when observations are not available on a l l state variables. As i l l u s t r a t e d in Sections 4.4 and 4.5, these iterated methods provide a powerful tool (especially in an interactive environment) for handling the case where some state variables are unobserved. Again, our basic philosophy, of avoiding the f u l l nonlinearity that arises with the d i r e c t approach, l i e s behind the success of these methods. In Section 3.5, we present systematic approaches using break points and continuation parameters for bridging the gap between the r e l a t i v e l y coarse integral f i t t i n g technique and the f u l l nonlinear problem. These methods are highly interactive by nature, and again, they attempt to ease the approach to the f u l l nonlinear problem. As shown in Section 3.5, these methods can be useful for overcoming i n s t a b i l i t i e s . We end Chapter 3 with a discussion on how the various methods developed should be incorporated into an interactive package. A discussion of e f f e c t i v e strategies employing our special technigues in an interactive environment i s also presented at the end of Chapter 3. INTRODUCTION 3 Chapter 4 gives extensive experiments comparing the techniques of Chapters 2 and 3 on a s p e c i f i c problem. In Chapter 5, we continue our search for methods which reduce the e f f e c t of the nonlinearity associated with a d i r e c t approach. In p a r t i c u l a r a sequential approach i s shown to be e f f e c t i v e in several cases where a d i r e c t method encountered d i f f i c u l t i e s . When solving a parameter f i t t i n g problem involving a dynamic model, the p a r t i c u l a r strategy employed can be as important as the choice of algorithms. This i s e s p e c i a l l y the case when an interactive approach i s used. In Chapter 6 we present d e t a i l s of successful strategies on four "real world" problems where the observations were obtained from physical experiments and not computer simulations. (A condensation of our experience with parameter f i t t i n g in dynamic models is contained in the flow charts at the end of Chapter 3.) The d e t a i l s of such strategies of course vary from problem to problem; however, as experience with such problems grows, certain strategies emerge as being more e f f e c t i v e than others. One such strategy that has proved highly e f f e c t i v e involves the temporary freezing of parameters. INTRODUCTION 4 CHAPTER 1 NOTATION AND BACKGROUND 1.1 INTRODUCTION AND BASIC NOTATION We are interested in the problem of f i t t i n g dynamic models to observations and wish to pay special attention to the d i f f e r e n t i a l equation aspect of the problem. Our dynamic models are of the form y'=g(t,y,p) (1.1.1) Y(t 0)=y o(p) where y i s an n-vector of state variables, p i s an m vector of parameters, t i s the independent variable which we c a l l time for convenience, and * denotes d i f f e r e n t i a t i o n with respect to time. Along with the above i n i t i a l value problem we have a set of observations v; ,...,v^ taken at d i s t i n c t times t , . . . respectively where _>tfl , 1 = 1,..., k. Each v^ i s an r-vector where r<n. That i s , not a l l components of y need be observed. Define the weighted residual vector f of length kr by for s=l,...,r; 1=1,...,k; where v i s component s of v^ and y ., . (t.) i s the corresponding element of the vector y ( t . ) . 41. A) X A Wl[£ i)+A '""S 3 weighting factor. The weighted least squares problem i s to find p to minimize F(p)=f T(p)f (p) . (1.1.3) CHAPTER 1 5 The use of weights in the above function allows us to handle some maximum l i k e l i h o o d problems (see Section 2.4). Adapting the notation in Bard[5], l e t V p , - ( f * u - . > * i W T ' ( 1 - 1 - 4 ) k M(P)= X. e Ap)e^(p) . (1.1.5) 1 = 1 * where, for the moment, we are taking a l l weights equal to one. Further l e t the^v,"^ have normally distributed measurement errors with zero mean and covariance matrix V. When there are no errors in the t^ , the maximum l i k e l i h o o d estimate of p i s found by minimizing .5Tr(V _M(p)) (1.1.6) where Tr denotes the trace operator. (We are assuming the errors in the observations taken at d i f f e r e n t times are uncorrelated.) When V i s a diagonal matrix, (1.1.6) reduces to the form of the function in (1.1.3). It i s this special case we consider in Section 2.4. For a l i s t of some other objective functions we refer the reader to Bard[5]. The dynamic model considered above i s a special case of the standard dynamic model described by Bard[6,p221]. He considers problems where g and y of (1.1.1) are functions of a vector, x, of independent variables in addition to the above arguments and where the observations available are on variables which are given functions of t, y, p, and x. One of our aims in this CHAPTER 1 6 thesis i s to investigate special methods which handle some of the d i f f i c u l t i e s associated with f i t t i n g parameters in a d i f f e r e n t i a l equation, and, to avoid unnecessary complications, we confine our attention to models of the form (1.1.1) with observations taken d i r e c t l y on the state variables. The problem of finding p to minimize (1.1.3) can be nasty. Bard[6,p231] gives a concise description of some of the d i f f i c u l t i e s that can occur. Generally, the problem i s d i f f i c u l t because of the vast range of solutions that (1.1.1) can have as a function of p. This can result in a l l sorts of l o c a l minima for (1.1.3). S t a b i l i t y problems with the d i f f e r e n t i a l equation can also a r i s e . In some cases the solution to the i n i t i a l value problem (1.1.1) i s a discontinuous function of p and thi s can create d i f f i c u l t i e s . Also, many dynamic models are attempts to describe phenomena operating on d i f f e r e n t time scales and thus s t i f f i n i t i a l value problems can be expected to arise in practice. Parameter f i t t i n g in the predator-prey dynamic model y/ =p, Y, - P a y , vA / U+p^y, ) - P , y,a S ' (1.1.7) y^-p^+p^y, y a/(i+p sy, > is a t y p i c a l example of the type of problem considered in thi s thesis. For more d e t a i l s concerning this dynamic system, we refer the reader to Bazykin[7] and to Chapter 4 of thi s thesis. As shown in Chapter 4 (which i s devoted exclusively to this model) a poor i n i t i a l approximation to the optimal parameters CHAPTER 1 7 can often lead to a l o c a l minimum in parameter space at which the solution to the above i n i t i a l value problem i s q u a l i t a t i v e l y quite d i f f e r e n t from the observations. Several methods are presented in Chapter 3 which are designed to overcome poor i n i t i a l parameter estimates. Special methods are also developed in Chapter 3 for the important (and often d i f f i c u l t ) case when observations are not available on a l l state variables. We also consider methods designed to overcome i n s t a b i l i t i e s in the i n i t i a l value problem by the use of continuation parameters and break points. In Chapter 4 we present extensive experiments with many of the techniques developed in Chapter 3 applied to the above dynamic model. In Chapter 5 we present a promising technique for improving poor parameter estimates using a sequential reestimation approach. Experiments with this technique applied to problems involving the above dynamic model are also given in Chapter 5. Experience indicates that the successful resolution of a "real world" parameter f i t t i n g problem involving a dynamic model usually requires many optimization runs. Strategies such as freezing or rescaling parameters are also often useful. Frequently the model evolves as attempts are made to f i t i t to the data. It i s thus desirable to rapidly acquire experience with a given model. We conclude that an int e r a c t i v e approach can be valuable for resolving, in a reasonable time, a parameter f i t t i n g problem involving a dynamic model. This i s inherently expensive; however, as computer technology advances, the cost CHAPTER 1 8 factor becomes less important. It i s our view that the f i r s t point to consider when designing a good interactive package i s the set of numerical algorithms to be employed. The user-program interface should then be constructed to make optimal use of these algorithms. Of course the process works the other way too: interactive algorithms should be designed with a user-program interface in mind. One goal of t h i s thesis i s to develop dynamic model parameter f i t t i n g algorithms that exploit user in t e r a c t i o n . Another goal i s to organize a selection of these algorithms into an interactive package so that the various approaches developed complement one another. The end res u l t of t h i s work i s the interactive package PARFIT documented in Appendix A. 1.2 MOTIVATION Dynamic models occur extensively in practice. To be meaningful, such models must be related to physical observations and this often involves adjusting some parameters in the models. In t h i s section we b r i e f l y describe some areas where parameter f i t t i n g in dynamic models i s important. Chemistry i s one f i e l d where parameter f i t t i n g in dynamic models occurs that i s frequently mentioned in the l i t e r a t u r e (see for example, Rosenbrock and Storey[61,p.189,p.204], Bard[6,p.222], van Domselaar and Hemker[71], and Bellman et a l [ 9 ] ) . T y p i c a l l y a dynamic model i s set up to describe a chemical reaction. The unknown parameters are reaction rates and the state variables represent concentrations of various CHAPTER 1 9 reactants. Since reaction rates can vary greatly, s t i f f dynamic models are important in chemistry. An example of such a model is considered in Section 3.2. The study of parameters in dynamic models with a large number of state variables i s also of interest to chemists (see for example Farrow and Edelson[22]). Many dynamic models occur in the f i e l d s of medicine and biology. Some of these models involve organic chemistry, while others are more d i r e c t l y related to b i o l o g i c a l processes. There are models describing enzyme a c t i v i t y in the blood (van Domselaar[70], van Domselaar and Hemker[71]). Models describing blood c e l l population dynamics are currently of interest (Mackey[42]). Parameter estimation in a model involving the e l e c t r i c a l a c t i v i t y of the heart has been studied by Bellman et a l [ l l ] . B i o l o g i c a l processes frequently operate on d i f f e r e n t time scales and thus s t i f f dynamic models are important here. At present, there i s a strong interest in dynamic models describing ecological processes. Several such models are considered in th i s thesis. Often ecological data i s sparse and has a large random error and thi s makes parameter f i t t i n g d i f f i c u l t . Models describing predator-prey interactions occur extensively in ecology. In Chapters 4 and 6, we give several examples with such models using both simulated observations and observations obtained from physical experiments. For more background concerning parameter f i t t i n g in dynamic models in ecology, see for example Swartz and Bremermann[66], Vandermeer[69], Martin et al[44], Parker[53], Long[40]. CHAPTER 1 10 1.3 MORE BACKGROUND Following the observations of Bard[6,p220], we elect to use methods of the form ptS +' J = p U > R 7 F ( p t / > ) (1.3.1) to minimize F(p) in (1.1.3) where VF represents the gradient of F, i s a scalar, and R^ i s a matrix (usually p o s i t i v e d e f i n i t e ) . The Gauss-Newton (see for example [50,p.267]) and Levenberg-Marguardt[39],[43] methods are part i c u l a r examples of such technigues. To apply these methods v F must be found. There are b a s i c a l l y two ways this can be done. One way i s to calculate v F through f i n i t e differences- Since the determination of F(p) involves the integration of an i n i t i a l value problem, we expect the use of f i n i t e differences to approximate VF to be an expensive undertaking. More inportant, however, i s the fact that with a dynamic model, the accurate ca l c u l a t i o n of VF by f i n i t e differences can be a t r i c k y task. In p a r t i c u l a r l e t 3F(p)/3p. be approximated by the difference F(p+Ap. e. )-F(p) " (1.3.2) A P -where e- i s the unit vector with a 1 in position i . As pointed out by Bard[6,p226], several factors a f f e c t how well t h i s difference approximates dF/dp^ . For a good approximation, A P ; must be small; however, i f i t i s too small, rounding-error dominates and a poor approximation to the derivative i s obtained. More important, however, i s the fact that F(p) i s CHAPTER 1 11 obtained by integrating an i n i t i a l value problem and thus i t s accuracy depends on the d i s c r e t i z a t i o n used and on the order of the method used to numerically solve the d i f f e r e n t i a l equation. Thus for example i f F(p+Ap^.e^.) and F(p) are determined independently (each with i t s own discrete set of time values), they must be found with s u f f i c i e n t accuracy so that the difference approximation to ~dF/dp; i s v a l i d . This i s an expensive undertaking and i s not the proper way to proceed. It is more productive to think of F in terms of the discrete method used to find i t . Thus at a given point p in parameter space, "VF(p) i s approximated by integrating (1.1.1) m+1 times using the same discrete set of time values. This avoids for example the p o s s i b i l i t y of getting d i s c o n t i n u i t i e s in F(p) due to varying sets of discrete steps at neighboring points in parameter space. Of course such a discontinuity would play havoc with the f i n i t e difference approximation to the gradient. Thus to calculate F(p) and approximate vF(p) requires m+1 integrations of (1.1.1), only one of which involves error co n t r o l . However, (1.1.1) i s often a nonlinear i n i t i a l value problem. The second alternative for c a l c u l a t i n g VF, which we s h a l l use, employs the s e n s i t i v i t y equations. These are a set of linear i n i t i a l value problems coupled in only one d i r e c t i o n to the given i n i t i a l value problem (1.1.1). In our notation they are r* 3 J J (1.3.3) y, (0)Oy(0 f P ) / a p . CHAPTER 1 12 for j=l,...,m where the subscript p^ denotes p a r t i a l d i f f e r e n t i a t i o n with respect to p., and where g i s the Jacobian matrix (ag/ay). These equations may be obtained by d i f f e r e n t i a t i n g (1.1.1) with respect to p. To use the s e n s i t i v i t y equations, g and g must be found; however, for a wide selection of important dynamic models this i s not too d i f f i c u l t a task. For more d e t a i l s on s e n s i t i v i t y equations see Tomovic[67], Tomovic and Vukobratovic[68]. In Chapter 2, further d e t a i l s are given on the integration of the s e n s i t i v i t y equations. We observe that for models of the form (1.1.1), the solution to the s e n s i t i v i t y equations immediately gives V F(p). Following the notation used in (1.1.2), denote by J the kr x m Jacobian matrix of f with respect to p. The elements of J are given by J„,„ * (P)=w 3Yj(«j (t.) (1.3.4) 4 ( 8 - 0 + 4 J ( j ^ A_ke.-iUA.-UA s=l,...,r; 1=1,...,k; j=l,...,m, and the gradient of F(p) is VF(p)=2J(p) Tf (p) . (1.3.5) We note that in the process of finding VF, we have found part of the Hessian matrix of F(p). (The Hessian matrix i s T kr 2(J J+XZ f-G.) 2=1 * * where G_ is the matrix of second p a r t i a l s of f with respect to o D P-) CHAPTER 1 13 Beale[8] distinguishes three basic problems in nonlinear parameter estimation. F i r s t there i s the problem of determining the optimal parameter vector p. This i s primarily a matter of numerical analysis and i t i s the problem we concentrate on. Second there i s the problem of defining a t h e o r e t i c a l l y s a t i s f a c t o r y confidence region or an approximate confidence region for p. Third, Beale i d e n t i f i e s the problem of describing this confidence region so that i t can be e a s i l y interpreted. Beale deals extensively in [8] with the second problem. Although we are primarily concerned with the f i r s t problem in t h i s thesis, some attention to the second and thi r d problems i s mandatory since a measure of the r e l i a b i l i t y of parameters greatly enhances their value to the model builder. Also, as seen in Chapter 4, a study of some of the s t a t i s t i c a l aspects of the problem can be useful for detecting linear relationships among parameters in a model. Following Bard[6,p.187], we make the DEFINITION 1.3.1: The Y - j o i n t confidence region i s a bounded closed subset S(W) in parameter space depending on the data sample W such that Pr [p*6S (W) ] = V (1.3.6) for a l l possible data samples W where p* i s the exact (and unattainable) value for the parameter vector, and Pr denotes p r o b a b i l i t y . S p e c i f i c a l l y , we use for S(W) an m dimensional e l l i p s o i d CHAPTER 1 14 centred at the estimate p" of p*. Following Bard[6,pl89], we approximate f(p) defined in (1.1.2) by a linear function in a neighborhood of p: f (p)atf (p)+J(p) (p-p) . (1.3.7) We assume the errors in the observations are normally dis t r i b u t e d with zero mean. Form the theory of multiple linear regression with V representing the kr x kr covariance matrix for the observations, the variable ^ = (p-p) T(J T(p)V- lJ(p) ) (p-p) (1.3.8) has a ~\ a d i s t r i b u t i o n with m degrees of freedom. (We are T -I considering the general objective function f V f.) Unless stated otherwise, J i s evaluated at p" in the following discussion. Also, J=f ( p ) T V _ , f (p) (1.3.9) has a 'X d i s t r i b u t i o n independent of <J with kr-m degrees of freedom. Thus (kr-m)& m J has an F m ^ ^ / l_ / > n d i s t r ibution ( m degrees of freedom in the numerator, kr-m degrees of freedom in the denominator). In par t i c u l a r when the covariance matrix V i s of the form o-*I, (that i s , when a l l observations are independent) then CHAPTER 1 15 _ T T (kr-m)J (kr-m)(p-p) J J(p-p) mj mf(p) Tf(p) - T T (kr-m)(p-p) J J(p-p) = (1.3.10) mF(p) Thus the"B-joint confidence region i s _ T T _ . (kr-m) (p-p) J J (p-p) -» — < 1 - 3 - 1 1 ' which i s an m dimensional e l l i p s o i d in parameter space. Following van Domselaar and Hemker[71], we can use the singular value decomposition of J (p") to extract further information on this e l l i p s o i d . Let J(P)=Q(P)X(P)R"r(P) (1.3.12) where Q and R are orthogonal matrices of size kr x m and m x m and X i s the mxm diagonal matrix [diag(s , . . . , s^)] of singular values arranged in descending order of magnitude. Let kr-m _ (1.3.13) &q=R T(p)(p-p). Our e l l i p s o i d may thus be written | S q : Sq " V Sq< 6^ • (1.3.14) Thus the p r i n c i p a l axes of the confidence region have lengths yi/s- , j=l,...,m. For our confidence intervals on p., j=l,...,m J we take the projections of the above e l l i p s o i d onto the CHAPTER 1 16 coordinate axes in p-space with the o r i g i n translated to the estimate p. Thus the confidence i n t e r v a l for p. i s _ j [p. - Vvf (J T (P)J(P))V; / P. +Vf(j"r(p)J(P)r' ]. (i.3.i5) This i s the confidence i n t e r v a l we use in our program PARFIT described in Appendix A. To find (JTT)~* we use (J Tjf'=R 2~V. (1.3.16) The expected value of p-p i s zero and in the special case considered above when V=«-al, the covariance matrix for p-p* is E( ( P - P * ) (p-p*)~^)=<>-a(JTJ)", (1.3.17) where E(.) denotes expectation (see for example van Domselaar and Hemker[71], Bard[6,p.59]). This matrix can be found using (1.3.16). The matrix of corr e l a t i o n c o e f f i c i e n t s has elements ( J T J ) : ' . - J (1.3.18) *. T -I where fSA- = (J J)^. . Thus using the singular value decomposition, y0.j=cps(S/. , SJ ) (1.3.19) where S. , S. are the i 1 th and j 1 th row vectors of S=R2"! F i n a l l y we note that the above confidence intervals were derived under the assumption that f(p) could be well approximated by a linear function near p. This i s often not the case. Bard[6,pl91] gives a simple empirical way of checking t h i s . We want the l i n e a r i t y approximation to hold over the confidence region that has been found. That i s F(p) should be CHAPTER 1 17 near F(p)+.5(p-p) (J TJ) (p-p) in the confidence region. This can e a s i l y be checked at the boundary of the confidence region. We also note that when F(p) is large (and f i s only moderately nonlinear), then J T J i s a poor approximation to the Hessian matrix of F(p) and the quadratic approximation to F(p) stated above cannot be very good. Thus the confidence region stated above loses v a l i d i t y as the residuals increase. This can also be seen by observing that £ in (1.3.13) varies d i r e c t l y as F(p), and thus as F(p) increases, the confidence intervals can quickly become larger than the parameters themselves. For further comments on thi s case see Rosenbrock and Storey[61,p202]. We end this chapter with a b r i e f discussion on conditioning. There i s a close connection between nonlinear and linear least squares problems. For example, the Gauss-Newton i t e r a t i v e method for solving a nonlinear least squares problem can be viewed in terms of a sequence of linear least squares problems. That i s given an estimate p W for the parameter vector which minimizes (1.1.3), we seek a new estimate p ^ + , ) for t h i s minimizing parameter vector such that | |f ( p W ) + J ( p W ) I |* (1.3.20) is minimized where _p<«> . (1.3.21) CHAPTER 1 18 (In practice a robust least squares optimization procedure must do more. For example, i t should guarantee a reduction in the sum of the squares of the residuals after each iteration.) For s i m p l i c i t y of notation in the following discussion, we neglect superscripts and we l e t -$+'^=x. Also, we assume J i s of f u l l rank.; The condition number of the matrix J i s defined to be T((J) = I I J| l a l |J*I I a where J*= (J TJ)"' J T is the pseudo-inverse of J. F i r s t we consider the e f f e c t of small errors in f. The vector *x which minimizes ||f-Jx||* (1.3.22) is given by x =J f. When J i s of f u l l rank x i s unique. Let f be an approximation to f and l e t f and f be projections of f and "f* onto the range space of J. Provided f^40, i t follows (see for example Stewartf65,p.221]) M j V j ^ t l l , \(J) | I f . - f l I L * < (1.3.23) I U T f | l A H f r l l a Thus when J i s i l l - c o n d i t i o n e d (^f(J) i s large) or when the projection of f onto the range space of J i s small, r e l a t i v e l y small errors in f can have a strong influence on the accuracy with which we can determine ~x . The effects of errors in J are much more complicated. The following theorem (see for example Stewart[65,p.223]) addresses thi s s i t u a t i o n . THEOREM 1.3.1: Let J be of f u l l rank and l e t f be defined as above. Let E CHAPTER 1 19 be a matrix of the same dimensions as J and l e t E^, fp and E^, be projections of E and f onto the range space of J and the orthogonal complement of the range space of J respectively. (The projection of a matrix onto a subspace i s defined to be the matrix whose i 1 t h column i s the projection of the i 1 t h column of the given matrix onto the subspace.) If I U 1"! I A I I E , | lA<.5 then J+E i s of f u l l rank and i i x - i r i L IIEJL j i E j i A i i f j i <2 'X 5 + 4 ^ — 1 C.J^+s^—L-J- (1.3.24) l l x l l , . i u i i A I IJI 1^1 I f ^ l IA " J u t where x=(J+E)Tf and x =JTf. As noted by Stewart[65,p.224], i f f i s almost in the range space of J then "X i s the condition number of the least squares problem, while i f I I f ^ 1 \gL/1 I f^ I 1^ i s large then 7f i s the e f f e c t i v e condition number of the least squares problem. Thus depending on the orientation of f with respect to the range space of J, the least squares problem can be extremely sensitive to the condition number of J. The condition number of J can be influenced by certain transformations in parameter space. At times, such transformations can d r a s t i c a l l y reduce the condition number; however, on other occasions they can worsen the conditioning of a problem. For example consider the logarithmic transformation of Pj where we transform to "p. =ln (py ) . We consider the e f f e c t of t h i s transformation on the conditioning of the s t i f f problem CHAPTER 1 20 y'=-(1-y )y +p y ' ' (1.3.25) y^=p, ( d - y x ) y , - ( P * + P s ) y a > with i n i t i a l condition y(0) = ( l , 0 ) T at the point (1000, .99, .01) T in parameter space. Without any scaling, the condition number was approximately 59600. The matrix J was evaluated using the observation times given in Section 3.2 where this i n i t i a l value problem i s given further consideration. The trapezoidal integration scheme available in the package PARFIT described in the appendices was used to approximate J. For this problem, a logarithmic scaling of the parameters reduced the condition number to approximately 4.4. There are cases, however, where thi s scaling increases the condition number (this occurs for example, in certain cases with exponential f i t t i n g problems), and thus i t must be used with caution. CHAPTER 1 21 CHAPTER 2 OPTIMIZATION AND INTEGRATION 2.1 NONLINEAR LEAST SQUARES Our goal i s to concentrate on the d i f f e r e n t i a l equation aspect of parameter f i t t i n g in dynamic models and not to compare the fine points of various optimization algorithms. However, the nonlinear least squares problem occurs repeatedly in th i s thesis and thus a b r i e f discussion of this problem i s in order. When choosing a method, i t i s important to keep in mind that the e f f i c i e n c y and r e l i a b i l i t y of an optimization program depend on both the p a r t i c u l a r algorithm and the d e t a i l s of i t s implementation. We must choose between methods designed s p e c i f i c a l l y for least squares problems, and more general methods designed for nonlinear optimization. For our par t i c u l a r problem the cal c u l a t i o n of f i r s t derivatives i s often expensive (through the s e n s i t i v i t y equations for example) and the calc u l a t i o n of second derivatives i s even worse. Thus we elect to use only f i r s t d erivative methods and we must s e t t l e for approximations to the Hessian matrix. Some comparisons between least squares methods and more general optimization methods applied to least squares problems (Bard[5], Bus et al[14]) indicate special least squares methods are to be preferred. We recognize, however, that there are cases when a more general method can be superior (see Ramsin and Wedin[57], McKeown[46]). The results of Bus et al favor the CHAPTER 2 22 Levenberg-Marquardt method ( [ 3 9 ] , [ 4 3 ] ) over more general optimization methods and over the Gauss-Newton method, (we include under the name Gauss-Newton, modified Gauss-Newton algorithms employing step length adjustment) while Bard finds his implementation of the Levenberg-Marquardt method to be as r e l i a b l e but s l i g h t l y less e f f i c i e n t than his implementation of the Gauss-Newton method. Bard, however, employs some special techniques in his implementation of the Gauss-Newton algorithm that are absent in the experiments of Bus et a l . This further points out the s e n s i t i v i t y of test results to pa r t i c u l a r d e t a i l s of implementation. Of course a l l results are also problem dependent. The Levenberg-Marquardt method appears to be a good algorithm for the nonlinear least squares problem and we employ i t extensively in this thesis. S p e c i f i c a l l y we iterate according to p f r " ; =p(f> - ( j T ( p ( ^ )j(p(P , + X i ) - ' j T ( P ^ ) f ( P V ) (2.1.1) where J, f, p were defined in Section 1.1 and X i s a posit i v e parameter chosen so that the sum of the squares of the residuals i s reduced by the above i t e r a t i o n . -r In the case when J J i s pos i t i v e d e f i n i t e (that i s , when J is of f u l l rank), we have the following important facts about th i s technique (see Marquardt[43]). (1) Let ^ = p < * " > _p(*> _ T h a f c . s ( J T J + X l ) S = - J T f (2.1.2) CHAPTER 2 23 where we have d r o p p e d t h e s u p e r s c r i p t s . I t f o l l o w s t h a t $ =- ( J T J + X l ) " l J T f (2.1.3) m i n i m i z e s o v e r MJi + f l l * (2.1.4) : MSI 1^=1 IS.II^. (2.1.5) (2) For S> s a t i s i f y i n g (2.1.2) , | | % (X ) | £ i s a c o n t i n u o u s monotone d e c r e a s i n g f u n c t i o n o f X and l i m | IS (A) I |*=0. (2.1.6) (3) The q u a n t i t y ^ = c o s - ' ( £ . S ) (2.1.7) where %^=-JTf. i s a c o n t i n u o u s monotone d e c r e a s i n g f u n c t i o n o f X and l i m * = 0 . (2.1.8) Thus as X-*-« , t h e d e s c e n t d i r e c t i o n S g i v e n by (2.1.2) -r a p p r o a c h e s t h e s t e e p e s t d e s c e n t d i r e c t i o n , g i v e n by - J f , and i t s m a g n i t u d e a p p r o a c h e s z e r o . T h i s c an c r e a t e p r o b l e m s i f t h i s a l g o r i t h m i s n o t c a r e f u l l y i m p l e m e n t e d . T h a t i s , we do n o t want X t o become so l a r g e t h a t we a r e t a k i n g v e r y s m a l l s t e p s i n a d i r e c t i o n t h a t i s e s s e n t i a l l y t h e s t e e p e s t d e s c e n t d i r e c t i o n . CHAPTER 2 24 To overcome th i s possible drawback, a check i s made to see i f % is within 45° of the steepest descent d i r e c t i o n . If i t i s , A i s not increased but instead a search in the d i r e c t i o n of the current £ i s carried out to obtain a better parameter vector. Marquardt[43] outlines the necessity of using this strategy. The following theorem (see Dennis[19]) gives conditions for the l o c a l convergence of the Levenberg-Marquardt method. THEOREM 2.1.1: l e t "p be a l o c a l minimum of F(p) and l e t ^ / be the smallest eigenvalue of J ( p ) T J ( p ) . Let 5( be a scalar such that for a l l p in a neighborhood of p~, I | (J(p)-J(p) ) T f (P) | | <*||p-p|L (2.1.9) a — «*• If Y<yi/then for any bounded sequence A ( ^ ^ of real numbers, there exists a neighborhood of p such that i f p i s in this neighborhood, the Levenberg-Marquardt i t e r a t i o n s defined by and \ A ^ j converge to p. For further reading concerning the convergence of the Levenberg-Marquardt algorithm see Osborne[51],[52]. There i s a l o t of choice available in a par t i c u l a r implementation of the Levenberg-Marquardt method. Much of thi s centres around the strategy for adjusting X- For a summary of some of the strategies we refer the reader to Van Loan[72]. Marquardt[43] suggests that we rescale parameter space at each i t e r a t i o n in such a way that J J in the scaled parameters has diagonal elements equal to one. The rationale for this CHAPTER 2 25 scaling i s that the Levenberg-Marquardt method i s biasing the descent d i r e c t i o n towards the steepest descent d i r e c t i o n and the steepest descent d i r e c t i o n i s scale dependent. Our implementation of the Levenberg-Marquardt technique has th i s scaling available as an option. Experimental results indicate that at times this scaling can be quite e f f e c t i v e . A l i s t i n g of our implementation i s given in Appendix B. Next, we mention a couple of recent developments in nonlinear least squares problems which, when they become more f u l l y understood, may be very useful for our problem where function and gradient evaluations are expensive. Steen and Byrne[64] propose an interesting nonlinear least squares algorithm which adjusts the descent d i r e c t i o n between the steepest descent and Gauss-Newton directions in a complicated, but (experimentally) apparantly more e f f i c i e n t way than that of the Levenberg-Marquardt method. Also, their algorithm does not suffer from the stepsize going to zero as the steepest descent d i r e c t i o n i s approached. This algorithm i s of pa r t i c u l a r interest to us because i t requires a subs t a n t i a l l y fewer number of function evaluations than does the Levenberg-Marquardt method on a f a i r l y wide range of test problems considered in [64]. On most of the test problems considered, this method i s superior to the similar SPIRAL method proposed by Jones[34]. The second recent development of p a r t i c u l a r interest involves work on the large residual least squares problem. In CHAPTER 2 26 th i s case J J i s no longer a good approximation to the Hessian matrix, and the Gauss-Newton and Levenberg-Marquardt methods are slowed down. This s i t u a t i o n can arise when we have a l o t of noise in the observations, when we have a poor model, or when our least squares algorithm i s converging to a l o c a l minimum with a large re s i d u a l . (As the experiments of Chapter 4 indicate, numerous l o c a l minima can occur when we try to estimate parameters in dynamic models.) Recently, there has been some interesting work done on special algorithms which approximate the second derivatives in the Hessian of a least squares problem by techniques modelled after the quasi-Newton methods. (See for example Dennis[18,p.171,177], Dennis et al[20].) When the r e l i a b i l i t y of such methods increases, their application to parameter f i t t i n g problems in dynamic models seems worthwhile. If a reduction i s achieved in the number of numerical integrations required, the extra work invested to approximate the Hessian matrices should be well worth i t . For a good discussion of recent work in nonlinear least squares, we refer the reader to Dennis[19]. 2.2 INTEGRATION OF MODEL AND SENSITIVITY EQUATIONS In the course of f i t t i n g parameters in a dynamic model, several d i f f e r e n t i n i t i a l value problems must be solved. That i s , every time the parameter vector changes, a new i n i t i a l value problem must be solved. Moreover, the solutions to these problems can vary d r a s t i c a l l y . Consider, for example, the algae growth model of Section 6.3 where small changes in the parameter CHAPTER 2 27 values can produce huge changes in the solution to the i n i t i a l value problem. To handle such problems, a parameter f i t t i n g algorithm requires a good general purpose numerical package for solving i n i t i a l value problems. The a b i l i t y to handle parameter estimation problems involving s t i f f dynamic models i s also important in practi c e . For several examples with such problems we refer the reader to van Domselaar and Hemker[71]. In th i s thesis, we employ the automatic integration package developed by Gear[25], [26], [27]. In addition to the standard predictor corrector methods, th i s package has available a set of s t i f f l y stable multistep methods of varying order. In our package, PARFIT, the user can e a s i l y switch between a regular multistep method and a s t i f f l y stable method. When f i t t i n g parameters in a dynamic model, the need for a general integration program i s clear; however, often a much simpler integration procedure i s adequate. Of course to mimimize the error introduced into the parameter estimates by the d i s c r e t i z a t i o n of the i n i t i a l value problem, an integration scheme with error control should be used. In the interests of economy, i t i s advisable to sta r t with a simple integration scheme not employing stepsize control i f this i s possible. In par t i c u l a r we make use of the trapezoidal method without error co n t r o l . In PARFIT, the discrete times used by the trapezoidal method are the "sample times" which by default are the observation times; however, the sample times can be i n t e r a c t i v e l y modified. Use of the trapezoidal method allows us CHAPTER 2 28 to handle s t i f f problems. F i n a l l y we note that other integration schemes can e a s i l y be added to PARFIT. In addition to integrating the given i n i t i a l value problem, the s e n s i t i v i t y equations defined in Section 1.3 must also frequently be integrated. The way the s e n s i t i v i t y equations (which are coupled in only one d i r e c t i o n to the given i n i t i a l value problem) are integrated can be c r u c i a l to the success of a parameter f i t t i n g algorithm. For example, as observed by Bard[6,p.231], the integration of the s e n s i t i v i t y equations at a point p in parameter space should not have any influence on the discrete steps used in the integration of (1.1.1) at p. That i s , F(p) should be independent of whether or not gradient information i s extracted at p. The s e n s i t i v i t y equations in theory provide a means of determining how the continuous solution y(t) to the given i n i t i a l value problem varies as a function of the parameter vector p. However, i n practice the given i n i t i a l value problem can only be solved approximately according to some discrete model (for example the trapezoidal method). For the purposes of numerical optimization, i t i s the solution to thi s discrete analog of the continuous problem that i s being f i t to the observations, and when finding the gradients of the objective function we r e a l l y want to know how the solution to the discrete approximation to the i n i t i a l value problem varies as a function of p at the observation times. Thus for the purposes of numerical optimization, the way the s e n s i t i v i t y equations are integrated should be related to the CHAPTER 2 29 solution method of the o r i g i n a l i n i t i a l value problem. This i s e s p e c i a l l y important i f a coarse approximation to the given i n i t i a l value problem i s used. There i s an analog to the above considerations in optimal control problems where the discrete approximation to the adjoint equations should be t a i l o r e d to the pa r t i c u l a r d i s c r e t i z a t i o n of the state equations. This can be a tedious undertaking in control problems as demonstrated by Kelly and Denham[35]. However, the main d i f f i c u l t y with control problems arises because the state equations are integrated forward in time and the adjoint equations are integrated backward in time. Fortunately for our purposes since we are integrating in only one d i r e c t i o n in time, the r e l a t i o n of the discrete s e n s i t i v i t y equations to the discrete state equations i s much simpler. In pa r t i c u l a r a l l we must do i s to ensure that the same method with the same discrete set of time values i s used on the s e n s i t i v i t y equations as i s used on the state equations. To be more s p e c i f i c , the diagram in Figure 2.2.1 must commute where S c represents the operator which produces s e n s i t i v i t y equations from an i n i t i a l value problem, D4- represents a d i s c r e t i z a t i o n operator, and S Q represents the operator which produces discrete s e n s i t i v i t y equations from discrete state equations. In our numerical integration procedures we take advantage of the one way coupling between the s e n s i t i v i t y equations and the state equations. S p e c i f i c a l l y , we integrate the state equations from time t to time t+h (under error control i f applicable). Then CHAPTER 2 30 continuous state eqns D. discrete state eqns continuous s e n s i t i v i t y eqns D. discrete s e n s i t i v i t y eqns Figure 2.2.1 Di s c r e t i z a t i o n of s e n s i t i v i t y equations assuming there are m parameters and n state equations, we integrate each of the m s e n s i t i v i t y i n i t i a l value problems, each having n equations, from time t to time t+h using exactly the same d i s c r e t i z a t i o n that was used to integrate the state equations. This i s consistent with the comments by Bard[6,p.231] mentioned e a r l i e r in th i s section. We note that the s e n s i t i v i t y equations are linear i n i t i a l value problems. When Gear's program i s used, the above technique involves extracting information on the current stepsize and order from the integration package, and then integrating the s e n s i t i v i t y equations over the same step with the same order method. Gear's program employs the Nordsieck[49] formulation of a multistep method where approximations to higher derivatives are stored CHAPTER 2 31 instead of previous values. This makes changing the stepsize very easy. Our integration of the s e n s i t i v i t y equations employs exactly the same technique in phase with Gear's package. For more d e t a i l s see Appendix B. No error control i s used when integrating the s e n s i t i v i t y equations. As observed by van Domselaar and Hemker[71], an integration scheme has the same s t a b i l i t y properties on the s e n s i t i v i t y systems as i t has on the system of state equations. 2.3 WHEN TO INTEGRATE THE SENSITIVITY EQUATIONS We must decide when to integrate the s e n s i t i v i t y equations. Two strategies are considered. These extra linear equations may be integrated every time (1.1.1) i s integrated (strategy one), or they may be integrated only when the Jacobian matrix and the gradient are required (strategy two). We expect the choice of strategy to depend in part on the implementation d e t a i l s of the optimization algorithm employed. For example, i f several objective function evaluations are required at each i t e r a t i o n , then the f i r s t strategy would c l e a r l y be i n e f f i c i e n t . On the other hand, i f only one objective function evaluation i s required per i t e r a t i o n (this i s generally not the case, although at times several i t e r a t i o n s follow this pattern), then strategy one would hold the advantage. To further complicate matters, the integration of the s e n s i t i v i t y equations i s generally easier than the integration of (1.1.1) which i s usually nonlinear and at times even s t i f f . Also, the step size adjustments are made when integrating (1.1.1) and not when integrating the CHAPTER 2 32 s e n s i t i v i t y equations. To aid in a more detailed analysis of the two alte r n a t i v e s , we make the following d e f i n i t i o n s . (a) Let W denote the basic unit of work for a pa r t i c u l a r problem defined as the work required to integrate (1.1.1). (b) Let w denote the fr a c t i o n of the work W required to integrate one s e n s i t i v i t y equation. (c) Let m be the number of s e n s i t i v i t y equation systems. This is just the number of parameters. (d) Let I be the number of it e r a t i o n s in the optimization run. (e) Let J be the number of objective function evaluations where no gradient i s required. These are the evaluations in the searches during the i t e r a t i o n s . Under the f i r s t strategy, the number of times the i n i t i a l value problem (1.1.1) i s solved i s I+J-(1-1)=J+1 (2.3.1) and t h i s i s also the number of times the s e n s i t i v i t y equations are solved. Therefore the t o t a l function evaluation work done in solving for the optimal parameters i s W, =(J+l)W+m(J+l)wW. (2.3.2) Under the second strategy, the number of times the i n i t i a l value problem i s solved i s I+J, and the number of times the s e n s i t i v i t y equations are solved i s I. Therefore the t o t a l function evaluation work done in solving for the optimal parameters i s CHAPTER 2 33 Wa= (I+J)W+mIwW. (3.3.3) Thus W /W =(J+1)(mw+1)/(I+J+mlw) ' *• (2.3.4) =(J/I+1/I)(mw+l)/(l+J/I+mw). We neglect 1/1 compared to J / I . This i s equivalent to ignoring the s t a r t i n g function evaluation in the f i r s t strategy and thus in most cases, t h i s approximation i s not too s i g n i f i c a n t . Thus Wy /W XJ/I (mw+1)/(J/I+(mw+1) ) (2.3.5) Table 2.3.1 contains some values for W,/Wa as a function of J/I and mw. Entries less than 1 correspond to cases where i t i s v mw J/K .5 1 2 4 8 i 0 .6 0 .667 0 .75 0 .833 0 .9 1.2 0 .667 0 .75 0 .857 0 .968 1 .06 1.5 0 .75 0 .857 1 .0 1 .15 1 .29 2 0 .857 1 .0 1 .2 1 .43 1 .64 Table 2.3.1 Work rati o s W /W„ — \i-—a. more e f f i c i e n t to integrate the s e n s i t i v i t y equations each time the i n i t i a l value problem i s integrated. To carry t h i s analysis further, we need some t y p i c a l values for w. Consider Euler's method with no stepsize adjustment. Assume there are n state variables and l e t the work required to evaluate one component of a vector function or one element of a Jacobian matrix be Wc. The i n i t i a l value problem (1.1.1) i s CHAPTER 2 34 y'=g(t,y,p) and the s e n s i t i v i t y equations are of the form y.' =g u(t,y,p)y. +g_ ,j=i,...,m. We do not make any special allowances for parameters that occur only in the i n i t i a l conditions (thus making g =0) in the following analysis. Also, additions and mul t i p l i c a t i o n s associated with taking an Euler step are neglected. The work required to advance one time step in the solution of (1.1.1) i s nWc, and the work required to advance one step in the solution of one s e n s i t i v i t y equation system i s (na/m+n)Wc. Thus mw=(na+mn)/n=n+m. (2.3.6) In the case of Euler's method without error control, i t seems advisable to solve the s e n s i t i v i t y equations only when required. The same conclusion applies to e x p l i c i t multistep methods without error control since to advance one step in time requires only one evaluation of the vector function on the right hand side of (1.1.1) (see for example Gear[25,p.104]). The s i t u a t i o n i s , however, d i f f e r e n t with i m p l i c i t multistep methods. Here a generally nonlinear system of algebraic equations must be solved for each advance of one time step when integrating (1.1.1). If a predictor-corrector method i s used to solve the these nonlinear systems of equations then t y p i c a l l y two or three corrections are required at each time step (Gear[25,p.114]). Assume the predictor - corrector method CHAPTER 2 35 ends with an evaluation of y' at the new time value. To be concrete, assume four evaluations of g are required per time step. To solve the i m p l i c i t equations associated with the integration of the s e n s i t i v i t y equations, no extra function evaluations beyond those required for an e x p l i c i t multistep method are needed. However, m linear systems of equations each involving the same nxn matrix must be solved. This takes on the order of n /3+mn mul t i p l i c a t i o n s (see for example Stewart[65,p.136]). The number of mul t i p l i c a t i o n s required to evaluate a component of g can vary greatly between problems. To describe t h i s v a r i a t i o n , l e t Wc require 1 m u l t i p l i c a t i o n s . We w i l l vary 1 in the following analysis. The work required to advance one time step , without error control, in the solution of (1.1.1) i s 4nWc=4nl mul t i p l i c a t i o n s (2.3.7) and the number of mu l t i p l i c a t i o n s required to advance one step in the solution of a s e n s i t i v i t y equation i s (na/m+n) l+n 3/(3m)+2n i. (2.3.8) The m u l t i p l i c a t i o n s required to form g y have been included in the above estimate. Thus mw=((n+m)l+na/3+2nm)/(41). (2.3.9) Some values for mw and for W^W^ are given in Table 2.3.2. The ordered pair (m,n) i s given below the mw values. From Table 2.3.2, we see there are cases when i t i s advantageous to CHAPTER 2 36 1 = 5 1 = 25 \mw 1.92 3.92 3.27 6.17 1.38 2.38 2.25 3.43 J / I \ (3,2) (3,5) (6,2) (6,5) (3,2) (3,5) (6,2) (6,5) 1.0 0.745 0.831 0.810 0.878 0.704 0.772 0.765 0.816 1.2 0.850 0.965 0.937 1.03 0.798 0.886 0.877 0.944 1.5 0.991 1.15 1.11 1.24 0.921 1.04 1.03 1.12 2.0 1.19 1.42 1.36 1.56 1.09 1.26 1.24 1.38 Table 2.3.2 Some work r a t i o s f o r a p r e d i c t o r - c o r r e c t o r method i n t e g r a t e the s e n s i t i v i t y equations every time the n o n l i n e a r i n i t i a l value problem i s i n t e g r a t e d . T h i s s t r a t e g y would be even more d e s i r a b l e i f an e r r o r c o n t r o l were used on the n o n l i n e a r i n i t i a l value problem. The case when the i n i t i a l value problem i s s t i f f r e q u i r e s s p e c i a l a t t e n t i o n . In t h i s case i t i s d e s i r a b l e f o r s t a b i l i t y reasons to use an i m p l i c i t i n t e g r a t i o n scheme; however, a p r e d i c t o r - c o r r e c t o r method can r e q u i r e very small step s i z e s to converge and thus a Newton-like method i s i n d i c a t e d (Gear[25,p.216]). T h i s r e q u i r e s i n f o r m a t i o n on g ; however, the exact g i s not r e q u i r e d and the usual s t r a t e g y i s to update g^ only when necessary (see Gear[25,p.217]). To be co n c r e t e , we assume oL e v a l u a t i o n s of g are r e q u i r e d per time step i n the s o l u t i o n of the giv e n i n i t i a l value problem ( u s u a l l y << <1.). There i s a f u r t h e r c o m p l i c a t i o n i n f i n d i n g work estimates i n the s t i f f case. I f the s e n s i t i v i t y equations are to be i n t e g r a t e d then we must f i n d g and the f a c t o r s of the a s s o c i a t e d l i n e a r J CHAPTER 2 37 system matrix at each time step. Thus this information i s f r e e l y available when we are integrating the given dynamic system. In the following estimates we assume no advantage i s taken of th i s free information. We have mw=( (n+m) l+na/3+2nm) / (4 (n+1) +<* (nl+n A/3) ) . (2.3.9) In Tables 2.3.3 and 2.3.4, results analogous to those in Table 2.3.2 are given for the cases <<=.3 and <<=.6 respectively. We conclude that for i m p l i c i t methods, i t can be advantageous to follow strategy one, e s p e c i a l l y when the problem is s t i f f . Of course the optimal strategy i s strongly dependent on the optimization method used and for strategy one to be best, exact searches at each i t e r a t i o n should not be made. F i n a l l y we note that from the results on ten test problems considered in [5], J/I^1.2 for the Levenberg-Marquardt method. CHAPTER 2 38 1 = 5 1 = 25 \ 1.22 1.57 2.08 2.47 1.12 1.49 1.83 2.15 J / I \ (3,2) (3,5) (6,2) (6,5) (3,2) (3,5) (6,2) (6,5) 1.0 0.690 0.720 0.755 0.776 0.680 0.713 0.739 0.759 1.2 0.779 0.818 0.864 0.891 0.766 0.810 0.842 0.869 1.5 0.895 0.947 1.01 1.05 0.879 0.936 0.980 1.02 2.0 . 1.05 1.12 1.21 1.27 1.03 1.11 1.17 1.22 Table 2.3.3 Work ratios for a s t i f f method (a(=.3) 1 = 5 1.31 (3,5) 1.88 (6,2) 2.06 (6,5) 1 = 25 0.997 1.19 1.62 1.72 (3,2) (3,5) (6,2) (6,5) 1.0 1.2 1.5 2.0 0.678 0.697 0.764 0.789 0.875 0.909 1.02 1.07 0.742 0.753 0.847 0.862 0.986 1.01 1.18 1.21 0.666 0.687 0.750 0.775 0.857 0.891 0.999 1.05 0.724 0.731 0.823 0.832 0.954 0.966 1.13 1.15 Table 2.3.4 Work rati o s for a s t i f f method (<<=.6) 2.4 AN EXAMPLE WITH KNOWN DIAGONAL COVARIANCE MATRIX Consider the i n i t i a l value problem Y ; = - P , Y ^ + P ^ y » - P / Y , Y * + p * y 3 - p * y * Y s + P * Y 5 - - P < Y ^ Y * y j =P, y, y a - P Z Y 3 - P , Y ^ - ^ Y ^ (2.4. I ) y ^ y * y « - p 5 - V P * y a y * with the i n i t i a l condition CHAPTER 2 39 y(0) = ( l , 1, 0, 0, 0 ) T . (2.4.2) This problem i s considered by Bard[5]. Observations used by Bard on a l l f i v e state variables are given in Table 2.4.1. The Time Observations 10y3 lOOOy^ 10 00y^. 12. 5 .945757 .961201 .494861 154976 .111485 25 .926486 .928762 .690492 314501 .236263 37. 5 .917668 .915966 .751806 „ 709300 .311747 50 .928987 .917542 .771559 1 .19224 .333096 62. 5 .927782 .920075 .780903 1 .68815 .340324 75 .925304 .912330 .790539 2 .19539 .356787 87. 5 .925083 .917684 .783933 2 .74211 .358283 100 .917277 .907529 .779259 3 .20025 .361969 Table 2.4.1 Observations problem we consider, designated problem 3dl by Bard[5], assumes a known diagonal covariance matrix for the error in the observations on the state variables of V=diag(25E-6, 25E-6, 25E-8, 25E-10, 25E-12). (2.4.3) The objective function i s .5Tr (V_/M(p) ) (2.4.4) where M(p) i s defined in'Section 1.1. This i s just a maximum lik e l i h o o d estimate with the observation times known exactly. Since V i s a diagonal matrix, t h i s problem can be handled by our weighted least squares formulation. For convenience we ignore the .5 and minimize Tr (v'M(p)). Now CHAPTER 2 40 V =diag(4E4, 4E4, 4E6, 4E8, 4E10) and thus we weight the residuals associated with state variables 1, 2, 3, 4, and 5 by 200, 200, 2000, 20000, and 200000 respectively. Bard's starting approximation for p was (.01, .01, .001, .001, .02, .001)T Our implementation of the Levenberg-Marquardt method using Gear's predictor-corrector implementation to integrate the i n i t i a l value problem gave the results in Table 2.4.2. Our Component of p Our estimate Estimate in [5] 1 .6358233E-2 .6358106E-2 2 .6774440E-1 .6774396E-1 3 .5920433E-4 .5916273E-4 4 .4943161E-3 .4943798E-3 5 .1018610 .1018756 6 .4204069E-3 .4202537E-3 Table 2.4.2 Optimization results minimum for the expression in (2.4.4) was 21.38429 and Bard's minimum was 21.37944. Considering the complexity of the programs and the fact that we used Hermite interpolation to get integration results at the observation times, these results seem to be in good agreement. We note that no constraints were required to get the above optimum. Bard uses penalty functions to impose the constraint p->0, j=l,...,6 on this problem. They appear unnecessary for us; however, Bard[5,pl85] does require CHAPTER 2 41 the imposition of constraints to successfully resolve some parameter f i t t i n g problems involving the above dynamic system. CHAPTER 2 42 CHAPTER 3 SPECIAL METHODS FOR THE INTERACTIVE APPROACH 3.1 THE INTERACTIVE APPROACH Interactive techniques provide a powerful tool for nonlinear parameter estimation in general and they are es p e c i a l l y valuable for d i f f i c u l t problems such as those involving dynamic models. Indeed, even the resolution of simple nonlinear parameter estimation problems often requires several runs to adjust such things as termination c r i t e r i a , and star t i n g approximations to the parameters. With dynamic models, the problems of i n s t a b i l i t i e s , o v e r s t a b i l i t i e s , and at times numerous l o c a l minima in the optimization problem can make parameter estimation a tedious task. An interactive approach using algorithms designed with user intervention in mind can reduce many of the d i f f i c u l t i e s associated with parameter estimation in dynamic models. However, for the optimum use of any parameter estimation package there i s no substitute for a good understanding of the model under consideration. The design of a good interactive package i s an involved task. For a detailed introduction to interactive applications in numerical analysis see Smith[63]. The work by Aaro[l],[2] on a software system for interactive computing seems to hold promise for producing good transportable interactive packages with good user interfaces. We l i s t below some of the major considerations involved in producing a good interactive parameter estimation CHAPTER 3 43 package. (1) Special algorithms that exploit user judgement and intervention should play a large role in the interactive package. The development of such algorithms for the f i t t i n g of parameters in dynamic models i s one goal of thi s thesis. (2) An extensive set of commands should be available; however, a ty p i c a l user should not be forced to learn a detailed command language in order to use the program. One way to att a i n t h i s goal i s for the inte r a c t i v e program to display l i s t s of options (such as output options for example) and to prompt the user for the necessary d e t a i l s to complete a command. There i s , however, a tradeoff here and for certain highly r e p e t i t i v e commands (such as those involved with stepping through a nonlinear optimization intera c t i v e l y ) prompting should be kept to a minimum. F i n a l l y , the program should be r e l a t i v e l y "user proof". That i s , internal checking should be done so that regardless of what the user enters, the program should not end with a terminal error. Making a package user proof i s large l y a matter of detailed programming, and in the interests of e f f i c i e n t program development, i t should be l e f t u n t i l near the end. Another goal of t h i s thesis i s to develop a set of simple commands that are useful for interactive parameter f i t t i n g in dynamic models. (3) Careful consideration must be given to what information the interactive program displays. Graphical information seems CHAPTER 3 44 to be the most useful. In the case of parameter f i t t i n g in dynamic models, integration r e s u l t s , observations, and in the two state variable case, phase plane plots are obvious candidates for graphical display. A decision must be made on how the user should describe the parameter f i t t i n g problem to the interactive package. There are b a s i c a l l y two choices. F i r s t , the dynamic model can be defined through a user written subroutine, which i s separately compiled and then loaded with the interactive package. This subroutine can also contain other necessary a n a l y t i c a l information (such as Jacobian matrices) required by the integration and parameter f i t t i n g algorithms. A l t e r n a t i v e l y , the model can be entered and modified i n t e r a c t i v e l y (and of course saved on f i l e for later use so that i t need not be re-entered each time an interactive session begins). This i s a more v e r s a t i l e approach, but i t requires extensive programming. Its main advantage i s that the user can i n t e r a c t i v e l y modify as well as f i t a model. Starting with a simple model and gradually working up to a more complex model i s one way of getting starting approximations to parameters (see Bard[6,p.123]). However, the consequences of adding a new term to a dynamic model can be dramatic and interactive modification of dynamic models demands a l o t from the user. In the interests of e f f i c i e n c y , i nteractive model entry and modification require that any p a r t i a l derivatives needed by the numerical CHAPTER 3 45 algorithms should be found symbolically. This i s fe a s i b l e , but i t adds to the complexity of the ove r a l l program. The second option described above i s a good long range goal; however, i t i s e s s e n t i a l l y a matter of programming and not numerical analysis and i t should wait u n t i l the numerical aspects of the interactive package have been settled upon. This second option can be added later to a working package using the f i r s t option. (5) A good interactive package should be well structured so that new commands can e a s i l y be added, and so that existing commands can be e a s i l y modified and extended. Also attention should be paid to the d e s i r a b i l i t y of eventually producing a transportable program. Transportability i s es p e c i a l l y sensitive to the way the package uses graphics software. Thus i t is desirable to is o l a t e the interface between the interactive program and pa r t i c u l a r graphics procedures. Of course, the interactive program should only use generally available graphics operations. It i s our view that the f i r s t goal (to develop good interactive algorithms for the parameter f i t t i n g problem) i s the most important in that decisions made here influence the d e t a i l s concerning the way the other goals are attained, and even more importantly, the algorithms employed play a major role in determining how effective, the ov e r a l l package i s . Of course good algorithms can be degraded i f the user-machine interface i s neglected. CHAPTER 3 46 An e f f i c i e n t way to proceed seems to be to develop an experimental interactive package concentrating on the f i r s t goal, but also paying strong attention to the second t h i r d and f i f t h goals. This i s the strategy we have employed in developing our interactive package PARFIT described in d e t a i l in Appendix A. In this chapter, we consider several special techniques for f i t t i n g parameters in d i f f e r e n t i a l equations. Our goal i s the development of techniques which lend themselves well to an int e r a c t i v e approach, and which are less sensitive to the i n i t i a l parameter guess than the d i r e c t approach using the s e n s i t i v i t y equations. However, there i s a tradeoff and coarse but well behaved methods should only be expected to give approximate values to the optimal parameters and should not for example be expected to distinguish between neighboring l o c a l minima in the f u l l least squares problem. The f i r s t special method we consider i s the derivative f i t t i n g approach. This i s one of the most straightforward of the coarser methods. For experimental results with another implementation of this technique, we refer the reader to Swartz and Bremermann[66]. 3.2 DERIVATIVE FITTING (DFIT) Assume observations are given d i r e c t l y on a set of state variables in the dynamic model under consideration. Thus, t h e o r e t i c a l l y , the desired derivatives of these state variables can be approximated as follows: (For the moment, we assume observations are available on a l l n components of the state CHAPTER 3 47 vector y(t).) Each component y^-vt) of y(t) i s approximated by a function s^ . (t) f i t t i n g the data at the observation points which i s at least continuously d i f f e r e n t i a b l e . This can i d e a l l y be done i n t e r a c t i v e l y . The problem of finding p to minimize F(p) in (1.3.1) can now be approximated by the problem of finding p to minimize I Is'(t)-g(t,s(t),p) I |* (3.2.1) T where s(t) = (s ; (t) ,...,s„ (t)) . Since s(t) generally approximates noisy data, a careful determination of p to minimize the above expression in the L a sense cannot be j u s t i f i e d . Thus for computational purposes, we minimize the semi-norm on c' [t0 ,t^] defined by ( S I I Is' ( t t ) - g ( t / ,s{tx ) ,p) | |* (3.2.2) where £t^ : l<l<k^ i s the set of observation times introduced in Section 1.1. Our parameter f i t t i n g problem has thus become a problem in nonlinear functional approximation which i s much cheaper and usually much easier than the o r i g i n a l problem. Indeed, i f g i s linear in p, a l l we have i s an ordinary linear least squares problem. The above observations make the derivative f i t t i n g approach very a t t r a c t i v e ; however, as noted by Bard[6p.128], i t has some severe • flaws. These occur in part because i t requires approximations to y(t) and y'(t) and good approximations to these functions are often hard to obtain. This i s the case for CHAPTER 3 48 example when the observations have large separations in time or when the data i s noisy. Bard further notes that an error analysis for the parameter estimates i s d i f f i c u l t with this approach. This l a t t e r drawback i s not too important from our point of view since we propose this method only as an intermediate technique and for the f i n a l determination of parameters and confidence i n t e r v a l s , we recommend the use of the s e n s i t i v i t y equations. A more severe drawback of this method i s that with a poor approximation to y or y 1 , parameters may be produced at which the solution to the i n i t i a l value problem deviates greatly from the observations or even blows up. This can be remedied at times by the technique of the next section where there i s no need for an approximation to y ' ( t ) . The errors in the approximations to y and y' are not the only factors a f f e c t i n g the r e l i a b i l i t y of this technique. Other c h a r a c t e r i s t i c s of the least squares problem of minimizing (3.2.2) must also be considered. In p a r t i c u l a r the results concerning conditioning mentioned at the end of Chapter 1 are important. For example, in view of (1.3.23), i t i s possible for r e l a t i v e l y small errors in our approximation to y' to have a large influence on the parameters estimated by the DFIT method. This i s e s p e c i a l l y important to keep in mind since the d i f f e r e n t i a t i o n of data tends to be error prone. The integral f i t t i n g method of the next section avoids this dependence on approximated derivatives. To implement the derivative f i t t i n g approach, a technigue CHAPTER 3 49 f o r using the o b s e r v a t i o n s to approximate y ( t ) and y ' ( t ) i s r e q u i r e d . I f there are f a i r l y l a r g e e r r o r s i n the data, i t i s not reasonable to use a d i f f e r e n c i n g technique to approximate y', and some smoothing method i s c a l l e d f o r . In our package PARFIT, we use l e a s t squares piecewise c u b i c s p l i n e approximations and l e a s t squares piecewise c u b i c Hermite approximations to the da t a . Approximations are made i n d i v i d u a l l y on each observed s t a t e v a r i a b l e and the p o s i t i o n i n g of the j o i n t s f o r each piecewise polynomial i s l e f t to the user. T h i s should i d e a l l y be done i n t e r a c t i v e l y . Cubic s p l i n e approximations are adequate f o r many cases; however, they run i n t o d i f f i c u l t i e s when sharp bends occur i n the f u n c t i o n being approximated. T h i s causes problems, f o r example, when the dynamic model under c o n s i d e r a t i o n i s s t i f f . Piecewise c u b i c Hermite approximations which are C ' as opposed to the C 3 c u b i c s p l i n e approximations do not have as gre a t a problem with sharp bends and are thus more s u i t e d f o r handl i n g the sharp turns that occur i n s t i f f problems. A couple of l i m i t a t i o n s with the d e r i v a t i v e f i t t i n g technique immediately come to mind. In the problem formulated i n S e c t i o n 1.1, i t was p o s s i b l e to have parameters a r i s i n g o n l y through the i n i t i a l c o n d i t i o n s . A d e r i v a t i v e f i t t i n g a l g o r i t h m cannot g i v e us any i n f o r m a t i o n on these parameters. Another d i f f i c u l t y a r i s e s with d e r i v a t i v e f i t t i n g when o b s e r v a t i o n s are not a v a i l a b l e on a l l the s t a t e v a r i a b l e s , but there i s a way around t h i s problem i n some of the cases where no parameters CHAPTER 3 50 occur exclusively in the subset of state equations corresponding to unobserved state variables. S p e c i f i c a l l y , we do a derivative f i t on the subset of state equations corresponding to observed state variables. At each least squares function evaluation at a point in parameter space during the i t e r a t i v e solution of this reduced derivative f i t t i n g problem, we integrate the subset of state equations corresponding to unobserved state variables. Thus, up to date information i s always available on the unobserved state variables in the reduced derivative f i t t i n g problem. However, th i s method has severe l i m i t a t i o n s . The obvious l i m i t a t i o n i s that i t only applies to a r e s t r i c t e d set of problems. Another l i m i t a t i o n involves s t a b i l i t y problems which can arise when a subset of a system of d i f f e r e n t i a l equations i s integrated. Nevertheless, t h i s technique has experimentally proven successful in some cases and for th i s reason we mention i t here. Next we give an example of the kind of s t a b i l i t y problem that can occur. Consider the problem y'=Gy (3.2.6) where y i s of length 2 and "-30 -22' 28 20 G = (3.2.7) The eigenvalues of G are -8 and -2 and the solution to (3.2.6) decays exponentially; however, i f we f i x y =sf (t) and integrate CHAPTER 3 51 y;J = 2 8 s / ( t ) + 2 0 y 4 , (3.2.8) depending on s ; ( t ) of course, the s o l u t i o n can grow e x p o n e n t i a l l y . The s t a b i l i t y problems that can a r i s e with the above m o d i f i e d d e r i v a t i v e f i t t i n g technique a l s o have an impact on the d e s i g n of s t a b l e i t e r a t e d i n t e g r a l and d e r i v a t i v e f i t t i n g a l g o r i t h m s i n S e c t i o n 3.4. To i l l u s t r a t e the d e r i v a t i v e f i t t i n g technique, c o n s i d e r the f o l l o w i n g problem i n v o l v i n g a set of coupled chemical r e a c t i o n s (see van Domselaar and Hemker[71]). The s t a t e equations are y>=_(l_y )y +p y ' * * (3.2.9) y'=p ( ( 1 - y ) y - ( p +p )y ) a. I A. ' A 3 3 and the i n i t i a l c o n d i t i o n i s y(0) = ( l , 0 ) r . (3.2.10) Th i s r e p r e s e n t s a model of a chemical r e a c t i o n and a l l parameters should remain p o s i t i v e . Observations were generated by i n t e g r a t i n g the above i n i t i a l value problem at (1000, 0.99, 0.01)T A l l simulated o b s e r v a t i o n s i n t h i s t h e s i s were found by i n t e g r a t i o n under e r r o r c o n t r o l . In the i n t e r e s t s of economy, a l l parameter f i t t i n g i n t e g r a t i o n s were done without s t e p s i z e adjustment unless otherwise i n d i c a t e d . For t h i s s t i f f problem, we used o b s e r v a t i o n s on both s t a t e v a r i a b l e s at the same o b s e r v a t i o n times that were used i n [71] . These o b s e r v a t i o n s (to four f i g u r e s ) are l i s t e d i n Table 3.2.1. The o b s e r v a t i o n s CHAPTER 3 52 Time y a Time y, 0 . 0 0 0 2 0 .9998 0. 1 6 4 8 0.06 0 .9991 0. 4 9 9 8 0 . 0 0 0 4 0 .9997 0. 2 7 5 3 0.08 0 .9 9 8 9 0. 4 9 9 7 0 . 0 0 0 6 0 .9 9 9 6 0. 3 4 9 3 0.1 0 . 9 9 8 9 0. 4 9 9 7 0 . 0 0 0 8 0 .9 9 9 6 0. 3 9 9 0 1.0 0 . 9 9 4 5 0. 4 9 8 6 0.001 0 .9996 0. 4 3 2 2 2.0 0 . 9 8 9 5 0. 4 9 7 4 0 . 0 0 1 2 0 .9 9 9 5 0. 4 5 4 5 5.0 0 .9747 0. 4 9 3 6 0 . 0 0 1 4 0 . 9 9 9 5 0. 4 6 9 5 1 0 . 0 0 .9502 0. 4 8 7 2 0 . 0 0 1 6 0 .9 9 9 5 0. 4 7 9 5 1 5 . 0 0 .9260 0. 4 8 0 8 0 . 0 0 1 8 0 .9 9 9 5 0. 4 8 6 2 2 0 . 0 0 .9021 0. 4 7 4 3 0.002 0 .9 9 9 5 0. 4 9 0 7 2 5 . 0 0 .8 7 8 6 0. 4 6 7 7 0.02 0 .99 9 3 0. 4 9 9 8 3 0 . 0 0 .85 5 3 0. 4 6 1 0 0.04 0 .99 9 3 0. 4 9 9 8 Table 3.2.1 Observations for s t i f f problem on y were approximated with a least squares piecewise cubic Hermite polynomial with one j o i n t at t=10, and the observations on y were approximated with a least squares piecewise cubic Hermite polynomial with j o i n t s at t=.0007, .0014, .0016, 15. Our star t i n g guess at the parameters was p('J =(50, 5, . 5 ) r . The DFIT method found the point (969.6, 1.113, -.1080) T in parameter space. Unfortunately, p^ i s negative. This suggests we try a square root transformation of p3 and estimate 'p^ = fp~3. With th i s transformation, the DFIT method found the point (unsealed) (969.6, 1.005, 9.33E-7)7" CHAPTER 3 53 in parameter space. We note that the DFIT method was quite capable of finding a good approximation to p ;. This should be compared with the results in the next section where the IFIT method was applied, to thi s problem. The d i r e c t method using the s e n s i t i v i t y equations also succeeded when i t was started from p given above; however, i t was slow to begin modifying p ; upward from 50. Before we give our next set of test problems, some conventions must be established concerning the presentation of graphical information. In p a r t i c u l a r , we use the following conventions: Observation points on y Observation points on y A Integration results on y Integration results on y^ Smoothing of data for y Smoothing of data for y a Guessed behavior for y Guessed behavior for y X a. Phase plane t r a j e c t o r i e s are s o l i d l i n e s . The above conventions are s u f f i c i e n t for most of our plot s . Other conventions are introduced as they become necessary. We use the method developed by McConalogue[45] to produce smooth curves for our p l o t s . In t h i s section and Section 3.3, we make some comparisons between derivative f i t t i n g and integral f i t t i n g on a problem CHAPTER 3 54 involving a change in equilibrium. The ef f e c t of noisy data on these two parameter estimation techniques i s investigated for the test problem, y'=p yi+pay,a+pl y, -y. ' ' ' 51 1 * ' * ( 3 . 2 . 1 1 ) y' =p y with the i n i t i a l condition y ( 0 ) = ( 1 . 5 , 1.0 ) T ( 3 . 2 . 1 2 ) Our f i r s t set of observations was obtained by integrating the above problem at the parameter vector (-.1, - 1 , 2 . 4 , .1)T These generated observations, at times .5 ( . 5 ) 2 0 , are shown in Figure 3 . 2 . 1 . For c l a r i t y we present graphical results only on y . State variable y does not go through any rapid jumps. Observations are used on both state variables. A piecewise cubic Hermite approximation to the observations on y using j o i n t s at t=2, 4, 6, 7, 8, 9, 1 0 , 1 2 , 1 4 , 18 i s also shown in Figure 3 . 2 . 1 . The j o i n t s for the corresponding approximation to the observations on y were at t=3, 8, 1 5 . Using these approximations to the observations, the DFIT method produced the parameters ( - . 0 7 7 4 0 , - . 7 9 9 5 , 1 . 7 2 5 , .09948)T No starting approximation to the parameters was required since the DFIT method involved solving a linear least squares problem CHAPTER 3 55 I UJ (— cr t— mg oi -t 0 . 0 0 4.00 8.00 TIME I J2.00 ~1 1B.0D Figure 3.2.1 Eguilibrium change—no error in observations 20.00 in this case. Integration results for y ; at the above parameters are also shown in Figure 3.2.1. A l l integrations of (3.2.11) were done using stepsize adjustment. These results should be compared with those in Figure 3.3.1 where results obtained with the IFIT method applied to this problem are shown. Next a normally distributed random error with mean 0 and standard deviation <r =1 was introduced into the above observations. The resulting observations on y and • their piecewise cubic Hermite approximation using j o i n t s at t=6,7,9,14 are shown in Figure 3.2.2. Joints at t=6,12 were used for the smoothing of the observations on y . Using this smoothing, the DFIT method produced the parameters T (-.03942, -.4047, 1.168, .1007). CHAPTER 3 56 "1 : 1 1 1 —' ) O.CD 4.00 8.0(1 12.00 16.00 20J10 TIME Figure 3.2.2 Equilibrium change—error in observations (tr=l) Integration results for y at these parameters are also shown in Figure 3.2.2. These results should be compared with those in Figure 3.3.2 where results with the IFIT method are presented for this problem. Our next experiment involved an increase in the noise; however, the change in equilibrium i s s t i l l v i s u a l l y discernable. A normally di s t r i b u t e d random error with mean 0 and standard deviation o-=2 was introduced into the observations. The resulting observations along with a piecewise cubic Hermite smoothing function are shown in Figure 3.2.3. The jo i n t s for the smoothing functions were the same as for the case<r=l. With this smoothing , the DFIT method produced the parameters (-.02113, -.2394, .6449, .1023)T CHAPTER 3 57 o s_| , , , , , O.DD 4.00 8.00 J2.00 16.00 20.00 TINE F i g u r e 3.2.3 E q u i l i b r i u m c h a n g e — e r r o r i n o b s e r v a t i o n s (<r=2) I n t e g r a t i o n r e s u l t s f o r y at these parameters are shown i n Fi g u r e 3 . 2 . 3 . By comparison with F i g u r e 3 . 3 . 3 , we see that the IFIT method produced much b e t t e r r e s u l t s i n t h i s case. 3.3 INTEGRAL FITTING (IFIT) The d e r i v a t i v e f i t t i n g approach works w e l l i n many cases; however, i t has the drawback that i t r e q u i r e s the numerical d i f f e r e n t i a t i o n of (at times) n o i s y data. Thus i t i s reasonable to t r y to f i t i n t e g r a l s i n s t e a d of d e r i v a t i v e s . For some a d d i t i o n a l background to i n t e g r a l f i t t i n g we r e f e r the reader to B a r d [ 6 , p . 2 1 9 ] . As i n the p r e v i o u s s e c t i o n , we assume an approximation to the d e s i r e d s o l u t i o n to our i n i t i a l value problem i s a v a i l a b l e i n the n-vector s (t) = (s, ( t ) , . . . , s ^ ( t ) ) T which i s obtained by some smoothing technique. The i n t e g r a l CHAPTER 3 58 f i t t i n g problem i s to find the vector p to minimize f r ( p ) f ( p ) where f*ci-OW = ( Y * W - ( P > + j g , ( t , s ( t ) f P ) d t ) - s . (t^) (3.3.1) where 1=1,...,k; i=l,...,n; y (p) i s the i 1 t h component of the vector of star t i n g values for the i n i t i a l value problem y'=g(t,y,p), and g^.(t,s,p) i s the i 1 th component of g(t,s,p). The Jacobian matrices for the linear least square problems that arise when p i s found i t e r a t i v e l y have elements J»i = 2 1 * ( 3 = 1 ' ' k n ; J = 1 ' ' m (3.3.2) which are found by evaluating the integrals ±y_oi(p)+ j 3g,(t,s,p)dt (3.3.3) for 1=1,...,k; i=l,...,n; j=l,...,m. Note that we are not solving our i n i t i a l value problem with these i n t e g r a l s . We are just integrating functions of time since s(t) i s known. Thus in terms of the number of evaluations of the function g, th i s method i s equivalent to the derivative f i t t i n g algorithm when a simple integration method such as the trapezoidal method using the times t Q , t i , . . . , t ^ i s employed. Furthermore, t h i s method has the same degree of l i n e a r i t y as does the derivative f i t t i n g method. In par t i c u l a r the above least squares problem for p i s linear in p when g and Y c(p) are CHAPTER 3 59 linear in p. Also, the integral f i t t i n g method can provide information on parameters which occur only in the i n i t i a l conditions, and thi s can be very useful. As demonstrated in Chapter 4, access to the i n i t i a l conditions can also be very important when no parameters occur in the i n i t i a l conditions. One might expect the IFIT problem to be better conditioned than the DFIT problem; however, i t i s possible for the integral f i t t i n g problem to be singular even when the derivative f i t t i n g problem i s well conditioned, but this does not appear to be a serious drawback in practi c e . To see how thi s s i n g u l a r i t y can ari s e , consider the case when there are 2 parameters, 1 state variable, and three observation times, t , t A , and t 3 with equal spacing h. Let the i n i t i a l condition (at t ) be independent of the parameters and l e t the derivative f i t t i n g Jacobian be 0 1 1 0 -1 0 Using the trapezoidal integration method, the integral f i t t i n g Jacobian i s the rank one 2x2 matrix f h / 2 h/2l |_h/2 h/2j formed by multiplying the derivative f i t t i n g Jacobian by fh/ 2 h/2 0 "I |_ h/2 h h/2 We note again that this was a s p e c i a l l y contrived si t u a t i o n and such a d i f f i c u l t y does not appear to arise in practice. CHAPTER 3 60 Our f i r s t experiment with the IFIT method i s on the s t i f f problem (3.2.9). Using the same smoothing and p v as were used in the previous section, the IFIT method found the point (607.3, .9901, .009281)7" in parameter space. The parameter p was not as well approximated as i t was with the DFIT method; however, no parameters have gone negative and good approximations have been obtained for p and p . Next we give some experiments with the i n i t i a l value problem (3.2.11) involving a change in equilibrium. Using the observations and smoothing of the previous section for the case <r=0 (no e r r o r ) , the IFIT method gave the parameters (-.1006, -.9913, 2.569, .1004)T Integration results for y at this parameter vector are shown in Figure 3.3.1. Using the observations and smoothing of the previous section for the case<?-=l, the IFIT method gave the parameter vector (-.00354, -.5883, 2.223, .09735)T Integration results for y^ at the above parameters are shown in Figure 3.3.2. These results should be compared with those in Figure 3.2.2. F i n a l l y using the observations and smoothing functions of the previous section for the casee>-=2, the IFIT method gave the parameters (-.03277, -.2655, 1.813, .09433)'. CHAPTER 3 61 •h + + H>-0.00 4.DO 8.CO TIME I 32.03 15. QO —I 20. OD Figure 3.3.1 Equilibrium change—no error in observations — i 16.00 20.00 0.00 4.00 8.00 12.00 TIME Figure 3.3.2 Equilibrium change—error {*'=!) CHAPTER 3 62 integration results for y at the above parameters are shown in Figure 3.3.3. We observe that there was a substantial a ••'•3 DO TIME Figure 3.3.3 Equilibrium change—error (<r = 2) improvement over the corresponding results for the DFIT method shown in Figure 3.2.3. 3.4 ITERATED INTEGRAL AND DERIVATIVE FITTING In t h i s section, the important special case when observations are not available on a l l state variables i s considered. This s i t u a t i o n was mentioned b r i e f l y in Section 3.2; however, the technique presented there was highly r e s t r i c t i v e in the class of problems i t could handle and i t was prone to i n s t a b i l i t i e s . At times the model builder knows approximately how unobserved state variables should behave to be consistent with CHAPTER 3 63 the observed state variables. Thus an i n t e l l i g e n t guess can be made and f i c t i t i o u s observations on the unobserved state variables can be created. With these created observations along with the physical observations, the DFIT and IFIT methods can be applied. Our interactive package PARFIT contains f a c i l i t i e s for setting up guessed observations on unobserved state variables in the important special case when only two state variables are present. PARFIT could e a s i l y be modified to handle cases where more than two state variables are present; however, as the number of unobserved state variables increases, the p r a c t i c a l i t y of t h i s method diminishes. The use of created observations can be a great help in determining the proper parameters, but, as one would expect, the success of this method depends on how well the proper behavior of the unobserved state variables can be anticipated. This i s again a good place for an interactive approach. The model builder can i n t e r a c t i v e l y modify the guessed observations, apply the DFIT or IFIT methods, observe the integration results at the parameters obtained ( i f integration i s possible), and then readjust the created observations. There are also automatic ways of improving the guessed observations and we concentrate on such methods for the remainder of this section. One stable way to i t e r a t i v e l y improve guessed observations and reestimate parameters i s with a nonlinear block Gauss-Seidel technigue. (For a discussion of nonlinear Gauss-Seidel techniques, see for example Ortega and Rheinboldt[50,p.224].) CHAPTER 3 64 For s i m p l i c i t y of notation, we look at the two state variable case with observations available on only one state variable. The extension to n state variables with observations available on r state variables (r<n) i s immediate. Note, however, that generally the parameter estimation problem becomes more d i f f i c u l t as the number of unobserved state variables increases. Indeed, removing observations on a state variable can change a well defined parameter estimation problem into a singular problem. An example of this i s given in the next chapter. Without loss of generality for our two state variable discussion, we assume that observations are missing on the second state variable. For the discrete set of time values t^, 1=1,,,,N l e t s =s(t e) (where we are using a superscript to avoid confusion with the case when s(t) i s a vector) be the smoothed approximation to the observations on state variable y at time t^ , and l e t s ^ ^ s ' t t ^ ) be the corresponding approximation to the derivative of the observations at time t j . Let Cj be the approximation to the unobserved state variable y at time tp -Usually, the points tj , 1=1,...,N are the observation times defined in Section 1.1. We seek p and c=(c 0,... ,cN)r to minimize N Jt -r a N , F = 2 T (g, (t , ( s \ c )',p)-s*') +21 (d,(c)) (3.4.1) 1=0 ' * * 1=1 * where d (c) represents a d i s c r e t i z a t i o n of the second state CHAPTER 3 65 equation. That i s we want the c to approximately s a t i s f y the second state equation at the minimum of the above expression. For example i f the trapezoidal method i s used to d i s c r e t i z e the second state equation then d (c)=.5(g (t ,(s*-' ,c )^p)+ga (t , (s* ,c )T,p) ) A A x * (3.4.2) c -c. JL_ SL-I Define r v"p F= (£F, . . . ,3F) *P, 5P~ T (3.4.3) ^F= (*F, . . . , 3F) . Thus the gradient of F i s VF=(9 fF T, v;F T) T. A necessary condition for (p r,cV=(p T,c rr to minimize F i s that V F ( ( p r , c ' V ) = 0 . (3.4.4) One way to solve (3.4.4) i s to use the nonlinear block Gauss-Seidel method starting with an i n i t i a l guess (p1"' T , c t 0 , T ) T for ( p . T / C T ) T . There are of course other ways to solve for — T — T T (p ,c ) . For example, we could minimize (3.4.1) by a standard nonlinear least squares technique such as the Levenberg-Marquardt method. To be e f f i c i e n t , however, such CHAPTER 3 66 methods should take advantage of the p a r t i c u l a r sparsity structure of the Jacobian matrix associated with (3.4.1). The use of a block Gauss-Seidel technique on the other hand requires only a standard nonlinear least squares technique such as the Levenberg-Marquardt procedure, along with a least squares technique designed for cases where the approximations to the Hessian matrices are banded matrices. A description of our algorithm for minimizing (3.4.1) and an outline of a l o c a l convergence proof for t h i s method follows. Assume a sta r t i n g approximation (p^ 0 ) r ,c1*' r ) T to (p T,c* T) T i s given. F i r s t hold c fixed at c and determine p (l* to minimize (3.4.1). This is just a standard nonlinear least squares problem and we solve i t using the Levenberg-Marquardt method. Next with p fixed at p determine c °^ to minimize (3.4.1). This i s a sparse nonlinear least squares problem and we solve i t by the Gauss-Newton method using the normal equations and a standard l i b r a r y procedure for solving banded posit i v e d e f i n i t e linear systems by a Cholesky decomposition. Strategy (a) in Bard[5,pl75] was employed for step length adjustment in our sparse Gauss-Newton procedure. In practice f a i r l y rapid convergence was obtained with this part of the algorithm; however, we expect this to vary depending on the nonlinearity in y^ of the given i n i t i a l value problem. The o v e r a l l algorithm proceeds by successively reestimating p ^ and c ^ where on each estimation, the most recent information on p and c i s employed. Usually only a few f u l l i t e r a t i o n s were required for the algorithm to s e t t l e down to a value for CHAPTER 3 67 (P f C ) . The above algorithm i s just a special case with w=l of a block nonlinear successive over relaxation process (see for example Ortega and Rheinboldt[50,p.325, p.332]). Thus a l o c a l convergence proof i s standard and we confine ourselves to just a b r i e f outline of convergence-Let x=(p r,c r) T, "x = (p r,c" r) T and assume x i s a l o c a l minimum of F- That i s *F(x)=0. Assume further that F i s twice continuously d i f f e r e n t i a b l e and that the Hessian matrix H of F i s posi t i v e d e f i n i t e in the open neighborhood S a of x . S p l i t the Hessian into H(x)=D(x)-L(x)-L T(x) where D(x) i s a block diagonal matrix, and L(x) i s a block lower triangular matrix and where the entries in L corresponding to the blocks in D are zero. Since H(x) i s symmetric and posi t i v e d e f i n i t e D(x) i s i s symmetric and positive d e f i n i t e . Also D-«L is nonsingular for any oo and in pa r t i c u l a r for 0<cu<2. Thus (see Varga [73 ,p. 77] ) , for 0<«K2 It follows (Ortega and Rheinboldt[50,p.326]) that there exists an open b a l l S centred at "x and contained in S0 such where p (.) denotes spectral radius. that there i s a unique sequence in S s a t i s i f y i n g our CHAPTER 3 68 n o n l i n e a r block Gauss-Seidel a l g o r i t h m (d>=l) and 1irn (x ) =x. There are other ways to estimate p i n c o n j u n c t i o n with i t e r a t i o n s on guessed o b s e r v a t i o n s . For example i n s t e a d of e s t i m a t i n g p to minimize (3.4.1) for a f i x e d c, we could estimate p using the DFIT or IFIT methods. Use of the DFIT method to estimate p i s e q u i v a l e n t to minimizing (3.4.1) with 6X ( c ) = g a . ( t J - ' ( S < , C * ) T 'P)~°jt where cJ approximates the time d e r i v a t i v e of c at . The d e t a i l e d form of d^ depends on how cJ i s approximated. A l t e r n a t i v e l y , we could r e p l a c e the DFIT p o r t i o n of the above i t e r a t i v e a l g o r i t h m by an a p p l i c a t i o n of the IFIT method. T h i s has been found to work w e l l . We comment that the r e s u l t i n g a l g o r i t h m i s not e q u i v a l e n t to minimizing F ( ( p 7 " , c r ) T ) i n (3.4.1) with the f i r s t summation r e p l a c e d by i t s i n t e g r a l f i t t i n g c o u n t e r p a r t : N Cfj r * XL ( y , ( 0 ) + g ( t , ( s ( t ) , c ( t ) ) , p ) d t - s ( t ) ) . 1=0 ' «i ' J I f the above term i s put i n p l a c e of the f i r s t sum i n (3.4.1) then the l e a s t squares problem for updating c l o s e s the s p a r s i t y s t r u c t u r e t h a t was present with d e r i v a t i v e f i t t i n g . Thus we do not c o n s i d e r f u l l i t e r a t e d i n t e g r a l f i t t i n g methods, but o n l y i t e r a t e d i n t e g r a l f i t t i n g methods where p i s updated using the CHAPTER 3 69 IFIT method and c i s updated using a sparse technique. There i s another way c can be updated besides using the f u n c t i o n (3.4.1) and e x p e r i m e n t a l l y t h i s method has proven e f f e c t i v e ; however, i t s u f f e r s from p o t e n t i a l i n s t a b i l i t i e s and must be used with c a u t i o n . T h i s method simply i n v o l v e s the g e n e r a t i o n of c at the new parameter vector p by i n t e g r a t i n g the second s t a t e equation h o l d i n g y f i x e d at s ( t ) . (We are s t i l l assuming y ^ i s unobserved.) As i n d i c a t e d i n the d e r i v a t i v e f i t t i n g s e c t i o n of t h i s chapter, t h i s subsystem can be very unstable at a p a r t i c u l a r p o i n t p i n parameter space even when the system y'=g(t,y,p) has no s t a b i l i t y problems. An example where t h i s method works very w e l l i s given i n the next chapter. F i n a l l y we observe that no d i f f e r e n t i a l equation s t a b i l i t y problems occur when we update c to minimize (3.4.1). A couple of examples using i n t e g r a l f i t t i n g and our sparse Gauss-Newton method to improve guessed o b s e r v a t i o n s f o l l o w . Consider the L o t k a - V o l t e r r a p r e d a t o r - p r e y model[41]. y'=p y -p y y / / / a. ' a y'=-p y +p y y. Here y^ r e p r e s e n t s the prey and y & r e p r e s e n t s the predator and P »P f P 3 fP^ are a H p o s i t i v e . T h i s model i s a s p e c i a l case of the model co n s i d e r e d by Bazykin[7] which we look at i n Chapter 4. For our f i r s t example, we generated o b s e r v a t i o n s by i n t e g r a t i n g the above system s t a r t i n g at y =12, y^=2 and using p=(.15, .03, .8, . 1 ) T . CHAPTER 3 70 The observation times were 1(1)20 and observations were made available only on y . The generated observations on y and integration results for y^ at the above simulation parameters are shown in Figure 3.4.1. The function s ( t ) , defined by the TIME IS.DO 20.00 Figure 3.4.1 Simulation results and spline approximation least squares cubic spline approximation to the observations using j o i n t s at t=2, 3, 5, 9, 16, is also shown in Figure 3.4.1. The i n i t i a l guess at the observations i s shown in Figure 3.4.2. Successive improvements in the guessed observations are also shown in Figure 3.4.2. Integration results at successive parameter estimates are shown for y in Figure 3.4.3. Observations on y are also shown in Figure 3.4.3. Observe that the integral f i t t i n g problem i s linear and thus an i n i t i a l guess at the parameter values was not required. The least squares CHAPTER 3 CHAPTER 3 72 problem for updating the guessed observations i s also linear in th i s example. Convergence was not too sensitive to the height of the i n i t i a l approximation to y , but i t was sensitive to the position of the peak. This, however, i s quite e a s i l y adjusted i n t e r a c t i v e l y . Thus an iterated approach to this problem reduces a p o t e n t i a l l y nasty nonlinear problem (especially i f there are no good guesses at the values of p^ ,...,p ). to a simple i n t e r a c t i v e procedure of adjusting one quantity (the position of the peak) over a well defined i n t e r v a l . Of course some i n t u i t i v e idea about the "proper" behavior of y^ i s required; however, we would expect t h i s information to often be more readi l y available than a good approximation to the optimal parameter vector. Our second example i s the same as the above example except for the new observation times ,5(.5)12.5. Also our i n i t i a l guess at the behavior of y^ was much less informed than i t was for the previous example. The j o i n t s for the smoothing spline were at t=1.25, 2.5, 5, 9. Figure 3.4.4 shows the successive improvements in the guessed observations and Figure 3.4.5 compares integration results with observations for y at the successive parameter estimates. We observe that the i t e r a t i o n s on the guessed observations did not attain the maximum that y does in Figure 3.4.1; however, th i s does not seem c r i t i c a l in view of the results in Figure 3.4.5. CHAPTER 3 73 CHAPTER 3 74 3.5 CONTINUATION AND QUASI MULTIPLE SHOOTING The methods presented i n t h i s s e c t i o n are designed to br i d g e the gap between the coarse but w e l l behaved technigues of the p r e v i o u s s e c t i o n s and the f u l l n o n l i n e a r l e a s t squares problem. These coarse methods are good to s t a r t out with i f a good i n i t i a l approximation to the optimal parameter vector i s u n a v a i l a b l e . However, the parameters produced by these methods can be inadequate. For example the s o l u t i o n to the giv e n i n i t i a l value problem may blow up at the parameters found by a coarse method. The problem of i n s t a b i l i t i e s at the s t a r t i n g parameter values i s a common d i f f i c u l t y encountered when f i t t i n g parameters i n i n i t i a l value problems. V a r i o u s s t r a t e g i e s have been suggested (see f o r example Bard[6,p.233]); however, there does not appear to be any p r e f e r r e d technique. The methods suggested i n t h i s s e c t i o n lend themselves w e l l to an i n t e r a c t i v e a t t a ck on the problem. F i r s t we consid e r a c o n t i n u a t i o n method between the i n t e g r a l f i t t i n g (IFIT) technique and the f u l l l e a s t squares problem. Assume, f o r now, that o b s e r v a t i o n s are a v a i l a b l e on a l l s t a t e v a r i a b l e s . These o b s e r v a t i o n s p r o v i d e us with approximations s - ( t ) , i = l , . . . , n to the d e s i r e d behavior of the s t a t e v a r i a b l e s y - ( t ) , i = l , . . . , n . Consider the problem of f i n d i n g p to f i t the s o l u t i o n of u'=g(t, (l-*)s(t)+jru,p) (3.5.1) u ( t 0 ) = y 0 ( p ) CHAPTER 3 75 to s(t) in the least squares sense at the observation times where CKy<l. When y=0, thi s i s just the integral f i t t i n g technique and when 2f =1 i t i s the f u l l least squares problem (on the smoothed data). The eigenvalues of g u determine the s t a b i l i t y of the above i n i t i a l value problem. Now for y= (l- ' j f) s+jfu. Thus for small* , i t should be possible to integrate (3.5.1) even when i t i s unstable for a =1. Put another way, as y increases, more and more of the " d i f f e r e n t i a l equation nature" of the problem i s taken into account. There are two basic ways a continuation problem can be approached. One way i s to treat i t as an i n i t i a l value problem in the continuation parameter. This i s the Davidenko approach (for a good summary of Davidenko's work see Rail[56]). The second way i s to treat i t as a sequence of nonlinear problems each associated with a larger value of . This i s the Lahaye approach (see Rheinboldt[58],[59], Rail[56], Ficken[23]). For our purposes the Davidenko approach appears to involve an excessive amount of c a l c u l a t i o n . The Lahaye approach on the other hand lends i t s e l f well to an interactive attack on the problem. That i s we st a r t withy=0 (IFIT) and then successively f i t the solution to (3.5.1) to s(t) with progressively larger values for y If y i s increased too much and the integration of (3.5.1) becomes impossible, then Y can be i n t e r a c t i v e l y reduced u n t i l (3.5.1) can be integrated. Our limited experience CHAPTER 3 76 seems to indicate that t h i s approach by i t s e l f i s not very e f f e c t i v e (for a f u l l evaluation more experiments on a wide selection of problems are required); however, combined with the use of break points as described later in this section, the continuation approach seems to be a viable way to escape from an unstable region in parameter space. One of the main drawbacks with such technigues i s of course the expense involved. This however i s becoming less important with the increasing a v a i l a b i l i t y of powerful computers. Experimental results indicate that the use of break points in a quasi-multiple shooting technique along with a continuation method can be an e f f e c t i v e combination for handling the s t a b i l i t y problem. S p e c i f i c a l l y consider break points at T7<Ta<...<T8 corresponding to observation times t. i Further l e t denote a continuation parameter vector for the break points. The vector «< i s of length n where n i s the number of state variables and 0<.<*- <1 > i=l,...,n. If break points alone are used (that is,2f=l) then observations need not be available on a l l state variables. The components of << corresponding to unobserved state variables should be equal to 1. Let u -(T <-) denote the solution to (3.5.1) arrived at by integrating up to break point T ; (which for i>l involves passing through i-1 previous break points). In the process of integrating (3.5.1), CHAPTER 3 77 we reset u at the break points according to u"t'(T; ) =Au~ (TV ) +(I-A) s (T; ) where A=[diag(^ ( , . . . , * ^ ) ] , and (V ) acts as the new i n i t i a l condition at T. . Of course the appropriate modification to the solution of the s e n s i t i v i t y equations must also be made at the break points. For purposes of least squares approximation, the value u~ (Tj ) is used at T,-. Thus we can also weight the residuals at the break points with weights w^ ,w^ ,...,w^ . To summarize we have at our command (a) a continuation parameter Y (b) a set of break points T ,...,T (c) a continuation parameter vector << for the break points (d) a set of weights w ,...,w_ for the break points. This gives the user a powerful set of options to play with and for their optimal use, an interactive approach i s indicated. Extensive experiments over a wide range of problems are required before a proper evaluation can be made of the interactive f a c i l i t i e s suggested above. For the purposes of this thesis; however, we l i m i t ourselves to a few examples in this section which indicate the potential power of the above f a c i l i t i e s in an interactive environment. For d e t a i l s on another approach employing break points to aid in the f i t t i n g of parameters in dynamic models, we refer the reader to van Domselaar and Hemker[71]. For more background on shooting methods in general see for example Roberts and Shipman[60]. CHAPTER 3 78 Consider again the Lotka-Volterra predator - prey model described in the previous section. We w i l l try to f i t t h i s model to the data shown in Figure 3.5.1. This data i s contrived in 2.CO 4.30 —I 1 c.oo a.oo TIME 10.00 12.00 Figure 3.5.1 Data and smoothing for continuation tests 14.00 and there i s no reason to expect a good f i t with this model. Piecewise cubic Hermite least squares approximations to the data were used to define s ( t ) . The j o i n t s for s ; ( t ) were at t=l, 3, 6 and the j o i n t for s (t) was at t=5. The components of s(t) are shown in Figure 3.5.1. The observations were generated so that the IFIT method produced parameters where the solution to the Lotka-Volterra equations with i n i t i a l conditions at T y=(12, 2) blew up. It is worthy of note that a f a i r amount of experimentation was required before such observations could be contrived. That i s for this problem, the IFIT method did not CHAPTER 3 79 seem l i k e l y to produce parameters corresponding to an i n s t a b i l i t y . The IFIT technigue (jf=0) produced the parameters (-.7273, -.06719, 1.525, .3295) T at which the given i n i t i a l value problem was unstable. Since the observations have no physical meaning, there i s no reason to require the paramerters to be p o s i t i v e . We are just looking for a set of parameters to minimize a sum of squares. Using break points at .6, 1.6, 4.5, 9.5 and using = . 2, c< f (0 , 0) and with weights of 1 at the break points, the following parameters were found: (.07608, .02078, 1.032, .2150)T The i n i t i a l value problem was stable here and integration results at these parameters along with the observations are shown in Figure 3.5.2. The Levenberg-Marquardt technique had no problem converging from the above parameters to the optimum (-.6820, -.05733, .8184, .1941)T Integration results at the above paramters are shown in Figure 3.5.3. The sum of the squares of the residuals was approximately 196. We comment that i t i s not necessarily advantageous to proceed with the continuation process once s t a b i l i t y has been attained. Parameters giving r i s e to an i n s t a b i l i t y may be produced and even with the use of break points, the careful CHAPTER 3 80 s i I UJ r— . CL X * >\ X X X X / v X X X. + + + + + • v. • X y 0.00 2.00 4.03 I 1 6.00 8.00 • TIME 10.00 12.00 F i g u r e 3.5.2 R e s u l t s using break p o i n t s with V=.2 — i 14.00 4.00 TIME 10.00 12.00 F i g u r e 3.5.3 R e s u l t s at optimal parameters 14.00 CHAPTER 3 81 increasing of continuation parameters can prove to be a tedious and expensive undertaking. Again interactive monitoring and control of the o v e r a l l process i s desirable and i f the process appears to get stuck a good strategy i s to return to a smaller continuation parameter and modify the break points. The use of the continuation parameter without break points did not appear productive on th i s problem. For example consider the sequence of H's in Table 3.5.1. The process worked f a i r l y well u n t i l V got near .75. It then became d i f f i c u l t to Continuation f i t re s u l t S t a b i l i t y P_*_ ?! \ 0 -.7273 -.06719 1.525 .3295 u .3 -.5877 -.04820 1.524 .3341 u .6 u n s t a b 1 e .5 u n s t a b 1 e .4 -.5766 -.04640 1.397 .3092 u .5 -.5622 -.04465 1.242 .2783 u .6 u n s t a b 1 e .55 -.5697 -.04491 1.184 .2667 u .6 -.5759 -.04541 1.130 .2559 u .7 u n s t a b 1 e .65 u n s t a b 1 e .63 -.5723 -.04492 1.081 .2463 u .65 -.5704 -.04462 1.057 .2402 u .68 -.5741 -.04497 1.021 .2343 u .7 -.5809 -.04576 1.010 .2320 u .73 u n s t a b 1 e .72 -.5843 -.04610 .9881 .2279 u .74 -.5936 -.04724 .9853 .2270 u .76 -.9258 -.08865 1.043 .2637 u very d i f f i c u l t to continue Table 3.5.1 A continuation experiment increase Tf and d i f f i c u l t to do the optimizations. The use of break points alone (with2f=l) was e f f e c t i v e on CHAPTER 3 82 thi s problem; however, i n s t a b i l i t i e s arose very e a s i l y i f too few break points were used. Here again we have a s i t u a t i o n where the interactive approach can provide a powerful t o o l . Starting at the parameters determined by the IFIT algorithm, with T f = l , «< = (0,0) an i n s t a b i l i t y arose in the integration of (3.5.1) when break points were used at 1.8, 4, 7.5. With break points at .6, 1, 1.8, 4, 4.5, 6, 7.5, 10 the integration of (3.5.1) blew up just after t=5. With break points at a l l the observation times, (3.5.1) could be integrated and the optimal parameters obtained were (-.4166, -.02940, .6279, .1280) T. The Volterra equations were stable at these parameters and star t i n g from th i s point in parameter space, the Levenberg-Marquardt method produced the optimum i l l u s t r a t e d in Figure 3.5.3. We conclude that for th i s problem the use of break points alone i s e f f e c t i v e , but t r i c k y , while the use of break points along with a continuation parameter in the d i f f e r e n t i a l equation can be very e f f e c t i v e . The techniques of this section demand a degree of judgement from the user; however, they provide a framework in which to tackle the i n s t a b i l i t y problem and in th i s sense they are to be CHAPTER 3 83 preferred over blind probing in parameter space. 3.6 IMPLEMENTATION OF AN INTERACTIVE PACKAGE The development of a good interactive package i s an evolutionary process. Our package, PARFIT, represents the f i r s t stage in such a process. We outline below some of the sp e c i f i c a t i o n s that should be kept in mind as the package evolves. We also describe with the aid of flow charts how the various f a c i l i t i e s of PARFIT can be used to complement one another. In view of the increasing power of mini-computers and the inherent expense of using a program such as PARFIT, the f e a s i b i l i t y of implementing a version of PARFIT on a mini-computer appears to be a worthy topic of study. The work of A a r o f l ] , [2] should provide a very valuable tool for developing such an implementation. One goal when developing an interactive package for parameter f i t t i n g in d i f f e r e n t i a l equations should be to make special parameter f i t t i n g algorithms such as those of this chapter e a s i l y accessible and complementary. The package should also allow for the testing and addition of new algorithms. It is advantageous to be able to e a s i l y switch between techniques in an interactive manner since the success of a par t i c u l a r technique i s often c l o s e l y related to the problem on which i t i s employed. To f a c i l i t a t e the easy switching from one algorithm to another the interactive package must be c a r e f u l l y designed. For example piecewise polynomial smoothing of data i s used by several algorithms. Thus there should be a single data CHAPTER 3 84 structure for smoothing functions and i t should be accessible (and perhaps modifiable) by a l l the procedures using smoothed data. This i s just common sense; the general philosophy should be to define data structures at the most general l e v e l p r acticable. The various algorithms then function in an environment established by these data structures. It i s useful for t h i s environment to contain much more than just the set of data structures used by the numerical procedures, and i t i s convenient to think of i t as a data structure i t s e l f containing information on how the package i s to communicate with the user, on various c o n t r o l l i n g parameters such as the integration error c r i t e r i a and integration method, and on many other aspects of the operation of the package. We l i s t below some of the items which can be thought of as being part of the environment. (1) Communication mode: This governs the amount of program guidance and descriptive information provided during an interactive session. The way information i s entered (for example using a keyboard, or a l i g h t pen) i s also indicated here. (2) Echo f l a g s : These indicate what information produced during an interactive session i s to be retained for later hard copy output. (3) Graphics display c o n t r o l : This indicates what information i s to be displayed on the graphics device. For example the most recently generated data CHAPTER 3 85 (such as smoothing or integration results) may be displayed as the default. The user should be able to modify the display to include for example only selected state variables. The default display should be i n t e l l i g e n t enough to display appropriate plots during an interactive session. For example when smoothing i s being done, the data and approximating function for the given state variable should be displayed. When iterated integral f i t t i n g i s being done, the display of successive i t e r a t i o n s on the guessed observations would be valuable. It should of course be possible to override the graphics display control and request that s p e c i f i c data be plotted. (4) Numerical data structures: These contain, among other things, smoothing information and integration r e s u l t s . (5) Numerical control parameters: These include integration error c r i t e r i a and stepsize constraints as well as parameters used in the optimization procedures. (6) Problem and algorithm selection data structures: These indicate what integration and optimization procedures are being used. The use of various scalings (such as the square root or logarithmic scalings available in PARFIT) i s indicated here and an indication of any frozen parameters i s given here. In more sophisticated packages, constraints on parameters can be indicated here and a pa r t i c u l a r choice from a selection of various objective functions to optimize (from s t a t i s t i c a l CHAPTER 3 86 considerations) can be indicated here. (7) Notations accumulated during an interactive session: As a user experiments with a par t i c u l a r problem, he should have the f a c i l i t i e s to make notes. Thus a l i s t of notes containing for example "promising parameter values" can be created. The environment thus becomes t a i l o r e d to a pa r t i c u l a r problem. Of course i t should be possible to save the notes from run to run. There are two fundamental modes in which an interactive program can operate. One mode employs extensive control by the interactive program and the other r e l i e s on the user to i n i t i a t e the appropriate seguence of actions to solve a problem. A good approach when developing an interactive package of the f i r s t type i s to f i r s t build a package of the second type. Strategies for the solution to the problems under consideration are then developed through extensive use of this package and these strategies can eventually be incorporated into a package employing extensive program control over the solution strategy. PARFIT i s a package designed for user i n i t i a t e d and conducted strategies. Thus i t s e f f e c t i v e use requires a detailed knowledge of a l l i t s f a c i l i t i e s and how these can be used to complement one another in an e f f e c t i v e manner. In this section we give a description by way of flow charts of how the various approaches to the parameter estimation problem can be used together in an integrated package. The strategies outlined in these flow charts form the skeleton for a version of PARFIT that CHAPTER 3 87 would attempt to guide the user to the various f a c i l i t i e s required for the solution to a problem. The implementation of such a program i s a n o n t r i v i a l endeavor, and to j u s t i f y i t , there should be a good demand for such a package. Indeed, the sophistication required of the model builder to develop good dynamic models argues against the need for detailed computer control of the interactive process. When the user i s required to pick an appropriate strategy without program guidance, the strategies outlined in the following flow charts should be kept in mind. CHAPTER 3 88 ARE ALL STATES OBSERVED?^ NO YES TRY GUESSED OBSERVATIONS AND ITERATED IFIT OR DFIT—5 YES TRY IFIT OR DFIT-4 OR SEQUENTIAL METHOD—7 NO TRY SEQUENTIAL-7 OR INTERACTIVE METHOD-3 Figure 3.6.1 Overall strategy CHAPTER 3 89 1 i DO NONLINEAR OPTIMIZATION (AUTOMATIC OR INTERACTIVE) RETHINK MODEL Figure 3.6.2 Refined parameter f i t t i n g CHAPTER 3 90 Figure 3.6.3 Interactive optimization CHAPTER 3 91 4 t SMOOTH OBSERVATIONS IF NOT DONE 1 ESTIMATE PARAMETERS USING LINEARITY IF PRESENT SUCCESS? YES NO \ TRY COARSE METHOD-2 OR INTERACTVE PROBING-•3 TRY NEW SMOOTHING— -4 CONT. WITH B.P. -6 INTERACTIVE -3 Figure 3.6.4 Derivative and integral f i t t i n g CHAPTER 3 92 i SMOOTH OBSERVATIONS IF NOT DONE GUESS UNOBSERVED STATE IF NOT DONE 11 i ESTIMATE PARAMETERS RESTART IF REQUIRED EXPLOIT LINEARITY 12 fESUL1] SATISFACTORY? YES 1 NO ITERATIVE [MPR0VEMENT2-YES ITERATIVELY IMPROVE GUESSED OBSERVATIONS DISPLAY RESULT. TRY NEW GUESSED OBS 11 NEW COARSE METHOD—2 INTERACTIVE 3 NEW SMOOTHING 5 Fi g u r e 3.6.5 Guessed o b s e r v a t i o n s and i t e r a t i v e improvement CHAPTER 3 93 6 1 SMOOTH OBSERVATIONS AND SET STARTING p IF NOT DONE. ! SET A SELECTION OF BREAK POINTS (B.P.) WEIGHTS ON B.P. B.P. CONT. PAR. I.V.P. CONT. PAR i — ESTIMATE PARAMETERS. 1 Figure 3.6.6 Continuation and quasi multiple shooting CHAPTER 3 94 7 4 GUESS STARTING PARAMETERS 1 Figure 3.6.7 Sequential reestimation (not implemented in PARFIT) CHAPTER 3 95 CHAPTER 4 PARAMETER FITTING IN A PREDATOR-PREY DYNAMIC MODEL 4.1 INTRODUCTION We consider in d e t a i l the problem of f i t t i n g parameters in a predator-prey dynamic model studied by Bazykin[7]. This model has several aspects which are at t r a c t i v e to ecologists modelling population dynamics. Depending on the parameter values, there are several possible phase plane configurations for the model, some of which contain l i m i t cycles. Certain l i m i t s to population growth are also inherent in the model and this i s phys i c a l l y appealing. Unfortunately, the varied behavior of the model which makes i t r i c h from an ecological point of view , complicates the parameter f i t t i n g problem. For example, with a poor guess at the optimal parameters, we may find ourselves with a phase plane configuration quite d i f f e r e n t from the one indicated by the observations. The a b i l i t y of some of the methods of Chapter 3 to handle this problem i s explored in thi s chapter. Only one parameter in the model considered by Bazykin occurs nonlinearly in the d i f f e r e n t i a l equations, and thus we expect some of the more global methods of Chapter 3 to be well suited to thi s model. 4.2 A PREDATOR-PREY DYNAMIC MODEL The dynamic model considered in [7] has the form: CHAPTER 4 96 y; =P y -P,y, y x / ( i +p j ry ;)-P, Y? £ = _ p 3 Y * + p v y/ Y* 7 ( 1 + P * - Y / } _ p,y a* We confine ourselves to a very b r i e f interpretation of this model. For more d e t a i l s see [7]. State variable y represents the prey population, and state variable y a represents the predator population. The above equations evolved from the Lotka-Volterra model introduced in Section 3.4. Equations (4.2.1) reduce to the Lotka-Volterra model equations when P = P T F = P 7 = 0 - T n e term- 1/{l+p^y/ ) in the above equations represents the s a t i a t i o n of the predators. That i s , as the number of prey increases, the a b i l i t y of the predators to consume prey i s limit e d , and when the number of prey i s large, the growth rate of the predators becomes independent of the prey population. The terms y ( z and P^y* represent competition among the prey and among the predators respectively. As the prey population y increases, i t becomes limited by such things as t e r r i t o r y . Similar l i m i t a t i o n s apply to the predator population. We consider one of the special cases studied by Bazykin. In this case, P 7=0 and there are two equilibrium points A and B in the phase plane (see Bazykin[7]). Point A i s at (P3 /(P^"P5.P3 ) ' ) (P, (Pv"P,- P3 > - P , P 3 )/(P^~P i rP 3 and point B i s at ( P /\9. 0) . / 6 CHAPTER 4 97 Since a l l the parameters are p o s i t i v e , for A to have physical meaning, we require p <p (p -p p )/p . Bazykin further shows that i f point A i s stable then p >p p a /(1+p a ) A where A has coordinates (a^ , a a ) , and i f A has b i o l o g i c a l significance and i s unstable, i t i s necessary that a l i m i t cycle occur. We consider three s i t u a t i o n s : (1) A has b i o l o g i c a l significance and i s stable; (2) A has b i o l o g i c a l significance and i s unstable; and (3) A has no b i o l o g i c a l s i g n i f i c a n c e . Case (1) arises for example at the parameter values (1, .1, 3, 1, .1, .15)17 In t h i s case A i s at (4.29, 5.10) and B i s at (6.67, 0) . Case (2) arises at the point (.5, .1, 5, 1, .15, .01) ' in parameter space. In this case A i s at (20.0, 12.0) and i s unstable and B i s at (50.0, 0). Case (3) arises at the point (1, .1, 3, 1, .2, . 1 5 ) T in parameter space. In t h i s case A has no b i o l o g i c a l significance and B i s at (6.67, 0). Phase plane plots corresponding to the above three parameter vectors are shown in Figures 4.2.1, 4.2.2, and 4.2.3. We use integration results at the , above three points in parameter space to simulate observations for our test problems. In p a r t i c u l a r , we make use of the following three test CHAPTER 4 98 CHAPTER 4 99 d i 1 1 1 1 1 ^ 1 i i 2 00 3 00 4.00 5.00 6.00 7.00 8.00 3.00 ]0.00 Yl Figure 4.2.3 Phase plot for case (3) problems: PROBLEM 4.2.1: Observations are generated using the parameters of Figure r 4.2.1 and the i n i t i a l condition y(0)=(2, 8) . PROBLEM 4.2.2: Observations are generated using the parameters of Figure f 4.2.2 and the i n i t i a l condition y(0)=(24, 8) . PROBLEM 4.2.3: Observations are generated using the parameters of Figure 4.2.3 and the i n i t i a l condition y(0)=(10, 10)'. In a l l of the above problems simulation runs were made using Gear's program [27], No random error was introduced into the generated observations, and observations were generated for both CHAPTER 4 100 state variables at times „5(.5)12.5 in a l l cases. The observations for Problems 4.2.1, 4.2.2, and 4.2.3 are displayed in Figures 4.2.4, 4.2.5, and 4.2.6 respectively. We also show o o : 1 1 1 1 1 1 1 0.00 2.00 4.DD 6.00 8.00 10.00 12.00 14.00 TIME Figure 4.2.4 Observations for Problem 4.^.]L the cubic spline least squares approximations to the observations that are used throughout this chapter. In Figure 4.2.4, the j o i n t s for the spline approximating the observations on y are at t=4,8. The jo i n t s for the cubic spline approximating the y a observations are at t=7,10.5. In Figure 4.2.5, the joint s for the spline approximating the y observations are at t=6.5,11.25, and the jo i n t s for the spline approximating the y^ observations are at t=2,3.75,8.75. In Figure 4.2.6, the j o i n t s for the spline approximating the observations on y^ are at t=2.5,7.5, and the j o i n t for the CHAPTER 4 101 .00 Figure 4.2.5 Observations for Problem 4.2.2 0.00 2.00 4.00 6.00 8.00 10.00 12.00 24.00 TIME Figure 4.2.6 Observations for Problem 4.2.3 CHAPTER 4 102 spline approximating the observations on i s at t=5. 4.3 IMPROVING STARTING PARAMETERS In Table 4.3.1 we present results using the Levenberg-Marquardt method and employing the s e n s i t i v i t y equations (for convenience we c a l l this the FIT technique) along with results using the derivative f i t t i n g (DFIT) method followed by the FIT method to refine the parameter values, and results using the integral f i t t i n g (IFIT) method followed by the FIT method to refine parameter values. The derivative and integral f i t t i n g methods are used as i n i t i a l techniques to improve our approximations to the optimal parameter values. From Table Problem Starting p Results with indicated method ( )-a figure, C-convergence, L-l o c a l minimum FIT DFIT+FIT IFIT+FIT 4 . 2 . 1 ( 4 . 2 . 2 ) L ( 4 . 3 . 1 ) C C ( 4 . 2 . 3 ) C C C 4 . 2 . 2 ( 4 . 2 . 1 ) C C C ( 4 . 2 . 3 ) M 4 . 3 . 2 ) C C 4 . 2 . 3 ( 4 . 2 . 1 ) C M 4 . 3 . 3 ) C ( 4 . 2 . 2 ) C L ( 4 . 3 . 3 ) C Table 4.3.1 FIT compared with DFIT+FIT and IFIT+FIT 4.3.1, we see that both the DFIT and IFIT methods work well for improving the approximations to the optimal parameters. This i s not too- unexpected since the d i f f e r e n t i a l equations are linear in a l l but one parameter. We observe that a d i r e c t method using CHAPTER 4 103 the s e n s i t i v i t y equations can lead to d i f f i c u l t i e s . This r e f l e c t s the increased nonlinearity that arises when we employ d i r e c t integration of the i n i t i a l value problem. The s i t u a t i o n that arose when the Levenberg-Marquardt technique was applied to Problem 4.2.1 starting with the parameters of Figure 4.2.2 i s t y p i c a l of the sort of thing that can happen. In this case a l o c a l minimum at (3.211, -.4441, -3.405, 3.159, -1.182, .6314)"^" was found. Integration results at these parameters are shown in Figure 4.3.1. At this l o c a l minimum the peaks and troughs in CD O d-\ 1 1 1 1 1 :—I 1 0.00 2.00 4.00 6.00 8.00 - 10.00 12.00 14.00 TIME Figure 4.3.1 A l o c a l minimum for Problem _4 ._2. 1^ the observations are being balanced against one another. The FIT approach worked for Problem 4.2.2 sta r t i n g at the parameters of Figure 4.2.1, but d i f f i c u l t i e s were encountered. CHAPTER 4 104 S p e c i f i c a l l y , the parameter }\ in the Levenberg-Marquardt procedure had to be adjusted to avoid certain points in parameter space where the integration blows up. The use of constraints might also be useful here; however, i t i s noteworthy that no such d i f f i c u l t i e s arose with the DFIT+FIT and IFIT+FIT methods on th i s problem. The FIT approach to Problem 4.2.2 starting at the parameters of Figure 4.2.3 produced a l o c a l minimum at (3.087, .6111, 3.086, .7745, .2007, .08077)T Integration results at these parameters are shown in Figure 4.3.2. No d i f f i c u l t i e s arose in this case with the DFIT+FIT and (f..Tl , , , , , j - | 0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 TIME Figure 4.3.2 A l o c a l minimum for problem 4_'2.2 IFIT+FIT methods. Problem 4.2.3 requires further discussion. At f i r s t CHAPTER 4 105 glance, i t appears to be the simplest of the three problems since the observations seem to exhibit no special features. However, there i s a l o c a l minimum in parameter space corresponding to a solution to the d i f f e r e n t i a l equation which comes very close to the given observations. This appears to res u l t from the small peak in y ^ near t=0.5, and from the contrived nature of the problem with no random error in the observations. The observations st a r t at t=0.5 and i f we do not look at the i n i t i a l conditions, the small peak near t=0.5 i s i n v i s i b l e . Thus the DFIT method found a point in parameter space where th i s peak was absent. Consequently the DFIT+FIT. combination found a l o c a l minimum at the parameters (1.028, .2255, 1.290, .7186, .5493, .1517)T The sum of the squares of the residuals at th i s point in parameter space was approximately 10. Integration results at the above parameters are shown in Figure 4.3.3. If there were some random error in the observations, the solution shown in Figure 4.3.3 might appear quite adequate. However, in this contrived case there exists a more optimal solution with the objective function equal to zero. Our implementation of the IFIT method which has access to the i n i t i a l values had no d i f f i c u l t y with t h i s problem. The FIT approach to this problem managed to extract the global optimum, but not without some d i f f i c u l t i e s . Starting with the parameters corresponding to Figure 4.2.1, no problems arose; however, star t i n g with the CHAPTER 4 106 X d i 1 1 1 1 1 I 1 0.03 2.CO 4.00 - 5.00 8.00 10.00 12.00 14.00 TIME Figure 4.3.3 A l o c a l minimum for Problem _4._2._3 parameters of Figure 4.2.2, d i f f i c u l t i e s were encountered and X in the Levenberg-Marquardt procedure had to be increased to avoid certain regions in parameter space. 4.4 GUESSED OBSERVATIONS AND ITERATED METHODS In this section we present some experiments with the techniques introduced in Section 3.4. These technigues are designed for the important case when observations are not available on a l l state variables. F i r s t we experiment with the use of guessed observations for unobserved state variables. To f a c i l i t a t e the guessing of state variable behavior, an interpolating cubic spline i s passed through a set of guessed observation points. This eliminates the need to enter long l i s t s of guessed observations. Experiments are conducted on CHAPTER 4 107 PROBLEM 4.4.1: This problem i s the same as Problem 4.2.1 except observations are only available on state variable y and Y a(0)=8. I n i t i a l l y we looked at Problem 4.4.1 with Y A(0) an unknown parameter. However, t h i s problem was singular due to a linear r e a l t i o n s h i p between p a and the i n i t i a l condition on y^. The columns corresponding to these two parameters in the least squares Jacobian matrices are multiples of one another and the cor r e l a t i o n c o e f f i c i e n t between these two parameters i s 1. This occurs because the state equation involving y^ i s homogeneous in y . Thus y can be replaced by cy and the second state equation i s not altered, except for the i n i t i a l condition on y a which i s divided by c. However, in the f i r s t state equation, p^ is replaced by cp . Thus a r e l a t i o n exists between p and the i n i t i a l condition on y . a. To apply the derivative f i t t i n g method to Problem 4.4.1, we approximate the observations on y with the least squares cubic spline used for experiments with Problem 4.2.1 presented in Table 4.3.1. In Figure 4.4.1, some guesses at possible observations on y are shown. Referring to this figure, curve (a) is an interpolating cubic spline for the points (0, 8), (2.5, 1.5), (7, 6.5), (10, 4.5), (12.5, 5), curve (b) i s an interpolating cubic spline for the points (0, 8) , (2.5, 3) , (8.75, 5) , (12.5, 4) . CHAPTER 4 108 \\ \ 1 1 1 1 ; 1 : 1 I 0.00 2.00 4.00 6.CO B.00 10.00 12.00 U . C O TIME Figure 4.4.1 Guessed observations for problem 4.4.1 and curve (c) i s an interpolating cubic spline for the points (0, 8), (5, 3), (8.75, 5), (12.5, 4) , (that i s , one point was moved). The guessed observations shown in curve (a) are f a i r l y close to the observations on y a in Problem 4.2.1, while the guessed observations in curve (c) are substantially d i f f e r e n t from those of Problem 4.2.1. The guessed observations in curve (b) are intermediate to those in curves (a) and (c). End conditions for the interpolations are described under the CREOBS command in Appendix A. Figure 4.4.2 shows the integration results at the parameter vector (.2354, -.1977, 4.983, 1.049, .01350, .05878) ' CHAPTER 4 109 obtained with the FIT approach (with error controlled integrations) to problem 4.4.1 using the starting parameters of TIME Figure 4.4.2 FIT on Problem 4.4.1 10.OD 14.00 Figure 4.2.2. A l o c a l minimum has been obtained. Using points on curve (a) in Figure 4.4.1 for guessed observations, the DFIT+FIT, and IFIT+FIT combinations both produced the correct parameters. The starting parameter values were those corresponding to Figure 4.2.2. Using guessed observations from curve (b) in Figure 4.4.1, the DFIT+FIT combination found a l o c a l minimum at (.2507, -.3758, 1.676, 2.729, 1.698, .07026). Integration results at these parameters are shown in Figure 4.4^3. The IFIT+FIT combination with these guessed observations produced the correct minimum. When guessed observations from CHAPTER 4 110 CD d i 1 1 1 i r 1 1 0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 TIME "Figure 4.4.3 DFIT+FIT on Problem ._4. 1^ using guess (b) curve (c) in Figure 4.4.1 were used, both the DFIT and IFIT methods produced parameters from which the FIT method was unsuccessful due to i n s t a b i l i t i e s . The change of the position of one data point from t=2.5 to t=5 in the guessed observations of curves (b) and (c) of Figure 4.4.1, meant the difference between disaster and the agui s i t i o n of a minimum. The above experiments indicate that some interactive experimentation with guessed observations and with pa r t i c u l a r methods can be pr o f i t a b l e . Next some experiments using the i t e r a t i v e techniques of Section 3.4 for improving guessed observations are presented. These techniques can reduce the dependence on a good set of guessed observations. CHAPTER 4 I l l Three methods are considered. F i r s t we consider iterated derivative f i t t i n g where the function (3.4.1) with d^(c) defined by (3.4.2) i s minimized. Second, we consider iterated integral f i t t i n g where our parameter estimates are updated by the integral f i t t i n g technigue and the guessed observations are updated to minimize the function (3.4.1). F i n a l l y , we consider an i t e r a t i v e scheme where the parameters are updated using the inte g r a l f i t t i n g technigue, and the guessed observations are updated by integrating a subsystem of the given system of d i f f e r e n t i a l equations. Experiments are conducted on Problem 4.4.1. The case when there i s a random error in the observations i s considered in the next section. In a l l experiments, the i n i t i a l condition for the unobserved state variable remains fixed at 8. We observe that the least squares problem for updating the guessed observations with p held fixed i s linear in th i s example. Starting with the guessed observations of curve (b) in Figure 4.4.1 and the parameters corresponding to Figure 4.2.2, a l l three methods converged to a parameter vector from which the FIT method converged to the desired solution. In a l l cases, the observations on y were approximated with the spline shown in Figure 4.2.4. The iterated derivative and iterated integral f i t t i n g methods employing (3.4.1) produced similar results and we present graphical results only for the iterated derivative f i t t i n g case. In Figure 4.4.4, the observations on y and integration results for y at successive parameter estimates obtained with the CHAPTER 4 112 iterated derivative f i t t i n g method are shown. In Figure 4.4.5, 0.00 2.00 4.00 B.0O TIME 6.00 10.00 12.00 J'4.00 Figure 4.4.4 Iterated DFIT results the i t e r a t i o n s on the guessed observations corresponding to the results presented in Figure 4.4.4 are shown. We note that the derivative f i t t i n g method using guessed observations from curve (b) in Figure 4.4.1, did not produce parameters from which the FIT method could find the global minimum, but the iterated derivative f i t t i n g method did produce parameters from which the FIT method was successful. In th i s case the it e r a t i o n s were c r u c i a l to obtaining the desired solution. Next we consider i t e r a t i v e l y improving the guessed observations by integrating the unobserved state variable with the observed state variable held fixed at the smoothed approximation to the observations. In Figures 4.4.6 and 4.4.7 CHAPTER 4 113 UJ I— c r w \ \\\ 4? / to) ^ / / 0.00 2.00 4.00 5.00 TIME 8.00 10.00 12.00 14.00 Figure 4.4.5 Iterated DFIT guessed observation i t e r a t i o n s we give results with this technique applied to Problem 4.4.1 start i n g with the guessed observations of curve (b) in Figure 4.4.1. Figure 4.4.6 shows the observations on y and integration results on y at successive parameter estimates. With th i s technique, the difference between i t e r a t i o n (0) and i t e r a t i o n (1) was substantial, while i t e r a t i o n s (1) and (2) were e s s e n t i a l l y i d e n t i c a l . The it e r a t i o n s shown in Figure 4.4.7 correspond well to the simulation results in Figure 4.2.4. The FIT approach had no trouble converging to the global minimum start i n g at the parameters provided by this i t e r a t i v e process. CHAPTER 4 CHAPTER 4 115 4.5 THE PRESENCE OF NOISE In this section we consider a selection of the previous problems with a random error introduced into the observations. The problems considered are: PROBLEM 4.5.1 This problem i s the same as Problem 4.2.1 except a normally di s t r i b u t e d random error with zero mean has been introduced into the observations. The standard deviation for the error in y i s .5 and the standard deviation for the error in y i s 1. PROBLEM 4.5.2 This problem i s the same as Problem 4.2.2 except a normally d i s t r i b u t e d random error with zero mean and standard deviation 2 has been introduced into the observations. PROBLEM 4.5.3 This problem i s the same as Problem 4.5.1 except the observations on y a have been removed. The i n i t i a l condition for y i s fixed at 8. We smooth the observations for Problems 4.5.1, 4.5.2, and 4.5.3 using least squares piecewise cubic splines with the same jo i n t s as were used for the smoothings in Problems 4.2.1, 4.2.2, and 4.4.1 respectively. F i r s t we consider Problem 4.5.1. Starting at the parameters correspomding to Figure 4.2.2, a d i r e c t approach (FIT method) found the point (.3668, -.08827, 4.874, 1.237, -.4058, .07037) 7" CHAPTER 4 116 in parameter space. Integration results at this point were q u a l i t a t i v e l y quite d i f f e r e n t from the observations. The sum of the squares of the residuals at the above point in parameter space was approximately 515. The IFIT+FIT and DFIT+FIT methods both found a minimum at r (.8865, .04991, 2.983, .6827, -.009550, .1483). In both cases, the star t i n g parameters were those corresponding to Figure 4.2.2. The sum of the sguares of the residuals at the above point in parameter space was approximately 15.5. The FIT method also found the above minimum when i t was started from the parameters corresponding to Figure 4.2.3. Unfortunately, p^ _ i s negative. This suggests we try a square root scaling of p6_ to constrain i t to be p o s i t i v e . With this scaling and starting at the parameters of Figure 4.2.2, the IFIT+FIT combination produced the parameters (unsealed) r (.8962, .05327, 3.076, .7405, .001897, .1498). The sum of the squares of the residuals at the above parameters was approximately 15.5. Integration results at the above parameters are shown in Figure 4.5.1. F i n a l l y , with this s c a l i n g , the FIT method also produced the above parameters when started from the parameters of Figure 4.2.2. Next we consider Problem 4.5.2. Starting at the parameters corresponding to Figure 4.2.3, the FIT method drew us to the point CHAPTER 4 117 (5.544, .7024, 2.647, -0.6093, -.07095, 1.021) in parameter space at which the integration results and the observations were q u a l i t a t i v e l y quite d i f f e r e n t . (The integration results on y contained a rapid and damped o s c i l l a t i o n . ) The sum of the squares of the residuals at the above parameters was approximately 1000. Both the DFIT+FIT, and the IFIT+FIT methods found an optimum at r (.4018, .03851, 1.254, .07606, .01059, .001893) The sum of the squares of the residuals at the above parameters was approximately 147. Integration results at the above point CHAPTER 4 118 in parameter space are shown in Figure 4.5.2. The FIT method, 14.00 Figure 4.5.2 Results for Problem 4.5.2 using a square root scaling of p and p and starting at the parameters of Figure 4.2.3, also found the above point in parameter space. F i n a l l y , we consider Problem 4.5.3. The random error in the observations combined with the missing observations on y^ make thi s a rather nasty problem. The noise i s f a i r l y large in this example, but this much noise i s not uncommon in problems involving population counts. Starting with the parameters corresponding to Figure 4.2.2, the FIT approach found an optimum at 7" (.5945, .0005587, 5.760, .7368, -.1121, .1073) The sum of the squares of the residuals was approximately 5.4 at CHAPTER 4 119 the above parameters. Graphically, integration results at the above parameters look very good. However, p i s negative, and the integration results for y^ at the above parameters are very d i f f e r e n t from the simulation results shown in Figure 4.2.4. (the integration results for y^ have a spike which extends to around ya=400 at t~7.5.) Using the guessed observations from curve (b) in Figure 4.4.1, and sta r t i n g at the parameters corresponding to Figure 4.2.2, three i t e r a t i o n s of the iterated derivative f i t t i n g method produced the parameters (1.522, .3921, 3.186, 1.709, .3031, .1829)T The observations and integration results at the above parameters are presented in Figure 4.5.3. Using the same star t i n g conditions as for the previous experiment, three i t e r a t i o n s of the iterated integral f i t t i n g method (using subsystem integrations) gave the parameters (1.402, .2126, 3.584, 2.497, .4548, .2249)T Integration results at the above parameters are presented in Figure 4.5.4. Using the same star t i n g conditions as in the previous experiment, three i t e r a t i o n s of the iterated integral f i t t i n g method (using (3.4.1) to update the guessed observations) produced the parameters (1.388, .2153, 4.652, 2.493, .2981, .2275). Integration results at the above parameters are presented in CHAPTER 4 120 o 03 J4.DD TIME Figure 4.5.4 IFIT results (using subsystem integrations) CHAPTER 4 121 Figure 4.5.5. From each of the above parameter vectors, the FIT method found the same minimum that was found when the FIT method alone was used. Since results with the iterated methods appear f a i r l y TIME 8 . D O 10.00 12.00 Figure 4.5.5 IFIT results (using (3.4.1)) 14.00 good graphically, and the parameters generated by the iterated methods are po s i t i v e , i t appears worthwhile to try a square root scaling on p starting from the results of an iterated method. Starting from the results of the iterated integral f i t t i n g method (using (3.4.1)) and using this scaling, we found an optimum at (unsealed) T (.6859, .03515, 6.092, 1.481, .2708E-5, .1213). The sum of the squares of the residuals at the above point in parameter space was approximately 5.8. Integration results at CHAPTER 4 122 the above parameters are shown in Figure 4.5.6. If we return to the d i r e c t FIT approach starting with the -) i 1 1 ,— 4 -0C 6.00 8.00 10.00 12.00 TIME Figure 4.5.6 FIT results (p^scaled) st a r t i n g from IFIT results parameters corresponding to Figure 4.2.2 and with p scaled with a square root transformation, problems occur. The parameter p, becomes negative. The scaling of both p^ and p ^ d i d not prove very p r o f i t a b l e either. With th i s scaling and star t i n g at the parameters corresponding to Figure 4.2.2, we were drawn to a point in parameter space where a l l the parameters were p o s i t i v e , but where the integration results on y^ rapidly went to zero, and the results on y went to an equilibrium. Thus the iterated approach was extremely valuable for getting a solution to this problem. The use of guessed observations may be thought of as a means of guiding us to a preferred solution. CHAPTER 4 123 CHAPTER 5 SEQUENTIAL TECHNIQUES 5.1 INTRODUCTION The d i r e c t approach, using the s e n s i t i v i t y equations, for f i t t i n g parameters in dynamic models involves the choice of an i n i t i a l approximation to the optimal parameter vector, integrating at this point in parameter space, and then, with the aid of information from the solution to the s e n s i t i v i t y equations, finding a more optimal set of parameters. Often the i n i t i a l integration deviates greatly from the observations, and i t may even blow up. In these cases, the f i r s t few data points contain valuable information that can be used to improve the parameter estimates. It i s i n t u i t i v e l y appealing to use this information to improve some parameter values before we commit ourselves to a f u l l integration over the whole time i n t e r v a l under consideration. To carry this idea a l i t t l e further, an algorithm where we sequentially update parameter estimates each time taking into account a few more data points seems worthy of consideration. In a sense, such an algorithm i s using the observations to guide us along the correct path in state space. Sequential reestimation has received a great deal of attention. Much of this attention has been from a s t a t i s t i c a l point of view. For a concise introduction, we refer the reader to Young[76]. Problems involving the analysis of a large quantity of continuously a r r i v i n g data and requiring a "real CHAPTER 5 124 time" solution have been one of the main motivations for the development of sequential estimation techniques. A t y p i c a l example of such a s i t u a t i o n occurs in the estimation of missile t r a j e c t o r i e s from, say, radar data. Frequently, sequential techniques are used to estimate the state of a dynamic system; however, they can also be used to estimate parameters in a dynamic system. For a good discussion of sequential estimation techniques applied to dynamic systems, we refer the reader to Gelb[28]. As mentioned above, our motivation for considering sequential techniques does not come from the need to rapidly process a large quantity of data, but instead from the need to overcome poor i n i t i a l parameter estimates. Our goal i s to use the observations in a manner that addresses i t s e l f to the b a s i c a l l y sequential nature of an i n i t i a l value problem. 5.2 A SEQUENTIAL ALGORITHM There are many ways to approach the development of a sequential algorithm. At one extreme there i s the approach of solving a sequence of parameter f i t t i n g problems, each using progressively more data points. Since we are dealing with a dynamic model, such an approach would be expensive. At the other extreme, we have the stochastic approximation techniques where parameters are updated by less refined but very fast reestimation algorithms ([6,p.251], [3], [33]). The method explored in t h i s chapter f a l l s between these two extremes. We assume the data points are processed in batches ending CHAPTER 5 125 at the observation times t. i • •« , t« (5.2.1) Batch s i s the set of observations taken at the observation times fcJk t i ' t * -+2. A - I n-i / • • • f t . (5.2.3) where t f c + | i s the f i r s t observation time. Define f (p) according to (1.1.2) for the observation times (5.2.3) and l e t (P)=f(„, (P)f, r t (P) (o) (5.2.4) To s t a r t the sequential process we estimate p as well as possible to minimize (5.2.4). Note that k must be large enough that t h i s least squares problem makes sense. That i s , we do not want fewer data points than parameters. We expect this problem to often be singular since i t i s unreasonable to expect that an estimate of a l l the parameters can be obtained with just a few data points. Denote by p the optimal parameter vector obtained by minimizing (5.2.4), and denote the Jacobian matrix for f defined by (1.3.4), by J ^ (p). Next we include the batch of data points at the observation times Define f (p) according to (1.1.2) for the above observation CHAPTER 5 126 times. There i s some ambiguity to be resolved concerning f ( ) i (p) . The elements in f ( | ) (p) depend on the solution y(t) to (1.1.1) at the above observation times. We would, however, l i k e to integrate from time t ^ and not from time t f l in order to define f ^ (p). Thus i n i t i a l conditions are required at time t« for this integration. Consistent with the l i n e a r i z a t i o n employed below, we define the i n i t i a l condition on y . at time t*. by Y E > . ( t ^ )=y; (P ( - ) ) + (P"P(0, ) T(>Y;,...»%) (5.2.5) for i=l,...,n. Thus the i n i t i a l conditions for the integration between times t . and tA are functions of the parameters, and this must be considered when the s e n s i t i v i t y equations are integrated. Define the Jacobian matrix J ^ (p) corresponding to f ( | ) (p) according to (1.3.4). This matrix i s found by integrating the s e n s i t i v i t y equations from time t_^ to time t ^ . Let (AP) ( E J =P-P(OJ (5.2.6) and define f (p) by f l l )(p) = ( ( f ( f t ) (P ( e J )+J ( o J <P(„ ) U p > ( # ) ) T , f ^ (P) ^ (5.2.7) Our new parameter estimate, which we denote by p ^ , minimizes f (P)f (P)- (5.2.8) The Jacobian matrices for f (p) are CHAPTER 5 127 J (p) = (O J - . (p) (5.2.9) We have just approximated f^0)(p) by the f i r s t term in i t s Taylor expansion at the point p . The success of a sequential technique i s clos e l y related to the size of the region in which t h i s approximation i s accurate. We continue in the above manner reestimating p for each new batch of data points. Thus when batch s i s under consideration, we are finding the estimate p ^ j using f (P) = (P{.> ) + J (.) (P l 6 ) ) (AP), f w (p) (5.2.10) and J (p) = J (*> ( P (,) > Jfr> (P) The new parameter estimate, p, , minimizes f w _ r ( P ) f U ) ( P ) . (5.2.11) (5.2.12) Note that only J (p) is changing in J determination of p (p) during the } . This fact can be used to advantage when implementing a sequential algorithm. For example, i f we are using the singular value decomposition of J then information CHAPTER 5 128 obtained when producing the decomposition of J ( p ^ ^ ) can be used to e f f i c i e n t l y obtain the decomposition of J (p). 5.3 EXPERIMENTAL RESULTS We experiment with Problems 4.2.1, 4.2.2, 4.5.1, and 4.5.2. The l a s t two of these test problems have noise in the observations. The star t i n g parameters for our experiments with Problems 4.2.1 and 4.5.1 are those corresponding to Figure 4.2.2, and the star t i n g parameters for our experiments with Problems 4.2.2 and 4.5.2 are those corresponding to Figure 4.2.3. We comment that with these parameters a d i r e c t approach (FIT) was unsuccessful on a l l four problems. Our experiments with Problems 4.2.1 and 4.2.2 star t with an i n i t i a l batch of observations corresponding to the f i r s t f i v e observation times. We then proceed through the remaining observation times in increments of f i v e observation times. For Problems 4.5.1 and 4.5.2 the use of f i v e observation times proved to be i n s u f f i c i e n t to get started (the i n i t i a l parameter estimates were too inaccurate). For these two problems we star t with ten observation times and then proceed in increments of f i v e observation times. An interactive approach could be valuable here. For Problem 4.2.1, the parameter estimates after the f i r s t and l a s t batches were respectively T (.9743, .1281, 3.873, 2.101, .3153, .1473) and CHAPTER 5 1 2 9 ( 1 . 0 1 4 , . 1 1 7 8 , 3 . 5 4 8 , 1 . 4 0 6 , . 1 6 3 2 , ,1539)T For Problem 4 . 2 . 2 , the parameter estimates after the f i r s t and l a s t batches were respectively T (.5550, . 0 9 2 7 7 , 3 . 2 5 6 , . 6 3 0 4 , . 1 3 9 0 , . 0 1 2 3 1 ) and r ( . 4 6 8 7 , . 0 6 5 4 8 , 3 . 3 8 3 7 , . 4 2 2 6 , . 0 7 8 8 6 , . 0 0 8 3 6 4 ) . For Problem 4 . 5 . 1 , the parameter estimates after the f i r s t and l a s t batches were respectively ( . 6 6 8 9 , . 0 2 4 9 3 , 4 . 0 8 5 , 1 . 3 0 9 , . 0 6 2 3 2 , . 1 1 5 4 ) 7 " and ( . 7 6 9 0 , . 0 4 5 4 0 , 3 . 2 3 8 , . 8 0 0 6 , . 0 1 4 2 1 , .1286)7^ For Problem 4 . 5 . 2 , the parameter estimates after the f i r s t and l a s t batches were respectively ( 7 . 2 2 7 , 1 . 3 0 3 , 3 . 3 3 7 , 1 . 3 3 3 , . 3 3 3 4 , . 2 2 6 6 ) " ^ and T (.6987, . 2 8 2 9 , 6 . 9 1 7 , 2 . 7 0 3 , . 3 3 8 9 , . 0 1 3 7 7 ) . A d i r e c t approach (FIT method) converged to the desired solution for Problems 4 . 2 . 1 , 4 . 2 . 2 , and 4.5.1 starting at the f i r s t and l a s t estimates given above. A square root transformation of p^. was required in the case of Problem 4.5.1 to prevent p^ _ from becoming negative. The d i r e c t approach on CHAPTER 5 130 Problem 4.5.2, star t i n g from the parameters obtained using only the f i r s t batch of observations, found a l o c a l minimum at (3.624, .8877, 4.064, 1.277, .2642, .1002) r where the sum to the squares of the residuals was approximately 840. However, convergence of the d i r e c t method to the desired solution was obtained when we started from the f i n a l r e s u l t of the sequential pass on Problem 4.5.2. Thus the sequential updating was essential in th i s case. We summarize the above results in Table 5.3.1. We conclude that i t can be advantageous Problem FIT FIT FIT ( f i r s t batch) (last batch) 4.4.1 L C C 4.4.2 L C C 4.5.1 L C C 4.5.2 L L C (L-local minimum, C-desired minimum) Table 5.3.1 Results with sequential approach to consider the observations sequentially to obtain an improved approximation to the optimal parameters before we commit ourselves to a f u l l optimization attempt over the whole time i n t e r v a l . Indeed, in three of the above four cases a s u f f i c i e n t improvement to allow the FIT method to converge to the desired solution was obtained using only the f i r s t few observation points. A great deal of work remains to be done to f u l l y evaluate CHAPTER 5 131 the use of sequential methods for improving starting parameters; however, we have attained our limited goal of demonstrating the f e a s i b i l i t y of using a sequential strategy. CHAPTER 5 132 CHAPTER 6 REAL WORLD PROBLEMS 6.1 INTRODUCTION A l l the parameter f i t t i n g problems in thi s chapter involve physical observations as opposed to observations generated by a simulation. Such problems are a good deal more d i f f i c u l t than those using generated data. This d i f f i c u l t y occurs partly because the dynamic model under consideration often cannot, for any parameter values, give an adeguate description of the process being modelled. Also, experience indicates that the least squares surface for parameter f i t t i n g in dynamic models i s often plagued with numerous l o c a l minima. Starting with one of the more global methods of Chapter 3, i t i s f a i r l y easy to find one of these l o c a l minima. (See for example the experiments with Bazykin's model in Chapter 4.) The problem for the model builder i s to decide i f there i s a more optimal set of parameters somewhere else in parameter space or i f the qu a l i t a t i v e difference between the model and the data at the current minimum i s just the resu l t of a poor or incomplete model. This i s usually a d i f f i c u l t decision to make. Situations such as this arise frequently in nonlinear problems and a standard strategy i s to start optimizing from d i f f e r e n t points in parameter space. If the same minimum i s determined starting from several d i f f e r e n t points, then we can be more confident that the minimum i s a global minimum. An interactive CHAPTER 6 133 approach i s ideal for experimentally checking on a minimum; however, a good understanding of the model and the physical meaning of the parameters i s also very valuable. This l a t t e r understanding can be augmented through an interactive analysis of the effects of various parameter changes. An interactive parameter f i t t i n g package can be very useful in the design of models as well as in the f i t t i n g of individual models. For example, the model builder may start with a simple but incomplete model and find in the course of f i t t i n g i t that i t cannot account for some of the q u a l i t a t i v e behavior of the observations. This would be indicated for example i f the best f i t t i n g parameters produced a model which smoothed out a c r u c i a l peak in the data. With luck, in the course of f i t t i n g this model, some of i t s de f i c i e n c i e s may be determined and some insight into improvements may be gained. For this type of application, i t would be helpful to allow the dynamic r e d e f i n i t i o n of the model. We stress that the above process i s very tenuous and puts a large emphasis on the i n t u i t i o n and judgement of the model builder. It i s in such situations, however, where an interactive approach can be extremely advantageous. 6.2 A DYNAMIC MODEL FOR AGGRESSIVE AND DOCILE MICE The model considered in this section proposes a population consisting of two interacting types of mice to account for observations on the t o t a l mouse population. For an introduction to this problem, we refer the reader to Myers and Krebs[48] and CHAPTER 6 134 Krebs et al[37]. Let u be the population density of docile mice, and v the population density of aggressive mice. Let p^ be the basic b i r t h rate, p^ the basic death rate, and l e t p^ . describe the s e n s i t i v i t y of docile mice to crowding. Let c< be the proportion of aggressive mice in the o f f s p r i n g . Further assume the aggressive mice reproduce poorly (almost s t e r i l e ) and that this can be described by multiplying the basic b i r t h rate by u/(u+v). Under the effects of crowding, the docile mice are assumed to either emigrate or die, and this i s described by the term -p^ufu+v) in the equation for u'. The dynamic model i s thus u'=p, ( l - O u*/(u+v) -p u-p u(u+v) 3 H (6.2.1) v ' =p3p<. u a / (u+v) -pH v. Questionable assumptions such as those given above are t y p i c a l of dynamic models in ecology, and with such assumptions we should not be too disappointed i f the model cannot describe the observations very well. Mouse population measurements are available only on the t o t a l population u+v. A more general formulation of the parameter f i t t i n g problem in dynamic models (see Bard[6,p.221]) could handle this d i r e c t l y ; however, the problem can be transformed to conform to our formulation. In so doing, we arrive at a problem with observations on only one state variable and some of the techniques of Chapter 3 can be used to produce starting approximations to the parameters. If we define y =u, y =u+v, and p =p (l-«0 then the above dynamic CHAPTER 6 135 model may be written as y/=P6 y,a/y -p y y y ' 6 1 * f ' ^ (6.2.2) y'=p y a/y -p y -p y y . For our i n i t i a l conditions we take y (0)=p /(l+exp.(-pa ) ) ' (6.2.3) Y (0)=p, i where we have ensured that y (0)<y (0). Observe that a l l / a. parameters in this model, with the possible exception of p^, should be po s i t i v e . The 44 observations on y^ are shown graphically in Figure 6.2.1. We comment that i t i s with reference to the scale of Figure 6.2.1 Observations and spline approximation Figure 6.2.1 that the parameter estimates should be interpreted, CHAPTER 6 136 (The basic unit of time i s two weeks and the basic population density unit i s ten animals per acre.) A cubic spline approximation to the observations using j o i n t s at t=5,10,20,25,28,35 i s also shown in Figure 6.2.1. This problem i s r e l a t i v e l y d i f f i c u l t . In the following discussion, we outline a p a r t i c u l a r sequence of experiments which leads to a model which f i t s the observations quite well numerically. Such experiments, of necessity, involve a good deal of t r i a l and error, and thus an interactive approach i s i d e a l . In what follows, we try to give an indication of this i n teractive process. Since observations are available on only one state variable, the i t e r a t i v e methods of Section 3.4 may be useful for getting i n i t i a l approximations to the parameters. Furthermore, we observe that with the exception of p a , a l l the parameters occur l i n e a r l y in the residual functions of the IFIT and DFIT algorithms when guessed observations are used on y . However, to apply one of the i t e r a t i v e algorithms of Section 3.4, a sta r t i n g guess at the behavior of the unobserved state variable i s required. It must be less than y , and we expect i t to mimic in some sense the behavior of ya • A reasonable guess i s curve (0) in Figure 6.2.2. Note that one of the most prominent features of y / with respect to y^ i s the position of the proposed maximum of y^ . As indicated by the experiments in Chapter 4, the position of such a maximum can be c r i t i c a l . An interactive approach can be very valuable here. To get st a r t i n g CHAPTER 6 137 T I M E Figure 6.2.2 Iterations on guessed observations approximations to the parameters, we used the iterated integral f i t t i n g method (using (3.4.1)) taking advantage of the l i n e a r i t y of the parameters in (6.2.2). We froze p^ and p^ at 2.9 and 2 respectively and iterated on the remaining four parameters. Figure 6.2.2 shows successive improvements in the guessed observations on y . In Figure 6.2.3, the solution y^ (t) obtained from integrating (6.2.2) at the successive approximations to p are shown. The observations on y^ are also shown in Figure 6.2.3. The trapezoidal d i s c r e t i z a t i o n was used throughout this section. Curve (i) in Figure 6.2.3 corresponds to curve (i) in Figure 6.2.2. The parameters corresponding to curve (2) in Figure 6.2.3 were (2.9, 2.0, 1.234, .2445, .003339, .6114). CHAPTER 6 138 8 o UJ o y-°. I— ~* i n XX x x X X x x X x>< x .X x x * (a) X X/ ^ ^ = = = = = - — J L X x x x x x x X x , r 1 1 1 1 r i i 0.00 5.00 10.00 IS.00 20.00 2S.00 30.00 3S.00 40.00 45.00 T I M E Figure 6.2.3 Integrations at successive parameter approximations Graphically, there appears to be substantial room for improvement in the parameter values; however, the iterated integral f i t t i n g method has e f f i c i e n t l y provided starting approximations to the l a s t four parameters. Now that we have star t i n g approximations to the parameters at which the i n i t i a l value problem can be integrated to produce reasonable r e s u l t s , i t i s worthwhile to try to refine the parameters values using a d i r e c t approach employing the s e n s i t i v i t y equations. The Levenberg-Marquardt method gave the parameters T (2.828, 1.963, 1.142, .06483, .008582, .5279) Integeration results corresponding to th i s parameter vector are shown in Figure 6.2.4. The sum of the squares of the residuals CHAPTER 6 139 at the above point in parameter space was approximately 1350. Clearly, an improvement has been made. However, the peak in the o C3 m XX °J XX X X x x ' i I ! I 1 1 r 5.00 10.00 ]5.C0 20.00 25.00 30.00 35 00 40 00 TJME Figure 6.2.4 Optimum star t i n g from iterated IFIT results observations i s not being approximated well. Considering the way the model was derived, there i s no reason to expect that the peak in the observations can be well approximated. If we want to try to f i t the peak in the data, we can experiment at other points in parameter space and try other algorithms such as those presented in Section 3.5. An interactive package can be very valuable for such probing and experimenting. Extensive experiments and the use of special methods such as the quasi-multiple shooting technigue of Chapter 3 did not produce parameter values which described the peak well. A logarithmic transformation of p which works well on the following two CHAPTER 6 140 models did not work here either. It i s impossible to say no such parameters exist ; however, in view of the better f i t obtained later in t h i s section when more f l e x i b i l i t y was put into the model, we are led to the conclusion that the model cannot adequately describe the observations. Such a conclusion i s a matter of judgement and a l l a good numerical package can do i s to provide information to make that judgement more informed. It i s f a i r l y easy using an interactive approach to produce parameters that give integration results which peak in the same v i c i n i t y as do the observations. For example, the following parameters were obtained i n t e r a c t i v e l y : (1.4, 1.24, .8272718, .07, .006, .5037043)T The interactive procedure employed involved freezing parameters, optimizing on subspaces, and the experimental resetting of parameters. Integration results at these parameters are shown in Figure 6.2.5. The d i f f i c u l t y seems to be that the sudden drop in the observations cannot be imitated. Thus when we t r i e d a f u l l least squares optimization starting at the above parameters, we obtained a possibly l o c a l minimum at (1.292, 1.171, .9662, .06040, .01371, .6242)T Integration results at the above parameters are shown in Figure 6.2.6. The sum of the squares of the residuals was approximately 1150 at the above parameters. Observe that the results are s l i g h t l y d i f f e r e n t than those of Figure 6.2.4, but the peak in the data i s s t i l l not being imitated. Indeed, even CHAPTER 6 141 C 3 di i 1 1 1 1 1 1 1 1 o.oo 5.oo lo.oo :s.oo 20.00 25.00 30.0c 35.00 40.00 4S.00 TIME Figure 6.2.5 Integration at parameters found i n t e r a c t i v e l y the i nteractive optimization on subspaces continually drew the integration results down to balance off the peak and the trough in the observations. Of course we could weight some of the data points in the least squares problem to emphasize the peak, but this i s a r t i f i c i a l and experiments indicate that while a good approximation up to the peak can ea s i l y be obtained, the rapid drop in the observations cannot be imitated. Next we experimented with a model with more f l e x i b i l i t y in the form of another parameter. S p e c i f i c a l l y , we l e t the death rates for the docile and aggressive mice be d i f f e r e n t . Denote by p^ the death rate for docile mice and l e t p^ denote the death rate for aggressive mice. Our dynamic model now i s CHAPTER 6 142 XX ; i 1 ! 1 1 1 i : i 1 1 0.00 S.CD 10.00 35.00 30.00 25.00 30.CD 35.00 40.00 45.00 T I M E .. FYgure" "6T2V6"~ Optimum star t i n g from parameters found i n t e r a c t i v e l y y'=p yVy -p y -p y y, ' 6 ' * H ' ( 6 . 2 . 4 ) y'=p yVy.-py -p (y -y ) - P y y Using this model and starting at the parameters obtained i n t e r a c t i v e l y with p^=p^ , a l o c a l minimum to the f u l l unweighted least squares problem was found at ( 1 . 4 0 2 , 1 . 2 4 1 , . 8 3 6 5 , . 1 8 7 5 , . 0 1 1 7 4 , . 6 7 4 8 , . 0 0 5 7 6 3 ^ The sum of the squares of the residuals at the above parameters was appproximately 9 6 0 . Although p and p d i f f e r , integration results at the above parameters do not d i f f e r s ubstantially from those shown in Figure 6 . 2 . 6 . CHAPTER 6 143 This i s in r e a l i t y a l o c a l minimum. Later in t h i s section, we return to t h i s model and achieve a much smaller but s t i l l perhaps l o c a l minimum. The s p e c i f i c strategy used to obtain th i s new minimum grew out of experiments with the following model. Experimentally, the following model f i t s the observations well. It i s the same as the . previous model except for a modification in the growth term for the aggressive mice. As in Equation (6.2.1), l e t u be the population density of docile mice, and l e t v be the population density of aggressive mice. Our proposed dynamic model is u'=p u"V(u+v)-p u-p u(u+v) v'=p3uv/(u+v) -p^v (6.2.5) which with y =u, and y„=u+v becomes y' = (p -P )y a/y +(P +P -P )Y -P_ y - P Y , Y „ (6.2.6) 7 s- i a. The i n i t i a l conditions are given in (6.2.2). Our starting approximation to the optimal parameters was T (2.9, 2.5, .8, .07, .006, 1, .5), and the optimal parameters obtained were T (2.958, 2.630, .6503, .8892, -.007497, 1.108, .3108). The sum of the squares of the residuals at the above parameters was approximately 250. Graphically, the integration results at the above parameters are almost the same as those in Figure CHAPTER 6 144 6.2.7 except y a gets close to zero around t=36. Unfortunately, p^_ i s negative. We note that p^_ i s much smaller than the other six parameters and this suggests we try the logarithmic tr ansf ormation ^ =ln (p ) and e s t i m a t e d instead of p . It i s important to be sel e c t i v e when choosing a logarithmic transformation. For example, the rescaling of p to p^ by logarithmic transformations did not lead to the success described below when p alone was rescaled. Thus again we have a place for interactive experimentation. As mentioned at the end of Chapter 1, the use of logarithmic transformations also affects the conditioning of the problem. For example the condition number of the Jacobian matrix in the least sguares problem at the above parameters i s 1.1E5. If p i s scaled as suggested then the condition number becomes 1.3E4 and i f p ,...,p^ are a l l scaled by logarithmic transformations, the condition number becomes 2.6E4. With the transformed rj_ , we started the Levenberg-Marquardt procedure at the l a s t set of parameters preceding the f i r s t occurance of a negative value for p in the previous run, namely (2.88, 2.57, .728, .437, -5.17, .687, .465) T where we have renamed "p^ to be p^ _. The optimal parameter values found were (5.335, 3.011, .08038, 3.966, -6.895, 4.266, -.01545) 7" where now p^ has taken on a negative value. (The transformed values of scaled parameters are given in this discussion.) The CHAPTER 6 145 sum of the squares of the residuals was approximately 56 at the above parameters. Graphically, the integration results at the above parameters are very similar to those in Figure 6.2.7 except y curves up s l i g h t l y for t > 3 0 . The best parameters before p went negative were ' T ( 4 . 3 6 2 , 2 . 9 8 9 , . 1 3 0 7 , 2 . 6 4 4 , - 6 . 2 3 2 , 2 . 9 0 9 , . 0 1 3 0 3 ) . It appears that a logarithmic, transformation of p^ may be p r o f i t a b l e . With this transformation and starting at the above point in parameter space, we found the minimum ( 5 . 4 0 3 , 2 . 6 9 1 , . 1 0 4 2 , 3 . 5 0 7 , - 6 . 8 0 7 , 3 . 8 4 8 , -4.688)"T The sum of the squares of the residuals at these parameters was approximately 6 2 . Integration results at the above parameter values are shown in Figure 6 . 2 . 7 . From a numerical point of view, the f i t in Figure 6.2.7 appears excellent; however, from a b i o l o g i c a l point of view i t has a flaw. The f i r s t state variable gets caught at zero and y A goes to an equilibrium. B i o l o g i c a l l y , we would l i k e the solution to the i n i t i a l value problem to o s c i l l a t e in time. Numerically, i t appears that more data points should be available i f we want to look for o s c i l l a t o r y behavior. The remarkable improvement achieved when a logarithmic transformation was used on p^ _ suggests that we go back to the f i r s t two models and try this strategy. Starting with the ' parameters obtained i n t e r a c t i v e l y , t h i s strategy did not change the results with the f i r s t model; however, starting at the CHAPTER 6 146 UJ o CZ 10. CO Vf X \ V , y x x x x \ ^ x ~ x ^ - x 2 1 - - — " \ x x x x x * x x T 1 1 : 1 r-° -D 0 5 - C 0 lO.GE jS.DO 20.00 25.00 30.00 3S.no 40 00 ~ 4 5 00 TIME F i g u r e 6 . 2 . 7 " Optimum with l o g a r i t h m i c s c a l i n g on p^. and p_ parameters obtanied i n t e r a c t i v e l y and with p =p , t h i s s t r a t e g y worked very w e l l on the second model where two death r a t e s were used. As with the t h i r d model, p^ became negative and small and a l o g a r i t m i c t r a n s f o r m a t i o n of p was a l s o used. The f i n a l ? parameters obtained were (5.416, 2.350, 3.832, 3.394, -7.037, 3.822, -5.062)1" and the sum of the squares of the r e s i d u a l s and i n t e g r a t i o n r e s u l t s at the above parameters were e s s e n t i a l l y the same as those of Figure 6.2.7. I t i s i n t e r e s t i n g to note that f o r the parameter vector (4.205, 2.586, 2.905, 2.562, -6.636, 2.896, -5.298)T" the sum of the squares of the r e s i d u a l s was approximately 54000. CHAPTER 6 147 This sort of behavior i s t y p i c a l of i n i t i a l value problems and i s one reason why so many d i f f i c u l t i e s are encountered when f i t t i n g parameters in dynamic models. F i n a l l y , we comment that our investigation has been confined to numerical parameter f i t t i n g and no attempt has been made to interpret the parameter values in a physical sense. Before any parameters can be accepted, they must of course be p h y s i c a l l y reasonable. 6.3 A MODEL INVOLVING A CHANGE IN EQUILIBRIUM The model considered in t h i s section represents an early attempt to describe data collected from Lake Pl a c i d , B r i t i s h Columbia [75]. The observations are on phytoplankton in the lake. Throughout much of the summer, their t o t a l mass remains f a i r l y constant at a r e l a t i v e l y low l e v e l . Then, within the space of a few days, i t jumps to a much higher l e v e l . The next few observations contain a l o t of noise, but i t appears that the l e v e l of phytoplankton remains high u n t i l near the end of the year when i t drops back to a low l e v e l . The observations on the phytoplankton are shown in Figure 6.3.1. The units for the time axis are in days.and time t=0 corresponds to the beginning of May. One unit along the state variable axis in Figure 6.3.1 corresponds to 1/75 milligram of phytoplankton per l i t r e of water. It i s postulated that the sudden jump in the observations can be described by a dynamic model which loses a lower equilibrium and moves to a higher equilibrium. In p a r t i c u l a r we consider the dynamic model (proposed by Dr. C.J. Walters, Ins t i t u t e of Animal Resource Ecology, University of CHAPTER 6 148 F i g u r e 6.3.1 P h y t o p l a n k t o n o b s e r v a t i o n s and smoothing f u n c t i o n B r i t i s h Columbia) y' =p r (t) y ( l - y / 2 5 ) -p z (t) y 2 / (p/+y a ) (6.3.1) y(0)=3.2. I n t h i s model, y r e p r e s e n t s the d e n s i t y o f p h y t o p l a n k t o n , and z ( t ) r e p r e s e n t s the d e n s i t y o f the p r e d a t o r z o o p l a n k t o n . The f u n c t i o n r ( t ) r e p r e s e n t s the e f f e c t o f s u n l i g h t on the growth r a t e o f the p h y t o p l a n k t o n . The term r ( t ) p ; ( l - y / 2 5 ) d e s c r i b e s the growth o f the p h y t o p l a n k t o n , and the term - z ( t ) p y^/fp^+y 3) d e s c r i b e s the f e e d i n g e f f e c t o f the z o o p l a n k t o n . To determine the e q u i l i b r i a of the above dynamic model, we s e t y'=0, and s o l v e f o r y. Thus f o r y^O, the e q u i l i b r i a occur a t the r o o t s o f CHAPTER 6 149 P / r ( t ) (l-y/25) (p*+ya) -p 3 z (t) y=0 (6.3.2) Depending on p , p , p , r ( t ) , and z ( t ) , the above algebraic equation in y can have one, two, or three real roots. Thus i t is t h e o r e t i c a l l y possible to gain or lose an equilibrium when (6.3.1) i s integrated. For r(t) we take the function r(t)=exp(-((t-110)/55) a). (6.3.3) The function z(t) was obtained from physical observations. To produce a continuous approximation to z ( t ) , we used a least squares piecewise cubic Hermite approximation to the observations. The j o i n t s used in t h i s approximation were at t=60, 120, 180. The data points and the continuous approximation to z(t) are shown in Figure 6.3.2. The observations have been scaled so that the maximum ordinate i s 1. We take P C * =d, i , . s r for our i n i t i a l parameter estimate. The success in Section 3.3 of the integral f i t t i n g technique for this type of problem suggests we st a r t with this method. F i r s t we smoothed the observations with a least squares piecewise cubic Hermite polynomial with j o i n t s at t=110, 115, 140, 180. This smoothing function i s shown along with the observations in Figure 6.3.1. With this smoothing, the IFIT method produced the parameters (.1433, 5.509, .7300)7" CHAPTER 6 150 \ X O.OQ 80.00 160.00 200.00 2+3.00 Figure 6.3.2 Zooplankton densities Integration results at the above parameter vector are shown in Figure 6.3.3. Next we refined these parameters using the Levenberg-Marquardt technique and the s e n s i t i v i t y equations. This approach found the optimum (.1562, 3.336, .8963)'R where the sum of the squares of the residuals was approximately 556. Integration results at the above parameters are shown in Figure 6.3.4. A di r e c t approach using the s e n s i t i v i t y equations and s t a r t i n g at p ^ defined above also produced this r e s u l t . In Table 6.3.1, we l i s t some roots of (6.3.2) as a function of time at the above parameters. The results in thi s table indicate that at the star t of the time in t e r v a l under consideration there i s one r e l a t i v e l y low equilibrium. Later on CHAPTER 6 151 d ! 0 -r— -* t o I O.00 + + + + + + + —I 80. CO + + + + 0.00 120.00 TIME -1 160.00 Figure 6.3.3 Integration at IFIT results — i 200.00 -1 240.00 r = a . O l i o - | + + + + + + + + + t - 1 O.00 -1 80.00 —1 160.00 + + 0.00 120.00 TIME 2B0.00 Figure 6.3.4 Integration results at IFIT+FIT results i 240.00 CHAPTER 6 152 Time Number of real roots Real roots 1 1 1 3 3 1 1 3 3 1 1 0.07464 84 85 89 90 151 152 156 157 226 2.093 2.216 3.130 16.95 17.14 3.260 2.134 1.994 0.1436 5.142 10.83 9.331 5.395 13.45 16.47 16.60 12.03 Table 6.3.1 Roots of J_6_.3_-.2_L corresponding to Figure 6.3.4 we have three e q u i l i b r i a , and s t i l l later we have only one r e l a t i v e l y high equilibrium. S t i l l later in the time i n t e r v a l , we again acquire three e q u l i l i b r i a . At the end of the time i n t e r v a l , we are back to only one lower equilibrium. However, no rapid jumps to new e q u i l i b r i a are evident in Figure 6.3.4. In the results which follow, a more pronounced jump to a higher equilibrium was obtained. The above parameter vector corresponding to Figure 6.3.4 i s not the only point in parameter space where the solution to (6.3.1) f i t s the observations well. Starting at p defined above, and following an interactive strategy of freezing parameters, optimizing on subspaces, and resetting parameters, we arrived at the point (.9900, .7284, 2.796) r in parameter space. The sum of the squares of the residuals at the above l o c a l minimum was approximately 662. Integration CHAPTER 6 153 results at the above parameters are shown in Figure 6.3.5, Figure 6.3.5 Integration at i n t e r a c t i v e l y obtained optimum (Integrations under error control were employed near the above point in parameter space.) This point was d i f f i c u l t to find and i t i s unlikely that an automatic approach would have much luck in finding i t . The d i f f i c u l t y seems to stem from the nature of the least squares surface near the above point in parameter space. For example i f we change p_^ to .7 then the integration results become e s s e n t i a l l y zero for the f u l l time i n t e r v a l under consideration, and the sum of the squares of the residuals becomes 3085. Integration results corresponding to this sum of squares are shown in Figure 6.3.6. Starting from the parameters of Figure 6.3.6, the FIT approach again produced the optimum i l l u s t r a t e d in Figure 6.3.4. CHAPTER 6 154 In Table 6 . 3 . 2 , we l i s t some of the roots of ( 6 . 3 . 2 ) as a function of time at the parameters of Figure 6 . 3 . 5 . At the U J = r -CLIO-t— -« CO + + , + + + + + + + + + + + t + + + + + + + + + + + + + + + 1 11 i11 " 1 -==-=5 1 , 0.00 45.00 eo.00 120.00 160.00 200.00 240.00 TIME Figure 6.3.6 Integration near i n t e r a c t i v e l y obtained optimum Time Number of real roots Real roots 1 1 0 . 0 0 7 2 4 4 42 1 0 . 0 8 4 4 3 43 3 0 . 0 8 7 3 0 1 0 . 6 6 1 4 . 2 5 112 3 0 . 6 5 1 1 0 . 8 6 7 6 2 3 . 4 8 113 1 2 3 . 5 2 12 9 1 2 3 . 5 1 130 3 0. 6 3 1 6 0.8947 2 3 . 4 7 176 3 0 . 0 8 5 4 8 1 2 . 3 4 1 2 . 5 7 177 1 0 . 0 8 3 6 7 226 1 0 . 0 1 3 9 6 Table 6.3.2 Roots of J(j6.3_.2)_ corresponding to Figure 6.3.5 CHAPTER 6 155 sta r t of the time i n t e r v a l , we have only a r e l a t i v e l y low equilibrium. Around t=43, we acquire three e q u i l i b r i a , and around t=113, the lower two of these three e q u i l i b r i a vanish and we are l e f t with only a r e l a t i v e l y high equilibrium. Later, we again acquire three e q u i l i b r i a , and around t=177, the higher two of these three vanish and we are l e f t with a r e l a t i v e l y low equilibrium which remains u n t i l the end of the time i n t e r v a l . The disappearance of the lower two e q u i l i b r i a around t=113 corresponds to the rapid increase in the solution to (6.3.1) shown in Figure 6.3.5. The large error in the observations makes i t d i f f i c u l t to choose between the two solutions obtained for this problem; however, q u a l i t a t i v e l y the l a t t e r solution i s more pleasing. In conclusion, the inte r a c t i v e approach has provided us with a solution which q u a l i t a t i v e l y behaves in the desired fashion. It is now the task of the model builder to interpret and perhaps build on these r e s u l t s . When interpreting the parameters corresponding to to Figure 6.3.5, the model builder must of course take into account the dr a s t i c change possible in the sum of the squares of the residuals due to a r e l a t i v e l y small change in parameter space. 6.4 A REINDEER POPULATION GROWTH MODEL The model presented below represents an attempt to describe the reindeer population in Alaska from the year 1891 to the year 1963 [2]. The Bazykin predator-prey model discussed in Chapter 4 i s employed. The reindeer correspond to the predators and the CHAPTER 6 156 forage corresponds to the prey. Let y^ represent forage in units of 100 tons per square mile, and l e t y^ represent the reindeer population density in units of animals per square mile. We consider the dynamic model (proposed by Dr. C.J. Walters, Institute of Animal Resource Ecology, University of B r i t i s h Columbia) y'=p y (1-cy )-(r/100)y y /(p +y, ) ' ' ' ' 4 .* ' (6.4.1) K=pzry, Y*/ipz+Y< )_P«Y* where p^ represents the growth rate of the forage for small y^ in the absence of grazing, 1/c represents the equilibrium for the forage in the absence of grazing, r represents the reindeer feeding rate, p and p are measures of the reindeer feeding e f f i c i e n c y , and p^ represents the reindeer death rate. Of course a l l parameters should be p o s i t i v e . Our i n i t i a l conditions are yf (0)=10, y a(0)=.001 where t=0 corresponds to the year 1891. The i n i t i a l condition on y represents 1000 tons of forage per square mile. We take the constant c=.l. That i s , the equilibrium for the forage in the absence of grazing is 1000 tons per square mile, the i n i t i a l condition on y^ . The reindeer population density observations are derived from population counts over an area of approximately 20000 square miles. Thus the i n i t i a l condition on y represents very few (approximately 16) reindeer in this area. We f i x r at 2 (in units of tons per year) for the reindeer in this area. CHAPTER 6 157 Our i n i t i a l approximation to the parameters was p ( 0 ) =(.3, 1, .1, .06)T In Figure 6.4.1, the observations on y are shown along with integration results at the above parameter vector. There are no TC. UJ c— cr k g 1X5 . X x X X X XX XX x x X X X X X X X** X X ; w w o t f < < w ^ , , J ^ f ^ -I 1 1 1 0.00 8.00 16.00 24.00 32.00 40.00 48.00 55.00 64.00 72.00 TIME Figure 6.4.1 . Observations and integration results at p observations on the forage, and the i n i t i a l condition on y represents a rough guess. In the following discussion, we demonstrate through a set of experiments, the power of an interactive approach on this f a i r l y d i f f i c u l t problem. EXPERIMENT 1 It seems worthwhile to begin with a di r e c t attack on the problem. If this strategy succeeds, then we are finished, and CHAPTER 6 158 i f i t does not , we can try a more sophisticated strategy. Even i f this d i r e c t attempt does not work, i t may suggest further experiments. Starting at the above p ^ , the FIT approach produced the optimal parameters (-.2691, 24.82, .8093, .1763)!" The sum of the squares of the residuals at this mimimum was approximately 494. Integration results at these parameters are shown in Figure 6.4.2. The results look excellent graphically, T I M E Figure 6.4.2 Integration results for Experiment .1 but p^ i s negative and p ^ i s much too large. The 95% confidence intervals , as defined in Chapter 1, were (+4.041, +3788, +83.11, +.1385)T" The parameter p^ has strayed into a region where the least CHAPTER 6 159 squares surface i s very insens i t i v e to i t . The negative sign of p^ suggests we constrain i t to be pos i t i v e ; however, this strategy st a r t i n g at p ( 0 > did not prove e f f e c t i v e . (The parameter p/ became small, but the integration results remained e s s e n t i a l l y the same as those in Figure 6.4.1.) Freezing p/ at to) .3 and star t i n g at p did not help either; however, the next experiment was successful. EXPERIMENT 2 In this experiment, p^ was frozen at 1 and the optimization (#) was started at p defined above. This strategy found an optimum at -r (-.2667, 1., .2250, .1366) . The sum of the squares of the residuals at these parameters was approximately 757. Graphically, the solution appears to model the observations quite well; however, p^ i s negative. Our next experiment produced q u a l i t a t i v e l y promising behavior with a l l parameters p o s i t i v e . EXPERIMENT 3 In th i s experiment, p^ and p^ were frozen at .3 and 1 respectively, and we l e t p and p. star t at the values obtained 3 * in the previous experiment. Optimizing on the resulting two dimensional subspace of parameter space produced the parameters (.3, 1, .9704, 1.488)T CHAPTER 6 160 The sum of the squares of the residuals at the above parameters was approximately 20,000. Integration results at the above parameters are shown in Figure 6.4.3. In spite of the large sum o o ) a .00 Figure 6.4.3 Integration at results of Experiment 3 of squares, the results look promising. The model equations suggest that by reducing p and p , a much better result should be possible; however, interactive experiments along these l i n e s produced i n s t a b i l i t i e s very e a s i l y . We improve on the above results in the next experiment. EXPERIMENT 4 Starting at the parameters obtained in the previous experiment, and not employing any freezing, we found an optimum at the point CHAPTER 6 161 r (.04046, 14.61, 3.635, 2.669) in parameter space. The sum of the squares of the residuals at this point was approximately 490. Integration results at these parameters are shown in Figure 6.4.4. The parameters are now X^ x x x ^ x ^ ^ ^ X 1 1 1 1 1 1 0.00 3.00 16.00 24.00 32.00 40.00 46.00 56.00 64.00 72.00 T I M E Figure 6.4.4 Integration results at optimum of Experiment __ a l l positive and the integration results at these parameters f i t the data quite well. However, p^ i s again much too large, as are p_ and p^, and Y, does not appear to be as active as i t should be. It appears that the absence of observations on y j leaves too much f l e x i b i l i t y in the model. EXPERIMENT 5 Our l a s t experiment in this section i s with guessed observations and iterated integral f i t t i n g methods. The CHAPTER 6 162 smoothing of the o b s e r v a t i o n s was a c c o m p l i s h e d w i t h a l e a s t s q u ares c u b i c s p l i n e u s i n g j o i n t s a t t=5, 10, 21, 23, 42, 57, 59. The i n i t i a l guessed o b s e r v a t i o n s f o r the f o r a g e were p o i n t s on the c u b i c s p l i n e t h a t i n t e r p o l a t e s (0,10), (10,10), (20,9), (30,6), (50,3), (60,2), (72,1). The end c o n d i t i o n s f o r t h i s i n t e r p o l a t i o n a re d e s c r i b e d under the CREOBS command i n the PARFIT documentation i n Appendix A. The i t e r a t e d i n t e g r a l f i t t i n g approach (employing subsystem i n t e g r a t i o n s ) worked w e l l ; however, i t drew us t o the parameters found i n Experiment 1. F r e e z i n g p a and u s i n g the above guessed o b s e r v a t i o n s produced a n e g a t i v e p / on the f i r s t i t e r a t i o n , and f u r t h e r i t e r a t i o n s d i d not c o r r e c t t h i s s i t u a t i o n . Next, p^ and p^ were f r o z e n a t .3 and 1 r e s p e c t i v e l y . The i t e r a t e d i n t e g r a l f i t t i n g approach ( u s i n g (3.4.1)) produced the parameters c o r r e s p o n d i n g t o F i g u r e 6.4.3. W h i l e the methods employed i n t h i s experiment d i d not p r o v i d e any new parameter e s t i m a t e s , they a t l e a s t demonstrated the s t r o n g p r e f e r e n c e t h a t e x i s t s i n the model f o r the e s t i m a t e s o b t a i n e d i n the p r e v i o u s e x p e r i m e n t s . The above e x p e r i m e n t s are a d i s t i l l a t i o n o f a few f a i r l y s h o r t i n t e r a c t i v e s e s s i o n s . To g a i n comparable r e s u l t s w i t h a n o n i n t e r a c t i v e approach would take a good d e a l l o n g e r and would demand a l o t o f p a t i e n c e and d e t e r m i n a t i o n on the p a r t of the user . Much more c o u l d be done w i t h t h i s problem. For example, we CHAPTER 6 163 could search for other l o c a l minima. We could also t r y more parameters or d i f f e r e n t combinations of parameters. (For example, we could f i x one of the above parameters and l e t r be a parameter.) Our main goal, however, i s to gain experience with PARFIT on a variety of problems. Thus we turn to our next example, a problem involving an ocean plankton model. 6.5 AN OCEAN PLANKTON MODEL Our f i n a l example in this chapter involves a model of the stages in the l i f e cycle of certain ocean zooplankton. We consider a s i m p l i f i e d model where only three stages in the l i f e cycle are represented by state variables. Let y , y , and y represent the population densities of these three stages in units of population per cubic metre of sea water. The adult population density i s represented by Y3 - Physically, there are death rates p , p , and p„ associated with y , y , and y respectively, and there i s a transfer rate p^ from y to y^, and a transfer rate p^ _ from y^ to y^ . We drive the system with a function x(t) which represents the population density of the stage in the l i f e cycle before that represented by y^ . Let p^ represent the transfer rate from x(t) to y^ . Our dynamic model is thus ( [54]) y'=-(p +p )y +p x(t) y'=pv y, -(P^ +P f f-)y a (6.5.D y'=p y -p y 3 fT A. 3 3 We note that this model i s l i n e a r . Observations on a l l state CHAPTER 6 164 variables and on x(t) were taken using large p l a s t i c bags (suspended from the ocean surface) which were designed to is o l a t e samples of ocean water[16]. The observations are shown in Figures 6.5.1, 6.5.2, and 6.5.3. For these three graphs, time i s measures in days, observations on y are symbolized by , and integration results for y^ are represented by an unbroken l i n e . The function x(t) was approximated by 22.15+8.036t -.309.91? +.003156t5 . (6.5.2) The i n i t i a l conditions were y(l)=(.407, .271, .291)T and our s t a r t i n g parameter vector was p ( o ) =(.03, .02, .01, .1, .2, .06)T A l l parameters should remain between Q and 1. A d i r e c t approach (FIT method) found the point (.1256, -.1121 , .1626, .06709, .2219, .07167)7" in parameter space. The sum of the squares of the residuals at the above parameters was approximately 1690. Unfortunately p_ a. i s negative. We note that no parameters have exceeded 1 and this i s encouraging. Next we constrained p_ with a square root s c a l i n g . With this transformation, the FIT method found an optimum at (unsealed) r (.09615, 1.160E-6, .1087, .09013, .1523, .06958) CHAPTER 6 165 when i t was started from p *" defined above. The sum of the squares of the residuals at the above parameters was approximately 1710. A l l the parameters are now between 0 and 1. Integration results at the above parameters are shown along with the observations in Figures 6.5.1, 6.5.2, and 6.5.3. Numerically this was a rather easy problem compared with the other three examples considered in this chapter. V i s u a l l y , there appear to be peaks in the observations which could be approximated better, and this i s a subject for further study. o o + + ; H— , , , , , 1 ! OJJD 6.00 16.00 24.00 32.00 40.00 48.00 56.00 64.00 T I M E Figure 6.5.1 Observations and best f i t for y CHAPTER 6 166 t - ° ( X l O -r — — C D X X o.oo 8.DO 16.00 21.00 32.00 TIME . I 40.00 48.00 55.00 64.00 Figure 6.5.2 Observations and best f i t for y_t CHAPTER 6 167 CHAPTER 7 CONCLUSIONS AND FUTURE WORK 7.1 CONCLUSIONS Our goal was to develop techniques designed to overcome poor i n i t i a l approximations to the optimal parameters. The necessity for such technigues became evident with experiments (such as those of Section 4.2) using the d i r e c t approach employing the s e n s i t i v i t y equations. The basic conclusion from such experiments was that the d i r e c t approach often produced a highly nonlinear problem where the ef f e c t of small parameter changes could be dramatic. Thus we turned to such methods as derivative and integral f i t t i n g . As shown in Section 4.2, these approaches are very useful. It also became evident that to resolve a problem often required several runs employing d i f f e r e n t complementary procedures. Thus an interactive approach appeared to be a good way to proceed. The case when observations are not available on a l l state variables arises often in practice, and experiments indicated that this s i t u a t i o n can be very d i f f i c u l t i f approached d i r e c t l y with poor i n i t i a l parameter estimates. Thus we sought an extension of the derivative and integral f i t t i n g techniques that could handle th i s problem. As a result we developed the e f f e c t i v e approach of guessing the desired behavior of the unobserved states and then i t e r a t i v e l y improving this guess. As shown in Section 4.5, this approach can be very e f f e c t i v e when CHAPTER 7 168 the d i r e c t approach presents a l l sorts of d i f f i c u l t i e s . In keeping with our goal of developing tools for overcoming (at least in part) the d i f f i c u l t i e s associated with a d i r e c t approach, we investigated b r i e f l y in Chapter 5 the value of sequentially processing the observations. We conclude that this strategy can be superior to a d i r e c t attack on the problem. The main trouble with a coarse method i s that i t can produce parameters from which the dire c t approach i s s t i l l incapable of succeeding. For example the i n i t i a l value problem may be unstable at the parameters produced by the coarse method. In Section 3.5, we proposed methods employing continuation parameters and break points to bridge the gap between the coarse approach and the f u l l nonlinear problem. We conclude from the example of Section 3.5 that this approach can be e f f e c t i v e on problems involving an i n s t a b i l i t y . Our approach to the use of break points d i f f e r s from that of van Domselaar and Hemker[71], and we are unaware of any work connected with parameter estimation in dynamic models that involves the use of break points along with continuation parameters. We conclude from our experience on real world problems that with an in t e r a c t i v e approach i t i s possible, in a r e l a t i v e l y short time, to make substantial progress on these f a i r l y d i f f i c u l t problems. However, to be e f f e c t i v e such an interactive package must be well organized and have a f a i r l y wide range of options available. The implementation of such a package i s an evolutionary process. Our package PARFIT CHAPTER 7 169 represents the i n i t i a l stage in this process. 7.2 SUGGESTIONS FOR FUTURE WORK The iterated integral and derivative f i t t i n g methods developed in Chapter 3 appear to be worthy of further study. For example in the experiments presented in Section 4.4, the iterated integral f i t t i n g method using subsystem integrations worked much better than the iterated integral and derivative f i t t i n g methods employing (3.4.1). The reason behind this seems worthy of further study. Hopefully, such a study may lead to other e f f e c t i v e ways of updating the guessed observations. Other promising avenues of further research involve the use of continuation parameters and break points, and the use of sequential techniques for improving sta r t i n g parameters. F i n a l l y there i s the development of new and improved versions of a package such as PARFIT. The value of good software when approaching the sort of problems considered in this thesis cannot be over emphasized. CHAPTER 7 170 BIBLIOGRAPHY [1] Aaro, I., Design and implementation of a tool for interactive communication between user and program, Dept. of Info. P r o c , Roy. Inst. of Tech., Stockholm, Sweden, TRITA-NA-7705 (1977) [2] Aaro, I., Design and implementation of a software system for i n t e r a c t i v e s c i e n t i f i c computing, Dept. of Info. P r o c , Roy. Inst. of Tech., Stockholm, Sweden, TRITA-NA-7707 (1977) [3] Albert, A.E. and Gardner, L.A., Stochastic Approximation and Nonlinear Regression, M.I.T. Press, Cambridge, Mass. (1967). [4] A v i l a , J . , Continuation methods for nonlinear equations, SLAM J . Numer. Anal., 11 (1974) 102-122. [5] Bard, Y., Comparison of gradient methods for the solution of nonlinear parameter estimation problems, SIAM J. Numer. Anal. 7 (1970) 157-186. [6] Bard, Y., Nonlinear Parameter Estimation, Academic Press, New York (1974). [7] Bazykin, A.D., Volterra's system and the Michaelis-Menton equation in Problems in Mathematical Genetics, Ed. by V.A. Ratner, U.S.S.R. Acad. S c i . , Novosibrisk (1974) 103-142. (Available in English as Structural and dynamic s t a b i l i t y of model predator-prey systems (1975), Institute of Resource Ecology, University of B r i t i s h Columbia, Vancouver, B.C., Report R-3-R.) [8] Beale, E.M.L., Confidence regions in nonlinear estimation, J. Rjoy. Stat. S o c 22 (1960) 41-88. [9] Bellman, R., Jacquez, J . , Kalba, R., and Schwimmer, S., Quasilinearization and the estimation of chemical rate constants from raw kinetic data, Math. B i o s c i . 1 (1967) 71-76. [10] Bellman, R., Kagiwada, H., Kalaba, R., and Vasudevan, R., Quasi l i n e a r i z a t i o n and the estimation of d i f f e r e n t i a l operators from eigenvalues, Comm. ACM 11 (1968) 255-256. [11] Bellman, R., Kashef, B., and Vasudevan, R., The inverse problem of estimating heart parameters from cardiograms, Math. B i o s c i . 19 (1974) 221-230. B I B L I O G R A P H Y 171 [12] Berman, M., Weiss, M.F. and Shahn, E. , Some formal approaches to the analysis of kinetic data in terms of linear compartmental systems, Biophys. J. 2 (1962) 289-316. [13] Boggs, P.T. and Dennis, J.E. J r . , A s t a b i l i t y analysis for perturbed nonlinear i t e r a t i v e methods, Math. of Comp. 30 (1976) 199-215. [14] Bus, J.C.P., van Domselaar, B., and Kok, J . , Nonlinear least squares estimation, Math. Cent. Amsterdam, NW 17/75 (1975). [15] Byrne, G.D. and H a l l , CA. (Ed.) Njjmerical Solution of Systems of Nonlinear Algebraic Equations, Academic New York (1973) . Press, [16] CEPEX-Controlled Ecosystem Po l l u t i o n Experiment, P a t r i c i a Bay, Sannich, B r i t i s h Columbia, Canada, unpublished working paper (1976). [17] C u r t i s , A.R., and Edsberg, L. , Some investigations into data requirements for rate constant estimation, A.E.R.E., Harwell, HL. 73/3744 (1973). [18] Dennis, J.E. J r . , Some computational techniques for the nonlinear least squares problem, in Numerical Solution of Systems of Nonlinear Algebraic Equations Ed. by G.D. Byrne and CA. H a l l , Academic Press, New York (1973) 157-183. [19] [20] [21] [22] Dennis, J.E. J r . , Nonlinear least squares and equations, A.E.R.E. Harwell, Oxfordshire, CSS 32 (1976). Dennis, J.E. J r . , Gay, D.M., and Welsch, R.E., An adaptive nonlinear least squares algorithm, National Bureau of Economic Research, Cambridge, Mass. prelim, rep. (1977) . Edsberg, L., KEMPEX-II, A program package for interactive simulation of some chemical reactors. Dept. of Info. P r o c , Roy. Inst. of Tech., Stockholm, Sweden, TRITA-NA-7504 (1975) . Farrow, L. and approximation: fact VI (1974) 787-800. Edelson, D., The or f i c t i o n , Int. J. steady-state of Chem. Kin. [23] Ficken, F.A., The continuation methods for functional equations, Comm. Pure Appl. Math. 4 (1951) 435-456, Math. Rev. 13 (1952) 562-563. BIBLIOGRAPHY 172 [24] Fletcher, R., Generalized inverse methods for the least squares solution of systems of nonlinear equations, The Comput. J . , 10 (1968) 392-399. [25] Gear, C.W., Numerical I n i t i a l Value Problems in Ordinary D i f f e r e n t i a l Equations, Prentice-Hall, Inc., Englewood C l i f f s , N.J. (1971). [26] Gear, C.W., The automatic integration of ordinary d i f f e r e n t i a l equations, Comm. ACM 14 (1971) 176-179. [27] Gear, C.W., DIFSUB for solution of ordinary d i f f e r e n t i a l equations, Algorithm 407, Comm. ACM 14 (1971) 185-190. [28] Gelb, A. (Ed.), Applied Optimal Estimation, The M.I.T. Press, Cambridge, Mass. (1974). [29] G i l l , P.E., Golub, G.H., Murray, W. and Saunders, M.A., Methods for modifying matrix f a c t o r i z a t i o n s , Comp. S c i . Dept., Stanford, U., STAN-CS-72-322 (1972). [30] G i l l , P.E. and Murray, W., (Ed.) Numerical Methods for Constrained Optimization, Academic Press (1974). [31] Goldstein, A.A. and Price, J.F., An ef f e c t i v e algorithm for minimization, Numer. Math. 10 (1967) 184-189. [32] Golub, G.H., and Reinsch, C , Singular value decomposition and least squares solutions - Handbook series in linear algebra, Numer. Math. 14 (1970) 403-420. [33] Ho, Yu Chi, On the stochastic approximation method and optimal f i l t e r i n g theory, J. Math. Anal. and Appl. 6 (1962) 152-154. [34] Jones, A., SPIRAL-a new algorithm for non-linear parameter estimation using least squares. The Comput. J. 13 (1970) 301-308. [35] Kelley, H.J. and Denham, W.F., Modeling and adjoints for continuous systems, JOTA 3 (1969) 174-183. [36] Kowalik, J. and Osborne, M.R., Methods for Unconstrained Optimization Problems, American Else v i e r , New York (1968). [37] Krebs, C.J., Gainer, M.S., Kel l e r , B.L., Myers, J.H. and Tamarin, R.H., Population cycles in small rodents, Science 179 (1973) 35-41. BIBLIOGRAPHY 173 [38] Lawson, C.L. and Hanson, R.T., Solving Least Squares Problems, Prentice-Hall, Inc., Englewood C l i f f s , N.J. (1974) . [39] Levenberg, K. , A method for the solution of certain non-linear problems in least squares, Quart. Appl. Math. 2 (1944) 164-168. [40] Long, C.E., Model s t a b i l i t y , r e s i l i e n c e , and management of an aquatic community, Oecologia (Berl.) 17 (1974) 65-85. [41] Lotka, A., Elements of Mathematical Biology, Dover, Pub. Inc., New York (1956), republication of Elements of Physical Biology, Wilkins Co. Inc. (1924). [42] Mackey, M., O s c i l l a t i o n and chaos in physiological control systems, to appear in Science. [43] Marquardt, D.W., An algorithm for least squares estimation of nonlinear parameters, SIAM J. 11 (1963) 431-441. [44] Marten, G.G., Kleiber, P.M., and Reid, J.A.K., A computer program for f i t t i n g tracer k i n e t i c and other d i f f e r e n t i a l equations to data, Ecology 56 (1975) 752-754. [45] McConalogue, D.J., A q u a s i - i n t r i n s i c scheme for passing a smooth curve through a discrete set of points, The Comput. J. 13 (1970) 392-396. [46] McKeown, J.J., Specialised versus general-purpose algorithms for minimising functions that are sums of squared terms, Math. Prog. 9 (1975) 57-68. [47] Murray, W. (Ed.), Numerical Methods for Unconstrained Optimization, Academic Press, London (1972). [48] Myers, J.H. and Krebs, C.J., Population cycles in rodents, S c i e n t i f i c American, June (1974) 38-46. [49] Nordsieck, A., On the numerical integration of ordinary d i f f e r e n t i a l equations, Math. Comp. 16 (1962) 22-49. [50] Ortega, J.M. and Rheinboldt, W.C., Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York (1970). [51] Osborne, M.R., Some aspects of non-linear least squares ca l c u l a t i o n s , in Numer i c a l Methods for Non-linear Optimization, Ed. by F.A. Lootsma, Academic Press, London (1972) 171-189. BIBLIOGRAPHY 174 [52] Osborne, M.R., A class of methods for minimising a sum of squares, The Aust. Comp. J . 4 (1972) 164-169. [53] Parker, R.A., The influence of environmental driving variables on the dynamics of an aquatic ecosystem model, Verh. Internat. Verein. Limnol. 19 (1975) 47-55. [54] Parslow, J . and Sonntag, N. , Parameter estimation of the natural population, Institute of Oceanography, University of B r i t i s h Columbia, Vancouver, B.C., (in preparation). [55] Pearson, J.B., On nonlinear least squares f i l t e r i n g , Automatica, 4 (1967) 97-105. [56] R a i l , L.B., Davidenko's method for the solution of nonlinear operator equations, The Univ. of Wisconsin, Math. Res. Cent., MRC Tech. Summary Rep. 948 (1968). [57] Ramsin, H. and Wedin, P.-A*., A comparison of some algorithms for the nonlinear least squares problem, BIT 17(1977) 72-90. [58] Rheinboldt, W.C., On the solution of some nonlinear equations a r i s i n g in the application of f i n i t e element methods, U. of Maryland Tech. Rep. TR-362 (1975). [59] Rheinboldt, W.C., Numerical continuation methods for f i n i t e element applications, U. of Maryland Tech. Rep. TR-454 (1976), to appear in Formulation and Computational Algor ithms in F i n i t e Element Analysis, Proc. of U.S.-German Symp., MIT Press (1976) [60] Roberts, S.M. and Shipman, J.S., Two Point Boundary Value Problems: Shooting Methods, American Els e v i e r , New York (1972) . [61] Rosenbrock, H.H. and Storey, C , Computaional Techniques for Chemical Engineers, Pergamon Press, Oxford (1966) 189-208. [62] Skoog, R.O., Ecology of the Caribou (rangifer tarandus granti) in Alaska, Ph.D. Thesis, Univ. of C a l i f o r n i a , Berkely (1968). [63] Smith, L.B., The use of man-machine interaction in data f i t t i n g problems, Stanford U. , Comp. S c i . Dept. Rep. CS 131 (1969). [64] Steen, N.M. and Byrne, G.D., The problem of minimizing nonlinear functionals-I. Least squares, in Numer i c a l Solution of Systems of Nonlinear Algebraic Equations Ed. BIBLIOGRAPHY 175 by G.D. Byrne and C.A. H a l l , Academic Press, New York, (1973) 185-239. [65] Stewart, G.W., Introduction to Matr ix Computations, Academic Press, New York (1973). [66] Swartz, J. and Bremermann, H., Discussion of parameter estimation in b i o l o g i c a l modelling: algorithms for estimation and evaluation of the estimates, J. Math. Bio. 1 (1975) 241-257. [67] Tomovic", R. , S e n s i t i v i t y Analysis of Dynamic Systems, McGraw-Hill, New York, Trans. by D. Tornguist. (1963). [68] Tomovi£, R. and Vukobratovie", M., General S e n s i t i v i t y Theory, American Els e v i e r , Inc., New York (1972). [69] Vandermeer, J.H., The competitive structure of communities: an experimental approach with protozoa, Ecology 50 (1969) 362-371. [70] van Domselaar, B., A mathematical analysis of the hea r t - i n f a r c t (Dutch), Math. Cent., Amsterdam NN 4/74 (1974). [71] van Domselaar, B. and Hemker, P.W., Nonlinear parameter estimation in i n i t i a l value problems, Math. Cent., Amsterdam NW 18/75 (1975). [72] Van Loan, C , Lectures in least squares, Dept of Comp. S c i . , Cornell U., Ithaca, N.Y. TR 76-279 (1976). [73] Varga, R.S., Matrix Iterative Analysis, Prentice-Hall, Inc., Englewood C l i f f s , N.J. (1962). [74] Walters, C.J., and Hilborn, R., Adaptive control of f i s h i n g systems, J. Fish. Res. Board Can., 33 (1976) 145-159. [75] Walters, C.J., Institute of Animal Resource Ecology, University of B r i t i s h Columbia, Vancouver, B.C., personal communication. [76] Young, P., Recursive approaches to time series analysis, Inst. Math. Appl. 10 (1974) 209-224. BIBLIOGRAPHY 176 APPENDIX A PARFIT DOCUMENTATION Our int e r a c t i v e parameter f i t t i n g program, PARFIT, serves several purposes. It establishes an environment in which new algorithms can be tested. It i s an experimental tool for investigating the u t i l i t y of various commands for interactive parameter f i t t i n g in dynamic models. It i s also a device for studying the organization of an interactive program of this nature. F i n a l l y , i t i s a p r a c t i c a l tool for f i t t i n g parameters in dynamic models. PARFIT i s structured so that the various numerical algorithms can e a s i l y be extracted into individual procedures (see Chapter 3 for flow charts ou t l i n i n g the f a c i l i t i e s of PARFIT). As mentioned in Chapter 3, the development of an inter a c t i v e package such as PARFIT i s an evolutionary process. The documentation in this appendix describes the f i r s t stage in this process. APPENDIX A 177 PARFIT INTERACTIVE PARAMETER FITTING IN DYNAMIC MODELS CONTENTS 1. Introduction and notation 2. Information reguired by PARFIT 3. Temporary f i l e s used by PARFIT 4. The batch mode of PARFIT 5. Commands in PARFIT APPENDIX A 178 1. INTRODUCTION AND NOTATION The program PARFIT i s designed to allow a user to in t e r a c t i v e l y f i t parameters in an i n i t i a l value problem when observations are available on the solution to the problem. Its use does not require the learning of any complex set of commands. The program i s written in ALGOL W; however, a knowledge of FORTRAN i s s u f f i c i e n t for i t s use. S p e c i f i c a l l y the program handles i n i t i a l value problems of the form y'=g(t,y,p) y(t 0)=y 0(p) where y i s an n-vector of state variables, p i s an m-vector of parameters, t i s the independent variable which we w i l l c a l l time for convenience, and 1 indicates d i f f e r e n t i a t i o n with respect to time. Along with the i n i t i a l value problem, we have a set of observations v,,...,v^ taken at d i s t i n c t times t ,...,t^ respectively where tf may or may not equal t 0 and where t^ >t , 1 = 1,2,...,k. Each v^ i s an r-vector where r<n. That i s , not a l l components of y need be observed. Each , however, contains observations on the same components of y. The central part of PARFIT i s a command reader. This nucleus of the program t i e s together the various f a c i l i t i e s such as integration, optimization and p l o t t i n g . After PARFIT has executed a par t i c u l a r command, control returns to the command reader and PARFIT i s ready for the next command. This continues u n t i l a QUIT command i s issued. Once a command i s issued, PARFIT usually e l i c i t s from the user a l l the information required to execute the command. This p r i n c i p l e i s , in the interests of e f f i c i e n c y , violated s l i g h t l y in the interactive option of the FIT command. It i s useful to view the operation of PARFIT in terms of an environment and a set of commands which operate in and on this environment. The environment consists of such things as echo fl a g s , parameter scaling and freezing indicators, sample times, output options, and algorithm selection indicators. PARFIT i s modular in nature and i s designed to make the addition of new commands very easy. Existing commands are also easy to modify. For example at present the Levenberg-Marquardt technique i s used to solve nonlinear least squares problems, but other optimization methods can e a s i l y be added. New integration schemes can also e a s i l y be added. APPENDIX A 179 We summarize here some of the f a c i l i t i e s of PARFIT. For more d e t a i l s consult the command descriptions. There are extensive output f a c i l i t i e s for printing a variety of things such as Jacobian matrices, integration r e s u l t s , optimization d e t a i l s and smoothing r e s u l t s . Parameters can be frozen and scaled. There are f a c i l i t i e s for determining the optimal parameters in a least squares sense by automatic and by interactive techniques. There are f a c i l i t i e s for obtaining i n i t i a l approximations to the optimal parameters using such technigues as derivative and integral f i t t i n g , guessed observations and i t e r a t i v e improvement of guessed observations for unobserved state variables, and methods employing continuation parameters and break points. A subsection of PARFIT can be used in batch mode. Certain s t a t i s t i c a l information on the optimal parameters can be calculated and various control parameters governing the way PARFIT runs can be reset by the user. PARFIT functions with two sets of discrete time values. F i r s t we have the observation times where the observations v, ,...,v^ were taken and second we have the sample times. The sample times are the times at which plot points for the state variables and smoothed observations are taken and the times at which information on the continuous solution of the s e n s i t i v i t y equations (see below) can be extracted. For further d e t a i l s see the SAMPLE command and item 7 of the REPORT command. Before continuing our description of PARFIT, we must establish more notation. Define the weighted residual vector f by for s=l,...,r, 1=1,...k where r i s the length of vs , 1=1....k, v.iA i s component s of v 4 and y_ t x l ) (t^ ) i s the corresponding element of the vector y(t«). w/t(je-•)+/L 1 S a weighting factor. We note that f has length kr. We seek p to minimize F(p)=f T(p)f (p) . The gradient of F(p) i s VF(p)=2J T(p) f (p) where J(p) i s the kr x m matrix defined by J.. =3J; i=l,...,kr; j=l,...,m. The elements in J are found by integrating the s e n s i t i v i t y APPENDIX A 180 equations: Yj =g t a(t,y,p)y + g (t fy,p) Yi> ' P J - f y . ) , , - <P> A for j=l,...,m derived from the o r i g i n a l i n i t i a l value problem. In our notation yf. and g,.. are n-vectors of p a r t i a l derivatives of the components Jof y and' g with respect to p. , and g i s the Jacobian matrix of g with respect to the vector y. Our integration program i s s p e c i a l l y designed to take advantage of the fact that our s e n s i t i v i t y equations are linear and coupled in only one d i r e c t i o n to the o r i g i n a l nonlinear i n i t i a l value problem. A l l the step size and order adjustments are done on the nonlinear i n i t i a l value problem and the linear problems are e f f i c i e n t l y solved along with the nonlinear problem. This i s the same strategy as that adopted by van Domselaar and Hemker [71] . The Levenberg-Marquardt technique for finding the optimal parameter vector p uses the i t e r a t i o n pl%«> =p<*> - ( J T ( p frJ )J(p' f J )+>!)-' J T ( P C ? J ) f ( p ( ' J ) where 0_<A<«o. As in [71] we employ the singular value decomposition in our implementation of this algorithm. This avoids the forming of J T J with i t s associated squaring of the condition number. A scaling option i s also available in our implementation of the Levenberg-Marquardt method. F i n a l l y in our notation a steepest descent i t e r a t i o n to get the optimal p i s P"**3 = Pfr J - S J T ( P ( y J ) f ( P < " ) where S i s the step length referred to in the interactive option of the FIT command. To run PARFIT the information detailed in Section 2 must f i r s t be supplied. The pa r t i c u l a r run command for PARFIT depends on what f a c i l i t i e s of PARFIT the user desires. The basic run command i s $RUN PFIT:PAR1+PFIT:L1+CP+*PRPL0T 4=DP T=t thi s provides a l l aspects of PARFIT except the SMOOTH, DFIT, IFIT, CONTIN and CREATE commands, and l i m i t s the integration method to the trapezoidal method. The f i l e CP contains the object code for the procedure G_FUN described in the next section, and DP i s the data f i l e described in the next section. APPENDIX A 181 If CP i s the object code for the FORTRAN subroutine (named GF) version of G_FUN, then the MTS run command i s $RUN PFIT:PAR1+PFIT:L1+PFIT:LF+CP+*PRPL0T 4=DP T=t To run PARFIT with Gear's program and the CREATE command available, but without the DFIT, IFIT, CONTIN and SMOOTH commands, the MTS run command i s $RUN PFIT:PARl+PFIT:PAR2+PFIT:L12+CP+*PRPLOT 4=DP T=t If CP is a compiled FORTRAN subroutine, then as in the previous command, PFIT:LF must be used. To run PARFIT with everything but Gear's program and the CREATE command, the MTS run command is $RUN PFIT:PARl+PFIT:PAR3+PFIT:L13+CP+*PRPLOT+*NUMLIB 4=DP T=t (with the previous modification i f CP comes from a FORTRAN program). F i n a l l y , to run the complete PARFIT, the run command is $RUN PFIT:PAR1+PFIT:PAR2+PFIT:PAR3+CP+*PRPL0T+*NUMLIB 4=DP T=t (with the appropriate change for a FORTRAN CP). It i s suggested that a time l i m i t be put on a l l runs to avoid the p o s s i b i l i t y of unnecessary expense. A good strategy i s to start with the simple version of PARFIT and to use the more powerful f a c i l i t i e s when they become necessary. 2. INFORMATION REQUIRED BY PARFIT To run PARFIT, the user must supply a procedure which defines g(t,y,p) as well as the Jacobian functions g^ and qp and which provides i n i t i a l i z a t i o n information. Since PARFIT i s written in ALGOL W, i t i s natural that this program should also be in ALGOL W, but as indicated in the run commands of the previous section, a FORTRAN subroutine can be used. If a FORTRAN subroutine i s used, i t should be named GF and a l l i t s real arguments should be double precision. If an ALGOL W procedure i s used, i t should be called G_FUN and i t should have the following header: (The pa r t i c u l a r parameter names are of course not important and may be changed by the user.) PROCEDURE G_FUN(LONG REAL VALUE T; LONG REAL ARRAY Y(*); LONG REAL ARRAY P(*); INTEGER VALUE OPTION; LONG REAL ARRAY G(*); LONG REAL ARRAY DGY(*,*); LONG REAL ARRAY DGP(*,*); LONG REAL ARRAY ISEN(*,*)); APPENDIX A 182 where T i s the independent variable time, Y i s the vector of state variables y(t) of length n, P is the vector of parameters of length m, OPTION indicates which of various tasks G_FUN i s to perform: 1 to return g(t,y,p) and g (t,y,p) -1 to return just g(t,y,p) 2 to return g /.(t,y,p) 3 to return i n i t i a l y in Y and i n i t i a l values for y , j=l,...,m in ISEN, J -3 to return just the i n i t i a l y in Y. G returns the n-vector g(t,y,p) when reguired, DGY returns the n x n matrix g^(t,y,p) when required, DGP returns the n x m matrix g (t,y,p) when reguired, ISEN returns the nxm matrix of i n i t i a l values for the s e n s i t i v i t y eguations. An example of an ALGOL W procedure G_FUN for the problem y/=-(l-y_Jy. +p i Y ; i y'=p ((1-y )y -(p +p )y ) y ( 0 ) = l , y (0)=0 is given in Figure A . l . (This dynamic model i s considered by van Domselaar and Hemker[71].) Besides providing the procedure G_FUN, the user must provide a data f i l e containing the information outlined below. This data i s read under free format with blanks acting as deli m i t e r s . The f i r s t data card contains the model name associated with the p a r t i c u l a r set of d i f f e r e n t i a l equations. It consists of at most 30 characters with no embedded blanks. And i t cannot be the word CREATE (see the CREATE command). APPENDIX A 183 PROCEDURE G_FUN(LONG REAL LONG REAL ARRAY Y(*);LONG INTEGER VALUE OPTION;LONG LONG REAL ARRAY G(*);LONG VALUE T; REAL ARRAY REAL ARRAY REAL ARRAY P(*) ; G(*) ; DGY(*, ) DGP(*,*) LONG REAL ARRAY ISEN(*,*)) A MODEL DEFINITION PROCEDURE; LONG REAL ARRAY BEGIN COMMENT EXAMPLE OF CASE ABS OPTION OF BEGIN BEGIN G (1) :=- (1. L-Y (2) ) *Y (1) +P (2) *Y (2) ; G(2) :=P(1)*((l.L-Y(2))*Y(1)-(P(2)+P(3))*Y (2)) THEN IF OPTION>0 BEGIN DGY(1,1) DGY(1,2) DGY(2,1) DGY(2,2) END; END; BEGIN DGP(1,1) DGP(1,2) DGP(1,3) DGP(2,1) DGP(2,2) DGP(2,3) END; BEGIN Y(l) :=1.L; Y(2) :=0.L; IF OPTION>0 BEGIN FOR I:=l FOR J:=l ISEN(I,J):=0.L; END; END END; END G FUN. :=-(l.L-Y(2)); :=Y(1)+P(2) ; :=P(1)*(1.L-Y(2)); :=-P(l) *(Y(1)+P(2)+P(3) ) = 0.L; =Y(2) ; = 0.L; = (1.L-Y(2) )*Y(1)-(P(2)+P(3) )*Y(2) =-P(l) *Y(2) ; =-P(l)*Y(2); THEN UNTIL UNTIL DO DO Figure A . l A t y p i c a l model d e f i n i t i o n procedure APPENDIX A 184 The second data card contains the i n i t i a l time. The next data card contains 5 integers separated by blanks. These integers are (1) number of state variables (2) number of components of p (3) number of components of v (4) number of observation times (5) maximum number of sample times The maximum number of sample times must be greater than the number of observation times by at least one when there are no observations at the i n i t i a l time. When there are observations at the i n i t i a l time, the maximum number of sample times can be greater than or equal to the number of observation times. The next set of data cards contains the observations. Several observations can be put on each card. The observations are ordered f i r s t by state variable and then by time. For example i f n=5 and state variables 2 and 5 are observed then a l l the observations on state variable 2 are entered and following this set of numbers, a l l the observations on state variable 5 are entered. It i s suggested, but not mandatory that observations on a new state variable s t a r t on a new card. The next set of data cards contains the observation times. These entries need not star t on a new card, but for c l a r i t y i t is suggested that they do. The next set of data i s a set of integers indicating which state variables have been observed. Next, an i n i t i a l guess at the optimal parameter values should be entered in the data f i l e . F i n a l l y , a set of weights corresponding to the observations may be entered by the user. This i s optional and i f no weights are entered, PARFIT by default sets a l l weights to one. If weights are to be entered, there should be an entry for each observation, these entries should be in the same order as the corresponding observations, and they must sta r t on a new card. The use of free format should make the data entry f a i r l y APPENDIX A 185 simple. For example real numbers are entered as -.1, -3.2, .005, 2, 5 etc. and integers are entered as 1, -5, 2 etc. Numbers in exponential notation follow FORTRAN conventions. 3. FILES USED BY PARFIT PARFIT uses a number of MTS temporary f i l e s when i t i s running. Normally the user need not be concerned with these f i l e s . However, i f PARFIT terminates with an error, the information in these f i l e s may be of value to the user. The temporary f i l e s used are: -SCI This f i l e takes output from the integeration and optimization procedures. It is always emptied before i t i s reused. -SC2 When required, this f i l e accumulates the information in -SCI for later output. , -GRAPH This f i l e takes the output from the pl o t t i n g procedures. -GRAPHSTORE This f i l e accumulates graphical data for later hard copy output. -REPRT This f i l e takes output from the report command. It also takes certain messages such as those indicating when a permanent copy of a graph has been requested. -AECHO This f i l e contains an echo of the sessions with the interactive option of the f i t command. -ECHO This f i l e contains echo information as requested through the ECHO command. It never contains any of the information that i s sent to -AECHO. APPENDIX A 186 4. THE BATCH MODE OF PARFIT PARFIT i s designed as an interactive program. However, there are cases when, for reasons of economy say, a user would be advised to run sections of PARFIT in batch mode. For example, th i s might be the case in the f i n a l stages of parameter estimation when good approximations to the optimal parameter values are available and where these values are being further refined through the automatic option of the FIT command. The MTS RUN commands for a batch run are the same as for an interactive run. The PARFIT commands should follow d i r e c t l y after the RUN command. A l t e r n a t i v e l y , the user may specify SCARDS=filename in the RUN command, in which case the PARFIT commands are read form the f i l e attached to SCARDS. When in batch mode, PARFIT prin t s the command designators on *SINK*. When in batch mode, -SCI and -SC2 are not used. A l l output that goes to -SCI in an interactive run goes d i r e c t l y to *SINK* when PARFIT i s runnumg in BATCH mode. 5. COMMANDS IN PARFIT The following are the commands currently available in PARFIT. (1) ECHO (2) OPTION (3) FREEZE (4) SCALE (5) SAMPLE (6) WEIGHT (7) PLOT (8) REPORT (9) SET (10) INTEG (11) PROBE (12) FIT (13) STATS (14) QUIT (15) SUSP (16) CREATE (17) SMOOTH (18) CREOBS (19) LINEAR (20) DFIT (21) IFIT (22) CONTIN With each of the above commands, we describe what action the command i n i t a i t e s and what interaction with PARFIT the user can expect after issuing the command. (1) ECHO The ECHO command allows the user to control the accumulation of a hard copy echo of an interactive session. When t h i s command i s issued, PARFIT requests an entry of 0 or 1. An entry of 0 turns off the echo and an entry of 1 turns the echo on. This command does not affect the echo associated with option 2 of the FIT command. At the start of an interactive session PARFIT turns the echo o f f . APPENDIX A 187 (2 ) OPTION This command allows the user to set certain control parameters in PARFIT that govern the way various procedures function. These control parameters are automatically given default values by PARFIT at the st a r t of a run. After issuing the OPTION command, PARFIT returns with the following message indicating what control parameters the user can change DESIGNATE CONTROL PARAMETER AND NEW VALUE (0 TO END) OUTPUT 1 INT. OUTPUT IN OPT. 0-YES,1-NO 2 JACOBIAN 0-FULL 1-LEAST SQUARES 3 GEAR'S METHOD 0-ADAMS , 1-STIFF 4 EPS FOR INTEGRATION 5 HMIN FOR INTEGRATION 6 HMAX FOR INTEGRATION 7 INTEGRATION PROCEDURE, 1-GEAR, 2-TRAPEZOIDAL 8 The convention of using a 0 to end a seguence of input data of indeterminate length i s used in several places in PARFIT. Control parameter 1 governs the dumping of information during an integration. This option i s useful during the debugging stages with a new model when the integration program encounters d i f f i c u l t i e s . This may happen for example i f G_FUN is returning undefined values to the integration procedures. The default value of this control parameter i s 0 in which case no output occurs. If this control parameter i s set to the integer n, then after every n integration steps, the time and state variables are printed on -SCI. Control parameter 2 indicates whether or not any output requested by a nonzero control parameter 1 i s required when the integration procedure has been called by an optimization procedure. This control parameter i s 0 i f output i s desired during an optimization and 1 otherwise. Its default value i s 1. Control parameter 2 acts as a safety on control parameter 1 in that the user must s p e c i f i c a l l y request integration output in an optimization run thus guarding against the chance of getting a large quantity of output by accident. Control parameter 3 selects the information to be printed when output of the Jacobian matrix i s requested. If this control parameter i s 0 then the f u l l Jacobian on a l l the state variables and at a l l the sample times i s printed. If this APPENDIX A 188 control parameter i s 1 then only the entries of the f u l l Jacobian that relate to observations on the state variables are printed. (That i s the Jacobian J i s printed.) The default value of control parameter 3 i s 0. The f u l l Jacobian i s of value to the model builder because i t can t e l l him (perhaps after further analysis) i f and where he should take further observations to better determine his parameters. This i s the case because the f u l l Jacobian matrix i s determined by solving the s e n s i t i v i t y equations and i s thus independent of the observations. Control parameter 4 indicates what option in Gear's integration procedure i s to be employed. If this control parameter i s 0 then integration of the i n i t i a l value problem i s done by Gear's implementation of an Adams' predictor corrector method[27]. If thi s control parameter i s 1 then the integration i s done by Gear's implementation of a multi-step method suitable for s t i f f problems[27]. The Adams' predictor-corrector method can be faster when s t a b i l i t y i s not a problem. The default value of control parameter 4 i s 1. Control parameter 5 contains the error c r i t e r i o n EPS to be used in gear's integration procedure. Its default value i s .01. As this value i s decreased, the user can expect his integrations to become more expensive. Among other things, the observation error should be considered when picking this control parameter. We comment that .01 i s a very weak error c r i t e r i o n . Control parameter 6 contains the minimum step size that integration procedures with stepsize control are allowed to use. Its default value i s .00001. Control parameter 7 contains the maximum step size that integration procedures with stepsize control are allowed to use. Its default value i s 5. Control parameter 8 indicates what integration procedure i s to be employed. If i t i s 1, Gear's program i s used. If i t i s 2, a trapezoidal method without error control i s employed. The default value of control parameter 8 i s 2. APPENDIX A 189 (3) FREEZE The FREEZE command allows the user to freeze selected parameters. These parameters then remain fixed u n t i l freezing i s removed. There must be at least two active parameters for the optimization procedures to function properly. The freezing in the int e r a c t i v e option of the FIT command i s at a higher l e v e l than the freezing indicated by the FREEZE command. When this command i s issued, PARFIT asks the user to enter a l i s t of subscripts of parameters to be frozen (0 to end). An entry of 0 removes a l l freezing. (4) SCALE The SCALE command allows the user to transform selected parameters. Currently logarithmic scaling and square root scaling are a v a i l a b l e . With logarithmic scaling for p^ , pj i s transformed according to 'p\=ln(Pj), and with square root scaling, Pj i s transformed according to 'pj. =Vp"j - Thus with logarithmic s c a l i n g , p- i s repalced by exp(p_ ) in the model and with square root scaling p_j i s repalced by "p^ . When t h i s command i s issued, PARFIT asks the user to enter pairs of integers indicating subscripts of parameters to be scaled and the scaling to be used. The integer 1 indicates logarithmic scaling and 2 indictes square root scaling. A 0 subscript terminates the entry of scaling instructions. An entry of 0 alone removes a l l scaling. When a parameter i s scaled, or descaled, i t s current numerical value i s automatically transformed. (5) SAMPLE This command allows the user to alter the set of sample times. As mentioned in the introduction, these are the times at which information i s extracted from the continuous problem. The default sample times are the i n i t i a l time and the observation times. When the user issues the SAMPLE command, a choice of three options i s presented. The f i r s t option restores the sample times to their default values. The second option allows the user to specify a uniform mesh of sample times sta r t i n g at the i n i t i a l time by entering the number of sample times (not counting the i n i t i a l time) and the sample time spacing. Of course an error results i f more sample times than are allowed (as indicated in the data f i l e ) are requested. With this option, no connection to the observation times i s maintained, and thus care must be used when thi s option i s in e f f e c t . For example, the FIT command cannot be used when thi s option i s in ef f e c t . The thi r d option i s for the interactive insertion of APPENDIX A 190 sample times between existing sample times. The user i s f i r s t asked i f a l i s t i n g of the existing sample times i s desired. If i t is desired, the user enters the subscript range (in the existing vector of sample times) where the l i s t i n g is desired. To insert sample times between existing sample times, the user l i s t s in sequence the upper index (in the vector of sample times) of the i n t e r v a l where the new times are to be inserted, and the number of points to i n s e r t . The points are inserted uniformly in the i n t e r v a l . The user can request insertion of times in several i n t e r v a l s . A 0 for an interval's upper index ends the input. This interactive insertion option can be of value when the sample times straddle a time i n t e r v a l where the d i f f e r e n t i a l equation solution warrants further investigation (for example, i t might be taking a sudden jump.) (6) WEIGHT The WEIGHT command allows the user to i n t e r a c t i v e l y enter the weights for the observations. There are two options. F i r s t , the weights can a l l be set to their default value of one. Second, weights can be specified on selected observations on selected state variables which have been observed. Under the second option, the user i s given the chance to take a permanent copy of the weights in the scratch f i l e -WEIGHT. (7) PLOT The PLOT command allows the user to select various items to p l o t . When the command i s issued, the following instructions appear SEQUENCE OF ITEMS TO PLOT (END WITH 0) STATE VARIABLES 1 OBSERVATIONS 2 SMOOTHED OBSERVATIONS 3 GUESSED OBSERVATIONS 4 PHASE PLOT 5 Of course before item 1 can be selected, an integration must have been performed, and before item 3 can be selected, the SMOOTH command must have been used, and before item 4 can be selected, the CREOBS command must have been used. Item 5 applies only to 2 state variable problems. Before i t can be used an integration must have been performed. After the desired items are selected, the user i s given further choice. For example i f items 1 and 2 are selected, the user i s asked which state variables and which observations ( i . e . on which state variables) he wants plotted. This f l e x i b i l i t y allows the user APPENDIX A 191 to i s o l a t e various aspects of the problem. After the user has described the desired plo t , a mini-print-plot appears at his terminal. The abscissae for the plotted points are the sample times. After the plot i s completed, the user i s asked i f a permanent record i s required. If the answer i s y for yes, a large scale version of the print plot i s accumulated in the f i l e -GRAPHSTORE for later output. A plot number (starting at 1) i s associated with each graph accumulated and when a plot i s accumulated, a message indication the current parameter values and the current plot number i s written on the f i l e -REPRT. The interface to the p l o t t i n g programs i s confined to one procedure in PARFIT and thus i t i s easy to modify the p l o t t i n g f a c i l i t i e s of PARFIT and the p l o t t i n g hardware employed can e a s i l y be altered. (8) REPORT This command controls a set of output procedures with which PARFIT can display various information to the user. When this command i s issued, PARFIT returns with the following message indicating what items the user can have printed. ENTER LIST OF ITEMS TO PRINT (END WITH 0) GENERAL DATA 1 OBS 2 PAR 3 CREATION 4 PTS AND STATE VARS—5 SMOOTHING DATA 6 JACOBIAN 7 INTEG./OPT. DETAILS-8 STATISTICAL DATA 9 OPTION SETTINGS 10 WEIGHTS 11 GUESSED OBS 12 new items can e a s i l y be added to this l i s t . The requested information i s displayed at the terminal and also put in the f i l e -REPRT so that a permanent record can be taken at the end APPENDIX A 192 of a run with PARFIT. When item 1 i s selected, PARFIT displays the following basic information on the pa r t i c u l a r problem under consideration (a) the number of parameters (b) the number of state variables (c) the number of state variables observed (d) a l i s t of state variables observed (e) the number of observation times Selection of item 2 causes a l i s t of observations along with observation times to be printed. Selection of item 3 causes the starting parameter values to be printed (that i s the values read from the data f i l e ) along with the current parameter values and the freezing and scaling status of each parameter. Report item 4 prints information for the special s i t u a t i o n when test observations have been created by a simulation run. The parameter values for the simulation run are printed along with the standard deviation of the random error introduced into the generated observations. Report item 5 pri n t s the parameter values used in the l a s t integration of the i n i t i a l value problem along with the sample times and integration results at these sample times. Report item 6 pri n t s smoothing information generated by the SMOOTH command. F i r s t the type of smoothing used—either least squares cubic spline or least squares cubic Hermite i s indicated. Next the j o i n t s used for the piecewise polynomial smoothing function are displayed and f i n a l l y the smoothed observations and the smoothed derivatives at the sample times are l i s t e d . Report item 7 pri n t s the parameter values used in the l a s t integration along with the jacobian matrix. Whether or not a f u l l Jacobian matrix i s printed depends on control parameter 3 which can be altered in the option conmmand. APPENDIX A 193 Report item 8 prints optimization and or integration data. When an automatic optimization run i s made, detailed information on the run (eg. sums of squares of residuals, parameter values etc.) i s written on the scratch f i l e -SCI. Depending on the settings of control parameters 1 and 2 (see the option command), integration information may be written on -SCI. When the user selects item 8 and when PARFIT i s running i n t e r a c t i v e l y , the contents of -SCI are displayed at the terminal. Since a l o t of r e l a t i v e l y useless output may be present in -SCI, the user i s asked i f the contents of -SCI are to be accumulated for later output. If a later hard copy i s desired, the contents of -SCI are accumulated in -SC2. The f i l e -SCI i s emptied before i t is next reguired to accept output from PARFIT. An output reference number i s attached to each use of -SCI. When an accumulation i s made to -SC2, a message to this e f f e c t along with the current output reference number i s written on -REPRT and the output reference number i s incremented by 1. This allows a coordinated interpretation of the output from PARFIT. Report item 9 prin t s s t a t i s t i c a l data resulting from the STATS command. The F d i s t r i b u t i o n value used in determining confidence intervals on the parameters i s displayed along with i t s corresponding percentile value and number of degrees of freedom in the numerator and number of degrees of freedom in the denominator. The sum of the squares of the residuals is printed and the parameter values along with their confidence intervals are printed. F i n a l l y , the corr e l a t i o n and covariance matrices are printed. Report item 10 prints the current control parameter settings along with the default values for the control parameters. Report item 11 print s the weights given to the observations. Report item 12 prints the guessed observations and corresponding derivatives generated by the CREOBS command. (9) SET This command allows the user to redefine an element of the parameter vactor p by entering the integer subscript of the element and the new value of the element. This can be repeated for as many elements as desired. Entry of a 0 for a subscript terminates the command. For example to set the f i r s t and fourth APPENDIX A 194 parameters to 3. and 6.2 respectively, the user would enter 1 3. 4 6.2 0 (The above set of numbers need not a l l be on one line.) Parameter subscripts refer to the f u l l parameter vector and no allowance for frozen parameters i s required. Scaling i s ignored by the SET command. (10) INTEG This command requests that the i n i t i a l value problem be numerically integrated using the current parameter values. The integration technique used i s determined by control parameter's 4 and 8 which can be reset by the OPTION command. The integration results at the sample times are stored in an array for later use. For example these values may be plotted at some lat e r time in an interactive session. When this command i s issued, the user i s asked i f the Jacobian i s desired, and i f i t is the s e n s i t i v i t y eguations are integrated along with the given model equations and the f u l l Jacobian i s stored for later analysis. (for example by the PROBE command). If the integration procedure runs into d i f f i c u l t y , an error message i s printed and the user i s returned to the command section of PARFIT. A t y p i c a l d i f f i c u l t y with the integration procedure i s that i t cannot meet the error c r i t e r i o n with the current minimum step s i z e . (11) PROBE This command allows the user to investigate, among other things, the condition at the current point p in parameter space of the Jacobian matrix associated with the least squares problem. The user i s asked i f an integration i s required to determine the Jacobian at the current parameter values. ( i t may not be i f for example INTEG with a Jacobian option has just been executed.) A singular value decomposition i s done on the Jacobian for the least squares problem. The user is given the option of taking a permanent record of the PROBE results in the f i l e -REPRT. Furthermore, i f control parameter 1 i s set to request output in an integration, the user i s given the chance to view the output and accumulate i t in -SC2. Since the singular value decomposition i s available when the PROBE command has been executed, the potential exists for adding a procedure here to further analyse the problem at the current point in parameter space. APPENDIX A i 195 (12) FIT The FIT command puts the user in control of optimization procedures which apply d i r e c t l y to the least squares minimization problem mentioned in the introduction (as opposed to the DFIT command for example). Currently the user has two main choices with the FIT command. The f i r s t option uses the Levenberg-Marquardt algorithm. From a starting guess, this algorithm attempts to determine the optimum parameters automatically without user intervention. For e f f e c t i v e use of this procedure, the i n i t i a l guess at the optimal parameter vector should be f a i r l y good. The user i s asked to supply a start i n g value for the parameter A used in the Levenberg-Marquardt algorithm as well as error tolerances for termination of the automatic optimization run. A negative value for X t e l l s PARFIT to pick i t s own star t i n g value for X. At times X must be adjusted upward i n t e r a c t i v e l y to avoid points in parameter space where the d i f f e r e n t i a l equation cannot be integrated. The termination c r i t e r i o n take the form of a r e a l t i v e (e, ) and an absolute (e a) error tolerence. Termination of the optimization run occurs when either where F f i s the sum of the squares of the residuals on the q'th i t e r a t i o n . Of course computation costs increase as e, and e a are decreased and for fine tolerences a batch run of PARFIT i s probably advisable. Choice of e, and e a should of course depend on the accuracy of the observations and on the error c r i t e r i o n chosen for the numerical integration technique. Our implementation of the Levenberg-Marquardt algorithm has a provision for automatic scaling so that the diagonal elements of J J are a l l 1. To request scaling, e, and e A should both be negative. PARFIT uses their absolute values for the termination c r i t e r i a when scaling i s requested. The use of scaling can dramatically speed up convergence. The second option under the FIT command provides an inter a c t i v e optimization approach where the user has extensive control over PARFIT through a set of optimization commands. Among other thing ys, the user can reset parameters, freeze selected parameters, and plot graphs. Since i t i s anticipated that these commands w i l l be used very frequently, the descriptive messages from PARFIT are kept to a minimum and the command designators are very short. A description of the APPENDIX A 196 currently implemented optimization commands follows. (a) T This i s the technique command where the user can choose either the Levenberg-Marquardt or the steepest descent optimization method. To request the steepest descent technique the user should enter T SD and then h i t the return. One or more blanks must separate the T and the SD. To request the Levenberg-Marquardt technique, the user should enter T MARQ The default i s the Levenberg-Marguardt technigue. (b) M By entering M r where r i s a real number, the current value of X for the Levenberg-Marquardt technique or step length for the steepest descent technique i s multiplied by r and this product replaces the current X or step length. A new i t e r a t i o n of the current optimization technique i s then attempted. PARFIT then reports on the success of this attempt and the user i s asked to enter a new optimization command. The default value of X i s .01. There i s no default value for the step length for the steepest descent method. The user should pick a starting value for the step length after observing the gradient. (c) N By entering N r where r i s a real number, the current value of X or step length (depending on the technique) i s replaced by r. APPENDIX A 197 (d) V This i s the view command. It does not take any arguments. It requests a display of the current parameters, and current gradient components. Frozen parameters are indicated—see command (e). The current technique i s also displayed along with basic information associated with this technique. F i n a l l y , the current sum of squares i s displayed. (e) F This i s the freeze command. This command acts at a higher l e v e l than the main FREEZE command. It allows the user to s e l e c t i v e l y freeze various parameters at their current values and to continue the optimization on a subspace of parameter space. A 0 i s used to terminate the l i s t of parameters to be frozen. For example, to freeze the f i r s t and thi r d parameters, the user would enter F 1 3 0 Currently, no special programming i s implemented to take advantage of the fewer s e n s i t i v i t y equations present when we are working on a subspace. (f) DF This i s the defreeze command. It removes a l l or some of the parameters from the l i s t of frozen parameters. This command does not influence freezing set by the FREEZE command. To remove a l l parameters from the l i s t of frozen parameters enter DF 0 To remove freezing on say the third and fourth parameters enter DF 3 4 0 (g) SET This i s the SET command and i t i s i d e n t i c a l , except for the pri n t i n g of guiding instructions, to the main SET APPENDIX A 198 command. (h) PLOT This command is i d e n t i c a l to the main PLOT command. The state variables plotted are those from the l a s t integration. Thus i f a plot i s requested after an unsuccessful Marquardt i t e r a t i o n attempt, the state variables plotted are those at the l a s t set of t r i a l parameters. (i) Q This i s the quit command for the interactive optimization subsection. It returns control to the main command section of PARFIT. A detailed record of a l l commands and a l l command results (including mini-print-plots) that occur in an interactive optimization session i s kept in the f i l e -AECHO. The user has the option of taking a hard copy of thi s f i l e at the end of a run with PARFIT. (13) STATS This command reguests PARFIT to produce certain s t a t i s t i c a l information on the parameters after they have been optimized by the FIT command. An assumption of l i n e a r i t y in the parameters near the optimum i s made. The user i s requested to enter the percentile for the confidence i n t e r v a l s . The program then finds the reqired value of the F d i s t r i b u t i o n with the appropriate degrees of freedom. The confidence inte r v a l s on the parameters and the co r r e l a t i o n and covariance matrices are calculated when the STATS command i s issued. (14) QUIT This command terminates a run with PARFIT. Before execution i s terminated, the user i s given the chance to take a permanent copy of some or a l l of the information accumulated APPENDIX A 199 during the interactive session. In batch mode a permanent copy of the accumulated plots i s automatically taken. (15) SUSP This command suspends execution of PARFIT and returns the user to MTS. The MTS command $RESTART causes the execution of PARFIT to be resumed. One use of this might be to examine various scratch f i l e s PARFIT has created. (16) CREATE This command allows the user to make a simulation run with his model and to generate a data f i l e from this simulation run. A primary purpose of this f a c i l i t y i s in the debugging of new procedures in PARFIT. The user i s asked for the parameter values for the simulation run, the observation times, the state variables observed, and the error tolerence, maximum step siz e , and minimum step size for Gear's integration program. (If the integration run does not succeed, i t s constraints can be reset and a new run can be made.) The user i s further asked for the standard deviation of the random error in the generated observations and the vector of starting values for the parameters. F i n a l l y the user can have the generated data put in the f i l e -DATA. When the create command i s to be used, the data f i l e should have a model name of CREATE in i t and of course only the f i r s t three data cards described in section 2 are required when the CREATE command i s to be used. (17) SMOOTH The SMOOTH command allows the user to f i t a least squares piecewise polynomial to the observations on each state variable observed. This i s normally used in preparation for the DFIT IFIT and CONTIN commands. The user has the choice of using either a piecewise cubic spline or a piecewise cubic Hermite polynomial—the l a t t e r should be used i f the observations take any sudden violent jumps. In either case, PARFIT sequentially goes through the state variables on which observations have been taken. On each state variable, the user i s asked to enter the number of j o i n t s and j o i n t positions for the indicated set of observations. A maximum of 15 jo i n t s for each piecewise polynomial i s allowed. It should be very easy to make this curve f i t t i n g aspect of PARFIT more in t e r a c t i v e , however for the present, the user must issue SMOOTH snd PLOT commands a l t e r n a t i v e l y when doing APPENDIX A 200 interactive curve f i t t i n g . (18) CREOBS This command allows the user to guess observations on the unobserved state variable in the two state variable case. This i s in preparation for options 2 and 3 of the DFIT command and options 2, 3, and 4 of the IFIT command. The user is asked to enter the number of abscissae and the abscissae for an interpolating cubic spline to approximate the guessed observations. The f i r s t and l a s t sample times must begin and end the l i s t of abscissae. Next the user i s asked to enter the corresponding ordinates. At each end, the interpolating cubic spline matches the slope of the l i n e joining the two points of interpolation closest to the given end. (19) LINEAR The LINEAR command allows the user to specify that the DFIT and IFIT least squares problems are linear and thus starting parameters are not required since no it e r a t i o n s are required to obtain the optimal parameters. Currently this f a c i l i t y i s implemented in options 2 and 3 of the DFIT command and in options 2 3 and 4 of the IFIT command. (20) DFIT This command has three options. The f i r s t option uses the results of the SMOOTH command to estimate optimal parameters by a rather coarse, but at times inexpensive technigue. Furthermore, th i s procedure can be useful when our i n i t i a l parameter values correspond to an unstable i n i t i a l value problem. There are however some r e s t r i c t i o n s on the class of problems the DFIT command can handle. It cannot handle problems where some parameters occur only in the i n i t i a l conditions, and for this version of the DFIT command, a l l parameters must occur in the subset of the d i f f e r e n t i a l eguations defining the i n i t i a l value problem that correspond to state variables on which observations have been taken. This program works by applying an automatic Levenberg-Marquardt procedure to the nonlinear curve f i t t i n g problem s'=g(t,s,p) where s^,- (t) , the i ' th component of the vector s ( t ) , i s a piecewise polynomial approximation to the observations on y.. The nonlinear curve f i t t i n g i s done in a least squares sense at the sample times. No weighting i s employed at present. If not APPENDIX A 201 a l l the state variables are observed, then at each i t e r a t i o n of the Levenberg-Marquardt procedure, a subset of the set of equations in our i n i t i a l value problem i s integrated (using s ( t ) , the current parameter vector and the integration technique indicated by control parameter's 2 and 8). Thus this technique can become expensive too. Also i n s t a b i l i t i e s at the star t i n g parameter values may arize in the subsystem i n i t i a l value problem. Furthermore, esp e c i a l l y i f the observations are far apart and have large errors, the parameters determined by t h i s technique are not very r e l i a b l e . However, they can serve as start i n g values for the FIT command. As in the automatic option of FIT command, the user i s asked to enter a star t i n g X along with a r e l a t i v e and an absloute error tolerence for the Levenberg-Marquardt procedure. The second option under the DFIT command currently applies only to the important special case when only two state variables are present and observations are available on only one of them. This option assumes the behavior of the unobserved state variable has been approximated using the CREOBS command. The observations on the other state variable must be smoothed with the SMOOTH command prior to using this option. This option of the DFIT command then f i t s derivatives using the smoothed and guessed observations. Only the nonfrozen parameters enter into the optimization. If a parameter occurs only in an i n i t i a l condition, i t must be frozen prior to the use of this command. If the least squares problem i s l i n e a r , (as indicated by the LINEAR command), then a linear least squares technique using the singular value decomposition i s employed. If the problem i s nonlinear, then the Levenberg-Marquardt algorithm i s employed and a st a r t i n g lambda and r e l a t i v e and absolute error tolerances must be supplied. The t h i r d option under the DFIT command provides a means of i t e r a t i v e l y improving the guessed observations with the aid of a sparse Gauss-Newton procedure. This option currently applies only to the two state variable case. Starting with guessed observations on the unobserved state variable, PARFIT attempts to find an optimal parameter vector p and corresponding guessed observations c= (c,, , .. . , C y ) r , where c^ i s the guessed observation at time t^ , to minimize ST ( g ( t , (s*,c ) T , p ) - s * ' ) a + Z. (d. (c))* where for notational convenience we have assumed y_ i s unobserved, where s =s(t x) (a superscript i s used to avoid confusion with our notation when s(t) i s a vector), and where d_^(c) represents a d i s c r e t i z a t i o n of the state equation corresponding to the unobserved state variable. Currently APPENDIX A \ 202 PARFIT uses the trapezoidal d i s c r e t i z a t i o n for (c). That i s d^(c)=.5(g (t , ( 5 * " , C j l_ ) r,p)+g ( t , (s*,c ) T,p)) c. -c PARFIT employs a nonlinear block Gauss-Seidel technique to find the optimal p and c. That i s PARFIT starts by f i x i n g c and finding the optimal parameter vector to minimize the above sum of sguares. Then with p fixed at this optimum, an optimal c i s determined to munimize the above sum of squares. This l a t t e r optimization i s accomplished through a Gauss-Newton procedure which takes advantage of the pa r t i c u l a r sparsity structure of the problem. The user i s requested to enter a tolerance e to define the stopping c r i t e r i o n for the i t e r a t i v e determination of c. The i t e r a t i o n terminates when two successive iterates c (t> and c1*"* s a t i s i f y l c ^ + , ) -c*' K e d c f * I+.001) for 1=0,...,N. There i s an option to f i x the i n i t i a l conditions of the guessed observations. PARFIT has f a c i l i t i e s for handling the case when the estimation of p for a fixed c i s a linear problem. When this problem i s not l i n e a r , the user i s reguested to enter a st a r t i n g lambda and r e l a t i v e and absolute error tolerances for the determination of the optimal p by the Levenberg-Marquardt technique. After an optimal c i s found, the user i s given the chance to further refine this vector by decreasing the tolerance e. F i n a l l y , after an i t e r a t i o n (the determination of a new p and a new c) the user i s given the option of doing another i t e r a t i o n or terminating the i t e r a t i v e process. If the process i s terminated, i t may be restarted by issuing the same DFIT command that i n i t i a t e d the i t e r a t i v e process. (21) IFIT This command i s similar to the DFIT command except here integrals are used instead of derivatives. Currently there are four options available under this command. The f i r s t option requires observations on a l l state variables and i t determines parameters which minimize f "^pjf (P) where APPENDIX A 203 *t f * U -.>•,• = y o „ ( P ) + K ( t , S ( t ) , p ) d t - S . (t) where 1=1,...,k; i=l,...,n; y a > . i s the i 1 t h component of the i n i t i a l condition vector and g y i s the i 1 t h component of g(t,s,p), and s(t) i s the smoothing function determined by the SMOOTH command. Currently there is no provision for the linear case and the Levenberg-Marquardt method i s used to obtain the optimal parameters. The second option under the IFIT command corresponds to the second option of the DFIT command. This option f i t s integrals to the smoothed observations and to the guessed observations. There are special f a c i l i t i e s for handling the linear case. Currently this option of the IFIT command applies only to the two state variable case. The t h i r d option under the IFIT command attempts to i t e r a t i v e l y improve the guessed observations by an experimentally e f f e c t i v e , but, occasionally, unstable technique. The i t e r a t i o n proceeds by f i r s t applying the method used in option two of the IFIT command to estimate the parameters, and then, holding the observed state variable equal to s ( t ) , a new set of guessed observations i s generated by integrating the state equation corresponding to the unobserved state variable. Currently t h i s option applies only to the two state variable case. As with the third option of the DFIT command, after an i t e r a t i o n i s complete, the user i s given the chance to terminate the i t e r a t i v e process. The fourth option under the IFIT command is very similar to the t h i r d option under the DFIT command. The only difference i s that the parameter vector i s updated using the procedure employed in the second option of the IFIT command. Provisions are available for linear parameter estimation problems. (22) CONTIN The CONTIN command provides the user with a technigue employing continuation methods and break points which i s designed to bridge the gap between the coarse integral f i t t i n g method and the f u l l least squares problem. A di r e c t continuation method i s available where the user can f i t the solution of u'=g(t, ( (l-*)s(t)+»u,p) u(t o)=y 0(p) APPENDIX A 204 to s(t) in the least squares sense at the observation times where 0<X<1 and where s(t) represents a smoothing of the observations. The user can experiment with various values for the continuation parameter y . When **=0, we have the f i r s t option of the IFIT command, and when 2f=l, we have the f u l l least squares problem (on the smoothed observations). Observations must be available on a l l state variables to use this strategy in the CONTIN command. Another f a c i l i t y available under this command involves the use of break points. The user can specify break points at times T <T <...<T_ corresponding to observation times The user can also specify a continuation parameter vector ^ for the break points. As we integrate the i n i t i a l value problem through the break point at time T_. , u i s reset according to u(T. )=Au"(T- ) + (I-A)s(T- ) where A=diag («<v ,.. . , , and u"(T,-) i s the result obtained by integrating up to time T,- . The user also has the chance to weight the break points with weights w,,...,w.. F i n a l l y , when break points alone are employed (_=1), observations need not be available on a l l state variables. The components of e< corresponding to unobserved state variables should be set equal to 1. This command has no implementation r e s t r i c t i o n s on the number of state variables. APPENDIX A 205 APPENDIX B SELECTED PROGRAM LISTINGS In this appendix we give selected l i s t i n g s from the code that defines PARFIT. PARFIT i s coded in ALGOL W, and thus the l i s t i n g s given below should be f a i r l y easy to read. In the interests of brevity and c l a r i t y , we have replaced a l l input/output statements with descriptive pseudo-statements indicated by %INPUT and %OUTPUT. In these statements, actual variables in the ALGOL W code are enclosed in brackets. PARFIT performs a major portion of i t s input and output using FORTRAN subroutines. This was done mainly to take advantage of the good "user proof" set of FORTRAN c a l l a b l e free format input procedures available at the University of B r i t i s h Columbia. External procedures are indicated by %EXTERNAL in the following l i s t i n g s . The l a s t few pages of l i s t i n g s are devoted to the declaration statements for the external procedures not previously l i s t e d . B rief descriptions of the functions of these procedures are also included. Occasionally, PARFIT performs operations on MTS (Michigan Terminal System) f i l e s . These operations are indicated with %FILE in the following l i s t i n g s . i APPENDIX B 206 %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% Main driving program for PARFIT %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% BEGIN COMMENT MAIN DRIVING PROGRAM FOR PARFIT START BY SETTING UP SIZE INFORMATION; INTEGER N_STATE , N_PAR , N_STATE_OBS , N__OBS , MAX__PTS ; STRING(31) MODEL; STRING(1) ANS; LOGICAL BATCH; LONG REAL INITIAL_TIME; COMMENT %EXTERNAL CMD_AL, CHECK__BATCH; COMMENT %EXTERNAL GEAR, TRAP, SVD_AL, MARQUARDT, PARFIT; COMMENT EXECUTION BEGINS HERE ######################################### #########################################; COMMENT ASSIGN UNIT NUMBERS FOR OUTPUT; COMMENT EMPTY TEMPORARY FILES; COMMENT DETERMINE IF IN BATCH MODE; CHECK_BATCH(BATCH); COMMENT IF IN BATCH OPTIMIZATION/INTEGRATION INFORMATION IS WRITTEN OUT DIRECTLY BY ASSIGNING SAME UNIT NUMBER FOR -SCI AND OUTPUT TO USER; COMMENT %INPUT FROM DATA FILE (MODEL),(INITIAL_TIME),(N_STATE),(N_PAR),(N_STATE_OBS), (N_OBS),(MAX_PTS); PARFIT (N_STATE , N_PAR,N_STATE_OBS ,N__OBS ,MAX_PTS ,MODEL , INITIAL_TIME,BATCH,CHECK_BATCH,GEAR,TRAP,SVD_AL, MARQUARDT,CMD__AL) ; IF BATCH=TRUE THEN COMMENT %FILE TAKE COPY OF -GRAPHSTORE; ELSE BEGIN COMMENT %OUTPUT TO USER IS A LISTING OF FULL ECHO DESIRED? Y OR N; COMMENT %INPUT (ANS); IF ANS="Y" THEN COMMENT %FILE TAKE COPY OF -ECHO; COMMENT %OUTPUT TO USER IS A COPY OF FULL PLOTS AND REPORTS DESIRED? Y OR N; COMMENT %INPUT (ANS); IF ANS="Y" THEN COMMENT %FILE TAKE COPY OF -REPRT, -GRAPHSTORE; COMMENT %OUTPUT TO USER IS A COPY OF INTEG./OPT. AND INTERACTIVE FIT DESIRED Y OR N; COMMENT %INPUT (ANS); APPENDIX B 207 IF ANS="Y" THEN COMMENT %FILE TAKE COPY OF -SC2, -AECHO; END; END. %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% Major procedure defining PARFIT %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE PARFIT(INTEGER VALUE N_STATE,FN_PAR, N_STATE_OBS,N_OBS,MAX_PTS; STRING(31) VALUE MODEL; LONG REAL VALUE INITIAL_TIME; LOGICAL VALUE BATCH; PROCEDURE CHECK_BATCH, GEAR, TRAP, SVD_AL, MARQUARDT, CMD_AL); BEGIN COMMENT MAJOR PROCEDURE IN PARFIT PROCEDURE EG_FUN(LONG REAL VALUE T; LONG REAL ARRAY Y(*); LONG REAL ARRAY P(*); INTEGER VALUE OPTION; LONG REAL ARRAY G ( * ) ; LONG REAL ARRAY DGY(*,*); LONG REAL ARRAY DGP(*,*); LONG REAL ARRAY ISEN(*,*)); BEGIN COMMENT INTERFACE TO USER DEFINED G_FUN TO ALLOW USE OF ENVIRONMENT FOR SCALING AND FREEZING OF PARAMETERS; COMMENT %EXTERNAL G_FUN; INTEGER KK; LONG REAL ARRAY PS(1::FN_PAR); LONG REAL ARRAY TDGP,TISEN(1::N_STATE,1::FN_PAR); IF PFRZ =TRUE THEN BEGIN COMMENT COPY NONFROZEN PARAMETERS AND FROZEN PARAMETERS TO PS; KK:=0; FOR I:=l UNTIL FN_PAR DO IF FRZ(I)=0 THEN BEGIN KK:=KK+1; PS(I):=P(KK); END ELSE PS(I):=FPAR(I); APPENDIX B 208 END ELSE FOR I:=l UNTIL FN_PAR DO PS(I):=P(I); IF PSCL=TRUE THEN COMMENT SCALING PRESENT; FOR I:=l UNTIL FN_PAR DO IF SCL(I)=1 THEN PS(I):=LONGEXP(PS(I)) ELSE IF SCL(I)=2 THEN PS(I):=PS(I)**2; G_FUN(T,Y,PS,OPTION,G,DGY,TDGP,TISEN); CASE ABS OPTION OF BEGIN BEGIN END; BEGIN IF PSCL=TRUE OR PFRZ=TRUE THEN BEGIN COMMENT HANDLE FREEZING AND SCALING; KK:=0; FOR I:=l UNTIL FN_PAR DO IF FRZ(I)=0 THEN BEGIN KK:=KK+1; IF SCL(I)=1 THEN BEGIN FOR J:=l UNTIL N_STATE DO DGP(J,KK) :=TDGP(J,I)*PS (I) ; END ELSE IF SCL(I)=2 THEN BEGIN FOR J:=l UNTIL N_STATE DO DGP(J,KK):=TDGP(J,I)*2.L*P(KK) END ELSE FOR J:=l UNTIL N_STATE DO DGP(J,KK):=TDGP(J,I); END; END ELSE FOR I:=l UNTIL N_STATE DO FOR J:=l UNTIL FN_PAR DO DGP(I,J):=TDGP(I,J); END; BEGIN IF OPTION>0 THEN IF PSCL=TRUE OR PFRZ=TRUE THEN BEGIN COMMENT HANDLE FREEZING AND SCALING; APPENDIX B 209 KK:=0; FOR I:=l UNTIL FN_PAR DO IF FRZ(I)=0 THEN BEGIN KK:=KK+1; IF SCL(I)=1 THEN BEGIN FOR J:=l UNTIL N_STATE DO ISEN(J,KK):=TISEN(J,I)*PS(I); END ELSE IF SCL(I)=2 THEN BEGIN FOR J:=l UNTIL N_STATE DO ISEN(J,KK):=TISEN(J,I)*2.L*P(KK); END ELSE FOR J:=l UNTIL N_STATE DO ISEN(J,KK):=TISEN(J,I); END; END ELSE FOR I:=l UNTIL N_STATE DO FOR J:=l UNTIL FN_PAR DO ISEN (I, J) :=TISEN(I,J) ; END END; END EG_FUN; COMMENT PROCEDURE EXTRACT__JACOBIAN (LONG REAL ARRAY JAC (*,*)) ; BEGIN COMMENT FROM A COMPLETE JACOBIAN AT ALL POINTS HIT AND ON ALL STATE VARIABLES, EXTRACT INTO JAC THE JACOBIAN DIRECTLY ASSOCIATED WITH THE OBSERVATIONS TAKEN, AND INCORPORATING THE WEIGHTS ON THE OBSERVATIONS; INTEGER M; COMMENT SET UP WEIGHTING SCALING OF LEAST SQUARES JAC; FOR J:=l UNTIL N_STATE_OBS DO FOR I:=l UNTIL N_OBS DO WT_VEC ( (1-1) *N_STATE__OBS+J) :=WT(I ,J) ; FOR K:=l UNTIL N_PAR DO BEGIN M:=0; FOR I:=l UNTIL N_PTS__HIT DO IF OBS_STATUS(I)~=0 THEN FOR J:=l UNTIL N_STATE_OBS DO BEGIN M:=M+1; JAC (M,K) :=JACOBIAN ( (I-1) *N__STATE+STATES_OBS (J) ,K) *WT_VEC(M); APPENDIX B 210 END; END; END EXTRACT_JACOBIAN; COMMENT PROCEDURE STANDARD_HIT( LONG REAL VALUE INITIAL_TIME; INTEGER ARRAY OBS_STATUS(*); LONG REAL ARRAY OBS_PLACE(*); INTEGER VALUE N_OBS; LONG REAL ARRAY PTS_TO_HIT(*); INTEGER RESULT N_PTS_HIT); BEGIN COMMENT SET SAMPLE TIMES AS INITIAL TIME AND OBSERVATION TIMES; I:=l; PTS_TO_HIT(l):=INITIAL_TIME; OBS_STATUS(1):=0; IF ABS(OBS_PLACE(1)-INITIAL_TIME)<1.1-5 THEN BEGIN OBS_STATUS(1):=1; I: =0; END; FOR J:=2-I UNTIL N_OBS DO BEGIN PTS_TO_HIT(J+I):=OBS_PLACE(J); OBS_STATUS(J+I):=J; END; N_PTS_HIT:=N_OBS+I; END STANDARD_HIT; COMMENT PROCEDURE COPY_TRANS(LONG REAL ARRAY A(*,*); LONG REAL ARRAY B ( * , * ) ; LONG REAL ARRAY JA(*,*); LONG REAL ARRAY JB(*,*); INTEGER VALUE M,N,JM,JN,AUX_INT); BEGIN COMMENT TO REDUCE PAGING, INTEGRATION PROCEDURES INSERT DATA BY COLUMNS IN NXM ARRAY B,THEN B(TRANSPOSE) IS COPIED TO A TO ACCESS A BY COLUMNS FOR PLOTTING ETC.; FOR J:=l UNTIL N DO FOR I:=l UNTIL M DO A(I,J):=B(J,I); IF AUX_INT=1 THEN FOR J:=l UNTIL JN DO FOR I:=l UNTIL JM DO JA(I,J):=JB(J,I); END COPY_TRANS; COMMENT APPENDIX B 211 COMMENT %EXTERNAL SPLINT_AL, SPLN_AL, DSPLN_AL, HERMIT_AL, HERM, DHERM, ECHOl; PROCEDURE READ_CMD_DATA(INTEGER ARRAY CMD_DATA(*)); BEGIN COMMENT READ A STRING OF INTEGERS SEPARATED BY BLANKS UNTIL A ZERO IS ENCOUNTERED, STORE THE INTEGERS INCLUDING THE ZERO IN THE VECTOR CMD_DATA; INTEGER I; I: =1; COMMENT %INPUT (CMD_DATA(I)); WHILE CMD_DATA(I)~ = 0 DO BEGIN I: =1+1; COMMENT %INPUT (CMD_DATA(I)); END; END READ_CMD_DATA; COMMENT COMMENT %EXTERNAL PLOT_COMMAND; COMMENT VARIABLE DECLARATIONS FOLLOW INTEGER ARRAY SCL,FRZ(1::FN_PAR); LONG REAL ARRAY CREOBS_JOINTS(1::15); LONG REAL ARRAY CREOBS_VAL(1::15); LONG REAL ARRAY CREOBS(1::MAX_PTS); LONG REAL ARRAY DCREOBS(1::MAX_PTS); LONG REAL ARRAY FPAR,START_PAR,SIM_PAR, INT_PAR,GRADIENT,PAR(1::FN_PAR); LONG REAL ARRAY OBS_PLACE , RESIDUAL__VECTOR (1: :N_OBS) ; LONG REAL ARRAY PTS_TO_HIT(1::MAX_PTS); INTEGER ARRAY STATES_OBS(1::N_STATE_OBS); LONG REAL ARRAY OBS(1::N_OBS,1::N_STATE_OBS); LONG REAL ARRAY WT (1: :N__OBS , 1: : N_STATE__OBS) ; REAL ARRAY WT_VEC(1::N_OBS*N_STATE_OBS); LONG REAL ARRAY OBS_SMOOTH (1: :MAX_PTS , 1: :N_STATE__OBS) ; LONG REAL ARRAY JOINTS(1::17,1::N_STATE_OBS); LONG REAL ARRAY SPLN_COEF(1::30,1::N_STATE_OBS); LONG REAL ARRAY HERM_COEF(1::16,1::4,1::N_STATE_OBS); INTEGER ARRAY NJOINTS,N_SPLN_PAR(1::N_STATE_OBS); LONG REAL ARRAY STATE(1::MAX_PTS,1::N_STATE); LONG REAL ARRAY JACOBIAN(1::MAX_PTS*N_STATE,1::FN_PAR); INTEGER DFIT_LIN,NO_CREOBS_JOINTS,CMD_NO,I,N_PTS_HIT; INTEGER ARRAY OBS_STATUS(1::MAX_PTS); LONG REAL ARRAY CON(1::FN_PAR); LONG REAL ARRAY COR,COV(1::FN_PAR,1::FN_PAR); LONG REAL STD_DEV; STRING(6) ARRAY COMMANDS(1::30); STRING(6) CMD;STRING(6) UNIT; LONG REAL FPROB;INTEGER SMF,WT_FLG; LONG REAL EPS,HMIN,HMAX; APPENDIX B 212 INTEGER PLOT_NUMBER,PERCENTILE,AUX_INT,OUTPUT, KFLAG,METHOD_FLAG,OUTPUTJSUP,OUT_SEG,JAC_OPTION, INT_PROC,N_PAR,JJ; STRING(1) ANS; LOGICAL PFRZ,PSCL,ECHO; LONG REAL SUM_SQ_RES; COMMENT ################################################# #################INITIALIZATION################## #################################################; ECHO:=FALSE; OUTPUT:=0; DFIT_LIN:=0; INT_PROC:=2; PLOT_NUMBER:=0; EPS:=.01; HMIN:=.00001; HMAX:=5; METHOD_FLAG:=1; OUTPUTJSUP: =1; OUT_SEG:=0; JAC_OPTION:=0; PFRZ:=FALSE;PSCL:=FALSE; FOR I:=l UNTIL FN__PAR DO FRZ(I):=SCL(I):=0; N_PAR:=FN_PAR; COMMENT DEFINE VECTOR OF COMMAND DESIGNATORS; COMMANDS (1) : = _ II PLOT " COMMANDS (2) : = _ II SET ? COMMANDS (3) : = _ II INTEG " ; COMMANDS (4) : = _ II FIT " ; COMMANDS (5) : = QUIT " ; COMMANDS (6) : = = " REPORT" ; COMMANDS (7) : = _ II CREATE" ; COMMANDS (8) : = 1 1 SAMPLE" ; COMMANDS (9) : = _ II SMOOTH" / COMMANDS (10) : = "DFIT COMMANDS (11) : = "SUSP •I COMMANDS (12) : = "OPTION •I COMMANDS (13) : = "STATS COMMANDS (14) • = "PROBE n COMMANDS (15) = "WEIGHT II COMMANDS (16) = "CONTIN •I COMMANDS (17) = "CREOBS COMMANDS (18) • = "IFIT •I COMMANDS (19) • = "FREEZE •I COMMANDS (20) : = "SCALE II COMMANDS (21) : = "ECHO •I COMMANDS (22) : = "LINEAR II COMMANDS (23) : = "END •I APPENDIX B 213 COMMENT COMMENT %OUTPUT TO USER -REPRT (MODEL); IF MODEL(0|6) ~= "CREATE" THEN BEGIN COMMENT READ IN DATA; COMMENT %INPUT FROM DATA FILE (OBS) , (OBS_PLACE) , (START__PAR) , (STATES_OBS) , (WT) , (WT_FLAG)—0 IF NO WEIGHTS IN DATA FILE; COMMENT SET ALL WEIGHTS TO 1 IF NOT PRESENT IN DATA FILE; IF WT_FLG=0 THEN FOR J:=l UNTIL N_STATE_OBS DO FOR I:=l UNTIL N_OBS DO WT(I,J):=1; COMMENT INITIALIZE PAR TO START_PAR; FOR I:=l UNTIL FN_PAR DO PAR(I):=FPAR(I):=START_PAR(I); END; COMMENT r COMMENT READ COMMAND DESIGNATOR AND IDENTIFY COMMAND; READ_CMD: COMMENT %OUTPUT TO USER ENTER COMMAND; COMMENT %INPUT (CMD); I:=l; WHILE COMMANDS(I)~="END " DO BEGIN IF COMMANDS(I)=CMD THEN BEGIN CMD_NO:=I;GO TO OUT; END; I:=1+1; END; COMMENT %OUTPUT TO USER COMMAND IN ERROR, RESPECIFY; GO TO READ_CMD; COMMENT OUT:IF ECHO=TRUE THEN ECH01(2); CASE CMD_NO OF BEGIN BEGIN COMMENT ### PLOT COMMAND ###; INTEGER KK; KK:=0; FOR I:=l UNTIL FN_PAR DO IF FRZ(I)=0 THEN BEGIN KK:=KK+1; FPAR(I):=PAR(KK); END; PLOT_COMMAND( OBS_PLACE, PTS_TO_HIT, STATE, OBS, OBS_SMOOTH,FPAR, N_OBS ,N__PTS_HIT,N_STATE_OBS ,N_STATE ,FN_PAR, APPENDIX B 214 STATES_OBS,PLOT_NUMBER, READ_CMD_DATA,CMD_AL,CREOBS) ; END; COMMENT BEGIN COMMENT ### SET COMMAND ###; INTEGER I; COMMENT %OUTPUT GIVE LIST OF PARAMETERS AND VALUES (END WITH 0) SUBSCRIPTS CORRESPOND TO UNFROZEN PARAMETER VECTOR % INPUT (IK-WHILE I~=0 DO BEGIN COMMENT %INPUT (FPAR(I)),(I); END; I: =0; FOR J:=l UNTIL FN_PAR DO IF FRZ(J)=0 THEN BEGIN I:=1+1; PAR(I):=FPAR(J); END; END; COMMENT BEGIN COMMENT ### INTEGRATE COMMAND ###; LONG REAL ARRAY INITY(1::1); LONG REAL ARRAY INITYP(1::1,1::1); AUX_INT:=0; COMMENT %OUTPUT TO USER IS THE JACOBIAN DESIRED? Y OR N; COMMENT %INPUT (ANS); IF ANS="Y" THEN AUX_INT:=1; BEGIN LONG REAL ARRAY B (1 : : N__STATE , 1: : N_PTS_HIT) ; LONG REAL ARRAY JB(1::N_PAR,1::N_PTS_HIT*N_STATE); CASE INT_PROC OF BEGIN COMMENT %OUTPUT TO USER INTEGRATION METHOD AND PERTINENT CONTROL PARAMETERS; END; IF OUTPUT~=0 THEN BEGIN COMMENT %FILE EMPTY -SCI; COMMENT %OUTPUT TO -REPRT OUTPUT REFERENCE NUMBER (OUT_SEG); CASE INT_PROC OF BEGIN COMMENT %OUTPUT TO -REPRT APPENDIX B 215 INDICATE INTEGRATION METHOD AND PERTINENT PARAMETERS; END; END; CASE INT_PROC OF BEGIN GEAR(PAR,PTS_TO_HIT,B,JB,N_STATE,N_PTS_HIT, EPS,HMIN,HMAX,N_PAR,AUX_INT,EG_FUN, KFLAG,OUTPUT,METHOD_FLAG,0,INITY,INITYP); TRAP(PAR,PTS_TO_HIT,B,JB,N_STATE,N_PTS_HIT, N_PAR,AUX_INT,EG_FUN,KFLAG,OUTPUT) END; AUX_INT:=1; COPY_TRANS(STATE,B,JACOBIAN,JB,N_PTS_HIT,N_STATE, N_PTS_HIT*N_STATE,N_PAR,AUX_INT); END; JJ:=0; FOR I:=l UNTIL FN_PAR DO IF FRZ(I)=0 THEN BEGIN JJ:=JJ+1; INT_PAR(I):=PAR(JJ); END ELSE INT_PAR(I):=FPAR(I); END; COMMENT BEGIN COMMENT ### FIT COMMAND ###; INTEGER I; LONG REAL ARRAY INITY(1::1); LONG REAL ARRAY INITYP(1::1,1:: 1) ; PROCEDURE FUNC(LONG REAL ARRAY P(*); LONG REAL RESULT F; LONG REAL ARRAY RES(*); LONG REAL ARRAY JAC(*,*); LONG REAL ARRAY GRAD(*); INTEGER RESULT EFLAG); BEGIN COMMENT JACOBIAN GRADIENT AND RESIDUAL INFORMATION; INTEGER M,N_RES,OUTl; LONG REAL SUM; COMMENT INTEGRATE; IF OUTPUT_SUP=l THEN OUTl:=0 ELSE OUT1:=OUTPUT; EFLAG:=0; BEGIN LONG REAL ARRAY B(1::N_STATE,1::N_PTS_HIT); LONG REAL ARRAY JB(1::N_PAR,1::N_PTS_HIT*N_STATE); CASE INT_PROC OF BEGIN APPENDIX B 216 BEGIN GEAR(P,PTS_TO_HIT,B,JB,N_STATE, N_PTS_HIT,EPS,HMIN,HMAX,N_PAR,1, EG_FUN,KFLAG,0UT1, METHOD_FLAG,0,INITY,INITYP); IF KFLAG~=1 THEN BEGIN EFLAG:=1; GO TO OUT; END; END; BEGIN TRAP(P,PTS_TO_HIT, B,JB,N_STATE, N_PTS_HIT,N_PAR,AUX_INT,EG_FUN,EFLAG,OUTl); IF EFLAG=1 THEN GO TO OUT; END END; AUX_INT:=1; COPY_TRANS(STATE,B,JACOBIAN,JB,N_PTS_HIT,N_STATE, N_PTS__HIT*N_STATE,N_PAR,AUX_INT) ; END; EXTRACT_JACOBIAN(JAC); FOR J:=l UNTIL N_STATE_OBS DO FOR I:=l UNTIL N_PTS_HIT DO IF OBS_STATUS(I)~=0 THEN BEGIN INTEGER INDEX; INDEX:=(OBS_STATUS(I)-1)*N_STATE_OBS+J; RES(INDEX):=(STATE(I,STATES_OBS(J))-OBS(OBS_STATUS(I), J))*WT_VEC(INDEX); END; N_RES:=N_STATE_OBS*N_OBS; F:=0; FOR I:=l UNTIL N_RES DO F:=F+RES(I)**2; SUM_SQ_RES:=F; COMMENT FORM GRADIENT; FOR I:=l UNTIL N_PAR DO BEGIN SUM:=0. ; FOR J:=l UNTIL N_RES DO SUM:=SUM+JAC(J,I)*RES(J); GRAD(I):=SUM; END; OUT: END FUNC; COMMENT %OUTPUT CHOOSE METHOD MARQUARDT 1 INTERACTIVE 2; COMMENT %INPUT (I) ; CASE I OF BEGIN BEGIN COMMENT FIT USING MARQUARDT1S TECHNIQUE ; LONG REAL LAM,EPS_R,EPS_A; APPENDIX B 217 COMMENT %OUTPUT TO USER ENTER STARTING LAMBDA, RELATIVE TOLERENCE, AND ABSOLUTE TOLERENCE MARQUARDT; COMMENT %INPUT (LAM),(EPS_R),(EPS_A); COMMENT %FILE EMPTY -SCI; COMMENT %OUTPUT CURRENT OUTPUT REFERENCE NUMBER IS (OUT_SEG) MARQUARDT USED IN FIT COMMAND (LAM),(EPS_R),(EPS_A); MARQUARDT (EPS_R, EPS_A, N_OBS*N_STATE__OBS , N_PAR, FUNC,PAR,LAM,SVD_AL); END; BEGIN COMMENT %EXTERNAL INTERACTIVE_OPT; IF ECHO=TRUE THEN ECH01(3); INTERACTIVE_OPT(N_OBS*N_STATE_OBS,N_PAR,FUNC, PAR,SVD_AL,OBS_PLACE(*), PTS_TO_HIT(*), STATE(*,*), OBS(*,*), OBS_SMOOTH(*,*), CREOBS(*), N_OBS,N_PTS_HIT,N_STATE_OBS, N_STATE, STATES_OBS(*), PLOT_NUMBER, READ_CMD_DATA, CMD_AL, PLOT_COMMAND); IF ECHO=TRUE THEN ECHOl(l); END END; JJ:=0; FOR I:=l UNTIL FN_PAR DO IF FRZ(I)=0 THEN BEGIN JJ:=JJ+1; INT_PAR(I) :=PAR(JJ) ; END ELSE INT_PAR(I):=FPAR(I); END; COMMENT BEGIN COMMENT ### QUIT COMMAND ###; GO TO FINISHED; END; COMMENT BEGIN COMMENT ### REPORT COMMAND ###; INTEGER ARRAY CMD_DATA(1::15); INTEGER II,UNI; COMMENT %OUTPUT TO USER ENTER LIST OF ITEMS TO PRINT (END WITH 0) GENERAL DATA 1 OBS 2 PAR 3 CREATION 4 APPENDIX B 2 PTS AND STATE V A R S — 5 SMOOTHING DATA 6 JACOBIAN 7 INTEG./OPT. DETAILS-8 S T A T I S T I C A L DATA 9 OPTION SETTINGS 10 WEIGHTS 11 GUESSED OBS 12; READ_CMD_DATA(CMD_DATA); FOR IREP:=1 UNTIL 2 DO BEGIN COMMENT % F I L E I F IREP=1 OUTPUT TO USER, I F IREP=2 OUTPUT TO -REPRT; II:=1; WHILE CMD_DATA(II)~=0 DO BEGIN CASE CMD_DATA(II) OF BEGIN BEGIN COMMENT OPTION 1, GENERAL S I Z E DATA; COMMENT %OUTPUT (FN_PAR),(N_STATE),(N_STATE_OBS) (N__OBS) , (STATES_OBS) ; END; BEGIN COMMENT OPTION 2, OBSERVATIONS; COMMENT %OUTPUT ( O B S ) , ( O B S _ P L A C E ) ; END; BEGIN COMMENT OPTION 3, PARAMETER VALUES; INTEGER KK; KK:=0; FOR I : = l UNTIL FN_PAR DO I F FRZ(I)=0 THEN BEGIN KK:=KK+1; F P A R ( I ) : = P A R ( K K ) ; END; COMMENT %OUTPUT (START_PAR),(FPAR) ( F R Z ) , ( S C L ) ; END; BEGIN COMMENT OPTION 4, DATA ON MODEL CREATION; COMMENT %OUTPUT RAMDOM ERROR USED, SIMULATION PARAMETERS (S T D _ D E V ) , ( S I M _ P A R ) ; END; BEGIN COMMENT OPTION 5, TIMES, AND STATE VARIABLES; COMMENT %OUTPUT INTEGRATION PARAMETERS (INT_PAR) SAMPLE TIMES ( P T S _ T O _ H I T ) , INTEGRATION PROCEDURE (INT_PROC) APPENDIX 219 INTEGRATION RESULTS (STATE); END; BEGIN COMMENT OPTION 6, SMOOTHING INFORMATION; CASE SMF OF BEGIN BEGIN COMMENT SPLINE SMOOTHING CASE; COMMENT %OUTPUT SPLINE SMOOTHING JOINTS, SMOOTHED OBSERVATIONS, SMOOTHED DERIVATIVES ON OBSERVED STATE VARIABLES; END; BEGIN COMMENT HERMITE CASE; COMMENT %OUTPUT HERMITE SMOOTHING JOINTS, SMOOTHED OBSERVATIONS, SMOOTHED DERIVATIVES ON OBSERVED STATE VARIABLES; END END; END; BEGIN COMMENT OPTION 7, JACOBIAN OUTPUT; COMMENT %OUTPUT INTEGRATION PARAMETERS (INT_PAR); IF JAC_OPTION=0 THEN COMMENT %OUTPUT FULL JACOBIAN ON ALL STATES AT ALL SAMPLE TIMES; ELSE COMMENT %OUTPUT LEAST SQUARES JACOBIAN; END; BEGIN COMMENT OPTION 8, OPTIMIZATION AND/OR INTEGRATION DATA; IF BATCH=FALSE THEN BEGIN COMMENT %FILE DISPLAY -SCI TO USER IF (UNIT) SPECIFIES OUTPUT TO USER; IF UNIT="-REPRT" THEN BEGIN STRING(1) ANS; COMMENT %OUTPUT TO USER IS INTEGRATION/OPTIMIZATION DATA TO BE ACCUMULATED FOR LATER OUTPUT Y OR N %INPUT (ANS); IF ANS="Y" THEN APPENDIX B 220 BEGIN COMMENT %OUTPUT TO -REPRT DATA ACCUMULATED IN -SC2 OUTPUT REFERENCE NUMBER IS (OUT_SEG) %FILE ACCUMULATE -SCI AT END OF -SC2; OUT_SEG:=OUT_SEG+l; END; END; END; END; BEGIN COMMENT OPTION 9, STATISTICAL DATA; INTEGER NI; Nl:=N_OBS-N_PAR; COMMENT %OUTPUT F DISTRIBUTION VALUE AT (PERCENTILE) PERCENT WITH (N_PAR) DEGREES OF FREEDOM IN NUMERATOR AND (NI) DEGREES OF FREEDON IN DENOMINATOR IS (FPROB), SUM OF SQUARES OF RESIDUALS (SUM_SQ_RES), CRRRELATION MATRIX(COR), COVARIANCE MATRIX (COV), PARAMETERS (PAR), CONFIDENCE INTERVALS (CON); END; BEGIN COMMENT OPTION 10 REPORT ON OPTION SETTINGS; COMMENT %OUTPUT OPTION SETTINGS: OUTPUT FLAG (OUTPUT), OUTPUT SUPPRESSION FLAG (OUTPUT_SUP), JACOBIAN OUTPUT MODE (JAC_OPTION), GEAR PROGRAM (METHOD_FLAG), GEAR PROGRAM EPS (EPS), GEAR PROGRAM MINIMUM AND MAXIMUM STEP (HMIN), (HMAX), INTEGRATION METHOD (INT_PROC); END; BEGIN COMMENT OPTION 11 REPORT ON WEIGHTS; COMMENT %OUTPUT OBSERVATION TIMES (OBS_PLACE), STATES OBSERVED (STATES_OBS), WEIGHTING FACTORS (WT); END; BEGIN COMMENT OPTION 12 CREATED OBSERVTIONS DATA; COMMENT %OUTPUT JOINTS OF INTERPOLATING SPLINE (CREOBS_JOINTS) ORDINATES FOR INTERPOLATING SPLINE (CREOBS_VAL) CREATED OBSERVATIONS (CREOBS) AND DERIVATIVES (DCREOBS) AT SAMPLE TIMES (PTS_TO_HIT); APPENDIX B 221 END END; II:=II+1; END; END; END; COMMENT BEGIN COMMENT ### CREATE COMMAND ###; INTEGER ARRAY INTDATA(1::7); COMMENT %EXTERNAL CREATE_DATA; INTDATA(1):=N_STATE; INTDATA(3):=N_OBS; INTDATA(4):=FN_PAR; INTDATA(5):=N_STATE_OBS; INTDATA(6):=MAX_PTS; INTDATA(7):=METHOD_FLAG; CREATE_DATA(SIM_PAR, START_PAR, FPAR, OBS_PLACE, PTS__TO_HIT, OBS, STATE, JACOBIAN, OBS_STATUS, STATES_OBS, INTDATA, MODEL, EG_FUN, GEAR, STANDARD_HIT,STD_DEV, I N I T I A L J T I M E , KFLAG,OUTPUT); N_PTS_HIT:=INTDATA(2); FOR I:=l UNTIL FN_PAR DO INT_PAR(I) :.=SIM_PAR(I) ; N_PAR:=FN_PAR; END; COMMENT BEGIN COMMENT ### SAMPLE COMMAND ###; COMMENT %EXTERNAL HITPTS; HITPTS(STANDARD_HIT, READ_CMD_DATA, PTS_TO_HIT, N_PTS_HIT,MAX_PTS, OBS_STATUS, OBS_PLACE, N_OBS, INITIAL_TIME); END; COMMENT BEGIN COMMENT ### SMOOTH COMMAND ###; INTEGER MET; COMMENT %OUTPUT SELECT METHOD CUBIC SPLINE 1 CUBIC HERMITE 2 APPENDIX B 222 %INPUT (MET); CASE MET OF BEGIN BEGIN COMMENT SPLINE CASE; INTEGER NJ,DEC-LONG REAL ARRAY W(l::N_OBS); FOR I:=l UNTIL N_OBS DO W(I):=1.; FOR ST:=1 UNTIL N_STATE_OBS DO BEGIN COMMENT %OUTPUT ENTER NUMBER OF JOINTS (MAX 15), AND JOINT POSITIONS FOR STATE VARIABLE (STATES_OBS(ST)) %INPUT (NJ); NJOINTS(ST):=NJ; FOR I:=l UNTIL NJ DO COMMENT %INPUT (JOINTS(I,ST)); DEG:=3; SPLINT_AL(OBS_PLACE,OBS(*,ST),W,N_OBS, SPLN_COEF(*,ST), DEG,JOINTS(*,ST)tNJOINTS(ST)); N_SPLN_PAR(ST):=DEG; COMMENT INSERT VALUES INTO OBS_SMOOTH; FOR I:=l UNTIL N_PTS_HIT DO OBS_SMOOTH (I ,ST) : =SPLN_AL (PTS__TO_HIT (I) , SPLN_COEF(*,ST),JOINTS(*,ST), NJOINTS(ST),N_SPLN_PAR(ST)); END; SMF:=1; END; BEGIN COMMENT HERMITE CASE; INTEGER NJ; LONG REAL ARRAY COEF(1::40,1::4); FOR ST:=1 UNTIL N_STATE_OBS DO BEGIN INTEGER FLAG; REDO: COMMENT %OUTPUT ENTER NUMBER OF JOINTS (MAX 15), AND JOINT POSITIONS FOR STATE VARIABLE (STATES_OBS(ST)) DO NOT INCLUDE FIRST OR LAST OBSERVATION %INPUT (NJ); NJOINTS(ST):=NJ; JOINTS(1,ST):=OBS_PLACE(1); FOR I:=l UNTIL NJ DO COMMENT %INPUT (JOINTS(1+1,ST)); JOINTS(NJ+2,ST):=OBS_PLACE(N_OBS); IF N_OBS<(2*NJ+4) THEN APPENDIX B 223 BEGIN COMMENT %OUTPUT TO USER EXCESS OF ((2*NJ+4-N_OBS)/2) JOINTS; GO TO REDO; END; HERMIT_AL(OBS_PLACE,OBS(*,ST),JOINTS(*,ST), N_OBSfNJ,COEF,40,FLAG); IF FLAG=1 THEN BEGIN COMMENT %OUTPUT SINGULAR PROBLEM—REDISTRIBUTE JOINTS OR REMOVE SOME JOINTS; GO TO REDO; END; FOR J:=l UNTIL 4 DO FOR I:=l UNTIL NJ+1 DO HERM_COEF(I,J,ST):=COEF(I,J); COMMENT INSERT VALUES INTO OBS SMOOTH; FOR I:=l UNTIL N_PTS_HIT DO OBS_SMOOTH (I ,ST) : =HERM (PTS__TO_HIT (I) , HERM_COEF(*,*,ST), JOINTS(*,ST),NJOINTS(ST)); END; SMF:=2; END END; END; COMMENT BEGIN COMMENT ### DFIT COMMAND ###; INTEGER ARRAY INTDATA(1::14); INTEGER METHOD; COMMENT %EXTERNAL DATAFT_COMMAND, DFIT_CRE, DFITITER,SPRGN; INTDATA(1):=N_PTS_HIT; INTDATA(2):=N_STATE_OBS; INTDATA(3):=N_STATE; INTDATA(4):=N_PAR; INTDATA(5):=SMF; INTDATA(6):=OUTPUT; INTDATA (7) : =METHOD__FLAG; INTDATA(9):=OUTPUT_SUP; INTDATA(10):=OUT_SEG; INTDATA(11):=INT_PROC; INTDATA(12):=DFIT_LIN; COMMENT %OUTPUT ENTER METHOD REGULAR 1 USING GUESSED OBS 2 ITERATED USING SPARSE GN 3; APPENDIX B 224 CASE METHOD OF BEGIN DATAFT_COMMAND(EG_FUN, PAR, INTDATA, OBS_SMOOTH, STATES_OBS, PTS_TO_HIT, JOINTS, SPLN_COEF, HERM_COEF, NJOINTS, N_SPLN_PAR,SPLN_AL,DSPLN_AL, HERM, DHERM, INITIAL_TIME,EPS,HMIN,HMAX); DFIT_CRE(EG_FUN, PAR, INTDATA, OBS_SMOOTH, STATES_OBS, PTS_TO_HIT, JOINTS, SPLN_COEF, HERM__COEF, NJOINTS, N_SPLN_PAR,SPLN_AL,DSPLN_AL, HERM,DHERM,CREOBS,DCREOBS); SPRGN(EG_FUN, PAR, INTDATA, OBS_SMOOTH, STATES_OBS, PTS_TO_HIT, JOINTS, SPLN_COEF, HERM_COEF, NJOINTS, N_SPLN_PAR,SPLN_AL,DSPLN_AL, HERM,DHERM,CREOBS,DCREOBS,DFITITER) END; END; COMMENT BEGIN COMMENT ### SUSPEND COMMAND ###; COMMENT SUSPEND EXECUTION AND RETURN TO OPERATING SYSTEM; END; COMMENT BEGIN COMMENT ### OPTION COMMAND ###; INTEGER I; COMMENT %OUTPUT DESIGNATE CONTROL PAR. AND NEW VALUE (0 TO END)" OUTPUT 1 INT. OUTPUT IN OPT. 0-YES, 1-NO 2 JACOBIAN 0-FULL 1-LEAST SQUARES 3 GEAR METHOD 0-ADAMS , 1-STIFF 4 EPS FOR INTEGRATION 5 HMIN FOR INTEGRATION 6 HMAX FOR INTEGRATION 7 INTEGRATION PROCEDURE 1-GEAR, 2-QUICK 8; WHILE I~=0 DO BEGIN CASE I OF BEGIN COMMENT %INPUT DEPENDING ON (I) (OUTPUT),(OUTPUT_SUP),(JAC_OPTION),(METHOD_FLAG), (EPS),(HMIN),(HMAX),(INT_PROC); END; COMMENT %INPUT (I) ; END; END; APPENDIX B 225 COMMENT BEGIN COMMENT ### STATS COMMAND ### THIS COMMAND ASSUMES THE JACOBIAN AND THE SUM OF THE SQUARES OF THE RESIDUAL ARE AVAILABLE AT THE OPTIMUM PARAMETERS OBTAINED; LONG REAL ARRAY INV,VD(1::N_PAR,1::N_PAR); LONG REAL ARRAY UD,JAC(1::N_STATE_OBS*N_OBS,1::N_PAR); LONG REAL ARRAY S(1::N_PAR); INTEGER M; LONG REAL SUM,SUMI,SUMJ,SIG_SQ,E; COMMENT %EXTERNAL FVALUE_AL; M:=N_STATE_OBS*N_OBS; COMMENT %OUTPUT ENTER (INTEGER) PERCENTILE FOR F DISTRIBUTION COMMENT %INPUT (PERCENTILE); FPROB:=FVALUE_AL(1.-PERCENTILE/100.,N_PAR, N_OBS*N_STATE_OBS-N_PAR); COMMENT %OUTPUT (FPROB); EXTRACT_JACOBIAN(JAC); SVD_AL(JAC,S,UD,VD,M,N_PAR,M,N_PAR,0,N_PAR,N_PAR); FOR J:=l UNTIL N_PAR DO FOR I:=l UNTIL N_PAR DO VD(I,J):=VD(I,J)/S(J); COMMENT FORM INVERSE OF (JACOBIAN TRANSPOSE TIMES JACOBIAN); FOR I:=l UNTIL N_PAR DO FOR J:=l UNTIL N_PAR DO BEGIN SUM:=0.; FOR K:=l UNTIL N_PAR DO SUM:=SUM+VD(I,K)*VD(J,K); INV(I,J):=SUM; END; SIG_SQ:=SUM_SQ_RES/(M-N_PAR); E:=N_PAR*FPROB*SIG_SQ; COMMENT CALCULATE CONFIDENCE INTERVALS; FOR I:=l UNTIL N_PAR DO CON(I):=SQRT(E*INV(I,I)); COMMENT CALCULATE COVARIANCE MATRIX; FOR I:=l UNTIL N_PAR DO FOR J:=l UNTIL N_PAR DO COV(I,J):=SIG_SQ*INV(I,J); COMMENT CALCULATE CORRELATION MATRIX; FOR I:=2 UNTIL N_PAR DO FOR J:=l UNTIL 1-1 DO BEGIN SUMI:=0;SUMJ:=0;SUM:=0; FOR K:=l UNTIL N PAR DO APPENDIX B 226 BEGIN SUM:=SUM+VD(I , K)*VD(J,K); SUMI:=SUMI+VD(I ,K)**2; SUMJ:=SUMJ+VD(J,K)**2; END; COR(I,J):=SUM/SQRT(SUMI*SUMJ); COR(J,I):=COR(I,J); END; FOR I:=l UNTIL N_PAR DO COR(I,I):=1.; END; COMMENT BEGIN COMMENT ### PROBE COMMAND ###; STRING(1) ANS; INTEGER M; LONG REAL ARRAY INITY(1::1); LONG REAL ARRAY INITYP(1::1,1::1); LONG REAL ARRAY UD,JAC(1::N_STATE_OBS*N_OBS,1::N_PAR) LONG REAL ARRAY VD(1::N_PAR,1::N_PAR); LONG REAL ARRAY S(1::N_PAR); COMMENT %OUTPUT IS AN INTEGRATION REQUIRED? Y OR N %INPUT (ANS); IF ANS="Y" THEN BEGIN IF OUTPUT~=0 THEN BEGIN COMMENT %FILE EMPTY -SCI %OUTPUT TO -REPRT INTEGRATION IN PROBE OUTPUT REFERENCE NUMBER IS (OUT_SEG); CASE INT_PROC OF BEGIN COMMENT %OUTPUT TO -REPRT INTEGRATION CONTROL PARAMETRERS; END; END; BEGIN LONG REAL ARRAY B(1::N_STATE,1::N_PTS_HIT); LONG REAL ARRAY JB(1::N_PAR,1::N_PTS_HIT*N_STATE) AUX_INT:=1; CASE INT_PROC OF BEGIN BEGIN GEAR(PAR,PTS_TO_HIT,B,JB,N_STATE,N_PTS_HIT, EPS,HMIN,HMAX,N_PAR,1,EG_FUN,KFLAG,OUTPUT, METHOD__FLAG, 0,INITY,INITYP); IF KFLAG~=1 THEN GO TO OUT; END; BEGIN APPENDIX B 227 TRAP(PAR,PTS_TO_HIT,B,JB,N_STATE,N_PTS_HIT, N_PAR,1, EG_FUN,KFLAG,OUTPUT); IF KFLAG=1 THEN GO TO OUT; END END; AUX_INT:=1; COPY_TRANS(STATE,B,JACOBIAN,JB,N_PTS_HIT,N_STATE, N_PTS_HIT*N_STATE,N_PAR,AUX_INT); END; JJ:=0 FOR I:=l UNTIL FN_PAR DO IF FRZ(I)=0 THEN BEGIN JJ:=JJ+1; INT_PAR(I):=PAR(JJ); END ELSE INT_PAR(I):=PAR(I); END; M:=N_STATE_OBS*N_OBS; EXTRACT_JACOBIAN(JAC); SVD_AL(JAC,S,UD,VD,M,N_PAR,M,N_PAR,0,N_PAR,N_PAR); COMMENT %OUTPUT TO USER PARAMETERS (PAR) SINGULAR VALUES (S) IS A REPORT OF PROBE REQUIRED?, Y OR N %INPUT (ANS); IF ANS="Y" THEN BEGIN COMMENT %OUTPUT TO -REPRT ###PROBE COMMAND### PARAMETERS (PAR) SINGULAR VALUES (S); IF OUTPUT~=0 THEN BEGIN COMMENT %OUTPUT IS INTEG OUTPUT TO BE DISPLAYED? Y OR N %INPUT (ANS); IF ANS="Y" THEN BEGIN COMMENT %FILE DISPLAY -SCI TO USER %OUTPUT IS INTEG OUTPUT TO BE ACCUMULATED? Y OR N %INPUT (ANS); IF ANS="Y" THEN BEGIN COMMENT %OUTPUT TO -REPRT INTEGRATION DATA ACCUMULATED IN PROBE OUTPUT REFERENCE NUMBER IS (OUT_SEG) %FILE ACCUMULATE -SCI AT END OF -SC2; OUT_SEG:=OUT_SEG+l; END; APPENDIX B 228 END; END; END; OUT:END; COMMENT BEGIN COMMENT ### WEIGHTING COMMAND ###; INTEGER CAS;INTEGER ARRAY ST_WT(1::N_STATE_OBS+l); INTEGER ST,SUB,SUB1; COMMENT %OUTPUT INDICATE OPTION SET ALL WEIGHTS TO 1 1 INDIVIDUAL ENTRY OF WEIGHTS 2 %INPUT (CAS); CASE CAS OF BEGIN BEGIN FOR J:=l UNTIL N_STATE_OBS DO FOR I:=l UNTIL N_OBS DO WT(I,J):=1; END; BEGIN COMMENT %OUTPUT ENTER LIST OF STATE VARIABLES WHOSE OBSERVATIONS ARE TO BE WEIGHTED. 0 TO END; READ_CMD_DATA(ST_WT); ST:=1; WHILE ST_WT(ST)~=0 DO BEGIN FOR I:=l UNTIL N_STATE_OBS DO IF STATES_OBS(I)=ST_WT(ST) THEN SUB1:=I; COMMENT %OUTPUT ENTER SEQUENCE OF OBSERVATION TIME SUBSCRIPTS AND NEW WEIGHTS FOR STATE VARIABLE (ST_WT(ST)) ENTER A ZERO SUBSCRIPT TO END INPUT %INPUT (SUB); WHILE SUB~=0 DO BEGIN COMMENT %INPUT (WT(SUB,SUBl)),(SUB); END; ST:=ST+1; END; COMMENT %OUTPUT DO YOU WANT THE WEIGHTS STORED IN THE FILE -WEIGHT? Y,N %INPUT (ANS); IF ANS="Y" THEN BEGIN COMMENT %OUTPUT TO -WEIGHT (WT); APPENDIX B 229 END; END END; END; COMMENT BEGIN COMMENT ### CONTINUATION COMMAND ###; INTEGER ARRAY INTDATA(1::15); COMMENT %EXTERNAL CONTI; INTDATA(1):=N_PTS_HIT; INTDATA(2):=N_STATE_OBS; INTDATA (.3) : =N_STATE ; INTDATA(4):=N_PAR; INTDATA(5):=SMF; INTDATA(6):=OUTPUT; INTDATA(7):=METHOD_FLAG; INTDATA(9):=OUTPUT_SUP; INTDATA(10):=OUT_SEG; INTDATA(11):=INT_PROC; INTDATA(12):=DFIT_LIN; CONTI(EG_FUN, PAR,0,INTDATA,OBS_SMOOTH,STATES_OBS, PTS_TO_HIT,JOINTS,SPLN_COEF,HERM_COEF, NJOINTS, N_SPLN_PAR,SPLN_AL,DSPLN_AL, HERM,DHERM,EPS,HMIN,HMAX); END; COMMENT BEGIN COMMENT ### CREOBS COMMAND ### THIS COMMAND ALLOWS THE USER TO CREATE OBSERVATIONS FOR AN UNOBSERVED STATE VARIABLE COMMENT %OUTPUT ENTER NO. OF JOINTS AND JOINT POSITIONS FOR STATE VARIABLE (3-STATES_OBS(1)) INCLUDE FIRST AND LAST SAMPLE TIMES % INPUT (NO_CREOBS__JOINTS) ; FOR I:=l UNTIL NO_CREOBS_JOINTS DO COMMENT %INPUT (CREOBS_JOINTS(I)); COMMENT %OUTPUT ENTER CORRESPONDING ORDINATES FOR CREATED OBS; FOR I:=l UNTIL NO_CREOBS_JOINTS DO COMMENT %INPUT (CREOBS_VAL(I)); BEGIN REAL ARRAY X,Y(1::NO_CREOBS_JOINTS); REAL ARRAY P (1: : 7 *NO__CREOBS_JOINTS) ; REAL ARRAY SI(1::2); REAL ARRAY T,S,SI,S2(1::N_PTS_HIT); COMMENT %EXTERNAL SMT; APPENDIX B 230 FOR I:=l UNTIL NO_CREOBS_JOINTS-l DO P(I):=0; FOR I:=l UNTIL NO_CREOBS_JOINTS DO BEGIN X(I):=CREOBS_JOINTS(I); Y(I):=CREOBS_VAL(I); END; SI (1) : = (Y(2)-Y(1) )/(X(2)-X(l) ) ; SI(2):=(Y(NO_CREOBS_JOINTS)-Y(NO_CREOBS_JOINTS-l))/ (X(NO_CREOBS_JOINTS)-X(NO_CREOBS_JOINTS-l)); FOR I:=l UNTIL N_PTS_HIT DO T(I):=PTS_TO_HIT(I); SMT(X,Y,P,SI,T,S,SI,S2,NO_CREOBS_JOINTS, 1,N_PTS_HIT);' FOR I:=l UNTIL N_PTS_HIT DO BEGIN CREOBS(I):=S(I); DCREOBS(I) :=S1 (I) ; END; END; END; COMMENT BEGIN COMMENT ### IFIT COMMAND ###; INTEGER ARRAY INTDATA(1::14); INTEGER METHOD; COMMENT % EXTERNAL CONT, IFIT__CRE , IFITI, SPRGN; INTDATA(1):=N_PTS_HIT; INTDATA(2):=N_STATE_OBS; INTDATA(3):=N_STATE; INTDATA(4):=N_PAR; INTDATA(5):=SMF; INTDATA(6):=OUTPUT; INTDATA(7):=METHOD_FLAG; INTDATA(9):=OUTPUT_SUP; INTDATA(10):=OUT_SEG; INTDATA(11):=INT_PROC; INTDATA(12):=DFIT_LIN; COMMENT %OUTPUT ENTER METHOD REGULAR 1 USING GUESSED OBS 2 ITERATED INTEG. SUBSYSTEMS 3 ITERATED USING SPARSE GN 4 %INPUT (METHOD); CASE METHOD OF BEGIN CONTI(EG_FUN, PAR,1,INTDATA,OBS_SMOOTH,STATES_OBS, PTS_TO_HIT,JOINTS,SPLN_COEF,HERM_COEF, NJOINTS, N_SPLN_PAR,SPLN_AL,DSPLN_AL, APPENDIX B 231 HERM,DHERM,EPS,HMIN,HMAX); IFIT_CRE(EG_FUN, PAR, INTDATA, OBS_SMOOTH, STATES_OBS, PTS_TO_HIT, JOINTS, SPLN_COEF, HERM_COEF, NJOINTS, N_SPLN_PAR,SPLN_AL,DSPLN_AL, HERM,DHERM,CREOBS,DCREOBS); IFITI(EG_FUN, PAR, INTDATA, OBS_SMOOTH, STATES_OBS, PTS_TO_HIT, JOINTS, SPLN_COEF, HERM_COEF, NJOINTS, N_SPLN_PAR,SPLN_AL,DSPLN_AL, HERM,DHERM,CREOBS,DCREOBS,IFIT_CRE); SPRGN(EG_FUN, PAR, INTDATA, OBS_SMOOTH, STATES_OBS, PTS_TO_HIT, JOINTS, SPLN_COEF, HERM_COEF, NJOINTS, <N_SPLN_PAR,SPLN_AL,DSPLN_AL, HERM,DHERM,CREOBS,DCREOBS,IFIT_CRE) END; END; COMMENT r BEGIN COMMENT ### FREEZE COMMAND ###; INTEGER SUB,KK; KK:=0; FOR I:=l UNTIL FN_PAR DO IF FRZ(I)=0 THEN BEGIN KK:=KK+1; FPAR(I):=PAR(KK); END; COMMENT %OUTPUT ENTER LIST OF SUBSCRIPTS FOR FROZEN PARAMETERS (0 TO END) (AN ENTRY OF 0 REMOVES ALL FREEZING) %INPUT (SUB); IF SUB=0 THEN BEGIN PFRZ:=FALSE; FOR I:=l UNTIL FN_PAR DO FRZ(I):=0; FOR I:=l UNTIL FN_PAR DO PAR(I):=FPAR(I); N_PAR:=FN_PAR; END ELSE BEGIN PFRZ:=TRUE; WHILE SUB~=0 DO BEGIN FRZ(SUB):=1; COMMENT %INPUT (SUB); END; END; KK:=0; APPENDIX B 232 FOR I:=l UNTIL FN_PAR DO IF FRZ(I)=0 THEN BEGIN KK:=KK+1; PAR(KK):=FPAR(I); END; N_PAR:=KK; END; COMMENT BEGIN COMMENT ### SCALE COMMAND ###; INTEGER SUB,KK; COMMENT UPDATE FPAR FOR RESCALING; KK:=0; FOR I:=l UNTIL FN_PAR DO IF FRZ(I)=0 THEN BEGIN KK:=KK+1; FPAR(I):=PAR(KK); END; COMMENT DESCALE FPAR IN PREPARATION FOR RESCALING; FOR I:=l UNTIL FN_PAR DO IF SCL(I)=1 THEN FPAR(I):=LONGEXP(FPAR(I)) ELSE IF SCL(I)=2 THEN FPAR(I):=FPAR(I)**2; COMMENT %OUTPUT ENTER LIST OF PARAMETER SUBSCRIPTS AND SCALING INDICATORS (0 SUBSCRIPT TO END) INDICATOR OF 1 GIVES LOG SCALING INDICATOR OF 2 GIVES SQUARE ROOT SCALING %INPUT (SUB); IF SUB=0 THEN BEGIN PSCL:=FALSE; FOR I:=l UNTIL FN_PAR DO SCL(I):=0; END ELSE BEGIN PSCL:=TRUE; WHILE SUB~=0 DO BEGIN COMMENT %INPUT (SCL(SUB)),(SUB); END; END; COMMENT RESCALE PARAMETER VALUES; FOR I:=l UNTIL FN_PAR DO IF SCL(I)=1 THEN FPAR(I):=LONGLN(FPAR(I)) ELSE IF SCL(I)=2 THEN FPAR(I):=LONGSQRT(FPAR(I)); COMMENT UPDATE PAR; APPENDIX B 233 KK:=0; FOR I:=l UNTIL FN_PAR DO IF FRZ(I)=0 THEN BEGIN KK:=KK+1; PAR(KK):=FPAR(I); END; END; COMMENT BEGIN COMMENT ### ECHO COMMAND ###; INTEGER INT; COMMENT %OUTPUT ENTER 1 TO BEGIN ECHO, 0 TO END ECHO %INPUT (INT); IF INT=1 THEN ECHO:=TRUE ELSE ECHO:=FALSE; IF INT=1 THEN ECHOl(l) ELSE ECHOl(3); END; COMMENT BEGIN COMMENT ### LINEAR COMMAND ### TO INDICATE LINEARITY IN IFIT, DFIT; COMMENT %OUTPUT ENTER 1 IF IFIT, DFIT GIVE LINEAR PROBLEMS ENTER 0 TO REMOVE LINEARITY SETTING %INPUT (DFIT_LIN); END END; GO TO READ_CMD; FINISHED: IF ECHO=TRUE THEN ECHOl(3); END PARFIT. %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% Procedure to modify sample times %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE HITPTS (PROCEDURE S T AND A RD__H IT, READ_CMD_DATA; LONG REAL ARRAY PTS_TO_HIT(*); INTEGER VALUE RESULT N_PTS_HIT; INTEGER VALUE MAX_PTS; INTEGER ARRAY OBS_STATUS(*); LONG REAL ARRAY OBS_PLACE(*); INTEGER VALUE N_OBS; LONG REAL VALUE INITIAL_TIME); BEGIN APPENDIX B 234 COMMENT MODIFICATION OF SAMPLE TIMES (DEFAULT, UNIFORM MESH, OR INSERTION); INTEGER HIT_OPTION; COMMENT %OUTPUT TO USER PICK OPTION OBS PTS+INITIAL TIME--1 UNIFORM MESH 2 INTERACTIVE INSERTION-3 %INPUT SELECTION IN (HIT_OPTION); CASE HIT_OPTION OF BEGIN ' BEGIN COMMENT STANDARD SCHEME—OBSERVATION POINTS PLUS INITIAL TIME; STANDARD_HIT(INITIAL_TIME,OBS_STATUS,OBS_PLACE,N_OBS, PTS_TO_HIT,N_PTS_HIT); END; BEGIN COMMENT A UNIFORM MESH STARTING WITH INITIAL_TIME; INTEGER N;LONG REAL DEL; COMMENT %OUTPUT TO USER ENTER NUMBER OF POINTS AND POINT SPACING %INPUT (N),(DEL); PTS_TO_HIT(l):=INITIAL_TIME; FOR I:=l UNTIL N DO PTS_TO_HIT(1+1):=INITIAL_TIME+I*DEL; N_PTS_HIT:=N+1; END; BEGIN COMMENT INTERACTIVE INSERTION OF POINTS BETWEEN EXISTING POINTS; INTEGER L1,L2,K,J; LONG REAL ARRAY NEW_PTS_TO_HIT(1::MAX_PTS); INTEGER ARRAY NEW_OBS_STATUS(1::MAX_PTS); INTEGER ARRAY CMD_DATA(1::21); COMMENT %OUTPUT TO USER IS A LISTING OF SAMPLE TIMES REQUIRED? IF NOT ENTER 0, IF YES ENTER SUBSCRIPT LIMITS %INPUT (LI); IF L1~=0 THEN BEGIN COMMENT %INPUT (L2) %OUTPUT TO USER LIST OF SAMPLE TIMES BETWEEN SUBSCRIPTS L l AND L2; END; COMMENT %OUTPUT TO USER ENTER SEQUENCE OF UPPER INDICIES AND NUMBER OF TIMES TO INSERT BETWEEN INDICATED TIME AND PREVIOUS TIME (END WITH 0).; READ_CMD_DATA(CMD_DATA); K:=l; APPENDIX B 235 J:=0; FOR I:=l UNTIL N_PTS_HIT DO IF CMD_DATA(K)=I THEN BEGIN COMMENT INSERTION OF POINTS; LONG REAL DEL; INTEGER N_INS; N_INS:=CMD_DATA(K+l); DEL: = (PTS_TO__HIT(I) -PTS_TO_HIT (I-1) )/(N_INS+l) ; FOR R:=l UNTIL N_INS DO BEGIN J:=J+1; NEW_PTS_TO_HIT(J):=PTS_TO_HIT(1-1)+R*DEL; NEW_OBS_STATUS(J):=0; END; J:=J+1; NEW_PTS_TO_HIT(J):=PTS_TO_HIT(I); NEW_OBS_STATUS(J):=OBS_STATUS(I); K:=K+2; END ELSE BEGIN COMMENT COPYING OF OLD POINT; J:=J+1; NEW_PTS__TO_HIT (J):=PTS_TO_HIT(I); NEW_OBS_STATUS(J):=OBS_STATUS(I); END; N_PTS_HIT:=J; COMMENT COPY NEW ARRAYS; FOR I:=l UNTIL N_PTS_HIT DO BEGIN OBS_STATUS(I):=NEW_OBS_STATUS(I); PTS_TO_HIT(I):=NEW_PTS TO HIT(I); END; END END; END HITPTS. % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % ! Trapezoidal integration procedure %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%! PROCEDURE TRAP(LONG REAL ARRAY PAR(*); LONG REAL ARRAY PTS_TO_HIT(*); LONG REAL ARRAY STATE(*,*); LONG REAL ARRAY JACOBIAN(*,*); INTEGER VALUE N_STATE,N_PTS_HIT,N_PAR,AUX_INT; PROCEDURE FUN; APPENDIX B 236 INTEGER RESULT EFLAG; INTEGER VALUE OUTPUT); BEGIN LONG REAL ARRAY PREV_G,G,PREV_Y,Y(1::N_STATE); LONG REAL ARRAY PREV_DGP,DGP,PREV_SENSE, SENSE(1::N_STATE,1::N_PAR); LONG REAL ARRAY PREV_DGY,DGY(1::N_STATE,1::N_STATE); LONG REAL T,PREV_T; INTEGER OUT_COUNT,M; LONG REAL CRIT; PROCEDURE INT_STEP; BEGIN COMMENT INTEGRATE WITH TRAP. RULE FROM PREV_T TO T; COMMENT IF AUX_INT IS NOT 0, INTEGRATE SENSITIVITY EQNS; LONG REAL ARRAY RES(1::N_STATE); LONG REAL ARRAY X,DGY_DYP, B (1::N_STATE,1::N_PAR); LONG REAL ARRAY TMP,A(1::N_STATE,1::N_STATE); INTEGER ARRAY IPERM(1::2*N_STATE); LONG REAL H,SUM,SSQD,SSQ; INTEGER MAXIT; COMMENT %EXTERNAL FSLE_AL; MAXIT:=12; H:=T-PREV_T; COMMENT NEWTON METHOD ON NONLINEAR SYSTEM FOR TIME STEP; FOR I:=l UNTIL N_STATE DO Y(I):=PREV_Y(I); FOR I:=l UNTIL MAXIT DO BEGIN FUN(T, Y,PAR,1,G,DGY,DGP,SENSE); COMMENT FORM JACOBIAN FOR NONLINEAR SYSTEM; FOR I:=l UNTIL N_STATE DO FOR J:=l UNTIL N_STATE DO A(I,J):=-DGY(I,J)*H/2.; FOR I:=l UNTIL N_STATE DO A(I,I):=1.+A(I,I); FOR I:=l UNTIL N_STATE DO B (1,1):=-Y(I)+PREV_Y(I)+H/2.*(G(I)+PREV_G(I)); FSLE_AL(N_STATE,N_STATE,A,1,N_STATE, B, X,IPERM,N_STATE,TMP); COMMENT UPDATE AND GET SUP NORM; SSQD:=0.; SSQ:=0.; FOR I:=l UNTIL N_STATE DO BEGIN SSQD:=SSQD+X(1,1)**2; Y(I) :=X(I,1)+Y(I) ; SSQ:=SSQ+Y(I)**2; END; IF SSQ<1.'-3L THEN SSQ:=1.'-3L; IF (SSQD/SSQ)<(CRIT**2) THEN GO TO OUT; END; COMMENT %OUTPUT TO USER ABNORMAL EXIT IN NEWTON IN INTEGRATION STEP; APPENDIX B 237 EFLAG:=1; GO TO STOP; OUT:IF AUX_INT~=0 THEN BEGIN COMMENT THIS BLOCK SOLVES AUXILIARY LINEAR PROBLEMS; FUN(T,Y,PAR,2,G,DGY,DGP,SENSE); FOR I:=l UNTIL N_STATE DO FOR J:=l UNTIL N_PAR DO BEGIN SUM:=0.; FOR K:=l UNTIL N_STATE DO SUM:=SUM+PREV_DGY(I,K)*PREV_SENSE(K,J); DGY_DYP(I,J):=SUM; END; COMMENT SET UP RIGHT HAND SIDES FOR DISCRETE LINEAR PROB; FOR I:=l UNTIL N_STATE DO FOR J:=l UNTIL N_PAR DO B(I,J):=PREV_SENSE(I,J)+H/2.*(PREV_DGP(I,J) +DGY_DYP(I,J)+DGP(I,J)); COMMENT SOLVE SYSTEMS; FSLE_AL(N_STATE,N_STATE,A,N_PAR,N_STATE,B,SENSE,IPERM, N_STATE,TMP); END; END INT_STEP; COMMENT MAIN PROGRAM STARTS HERE; CRIT:=1-'-5L; EFLAG:=0; OUT_COUNT:=0; M:=0; COMMENT SET UP INITIAL CONDITIONS; PREV_T:=PTS_TO_HIT(1); FUN(PREV_T,PREV_Y,PAR,3,G,DGY,DGP,PREV_SENSE); FOR I:=l UNTIL N_STATE DO STATE(I,1):=PREV_Y(I); FUN(PREV_T,PREV_Y,PAR,1,PREV_G, PREV_DGY,DGP,SENSE); IF AUX_INT~=0 THEN BEGIN FUN(PREV_T,PREV_Y,PAR,2,PREV_G, PREV_DGY,PREV_DGP,SENSE); FOR J:=l UNTIL N__STATE DO BEGIN M:=M+1; FOR L:=l UNTIL N_PAR DO JACOBIAN (L,M) : =PREV__SENSE (J , L) ; END; END; FOR I:=2 UNTIL N_PTS_HIT DO BEGIN T:=PTS_TO__HIT (I) ; INT_STEP; COMMENT OUTPUT OPTION; APPENDIX B 238 I F OUTPUT~=0 THEN BEGIN OUT__COUNT: =0UT_C0UNT+1; I F OUT_COUNT=OUTPUT THEN BEGIN COMMENT OUTPUT TO -SCI TIME AND STATE VARIABLES; OUT_COUNT:=0; END; END; COMMENT INSERT DATA INTO STATE AND UPDATE; FOR J : = l UNTIL N_STATE DO BEGIN S T A T E ( J , I ) : = P R E V _ Y ( J ) : = Y ( J ) ; P R E V _ G ( J ) : = G ( J ) ; PREV_T:=T; FOR K : = l UNTIL N_STATE DO P R E V _ D G Y ( J , K ) : = D G Y ( J , K ) ; END; COMMENT INSERT DATA INTO JACOBIAN I F REQUIRED; I F AUX_INT~=0 THEN BEGIN FOR J : = l UNTIL N_STATE DO FOR K : = l UNTIL N_PAR DO BEGIN P R E V _ S E N S E ( J , K ) : = S E N S E ( J , K ) ; P R E V _ D G P ( J , K ) : = D G P ( J , K ) ; END; FOR J : = l UNTIL N_STATE DO BEGIN M:=M+1; FOR L : = l UNTIL N_PAR DO J A C O B I A N ( L , M ) : = S E N S E ( J , L ) ; END; END; END; STOP:END TRAP. %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% S e n s i t i v i t y e q u a t i o n s o l u t i o n and i n t e r f a c e t o G e a r ' s c o d e %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE GEAR(LONG REAL ARRAY P ( * ) ; LONG REAL ARRAY P T S ( * ) ; LONG REAL ARRAY Y _ P T S ( * , * ) ; LONG REAL ARRAY J A C ( * , * ) ; INTEGER VALUE N,N_PTS; LONG REAL VALUE EPS,HMIN,HMAX; INTEGER VALUE N_PAR,AUX_INT; APPENDIX B 239 PROCEDURE FUN; INTEGER RESULT KFLAG; INTEGER VALUE OUTPUT; INTEGER VALUE METHOD_FLAG; INTEGER VALUE INIT; LONG REAL ARRAY INITY(*); LONG REAL ARRAY INITYP(*,*)); BEGIN REAL ARRAY PW(1::N*N); LONG REAL ARRAY LY(1::8,1::N); LONG REAL ARRAY LSAVE(1::10,1::N+3); COMMENT ASSUMING N IS AT MOST 15 IN DECLARATION OF SAVE; LONG REAL ARRAY LYMAX,LERROR(1::N); LONG REAL LT,LH,LHMIN,LHMAX,LEPS,LNEXT_PT,HPl,HP2,LTS; INTEGER JSTART,MAXDER,MAX_STEP,STEP_COUNT,PT; INTEGER OUT_COUNT; LONG REAL ARRAY G(1::N); LONG REAL ARRAY DGY(1::N,1::N); LONG REAL ARRAY DGP,SENSE(1::N,1::N_PAR); LONG REAL ARRAY Z(1::N); COMMENT DECLARATIONS FOR HERMITE INTERPOLATION; LONG REAL ARRAY HERM_SAVE(1::2,1::N,1::1+N_PAR); LONG REAL ARRAY INTERP(1::4,1::N,1::1+N_PAR); COMMENT DECLARATIONS FOR AUXILIARY INTEGRATIONS; LONG REAL ARRAY STORE_DER(1::8,1::N,1::N_PAR); LONG REAL ARRAY AA(1::8); INTEGER PASS,ORDER; INTEGER MJAC; COMMENT %EXTERNAL DIFF; LONG REAL PROCEDURE POLYVAL(LONG REAL VALUE X; LONG REAL ARRAY COEF(*); INTEGER VALUE N_COEF); BEGIN LONG REAL Y; Y:=0.; FOR I:=N_COEF STEP -1 UNTIL 2 DO Y:=X*(COEF(I)+Y); Y+COEF(1) END POLYVAL; PROCEDURE INTERPOLATE_COEF; BEGIN LONG REAL ARRAY COEF(1::4); LONG REAL ARRAY DATA(1::6); PROCEDURE HERMITE(LONG REAL ARRAY DATA(*); LONG REAL ARRAY COEF(*)); BEGIN COMMENT CUBIC HERMITE INTERPOLATION. DATA CONTAINS ABSCISSAE, ORDINATES, AND SLOPES IN PAIRS. COEF CONTAINS THE COEFFICIENTS STARTING WITH THE CONSTANT TERM IN THE CUBIC; APPENDIX B 240 COEF (4) : = (DATA(5)+DATA(6) ) / (DATA (2 ) -DATA (1') ) **2 -2.0L*(DATA(4) -DATA(3))/(DATA(2)-DATA (1))**3; COEF(3):=(DATA(6)-DATA(5))/(2.0L*(DATA(2)-DATA(1))) -1.5L*(DATA(2)+DATA(1))*COEF(4); COEF(2):=DATA(5)-3.0L*DATA(1)**2*COEF(4) -2.0L*DATA(1)*COEF(3); COEF(l):=DATA(3)-DATA(1)**3*COEF(4)-DATA(1)**2*COEF(3) -DATA(1)*COEF(2); END HERMITE; DATA(1):=LT-HPl; DATA(2):=LT; FOR I:=l UNTIL N DO BEGIN DATA(3):=HERM_SAVE(1,1,1); DATA(4):=LY(1,1); DATA(5):=HERM_SAVE(2,1,1); DATA(6):=LY(2,I)/HP1; HERMITE(DATA,INTERP(*,1,1) ) ; IF AUX_INT~=0 THEN FOR K:=l UNTIL N_PAR DO BEGIN COMMENT INTERPOLATION ON AUXILIARY PROBLEMS; DATA(3):=HERM_SAVE(1,1,K+l); DATA(4):=STORE_DER(l,I,K); DATA(5):=HERM_SAVE(2,1,K+l); DATA(6):=STORE_DER(2,I,K)/HPl; HERMITE(DATA,INTERP(*,I,K+l)); END; END; END INTERPOLATE_COEF; COMMENT MANAGEMENT PROGRAM STARTS HERE; OUT_COUNT:=0; PASS:=1; MJAC:=0; MAX_STEP:=200; COMMENT ONLY USE UP TO FOURTH ORDER METHOD CONSISTENT WITH INTERPOLATION; MAXDER:=4; LHMAX:=HMAX; LHMIN:=HMIN; LEPS:=EPS; PT:=2; LH:=(PTS(2)-PTS(1))/3; COMMENT MAKE SURE STARTING H IS LESS THAN .1; IF LH>.1 THEN LH:=.l; COMMENT FOR USE WHEN AUX_INT~=0; HP2:=LH; COMMENT INITIALIZE; IF INIT=1 THEN BEGIN FOR I:=l UNTIL N DO Z(I):=INITY(I); END ELSE FUN(PTS(1),Z,P,-3,G,DGY,DGP,SENSE); APPENDIX B 241 FOR I:=l UNTIL N DO BEGIN LY(1,I):=Z(I); Y_PTS(1,1):=Z (I) ; LYMAX(I):=1.; END; COMMENT INITIALIZE NON JACOBIAN PART OF HERM_SAVE; FOR I:=l UNTIL N DO HERM_SAVE(1,I,1):=Z(I); FUN(PTS(1),Z,P,-1,G,DGY,DGP,SENSE); FOR I:=l UNTIL N DO HERM_SAVE(2,1,1):=G(I); LT:=PTS (1) ; STEP_COUNT:=0; LNEXT_PT:=PTS(PT); JSTART:=0; COMMENT USE MULTISTEP METHOD SUITABLE FOR STIFF PROBLEMS; WHILE PT<=N_PTS DO BEGIN STEP_COUNT:=STEP_COUNT+l; IF STEP_COUNT>MAX_STEP THEN BEGIN COMMENT IOUTPUT TO USER OVER (MAX_STEP) STEPS USED IN GEAR—GAVE UP; GO TO STOP; END; COMMENT FOR USE WITH AUX_INT~=0; LTS:=LT; DIFF(N,LT,LY,LSAVE,LH,LHMIN,LHMAX,LEPS,METHOD_FLAG,LYMAX, LERROR,KFLAG,JSTART,MAXDER,AA,ORDER,FUN,P,PW); COMMENT PROCESS COMPLETION CODE; IF KFLAG<0 THEN BEGIN CASE ABS KFLAG OF BEGIN COMMENT %OUTPUT PRINT TO USER LINE INDEXED BY KFLAG H=HMIN USED, ERROR NOT ATTAINED MAX ORDER SPECIFIED IS TOO LARGE NO CORRECTOR COMVERGENCE FOR H>HMIN REQUESTED ERROR TOO SMALL FOR PROB.; END; IF OUTPUT~=0 THEN CASE ABS KFLAG OF BEGIN COMMENT %OUTPUT TO -SCI THE SAME MESSAGE AS SENT TO USER IN LAST OUTPUT STATEMENT; END; GO TO STOP; END; HPl:=LT-LTS; LTS:=LT; IF AUX INT~=0 THEN APPENDIX B 242 BEGIN INTEGER ARRAY IPERM(1::2*N); LONG REAL ARRAY TMPR,U(1::N,1::N); LONG REAL ARRAY B,F_PRED(1::N,1::N_PAR); COMMENT %EXTERNAL FSLE_AL; COMMENT AUXILIARY SYSTEMS BEING INTEGRATED; IF PASS=1 THEN BEGIN COMMENT FIRST PASS INITIALIZE VARIABLES; IF INIT=1 THEN BEGIN FOR J:=l UNTIL N DO FOR K:=l UNTIL N_PAR DO SENSE(J,K):=INITYP(J,K); END ELSE FUN(PTS(1),Y_PTS(1,*),P,3,G,DGY,DGP,SENSE); FOR J:=l UNTIL N DO FOR K:=l UNTIL N_PAR DO STORE_DER(1,J,K):=SENSE(J,K); FOR J:=l UNTIL N DO BEGIN MJAC:=MJAC+1; FOR L:=l UNTIL N_PAR DO JAC(L,MJAC):=STORE_DER(l,J,L); END; COMMENT INITIALIZE FIRST DERIVATIVE PART OF STORE_DER; FUN(PTS(1),Y_PTS(1,*),P,1,G,DGY,DGP,SENSE); FUN(PTS(1),Y_PTS(1,*),P,2,G,DGY,DGP,SENSE); FOR K:=l UNTIL N_PAR DO FOR I:=l UNTIL N DO BEGIN LONG REAL SUM; SUM:=0.; FOR J:=l UNTIL N DO SUM:=SUM+DGY(I,J)*STORE_DER(1,J,K); STORE_DER(2,I,K):=HP2*(SUM+DGP(I,K)); END; COMMENT INITIALIZE PART OF HERM_SAVE FOR JACOBIAN; FOR J:=l UNTIL N DO FOR K:=l UNTIL N_PAR DO BEGIN HERM_SAVE(1,J,K+l):=STORE_DER(1,J,K); HERM_SAVE(2,J,K+l):=STORE_DER(2,J,K)/HP2; END; PASS:=2; END; IF HP1~=HP2 THEN BEGIN COMMENT STEP HAS CHANGED AND STORED DERIVATIVES NEED RESCALING; APPENDIX B 243 LONG REAL RATIO,FACTOR; FACTOR:=1.L; RATIO:=HP1/HP2; FOR I:=2 UNTIL ORDER+1 DO BEGIN FACTOR:=FACTOR*RATIO; FOR J:=l UNTIL N DO FOR K:=l UNTIL N_PAR DO STORE_DER(I,J,K):=STORE_DER(I,J,K)*FACTOR; END; END; COMMENT UPDATE HP2; HP2:=HP1; COMMENT FIND PREDICTED VALUE BY MULTIPLYING STORE_DER BY PASCAL TRIANGLE MATRIX; FOR K:=l UNTIL N_PAR DO FOR J:=2 UNTIL ORDER+1 DO FOR J1:=J UNTIL ORDER+1 DO BEGIN INTEGER J2; J2:=ORDER-Jl+J; FOR I:=l UNTIL N DO STORE_DER(J2,I,K):=STORE_DER(32,I,K) +STORE_DER(J2+l,I,K); END; COMMENT SAVE HPl TIMES PREDICTED R.H.S. TO DE'S IN F_PRED; FOR J:=l UNTIL N DO FOR K:=l UNTIL N_PAR DO F_PRED(J,K):=STORE_DER(2,J,K); COMMENT SET UP R.H. SIDES FOR ALGEBRAIC LINEAR SYSTEMS; FOR J:=l UNTIL N DO FOR K:=l UNTIL N_PAR DO B(J,K):=STORE_DER(l,J,K)-AA(l)*STORE_DER(2,J,K); COMMENT ADD IN INHOMOGENEOUS TERMS; FOR I:=l UNTIL N DO Z(I):=LY(1,1); FUN(LT,Z,P,2,G,DGY,DGP,SENSE); FOR K:=l UNTIL N_PAR DO FOR J:=l UNTIL N DO B(J,K):=B(J,K)+HP1*AA(1)*DGP(J,K); COMMENT SET UP MATRIX FOR LINEAR PROBLEMS; FOR I:=l UNTIL N DO Z(I):=LY(1,1); FUN(LT,Z,P,1,G,DGY,DGP,SENSE); FOR I:=l UNTIL N DO FOR J:=l UNTIL N DO U(I,J):=-HPl*AA(1)*DGY(I,J); FOR I:=l UNTIL N DO U(I,I):=U(I,I)+1.; COMMENT SOLVE LINEAR SYSTEMS; FSLE_AL(N,N,U,N_PAR,N,B,SENSE,IPERM,N,TMPR); COMMENT STORE SOLUTION AT NEW TIME IN STORE_DER; FOR J:=l UNTIL N DO APPENDIX B 244 FOR K:=l UNTIL N_PAR DO STORE_DER(1, J , K):=SENSE(J,K); COMMENT FIND R.H.S. OF DE 1S AT NEW POINT; FOR K:=l UNTIL N_PAR DO FOR I:=l UNTIL N DO BEGIN LONG REAL SUM; SUM:=0.; FOR J:=l UNTIL N DO SUM:=SUM+DGY(I,J)*STORE_DER(1,J,K); STORE_DER(2,I,K):=HP1*(SUM+DGP(I,K)); END; COMMENT CORRECT HIGHER DERIVATIVES; IF ORDER>1 THEN FOR I:=3 UNTIL ORDER+1 DO FOR J:=l UNTIL N DO FOR K:=l UNTIL N_PAR DO STORE_DER(I,J,K):=STORE_DER(I,J,K)+AA(I) *(STORE_DER(2,J,K)-F_PRED(J,K)); COMMENT IF REQUIRED INSERT NEXT HIGHER DERIVATIVES IN STORE_DER; IF JSTART>ORDER THEN FOR J:=l UNTIL N DO FOR K:=l UNTIL N_PAR DO STORE_DER(ORDER+2,J,K):=AA(ORDER+1)*(STORE_DER(2,J,K) -F_PRED(J,K))/(ORDER+1); END; COMMENT OUTPUT OPTION; IF OUTPUT~=0 THEN BEGIN OUT_COUNT:=OUT_COUNT+l; IF OUT_COUNT=OUTPUT THEN BEGIN LONG REAL ARRAY V(1::N); LONG REAL TIME; FOR I:=l UNTIL N DO V (I) :=LY(1,1) ; TIME:=LT; COMMENT %OUTPUT TO -SCI TIME AND STATE VARIABLES; OUT_COUNT:=0; END; END; COMMENT CHECK IF PASSED A POINT; IF LT> =LNEXT_PT THEN BEGIN LONG REAL E; INTERPOLATE_COEF; WHILE LT>=LNEXT_PT DO BEGIN E : = (PTS(PT)-LT)/HP1; COMMENT INSERT DATA INTO Y_PTS; APPENDIX B 245 FOR I:=l UNTIL N DO Y_PTS(I,PT):=POLYVAL(PTS(PT),INTERP(*,I,1),4); COMMENT INSERT DATA INTO JACOBIAN IF REQUIRED; IF AUX_INT~=0 THEN FOR J:=l UNTIL N DO BEGIN MJAC:=MJAC+1; FOR L:=l UNTIL N_PAR DO JAC(L,MJAC):=POLYVAL(PTS(PT),INTERP(*,J,L+l),4); END; PT:=PT+1; IF PT>N_PTS THEN GO TO STOP; LNEXT_PT:=PTS(PT); END; STEP_COUNT:=0; END; COMMENT UPDATE HERM_SAVE; FOR J:=l UNTIL N DO BEGIN HERM_SAVE(1,J,1):=LY(1,J); HERM_SAVE(2,J,1):=LY(2 , J)/HPl; END; IF AUX_INT~=0 THEN FOR J:=l UNTIL N DO FOR K:=l UNTIL N_PAR DO BEGIN HERM_SAVE(1,J,K+1):=STORE_DER(1 , J , K); HERM_SAVE(2 , J , K+l) :=STORE_DER(2,J,K)/HPl; END; COMMENT RESET JSTART; JSTART:=1; END; STOP:END GEAR. %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% Marguardt procedure %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE MARQUARDT(LONG REAL VALUE EPS_R,EPS_A; INTEGER VALUE M,N; PROCEDURE FUNC; LONG REAL ARRAY P(*); LONG REAL VALUE LAM; PROCEDURE SVD_AL); BEGIN COMMENT MARQUARDT-LEVENBERG TECHNIQUE—AN ADAPTATION OF THE VERSION USED BY BARD (1970), SIAM J . NUMER. ANAL. 7, 157-186—OPTIONAL SCALING OF PROBLEM IF EPS_R, EPS_A ARE BOTH NEGATIVE; LONG REAL ARRAY UD,JAC(1::M,1::N); APPENDIX B 246 LONG REAL ARRAY RES,RES1(1::M); LONG REAL ARRAY Pi,S,GRAD,GRADl,AUX_VEC,Z,DELTA,C(1::N); LONG REAL ARRAY VD(1::N,1::N); INTEGER ERROR_COUNT,MAXIT,MAXINT,MAXERR,EFLAG,DECLAM; LONG REAL F,Fl,SUM; LOGICAL CONV,SCALE; LONG REAL PROCEDURE COSINE(LONG REAL ARRAY V l ( * ) ; LONG REAL ARRAY V2(*)); BEGIN COMMENT FIND COSINE OF ANGLE BETWEEN VI AND V2; LONG REAL S1,S2,S3; S1:=0.L; S2:=0.L; S3:=0.L; FOR I:=l UNTIL N DO BEGIN S1:=S1-V1(I)*V2(I); S2:=S2+V1(I)**2; S3:=S3+V2(I)**2; END; Sl/LONGSQRT(S2*S3) END COSINE; COMMENT INITIALIZE AND SET BOUNDS ON EFFORT IN OPTIMIZATION ATTEMPT; MAXIT:=25; MAXINT:=4; MAXERR:=3; CONV:=FALSE; ERROR_COUNT:=0; DECLAM:=0; IF EPS_R<0 AND EPS_A<0 THEN BEGIN SCALE:=TRUE; EPS_R:=-EPS_R;EPS_A:=-EPS_A; COMMENT %OUTPUT TO USER, -SCI SCALING USED IN MARQUARDT; END ELSE BEGIN SCALE:=FALSE; FOR I:=l UNTIL N DO C(I):=1.L; END; COMMENT INITIAL LEAST SQUARES FUNCTION EVALUATION; FUNC(P,F,RES,JAC,GRAD,EFLAG); IF EFLAG=1 THEN BEGIN COMMENT HANDLE ERROR RETURN %OUTPUT TO USER, -SCI ERROR IN FIRST FUNCTION CALL IN MARQUARDT; WRITE("ERROR IN FIRST FUNC CALL IN MARQUARDT"); GO TO STOP; END; COMMENT %OUTPUT TO USER, -SCI INITIAL SUM OF SQUARES (F); COMMENT %OUTPUT TO USER INITIAL SUM OF SQUARES IS (F); APPENDIX B 247 FOR ITER:=1 UNTIL MAXIT DO BEGIN IF SCALE=TRUE THEN BEGIN LONG REAL SUM; FOR J:=l UNTIL N DO BEGIN SUM:=0.L; FOR I:=l UNTIL M DO SUM:=SUM+JAC(I,J)**2; C(J):=LONGSQRT(SUM); END; FOR J:=l UNTIL N DO FOR I:=l UNTIL M DO JAC(I,J):=JAC(I,J)/C(J); FOR I:=l UNTIL N DO GRAD(I):=GRAD(I)/C(I); END; COMMENT FIND SINGULAR VALUE DECOMPOSITION OF JAC; SVD_AL(JAC,S,UD,VD,M,N,M,N,0,N,N) ; IF ITER=1 THEN BEGIN COMMENT INITIALIZE LAM; LAM: = (IF LAM<0 THEN .OIL ELSE LAM); END; COMMENT PREPARE FOR AN ITERATION; FOR I:=l UNTIL N DO BEGIN SUM:=0.L; FOR J:=l UNTIL M DO SUM:=SUM+UD(J,I)*RES(J); AUX_VEC(I):=SUM*S(I); END; COMMENT DETERMINE LAM; REDO:FOR I:=l UNTIL N DO Z(I):=AUX_VEC(I)/(S(I)**2+LAM); FOR I:=l UNTIL N DO BEGIN SUM:=0.L; FOR J:=l UNTIL N DO SUM:=SUM+VD(I,J)*Z(J); DELTA(I):=-SUM; END; FOR I:=l UNTIL N DO Pi (I) :=P(I)+DELTA(I)/C(I) ; COMMENT %OUTPUT TO -SCI TRIAL LAMBDA (LAM), TRIAL PARAMETER VECTOR (PI); COMMENT FIND LEAST SQUARES FUNCTION AT TRIAL PARAMETERS; FUNC(Pi,F1,RES1,JAC,GRADl,EFLAG); IF EFLAG=1 THEN BEGIN COMMENT HANDLE ERROR RETURN; ERROR_COUNT:=ERROR_COUNT+l; APPENDIX B 248 IF ERROR_COUNT=MAXERR THEN BEGIN COMMENT %OUTPUT TO USER, -SCI (MAXERR) FUNCTION ERRORS; GO TO STOP; END ELSE BEGIN LAM:=LAM*10.L; DECLAM:=0; GO TO REDO; END; END; ERROR_COUNT:=0; COMMENT %OUTPUT TO -SCI TRIAL SUM OF SQUARES ( F l ) ; IF F K F THEN COMMENT DECREASE LAM AFTER TWO SUCCESSFUL FUNCTION REDUCTIONS; BEGIN DECLAM:=DECLAM+1; IF DECLAM=2 THEN BEGIN LAM:=(IF (.1L*LAM>1.'-10L) THEN .1L*LAM ELSE l . ' - l O L ) ; DECLAM:=0; END; COMMENT CHECK FOR CONVERGENCE; IF (F-FKEPS_R*F1+EPS_A) OR (Fl<EPS_A) THEN CONV:=TRUE; GO TO UPDATE; END ELSE BEGIN IF (COSINE(DELTA,GRAD))<.707 THEN BEGIN COMMENT INCREASE LAM; LAM:=10.L*LAM; DECLAM:=0; GO TO REDO; END ELSE BEGIN COMMENT DIRECTION TOO CLOSE TO STEEPEST DESCENT, THEREFORE DO NOT INCREASE LAM, BUT INTERPOLATE; LONG REAL GA,R0,Rl,Wl; COMMENT %OUTPUT TO USER, -SCI INTERPOLATING; GA:=0.L; R0:=1.L; APPENDIX B 249 FOR I:=l UNTIL N DO GA:=GA+DELTA(I)*GRAD(I); FOR INTERP:=1 UNTIL MAXINT DO BEGIN R1:=GA*R0**2/(2.L*(GA*R0+F-F1)); W1:=(IF (.75L*R0<R1) THEN .75L*R0 ELSE R l ) ; R0:=(IF (.25L*R0>W1) THEN .25L*R0 ELSE Wl); REPEAT:FOR I:=l UNTIL N DO PI(I):=P(I)+RO*DELTA(I)/C(I); COMMENT %OUTPUT TO -SCI TRIAL PARAMETER VECTOR ( P i ) ; COMMENT FUNCTION EVALUATION AT TRIAL PARAMETERS; FUNC(PI,Fl,RES1,JAC,GRADl,EFLAG); IF EFLAG=1 THEN BEGIN COMMENT HANDLE ERROR RETURN; ERROR_COUNT:=ERROR_COUNT+l; IF ERROR_COUNT=MAXERR THEN BEGIN COMMENT %OUTPUT TO USER, -SCI (MAXERR) FUNCTION ERRORS IN INTERPOLATION PART; GO TO STOP; END ELSE BEGIN R0:=R0*.5L; GO TO REPEAT; END; END; ERROR_COUNT:=0; COMMENT %OUTPUT TO USER, -SCI TRIAL SUM OF SQUARES (Fl) ; IF F K F THEN BEGIN IF (F-FKEPS_R*F1+EPS_A) OR (FKEPS_A) THEN CONV:=TRUE; GO TO UPDATE; END; END INTERP; COMMENT %OUTPUT TO USER, -SCI (MAXINT) INTERPOLATIONS TRIED, NO REDUCTION IN SUM OF SQUARES; GO TO STOP; END; END; UPDATE: FOR I:=l UNTIL N DO A P P E N D I X B 250 BEGIN P(I) :=P1(I) ; GRAD(I):=GRADl(I); END; F:=F1; COMMENT %OUTPUT TO USER NEW SUM OF SQUARES IS (F); FOR I:=l UNTIL M DO RES(I):=RES1(I); IF CONV=TRUE THEN GO TO FINISHED; END ITER; COMMENT %OUTPUT TO USER, -SCI OVER (MAXIT) ITERATIONS REQUIRED; GO TO STOP; FINISHED: COMMENT %OUTPUT TO USER, -SCI FINAL PARAMETERS (P), FINAL GRADIENT (GRAD), FINAL SUM OF SQUARES (F); STOP:END MARQUARDT. %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% Interactive optimization %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE INTERACTIVE_OPT(INTEGER VALUE M,N; PROCEDURE FUNC; LONG REAL ARRAY P(*); PROCEDURE SVD_AL; LONG REAL ARRAY OBS_PLACE(*); LONG REAL ARRAY PTS__TO_HIT (* ) ; LONG REAL ARRAY STATE(*,*); LONG REAL ARRAY OBS(*,*); LONG REAL ARRAY OBS_SMOOTH(*,*); LONG REAL ARRAY CREOBS(*); INTEGER VALUE N_OBS,N_PTS_HIT,N_STATE_OBS ,N_STATE; INTEGER ARRAY STATES_OBS(*); INTEGER VALUE RESULT PLOT_NUMBER; PROCEDURE READ_CMD_DATA; PROCEDURE CMD_AL; PROCEDURE PLOT_COMMAND); BEGIN STRING(6) ARRAY COMMANDS(1::20); STRING(6) CMD; INTEGER CMD_CTR,CMD_NO,TECH,NP,EFLAG,ECT; LONG REAL LAMDA,DEL,F,TRIAL_F,SUM; LONG REAL ARRAY AUX_VEC,S,SP,TRIAL_SP, TRIAL__P , GRAD, TRIAL_GRAD (1: :N) ; APPENDIX B 251 INTEGER ARRAY FREEZE(1::N); LOGICAL NUM,INIT,SUCCESS; LONG REAL ARRAY RES,TRIAL_RES(1::M); LONG REAL ARRAY TRIAL_JAC,JAC,AJAC,UD(1::M,1::N); LONG REAL ARRAY VD(1::N,1::N); COMMENT %EXTERNAL ECHO; PROCEDURE ECHO_CHECK; BEGIN COMMENT ECHO 3270 CONVERSATION BUFFER; IF ECT>80 THEN BEGIN ECHO(2); END; ECT:=0; END ECHO_CHECK; PROCEDURE MARQ_PREP; BEGIN COMMENT EXTRACT JACOBIAN AND PARAMETERS ON NONFROZEN COMPONENTS; INTEGER J J ; LONG REAL SUM; JJ:=0; FOR J:=l UNTIL N DO IF FREEZE(J)=0 THEN BEGIN JJ:=JJ+1; FOR I:=l UNTIL M DO AJAC(I,JJ):=JAC(I,J); SP(JJ) :=P(J) ; END; NP:=JJ; SVD_AL(AJAC, S , UD, VD, M , N, M , NP , 0 , NP , NP) ; FOR I:=l UNTIL NP DO BEGIN SUM:=0; FOR J:=l UNTIL M DO SUM:=SUM+UD(J,I)*RES(J); AUX_VEC(I):=SUM; END; FOR I:=l UNTIL NP DO AUX_VEC(I):=AUX_VEC(I)*S(I); END MARQ_PREP; PROCEDURE INITIALIZE; BEGIN COMMENT INITIALIZE; FUNC(P,F,RES,JAC,GRAD,EFLAG); IF EFLAG=1 THEN BEGIN INIT:=FALSE; NUM:=FALSE; GO TO READ_CMD; END; APPENDIX B 252 CASE TECH OF BEGIN BEGIN COMMENT STEEPEST DESCENT CASE; COMMENT I N I T I A L I Z A T I O N AUTOMATIC; END; BEGIN COMMENT MARQUARDT; MARQ_PREP; END END; INI T : = F A L S E ; END I N I T I A L I Z E ; PROCEDURE UPDATE; BEGIN CASE TECH OF BEGIN BEGIN COMMENT UPDATE FOR STEEPEST DESCENT; FOR I : = l UNTIL N DO BEGIN P ( I ) : = T R I A L _ P ( I ) ; G R A D ( I ) : = T R I A L _ G R A D ( I ) ; END; F:=TRIAL_F; END; BEGIN COMMENT UPDATE FOR MARQUARDT; F:=TRIAL_F; FOR I : = l UNTIL N DO BEGIN P ( I ) : = T R I A L _ P ( I ) ; G R A D ( I ) : = T R I A L _ G R A D ( I ) ; END; FOR I : = l UNTIL M DO RES ( I ) : = T RIAL_RES(I) ; FOR J : = l UNTIL N DO FOR I : = l UNTIL M DO J A C ( I , J ) : = T R I A L _ J A C ( I , J ) ; MARQ_PREP; END END; SUCCESS:=FALSE; END UPDATE; COMMENT I N I T I A L I Z E ; ECT:=0; ECHO(l); COMMENT %OUTPUT TO USER -REPRT ######INTERACTIVE OPTIMIZATION ATTEMPT*#####; INIT:=TRUE; LAMDA:=.01; APPENDIX B 253 TECH:=2; SUCCESS:=FALSE; NUM:=FALSE; FOR I:=l UNTIL N DO FREEZE (I) :=0; COMMENT SET UP COMMAND DESIGNATORS; COMMANDS(1):="T "; COMMANDS(2):="M COMMANDS(3):="N "; COMMANDS(4):="V "; COMMANDS(5):="F "; COMMANDS(6):="DF COMMANDS(7):="SET "; COMMANDS(8):="Q "; COMMANDS(9):="PLOT "; COMMANDS(10):="END "; COMMENT READ COMMAND DESIGNATOR AND IDENTIFY; READ__CMD: COMMENT %OUTPUT TO USER ****ENTER OPTIMIZATION COMMAND****; ECHO_CHECK; COMMENT %INPUT (CMD); CMD_CTR:=1; WHILE COMMANDS(CMD_CTR)~="END " DO BEGIN IF COMMANDS(CMD_CTR)=CMD THEN BEGIN CMD_NO:=CMD_CTR; GO TO OUT; END; CMD_CTR:=CMD_CTR+1; END; COMMENT %OUTPUT TO USER COMMAND IN ERROR RESPECIFY; GO TO READ_CMD; OUT:CASE CMD_NO OF BEGIN BEGIN COMMENT CHOICE OF TECHNIQUE COMMAND; STRING(6) MET; INTEGER TECH1; ECT:=ECT+1; TECH1:=TECH; COMMENT %INPUT (MET); IF MET="SD " THEN TECH:=1 ELSE IF MET="MARQ " THEN TECH:=2 ELSE BEGIN COMMENT %OUTPUT TO USER ERROR IN TECHNIQUE SPECIFICATION; GO TO READ_CMD; END; APPENDIX B 254 I F TECH~=TECH1 THEN INIT:=TRUE; END; BEGIN COMMENT MULTIPLY OPTIMIZATION PARAMETER BY A FACTOR; LONG REAL FACTOR; ECT:=ECT+3; COMMENT %INPUT (FACTOR); CASE TECH OF BEGIN DEL:=DEL*FACTOR; LAMDA:=LAMDA*FACTOR END; NUM:=TRUE; END; BEGIN COMMENT NEW OPTIMIZATION PARAMETER; ECT:=ECT+3; CASE TECH OF BEGIN BEGIN COMMENT %INPUT ( D E L ) ; END; BEGIN COMMENT %INPUT ( D E L ) ; END END; NUM:=TRUE; END; BEGIN COMMENT VIEW COMMAND; STRING(6) UNIT; ECT:=ECT+10; I F INIT=TRUE THEN I N I T I A L I Z E ELSE I F SUCCESS=TRUE THEN UPDATE; BEGIN COMMENT %OUTPUT TO USER PARAMETERS, GRADIENT, FREEZING; CASE TECH OF BEGIN BEGIN COMMENT STEEPEST DESCENT; COMMENT %OUTPUT TO USER STEEPEST DESCENT CURRENT STEP ( D E L ) ; END; BEGIN COMMENT MARQUARDT DATA; COMMENT %OUTPUT TO USER MARQUARDT, (LAMDA), ( S ) ; END END; COMMENT %OUTPUT TO USER APPENDIX B I 255 CURRENT SUM OF SQUARES ( F ) ; END; END; BEGIN COMMENT FREEZE COMMAND; INTEGER CMP; I F SUCCESS=TRUE THEN UPDATE; COMMENT %INPUT (CMP); WHILE CMP~=0 DO BEGIN ECT:=ECT+1; FREEZE(CMP):=1; COMMENT %INPUT (CMP); END ; INIT:=TRUE; END; BEGIN COMMENT DEFREEZE COMMAND; INTEGER CMP; I F SUCCESS=TRUE THEN UPDATE; COMMENT %INPUT (CMP); I F CMP=0 THEN BEGIN COMMENT REMOVE A L L FREEZING; FOR I : = l UNTIL N DO F R E E Z E ( I ) : = 0 ; END ELSE WHILE CMP~=0 DO BEGIN ECT:=ECT+1; FREEZE(CMP):=0; COMMENT %INPUT (CMP); END; INIT:=TRUE; END; BEGIN COMMENT SET COMMAND; INTEGER CMP; I F SUCCESS=TRUE THEN UPDATE; COMMENT %INPUT (CMP); WHILE CMP~=0 DO BEGIN COMMENT %INPUT ( P ( C M P ) ) , (CMP); ECT:=ECT+1; END; INIT:=TRUE; END; BEGIN COMMENT QUIT COMMAND; I F SUCCESS=TRUE THEN UPDATE; ECHO(3) ; APPENDIX B 256 GO TO STOP; END; BEGIN COMMENT PLOT COMMAND; PLOT_COMMAND( OBS_PLACE, PTS_TO_HIT,STATE,OBS,OBS_SMOOTH,P, N_OBS,N_PTS_HIT,N_STATE_OBS,N_STATE,N, STATES_OBS,PLOT_NUMBER, READ_CMD_DATA, CMD_AL,CREOBS); ECT:=ECT+25; END END; COMMENT CARRY OUT NUMERICAL WORK I F REQUIRED; I F NUM=TRUE THEN BEGIN I F INIT=TRUE THEN I N I T I A L I Z E ELSE I F SUCCESS=TRUE THEN UPDATE; CASE TECH OF BEGIN BEGIN COMMENT STEEPEST DESCENT ITERATION ATTEMPT; FOR I : = l UNTIL N DO I F F R E E Z E ( I ) = 0 THEN T R I A L _ P ( I ) : = P ( I ) - D E L * G R A D ( I ) ELSE T R I A L _ P ( I ) : = P ( I ) ; ECT:=ECT+3; COMMENT %OUTPUT TRI A L PARAMETERS ( T R I A L _ P ) ; F U N C ( T R I A L _ P , T R I A L _ F , T R I A L _ R E S , T R I A L _ J A C ,TRIAL_GRAD,EFLAG); I F EFLAG=1 THEN BEGIN NUM:=FALSE; GO TO READ_CMD; END; BEGIN I F TRIAL_F<F THEN BEGIN COMMENT REDUCED R E S I D U A L — T E L L USER; ECT:=ECT+3; COMMENT %OUTPUT TO USER STEEPEST DESCENT STEP SUCCESSFUL NEW SUM OF SQUARES ( T R I A L _ F ) , CHANGE I N SUM OF SQUARES ( F - T R I A L _ F ) ; SUCCESS:=TRUE; END ELSE APPENDIX B BEGIN COMMENT STEP UNSUCCESSFUL; ECT:=ECT+3; COMMENT %OUTPUT TO USER STEEPEST DESCENT DID NOT REDUCE RESIDUAL NEW SUM OF SQUARES ( T R I A L _ F ) ; END; COMMENT %OUTPUT TO USER STEP WAS ( D E L ) ; END; END; BEGIN COMMENT MARQUARDT ITERATION ATTEMPT; LONG REAL ARRAY Z(1::N); INTEGER J J ; FOR I : = l UNTIL NP DO Z ( I ) : = A U X _ V E C ( I ) / ( S ( I ) * * 2 + L A M D A ) ; FOR I : = l UNTIL NP DO BEGIN SUM:=0; FOR J : = l UNTIL NP DO SUM:=SUM+VD(I,J)*Z(J) T R I A L _ S P ( I ) : = S P ( I ) - S U M ; END; JJ:=0 FOR I : = l UNTIL N DO BEGIN I F F REEZE(I)=0 THEN BEGIN J J : = J J + 1 ; T R I A L _ P ( I ) : = T R I A L _ S P ( J J ) END ELSE T R I A L _ P ( I ) : = P ( I ) ; END; BEGIN COMMENT %OUTPUT TO USER TR I A L PARAMETERS ( T R I A L _ P ) ; ECT:=ECT+3; END; F U N C ( T R I A L _ P , T R I A L _ F , T R I A L _ R E S , TRIAL__JAC , TRIAL_GRAD , EFLAG) ; I F EFLAG=1 THEN BEGIN NUM:=FALSE; GO TO READ_CMD; END; BEGIN I F TRIAL_F<F THEN BEGIN COMMENT REDUCED R E S I D U A L — T E L L USER; ECT:=ECT+3; COMMENT %OUTPUT TO USER 258 MARQUARDT STEP SUCCESSFUL NEW SUM OF SQUARES (TRIAL_F) CHANGE IN SUM OF SQUARES (T-TRIAL_F); SUCCESS:=TRUE; END ELSE BEGIN ECT:=ECT+3; COMMENT %OUTPUT TO USER MARQUARDT DID NOT REDUCE RESIDUAL NEW SUM OF SQUARES (TRIAL_F); END; COMMENT %OUTPUT TO USER LAMBDA IS (LAMDA); ECT:=ECT+1; END; END END; NUM:=FALSE; END; GO TO READ_CMD; STOP:END INTERACTIVE OPT. ; % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % P l o t t i n g procedure %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE PLOT_COMMAND(LONG REAL ARRAY LOBS_PLACE(*) LONG REAL ARRAY LPTS_TO_HIT(*); LSTATE(*,*); LOBS(*,*); LOBS_SMOOTH(*,*); PAR(*); OBS,N_PTS_HIT,N STATE LONG LONG LONG LONG ARRAY ARRAY ARRAY ARRAY REAL REAL REAL REAL INTEGER VALUE N ,N_STATE,N_PAR; INTEGER ARRAY STATES_OBS(*); INTEGER VALUE RESULT PLOT_NUMBER; PROCEDURE READ_CMD_DATA; CMD_AL; ARRAY LCREOBS(*)) ; OBS PROCEDURE LONG REAL BEGIN COMMENT PLOT COMMAND FOR STATE VARIABLES, OBSERVATIONS, SMOOTHING, GUESSED OBSERVATIONS, AND PHASE PLOTS; INTEGER ARRAY STATE_PLOT(1::N_STATE+1); OBS_PLOT,SMOOTH_PLOT,SCRATCH(1::N_STATE_OBS+l) PLOT_ITEMS(l::5); CRE PLOT,PH PLOT(1::2); INTEGER ARRAY INTEGER ARRAY INTEGER ARRAY STRING(1) ANS APPENDIX B 259 INTEGER I ; LONG REAL X S I Z E , Y S I Z E ; STRING(10) LABELX,LABELY; COMMENT DECLARE SINGLE PRECISION ARRAYS TO PASS DATA TO GRAPHICS PROCEDURES; REAL ARRAY OBS_PLACE(1::N_OBS); REAL ARRAY PTS_TO_HIT(1::N_PTS_HIT); REAL ARRAY STATE(1::N_PTS_HIT,1::N_STATE); REAL ARRAY OBS(1::N_OBS,1::N_STATE_OBS); REAL ARRAY OBS_SMOOTH(1::N_PTS_HIT,1::N_STATE_OBS); REAL ARRAY CREOBS(1::N_PTS_HIT,1::1); PROCEDURE MAX_MIN(REAL ARRAY X ( * ) ; I N T E G E R VALUE N; REAL RESULT MAX,MIN); BEGIN COMMENT FIND MAXIMUM AND MINIMUM OF X; MAX:=X(1);MIN:=X(1); FOR I : = l UNTIL N DO BEGIN I F X(I)>MAX THEN M A X : = X ( I ) ; I F X ( I ) < M I N THEN M I N : = X ( I ) ; END; END MAX_MIN; PROCEDURE PLOT; BEGIN COMMENT THIS PROCEDURE CONTAINS THE INTERFACE TO ALL GRAPHICS PROCEDURES; REAL MAX_X,MAX_Y,MAX_XT,MAX_YT, MI N_X , MI N_Y , MI N_X T , MI N_Y T ; INTEGER I,NEW; COMMENT %EXTERNAL ALGRAF_AL, PLOT_AL ALSIZE__AL, ALSCAL_AL, A L A X I S _ A L ; PROCEDURE MAX_MIN_MTX(INTEGER ARRAY P O S ( * ) ; REAL ARRAY X ( * , * ) ; INTEGER VALUE N ) ; BEGIN INTEGER J ; J : = l ; WHILE POS(J)~=0 DO BEGIN MAX_MIN(X(*,POS(J)),N,MAX_YT,MIN_YT); I F MAX_YT>MAX_Y THEN MAX__Y : =MAX_YT; I F MIN_YT<MIN_Y THEN MIN_Y:=MIN_YT; J:=J+1; END; END MAX_MIN_MTX; PROCEDURE PLOT_HELP(INTEGER ARRAY P O S ( * ) ; R E A L ARRAY X ( * ) ; REAL ARRAY Y ( * , * ) ; INTEGER VALUE N,NS); BEGIN INTEGER J ; J : = l ; APPENDIX B 260 WHILE POS(J)~=0 DO BEGIN ALGRAF_AL(X,Y(*,POS(J)),NEW*N,-5*NS-POS(J)); IF NEW>0 THEN NEW:=-1; J:=J+1; END; END PLOT_HELP; COMMENT DETERMINE BOUNDS FOR SCALING; MAX_X:=-1.'50;MIN_X:=1.'50; IF PLOT_ITEMS(1)=5 THEN MAX_MIN(STATE(*,1),N_PTS_HIT,MAX_X,MIN_X) ELSE BEGIN FOR I:=l UNTIL 5 DO IF PLOT_ITEMS(I)=2 THEN MAX_MIN(OBS_PLACE,N_OBS,MAX_X,MIN_X); MAX_MIN(PTS_TO_HIT,N_PTS_HIT,MAX_XT,MIN_XT); IF MAX_XT>MAX_X THEN MAX_X:=MAX_XT; IF MIN_XT<MIN_X THEN MIN_X:=MIN_XT; END; MAX_Y:=-1.'50;MIN_Y:=1.'50; I: =1; WHILE PLOT_ITEMS(I)~=0 DO BEGIN CASE PLOT_ITEMS(I) OF BEGIN MAX_MIN_MTX(STATE_PLOT,STATE,N_PTS_HIT); MAX_MIN_MTX(OBS_PLOT,OBS,N_OBS); MAX_MIN_MTX(SMOOTH_PLOT,OBS_SMOOTH,N_PTS_HIT); MAX_MIN_MTX(CRE_PLOT,CREOBS,N_PTS_HIT); MAX_MIN(STATE(*,2),N_PTS_HIT,MAX_Y,MIN_Y) END; I:=1+1; END; COMMENT DO THE PLOTTING; ALSIZE_AL(XSIZE-1.'-5,YSIZE-1.'-5); ALSCAL_AL(MIN_X,MAX_X,MIN_Y,MAX_Y); ALAXIS_AL(LABELX,50,LABELY, 50); NEW:=1; I : =1; WHILE PLOT_ITEMS(I)~=0 DO BEGIN CASE PLOT_ITEMS(I) OF BEGIN PLOT_HELP(STATE_PLOT,PTS_TO_HIT,STATE,N_PTS_HIT,0); PLOT_HELP(OBS_PLOT,OBS_PLACE,OBS,N_OBS,l); PLOT_HELP(SMOOTH_PLOT,PTS_TO_HIT,OBS_SMOOTH, N_PTS_HIT,2); PLOT_HELP (CRE_PLOT, PTS_TO_HIT , CREOBS , N_PTS__HIT, 3) ; PLOT_HELP(PH_PLOT,STATE(*,1),STATE,N_PTS_HIT,4) END; APPENDIX B 261 I:=I+1; END; COMMENT PREPARE FOR NEXT PLOT; PLOT_AL(12.0,0,-3); END PLOT; COMMENT EXECUTION BEGINS HERE FIRST COPY ARRAYS TO SINGLE PRECISION COUNTERPARTS; FOR I:=l UNTIL N_OBS DO OBS_PLACE(I):=LOBS_PLACE(I); FOR I:=l UNTIL N_PTS_HIT DO PTS_TO_HIT(I):=LPTS_TO_HIT(I); FOR J:=l UNTIL N_STATE DO FOR I:=l UNTIL N_PTS_HIT DO STATE(I,J):=LSTATE(I,J); FOR J:=l UNTIL N_STATE_OBS DO FOR I:=l UNTIL N_OBS DO OBS(I,J):=LOBS(I,J); FOR J:=l UNTIL N_STATE_OBS DO FOR I:=l UNTIL N_PTS_HIT DO OBS_SMOOTH(I,J):=LOBS_SMOOTH(I,J); FOR I:=l UNTIL N_PTS_HIT DO CREOBS(1,1):=LCREOBS(I); XSIZE:=5.; YSIZE:=3.; COMMENT %OUTPUT TO USER SEQUENCE OF ITEMS TO PLOT(END WITH 0) STATE VARIABLES 1 OBSERVATIONS 2 SMOOTHED OBSERVATIONS 3 GUESSED OBSERVATIONS 4 PHASE PLOT • 5; READ__CMD_DATA (PLOT_I TEMS) ; LABELX:="TIME";LABELY:="STATE"; I: =1; WHILE PLOT_ITEMS(I)~=0 DO BEGIN CASE PLOT_ITEMS(I) OF BEGIN BEGIN COMMENT %OUTPUT TO USER STATE VARIABLES(END WITH 0); READ_CMD__DATA (STATE_PLOT) ; END; BEGIN INTEGER II; COMMENT %OUTPUT TO USER OBSERVED VARIABLES TO PLOT (END WITH 0); READ_CMD_DATA(SCRATCH); II:=1; WHILE SCRATCH(II)~=0 DO BEGIN APPENDIX B 262 FOR J:=l UNTIL N_STATE_OBS DO IF SCRATCH(II)=STATES_OBS(J) THEN OBS_PLOT(II) :=J; II:=II+1; END; OBS_PLOT(II):=0; END; BEGIN INTEGER II; COMMENT %OUTPUT SMOOTHED OBSERVATIONS TO PLOT (END WITH 0); READ_CMD_DATA(SCRATCH); II:=1; WHILE SCRATCH(II)~=0 DO BEGIN FOR J:=l UNTIL N_STATE_OBS DO IF SCRATCH(II)=STATES_OBS(J) THEN SMOOTH_PLOT(II) :=J; II:=II+1; END; SMOOTH_PLOT(II):=0; END; BEGIN COMMENT GUESSED OBSERVATIONS CASE; CRE_PLOT(1):=1;CREJPLOT(2):=0; END; BEGIN COMMENT PHASE PLOT CASE; PH_PLOT(l):=2;PH_PLOT(2):=0; LABELX:="Yl";LABELY:="Y2"; END END; I:=I+1; END; COMMENT %FILE EMPTY FILE -GRAPH; PLOT; COMMENT %FILE DISPLAY -GRAPH TO USER %OUTPUT TO USER IS A PERMANENT PLOT REQUIRED ANS. Y OR N %INPUT (ANS); IF ANS="Y" THEN BEGIN XSIZE:=10; YSIZE:=10; COMMENT %FILE EMPTY -GRAPH; PLOT; COMMENT %FILE ACCUMULATE -GRAPH IN -GRAPHSTORE; PLOT_NUMBER:=PLOT_NUMBER+l; COMMENT %OUTPUT TO USER, -REPRT CURRENT PLOT_NUMBER (PLOT_NUMBER) APPENDIX B 263 CURRENT PARAMETERS (PAR) DESCRIPTION OF PLOT CONTENTS; END; END PLOT COMMAND. %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% Sparse Gauss-Newton for iterated methods %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE SPRGN(PROCEDURE G_FUN; LONG REAL ARRAY PAR(*) ; INTEGER ARRAY INTDATA(*); LONG REAL ARRAY OBS_SMOOTH(*,*); INTEGER ARRAY STATES_OBS(*); LONG REAL ARRAY PTS_TO_HIT(*); LONG REAL ARRAY JOINTS(*,*); LONG REAL ARRAY SPLN_COEF(*,*); LONG REAL ARRAY HERM_COEF(*,*,*) ; INTEGER ARRAY NJOINTS(*); INTEGER ARRAY N_SPLN_PAR(*); LONG REAL PROCEDURE SPLN_AL,DSPLN_AL,HERM,DHERM; LONG REAL ARRAY CREOBS(*); LONG REAL ARRAY DCREOBS(*); PROCEDURE ITER_PAR); BEGIN COMMENT ITERATED IFIT AND DFIT IN THE TWO STATE VARIABLE CASE USING A SPARSE GAUSS-NEWTON METHOD TO IMPROVE GUESSED OBSERVATIONS (2 OPTIONS, FIXED INITIAL CONDITIONS ON GUESSED OBSERVATIONS, AND VARIABLE INITIAL CONDITIONS FOR GUESSED OBSERVATIONS; PROCEDURE ITER_FUNC(LONG REAL ARRAY C(*); LONG REAL RESULT F; LONG REAL ARRAY AHES(*); LONG REAL ARRAY GRAD(*); INTEGER VALUE FULL); BEGIN COMMENT LEAST SQUARES FUNCTION FOR ITERATIVELY IMPROVING THE GUESSED OBSERVATIONS USING A SPARSE GAUSS NEWTON METHOD; LONG REAL ARRAY A,JA(1::N_PTS_HIT); LONG REAL ARRAY B (1: :N_PTS__HIT-1) ; LONG REAL ARRAY JB (1: :N__PTS__HIT-1 ,1: : 2) ; LONG REAL ARRAY DGY,PREV_DGY(1::2,1::2); LONG REAL ARRAY DGP,SENSE(1::1,1::1); LONG REAL ARRAY G,PREV_G,Y(1::2); LONG REAL HI; Y(SM_Y):=OBS_SMOOTH(1,1); Y(CR_Y):=C(1); APPENDIX B 264 G_FUN(PTS_T0_HIT(1),Y,PAR,1,PREV_G,PREV_DGY,DGP,SENSE); A(l):=PREV_G(SM_Y) -(CASE SMF 0F(DSPLN_AL(PTS_T0_HIT(1), SPLN_C0EF(*,1),JOINTS(*,1), NJOINTS(1),N_SPLN_PAR(1)), DHERM(PTS_TO_HIT(l),HERM_COEF( * , * , 1),JOINTS(*,D, NJOINTS(1)))); JA(1):=PREV_DGY(SM_Y,CR_Y); FOR I:= 2 UNTIL N_PTS_HIT DO BEGIN Y(SM_Y):=OBS_SMOOTH(I,1); Y(CR_Y):=C (I) ; G_FUN(PTS_TO_HIT(I),Y,PAR,1,G,DGY,DGP,SENSE); A(I):=G(SM_Y) -(CASE SMF OF(DSPLN_AL(PTS_TO_HIT(I), SPLN_COEF(*,1),JOINTS(*,1), NJOINTS(1),N_SPLN_PAR(1)), DHERM(PTS_TO_HIT(I),HERM_COEF(*,*,1), JOINTS (*,D, NJOINTS(1)))); JA(I):=DGY(SM_Y,CR_Y); HI:=PTS_TO_HIT(I)-PTS_TO_HIT(1-1); B(I-l):=.5L*(G(CR_Y)+PREV_G(CR_Y))-(C(I)-C(1-1))/HI; JB(1-1,1) : = .5 L * P REV_DGY(CR_Y,CR_Y)+1.L/HI; JB(1-1,2):=.5L*DGY(CR_Y,CR_Y)-l.L/HI; FOR J:=l UNTIL 2 DO PREV_G(J):=G(J); FOR J:=l UNTIL 2 DO FOR K:=l UNTIL 2 DO PREV_DGY(J,K):=DGY(J,K); END; IF FULL=1 THEN BEGIN COMMENT FORM AHES TO CONFORM TO DFBAND; AHES(l):=JB(1,1)**2+JA(l)**2; FOR I:=2 UNTIL N_PTS_HIT-1 DO AHES (2* (I-D+l) :=JA(I) **2+JB (1-1,2) **2+JB(I ,1) **2; AHES(2*(N_PTS_HIT-1)+1):=JA(N_PTS_HIT)**2 +JB(N_PTS_HIT-1,2)**2; FOR I:=l UNTIL N_PTS_HIT-1 DO AHES(2*1):=JB(I,1)*JB(I,2); COMMENT FORM GRADIENT; FOR I:=l UNTIL N_PTS_HIT DO BEGIN GRAD(I):=JA(I)*A(I); IF 1=1 THEN GRAD(I):=GRAD(I)+JB(1,1)*B(1) ELSE IF I=N_PTS_HIT THEN GRAD(I):=GRAD(I)+ JB(N_PTS_HIT-1,2)*B(N_PTS_HIT-1) ELSE GRAD(I):=GRAD(I)+JB(I-1,2)*B(I-1)+JB(I,1)*B(I); END ; APPENDIX B 265 END; COMMENT FORM F; F:=0.L; FOR I:=l UNTIL N_PTS_HIT DO F:=F+A(I)**2; FOR I:=l UNTIL N_PTS_HIT-1 DO F:=F+B(I)**2; END ITR_FUNC; COMMENT PROCEDURE ITER_FUNC_FIXIC(LONG REAL ARRAY C(*); LONG REAL RESULT F; LONG REAL ARRAY AHES(*); LONG REAL ARRAY GRAD(*); INTEGER VALUE FULL); BEGIN COMMENT LEAST SQUARES FUNCTION FOR ITERATIVELY IMPROVING THE GUESSED OBS FIXED IC ON GUESSED OBS; LONG REAL ARRAY A, JA (1: :N_PTS__HIT-1) ; LONG REAL ARRAY B(1::N_PTS_HIT-1); LONG REAL ARRAY JB(1::N_PTS_HIT-1,1::2); LONG REAL ARRAY DGY,PREV_DGY(1::2,1::2); LONG REAL ARRAY DGP,SENSE(1::1,1::1); LONG REAL ARRAY G,PREV_G, Y (1 : : 2); LONG REAL HI; G_FUN(PTS_TO_HIT(1),Y,PAR,-3,G,DGY,DGP,SENSE); Y(CR_Y):=CREOBS(1); G__FUN (PTS__TO_HIT (1) , Y, PAR, 1, PREV_G , PREV_DGY, DGP , SENSE) ; FOR I:=2 UNTIL N_PTS_HIT DO BEGIN Y(SM_Y):=OBS_SMOOTH(1,1); Y(CR_Y):=C(1-1); G_FUN(PTS_TO_HIT(I),Y,PAR,1,G,DGY,DGP,SENSE); A(I-l):=G(SM_Y) -(CASE SMF OF(DSPLN_AL(PTS_TO_HIT(I), SPLN_COEF(*,l),JOINTS(*,1), NJOINTS(l),N_SPLN_PAR(1)), DHERM(PTS_TO_HIT(I),HERM_COEF(*,*,1),JOINTS(*,l), NJOINTS(1)))); JA(1-1):=DGY(SM_Y,CR_Y); HI:=PTS_TO_HIT(I) -PTS__TO_HIT (1-1) ; B(I-l):=.5L*(G(CR_Y)+PREV_G(CR_Y))-(C(I-l) -(IF I>2 THEN C(I-2) ELSE CREOBS(1)))/HI; JB(1-1,1):=.5L*PREV_DGY(CR_Y,CR_Y)+1.L/HI; JB(1-1,2):=.5L*DGY(CR_Y,CR_Y)-1.L/HI; FOR J:=l UNTIL 2 DO PREV_G(J):=G(J); FOR J:=l UNTIL 2 DO FOR K:=l UNTIL 2 DO PREV_DGY(J,K):=DGY(J,K); APPENDIX B 266 END; IF FULL=1 THEN BEGIN COMMENT FORM AHES TO CONFORM TO DFBAND; FOR I:=l UNTIL N_PTS_HIT-2 DO AHES(2*(1-1)+1):=JA(I)**2+JB(I,2)**2+JB(1+1 f1)**2; AHES(2*(N_PTS_HIT-2)+1):=JA(N_PTS_HIT-1)**2 +JB(N_PTS_HIT-1,2)**2; FOR I:=l UNTIL N_PTS_HIT-2 DO AHES(2*1):=JB(1+1,1)*JB(1+1,2); COMMENT FORM GRADIENT; FOR I:=l UNTIL N_PTS_HIT-1 DO BEGIN GRAD(I):=JA(I)*A(I); IF 1=(N_PTS_HIT-1) THEN GRAD(I):=GRAD(I)+ JB(N_PTS_HIT-1,2)*B(N_PTS_HIT-1) ELSE GRAD(I):=GRAD(I)+JB(I,2)*B(I)+JB(1+1,1)*B(1+1); END; END; COMMENT FORM F; F:=0.L; FOR I:=l UNTIL N_PTS_HIT-1 DO F:=F+A(I)**2+B(I)**2; END ITER_FUNC_FIXIC; COMMENT PROCEDURE SPARSE_GN(LONG REAL ARRAY C(*); INTEGER VALUE NC; PROCEDURE ITER_FUNC); BEGIN COMMENT ITERATIVE UPDATING OF GUESSED OBSERVATIONS USING A SPARSE GAUSS NEWTON METHOD; LONG REAL F,Fl,RATIO,DET,GA,RO,Rl,Wl; LONG REAL ARRAY AHES(1::2*NC); LONG REAL ARRAY CI,GRAD,DELTA(1::NC); INTEGER JEXP,JAY; LOGICAL CONV; COMMENT %EXTERNAL DFBAND; CONV:=FALSE; FOR ITER:=1 UNTIL 25 DO BEGIN ITER_FUNC(C,F,AHES,GRAD,1); IF ITER=1 THEN COMMENT %OUTPUT TO USER STARTING SUM OF SQUARES IN SPARSE GAUSS NEWTON IS (F); IF ITER=1 THEN JAY:=0; JAY:=JAY DIV 2; R0:=l.L/2**(JAY); COMMENT SOLVE FOR DELTA; FOR I:=l UNTIL NC DO DELTA(I):=-GRAD(I); APPENDIX B 267 RATIO:=1.'-7L; DFBAND(AHES,DELTA,NC,2,1,RATIO,DET,JEXP, 0) ; COMMENT FIND STEP LENGTH; FOR I:=l UNTIL NC DO CI (I) :=C(I)+RO*DELTA(I) ; COMMENT CHECK FOR CONVERGENCE; FOR I:=l UNTIL NC DO IF (ABS(C(I)-C1(I))>(TOL*ABS(CI(I))+.0 01L)) THEN GO TO CON; CONV:=TRUE; GO TO UPDATE; CON: ITER__FUNC (CI,Fl,AHES,GRAD,0) ; IF F1>F THEN BEGIN COMMENT INTERPOLATE; GA:=0.L; FOR I:=l UNTIL NC DO GA:=GA+DELTA(I)*GRAD(I); FOR INTERP:=1 UNTIL 5 DO BEGIN JAY:=JAY+1; R1:=GA*R0**2/(2.L*(GA*R0+F-F1) ) ; W1:=(IF (.75L*R0<R1) THEN .75L*R0 ELSE Rl); R0:=(IF (.25L*R0>W1) THEN .25L*R0 ELSE WI); FOR I:=l UNTIL NC DO CI(I):=C(I)+R0*DELTA(I); COMMENT CHECK FOR CONVERGENCE; FOR I:=l UNTIL NC DO IF (ABS(C(I)-CI(I))>(TOL*ABS(CI(I))+.001L)) THEN GO TO CONT; CONV:=TRUE; GO TO UPDATE; CONT:ITER_FUNC(CI,Fl,AHES,GRAD,0); IF F K F THEN GO TO UPDATE; END INTERP; COMMENT %OUTPUT TO USER TERMINATING—OVER 5 INTERPOLATIONS REQUIRED; GO TO FINISHED; END; UPDATE:FOR I:=l UNTIL NC DO C(I) :=C1 (I) ; COMMENT %OUTPUT TO USER SUM OF SQUARES IS ( F l ) ; IF CONV=TRUE THEN GO TO FINISHED; END ITER; COMMENT %OUTPUT TO USER TERMINATING—SPARSE GAUSS NEWTON NEEDS MORE THAN 25 ITERATIONS TO MEET ERROR CRITERION; FINISHED: END SPARSE_GN; INTEGER N_PTS_HIT,N_PAR,SMF,OUTPUT,METHOD_FLAG,OUTPUT_SUP, OUT_SEG , INT_PROC , SM_Y , CR_Y, NO__ITER, EFLAG ; APPENDIX B 268 LONG REAL ARRAY B(1::1,1::INTDATA(1)); LONG REAL ARRAY BJAC(1::1,1: : 1) ; COMMENT COMMAND PROPER FOLLOWS; STRING(1) ANS,ANSI; LONG REAL TOL; N_PTS_HIT:=INTDATA(1) ; N_PAR:=INTDATA(4); SMF:=INTDATA(5); OUTPUT:=INTDATA(6); METHOD__FLAG : =INTDATA ( 7 ) ; OUTPUT_SUP:=INTDATA(9); OUT_SEG:=INTDATA(10); INT_PROC:=INTDATA(ll); SM_Y:=STATES_OBS(1); CR_Y:=3-SM_Y; COMMENT %OUTPUT TO USER IS INITIAL VALUE OF GUESSED OBSERVATIONS FIXED? Y OR N %INPUT (ANSI) %OUTPUT ENTER RELATIVE TOLERANCE ON CHANGE IN ITERATES FOR TERMINATION OF SPARSE GAUSS NEWTON %INPUT (TOL); REPT:ITER_PAR(G_FUN,PAR,INTDATA,OBS_SMOOTH,STATES_OBS, PTS_TO_HIT,JOINTS,SPLN_COEF,HERM_COEF,NJOINTS,N_SPLN_PAR, SPLN_AL,DSPLN_AL,HERM,DHERM, CREOBS,DCREOBS); COMMENT SET UP AND OPTIMIZE SPARSE PROB; REDO:IF ANS1~="Y" THEN SPARSE_GN(CR.EOBS,N_PTS_HIT,ITER_FUNC) ELSE BEGIN LONG REAL ARRAY C(1::N_PTS_HIT-1); FOR I:=l UNTIL N__PTS_HIT-1 DO C (I) : =CREOBS (1 + 1) ; SPARSE_GN(C,N_PTS_HIT-1,ITER_FUNC_FIXIC); FOR I:=l UNTIL N_PTS_HIT-1 DO CREOBS(1+1):=C(I); END; COMMENT %OUTPUT IS A FURTHER REFINEMENT OF THE GUESSED OBSERVATIONS DESIRED? Y OR N %INPUT (ANS); IF ANS="Y" THEN BEGIN COMMENT %OUTPUT ENTER NEW TOLERANCE FOR SPARSE PROBLEM %INPUT (TOL); GO TO REDO; END; COMMENT %OUTPUT APPENDIX B 269 IS ANOTHER ITERATION DESIRED? Y OR N %INPUT (ANS); IF ANS="Y" THEN GO TO REPT; OUT:END SPRGN. %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% Derivative f i t t i n g procedure %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE DATAFT_COMMAND(PROCEDURE G_FUN; LONG REAL ARRAY PAR(*); INTEGER ARRAY INTDATA(*); LONG REAL ARRAY OBS_SMOOTH(*,*) ; INTEGER ARRAY STATES_OBS(*); LONG REAL ARRAY PTS_TO_HIT(*); LONG REAL ARRAY JOINTS(*,*); LONG REAL ARRAY SPLN_COEF(*,*); LONG REAL ARRAY HERM_COEF(*,*,*); INTEGER ARRAY NJOINTS(*); INTEGER ARRAY N_SPLN_PAR(*); LONG REAL PROCEDURE SPLN_AL, DSPLN_AL,HERM,DHERM; LONG REAL VALUE INITIAL_TIME,EPS,HMIN,HMAX); BEGIN COMMENT DATAFIT COMMAND; INTEGER. N_PTS_HIT, N_STATE__OBS , N_STATE , N_PAR, SMF, OUTPUT ; INTEGER OUTPUT_SUP,OUT_SEG; INTEGER METHOD_FLAG,INT_PROC; LONG REAL LAM,EPS_R,EPS_A; LONG REAL ARRAY DJAC(1::1,1::1); INTEGER KFLAG; LONG REAL ARRAY STATE_DATA(1::INTDATA(1),1::INTDATA(3)); COMMENT %EXTERNAL GEAR, TRAP, SVD_AL, MARQUARDT, CMD_AL; PROCEDURE DATA_FUNC(LONG REAL ARRAY P(*); LONG REAL RESULT F; LONG REAL ARRAY RES(*); LONG REAL ARRAY JAC(*,*); LONG REAL ARRAY GRAD(*); INTEGER RESULT EFLAG); BEGIN COMMENT THIS PROCEDURE PROVIDES THE PERTINENT INFORMATION TO A NONLINEAR LEAST SQUARES ALGORITHM USED WITH THE DFIT METHOD; INTEGER K,M,OUTl; LONG REAL SUM; LONG REAL ARRAY INITY(1::1); LONG REAL ARRAY INITYP(1::1,1::1); LONG REAL ARRAY G(1::N_STATE); APPENDIX B 270 LONG REAL ARRAY DGY(1::N_STATE,1::N_STATE); LONG REAL ARRAY DGP,SENSE(1::N_STATE,1::N_PAR); K:=0; IF N_STATE_OBS~=N_STATE THEN BEGIN COMMENT NOT ALL STATE VARIABLES ARE OBSERVED; INTEGER ARRAY SCRATCH(1::N_STATE); INTEGER ARRAY STATES_NOT_OBS(1::N_STATE-N_STATE_OBS); LONG REAL ARRAY STATE_INTEG(1::N_PTS_HIT, 1::N_STATE-N_STATE_OBS); INTEGER L,N_STATE_NOT_OBS; PROCEDURE G_DATA(LONG REAL VALUE T; LONG REAL ARRAY Y(*); LONG REAL ARRAY P(* ) ; INTEGER VALUE OPTION; LONG REAL ARRAY G(*); LONG REAL ARRAY DGY(*,*); LONG REAL ARRAY D2(*,*); LONG REAL ARRAY D3(*,*)); BEGIN COMMENT INTERFACE TO G_FUN FOR DE SOLVER WHEN A SUBSET OF THE STATE VARIABLES ARE BEING INTEGRATED; LONG REAL ARRAY Y_A,DY_A(1::N_STATE); LONG REAL ARRAY DGY_A(1::N_STATE,1::N_STATE); LONG REAL ARRAY D2,D3(1: : 1,1 : : 1) ; FOR I:=l UNTIL N_STATE_NOT_OBS DO Y_A(STATES_NOT_OBS(I)):=Y(I); FOR I:=l UNTIL N_STATE_OBS DO CASE SMF OF BEGIN Y_A(STATES_OBS(I)):= SPLN_AL(T,SPLN_COEF(*,I),JOINTS(*,I) ,NJOINTS(I),N_SPLN_PAR(I)); Y_A(STATES_OBS(I)):= HERM(T,HERM_COEF( * , * , I),JOINTS(*,I) ,NJOINTS(I)) END; G_FUN(T,Y_A,P,OPTION,DY_A,DGY_A,D2,D3); IF OPTION=l THEN COMMENT CONSTRUCT SUB JACOBIAN; FOR I:=l UNTIL N_STATE_NOT_OBS DO FOR J:=l UNTIL N_STATE_NOT_OBS DO DGY(I,J):=DGY_A(STATES_NOT_OBS(I),STATES_NOT_OBS(J)); FOR I:=l UNTIL N_STATE_NOT_OBS DO G(I):=DY_A(STATES_NOT_OBS(I)); IF OPTION=-3 THEN FOR I:=l UNTIL N_STATE_NOT_OBS DO Y(I):=Y_A(STATES_NOT_OBS(I)); END G_DATA; COMMENT FORM VECTOR OF STATES NOT OBSERVED; FOR I:=l UNTIL N_STATE DO SCRATCH(I):=1; APPENDIX B 271 FOR I:=l UNTIL N_STATE_OBS DO SCRATCH(STATES_OBS(I)):=0; N_STATE_NOT_OBS:=N_S TATE-N_STATE_OB S; L:=0; FOR I:=l UNTIL N_STATE DO IF SCRATCH(I)~=0 THEN BEGIN L:=L+1; STATES_NOT_OBS(L):=SCRATCH(I); END; EFLAG:=0; COMMENT INTEGRATION; IF OUTPUT_SUP=l THEN OUTl:=0 ELSE OUTl:=OUTPUT; BEGIN LONG REAL ARRAY B(1::N_STATE_NOT_OBS,1::N_PTS_HIT); CASE INT_PROC OF BEGIN BEGIN GEAR(P,PTS_TO_HIT,B,DJAC,N_STATE_NOT_OBS, N_PTS_HIT,EPS,HMIN,HMAX, N_PAR,0,G_DATA,KFLAG,OUTl,METHOD_FLAG, 0,INITY,INITYP); IF KFLAG~=1 THEN BEGIN EFLAG:=1; GO TO OUT; END; END; BEGIN TRAP(P,PTS_TO_HIT,B,DJAC,N_STATE_NOT_OBS, N_PTS_HIT,N_PAR,0,G_DATA,EFLAG,OUTl); IF EFLAG=1 THEN GO TO OUT; END END; FOR J:=l UNTIL N_STATE_NOT_OBS DO FOR I:=l UNTIL N_PTS_HIT DO STATE_INTEG(I,J):=B(J,I); END; COMMENT COPY INTEGRATION DATA INTO STATE DATA; FOR I:=l UNTIL N_PTS_HIT DO FOR J:=l UNTIL N_STATE_NOT_OBS DO STATE_DATA (I, STATES_NOT_OBS (J) ) : =STATE__INTEG (I , J) ; END; FOR I:=l UNTIL N_PTS__HIT DO BEGIN G_FUN(PTS__TO_HIT(I) , STATE_DATA (I, * ) , P , 1, G, DGY, DGP , SENSE) ; G_FUN(PTS_TO_HIT(I),STATE_DATA(I,*),P,2,G,DGY,DGP, SENSE); FOR J:=l UNTIL N_STATE_OBS DO BEGIN APPENDIX B 272 K:=K+1; CASE SMF OF BEGIN RES (K) :=G(STATES_OBS(J))-DSPLN_AL(PTS_TO_HIT(I) , SPLN_COEF(*,J), JOINTS(*,J),NJOINTS(J),N_SPLN_PAR(J)); RES(K):=G(STATES_OBS(J))-DHERM(PTS_TO_HIT(I), HERM_COEF(*,*,J), JOINTS(*,J),NJOINTS(J)); END; FOR L:=l UNTIL N_PAR DO JAC(K,L):=DGP(STATES_OBS(J),L); END; END; COMMENT FORM SUM OF SQUARES OF RESIDUAL; SUM:=0.; M: =K ; FOR I:=l UNTIL M DO SUM:=SUM+RES(I)**2; F:=SUM; COMMENT FORM GRADIENT IF REQUIRED; BEGIN FOR I:=l UNTIL N_PAR DO BEGIN SUM:=0.; FOR J:=l UNTIL N_PTS_HIT*N_STATE_OBS DO SUM:=SUM+JAC(J,I)*RES(J); GRAD(I):=SUM; END; END; END DATA_FUNC; COMMENT COMMAND PROPER FOLLOWS; N_PTS_HIT:=INTDATA(1); N_STATE_OBS:=INTDATA(2); N_STATE:=INTDATA(3); N_PAR:=INTDATA(4); SMF:=INTDATA(5); OUTPUT:=INTDATA(6); METHOD_FLAG:=INTDATA(7); OUTPUT_SUP:=INTDATA(9); OUT_SEG:=INTDATA(10); INT_PROC:=INTDATA(ll); FOR I:=l UNTIL N_PTS_HIT DO FOR J:=l UNTIL N_STATE_OBS DO STATE_DATA(I,STATES_OBS(J)):=OBS_SMOOTH(I,J); COMMENT %OUTPUT ENTER STARTING LAMBDA RELATIVE AND ABSSOLUTE ERROR TOLERENCES FOR MARQUARDT PROCEDURE %INPUT (LAM),(EPS_R),(EPS_A) %FILE EMPTY TEMPORARY FILE -SCI APPENDIX B 273 %OUTPUT TO -REPRT MARQUARDT USED IN DFIT COMMAND—OPTION 1 LAMBDA, RELATIVE, AND ABSOLUTE ERROR TOLERENCES ARE (LAM),(EPS_R),(EPS_A); MARQUARDT(EPS_R,EPS_A,N_PTS_HIT*N_STATE_OBS,N_PAR,DATA_FUNC, PAR,LAM,SVD_AL); OUT:END. %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% Derivative f i t t i n g with guessed observations %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE DFIT_CRE(PROCEDURE G_FUN; LONG REAL ARRAY PAR(*); INTEGER ARRAY INTDATA(*); LONG REAL ARRAY OBS_SMOOTH(*,*) ; INTEGER ARRAY STATES_OBS(*); LONG REAL ARRAY PTS_TO_HIT(*); LONG REAL ARRAY JOINTS(*,*); LONG REAL ARRAY SPLN_COEF(*,*); LONG REAL ARRAY HERM_COEF(*,*,*) ; INTEGER ARRAY NJOINTS(*); INTEGER ARRAY N_SPLN_PAR(*); LONG REAL PROCEDURE SPLN_AL, DSPLN_AL,HERM,DHERM; LONG REAL ARRAY CREOBS(*); LONG REAL ARRAY DCREOBS(*)); BEGIN COMMENT DFIT ON 2 STATE VARIABLES WHEN OBSERVATIONS ON ONE HAVE BEEN GUESSED AT. SPECIAL HANDLING OF LINEAR PROBLEM; INTEGER N_PTS_HIT,N_PAR,SMF,DFIT_LIN,KK; LONG REAL LAM,EPS_R,EPS_A; INTEGER SM_Y,CR_Y,M,OUT_SEG; COMMENT %EXTERNAL SVD_AL, MARQUARDT, CMD_AL; PROCEDURE CRE_FUNC(LONG REAL ARRAY P(*); LONG REAL RESULT F; LONG REAL ARRAY RES(*); LONG REAL ARRAY JAC(*,*); LONG REAL ARRAY GRAD(*); INTEGER RESULT EFLAG); BEGIN COMMENT THIS PROCEDURE PROVIDES INFORMATION TO THE LEAST SQUARES MINIMIZER WHEN GUESSED OBSERVATIONS ARE USED; LONG REAL ARRAY SENSE,DGY(1::1,1::1); LONG REAL ARRAY Y,G(1::2); LONG REAL ARRAY DGP(1::2,1::N_PAR); INTEGER K,KK; APPENDIX B 274 LONG REAL SUM; EFLAG:=0; K:=0; FOR I:=l UNTIL N_PTS_HIT DO BEGIN Y(SM_Y):=OBS_SMOOTH(1,1); Y(CR_Y):=CREOBS(I); G_FUN(PTS_TO_HIT(I),Y,P,-1,G,DGY,DGP,SENSE); G_FUN(PTS_TO_HIT(I),Y,P,2,G,DGY,DGP,SENSE); FOR J:=l UNTIL 2 DO BEGIN K:=K+1; IF SM_Y=J THEN BEGIN CASE SMF OF BEGIN RES(K):=G (J)-DSPLN_AL (PTS_TO__HIT (I), SPLN_COEF(*,l),JOINTS(*,1),NJOINTS(1), N_SPLN_PAR(1)); RES(K):=G(J)-DHERM(PTS_TO_HIT(I), HERM_COEF(*,*,l),JOINTS(*,1),NJOINTS(1)) END; END ELSE RES(K):=G(J)-DCREOBS(I); FOR L:=l UNTIL N_PAR DO JAC(K,L):=DGP(J,L); END; END; COMMENT FORM F; F:=0; FOR I:=l UNTIL M DO F:=F+RES(I)**2 ; COMMENT FORM GRADIENT; FOR I:=l UNTIL N_PAR DO BEGIN SUM:=0; FOR J:=l UNTIL M DO SUM:=SUM+JAC(J,I)*RES(J); GRAD(I):=SUM; END; END CRE_FUNC; COMMENT COMMAND PROPER FOLLOWS; N_PTS_HIT:=INTDATA(1); N_PAR:=INTDATA(4); SMF:=INTDATA(5); OUT_SEG:=INTDATA(10); DFIT_LIN:=INTDATA(12); SM_Y:=STATES_OBS(1); CR_Y:=3-SM_Y; M:=2*N_PTS_HIT; COMMENT %FILE EMPTY TEMPORARY FILE -SCl; IF DFIT LIN~=1 THEN APPENDIX B 275 BEGIN COMMENT %OUTPUT ENTER STARTING LAMBDA, RELATIVE AND ABSOLUTE TOLERENCES FOR THE MARQUARDT PROCEDURE %INPUT (LAM),(EPS_R),(EPS_A) %OUTPUT TO -SCI MARQUARDT CALLED IN DFIT OPTION 2 OUTPUT REFERENCE NUMBER IS (OUT_SEG) LAMBDA, RELATIVE, AND ABSOLUTE ERROR TOLERENCES ARE (LAM),(EPS_R),(EPS_A); MARQUARDT(EPS_R,EPS_A,M,N_PAR,CRE_FUNC,PAR,LAM,SVD_AL); END ELSE BEGIN COMMENT LINEAR LEAST SQUARES USING SVD; LONG REAL SING_CUTOFF; LONG REAL ARRAY JAC(1::M,1::N_PAR); LONG REAL ARRAY RES(1::M); LONG REAL ARRAY SP,S,GRAD(1::N_PAR); LONG REAL ARRAY V(1::N_PAR,1::N_PAR); LONG REAL ARRAY A(1::M,1::N_PAR+1); LONG REAL SUM,F; INTEGER K; LONG REAL ARRAY U(1::M,1::N_PAR); INTEGER EFLAG; SING_CUTOFF:=.00001L; CRE_FUNC(PAR,F,RES,JAC,GRAD,EFLAG); FOR J:=l UNTIL N_PAR DO FOR I:=l UNTIL M DO A(I,J):=JAC(I,J); K:=0; FOR I:=l UNTIL N_PTS_HIT DO BEGIN FOR J:=l UNTIL 2 DO BEGIN K:=K+1; IF SM_Y=J THEN BEGIN CASE SMF OF BEGIN A(K,N_PAR+1):=DSPLN_AL(PTS_TO_HIT(I), SPLN_COEF(*,1),JOINTS(*,1),NJOINTS(l), N_SPLN_PAR(1)); A(K,N_PAR+1):=DHERM(PTS_TO_HIT(I), HERM_COEF(*,*,1),JOINTS(*,1),NJOINTS(l)) END; END ELSE A(K,N_PAR+1):=DCREOBS(I); END; END; SVD_AL(A,S,U,V,M,N_PAR,M,N_PAR,1,0,N_PAR); APPENDIX B 276 FOR I:=l UNTIL N_PAR DO IF (S(I)/S(1)>SING_CUTOFF) THEN SP(I):=A(I,N_PAR+1)/S(I) ELSE SP(I):=0. ; FOR I:=l UNTIL N_PAR DO BEGIN SUM:=0. ; FOR J:=l UNTIL N_PAR DO SUM:=SUM+V(I,J)*SP(J); PAR(I):=SUM; END; COMMENT %OUTPUT TO USER LINEAR DFIT-OPTION 2 SINGULAR VALUE REJECTION LEVEL IS (SING_CUTOFF); CRE_FUNC(PAR,F,RES,JAC,GRAD,EFLAG); COMMENT %OUTPUT TO USER IN LINEAR PART SUM OF SQUARES OF RESIDUALS IS (F); END; END DFIT CRE. %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% Iterated derivative f i t t i n g procedure %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE DFITITER(PROCEDURE G_FUN; LONG REAL ARRAY PAR(*); INTEGER ARRAY INTDATA(*); LONG REAL ARRAY OBS_SMOOTH(*,*); INTEGER ARRAY STATES_OBS(*); LONG REAL ARRAY PTS_TO_HIT(*); LONG REAL ARRAY JOINTS(*,*); LONG REAL ARRAY SPLN_COEF(*,*); LONG REAL ARRAY HERM_COEF(*,*,*); • INTEGER ARRAY NJOINTS(*); INTEGER ARRAY N_SPLN_PAR(*); LONG REAL PROCEDURE SPLN_AL,DSPLN_AL,HERM,DHERM; LONG REAL ARRAY CREOBS(*); LONG REAL ARRAY DCREOBS(*)); BEGIN COMMENT DFIT ON 2 STATE VARIABLES TO GO WITH SPARSE GAUSS NEWTON TO IMPROVE GUESSED STATE; INTEGER N_PTS_HIT,N_PAR,SMF,DFIT_LIN,KK; LONG REAL LAM,EPS_R,EPS_A; INTEGER SM_Y,CR_Y,M,OUT_SEG; LONG REAL ARRAY V0(l::2); COMMENT %EXTERNAL SVD_AL,MARQUARDT,CMD_AL; PROCEDURE CRE_FUNC(LONG REAL ARRAY P(*); LONG REAL RESULT F; LONG REAL ARRAY RES(*); APPENDIX B 277 LONG REAL ARRAY JAC(*,*); LONG REAL ARRAY GRAD(*); INTEGER RESULT EFLAG); BEGIN COMMENT THIS PROCEDURE PROVIDES INFORMATION TO THE LEAST SQUARES MINIMIZER WHEN GUESSED OBSERVATIONS ARE USED; LONG REAL ARRAY SENSE,DGY(1::1,1::1) ; LONG REAL ARRAY Y,G,INT_G,PREV_G(1::2); LONG REAL ARRAY DGP,INT_DGP,PREV_DGP(1::2,1::N_PAR); INTEGER K,KK; LONG REAL SUM,DEL; EFLAG:=0; K:=0; Y(SM_Y):=OBS_SMOOTH(l,l); Y(CR_Y):=CREOBS(1); G__FUN(PTS_TO_HIT(l) , Y,P ,-1,PREV_G ,DGY ,DGP , SENSE ) ; G_FUN(PTS_TO_HIT(l),Y,P,2,G,DGY,PREV_DGP,SENSE); FOR I:=2 UNTIL N_PTS_HIT DO BEGIN Y(SM_Y):=OBS_SMOOTH(1,1); Y(CR_Y):=CREOBS(I); G_FUN(PTS_TO_HIT(I),Y,P,-1,G,DGY,DGP,SENSE); G_FUN(PTS_TO_HIT(I),Y,P,2,G,DGY,DGP,SENSE); DEL: = (PTS_TO_HIT(I)-PTS_TO__HIT(I-l) ) ; FOR J:=l UNTIL 2 DO BEGIN K:=K+1; IF SM_Y=J THEN BEGIN CASE SMF OF BEGIN RES (K) :=G(J)-DSPLN_AL(PTS_TO__HIT(I) , SPLN_COEF(*,l),JOINTS(*,1),NJOINTS(1), N_SPLN_PAR(1)); RES(K) :=G(J)-DHERM(PTS_TO__HIT(I) , HERM_COEF(*,*,1),JOINTS(*,1),NJOINTS(1)) END; END ELSE RES(K):=.5L*(G(J)+PREV_G(J))-(CREOBS(I)-CREOBS(I-l))/DEL; KK:=0; FOR L:=l UNTIL N_PAR DO BEGIN KK:=KK+1; IF SM_Y=J THEN JAC(K,KK):=DGP(J,L) ELSE JAC(K,KK):=.5L*(DGP(J,L)+PREV_DGP(J,L)); END; END; COMMENT UPDATE PREV_G,PREV_DGP; APPENDIX B 278 FOR I:=l UNTIL 2 DO PREV_G(I):=G(I); FOR I:=l UNTIL 2 DO FOR J:=l UNTIL N_PAR DO PREV_DGP(I,J):=DGP(I,J); END; COMMENT FORM F; F:=0; FOR I:=l UNTIL M DO F:=F+RES(I)**2; COMMENT FORM GRADIENT; FOR I:=l UNTIL N_PAR DO BEGIN SUM:=0; FOR J:=l UNTIL M DO SUM:=SUM+JAC(J,I)*RES(J); GRAD(I):=SUM; END; END CRE_FUNC; COMMENT COMMAND PROPER FOLLOWS; N_PTS_HIT:=INTDATA(1); N_PAR:=INTDATA(4); SMF:=INTDATA(5); OUT_SEG:=INTDATA(10); DFIT_LIN:=INTDATA(12); SM_Y:=STATES_OBS(1); CR_Y:=3-SM_Y; M:=2*(N_PTS_HIT-1); COMMENT %FILE EMPTY TRMPORARY FILE -SCI; IF DFIT_LIN~=1 THEN BEGIN COMMENT %OUTPUT ENTER STARTING LAMBDA, RELATIVE AND ABSOLUTE TOLERENCES FOR MARQUARDT %INPUT (LAM),(EPS_R),(EPS_A) %OUTPUT TO USER IF IN BATCH, TO -SCI IF NOT, MARQUARDT USED IN DFIT FOR SPARSE GAUSS NEWTON OUTPUT REFERENCE NUMBER IS (OUT_SEG), LAMBDA, RELATIVE AND ABSOLUTE TOLERENCES ARE (LAM),(EPS_R),(EPS_A); MARQUARDT(EPS_R,EPS_A,M,N_PAR,CRE_FUNC,PAR,LAM,SVD_AL) END ELSE BEGIN COMMENT LINEAR LEAST SQUARES USING SVD; LONG REAL SING_CUTOFF; LONG REAL ARRAY JAC(1::M,1::N_PAR); LONG REAL ARRAY RES(1::M); LONG REAL ARRAY SP,S,GRAD(1::N_PAR); LONG REAL ARRAY V(1::N_PAR,1::N_PAR); LONG REAL ARRAY A(1::M,1::N_PAR+1); LONG REAL SUM,F; INTEGER K; APPENDIX B 279 LONG REAL ARRAY U(1::M,1::N_PAR); INTEGER EFLAG; SING_CUTOFF:=.000OIL; CRE_FUNC(PAR,F,RES,JAC,GRAD,EFLAG); FOR J:=l UNTIL N_PAR DO FOR I:=l UNTIL M DO A(I,J):=JAC(I,J); K:=0; FOR I:=2 UNTIL N_PTS_HIT DO BEGIN FOR J:=l UNTIL 2 DO BEGIN K:=K+1; IF SM_Y=J THEN BEGIN CASE SMF OF BEGIN A(K,N_PAR+1):=DSPLN_AL(PTS_TO_HIT(I), SPLN_COEF(*,l),JOINTS(*,1),NJOINTS(1), N_SPLN_PAR(1))-VO(J); A(K,N_PAR+1):=DHERM(PTS_TO_HIT(I), HERM_COEF(*,*,l),JOINTS(*,1),NJOINTS(l))-V0(J) END; END ELSE A(K,N_PAR+1):=(CREOBS(I)-CREOBS(1-1))/ (PTS_TO_HIT(I)-PTS_TO_HIT(I-l)); END; END; SVD_AL(A,S,U,V,M,N_PAR,M,N_PAR,1,0,N,_PAR); FOR I:=l UNTIL N_PAR DO IF (S(I)/S(1)>SING_CUTOFF) THEN SP(I):=A(I,N_PAR+1)/S(I) ELSE SP(I) :=0.; FOR I:=l UNTIL N_PAR DO BEGIN SUM:=0.; FOR J:=l UNTIL N_PAR DO SUM:=SUM+V(I,J)*SP(J); PAR(I):=SUM; END; COMMENT %OUTPUT TO USER SOLUTION IN LINEAR DFIT FOR SPARSE GAUSS NEWTON IS (PAR) SINGULAR VALUES ARE (S) SINGULAR VALUE REJECTION RATIO IS (SING_CUTOFF); CRE_FUNC(PAR,F,RES,JAC,GRAD,EFLAG); COMMENT %OUTPUT TO USER SUM OF SQUARES OF RESIDUALS IS (F); END; END DFITITER. APPENDIX B 280 %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% Integral f i t t i n g using created observations %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE IFIT_CRE(PROCEDURE G_FUN; LONG REAL ARRAY PAR(*); INTEGER ARRAY INTDATA(*); LONG REAL ARRAY OBS_SMOOTH(*,*) ; INTEGER ARRAY STATES__OBS (*) ; LONG REAL ARRAY PTS_TO_HIT(*); LONG REAL ARRAY JOINTS(*,*); LONG REAL ARRAY SPLN_COEF(*,*) ; LONG REAL ARRAY HERM_COEF(*,*,*); INTEGER ARRAY NJOINTS(*); INTEGER ARRAY N_SPLN_PAR(*) ; LONG REAL PROCEDURE SPLN_AL,DSPLN_AL,HERM,DHERM; LONG REAL ARRAY CREOBS(*); LONG REAL ARRAY DCREOBS(*)); BEGIN COMMENT IFIT ON 2 STATE VARIABLES WHEN OBSERVATIONS ON ONE HAVE BEEN GUESSED AT—LINEAR OPTION AVAILABLE; INTEGER N_PTS_HIT,N_PAR,SMF,DFIT_LIN,KK; LONG REAL LAM,EPS_R,EPS_A; INTEGER SM_Y,CR_Y,M,OUT_SEG; LONG REAL ARRAY V0(l::2); COMMENT %EXTERNAL SVD_AL,MARQUARDT,CMD_AL; PROCEDURE CRE_FUNC(LONG REAL ARRAY P(*); LONG REAL RESULT F; LONG REAL ARRAY RES(*); LONG REAL ARRAY JAC(*,*); LONG REAL ARRAY GRAD(*); INTEGER RESULT EFLAG); BEGIN COMMENT THIS PROCEDURE PROVIDES INFORMATION TO THE LEAST SQUARES MINIMIZER WHEN GUESSED OBSERVATIONS ARE USED; LONG REAL ARRAY SENSE,DGY(1::1,1:: 1) ; LONG REAL ARRAY Y, G,INT_G,PREV_G(1::2); LONG REAL ARRAY DGP,INT_DGP,PREV_DGP(1::2,1::N_PAR); INTEGER K,KK; LONG REAL SUM,DEL2; EFLAG:=0; K:=0; Y(SM_Y):=OBS_SMOOTH(1,1); Y(CR_Y):=CREOBS(1); G_FUN(PTS_TO_HIT(l),Y,P,-1,PREV_G,DGY,DGP,SENSE); G_FUN(PTS_TO_HIT(1),Y,P,2,G,DGY,PREV_DGP,SENSE); G_FUN(PTS_TO_HIT(1),INT_G,P,3,G,DGY,DGP,INT_DGP); FOR I:=l UNTIL 2 DO VO (I) : =INT__G (I) ; APPENDIX B 2 FOR I:=2 UNTIL N_PTS_HIT DO BEGIN Y(SM_Y):=OBS_SMOOTH(I,1); Y(CR_Y):=CREOBS(I); G_FUN(PTS_TO_HIT(I),Y,P,-1,G,DGY,DGP,SENSE); G_FUN(PTS_TO_HIT(I),Y,P,2,G,DGY,DGP,SENSE); DEL2:=(PTS_TO_HIT(I)-PTS_TO_HIT(1-1))/2.L; FOR I:=l UNTIL 2 DO INT_G(I):=INT_G(I)+DEL2*(G(I)+PREV_G(I)); FOR I:=l UNTIL 2 DO FOR J:=l UNTIL N_PAR DO INT_DGP(I,J):=INT_DGP(I,J)+DEL2*(DGP(I,J)+PREV_DGP(I,J)) FOR J:=l UNTIL 2 DO BEGIN K:=K+1; IF SM_Y=J THEN BEGIN CASE SMF OF BEGIN RES(K):=INT_G(J)-SPLN_AL(PTS_TO_HIT(I), SPLN_COEF(* f1),JOINTS(*,1),NJOINTS(l), N_SPLN_PAR(1)); RES(K):=INT_G(J)-HERM(PTS_TO_HIT(I), HERM_COEF(*,*,1),JOINTS(*,1),NJOINTS(1)) END; END ELSE RES(K):=INT_G(J)-CREOBS(I); FOR L:=l UNTIL N_PAR DO JAC(K,L):=INT_DGP(J,L); END; COMMENT UPDATE PREV_G,PREV_DGP; FOR I:=l UNTIL 2 DO PREV_G(I):=G(I); FOR I:=l UNTIL 2 DO FOR J:=l UNTIL N_PAR DO PREV_DGP(I,J):=DGP(I,J); END; COMMENT FORM F; F:=0; FOR I:=l UNTIL M DO F:=F+RES(I)**2; COMMENT FORM GRADIENT; FOR I:=l UNTIL N_PAR DO BEGIN SUM:=0; FOR J:=l UNTIL M DO SUM:=SUM+JAC(J,I)*RES(J); GRAD(I):=SUM; END; END CRE_FUNC; COMMENT COMMAND PROPER FOLLOWS; N_PTS_HIT:=INTDATA(1); N_PAR:=INTDATA(4); APPENDIX 282 SMF:=INTDATA(5) ; OUT_SEG:=INTDATA(10) ; DFIT_LIN:=INTDATA(12); SM_Y:=STATES_OBS(1); CR_Y:=3-SM_Y; M:=2*(N_PTS_HIT-1); COMMENT %FILE E,PTY TEMPORARY FILE -SCI; IF DFIT_LIN~=1 THEN BEGIN COMMENT %OUTPUT TO USER ENTER STARTING LAMBDA, RELATIVE AND ABSOLUTE TOLERENCES FOR MARQUARDT PROCEDURE %INPUT (LAM),(EPS_R),(EPS_A) %OUTPUT TO -SCI MARQUARDT USED IN IFIT OPTION 2 LAMBDA, RELATIVE AND ABSOLUTE TOLERENCES ARE (LAM),(EPS_R),(EPS_A); MARQUARDT (EPS_R, EPS_A , M , N__PAR, CRE_FUNC , PAR, LAM , SVD_AL) ; END ELSE BEGIN COMMENT LINEAR LEAST SQUARES USING SVD; LONG REAL SING_CUTOFF; LONG REAL ARRAY JAC (1 : :M , 1: : N__PAR) ; LONG REAL ARRAY RES(1::M); LONG REAL ARRAY SP,S,GRAD(1::N_PAR); LONG REAL ARRAY V(1::N_PAR,1::N_PAR); LONG REAL ARRAY A(1::M,1::N_PAR+1); LONG REAL SUM,F; INTEGER K; LONG REAL ARRAY U(1::M,1::N_PAR); INTEGER EFLAG; SING_CUTOFF:=.0 0001L; CRE_FUNC(PAR,F,RES,JAC,GRAD,EFLAG); FOR J:=l UNTIL N_PAR DO FOR I:=l UNTIL M DO A(I,J):=JAC(I,J); K:=0; FOR I:=2 UNTIL N_PTS_HIT DO BEGIN FOR J:=l UNTIL 2 DO BEGIN K:=K+1; IF SM_Y=J THEN BEGIN CASE SMF OF BEGIN A(K,N_PAR+1):=SPLN_AL(PTS_TO_HIT(I), SPLN_COEF(*,1),JOINTS(*,1),NJOINTS(1), N_SPLN_PAR(1))-VO(J); A(K,N_PAR+1):=HERM(PTS_TO_HIT(I), APPENDIX B 283 HERM_C0EF(*,*,1),JOINTS(*,1),NJOINTS(1))-V0(J) END; END ELSE A(K,N_PAR+1):=CREOBS(I)-VO(J); END; END; SVD_AL(A,S,U,V,M,N_PAR,M,N_PAR,1,0,N_PAR); FOR I:=l UNTIL N_PAR DO IF (S(I)/S(1)>SING_CUTOFF) THEN SP(I):=A(I,N_PAR+1)/S(I) ELSE SP (I) :=0.; FOR I:=l UNTIL N_PAR DO BEGIN SUM:=0.; FOR J:=l UNTIL N_PAR DO SUM:=SUM+V(I,J)*SP(J); PAR(I):=SUM; END; COMMENT %OUTPUT TO USER LINEAR OPTION IN IFIT OPTION 2 PARAMETERS FOUND ARE (PAR), SINGULAR VALUES ARE (S), SINGULAR VALUE REJECTION LEVEL IS (SING_CUTOFF); CRE_FUNC(PAR,F,RES,JAC,GRAD,EFLAG); COMMENT %OUTPUT TO USER SUM OF SQUARES OF RESIDUALS IS (F); END; END IFIT CRE. %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% Iterated integral f i t t i n g (subsystem integration) %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE IFITI(PROCEDURE G_FUN; LONG REAL ARRAY PAR(*); INTEGER ARRAY INTDATA(*); LONG REAL ARRAY OBS_SMOOTH(*,*); INTEGER ARRAY STATES__OBS (* ) ; LONG REAL ARRAY PTS_TO_HIT(*); LONG REAL ARRAY JOINTS(*,*); LONG REAL ARRAY SPLN_COEF(*,*); LONG REAL ARRAY HERM_COEF(*,*,*); INTEGER ARRAY NJOINTS(*); INTEGER ARRAY N_SPLN_PAR(*); LONG REAL PROCEDURE SPLN_AL, DSPLN_AL,HERM,DHERM; LONG REAL ARRAY CREOBS(*); LONG REAL ARRAY DCREOBS(*); PROCEDURE IFIT_CRE); BEGIN APPENDIX B i 284 COMMENT ITERATED IFIT ON 2 STATE VARIABLES INTEGRATION OF SUBSYSTEMS TO UPDATE GUESSED OBSERVATIONS; COMMENT %EXTERNAL TRAP; PROCEDURE G(LONG REAL VALUE T; LONG REAL ARRAY Y(*); LONG REAL ARRAY P(*); INTEGER VALUE OPTION; LONG REAL ARRAY G(*); LONG REAL ARRAY DGY(*,*); LONG REAL ARRAY DGP(*,*); LONG REAL ARRAY ISEN(*,*)); BEGIN COMMENT INTERFACE TO G_FUN WHEN ONLY 1 STATE VAR IS INTEGRATED.; LONG REAL ARRAY Y1,G1(1::2); LONG REAL ARRAY DGP1,ISENl(1::2,1::N_PAR); LONG REAL ARRAY DGYl(1::2,1::2); Yl(CR_Y):=Y(1); Yl(SM_Y):=CASE SMF OF (SPLN_AL(T,SPLN_COEF(*,l),JOINTS(*,l),NJOINTS(l), N_SPLN_PAR(1)), HERM(T,HERM_COEF(*,*,1),JOINTS(*,1),NJOINTS(l))); G_FUN(T,Y1,P,OPTION,GI,DGYl,DGP1,ISENl); IF (OPTION=l) OR (OPTION=-l) THEN G(1):=Gl(CR_Y); IF OPTION=l THEN DGY (1,1) : =DGYl (CR_Y , CR__Y) ; IF OPTION=3 THEN Y(1):=Yl(CR_Y); END G; INTEGER N_PTS_HIT, N_PAR, SMF , OUTPUT , METHOD__FLAG, OUTPUT_SUP , OUT_SEG,INT_PROC,SM_Y,CR_Y,NO_ITER,EFLAG,OUTl; STRING(1) ANS; LONG REAL ARRAY B(1::1,1::INTDATA(1)); LONG REAL ARRAY BJAC(1::1,1::1); COMMENT COMMAND PROPER FOLLOWS; N_PTS_HIT:=INTDATA(1); N_PAR:=INTDATA(4); SMF:=INTDATA(5); OUTPUT:=INTDATA(6); METHOD_FLAG:=INTDATA(7); OUTPUT_SUP:=INTDATA(9); OUT_SEG:=INTDATA(10); INT_PROC:=INTDATA(ll); SM_Y:=STATES_OBS(1); CR_Y:=3-SM_Y; REPT:IFIT_CRE(G_FUN,PAR,INTDATA,OBS_SMOOTH,STATES_OBS, PTS_TO_HIT,JOINTS,SPLN_COEF,HERM_COEF,NJOINTS,N_SPLN_PAR, SPLN_AL,DSPLN_AL,HERM,DHERM, CREOBS,DCREOBS); COMMENT INTEGRATION OF SUBSYSTEM (TRAPEZOIDAL RULE IMPLEMENTED); IF OUTPUT_SUP=l THEN OUTl:=0 ELSE OUTl:OUTPUT; TRAP (PAR, PTS_TO_HIT , B , B JAC , 1, N_PTS__HIT, N_PAR , 0 , G , EFLAG , 0) ; APPENDIX B 285 IF EFLAG=1 THEN GO TO OUT; FOR I:=l UNTIL N_PTS_HIT DO CREOBS(I):=B(1,I); COMMENT %OUTPUT IS ANOTHER ITERATION DESIRED? Y OR N %INPUT (ANS); IF ANS="Y" THEN GO TO REPT; OUT:END IFITI. %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%'• Continuation and quasi-multiple shooting %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%! PROCEDURE CONTI(PROCEDURE G_FUN; LONG REAL ARRAY PAR(*); INTEGER VALUE MODE; INTEGER ARRAY INTDATA(*); LONG REAL ARRAY OBS_SMOOTH(*,*); INTEGER ARRAY STATES_OBS(*); LONG REAL ARRAY PTS_TO_HIT(*); LONG REAL ARRAY JOINTS(*,*); LONG REAL ARRAY SPLN_COEF(*,*); LONG REAL ARRAY HERM_COEF(*,*,*); INTEGER ARRAY NJOINTS(*); INTEGER ARRAY N_SPLN_PAR(*); LONG REAL PROCEDURE SPLN_AL, DSPLN_AL,HERM,DHERM; LONG REAL VALUE EPS,HMIN,HMAX); BEGIN COMMENT CONTINUATION FORM IFIT WITH BREAK POINTS AND WEIGHTING AT BREAK POINTS; INTEGER N_PTS_HIT,N_PAR,SMF,DFIT_LIN,KK; LONG REAL LAM,EPS_R,EPS_A; LONG REAL ARRAY INITY(1::1); LONG REAL ARRAY INITYP(1::1,1::1); INTEGER KFLAG; INTEGER NP,SM_Y,CR_Y,M,OUT_SEC-LONG REAL ARRAY V0(l::2); COMMENT %EXTERNAL TRAP, GEAR, SVD_AL, MARQUARDT, CMD_AL; PROCEDURE C_FUNC(LONG REAL ARRAY P(*); LONG REAL RESULT F; LONG REAL ARRAY RES(*); LONG REAL ARRAY JAC(*,*); LONG REAL ARRAY GRAD(*); INTEGER RESULT EFLAG); BEGIN COMMENT OPTIMIZATION FUNCTION FOR CONTINUATION METHOD; APPENDIX B 286 INTEGER M,N_RES,OUTl; LONG REAL SUM; INTEGER I_BREAK; PROCEDURE CG_FUN(LONG REAL VALUE T; LONG REAL ARRAY Y(*); LONG REAL ARRAY P(*); INTEGER VALUE OPTION; LONG REAL ARRAY G ( * ) ; LONG REAL ARRAY DGY(*,*); LONG REAL ARRAY DGP(*,*); LONG REAL ARRAY ISEN(*,*)); BEGIN COMMENT G_FUN INTERFACE FOR CONTINUATION METHOD USING QUASI MULTIPLE SHOOTING; LONG REAL ARRAY CY(1::N_STATE); IF (ABS OPTION)~=3 THEN BEGIN FOR I:=l UNTIL N_STATE DO CY(I):=Y (I)*GA(I); FOR I:=l UNTIL N_STATE_OBS DO CY(STATES_OBS(I)):=(CASE SMF OF (SPLN_AL(T,SPLN_COEF(*,I),JOINTS(*,I),NJOINTS(I), N_SPLN_PAR(I)), HERM(T,HERM_COEF(*,*,I),JOINTS(*,I),NJOINTS(I)))) *(1.L-GA(STATES_OBS(I)))+CY(STATES_OBS(I)); END; CASE ABS OPTION OF BEGIN BEGIN G_FUN(T,CY,P,OPTION,G,DGY,DGP,ISEN); IF OPTION>0 THEN BEGIN FOR J:=l UNTIL N_STATE DO FOR K:=l UNTIL N_STATE DO DGY(J,K):=DGY(J,K)*GA(K); END; END; BEGIN G_FUN(T,CY,P,OPTION,G,DGY,DGP,ISEN); END; BEGIN IF I_BREAK=1 THEN G_FUN(T,Y,P,OPTION,G,DGY,DGP,ISEN) ELSE BEGIN INTEGER INDX; INDX:=BREAK(I_BREAK-1); FOR I:=l UNTIL N_STATE DO Y(I):=STATE(INDX,I)*GA1(I); FOR I:=l UNTIL N_STATE_OBS DO Y(STATES_OBS(I)):=(CASE SMF OF ( SPLN_AL(T,SPLN_COEF(*,I),JOINTS(*,I), APPENDIX B 287 NJOINTS(I),N_SPLN_PAR(I) ) , HERM(T,HERM_COEF(*,*,I),JOINTS(*,I),NJOINTS(I))))* (1.L-GA1(STATES_OBS(I)))+Y(STATES_OBS(I)); IF OPTION>0 THEN FOR I:=l UNTIL N_STATE DO FOR J:=l UNTIL N_PAR DO ISEN(I, J):=GA1(I)*JACOBIAN((INDX-1)*N_STATE+I,J); END; END END; END CG_FUN; FOR I:=l UNTIL N_BREAK DO BEGIN COMMENT INTEG TO BREAK(I); LONG REAL ARRAY SAMPLE(1::BREAK(I)-BREAK(I-1)+1); INTEGER N_PTS; LONG REAL ARRAY B(1::N_STATE,1::BREAK(I)-BREAK(1-1)+1); LONG REAL ARRAY JB(1::N_PAR,1::(BREAK(I)-BREAK(1-1)+1)*N_STATE); INTEGER INDX,JI; I_BREAK:=I; N_PTS:=BREAK(I)-BREAK(1-1)+1; FOR J:=l UNTIL N_PTS DO SAMPLE(J):=PTS_TO_HIT(BREAK(I)-N_PTS+J); COMMENT INTEGRATE; IF OUTPUT_SUP=l THEN OUT1:=0 ELSE OUTl:=OUTPUT; EFLAG:=0; CASE INT_PROC OF BEGIN BEGIN GEAR(P,SAMPLE,B,JB,N_STATE,N_PTS,EPS,HMIN,HMAX, N_PAR,1,CG_FUN,KFLAG,OUTl,METHOD_FLAG,0, INITY,INITYP); IF KFLAG~=1 THEN BEGIN EFLAG:=1;GO TO OUT;END; END; BEGIN TRAP(P,SAMPLE,B,JB,N_STATE,N_PTS,N_PAR, 1,CG_FUN,EFLAG,OUTl); IF EFLAG=1 THEN GO TO OUT; END END; JI:=1; INDX:=BREAK(I_BREAK-1); IF INDX~=1 THEN JI:=2; FOR J:=JI UNTIL N_PTS DO FOR K:=l UNTIL N_STATE DO BEGIN STATE(INDX-1+J,K):=B(K,J); FOR L:=l UNTIL N_PAR DO JACOBIAN((INDX-2+J)*N_STATE+K,L):= JB(L,(J-1)*N_STATE+K); APPENDIX B 288 END ; END I; COMMENT EXTRACT NON WEIGHTED JACOBIAN AND FORM RESIDUAL; FOR K:=l UNTIL N_PAR DO BEGIN M:=0; FOR I:=l UNTIL N_PTS_HIT DO FOR J:=l UNTIL N_STATE_OBS DO BEGIN M:=M+1; JAC(M,K):=JACOBIAN((1-1)*N_STATE+STATES_OBS(J),K); RES(M):=STATE(I,STATES_OBS(J))-(CASE SMF OF( SPLN_AL(PTS_TO_HIT(I),SPLN_COEF(*,J),JOINTS(*,J) ,NJOINTS(J), N_SPLN_PAR(J)), HERM(PTS_TO_HIT(I),HERM_COEF(*,*,J),JOINTS(*,J), NJOINTS(J)))) END; END; N_RES:=M; COMMENT SPECIAL WEIGTING OF BREAK POINTS; FOR I:=l UNTIL N__BREAK DO BEGIN INTEGER INDX; INDX:=(BREAK(I)-1)*N_STATE_OBS; FOR J:=l UNTIL N_STATE_OBS DO RES(INDX+J):=RES(INDX+J)*W(I); FOR J:=l UNTIL N_STATE_OBS DO FOR K:=l UNTIL N_PAR DO JAC(INDX+J,K):=JAC(INDX+J,K)*W(I); END; F:=0.L; FOR I:=l UNTIL N_RES DO F:=F+RES(I)**2; COMMENT FORM GRADIENT; FOR I:=l UNTIL N_PAR DO BEGIN SUM:=0.L; FOR J:=l UNTIL N_RES DO SUM:=SUM+JAC(J,I)*RES(J); GRAD(I):=SUM; END; OUT:END C_FUNC; INTEGER N_STATE , N_STATE_OBS , OUTPUT__SUP , OUTPUT, INT_PROC , METHOD , METHOD__FLAG; LONG REAL ARRAY STATE(1::INTDATA(1),1::INTDATA(3)); LONG REAL ARRAY JACOBIAN(1::INTDATA(1) *INTDATA(3),1::INTDATA(4)); INTEGER N_BREAK; INTEGER ARRAY BREAK(0::50); APPENDIX B 289 LONG REAL ARRAY W(0::50); LONG REAL ARRAY GA,GA1(1::INTDATA(3)); N_PTS_HIT:=INTDATA(1); N_STATE_OBS:=INTDATA(2); N_STATE:=INTDATA(3); N_PAR:=INTDATA(4); SMF:=INTDATA(5); OUTPUT:=INTDATA(6); METHOD_FLAG:=INTDATA(7) ; OUTPUT_SUP:=INTDATA(9); OUT_SEG:=INTDATA(10); INT_PROC:=INTDATA(ll); IF MODE=0 THEN BEGIN INTEGER N__BRK; LONG REAL GAM; COMMENT %OUTPUT TO USER ENTER NUMBER OF BREAK POINTS ENTER 0 FOR NO BREAK POINTS %INPUT (N_BRK); IF N_BRK~=0 THEN BEGIN COMMENT %OUTPUT TO USER ENTER SAMPLE TIME SUBSCRIPTS FOR BREAK POINTS DO NOT INCLUDE FIRST OR LAST SAMPLE TIME; FOR I:=l UNTIL N_BRK DO COMMENT %INPUT BREAK(I); BREAK(0):=1;BREAK(N_BRK+1):=N_PTS_HIT; N_BREAK:=N_BRK+1; END ELSE BEGIN BREAK(0):=1; BREAK(1):=N_PTS_HIT; N_BREAK:=1; END; COMMENT %OUTPUT ENTER CONTINUATION PARAMETER FOR INITIAL VALUE PROBLEM %INPUT (GAM); FOR I:=l UNTIL N_STATE DO GA(I):=GAM; IF N_BRK~=0 THEN BEGIN COMMENT %OUTPUT ENTER CONTINUATION PARAMETER FOR BREAK POINTS. EACH COMPONENT CORRESPONDS TO THE STATE VARIABLE WITH THE SAME SUBSCRIPT; FOR I:=l UNTIL N_STATE DO COMMENT %INPUT (GAl(I)) %OUTPUT ENTER WEIGHTS AT BREAK POINTS APPENDIX B 290 ONE ENTRY FOR EACH BREAK POINT; FOR I:=l UNTIL N_BRK DO COMMENT %INPUT (W(I)); W(0):=1.L; W(N_BREAK):=1.L; END ELSE W(l):=1.L; END ELSE BEGIN COMMENT MODE NOT 0. IFIT OPTION; N_BREAK:=1; BREAK(0):=1; BREAK(1):=N_PTS_HIT; FOR I:=l UNTIL N_STATE DO GA(I):=0.L; W(l):=1.L; END; COMMENT %FILE EMPTY TEMPORARY FILE -SCI %OUTPUT ENTER STARTING LAMBDA, REL AND ABS TOL FOR MARQUARDT %INPUT (LAM),(EPS_R),(EPS_A); COMMENT %OUTPUT TO USER IF IN BATCH, TO -SCI IF NOT, MARQUARDT USED IN CONTINUATION PROCEDURE OUTPUT REFERENCE NUMBER (OUT_SEG), LAMBDA (LAM), RELATIVE ERROR TOLERENCE (EPS_R), ABSOLUTE ERROR TOLERENCE (EPS_A); MARQUARDT(EPS_R,EPS_A,N_PTS_HIT*N_STATE_OBS, N_PAR,C_FUNC,PAR,LAM,SVD_AL); END CONTI. %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% Description of externally defined procedures %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% %%%% PROCEDURE CMD_AL(STRING(80) VALUE CMD; INTEGER VALUE N); BEGIN COMMENT EXECUTE AN MTS COMMAND; END. PROCEDURE CHECK_BATCH(LOGICAL RESULT BATCH); BEGIN COMMENT CHECK IF IN BATCH MODE; END. PROCEDURE SVD_AL(LONG REAL ARRAY AD(*,*); LONG REAL ARRAY S(*); LONG REAL ARRAY UD(*,*); LONG REAL ARRAY VD(*,*); INTEGER VALUE MDIM,NDIM,M,N,NRHS,NU,NV); APPENDIX B 291 BEGIN COMMENT SINGULAR VALUE DECOMPOSITION USES U.B.C. FORTRAN VERSION OF PROCEDURE BY G. GOLUB AND C. REINSCH, NUMER. MATH. 14 (1970) 403-420; END. PROCEDURE G_FUN(LONG REAL VALUE T; LONG REAL ARRAY Y(*);LONG REAL ARRAY P(*); INTEGER VALUE OPTION;LONG REAL ARRAY G(*); LONG REAL ARRAY DGY(*,*);LONG REAL ARRAY DGP(*,*); LONG REAL ARRAY PREV_SENSE(*,*)); BEGIN COMMENT MODEL DEFINITION ; END. PROCEDURE ECHOl(INTEGER VALUE CAS); BEGIN COMMENT ECHO IBM 3270 CONVERSATION BUFFER (AS USED BY MTS AT U.B.C.) AND RESET POINTERS SO NO OVERLAP OCCURS; END. PROCEDURE SPLINT_AL(LONG REAL ARRAY X(*); LONG REAL ARRAY Y(*);LONG REAL ARRAY W(*); INTEGER VALUE N;LONG REAL ARRAY P(*); INTEGER VALUE RESULT M;LONG REAL ARRAY XJOINTS(*); INTEGER VALUE NJOINT); BEGIN COMMENT LEAST SQUARES SPLINE APPROXIMATION; END. LONG REAL PROCEDURE SPLN_AL(LONG REAL VALUE X; LONG REAL ARRAY P(*);LONG REAL ARRAY XJOINT(*); INTEGER VALUE NJOINT,M); BEGIN COMMENT CALCULATE SPLINE APPROXIMATION USING SPLINT_AL RESULTS; END. LONG REAL PROCEDURE DSPLN_AL(LONG REAL VALUE X; LONG REAL ARRAY P(*);LONG REAL ARRAY XJOINT(*); INTEGER VALUE NJOINT,M); BEGIN COMMENT CALCULATE DERIVATIVE APPROXIMATION USING SPLINT_AL RESULTS; END. PROCEDURE HERMIT_AL(LONG REAL ARRAY X(*); LONG REAL ARRAY Y(*);LONG REAL ARRAY KNOTS(*); INTEGER VALUE N,NKNOTS;LONG REAL ARRAY COEF(*,*); INTEGER VALUE NCI;INTEGER RESULT FLAG); BEGIN ' COMMENT LEAST SQUARES PIECEWISE CUBIC HERMITE APPROXIMATION; END. LONG REAL PROCEDURE HERM(LONG REAL VALUE X; APPENDIX B 292 LONG REAL ARRAY COEF(*,*);LONG REAL ARRAY JOINTS(*); INTEGER VALUE NJOINTS); BEGIN COMMENT CALCULATE HERMITE APPROXIMATION USING RESULTS OF HERMIT_AL; END. LONG REAL PROCEDURE DHERM(LONG REAL VALUE X; LONG REAL ARRAY COEF(*,*)?LONG REAL ARRAY JOINTS(*); INTEGER VALUE NJOINTS); BEGIN COMMENT CALCULATE DERIVATIVE APPROXIMATION USING RESULTS OF HERMIT_AL; END. PROCEDURE CREATE_DATA(LONG REAL ARRAY SIM_PAR(*); LONG REAL ARRAY START_PAR(*);LONG REAL ARRAY PAR(*); LONG REAL ARRAY OBS_PLACE(*);LONG REAL ARRAY PTS_TO_HIT(*); LONG REAL ARRAY OBS(*,*);LONG REAL ARRAY STATE(*,*); LONG REAL ARRAY JACOBIAN(*,*);INTEGER ARRAY OBS_STATUS(*); INTEGER ARRAY STATES_OBS(*);INTEGER ARRAY INTDATA(*); STRING(31) VALUE MODEL;PROCEDURE EG_FUN,GEAR,STANDARD_HIT; LONG REAL RESULT STD_DEV;LONG REAL VALUE INITIAL_TIME; INTEGER VALUE KFLAG,OUTPUT); BEGIN COMMENT SIMULATE OBSERVATIONS; END. LONG REAL PROCEDURE FVALUE_AL(LONG REAL VALUE P; INTEGER VALUE N1,N2); BEGIN COMMENT STATISTICAL F DISTRIBUTION; END. PROCEDURE SMT(REAL ARRAY X(*); REAL ARRAY Y(*);REAL ARRAY P(*); REAL ARRAY SI(*);REAL ARRAY T(*); REAL ARRAY S(*);REAL ARRAY S l ( * ) ; REAL ARRAY S2(*);INTEGER VALUE N,IFF,M); BEGIN CUBIC SPLINE INTERPOLATION; END. PROCEDURE FSLE_AL(INTEGER VALUE N,NDIMA; LONG REAL ARRAY A(*,*);INTEGER VALUE NSOL,NDIMBX; LONG REAL ARRAY B ( *,*) ;LONG REAL ARRAY X(*,*); INTEGER ARRAY IPERM(*);INTEGER VALUE NDIMT; LONG REAL ARRAY TMP(*,*)); BEGIN COMMENT SOLVE A LINEAR SYSTEM OF EQUATIONS; END. PROCEDURE DIFF(INTEGER VALUE N; LONG REAL VALUE RESULT LT;LONG REAL ARRAY LY(*,*); LONG REAL ARRAY LSAVE(*,*);LONG REAL VALUE RESULT LH; LONG REAL VALUE LHMIN,LHMAX,LEPS;INTEGER VALUE METHOD_FLAG; LONG REAL ARRAY LYMAX(*);LONG REAL ARRAY LERROR(*); APPENDIX B INTEGER VALUE RESULT KFLAG; INTEGER VALUE RESULT JSTART; INTEGER VALUE MAXDER;LONG REAL ARRAY A A ( * ) ; INTEGER RESULT ORDER;PROCEDURE FUN; LONG REAL ARRAY P ( * ) ; R E A L ARRAY PW(*)); BEGIN COMMENT ALGOL FORTRAN INTERFACE TO GEAR'S CODE; END. PROCEDURE ECHO(INTEGER VALUE C A S ) ; BEGIN COMMENT SIM I L A R TO E C H O l — U S E D I N INTERACTIVE F I T ; END. PROCEDURE ALGRAF_AL(REAL ARRAY X ( * ) ; REAL ARRAY Y ( * ) ; INTEGER VALUE N,NS); BEGIN COMMENT PLOT A SET OF DATA POINTS; END. PROCEDURE PLOT_AL(REAL VALUE X,Y;INTEGER VALUE IPEN) BEGIN COMMENT MOVE PLOTTING PEN TO ( X , Y ) , CAN BE UP OR DOWN DEPENDING ON I P E N ; END. PROCEDURE A L S I Z E _ A L ( R E A L VALUE X S I Z E , Y S I Z E ) ; BEGIN COMMENT SET S I Z E OF PLOT; END. PROCEDURE A L S C A L _ A L ( R E A L VALUE XMIN,XMAX,YMIN,YMAX); BEGIN COMMENT SCALE DATA TO F I T PLOT; END. PROCEDURE A L A X I S _ A L ( S T R I N G ( 5 0 ) VALUE LABELX; INTEGER VALUE NX;STRING(50) VALUE LABELY; INTEGER VALUE N Y ) ; BEGIN COMMENT DRAW AXES; END. PROCEDURE DFBAND(LONG REAL ARRAY D A ( * ) ; LONG REAL ARRAY DB(*);INTEGER VALUE N,LHB,NRHS; LONG REAL VALUE RESULT RATIO;LONG REAL RESULT DET; INTEGER RESULT JEXP;INTEGER VALUE NS C A L E ) ; BEGIN COMMENT SOLVE A BANDED SYSTEM OF LINEAR EQUATIONS WITH A P O S I T I V E D E F I N I T E MATRIX USING A CHOLESKY DECOMPOSITION; END.
- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Numerical parameter estimation in differential equations
Open Collections
UBC Theses and Dissertations
Featured Collection
UBC Theses and Dissertations
Numerical parameter estimation in differential equations Benson, Maurice W. 1977
pdf
Page Metadata
Item Metadata
Title | Numerical parameter estimation in differential equations |
Creator |
Benson, Maurice W. |
Date Issued | 1977 |
Description | The problem of numerical least squares parameter estimation in differential equations is considered. Several new algorithms that pay particular attention to the differential equation aspect of the problem are presented. These reduce some of the difficulties encountered when the problem is treated solely as a question of nonlinear optimization. The extremely powerful interactive approach is considered and an interactive package incorporating standard techniques using sensitivity equations along with a selection of our special algorithms is presented. We consider methods involving the fitting of integrals and derivatives using piecewise polynomial approximations to the observations. Continuation methods with a quasi multiple shooting technique to bridge the gap between these coarse but well behaved methods and the full least squares problem are explored. Special methods are developed for the important case of two state variables with observations available on only one of them. In particular we consider algorithms which use an initial guess at the behavior of the unobserved state variable and then iteratively improve this guess. The need for effective algorithms for fitting population growth models in ecology is one motivation for this thesis. We devote a chapter to an important predator-prey model of population dynamics and extensive experiments are presented which demonstrate some of the typical difficulties which can arise and which illustrate the ability of our algorithms to overcome some of these difficulties. Some special problems involving jumps from one equilibrium to another (loosely referred to as catastrophes) are examined. This type of model has important applications in ecology. Models involving stiff differential equations are also considered. A short chapter is devoted to the use of sequential reestimation techniques. Experiments indicate that such methods can be useful for improving a crude initial guess at the parameters and this improvement can be crucial for the successful solution of the problem. Finally a chapter is devoted to a selection of "real world" problems. It is on such problems that the true value of an algorithm is determined. |
Genre |
Thesis/Dissertation |
Type |
Text |
Language | eng |
Date Available | 2010-03-01 |
Provider | Vancouver : University of British Columbia Library |
Rights | For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use. |
DOI | 10.14288/1.0051795 |
URI | http://hdl.handle.net/2429/21231 |
Degree |
Doctor of Philosophy - PhD |
Program |
Computer Science |
Affiliation |
Science, Faculty of Computer Science, Department of |
Degree Grantor | University of British Columbia |
Campus |
UBCV |
Scholarly Level | Graduate |
AggregatedSourceRepository | DSpace |
Download
- Media
- 831-UBC_1978_A1 B45.pdf [ 12.49MB ]
- Metadata
- JSON: 831-1.0051795.json
- JSON-LD: 831-1.0051795-ld.json
- RDF/XML (Pretty): 831-1.0051795-rdf.xml
- RDF/JSON: 831-1.0051795-rdf.json
- Turtle: 831-1.0051795-turtle.txt
- N-Triples: 831-1.0051795-rdf-ntriples.txt
- Original Record: 831-1.0051795-source.json
- Full Text
- 831-1.0051795-fulltext.txt
- Citation
- 831-1.0051795.ris
Full Text
Cite
Citation Scheme:
Usage Statistics
Share
Embed
Customize your widget with the following options, then copy and paste the code below into the HTML
of your page to embed this item in your website.
<div id="ubcOpenCollectionsWidgetDisplay">
<script id="ubcOpenCollectionsWidget"
src="{[{embed.src}]}"
data-item="{[{embed.item}]}"
data-collection="{[{embed.collection}]}"
data-metadata="{[{embed.showMetadata}]}"
data-width="{[{embed.width}]}"
async >
</script>
</div>
Our image viewer uses the IIIF 2.0 standard.
To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0051795/manifest