Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

The derivation of optimal control laws and the synthesis of real-time optimal controllers for a class.. 1965

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
UBC_1965_A1 C5.pdf [ 4.92MB ]
Metadata
JSON: 1.0103261.json
JSON-LD: 1.0103261+ld.json
RDF/XML (Pretty): 1.0103261.xml
RDF/JSON: 1.0103261+rdf.json
Turtle: 1.0103261+rdf-turtle.txt
N-Triples: 1.0103261+rdf-ntriples.txt
Citation
1.0103261.ris

Full Text

The U n i v e r s i t y of B r i t i s h Columbia FACULTY OF GRADUATE STUDIES PROGRAMME OF THE FINAL ORAL EXAMINATION FOR THE DEGREE OF DOCTOR OF PHILOSOPHY of WAH-CHUN CHAN B. S c , National Taiwan University, 1958 M.Sc, The University of New Brunswick, 1961 TUESDAY, FEBRUARY 23, 1965, AT 3:30 P.M. IN ROOM 208, MacLEOD BUILDING COMMITTEE IN CHARGE Chairman: I. McT. Cowan E. V. Bohn F. Noakes C. Brockley A. C, Soudack E. Leimanis- Y. N. Yu External Examiner: I. H. Mufti National Research Council D i v i s i o n of Mechanical Engineering Ottawa, Ontario THE DERIVATION OF OPTIMAL CONTROL LAWS AND THE SYNTHESIS OF REAL-TIME OPTIMAL CONTROLLERS FOR A CLASS OF DYNAMIC SYSTEMS ABSTRACT A method f or the so l u t i o n of a c l a s s of optimal control problems based on a modified steepest descent method i s discussed. This method i s suitable f o r the solut i o n of problems i n v a r i a t i o n a l calculus of the Mayer type^ and can be used to r e a l i z e comparatively simple on-line optimal c o n t r o l l e r s by means of analogue computer techniques. The essence of the modified steepest descent method i s to search for the optimum value of a performance function by replacing a search i n function space by a search i n parameter space. In general, an i t e r a t i v e type of search f o r the optimum value of the perfor- mance function i s required. However, i n c e r t a i n classes of problems the optimal control v a r i a b l e can be expressed as a function of the system state variables and no i t e r a t i o n i s necessary. Several optimal control problems for the rocket f l i g h t problem are studied and optimal control laws are derived as functions of the system state v a r i a b l e s . Experimental r e s u l t s show that the method i s very s a t i s f a c t o r y . A PACE 231-R analogue computer i s used to solve the sounding rocket problem. A more complex problem, the two-dimensional z e r o - l i f t rocket f l i g h t problem, i s solved using the modified method of steepest descent and an electromechanical f l i g h t simu- l a t o r . The experimental r e s u l t s obtained with the f l i g h t simulator show that the modified steepest descent method i s p r a c t i c a l and show promise of being useful i n the design of real-time optimal c o n t r o l l e r s . GRADUATE STUDIES F i e l d of Study: E l e c t r i c a l Engineering . Analogue Computers ' E. V. Bohn El e c t r o n i c Instrumentation F. K. Bowers Nonlinear Systems A. C. Soudack Applied Electromagnetic Theory G. B. Walker Related Studies: Theory and Applications of D i f f e r e n t i a l Equations J . F. Scott-Thomas Function of a Complex Variable Hsin Chu Dynamical Systsms I E. Leimanis THE DERIVATION OF OPTIMAL CONTROL LAVS AND THE SYNTHESIS OP REAL-TIME OPTIMAL CONTROLLERS FOR A CLASS OF DYNAMIC SYSTEMS by VAH-CHUN CHAN B.Sc, National Taiwan University, 1958 M.Sc, The University of New Brunswick, 1961 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENT FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in the Department of E l e c t r i c a l Engineering We accept this thesis as conforming to the required standard Members of the Department of E l e c t r i c a l Engineering The University of B r i t i s h Columbia JANUARY, 196 5 In p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t of the requirements f o r an advanced degree at the U n i v e r s i t y of B r i t i s h Columbia, I agree that the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r reference and study. I f u r t h e r agree that per- m i s s i o n f o r extensive copying of t h i s t h e s i s f o r s c h o l a r l y purposes may be granted by the Head of my Department or by h i s r e p r e s e n t a t i v e s . I t i s understood t h a t , c o p y i n g or p u b l i - c a t i o n of t h i s t h e s i s f o r f i n a n c i a l gain s h a l l not be allowed without my w r i t t e n permission* Department of E(z,Cstsi*-*i~*-^ llsi*- The U n i v e r s i t y of B r i t i s h Columbia, Vancouver 8, Canada Date ABSTRACT A method for the solution of a class of optimal control problems based on ̂ m o d i f i e d steepest descent method i s discussed. This method i s suitable f o r the solution of problems i n v a r i a t i o n a l calculus of the Mayer type, and can be used to r e a l i z e comparatively simple on-line optimal controllers by means of analogue computer techniques* The essence of the modified steepest descent method i s to search f o r the optimum value of a performance function by replacing a search i n function space by a search i n parameter space. In general, an i t e r a t i v e type of search for the optimum value of the performance function i s required* However, i n certain classes of problems the optimal control variable can be expressed as a function of the system state variables and no i t e r a t i o n i s necessary© Several optimal control problems for the rocket f l i g h t problem are studied and optimal control laws are derived as functions of the system state variables* Experimental results show that the method i s very s a t i s f a c t o r y . A FACE 231-R analogue computer i s used to solve the sounding rocket problem* A more complex problem^ the two—dimensional z e r o - l i f t rocket f l i g h t problem, i s solved using the modified method of steepest descent and an electromechanical f l i g h t simulator. The experi- mental results obtained with the f l i g h t simulator show that the modified steepest descent method i s p r a c t i c a l and show promise of being useful i n the design of real-time optimal controllers* TABLE OF CONTENTS Page LIST OF ILLUSTRATIONS v i ACKNOWLEDGEMENT v i i 1. INTRODUCTION .. 1 1.1 H i s t o r i c a l Note on the Theory of Optimal Processes 1 1.2 The P r i n c i p l e of O p t i m a l i t y 2 1.3 The Method of Dynamic Programming 2 1.3.1 The P r i n c i p l e of O p t i m a l i t y as a Numerical Technique 4 1.3.2 The Problem of D i m e n s i o n a l i t y ....... 5 1.3.3 The Euler-Lagrange Equations ........ 6 1.3.4 The Legendre-Clebsch C o n d i t i o n ...... 7 1.3.5 The Weierstrass C o n d i t i o n ........... 8 1.3.6 The T r a n s v e r s a l i t y C o n d i t i o n ........ 8 1.3.7 The Veierstrass-Erdmann Corner Co n d i t i o n s 9 1.3.8 The I n e q u a l i t y C o n s t r a i n t 10 1.3.9 The Lagrange M u l t i p l i e r s 10 1.3.10 The Dynamic Programming Approach f o r the Case of Two F i x e d End P o i n t s 15 1.4 The Gradient Method 16 1.4.1 Numerical Computation by the Steep- e s t Descent Method 20 1.4.2 The Steepest Descent Method f o r F i n d i n g the Minimum of a F u n c t i o n a l . 20 1.5 The C a l c u l u s of V a r i a t i o n s and the Theory of Optimal C o n t r o l 21 1.6 The A d j o i n t System and the Euler-Lagrange Equation 24 1.7 The Maximum P r i n c i p l e 25 i i i Page 1.8 The F i r s t Integral 28 1.9 The Modified Steepest Descent Method 29 1.10 Remarks 34 2. OPTIMAL CONTROL PROCESSES FOR ROCKET FLIGHT PROBLEMS 36 2.1 Introduction 36 2.2 Formulation of Rocket F l i g h t Problems by Means of the Calculus of Variations 36 2.2.1 Basic Assumptions and Equations of Motion 37 2.2.2 Formulation of the Rocket F l i g h t Problem 38 2.3 Analytical Study of Optimal Control for the Sounding Rocket Problem 40 3. OPTIMAL FEEDBACK CONTROL SYSTEMS 61 3.1 Introduction * 61 3.2 The Concept of Optimal Feedback Control and the Synthesis of Optimal Controllers ....... 61 3.2.1 A Multivariable Optimal Feedback Control System 64 3.2.2 Synthesis of Optimal Control Laws for Rocket F l i g h t 64 3.3 Analogue Computer Technique for the Synthesis of Optimal Controllers 73 3.4 Analogue Computer Study of the Sounding Rocket Problem 75 3.5 Some Other Possible Optimal Controllers .... 81 4. THE MODIFIED STEEPEST DESCENT METHOD 83 4.1 Introduction . 83 4.2 Basic Concept of the Modified Steepest Descent Method .... 83 4.3 P o s s i b i l i t y of P r a c t i c a l Applications ...... 88 4.4 Further Investigations .« 88 iv Page 5. FLIGHT SIMULATOR AND ANALOGUE SIMULATION 91 5.1 Introduction .. 91 5.2 Basic Components of a F l i g h t Simulator ...... 91 5.3 Simulation of the Optimal Control Law ....... 92 5.4 Analysis of a Test Problem .................. 93 5.5 Experimental Test of the Modified Steepest Descent Method 96 6. CONCLUSION 104 REFERENCES 106 APPENDIX . The Euler-Lagrange Equations for Rocket F l i g h t Problems 107 v LIST OF ILLUSTRATIONS Figure Page 1.1 The Fi n a l Stage and the Terminal Condition .. 16 1.2 A General Optimal Process 31 1.3 True Minimum and Local Minimum 33 2.1 The Forces Acting on a Rocket 37 2.2 The State Variables 60 2.3 The Lagrange M u l t i p l i e r s 60 3.1 A General Multivariable Optimal Feedback Con- t r o l System 63 3.2 A Multivariable Optimal Feedback Control System 65 3.3 The Modes of Control for Optimal Rocket F l i g h t 74 3.4 Synthesis of Optimal Controllers by Means of Analogue Computers 76 3.5 Analogue Computer Program for the Sounding Rocket Problem 77 3.6 Experimental Results for the Sounding Rocket Problem 79 4.1 An Optimal Controller for a General Process . 89 5.1 Three Modes of Thrust Control 94 5.2 A Subclass of Admissible Trajectories ....... 101 5.3 Determination of Approximate I n i t i a l Values of the State Variables 101 5.4 A Parti c u l a r Set of Approximate I n i t i a l Values of the State Variables 102 5.5 Optimum Performance Function 103 v i ACKNOWLEDGEMENT The author wishes to express his profound gratitude to his supervising professor, Dr. E.V. Bohn, for continuous guidance and assistance throughout the research project and also for great patience i n reading the manuscript of thi s t h e s i s . In the progress of the project, the author i s deeply indebted to Dr. Y.N. Yu who has always given him encouragement. The author would l i k e to thank Dr. E. Leimanis of the Mathematics Department for many helpful suggestions, Mr. F.G. Berry for assistance with the modification of the CF-100 f l i g h t simulator, and Mr. J.W* Sutherland for assistance i n the solution of the sounding rocket problem on the PACE 231-R analogue computer. Sincere appreciations are expressed for the award of a studentship for 1961-1964 by the National Research Council and Dr. F. Noakes, Head of the E l e c t r i c a l Engineering Department. Sincere appreciation i s also expressed for the jo i n t f i n a n c i a l support of thi s project by the National Research Council term grant A.68 and the Defe nee Research Board grant No. 4003—01. 1. INTRODUCTION 1.1 H i s t o r i c a l Note on the Theory of Optimal Processes The c l a s s i c a l theory of the calculus of variations was developed by Euler and Lagrange at the end of the eighteenth century. Euler obtained the necessary condition for a relat i v e weak minimum i n the form of an equation, now known as the Euler equation, Lagrange introduced the Lagrange mu l t i p l i e r to f a c i l i t a t e the formulation of minimum problems subject to con- s t r a i n t s . The Lagrange equation i n mechanics has the same form as the Euler equation. The Euler equation i s , therefore, also referred to as the Euler-Lagrange equation. In this thesis the name Euler-Lagrange equation instead of Euler equation i s used. The method of dynamic programming was developed by B e l l - man i n the l a s t decade and i s e s s e n t i a l l y a numerical technique suited for d i g i t a l computation. Recently Pontryagin developed a mathematically rigorous theory of optimal control which i s called the maximum p r i n c i p l e . A further computational technique available to solve minimum problems i s the gradient method or the method of steepest descent. The gradient method has been applied by Kelley for solving optimal f l i g h t path problems A similar scheme has been developed by Bryson and his colleagues^^ • Bohn^*^ has presented a modified approach for solving optimal control problems which appears suitable for computing the instantaneous control policy i n real time . This thesis i s concerned with the development of thi s method which, for reasons that w i l l be given l a t e r i n the thesis, i s called the modified steepest descent method. Chapter 1 gives a br i e f review of the various techniques 2 mentioned above. 1.2 The Prin c i p l e of Optimality (5) The p r i n c i p l e of optimality ' states that "an optimal p o l i c y has the property that whatever the i n i t i a l state and the i n i t i a l decision are, the remaining decisions must con- sti t u t e an optimal policy with regard to the state resulting from the f i r s t decision". This p r i n c i p l e plays the fundamental role i n the theory of dynamic programming. (6) 1.3 The Method of Dynamic Programming . The theory of dynamic programming i s based on the pr i n c i p l e of optimality. It gives a systematic approach for determining a numerical solution to minimum problems. In theory, dynamic programming i s a very general approach, how- ever, i n practice, i t has r e s t r i c t e d a p p l i c a b i l i t y because of the problem of dimensionality. In this section the basic technique of dynamic programming i s discussed. Consider the problem of minimizing the functional J J(x) =\ F(t,x,x) dt (1.1) where the vector notation / \ * _ dx x — v x^»»«»» x ny» x — and x(0) = c = (c1t»..,cn) i s used. The dynamic programming approach to minimizing J i s to consider the function f (t,x) = Min \ F(r,x, *£) dT (1.2) I t i s evident that and that f(T,x(T)) = 0 f(0,c) = Min J(x) The p r i n c i p l e of optimality applied to (1.2) y i e l d s f( t , x ) = Min l x J ,-t+At t F<T,x, )dT +' -T >+At F( T,x, |^ ) dT F ( t , X j X ) A t + f(t+At,x+xAt) + O(At) (1.3) (1.4) Thus f ( t , x ) = Min f *\ where O(At) indicates terms of the order of (At) . Expanding (l.4) i n a power series about (t,x) and l e t t i n g At —»- 0, yi e l d s (1.5) F(t,x,x) + ^ + 2^ ^ X j .1=1 3 0 = Min \ •J The solution of (1.5) must s a t i s f y the following two nonlinear p a r t i a l d i f f e r e n t i a l equations F + & + V Of_ ; = 0 rYt £—> OX. 1 n 6 f 6 t 3=1 6: (1.6) and 6 F _ ^ 6 f 6 x . 6 x . = 0, j = 1,2, .. * (1.7) 4 Thus the o r i g i n a l problem of minimizing the functional J of (1*1) i s transformed into the problem of solving the nonlinear p a r t i a l d i f f e r e n t i a l equations, (1.6) and (l*7) for f. In general these nonlinear p a r t i a l d i f f e r e n t i a l equations can not be solved d i r e c t l y . 1•3•1 The P r i n c i p l e of Optimality as a Numerical Technique an a n a l y t i c a l solution. A numerical solution may be obtained by the use of d i g i t a l computers. In order to employ d i g i t a l com- puters for the numerical solution of (l.6) and (1.7), i t i s necessary to convert the nonlinear p a r t i a l d i f f e r e n t i a l equations into a f i n i t e - d i f f e r e n c e equation. A more convenient method of solution i s to solve for the functional f of (1.2) by minimizing a discrete approximation of the form Most problems i n optimal control are far too complex for N-1 ) At (1.8) i= k where x(iAt) and where the derivative x i s approximated by (i) _ ; ( i ) — X , ( x ( i + l ) _ X ( i ) ) / A t Let u and introduce the sequence of functions (i) )At (1.9) 'for - o o<c <oo , k = 0 , 1 , j N j y l .Then ' N and f M ( T , c ) = 0 ( 1 . 1 0 ) • • N - l f k(kAt,c) = Min F(kAt,c,u )At + ̂  F ( i A t , x ^ j U ^ ^ A t [u) i=k+ l = Min F(kAt {,c,u )At + f k + 1 ( (k+l)At,c+uAt) W ( 1 . 1 1 ) Equation ( l . l l ) i s the basis of the dynamic programming (5) method for the solution of minimum problems . 1 * 3 . 2 The Problem of Dimensionality The numerical solution of ( l . l l ) requires the tabulation and storage of sequences of functions of n variables. This introduces some complications. To i l l u s t r a t e t h i s , consider the case of a two-dimensional problem where c = ( c l * c 2 ^ u = (u 1,u 2) Assume that c^ and a r e both allowed to have one hundred values* Since the number of di f f e r e n t values for c^ and i s 1 0 ^ , the tabulation of the values of f(c^,C2»T) for a par t i c u l a r value of 4 T requires a memory capable of storing 1 0 numbers. Moreover, since the recurrence r e l a t i o n requires that f(c,T) i s stored while the values for T+At are calculated, and since the values of u^ and must also be stored, the memory must be capable of 4 storing at least 4 x 1 0 numbers. Generally speaking, with current d i g i t a l computers having memories of 32,000 words, only two-dimensional minimum problems can be handled unless some method for reducing dimensionality i s found. The problem becomes d i f f i c u l t to cope with for higher dimensions. As pointed out by Bellman, a three- dimensional trajectory problem involving three position variables and three v e l o c i t y variables, treated by the dynamic programming approach results i n functions of six state variables. In this case, even i f each variable i s allowed to take only 10 9 d i f f e r e n t values, this leads to 10 values requiring an extremely large computer memory. 1.3.3 The Euler-Lagrange Equations A l l the necessary conditions i n the c l a s s i c a l theory of calculus of variations can be derived from the p r i n c i p l e of optimality. Consider the v a r i a t i o n a l problem discussed i n Section 1.3. The p r i n c i p l e of optimality y i e l d s the nonlinear p a r t i a l d i f f e r e n t i a l equations (1.6) and (1.7). D i f f e r e n t i a t i n g (1.7) with respect to t , gives n f x ( F . ) + A ^ A T + / ^ l d \ i . = 0 (1.12) dt ± / Ox-Ot Z_j 6x,6x i i J J i=l and p a r t i a l d i f f e r e n t i a t i o n of (1.6) with respect to x . gives x . / , O x • O"tox. T / V O x . Ax. x v V Thus n 6 2f ^ .ST'tft Substituting (1.13) i n (1.12) yiel d s j = 1,2,. d_ / 5P_ 6 F N 5 x ~ = °> .. , n, (1.14) which are the Euler-Lagrange equations. It i s also possible to derive (1.14) from the nonlinear p a r t i a l d i f f e r e n t i a l equations for f using the method of c h a r a c t e r i s t i c s . 1.3.4 The Legendre-Clebsch Condition The necessary condition for a minimum of (1.5) i s that the second derivative of the square brackets with respect to x^ must be p o s i t i v e . This leads to the Legendre-Clebsch condition or n n .2 Ox.Ox. I j i=l 3=1 3 6 2 F dx-j&x^ 6 2 F 6 2 F 6 % 2 > o, 6 2 F 6^6x2 6 2 F > o, (1.15) 6 2] 6 2 F d x 1 0 x 1 •••• d x 1 d x n 6 2 F ^ F _ _ 1»3.5 The Weierstrass Condition The Legendre-Clebsch Condition does not rule out the pos- s i b i l i t y of a re l a t i v e minimum. If F(t,x,x) i s an absolute minimum, i t follows from (l»6) that the following inequality must s a t i s f y j=l J j=l ° or F(t,x,X) - F(t,x,x) + (Xj " xj) ^ 7 ~ 0 (1.16) 3=1 j for a l l functions X. From (1.7), and (1.16) y i e l d s the Weierstrass condition for an absolute minimum. n ^ . X F(t,x,X) - F(t,x,x) - ^ " *j> of? ~ 0 ^ • 1 7 ^ 1.3.6 The Transversality Condition So far the discussion of the minimization of a functional i s r e s t r i c t e d to the case of fixed end points. Suppose now that the end points are variable. The necessary condition for a minimum of the functional i s that the d i f f e r e n t i a l of the function f ( t , x ) must vanish. Therefore Thus oT •x-r dt = - / x dx L-J ox. i (1.18) Multiplying (1.6) by dt gives n Of Pdt + dt + Of * x.dt = 0 j=l 3 Substituting (1,7) and (l«18) i n the above equation y i e l d s n n F, dx. = 0 (F - x j F « ) d t + This holds at both end points* Thus n n -i T 3=1 3 4 dx. + (F - F. ) dt 3=1 x. 3 = 0 (1.19) Equation (1.19) i s c a l l e d the transversality condition. 1«3.7 The Veierstrass—Erdmann Corner Conditions Many v a r i a t i o n a l problems of engineering interest have solutions which may have a f i n i t e number of corner points, where one or more of the derivatives x. have a discontinuity. Suppose • . 5f that x^ i s discontinuous» then^ since ^ x i s continuous, i t O F follows from (1.7) that — j — must be continuous at a corner. 6 x k Of Si m i l a r l y , ^ i s continuous and substituting (1.7) i n (1.6) yields n Of 6T P - E F • = - 3=1 J which i s also continuous at a corner. Therefore = F (1*20) 1 0 and n n (F - V " x F* )_ = (F - x F. ) (1.21) j=l j j=l j where the negative and positive signs denote trajectory positions immediately before and after a corner point, respectively. Equations ( l . 2 l ) and (l*20) are called the Weierstrass-Erdmann corner conditions. 1•3•8 The Inequality Constraint In many problems there may be inequality constraints on the independent variable u of ( l . l l ) (the so-called control v a r i a b l e ) . I f , for example^ lul < U where U i s the upper bound for the magnitude of u, then the choice of u^ at each i t e r a t i o n stage i n the dynamic programming approach i s r e s t r i c t e d and the computational aspect of the problem i s thereby simplified* (6) 1,3.9 The Lagrange M u t l i p l i e r s ' The Lagrange m u l t i p l i e r method i s the most suitable means for handling a minimum problem subject to constraints. Two dif f e r e n t kinds of Lagrange mu l t i p l i e r s which depend on the type of constraints are discussed i n this section. Consider the problem of minimizing the functional J(x) =\ H(t>x»x)dt, x(0) = c (1*22) Jo subject to the constraint G(t,x,x)dt = y 11 (1*23) '0 where y i s a given value* To solve the minimum problem the lower l i m i t i s considered variable so that the minimum f of J(x) becomes a function of three variables, t,x, and y. In other words, y i s considered as an additional variable. The solution of the minimum problem i s given by H(T,x, f*: )dT f(t,x*y) = Min where y i s determined by the equation of constraint (1.24) G(T,x, f^)dT = y (1*25) Equation (l.24) can be treated i n the same manner as was done previously for (l.2) y i e l d i n g f(t,x,y) = Min H(t,x>x)At + f(t+At, x+xAt, y-G(t,x,x)At)+ 0(At) (1*26) Proceeding as before, the following functional equation for f(t,x,y) i s obtained t 0 = Min Of n f * \ L 6f dt H(t,x.x) + + ^ x j ^ - - G(t,x,x) ^ (1.27) The solution of (1.27) must s a t i s f y the equations n TJ . Of' P 6f U = h* + A ~ G v — Ox. • Oy x j 3 x j (1.28) 12 and j=i J D i f f e r e n t i a t i o n of (l«28) "with respect to t, and p a r t i a l d i f f e r e n t i a t i o n of (l»29) with respect to x. yiel d s P a r t i a l d i f f e r e n t i a t i o n of (l»29) with respect to y yiel d s the following r e s u l t s : 6 2f , V " * 6 2 f „ 6 2 f or 0 = It d.32) Thus = constant (l.33) I t can be seen from (l»30) that i f a new variable (1-34) i s introduced, (1.30) results i n the Euler-Lagrange equations - F x =0, j = 1, 2,..., n (1.35) 13 where P = H + KG (1.36) Of This shows that — ^ plays the role of the Lagrange m u l t i p l i e r . In the case of the constraint being an integral form of (l.23), the Lagrange m u l t i p l i e r i s a constant. In general, the Lagrange m u l t i p l i e r i s not a constant. Consider the constraint to be of the form h(t,x,x*u) = 0 (1.37) or x = g(t,x,u), x(0) = c (1.38) where the control variable u = u(^,,,,u m) i s to be chosen so as to minimize the functional J(x)„ In this case, the Lagrange m u l t i p l i e r i s no longer a constant. For example, consider the problem of minimizing the time required to transfer the system described by (l.38) from the i n i t i a l state (c^,...,c n) to the f i n a l state (b^ , • • • ,bn).* The functional T = T(u) to be mini- mized i s subject to the constraints x,(T) =b., j = 1,2,...,n. (1.39) •J J This i s a minimum-time problem. By introducing the function f ( t , x ) = time required to transfer the system described by (1.38 ) from x to b and applying the p r i n c i p l e of optimality the equation f( t , x ) = Min At+f(t+At,x+gAt)+0(At) (1.40) i s obtained. Expanding the second term i n a power series and l e t t i n g the l i m i t as At 0 yie l d s the r e l a t i o n 0 = Min n 1 + f t + 3=1 3 The solution of (l.41) must s a t i s f y the equations n 0 = i + f t + 3=1 14 (1*41) (1.42) and n 0 = fx. 5 u t ' 1 = 1.2,...*m. 3=1 2 1 (1.43) P a r t i a l d i f f e r e n t i a t i o n of (1.42) with respect to x. yie l d s 3 o 2f 6 2f V 1- 6f ^>gk n 5t5x7 + Z_ Sx^oir g k + 2_, 5x7 oTT = 0 k=l k=l vk ^ j (1.44) Since <L_ f dt x. 3 6 5T x. J n E k=l 4_ <f ) 6x, v x.; dt ^ k o = 6 T^t + 2 s x 7 ( fx, ) gk k=l k A j (1.45) i t follows by substituting (l»44) into (1.45) that n ^ ^ f t fx. +E5T fx k = 0^' = 1'2 n' k=l (1.46) Introducing the Lagrange mult i p l i e r s *3 = fx. (1.47) into (1.46) y i e l d s k=l J The solution of the 2n+m equations (1.38), (1.43) and (l.48) gives the 2n+ra unknown functions which are A., x. and u^. The trajectory defined by these variables s a t i s f i e s the necessary conditions for a minimum-time trajectory. 1.3.10 The Dynamic Programming Approach to the Case of Two Fixed End Points The numerical technique discussed i n Section 1.2.1 allows a problem with two fixed end points to be replaced by an i n i t i a l - v a l u e problem* Consider the problem of minimizing the functional J(x) = \ F(t,x,x)dt (1.49) ^0 subject to the two end conditions x(0) = a, x(T) = b (1.50) Proceeding as i n Section 1*3*1 where u = x y i e l d s the r e l a t i o n F(c,u)At + f ( c , t ) (1.51) f(c+ uAt yt+At) = Min M The condition that the f i n a l values of x(t) be the as- signed values b must be s a t i s f i e d . This means i n effect that at the l a s t stage of the process, for any values of x., the choice of the control variables u. must be such as to resu l t i n x.(T) = b j * Consequently, the terminal constraints f i x the function f(c,T) given by the r e l a t i o n where thus f (c, (N-1) Ait) '= F(c,u) b-c u = At f(c,(N-l)Ajfc) = * ( c , ̂ r r ) At' (1.52) (1.530 (1.5 4) Here, b i s taken to be f i x e d and c i s considered to be variable, This i s shpwn i n Pig. 1*1« 0 (N-1)At T = NAt t F i g . 1.1. The f i n a l stage and the terminal condition In dynamic programming the terminal constraint s i m p l i f i e s the computation. Since f(c»T) i s determined by the terminal conditions, the remaining functions of the sequence f(c+uAt, t+At) are determined by means of ( l . 5 l ) with no further reference to the terminal conditions. 1.4 The Gradient Method (7) The gradient method or the method of steepest descent i s an elementary concept suitable for the solution of minimum problems* In recent years the computational convenience of the gradient method has l e d to a variety of applications. In order to present the basic idea of the gradient method* consider the problem of minimizing a continuous function f = f ( x ^ * . . ,x n) If an arc length i s defined by n ds' - £ 3=1 dx. 3 (1.55) the derivative of f along the arc i s n dx, df = V " Of ds / , Ox. * ds 3=1 3 (1.56) Introducing the constraint dx. E LLS. . 3=1 - 0 (1.57) by means of a Lagrange m u l t i p l i e r X y i e l d s n n 6f d x i + x df GTS Z. i O x • as 3=1 J -i dx. 1 - > , <a^> 3=1 where n 3=1 dx. _ l y i ~ ds n " Of A 1 r , 5x- ̂  + * L 1 - 2_, 3=1 (1.58) df P a r t i a l d i f f e r e n t i a t i o n of with respect to y. yields 18 6 /dfx 6f 0 1 oy" fe) = 53ET " 2 X yj (1.59) df For -jj-^- to be a maximum, the above equation must vanish: £ * - - 2 X y . =0 Hence y-;= 6f j 2X * 5x~ (1.60) (1.61) Substituting y. into (1.57) yie l d s 3 n Hence * = ± 2 n 3=1 X-p 2 J I 2 Substituting X into ( l . 6 l ) y i e l d s dx. 6f y j ~ ds 1 ~  ± 5x. n i=l •1 2 (1.62) , j = l*2,...,n. (1.63) and the maximum derivative of f with respect to s i s 1 ds — Z _ (oT7> 3=1 J (1.64) For the steepest descent d i r e c t i o n , the negative sign i s taken, while the positive sign i s taken for the steepest ascent d i r e c t i o n . Now consider x. as components of a vector x, the d x i J dx directions as components of the unit vector and the 19 Of p a r t i a l derivatives pr— as components of a gradient vector, Ox. then f f = ^ r a d f • i f d- 65) Introducing the function ds v = dT where T i s a parameter into (l.55) yi e l d s 1 V = n dx. 2 (dT 1) (1.66) Since dx. dx 1 ds dT ~ ds * dT i t follows from (l.63) and (1.66) that dx, dT ~ i Ox. n i=l (1.67) If V = k i=l where k i s a positive constant, i t follows that dx. d T = ± k 5 Of (1.68) For the steepest descent, the negative sign i s taken. This r e l a t i o n i s the basic condition of the steepest descent d i r - ection for f . 20 1.4*1 Numerical Computation by the Steepest Descent Method The numerical computation of the minimum of the function f ( x ^ , » • • y X n ) requires that the equation of steepest descent be approximated as a finte-difference equation, that i s , (1*68) i s written as 6f Ax.. * - k A T j g - , j = l,2,*.*,n. The proportionality constant k can be absorbed by the step size AT, hence x. may be written as 3 6f_ x,< 1 + 1> S x ^ - hU><gL)<i> , j = 1 , 2 , . . . , n . (1.69) where h = kAT and h ^ ^ = k^^AT . The process i s repeated u n t i l (m) a minimum of f(x^,...,x n) i s obtained at. ̂ ( x ). Equation (1.69) i s a general formula for i t e r a t i o n * The step size h may be adjusted to reduce the number of steps required. 1.4*2 The Steepest Descent Method for Finding the Minimum of a Functional Consider the problem of minimizing the functional f T J(x) = I F(t,x,x)dt, x(0)-= c (1.70) J o where x belongs to a class of admissible functions. Let x(t) = y(t) + h u ( t ) , u(0) = u(T) = 0 ( l . 7 l ) 21 where h i s a parameter, y(t) i s a f i r s t approximation and where ;, u i s to be found so that J(x) <C J ( y ) . Equation (1.7.0) can be written as J(h) = \ F(t,y+hu, y+hu)dt (1.72) JO The derivative of J(h) with respect to h i s 5E = f E (V 3 + Fx^ ) d t (1'73) J 0 j=l J X j Integrating the second term of (1.73) by parts yi e l d s I = i Z < v » v* (i-74) JO i = l 3 x i For the path of steepest descent (1.74) must be negative which i s the case i f u. i s chosen so that • 0 u (t) - V (1.75) x. j At the minimum of J , u.(t) = 0. 1.5 The Calculus of Variations and the Theory of Optimal Control The general problem of the calculus of variations can be formulated as a problem of Bolza, Lagrange or Mayer. These three formulations are t h e o r e t i c a l l y equivalent and the problem of Lagrange and Mayer can be considered as particular cases of the (8) problem of Bolza . 22 The problem of Bolza can be formulated as follows: Consider the set of functions X j ( t ) , j =1,2,...,n. s a t i s f y i n g the set of constraints ^ ( t j X j x ) = 0, i = 1,2,..., m < n which involves (n-m) degrees of freedom. (1.76) Assuming that the functions x.(t) and t are consistent with the boundary conditions at t=0 and at t=T, that i s , 0,x(0) T,x(T) =0, r = l,2,...,q. =0, p = q+1, . . . ,s ̂ 2n+2 (1.77) (1.78) then the problem i s to f i n d the special set of functions x.(t) which results i n a minimum for the functional T J = G(t,x) +\ H(t,x,x)dt - 0 JO (1.79) If the function G of (l.79) i s i d e n t i c a l l y zero, that i s i f , G(t,x) = 0 then the functional of (1.79) reduces to -T H(t,x,x)dt (1.80) This i s the problem of Lagrange. On the other hand, i f the integrand of (1.79) i s i d e n t i c a l l y zero, that i s i f , H(t,x,x) = 0 23 then the functional of (1.79) becomes - i T J = G(t,x) ->0 This i s the problem of Mayer. It i s of primary interest, to interpret the general problem of Bolza from the point of view of optimal control. The essential difference between the calculus of variations and the theory of optimal control i s that the derivatives i n the integrand of the functional J i n the calculus of variations are replaced by the control variables u, ( t ) . Thus, instead of considering the minimization of the functional subject to the constraints (1.81) the minimization of the Functional 0 0 subject to the constraints of the form x. - f.(t,x,u), j = l , 2 , . . . j n . (1.82) i s considered. Where u i s the set ( u, . , .,,u ) « 1 m In general the optimal control problem can be stated as follows? Given an i n i t i a l state (0,x(0)), f i n d the corresponding admissible control variables u^ defined :j.n the interval [|O,TJ for which the functional J assumes i t s minimum. If the set of control variables u^ can be determined as functions of the state variables x. so that the functional J 3 is minimum, then the set of control variables u^ can be obtained by feedback from the state variables at the output. In this case the control variables are of the form u k = L k ( x ) , k=l,2,...,m. (1.83) and the functions L^(x) are referred to as the control laws. The problem can therefore be stated as an optimal feedback control problem; Find the control laws such that when (l.83) i s substituted i n (1.82), the functional J assumes i t s minimum with regard to the set of a l l admissible control laws. 1.6 The Ad.joint System and the Euler-Lagrange Equation The equations of constraints (1.82) are, i n general, f i r s t order nonlinear d i f f e r e n t i a l equations. If these non- li n e a r d i f f e r e n t i a l equations are l i n e a r i z e d , one obtains a system of li n e a r d i f f e r e n t i a l equations of the form Sx. = y ^ s r ^ Sx. + y ^ - i - Su. (1.84) j=l J k=l K where the p a r t i a l derivatives are evaluated on the^ optimal trajectory. The adjoint system of (1.84) i s defined by n ^ 6f * i = " E *j 5^S 1 = 1.2 n. (1.85) 3=1 1 Consider now the problem of Mayer of Section 1.5, where the Euler-Lagrange equations are given by | T (F. ) - F x = 0, j = 1,2,...,n. (1.86) x. i and where n F = XX [Xi - f i=l substituting this function F i n the Euler-Lagrange equations yiel d s X ± = - E \j 5 ^ f i=l,2,..*,n. (1.87) j=l 1 The equations of (1.87) are exactly the same as equations of (1.85), thus the Euler-Lagrange equations i n the calculus of variations are the same as the adjoint system for the li n e a r i z e d equations of constraints. It should also be noted that the equations of (1.48) are the Euler-Lagrange equations, where the Lagrange mult i p l i e r s have the special meaning i n dynamic programming given by (1.47). (9) 1.7 The Maximum Principle Pontryagin and his co-authors have stated i n the book "The Mathematical Theory of Optimal Processes" that the method of dynamic programming lacks a rigorous l o g i c a l basis i n those cases where i t i s successfully made use of as a heuristic t o o l . The maximum princip l e gives a rigorous mathematical theory for optimal processes. Therefore, i t i s of theoretical interest to discuss b r i e f l y the minimum problem as i t i s formulated by the maximum p r i n c i p l e . 26 Consider the functional J = \ F(t,x,x)dt (1.88) J o where x = (x^,...,x n) and the problem i s to f i n d the minimum of J for a l l the admissible control variables u^ which transfer the point from x (0) to x (T). J J Let x 0 = P(t,x fu) (1.89) x- = u., j = l,2,...,n. (1.90) and form the H-function n H(p,x,u) = p QF + P ju.. (1.91) j=l where the variables p are defined by the r e l a t i o n s s d P i 6 r dt ~ 5" x. , i = 0,1,...,n, (1.92) x Hence dp. x p ^ = - P Q, i = 0,1,...,n, (1.93) then the r e l a t i o n of (1.93) gives dp dt £ = 0 (1.94) d p i OF dt = ~ p0 5x~" ' J' = 1»2,..•»!!. (1.95) The maximum p r i n c i p l e states that i n order for u and x to 27 define an optimal trajectory i t i s necessary that there exists a continuous vector function p = (pQ,...,pn) corresponding to u and x, such that 1. for every t, 0 < t < T , the function H attains i t s maximum at the point u, M(p,x) = Sup H(p,x,u) (1.96) M 2. at the terminal time T, the relations p Q(T)<:0, M [p(T), x(T) = 0 (1.97) are s a t i s f i e d . The equation of (1.96) implies that 5H 5u- = 0, j = 1,2,...,n. (1.98) P a r t i a l d i f f e r e n t i a t i o n of (1.96) with respect to u. yie l d s OTT = p0 + p j ' 3' = l t 2 , . . . , n . (1.99) By the equation of (1.98), the above equation becomes P 0 5 u ~ + P j = ° ' J = 1f2,.••,n. (1.100) J It follows from (1.100) that PQ ^ 0, otherwise a l l the p^ = 0, i = O j l ^ . B ^ . n . It i s seen from (1.94) and (l.97) that PQ i s a negative constant. It i s convenient to choose p Q = -1 so that (1.100) becomes OF Pj ^ Q ^ - » 3 = 1,2,...,n. (1,101) J 28 On the other hand, i f PQ = -1 i s substituted i n (1.95) and then integrated, i t gives P ; j = Pj(O) .+\ g£- ds, j = l,2,... rn. (1.102) J o 3 « replacing u, by x. i n (1*101) and substituting into (1.102), yi e l d s g- = ̂ (0)+f i : d s (1-l03) D i f f e r e n t i a t i n g this equation with respect to t yie l d s the Euler-Lagrange equations f r (P. ) - P x = 0, j = l>2,...,n. (1.104) X j j 1.8 The F i r s t Integral The solution of the Euler-Lagrange equations s a t i s f i e s the r e l a t i o n , n " - E ; i f i . ) - ^ (1-105) 0=1 a If F does not depend on the independent variable t e x p l i c i t l y , OF 5 t = 0 and the following f i r s t integral i s obtained. n x. F. = C (1.106) . , 3 x. 0=1 J where C i s the constant of integration. This r e l a t i o n i s calle d 29 the f i r s t integral of the Euler-Lagrange equations. 1.9 The Modified Steepest Descent Method The essence of the modified steepest descent method for solving minimum problems i s to consider a general process which i s described by a system of ordinary d i f f e r e n t i a l equations of the form x = f(x,u), x i(0) = c±f i = 1,2,...,n. (1.107) where x = (x^,«»*,x n) u = (u 1,...,u m) and f = (f^,.».,f n) The system under consideration i s assumed to move from a point x(0) to another terminal point x(T). Some of the terminal conditions of x(T) may be unspecified. The problem i s to mini- mize the performance function P(T,x(T)) by choosing a special set of control variables u^» This i s a problem of Mayer. The basic idea of the modified steepest descent method i s to con- sider the function P as a function of a set of unknown para- meters which are functions ofthe unknown i n i t i a l conditions of the state variables and the Lagrange m u l t i p l i e r s . Thus P = P(a) (1.108) where a = (a l t...,a n) = X 1(0),...,X r(0) > x r + 1 ( 0 ) , . . . , x n ( 0 ) ] and where X^(0) are the unknown i n i t i a l conditions for the Lagrange m u l t i p l i e r s . The problem under consideration can be formulated as follows: The function n E = Z ^ d ( X J - f a ) ( i a 0 9 ) i s formed where X. are the Lagrange m u l t i p l i e r s . At a minimum, the Euler-Lagrange equations f t ' ( F . ) = F , j = l , . . . , n . (1.110) and 0 = F , k = 1,...,m. (l .111) V must be s a t i s f i e d . Substituting (1.109) into (1,110) and ( l . l l l ) , y i elds the following equations ax, ^ Of. d t 1 = " Z^h 5^ ' d = 1 n' ( i a i 2 ) i=l J n O f 0 = E ^ i 0 l T » k = l,...,m. (1.113) i=l k By solving the system of (2n+m) d i f f e r e n t i a l equations of (1.107), (1.112) and (1.113), the (2n+m) unknown variables X j , X j , and u^ can be determined. The general scheme for the solution i s represented i n F i g . 1.2. The i n i t i a l values are sampled and introduced into a high speed repetitive trajectory computer. The performance function P i s determined and the Iteration Procedure u(t) Process x(t) f M9l (o) Traj ectory Computer (Analogue) P£a) Fig . 1 . 2 A general optimal process unknown i n i t i a l values are adjusted by an i t e r a t i v e procedure to minimize P. The sampled value of u i s introduced into the process. If there are no disturbances the state x(t) of the process w i l l correspond i n real time to the computed trajectory. In the above system the i n i t i a l values for the trajectory are the real-time values of the process variables. In most problems not a l l the i n i t i a l conditions are given and therefore a search procedure for the minimum of the function P must be employed. The important idea of the modified steep- est descent method i s to solve the preceeding (2n+m) equations subject to the condition that the derivatives of the perfor- mance function P with respect to the parameters a^ are always negative, that i s , 6P 5T: < 0, j = l f t . . , n , (1.114) The values of a. are unknown and can be determined by i t e r a t i o n . For each i t e r a t i o n the condition of (1.114) must be s a t i s f i e d . The modified steepest descent method does not rule out the p o s s i b i l i t y of a l o c a l minimum unless the entire range of parameter values are used which may not be p r a c t i c a l (see F i g . 1.3 where a^ results i n a true minimum and a^ results i n a l o c a l minimum). As for the numerical computation, i t i s assumed that the computation starts from a point AQ = ( & J Q) which may be arbitrary. The parameter a-^Q i s adjusted so that P decreases to a minimum. The remaining parameters can then be adjusted i n sequence i n the same manner. Proceeding i n this way, a new 33 F i g . 1.3 True minimum and lo c a l minimum point A^ = ( a j i ) i - s obtained. The general step may be summarized i n the following way* From a point A = (a. ) to the next point A r + 1 = ( a - j ( r + i ) ) i s found by a step-by- step procedure. 1. Adjust a- r̂ by a small amount to have, a smaller P u n t i l P starts to increase. 2. Repeat 1 for a 2 r » • • • » a n r » e a c h time- adjusting one parameter only. 3. Now a new point Ar+-^ = ( a-j( r+i)) I s obtained and the steps 1 and 2 are repeated u n t i l a minimum of P i s obtained. It i s important to note that for the adjustment of each Lj(r) 34 P ( a l ( r + 1 ) , a 2 r,...,a n r)<P(a l r,...,a n r) P ( a l ( r + l ) ' a2(r+l)» a 3 r * * " a n r ) < P ( a l ( r + l ) ' a 2 r ' * * - a n r ) • • o • • • o o o o » 0 o • • » • • • • ft » # • • » o o • * A o « P e • e « e • • s « o • • • • P ( a l ( r + l ) , , , , , a n ( r + l ) ) < P ^ a l ( r + l ) , 0 , a ' a ( n - l ) ( r + l ) ' a n f ) apply. 1.10 Remarks It i s of interest to compare the modified steepest descent method studied i n this thesis with other computational techniques. The standard v a r i a t i o n a l technique of the calculus of variations transforms the o r i g i n a l v a r i a t i o n a l problem into a problem i n the solution of ordinary d i f f e r e n t i a l equations involving two- point boundary conditions. To solve a two-point boundary value problem i s usually d i f f i c u l t from the computational pdiftt of view* Dynamic programming, i n theory, eliminates the two-point boundary value problem* However, i t introduces a new d i f f i c u l t y , the problem of dimensionality, which means that an extremely large d i g i t a l computer memory i s required. The gradient method or the steepest descent method was developed by Cauchy and has been independently applied to va r i a t i o n a l problems dealing with f l i g h t paths by Kelley and Bryson. This technique has been very successful. However, i t requires extensive d i g i t a l computing f a c i l i t i e s and does not appear suitable for developing comparatively simple real-time 35 optimal c o n t r o l l e r s . The modified steepest descent method i s p a r t i c u l a r l y suitable for the solution of certain classes of minimum problems by means of d i g i t a l or analogue computers. The analogue com- puter i s very convenient for solving trajectory problems. Another advantage of employing the analogue computer i s that i t i s then possible to construct comparatively simple real-time optimal c o n t r o l l e r s . Since the analogue computer solves problems i n a continuous manner, i t i s suitable for high-speed com- putation and feedback methods can be used for obtaining i t e r a t i v e solutions• 36 2. OPTIMAL CONTROL PROCESSES FOR ROCKET FLIGHT PROBLEMS 2.1 Introduction Ana l y t i c a l studies may f a c i l i t a t e the computation of the solution for optimal control problems. The i t e r a t i v e approach used i n the modified steepest descent method may also be greatly sim p l i f i e d i f an an a l y t i c a l expression for the optimal control law i n terms of state variables can be found. The calculus of variations i s the only suitable method for obtaining analytic information about the properties of the optimal control law and the optimal trajectory and i s therefore, of fundamental importance. This chapter i s devoted to the application of the calculus of variations to the problem of rocket f l i g h t and to a n a l y t i c a l studies for deriving optimal control laws. It i s also of theoretical interest to have a complete ana- l y t i c a l solution of a problem. This allows a study of the properties of the Lagrange multipliers which play an important role i n the determination of optimal control laws. On the other hand, the an a l y t i c a l solution can serve as a means for checking the accuracy of the analogue computations used i n the modified steepest descent method discussed i n Chapter 3. 2.2 Formulation of Rocket F l i g h t Problems by Means of the Cal- culus of Variations The determination of optimal t r a j e c t o r i e s for missiles, a i r c r a f t s and s a t e l l i t e s i s an important application of optimi- zation theory. Goddard recognized the calculus of variations as an important tool i n the analysis of rocket performance i n 1919. A general theory of rocket f l i g h t problems was recently developed 37 by Breakwell, F r i e d , Lawden, M i e l e , Leitman and o t h e r s . A b r i e f review of the rocket f l i g h t problem w i l l now be g i v e n . 2.2.1 Basi c Assumptions and Equations of Motion For the general f o r m u l a t i o n of the rocket f l i g h t problem, the f o l l o w i n g assumptions are made (see F i g . 2 . l ) : (1) The roc k e t i s considered as a p a r t i c l e or a p o i n t mass . (2) The power p l a n t of the rocket engine i s considered as an i d e a l engine, so that the e q u i v a l e n t e x i t v e l o c i t y V f o r the f u e l i s a constant. The t h r u s t i s taken e as where P i s a c o n t r o l parameter. (3) The E a r t h i s assumed to be f l a t , and the a c c e l e r a t i o n due to g r a v i t y i s taken to be constant. (4) The roc k e t moves i n a v e r t i c a l two—dimensional plane. 0 F i g . 2.1 The f o r c e s a c t i n g on a rocket 38 By these hypotheses the equations of motion for a rocket can be w r i t t e n ^ ^ as x - V cos 0 = 0 (2.1) h - V sin 0 = 0 (2.2) . D-V 0 cos ft) V + g sin 0 + = 0 (2.3) L + V P sin ft) 6 + f cos Q - ^ = 0 (2.4) m + 0 = 0 (2.5) where x i s the range, h i s the al t i t u d e , V i s the v e l o c i t y , g i s the acceleration due to gravity, L i s the l i f t , D i s the drag, m i s the mass, 0 i s the path i n c l i n a t i o n , and (o i s the angle between the thrust and the v e l o c i t y . The drag i s assumed to have the general form D = D(h,V,L) (2,6) and the engine char a c t e r i s t i c s of the rocket are represented as a function of a parameter a, that i s , the control parameter i s 0 = 0 ( a ) (2.7) 2.2.2 Formulation of the Rocket F l i g h t Problem The set of f i v e equations of motion, (2.1) to (2,5), involves one independent variable, the time t, and eight dependent variables, they are: x, h, V, 0, m, <o, L and 0 . Thus, the problem under consideration has three degrees of freedom, and three conditions for optimal performance can be imposed. In this con- nection, the optimal control problem of Mayer type, can be stated as follows: Among a l l sets of functionsx(t), h ( t ) , V ( t ) , 0 ( t ) , m(t), 3 9 co(t),L("t) and p ( t ) , s a t i s f y i n g the equations of motion, ( 2 . 1 ) to ( 2 . 5 ) , and certain prescribed end conditions, to determine the r i*f special set which minimizes the performance function where P = P(x,h,V,©,m,t) The end conditions are constraints imposed on the i n i t i a l and the f i n a l values of x, h, V, 0, m and t. In general, not a l l the end conditions are known. In the case that two additional constraining equations of the form ^f 6 = §>(x,h,V,0,m,L,p\a,t)= 0 ( 2 . 8 ) ip 7 = Y(x,h,V,0,mfL,P,a>,t) = 0 ( 2 . 9 ) are present, the problem has only one remaining degree of freedom, and one condition for optimal performance can be imposed. By introducing a set of Lagrange multipliers X ^ ( t ) , i = 1 , 2 , . . . , 7 , the so-called augmented function can be formed 7 P = ^ X. ( 2 . 1 0 ) i=l and the Euler-Lagrange equations are where x^= x, x 3 = ^ ' x 4 ~ ® ' x 5 = m ' x 6 = ̂ » x 7 = a ' a n d x 8 = to' As discussed i n the l a s t chapter, i f the augmented function F of ( 2 . 1 0 ) does not depend on the time t e x p l i c i t l y , the f i r s t i ntegral 7 k " O F 0 1 d x . = C ( 2 , 1 2 ) i=l i 40 exists. The Euler-Lagrange equations and the f i r s t integral for the rocket f l i g h t problem are given i n the Appendix, Several p o s s i b i l i t i e s exist for modifying the trajectory of a rocket. The elevator position, the thrust magnitude, and the thrust d i r e c t i o n can be controlled. Thus, for a given set of end conditions, an i n f i n i t e number of tr a j e c t o r i e s exist which are mathematically and physically possible. Among a l l the possible t r a j e c t o r i e s i t i s of interest to fi n d those tr a j e c t o r i e s which meet a requirement for optimal performance. Partic u l a r forms of the performance function P are: (1) P = —m , problems of minimizing the fuel con-t o sumption, (2) P = (3) P = , problems of minimizing the f l i g h t time t o t, - X L J t o f , problems of maximizing the range. 2.3 Anal y t i c a l Study of Optimal Control for the Sounding Rocket P r o b l e m ( 1 1 ' 1 2 ) The equations of motion for the rocket f l i g h t , (2,l) to (2.5)f are nonlinear d i f f e r e n t i a l equations, and the associated Euler-Lagrange equations, (A.l) to (A.8), are linear d i f f e r e n t i a l equations whose c o e f f i c i e n t s are functions of the state variables. If the equations of motion can be solved so that the state variables are functions of time, the Euler-Lagrange equations may be considered as l i n e a r d i f f e r e n t i a l equations with time varying 41 c o e f f i c i e n t s • Since there i s no systematic a n a l y t i c a l method for solving nonlinear d i f f e r e n t i a l equations, the determination of an ana l y t i c a l solution for the rocket f l i g h t problem i s extremely d i f f i c u l t and, i n general, i s not possible. However, anal y t i c a l solutions may be obtained i n special simple cases. A problem of interest i s the case of rocket f l i g h t i n a r e s i s t i n g medium* This problem can be solved a n a l y t i c a l l y i n the case of v e r t i c a l f l i g h t with a drag function of the form D = kV 2 exp (-ah) (2.13) where k and a are constants. The sounding rocket problem has been studied by many s c i e n t i s t s , such as, Hamel (1927), Oberth (1929), Malina and Smith (1938), Tsien and Evans (1951), and Leitmann (1957), etc. Much work, both numerical and a n a l y t i c a l , has been done on this problem. However, with the exception of t r i v i a l casesj no complete ana l y t i c a l solution has yet been obtained. The p a r t i a l a n a l y t i c a l results published i n the l i t e r a t u r e w i l l therefore be extended as f a r as possible i n an attempt to obtain a complete anal y t i c a l solution. It i s assumed that the following end conditions are specified: h ( t Q ) = h Q = 0 , h(t^) = hp = f i n a l altitude (given) V ( t o } = V o = 0 ' V ( t f ) = V f = 0 m(t Q) = mQ = unknown, m(t^) = m̂  = payload (given) where m i s the i n i t i a l mass which includes the mass of the f u e l , ' o The problem i s to minimize the fuel consumption required to 42 reach a specified altitude by cont r o l l i n g the thrust. The per- formance function P i s (m - m^). Since m̂  i s fixed, the problem i s equivalent to minimizing the i n i t i a l mass mQ. The Euler-Lagrange equation (A.17) shows that two diff e r e n t classes of subarcsexist for the optimal trajectory: (1) ^ = 0, subarcswith constant thrust. V (2) X c- X 0 — = 0, subarcswith variable thrust* 5 J m For the sounding rocket problem i t can be shown that im- pulsive boosting i s always required. In this case the equation of motion (A.12) may be approximated for the boosting period by the equation where t i s the i n i t i a l time and t, i s the end of the boosting i n -o 1 b t e r v a l . Solving (2.14) together with (A.13) yiel d s m * mQ exp (- |-) , t < t < t 1 . (2.15) e where m i s the i n i t i a l mass and m, i s the mass at the end of the o 1 boosting i n t e r v a l . The boosting i n t e r v a l i s often very short and the impulsive thrust i s extemely large. The to t a l time for the boosting period may then be taken as t-^- t = At, and the v e l o c i t y V i s suddenly increased from zero to V. while the mass decreases from m to m,» 1 o 1 The entire optimal trajectory has only three subarcs: The boosting subarc, the variable thrust subarc, and the coasting subarc (zero thrust). Integrating ( A . l l ) from t to t, yi e l d s f t r t h l = 1 V dt = 0 V dt ^© *o 4 3 since At i s very small and V i s f i n i t e , the above integral i s negligible and = Ah =• 0 ( 2 . 1 6 ) Let the mass flow of the impulsive boosting be p ^ . In- tegration of ( A . 1 3 ) gives m, — m = 1 o p dt rm •m ( 2 . 1 7 ) Since Pm i s extremely large, the product P mAt i s a f i n i t e quantity. Solving the Euler-Lagrange equations ( A . 1 4 ) to ( A . 1 6 ) y i e l d s and X 3 6 D 21 ~~ 2 0 \ m Oh A - „ , = X ™ + 2 0 •', ( 2 . 1 8 ) X 3 1 - * 3 0 + " ^ X 3 6 D " " A 2 + ~m" ST . dt = X 3 0 o -t, X 5 1 = X 5 0 +\ ~2 <VnT D > d t m (2.19) . I D , x 5 0 + ̂ 'm - V <-DM> - m X 3 D dt m = X_n + X . , N V (^- - — ) 5 0 3 0 e m, m 1 o ( 2 . 2 0 ) where the second subscript denotes the value of X ^ at the time t = t ^ , that i s , ̂ ("k^) = ^ i k * •'•be above approximations are 44 obtained by neglecting a l l integrals with respect to t since the time interval t^ — t Q i s n e g l i g i b l e . The drag function and i t s derivatives, and are f i n i t e during this i n t e r v a l . This can be seen from the drag function (2.13). Information about the end conditions of the Lagrange multi p l i e r s may be obtained from the transversality condition and the f i r s t i n t e g r a l . The transversality condition i s dm + X 0dh + X^dV + Xcdm + C dt 2 3 5 t f = 0 (2.21) t where C i s the f i r s t i n t e g r a l . Since m , t , and t„ are free, the transversality con-o o f J d i t i o n y i e l d s X 5 0 = -1 (2.22) and C = 0 (2.23) The transversality condition does not give any information about the f i n a l values of the Lagrange multipliers for this problem. However, the f i r s t integral (A.18) gives X 3 f = 0 (2.24) For the variable thrust subarc, ^ ^ 0, and i t follows from (A.17) that the condition V X5 " A 3lT = °' ti< t < t 2 ' ( 2 ' 2 5 ) must be s a t i s f i e d , where t ^ i s the time at which the thrust i s cut o f f . 45 The f i r s t integral (A.18) now reduces to X2Y - \3 (g + |) = 0, t 1 < t < t 2 . (2.26) It i s obvious that (2.26) also holds for the coasting subarc where 0 = 0 . D i f f e r e n t i a t i n g (2.25) with respect to t yie l d s m \5 + X5m - X 3 V e = 0 (2.27) Substituting (2.5), (A.15) and (A.16) into (2.27) gives X2 " m 1 ( f ~ + ^ = °' t i < t < t 2 • < 2* 2 8) e Substituting (2.13) into (2.28) yie l d s \r = m (l + rH' t x < : t < t 2 . (2.29) 3 e Eliminating ^ a n d ^ 3 between (2.26) and (2.29) gives mg - D(l 0, t 1 < t < t 2 . (2.30) e Equation (2.30) shows that the v e l o c i t y V can not be zero during the variable thrust period. Therefore impulsive boosting i s required. Moreover, equation (2.30) can be used to determine j the switching time t^ for the actual f l i g h t , and i t w i l l be used as a control law i n the next chapter for the analogue computation of the sounding rocket problem. Di f f e r e n t i a t i n g (2.30) with respect to t yie l d s Substituting (2.30) and (2.3l) into (A.12) gives 46 x V a V V a e e + t l - aY £_) JL 2} V 2K. e aV 2 + 4 T ~ V ^ e + 2 (2.32) Let v = rp— and b = —^> , then (2.32) can be written as e aV ^ • _ j * v _ v^ + (l-b)v-2b bV 2 D V e v + 4v + 2 (2.33) or b V e v 2 + 4v + 2 , dt = 5 dv g v[v + (l-b)v-2b[ (2.34) Integrating this equation from t-̂  to t gives where Since t = t n + — 1 g K l n ! i + li±bi l n v 2 + (l-b)v^2b v 1^+(l-b)v 1-2b + In r2v1+ (i-b) + K 2 V + ( 1 _ B ) -K 2 I 2v + (1-b) + K 2v 1+ (1-b) -K (2.35) K = J (1-b) 2 + 8b i t follows that h = V = vV dh = V vdt e Substituting (2.34) into t h i s equation yi e l d s d h = 1 ? v + 4v + 2 d y a v + (l-b)v-2b 47 Integrating t h i s equation from h, to h gives + 31b l n v 2+(l-b)v-2b 1 v^ + (l-b)v 1-2b K . (•2v1 + (l-b)4K 2 v + ( l_ b) K 1 + 2 l n^2v + (l-b)+K 2v 1 + (l-b) - K J (2.36) The mass m can be determined as a function of v and t by- rewriting (A .12) i n the form a = _ i (v + G + D } m V e nr e and then substituting (2.30) for ̂  into the l a s t equation. Thus m ~ ~ V V~ V U+v) e e or da = . ( d T + f _ d t ) Now substituting (2»34) for dt i n the above equation gives dm /, g -, . \ b v + 4v + 2 in V 7 v (1 +v) 2,/ n,\ e v ' v +(l-b)v-2b which can be integrated to the form l n m m m. (v + f- t) e + In 2^ v + v v z+ (l-b)v-2b or 48 m=ra .2 + y v 1 2 + (l-b)v 1-2b 1 v ^ + v1 v 2+(l-b)v-2b exp (2.37) To sblve the Euler-Lagrange equation (A.16), the following equations T p - D . m = V + x 3 x 5 m which are obtained from (A.12) and (2c25) are required. Sub- s t i t u t i n g these two equations into (A.16) gives X 5 = X 5(v +^-) e where v = V/V"e« Integrating this equation from t^ to t yie l d s X 5 = X exp (v t) (2.38) e Substituting this into (2.25) gives mX x 3 = — 51 exp (v t) (2.39) The Lagrange m u l t i p l i e r X 2 can be determined by the f i r s t i ntegral (2.26): mX c -i -n «2 - — ^ (g + exp (v + ^ t) (2.40) v V e X„ = For the coasting subarc, the thrust i s cut o f f , so that (3=0. Thus m = 0 and the mass m i s constant. Let m = at t = i<2> then = m̂ , and the equations of motion and the Euler- Lagrange equations become h - V = 0 (2.41) Y + g + ~— = 0 (2.42) mf m and where (2.43) *2 " =J 5E < 2 - ^ ) ^ = - X 2 + m 7 0 T <2'45> *5 = " X3 ~^2 ( 2 e 4 6 ) m̂ 2 where D = k V exp(-ah) Since v _ dV _ dV dh 5 i n c e V - dt ~ dh dt V = w 2 *Z dh e dh substituting this equation into (2.42) gives T 3 E + f 2 + ^ « P ( - ^ ) = ° e ° r ^ ( v 2 ) + ^ v 2 exp(-ah)+2 ab = 0 (2.47) Equation (2.47) i s a li n e a r d i f f e r e n t i a l equation with respect 2 / 2k — ah\ to v . It has an integrating factor of the form exp(- — e ), and can be written as 50 d_ dh 2 / 2k -ahx _vexp(- — e ) 2ab exp (- — e a^) (2.48) In order to integrate the right hand side of this equation, l e t y = e ~ a h , dy = -aydh and / 2k -ah\, u \ / 2k % dy exp (- — e )dh = - exp (- — y) -± J " ' ay The integration can be performed by expanding the exponential function i n a Taylor series. Thus r 0 0 n exp (c y ) ^ = l n (c y) + V J y £-i n.n 1 n=l and integrating (2.48) yie l d s 2 / - . I . _ /2k —ah\ v = 2 b exp ( e ) r am ̂  -ah + /_ 2k \ n g-anh «*» am^' n . n ; + C n=l - f ( h ) (2.49) where i s the constant of integration and i s given by ( 2k ^n g-anhf = ahj — am. n=l n . n ! Thus i s a known constant since h^ i s given. From (2.41) dh dh dt = V v e and (2.49) gives thus v =Vf (h) dt = ±- dh V e V H h j Integrating t h i s equation gives t = t 2 + ±- dh e J h . V f (y) (2.50) Substituting the f i r s t integral (2.26) into (2.45) for \ 0 y i e l d s 51 Since « dX~ dX 0 j , dX 0 , _ 3 _ 3 dh _ v 3 A3 ~ dt - dh ° dt _ v dh thus ax. dh _ V X3 / „ . kV 2 -ahs or ax. L_ e " a h d h _ s _ ah mf V 2 v 2 e But (2.49) gives thus vZ = f(h) ax. k_ e~ah d h _ _g_ dh m̂  e d h ~ 2 fThT Integrating this equation yi e l d s X.j = exp r-^ L_ am. -ah _g_ V 2 e ] + c. = F(h) (2.51) where C 9 i s the constant of integration and i s given by h e h 2 (2.52) The Lagrange mu l t i p l i e r X 2 can be obtained from the f i r s t integral k v ^ 2 , *o = 1 # L (g e" a h) (2.53) Substituting (2.5l) into (2.46) gives 52 X 5 = . P ( h ) S ! e-ah • d A _ d X , d X c 5 d t d h d t d h and V = V v = V \J f(h) thus dX c -kV , , ^ 1 = — f v / i l M P ( h ) e " a h dh m f 2 Integrating t h i s equation gives _ k V r h X 5 = ^ p \ h 2 P(y) V f ( y ) e" ay dy + C 3 (2.54) where i s the constant of integration. For the further d i s - cussion i t w i l l be convenient to give a summary for the solution of the sounding rocket problem. (l) For the boosting subarc ( 0 - ^ t < t ^ ) , where t Q = 0* h 1 = 0 (2.16) V suddenly increases from zero to V l m = m exp(- % — ) , where m i s o * V ' o e unknown. (2.15) t x = A t = 0 X 2 = X 2 Q (2.18) ^ 3 = A 3 Q (2.19) X5 S X50 + X30 V e £ ' m"> ( 2' 2 0> o where X 5 Q = -1 (2.22) The f i r s t integral i s A 2 V - x 3 ( g + 2) - U - * It) = o 53 (2) For the variable thrust subarc, ( t ^ ^ t - ^ t 2 ) , y - v + L±± l n v '+ (1 - b)v -2 b v 2 1 v 2 '̂  + (1 - b ) v x- 2b , K . f 2 v i + - b ) + K 2v + ( l - b) - K 11 + 2 l n l 2 v + (1 - b) + K * 2v x + (1 - D) - K J J m = m v 2 + v V ; L 2 .+ (1 - b)v1 2b 1 V l 2 + v l v 2 + ( l - b)v - 2b e exp (2.36) (2.37) t = !e [ l n I i + I l ^ _ b i l n v 2 + (1 - b)v - 2b g L v 2 v , 2 + (1 - b ) v x - 2b K f 2 v 1 + (1 - b) +K 2 v +, ( l _ b ? _ + 2 i n [ 2 v + (1 - b) + K y 1 + (l-b) - K (2.35) where b = — ^ t K =./(l - b ) 2 + 8b and aV. '3 m m X51 v V ; (g + £) exp (v + t) m X 51 3 - y m exp (v t) A 5 = A 5 1 exp (v + ̂ |- t) e The f i r s t integral i s X 0V - X~(g + -) = 0 m mg - D(l + f-) = 0 e (2.25) (2.40) (2.39) (2.38) (2.26) (2.30) (3) For the coasting subarc, ( t 0 ^ t ^ t „ ) , 54 2 - /2k - a h x v = 2b exp ( e ) /_ 2k \ n —anh oo am „ -ah + + C\ n=l n . n J where = f(h) = ah^ n / 2k \ -anh„ ~ ( - i s r : ) e f n=l h n , n dh (2.49) (2.50) m = m̂  = constant x _ lihl (s + k v 2 V e 2 e-ah }  A2 - v V V g + ~ i i n e I e i X- = exp r- — L am„ -ah __g_ e Jh, dy fTTT. (2.53) + C, A F(h) (2.51) -kV. k5 = _ 2 -  \ i — I F(y) V f(y) e ~ a y dy + C- (2.54) where C 2 = - exp The f i r s t i n t e gral i s _ JL_ p - a h f _ _£ am. dy 1 e J h 2 (2.52) X 2 V - *3 <« + m~) = 0 (2,26) It i s evident that the form of the analytical solution i s 55 very complicated. On the coasting subarc, the anal y t i c a l solution cannot be expressed i n a closed form. However, by the use of d i g i t a l computers an accurate numerical solution may be (12) obtained. For example, Leitmann has obtained the optimal thrust program as a function of time, using a d i g i t a l computer and the an a l y t i c a l results to obtain the optimal trajectory. In Leitmann's method the trajectory was solved i n reverse time, starting at the f i n a l point. Although the an a l y t i c a l solution has a complicated form i t s t i l l y i e l d s interesting information about the optimal trajectory of the sounding rocket problem. This w i l l be d i s - cussed i n the following section, (l) The Optimal Controller The entire optimal trajectory has three subarcs (the impulsive boosting subarc, the variable thrust subarc and the coasting subarc) and associated with these subarcs are three di f f e r e n t types of thrust programs. These are impulsive thrust, variable thrust and zero thrust. This means that the optimal controller has three modes of operation. The f i r s t and the l a s t modes are ones of maximum and zero thrust respectively. The variable thrust mode i s controlled by the optimal controller which must also determine the instants at which modes are switched. A possible optimal controller can be obtained by means of (2.30). The method whereby (2.30) i s used to obtain the optimal control law i s to consider (2.30) e • = mg - D(l + ̂ -) (2.55) e 56 as an error signal. The signal e i s fed into a high gain ampli- f i e r and the amplifier output is used to control the fuel flow, A detailed discussion and some other possible optimal control laws w i l l be studied i n the next chapter, (2) The I n i t i a l Values of the Lagrange M u l t i p l i e r s The Lagrange mult i p l i e r s play an important role i n the present study of optimal c o n t r o l l e r s . In the general case^ the control law depends on the Lagrange m u l t i p l i e r s . Usually the i n i t i a l conditions of Lagrange mult i p l i e r s are not a l l known and the controller must then compute the unknown i n i t i a l conditions. The sounding rocket problem has two unknown i n i t i a l Lagrange multipliers, X O A and X zv 30* It follows from the an a l y t i c a l study that both X^Q a n < i X-JQ are negative. This statement can be proved by the following argument: At the end of boosting, that i s at the time t-^, the ana l y t i c a l solution gives X 2 1 = X 2 Q (2.18) X 3 1 = X 3 Q (2.19) X 5 1 - - l + X 3 0 V e ( i - - ^ ) (2.20) 1 o and X 5 1 - X 3 ; ^ = 0 (2.25) The l a s t equation can be approximated: V A51 = X30 m^ 57 Substituting the above equation into ( 2 . 2 0 ) and solving for A o y i e l d s m S O--V 2 , ( 2 - 5 6 ) e Equation ( 2 . 5 6 ) shows that X-JQ must be negative, since mQ and V G are positive quantities. It follows from ( 2 . 1 9 ) that X^ must be negative. Furthermore, the f i r s t integral ( 2 . 2 6 ) shows that * 2 1 T l - * 3 1 < « + « f > - ° where , g, and m̂  are positive, and X ^ i s negative. Thus X^-must be negative and from ( 2 . 1 8 ) X^q must be negative. In conclusion, a l l the Lagrange multipliers i n the sounding rocket problem must have negative i n i t i a l values. ( 3 ) A Qualitative Study of the Motion of the Sounding Rocket Problem A qualitative study often gives a better understanding of a problem. The general behaviour of the state variables and the Lagrange mu l t i p l i e r s may be obtained from the an a l y t i c a l solution. The altitude h i s always increasing along the entire tr a j ectory. For the boosting subarc, the an a l y t i c a l solution shows that V i s increasing and that both m and X^ are decreasing, but X 2 and X^ are almost constant. For the variable thrust subarc, the optimum thrust gives an optimum v e l o c i t y . Equation ( 2 . 3 6 ) shows that V must increase since h i s increasing a l l the time. The mass m i s determined by the equation (see ( 2 . 3 0 ) ) . 58 - = | d + fo s e = k e ff exp(ah) Since m i s decreasing, i t follows from the above equation that the denominator, ge a*\ increases faster than the numerator k V 2 ( l + The Lagrange mult i p l i e r s X„ and X,, increase because they e have positive time derivatives and X,. decreases because i t has a negative derivative with respect to t (see (A.14), (A»15) and (A.16)). For the coasting subarc, the drag i s small at high a l t i t u d e , and the thrust i s zero, thus the v e l o c i t y i s approxi- mately equal to V 2 ~ g("t-t 2) (see (A.12)). The altitude h increases u n t i l V becomes zero. The Lagrange multipliers X 2 and X^ remain almost constant for the coasting subarc, since their time derivatives are negligible (see (A.14) and (A.16)) and X-j increases to i t s f i n a l value X^^ with a slope approxi- mately equal to -X 2 (see (A« 15))« The a n a l y t i c a l solution for the coasting subarc contains an integral* The integrand i s l / f ( h ) and i s i n f i n i t e at h = h^ since f(h^) = v f = 0 . The integrals h f . h f /ty and \ W'̂ N are, however, f i n i t e . The singular 2 - 2 nature of the integrand makes a dir e c t d i g i t a l computation using the a n a l y t i c a l results d i f f i c u l t . If the approximation V = V_- g(t - O for the coasting subarc i s made, the function • .—12 can be used to compute the f (h) = v = V e 2 - -~ 2 r v g(t - t 2 r 2 above two integrals, i 59 The following curves i n F i g . 2.2 and F i g . 2.3 i l l u s t r a t e the general behavipur of the state variables and Lagrange m u l t i p l i e r s . F i g . 2.3 The Lagrange multipliers 3. OPTIMAL FEEDBACK CONTROL SYSTEMS 61 3.1 Introduction The general problem i n optimal control i s the determination of the inputs to a system subject to certain con- straints so that the state of the system follows a trajectory r e s u l t i n g i n the optimization of a given performance c r i t e r i o n . In other words, the problem i s to determine the control variable as a function of time so that the system s a t i s f i e s the specified c r i t e r i o n . This i s ess e n t i a l l y an open loop control system and, from the control engineering point of view, may not be sati s f a c t o r y . The control variable resulting i n optimum performance can be determined a n a l y t i c a l l y only for very simple systems, for example, the constant c o e f f i c i e n t l i n e a r system. Furthermore, the open loop control has the disadvantage that disturbances existing i n a physical system results i n non- optimum performance. Therefore, a closed loop feedback control system i s desirable. This chapter i s devoted to the study of feedback optimal control systems. Specific problems are studied and the optimal control for each case i s derived as a function of the system state v a r i a b l e s . 3.2 The Concept of Optimal Feedback Control and the Synthesis of Optimal Controllers : Optimal controllers synthesized by use of the calculus of variations r e s u l t i n a multivariable type of control systems. In general, a multivariable optimal control system consists of 6 2 two subsystems. These are the plant and the so-called adjoint system. The plant i s usually described by a set of d i f f e r e n t i a l equations and the adjoint system corresponds to the Euler- Lagrange equations. The interre l a t i o n s h i p between these two subsystems i s shown i n F i g . 3.1. an n by m optimal feedback control system, where n refers to the number of the state variables x ( t ) , and m refers to the number of the control variables u ( t ) . The following matrix notations are used i n F i g . 3.1. The system i l l u s t r a t e d i n F i g . 3.1 may be considered as x 2 ( t ) * x(t) n x 1 matrix of state variables. x (t) n _ x x ( t ) X(t) n x 1 matrix of the Lagrange m u l t i p l i e r s . u x ( t ) u(t) m x 1 matrix of control variables. P(a m x 1 matrix of the terminal values of x(t) and t . P m The performance function P i s to be optimized. The number of elements of the u(t) matrix i s always the same as that x ( t „ ) Plant j d t l Performance Cr i t e r i o n ) , Adjoint Optimal u(t) System Controller F i g . 3.1 A general multivariable optimal feedback control system 64 of the P matrix. 3.2.1 A Multivariable Optimal Feedback Control System In some cases the optimal control law may not contain the Lagrange m u l t i p l i e r X(t) e x p l i c i t l y . The control variable u(t) may then be determined as a function of the state variable x ( t ) . In th i s case the general multivariable feedback control system described i n F i g . 3.1 reduces to the form shown i n F i g . 3.2. The following sections discuss optimal controllers of this type for a variety of f l i g h t conditions. 3.2.2 Synthesis,of Optimal Control Laws for Rocket F l i g h t In the study of optimal control systems the synthesis of the optimal controller i s a major problem. In the case of optimal feedback control systems the determination of the optimal control law i s of primary importance. The simplified problems of rocket f l i g h t have been formulated i n the Appendix, and they w i l l be studied i n this section. These simplified prdblems have one degree of freedom. Thus there exists only one optimal control variable i n these problems• (l) The V e r t i c a l F l i g h t (Sounding Rocket) Problem - It follows from Chapter 2 that optimal condition for the variable thrust subarc i s X5 " X3 S2- = 0 (2-25> Actually, this condition holds true for a l l the four problems Optimal Controller u(t) (t) Performance C r i t e r i o n 3.2 A multivariable optimal feedback control system discussed i n this chapter. D i f f e r e n t i a t i n g (2.25) with respect to ti yi e l d s m X 5 + m A - V e X 3 = 0 (3.1) It follows from Chapter 2, Section 2.3 that (3.1) leads to equation (2.30)j that i s A V f = mg - D(l = 0 (3.2) e where f i s calle d the switching function. The boosting s stage terminates when f goes through zero. D i f f e r e n t i a t i n g s (3.2) with respect to t gives mg - D-|- - (1 (|2 Y - aDh) = 0 (3.3) e e The equations of motion, ( A . l l ) , (A.12) and (A.13) can be used to eliminate m, V and h i n the above equation resulting i n u = (3 D (g + |) (2 + 21) + a V 2 (1 +T-;) _ e g V + | (2V e + 3V) (3.4) which gives the optimal control variable as a function of the state variables for the variable thrust subarc. (2) The Horizontal F l i g h t Problem, The equations for optimal horizontal f l i g h t are derived i n a manner similar to the problem of v e r t i c a l f l i g h t . After 67 substituting (Ao23), (A.25) and (A.26) into (3.1) the following equation results X„ =W + X, V 4 mV 1 e 3 m m o» (3.5) The f i r s t i ntegral for the variable thrust subarc i s X,V - X, - = 0 1 3 m (A.29) Solving this equation for X^, and (A.27) for X^ and then sub- s t i t u t i n g into (3.5), yields the condition 6: which must be s a t i s f i e d by the optimal variable thrust subarc Here L = mg and D = D ( V , L ) Expressing (3.6) i n the form 6D 6B D ( V - V e) +V V e - m g V § £ = 0 and then d i f f e r e n t i a t i n g with respect to t yiel d s V D + (V - V e ) ( ^ V + ^ L ) + V V 6D 6v - mgV (5LOV V + ~ L Substituting L = mg into the above equation gives n ^ ,r 6D 6D „ v d 2 D ,r d 2p " L 5L " M G 5L + V V e ^ 2 " m g v 5L5V_ - mg e 6L + V V e 5VOL " m g V ^2_j = 0 68 Let A(m,V,L) = D + V ̂  - .g ̂  + V Vfi ^ - ,g T ^ n( v A -ir 0D , „. ,, 6 2D , R Q2D B(m,V,L) = -Ve ^ + V V e ^ - mg V — and substituting (A.22) and (A.23) into the previous equation yiel d s the optimal control variable u = 0 A D (3.7) AV"e - mgB (3) The Arbitrary Inclined Rectilinear F l i g h t Problem, This i s a more general case and includes th€ v e r t i c a l and horizontal f l i g h t problems. The derivation of the optimal control variable i s the same. Substituting (A.34), (A.37) and (A.38) into (3.1) and using the optimal condition (2.25) for the variable thrust subarc, the following equation %s obtained. *4-mT + h Y e C 0 S ° + X 2 V e s i n Q " X3 (m" + lr 5T> = 0 (3.8) The f i r s t integral for this problem along the variable thrust subarc i s given by (A.4l) A-, JJ X± cos 0 + X2 sin © - ̂  (̂  + g sin Q) = 0 The Euler-Lagrange equation (A.39) gives X4 = X3 V 5 T It follows from the above two equations and (3.8) that the optimal variable thrust subarc must s a t i s f y the condition 69 f g = D(V - V e) + Y V e | - m g (V e sin 0 + V cos 0 ^ ) = 0 (3.9) where L = mg cos 0, D = D(h,V,L) and 0 i s a constant. It can be seen that (3.2) and (3.6) are special cases of (3.9). D i f f e r e n t i a t i n g (3.9) with respect to t yields + V cos 0 £°j) mg V cos 0 ̂  - mg V cos 0 ( g ^ V + h + g | £) = 0 By means of (A.32), (A.33), (A.34) and the equation L = m g cos 0 The previous expression can be solved for 0 yi e l d i n g the optimal control variable u = P _ mC - A(mg sin 0 + D ) / _ N \ mB - V A U.iO; e , » -A « _,_ w 6 2 D n dn w „ u 2 p where A = D + V V E — - mg cos 0 - mg V cos 0 Q ^ J J B = g cos 0 . ( Y " V 6T + W e 5VOL " m g C 0 S ° ^ 2 - V e t a n 9 " V 6T 70 2 2 C = ( T - V e)V sin © + sin 0 ^ - mg V 2 sin © cos © (4) The Z e r o - l i f t F l i g h t Problem. Substituting (A.48) and (A.53) into (3.1) yie l d s the equation *3 = " X3 m V ( 3 a i ) e The optimal condition for the variable thrust subarc i s given by.(A.54) V Ac - X- ~ = 0 5 3 m Substituting this into (A.53) gives h = ̂  ( V " D> (3'12> e It follows from (A.5l) that A^ = -Aj C O S © - A 2 sin 6 + ~ y7" ~ ^4 2 C 0 S 0 Substituting (3,11) into the above equation yields - A = S - + A, V cos © + A 0 V sin © - A;, — + A. # cos © 3 mv 1 2 3 m 4 V e = 0 (3.13) The f i r s t integral for the variable thrust subarc i s Aj V cos © + A 2 V sin © - A 3 ( ^ + g sin ©) - A 4 ^ cos © = 0 (3.14) The Euler-Lagrange equation (A.50), 71 with the aid of (3.1l), c a n D e written as \ 2 = a V g X 3 (3.16) Integrating (3.16) gives X 2 = a V e X 3 + C 2 (3.17) where C~ i s the i n i t i a l condition of X«~- a V X--.. d. <ZV e _}U Subtracting (3.13) from (3.14) and solving for X^ yields X V X„ = -x—2 TT (~- + - - g sin ©) (3.18) 4 2 g cos 0 mV m s y v ' B e Substituting (3.17) and (3.18) into (3.13) and solving for X 3 results i n 2V(C 1 cos 0 + C 2 sin ©) DV 3D ~ - + — + g sin © - 2aW sin 0 mv m 5 e X3 ~ (3.19) e where X^ = i s a constant, a result which follows from the Euler-Lagrange equation (A.49). Now l e t t i n g s = B(v!g ; m !o) ( 3 - 2 ° ) where A(©,V) = 2 V(C;L cos © + C 2 sin ©) (3.2l) B(V,h,m,©) = + ^ + g sin S - 2aW sin © e e (3.22) and d i f f e r e n t i a t i n g (3.20) gives X . ^ 5 ^ (3.23) J B^ It follows from (3.1l) and (3.20) that 72 Eliminating by the aid of (3.23) gives A B D where mV = B A - A B (3.25) « A B O A v . O A 6A mQ O  / • Q Dx 6A V e P 6A 6B 6v cos O v ^ O B ' 6B ' ^ 6B * v + 6 h h + 6m" m + 6© 9 OB / • n = s i n 6 _6B £ 6m K D^ ^ 6B V e P A 6B v . Q - —J + vr? + s-j- V sin 0 nr 0^ m O n U g A 5© f cos © Substituting A and B into (3.25) and solving for (3 results i n the optimal control variable for the variable thrust subarc where and u = (3 1 ~ F m V e g sin © 6B . 6v + A E c o s 0 6B V O© - B g Sill © jyf B D 6A m OY _ i£ V A D 6B A 6A cos © ^ (3.26) T? A A Ve 6B . 6B B Ve O A * - - 5v A m 6m m 6v 6A 6V 6A 5© 6B 6v 6B 5h 6B 6© 2 Cj cos © + 2 C 2 sin © r-2 Cj V sin © + 2 C 2 V cos © + ^ - 2a V sin © mVe mV e e g cos © - 2 a V V g cos © 73 dm" = - — ( 3 + V"' w m e By the aid of equations (3.17), (3.18), (3.20) and (3.14), the switching function f can be obtained f = C, V cos © + 0 o V sin 0 + a V V sin © | - ̂  g sin © s i £ e D dx> _ ADV _ 3AD _ ( x 2mBY 2mB - U \i^U e The optimal control law for the four di f f e r e n t problems of rocket f l i g h t has been derived. For this class of optimal control problems the fuel consumption has been minimized. How- ever, the technique can also be applied to problems of maximum range and minimum f l i g h t time, etc. The following block diagram represents the control scheme f o r . a l l four problems. There are i n each problem three modes of control corresponding to the boosting subarc, the variable thrust subarc and the coasting subarcs (see Fig* 3.3), The switching time t^ i s determined when the switching function f goes through zero (see (3.2), (3.6), (3.9) and (3.27)) s The controller then operates to keep f = 0 u n t i l the cut-off time i s reached. In the problem of z e r o - l i f t f l i g h t * the i n i t i a l values of the Lagrange multipliers , ̂  a n < ! enter into the optimal control law. The method for evaluating the i n i t i a l values i s discussed i n Chapter 4. 3.3 Analogue Computer Technique for the Synthesis of Optimal Controllers The conditions for optimal control derived i n the l a s t Impulsive Boosting or Maximum Thrust u(t) x(t ) o • — 1 Optimal Control Law M t o ) x(t) Performance Cri t e r i o n F i g . 3.3 The modes of control for optimum rocket f l i g h t 4^ section can be used to synthesize optimal co n t r o l l e r s . D i g i t a l computers are suitable for numerical computation. However, analogue computers appear better suited for the synthesis of comparatively simple real-time c o n t r o l l e r s . The lengthy i t e r a t i v e computations of the d i g i t a l computer are replaced by r e l a t i v e l y high—speed feedback loops where an error signal i s applied to a high—gain amplifier and the amplifier output can be used as the optimal control variable. The block diagram of F i g . 3.4 shows thi s technique. 3.4 Analogue Computer Study of the Sounding Rocket Problem The analogue computer technique discussed i n Section 3.3 w i l l now be applied to the sounding rocket problem, A PACE 231-R analogue computer was used and a schematic diagram of the computer program i s i l l u s t r a t e d i n F i g , 3.5. The problem i s computed backward i n time. In F i g . 3.5 the error signal i s given by the switching function f = e(t) k mg - D(l + ) (3.28) e and the control variable by u(t) = -K e(t) (3.29) The reason for computing the problem backward i n time i s that the f i n a l v e l o c i t y , a l t i t u d e , and mass are known. Thus for backward time computation no i t e r a t i o n i s required for determining the optimal trajectory. The numerical values chpsen are the following: Impulsive Boosting or Maximum Thrust u(t) 1 Optimal Control Law M t o ) Performance P C r i t e r i o n Pig. 3.4 Synthesis of optimal controllers by means of analogue computers ON Pig. 3.5 Analogue computer program for the sounding rocket problem 7 8 h f = 4 , 8 8 9 , 5 0 0 f t . = 1 0 slug V f = 0 ft/sec D = k V 2 e " a h V e = 5 5 0 0 ft/sec k = 1 0 ~ 4 slug - f t , a = 1 / 2 2 0 0 0 f t " 1 K = 1 0 0 The res u l t i n g state variables are shown i n F i g . 3 . 6 where T = - t is the backward time variable. The function E ( T ) i s used to determine the instant T 2 > when E ( 7 " 2 ) = ®* At T = 7" 2 the following values are obtained: h 2 = 6 2 , 6 0 0 f t . V 2 = 5 , 3 1 3 ft/sec = 1 0 slug X , = 1 6 1 . 3 sec. u 2 = 0 . 7 2 sl^g/sec. and the feedback computation of thrust based on E ( T ) = 0 i s introduced by means of a relay. At T = , the following values are obtained: = 0 V 1 = 2 2 7 5 ft/sec m. ± = 2 0 . 8 5 slug 1^ = 1 7 9 . 5 sec. u^ = 0 . 5 slug/sec At T= TQt the i n i t i a l mass including fuel i s 4,889,500 4,000,000 2,000,000 " 79 161.3 lOOe 32,200 20,000-- 10,000-- 0 T sec F i g . 3.6 Experimental results for the sounding rocket problem 80 Vj mo ~ m l e x p V̂"̂  e - 31.5 slug At the instant T= 7"2, a relay switches the control variable u into the input of the mass integrator* For the coasting subarc the input to the mass integrator i s zero and the mass i s constant* At the f i n a l altitude the v e l o c i t y i s zero and the error signal e(T) i s m^g. Since both D and V increase with T i t can be seen from (3.28) that the error signal decreases to zero. At T = T 2 ^he relay operates and the rocket enters the variable thrust subarc. When h = 0, a second relay i s used to clamp a l l integrator inputs at zero, freezing the operation, (12) Leitmann v ' has used the a n a l y t i c a l results (see Section 2.3) and an IBM 701 d i g i t a l computer for the solution of the sounding rocket problem with the same given data as was used i n this section. His results are h 2 = 62,576 f t . V 2 = 5,308 ft/sec m2 = 10 slug u 2 = 0.74 slug/sec T - 7"2 = 18.7 sec. nij = 21 slug m =31.4 slug 0 6 Uj = 0.51 slug/sec, i n general this approach of using the a n a l y t i c a l result to compute the solution i s not possible, since the a n a l y t i c a l re s u l t i s not obtainable. However, the approach of F i g . 3.4 has general a p p l i c a b i l i t y . Comparison of the results shows that the 81 experimental results for the sounding rocket problem are very satisfactory. 3.5 Some Other Possible Optimal Controllers In the preceding section the switching function given by (3.28) has been used for the synthesis of the optimal control variable u by an analogue computer. The switching instant 7"2 separating the coasting subarc from the variable thrust subarc i s determined by f ( T 0 ) = 0. On the variable thrust subarc a feedback loop around a high-gain amplifier i s used to s a t i s f y the condition for optimal control which requires that e(T) = 0. It should be noted that the switching function f (T) i s a S function of state variables. In the general case of Fig* 3.1 such a switching function may not be obtainable. In thi s case some other means must be used i n order to determine the control variable u for the optimal trajectory. These can be obtained from the switching function e 2 k m K5 - V e X 3 (3.30) and the f i r s t integral (provided i t exists, see (A.18)). y e 3 k C - X 2 V - X 3(g + |) - 0(X 5- X 3 ^ ) (3.31) Therefore there are three possible functions which can be used for the synthesis of control variable u for the optimal trajectory by means of a high—gain amplifier. These are e1 = mg - D(l + (r-) (3.32) e e 2 = m X 5 - V e X 3 (3.33) £ 3 = C - X 2 V - X 3 ( g + £ ) -f3(X 5- X 3 ^ ) (3,3.4) 82 A switching function of the type given by (3.32) i s preferable since i t results i n an extremely simple con t r o l l e r . Otherwise the Lagrange mult i p l i e r s must be computed. In such a case and can be used i n the same manner as was used. It should be noted, howevery that e-j = 0 for the complete trajectory and i s not, therefore> a switching function even though i t can be used to synthesize the control variable u. In order to use (3.33), the Lagrange multipliers X^ and must be solved simultaneously with the equations of motion. It i s of interest to note that X^ and X^ can be obtained by solving the two d i f f e r e n t i a l equations (see (3.1l) and (3.12)) X 3 = - ^ f (3.35) e e If the f i r s t integral i s to be used for synthesizing the control variable u for the optimal trajectory, the complete set of Euler- Lagrange equations must be solved. This i s much more complicated than the case of solving equations (3.35) and (3.36). 4. THE MODIFIED STEEPEST DESCENT METHOD 83 4.1 Introduction Computational methods for the solution of optimization problems have had two primary directions i n the past: The direct approach and the i n d i r e c t approach. In the d i r e c t approach, equations of motion are solved by selecting an i n i t i a l control variable and then performing an i t e r a t i o n on the control variable so that each new i t e r a t i o n improves the performance function to be optimized. The i n d i r e c t approach involves the development of an i t e r a t i v e technique for solving the equations of motion and the Euler-Lagrange equations. The d i r e c t approach i s usually associated with the gradient method or the method of steepest descent. In this chapter a modified steepest descent method i s described for the solution of optimization problems which can be programmed on analogue computers. 4.2 Basic Concept of the Modified Steepest Descent Method The Mayer formulation of v a r i a t i o n a l problems has been discussed i n Chapter 2. In the case of the four rocket f l i g h t problems studied i n Chapter 3, the optimal control variable can be determined as a function of state variables and feedback control methods can be employed. In general, the control variable u for the optimal trajectory may involve Lagrange multipliers and the computation of u becomes much more complicated. The basis of the modified steepest descent i s to search for the optimum value of the performance function by replacing a 84 search i n function space by a search i n parameter space. This greatly reduces the dimensionality of the problem. The per- formance function i s considered as a function of unknown terminal conditions. The f i n a l state of the system i s determined by the solution of the equations of motion and the i n i t i a l values of the state variables. The control variable for the optimal trajectory i s determined by the state variables and Lagrange m u l t i p l i e r s . The performance function may, therefore, be considered as a function of the unknown terminal conditions for the state variables and Lagrange m u l t i p l i e r s . In theory, i f the terminal conditions for the state variables and Lagrange multi p l i e r s are a l l known, the optimization problem can be solved by the method discussed i n Section 3.2. usually not a l l known. This complicates the synthesis of the control variable u for the optimal trajectory. In such cases some of the terminal conditions may be approximately determined by some means, and then the performance function i s optimized with respect to the remaining terminal conditions, using the gradient method. This i s the essential feature of the modified method of steepest descent. Consider the problem of minimizing the performance function In many p r a c t i c a l problems the terminal conditions are P = P ( a i , . • . t (4.1) = P(t,x) J t 0 subject to the equations of motion x. = f.(t,x,u), j = l , . . . , n . J J 85 (4.2) where x = (x, ,...,x )• u = (u,»... 9u ), and the functions P 1 n 7 1 m and f. are given functions of their arguments., 3 Following the theory of calculus of variations, the augmented function n 3=1 i s formed which s a t i s f i e s the Euler-Lagrange equations ( s ) ~" "—"—- = 0 s j = J. P » o o ) n » dt 6 F 6 u , 6x..< ...6x. j a, = o, Ii. '— I ;i o e » ^ H i i and the transversality condition dP + (F - n x N V — — x.; dt + > — 7 — dx. . /Ox. J 4-v6x. J 3=1 3 3=1 3 = 0 — t 0 (4.3) (4.4) (4.5) Substituting the function F into equations (4.4) and (4.5) gives n of, X. and n X3 = "* Z A i 6 T i=l n (4.6) dP - 3=1 X . f . dt + > X - dx ^ 3 3 ^_—/ 3 3 3=1 ->t J f 0 (4.7) 0 If the function F does not depend on t e x p l i c i t l y , the f i r s t integral e x i s t s ! 86 = C (4.8) that i s n "S~^ X.f . / i 3 3 = C (4.9) 3=1 It follows from the transversality condition that i f either t or t n i s free the f i r s t integral i s equal to zero. subject to the conditions (4.7) and (4.9) so that the perform mance function P i s a minimum. Note that the transversality condition y i e l d s information about the terminal values of the A. If the f i r s t integral i s known, i t may give some information about the terminal values of x and A. However, usually not a l l terminal values of x are given and not a l l terminal values of X can be determined by the transversality condition and the f i r s t i n t e g r a l . For a minimum problem having n state variables x. the performance function P w i l l , i n general, have n unknown parameters a.. If the f i r s t integral i s known (provided i t e x i s t s ) , only 3 (n-1) unknown parameters are independent. In order to reduce the dimensionality a f i r s t approximation of these (n-l) unknown parameters may be obtained by computing a subclass of admissible t r a j e c t o r i e s which s a t i s f y the equations of motion and the known terminal conditions of the state variables. The subclass of admissible t r a j e c t o r i e s i s taken to s a t i s f y some, but not necessarily a l l , the terminal conditions for X, The i n i t i a l values of x and X for the optimal trajectory can now be determined The computational technique for the solution of the optimization problem i s to solve equations (4.2) and (4.6) 87 by the method of steepest descent. In general, a computer program using the modified steepest descent method could proceed as follows. In order to simplify the discussion i t i s assumed that more i n i t i a l values of the state variables than f i n a l values are known. (1) A suitable control u n i s selected as a f i r s t approximation and the equations of motion are solved forward i n time. If ^ ( t ^ ) i s known and x^(t^) i s unknown, an approximation to ^ ( t ^ ) can be obtained by adjusting x, (t.) u n t i l the f i n a l value of Xj^ takes on the prescribed value X j ^ t ^ K If both terminal values x. (t.) and x. (t„) of a k 1 k f state variable x^(t) are unknown, a f i r s t approxi- mation to ^ ( t ^ ) can be determined by minimizing the performance function P by the steepest descent method. The t r a j e c t o r i e s determined i n t h i s manner form a subclass of admissible t r a j e c t o r i e s . (2) Y i t h the previously determined admissible trajectory the equations of motion and Euler-Lagrange equations are simultaneously solved backward i n time. The unknown terminal values Xj(t^) are adjusted at t = t^ by i t e r a t i o n u n t i l the prescribed i n i t i a l values of the corresponding X• are obtained. A f i r s t approximation of i n i t i a l values for x and X has now been determined. (3) The equations of motion and Euler-Lagrange equations are simultaneously solved by the feedback control method (see F i g . 1.2) forward i n time. The con- t r o l l e r i s introduced by the feedback control 88 technique and the value of the performance function i s noted. This subclass of trajectories have a variable thrust subarc and the thrust for this subarc i s determined by the optimal control law. (4) The unknown i n i t i a l values of x and A are adjusted according to the modified method of steepest descent u n t i l the performance function i s minimized. 4.3 P o s s i b i l i t y of P r a c t i c a l Applications In practice, there i s often a need for a low cost and comparatively simple on-line method for the solution of optimal control problems. At the present time many of the computational techniques existing i n various industries often require the use of a large capacity general purpose d i g i t a l computer. For economical reasons, this may not be acceptable i n many possible applications. However, the modified steepest descent method can (4) be used to rea l i z e comparatively simple on-line c o n t r o l l e r s . The instantaneous control policy i n real time may be obtained from an analogue computer which operates on a fast time scale. The trajectory i n state space i s solved by an analogue computer and a d i g i t a l computer stores the data for the steepest descent adjustment of the unknown parameters. This modified steepest descent method takes account of random disturbances since a new control policy i s computed for each trajectory* (see F i g . 4.1). 4*4 Further Investigations The general idea of the modified steepest descent method based on the ind i r e c t approach of the calculus of variations Hill-Climbing Computer (Digital) u(t) Process u(t) a. a n ^ ( t ) x'(t) n ' / Trajectory Computer (Analogue) P = P(a 1,..,a n) Pig, 4*1 An optimal controller for a general process 00 90 seems a very effective computational method. The high speed analogue computer i s p a r t i c u l a r l y suitable for the determination of t r a j e c t o r i e s and feedback methods can be used to synthesize the control v a r i a b l e . While computational experience with t h i s method i s l i m i t e d at the present time, i t s potential as a computational scheme for p r a c t i c a l applications deserves further studie s. It i s suggested that further investigations i n this method should be pursued to f a c i l i t a t e p r a c t i c a l applications to the following problems: 1. The application of d i g i t a l h i l l - c l i m b i n g or gradient methods for automatically optimizing the performance function. 2. Hybrid computational methods for automatically adjusting the unknown parameters. 3. The extension of the method to problems of many degrees of freedom. A l l these problems must be l e f t open for future investigations* 91 5. FLIGHT SIMULATOR AND ANALOGUE SIMULATION 5.1 Introduction Analogue computers may be divided broadly into d i r e c t analogues and i n d i r e c t , or functional, analogues. The p r i n c i p l e of operation of the dir e c t analogue computer i s based on a one- to-one correspondence between the behaviour of the analogue system and that of the physical system under study. In the in d i r e c t or functional analogue computer, the equations which describe a physical system are formulated by components, such as summers, integrators, m u l t i p l i e r s , etc. The f l i g h t simulator i s a functional analogue computer of the electromechanical type and i s i d e a l l y suited for the solution of trajectory problems. In order to study the rocket f l i g h t problem, a CF-100 f l i g h t simulator has been suitably modified. 5.2 Basic Components of the F l i g h t Simulator There are five basic components of the f l i g h t simulator. These are the summer, servo-amplifier, resolver, phase sensitive detector and relay. By means of these components mathematical operations can be performed. The summing amplifier, or the summer, carries out the arithmetic operations of sign inversion, m u l t i p l i c a t i o n by a constant and summation. The integration i s carried out by an electromechanical integrator. This integrator consists of a servo-amplifier, a servo-motor and a tachometer. A gear box i s used to couple the servo-motor to a linear 92 potentiometer which converts the shaft angle into a voltage. Furthermore, the integrator i s also used to generate functions and to carry out m u l t i p l i c a t i o n and d i v i s i o n . The resolver performs trigonometric operations involving the transformation of coordinates. The phase sensitive detector i s a device used to detect the phase change of an input signal with respect to a reference si g n a l . A relay i s energized when the input signal changes i t s phase. 5.3 Simulation of the Optimal Control Law This section i s devoted to the simulation of the optimal control law for the z e r o - l i f t rocket f l i g h t problem discussed i n Chapter 3. For the programming of this problem a large number of m u l t i p l i e r s and function generators are required. This cannot be handled by most ordinary analogue computers since only a small number of multipliers and function generators are normally available* The electromechancial computing units of a f l i g h t simulator are i d e a l l y suited for this type of problem. In the study of the theory of optimal rocket f l i g h t , i t has been shown that the optimal trajectory consists of three subarcs. Associated with each subarc i s a mode of con- t r o l for the control parameter 0. If impulsive boosting i s assumed, one of the subarcs may be computed a n a l y t i c a l l y . If the thrust program consists of maximum thrust, variable thrust and zero thrust, the maximum thrust mode must be included i n the simulator. In general there are, therefore, three modes of thrust control. It can be seen from the Appendix that the control parameter 93 (3 f appears i n both equations (A.46) and (A.48). The three modes of thrust control must, therefore, be applied to these two equations. The sequence of the modes i s important. It follows from the theory of rocket f l i g h t that the sequence of these modes ares Mode Is (3 = B , constant thrust, "max Fuel consumption i s at a constant rate and the mass i s a li n e a r function of time, m = m„ - (3 t . 0 rmax V Mode 2l = 0, variable thrust. The mass i s constrained.to s a t i s f y the variable thrust condition for optimal f l i g h t . Mode 3: P = 0, zero thrust. The mass i s constant. The zeros of the function A V f (m,/0 £ X 5 - X 3 ^ (5.1) can be used to define the three subarcs (see F i g . 5.1). The switching from Mode 1 to Mode 2 i s performed i n the simulator by a phase sensitive detector and a relay. In Mode 1, the relay i s i n the position for maximum thrust. When f (m.,\) becomes zero, the relay switches to Mode 2. During Mode 2 the control parameter 0 i s i m p l i c i t l y constrained so that f(m,A) = 0. For Mode 3, the signal representing the control parameter 0 i s shorted to ground. 5.4 Analysis of a Test Problem In order for the simulator to perform s a t i s f a c t o r i l y , f(m,X) Pig. 5.1 Three modes of thrust control various units must be calibrated. The ca l i b r a t i o n can be best performed by solving a simple problem of free motion described by the following d i f f e r e n t i a l equations: x = V cos 0 h = V sin 0 ( 5 V = -g sin 0 O = - ^ cos 0 The i n i t i a l conditions at t = 0 are x(0) = 0 h(0) = 0 V(0) = V ' o 0(0) = 0 95 where 0 < 0 q< J . The solution of this set of d i f f e r e n t i a l equations i s x = V cos 0 t o o h = V sin G t 4 g t 2 o o 2 ° V 2 = V q 2 + g 2 t 2 - 2 g V O sin 0 o t (5.3) tan (| + J) = tan (^ + /Jt2 - 2 ̂  sin 0 Q t + ^ _ Eliminating the sin 0 from the second and the t h i r d equations of (5.2) gives h = -Y V/g Integrating the above equation yields V 2 = Y2 - 2 g h (5.4) Since V cannot be zero, i t follows from the second equation of (5.2) that sin 0 must be zero at h . Furthermore, because of max 7 (5.4), V i s a minimum when h i s a maximum. From the solution for the v e l o c i t y of (5.3), i t i s seen that V . = V cos 0 (5.5) min o o s ' which i s extremely useful for c a l i b r a t i o n purposes. Another important fact i s that the v e l o c i t y i n the x- dir e c t i o n , that i s , x i s always constant. This gives a good check for the operation of the simulator. D i f f e r e n t i a t i n g the solution for the v e l o c i t y and equating i t to be zero gives 96 V t = — sin 0 (5.6) g and at this instant the v e l o c i t y reaches i t s minimum. The above equations were used to scale the voltages on the simulator so that for the mass used the trajectory covered a convenient range of an xy-recorder. 5.5 Experimental Test of the Modified Steepest Descent Method descent has been discussed i n Chapter 4. It would evidently be profitable to study a particular problem which can lead to a better understanding of the nature of the method. formance function to be minimized i s the fuel consumption. If the i n i t i a l mass mQ i s assumed to be given, the problem i s equivalent to maximizing the f i n a l mass m̂ . The i n i t i a l and f i n a l conditions are where mQ, x^ and h^ are given values. The following values of the state variables are unknown at the terminal points: © q , 0 V„, m„. Here m„ i s to be maximized. The basic idea for the method of modified steepest Consider the z e r o - l i f t rocket f l i g h t problem. The per- x ( t f ) = x f h ( t f ) = h f (5.7) The transversality condition for this problem i s 97 = 0 t - C dt + Xj dx + X 2 dh + X 3 dV + X 4 d© + (\ g-l)dm o (5.8) The quantities t Q , t ^ , Vft © 0» > m £ are free,so that C = 0, ^40 = °' ^3f = °» *4f = 0 a n d ^5f = l f a n d ^10' ^20' ^30' ^50* X J J and X ^ are unknown. The f i r s t integral (see(A.55)) i s XjV cos 0 + X 2V sin 0 - X 3 ( - + g sin 0) - X 4 f cos © - 8(X 5 V 3 m and for t = t ^ , B = 0, X 3^ = 0 and X ^ = 0. Hence tan ©„ = - ^ (5.10) r x 2 f Equation (5.10) gives a r e l a t i o n between ©^, X-^ and X^. From the Euler-Lagrange equation (A.49) i t i s seen that X^ i s a con- stant for the entire optimal trajectory. For this p a r t i c u l a r problem © q can not be 90°, as can be seen from the equation of motion (A.47) for 9. If 0 = 90°, and the l i f t i s zero, © i s zero i f V i s not zero, thus the ' o ' f i n a l point (x^,h^) cannot be reached. If 0 q < 90°, then V cannot be zero, otherwise © w i l l be i n f i n i t e at the i n i t i a l point. Thus an i n i t i a l v e l o c i t y i s essential which can be obtained by impulsive boosting. In this case, the computation starts with the variable thrust subarc, since the boosting subarc i s very short and may be neglected. 98 Consider now the case of impulsive boosting where there i s no constraint on the magnitude of the thrust. Let t^ be the time at the end of boosting, then t, - t = A t =" 0 1 o x, = x =0 1 0 h n = h =0 1 o V, £ 0 V m l ~ mo e x p (~ (5.11) e hi - x i o A21 ~ 20 A31 ~ 30 A 4 1 = A 4 Q = 0 X51 S X 5 0 + X 3 0 Ve^m7-m-) 1 o At t = t^, the variable thrust subarc starts, and V X 5 1 = X 3 1 m f <5-12> If the computation starts at t = t^, the i n i t i a l values for the state variables: , m̂  and 0^ are unknown. However, and m̂ are related by the r e l a t i o n V. m i = mQ exp (- =i) (5.13) e If the magnitude of the thrust i s constrained by the condition where |3 i s the maximum control parameter, the approximation of 99 (5.11) s t i l l can be applied, but the optimal trajectory w i l l start with maximum thrust subarc. Since the i n i t i a l v e l o c i t y V q i s zero, some a u x i l i a r y device i s required to avoid that 0 be i n f i n i t e at the s t a r t . This can be done by holding the rocket on a launcher with maximum thrust for a negl i g i b l y short time, and the rocket then starts with a maximum thrust subarc with an i n i t i a l angle O q less than 90°. This i s equivalent to the problem of starting with an i n i t i a l v e l o c i t y ^ 0 and an i n i t i a l mass given by V. m. £ mQ exp (- ̂ ) (5.15) Thus the optimal trajectory starts with the following i n i t i a l conditions: t. - t = At a* o i o x. ^ x = 0 1 o h. * h =0 1 o V. £ 0 V. m i = m 0 e x p (~ * l i A10 X 2 i 04. X20 * 3 i Q± X30 * 4 i so X40 k5i X50 (5.16) In this case the switching function may not reach zero at t = t ^ . The optimal trajectory must then start with a maximum thrust subarc. When the switching function (5.1) i s zero, the trajectory enters the variable thrust subarc. The computation starts at 1 0 0 t = t. with the i n i t i a l values of the state variables V., m. i 1 ' l and ©^ unknown. However, "V\ and are related by the equation ^ = mQ exp (- ^ ) (5*17) For s i m p l i c i t y , the drag function D used i n the simulation i s assumed to have the form D = D(V,h) e r2 = k V 2 " a h (5.18) * k 1 + ah To determine a f i r s t approximation for the i n i t i a l values of "V\ , HK and ©^, the trajectory i s considered to consist of a suitable constant thrust subarc or a maximum thrust subarc and a zero thrust subarc* A value i s selected and computed by (5,17). A suitable i n i t i a l value ©^ i s chosen and the length of the constant thrust subarc varied so that the f i n a l point (x^jhp) i s reached. F i g . 5.2 i l l u s t r a t e s the results obtained for various ©^, The value of m̂  for each of these t r a j e c t o r i e s i s noted and the results are plotted as shown i n F i g . 5*3o In t h i s manner 0^, V\ and are approximately determined. A pa r t i c u l a r set of data i s shown i n F i g . 5,4. A l l quantities on the simulator are i n terms of degrees of shaft rotation. Since ©^ i s now known at the f i n a l point, i t follows that A - j ^ and X^f a r e related by A 2 f = " c o t G f X l f (5*19) Note that at the f i n a l point, A-^ and X^f a r e the only unknowns. 1 0 1 F i g . 5.3 Determination of approximate i n i t i a l values for the state variables 102 m. 202°+ 201' 200C 199 u 198°I •v 70° 71° 72° 73° 74° 75° -0. I F i g . 5.4 A pa r t i c u l a r set of approximate i n i t i a l values of the state variables If is.known, can be computed by (5.19), Therefore, by selecting a X-^^t the equations of motion and the Euler-Lagrange equations can be solved backwards i n time. The Lagrange multi- p l i e r X-^f i s varied u n t i l the condition X^^ = 0 i s s a t i s f i e d . A l l i n i t i a l values are now specified and i t i s then possible to compute improved t r a j e c t o r i e s by introducing the optimal control for the trajectory and solving i t forward i n time. The f i n a l mass m̂  i s now considered as a function of the parameters: 9^, A ^ , A 2 i 7 A 3 i T a n d 0 P " k i m u m values of these parameters can be determined by the modified steepest descent method. The adjustment of the parameter values terminates when m̂  reaches a maximum* This approach proved f a i r l y successful on the f l i g h t simulator. The numerical res u l t i s i i i terms of degrees of ghaft rotation. Since the f l i g h t simulator does not have a high accuracy, no 103 precise numerical results have been obtained. However, a set of t r a j e c t o r i e s similar to F i g . 5.2 consisting of a maximum thrust subarc* a variable thrust subarc and a zero thrust subarc can be obtained* F i g . 5.5 i l l u s t r a t e s the performance function m_ considered as a function of the parameter a, . m̂  " 202° + 0 -*- a a, k k opt. F i g . 5.5 Optimum performance function At the point a^ are = a. opt. , the i n i t i a l values of the a l = Q± = 73° a 2 = V± = 50° a 3 = m± = 330° a 4 = X1± = 168° a5 = X 2 i = 219° a 6 = X 3 i = 253° a ? = X 4 i = 0° (This i s known) For t h i s problem the Lagrange m u l t i p l i e r X^ i s obtained from the f i r s t i n t e g r a l . Therefore, X^^ i s fixed by the f i r s t i n t e g r a l . 6. CONCLUSION 104 General optimal control problems formulated by the method of the calculus of variations with pa r t i c u l a r emphasis on the problem of Mayer have been studied. Special cases of optimal control can be realized by means of feedback control. The Lagrange mult i p l i e r s can be eliminated and the control variable for the optimal trajectory i s then a function of the state variables only. In this case the optimal control system can be treated as an optimal feedback control system. Analogue computer methods are convenient for the solution of such problems. The modified steepest descent method i s suitable for the solution of certain classes of optimal control problems. (1) For very complex problems the dimensionality of the problem can be reduced by using conventional i t e r a t i v e and gradient methods to determine subclasses of admissible tr a j e c t o r i e s i s a t i s f y i n g some, but not necessarily a l l , of the terminal conditions. Thei modified steepest descent method can then be used to optimize the performance function which i s con- ! V sidered to be a function of the remaining terminal con- d i t i o n s . (2) Simulator and analogue computer results show that the method i s p r a c t i c a l and can be used to synthesize r e a l - time optimal c o n t r o l l e r s . (3) For complex problems hybrid-computers are essential and are of considerable future i n t e r e s t . This thesis has dealt mainly with the analogue portion of the optimal con- t r o l l e r . The optimization of the performance function has 105 been performed by a manual search. In an actual system the optimization would be performed by a d i g i t a l computer (see F i g . 4.1). The analogue computer i s suitable for high speed trajectory computations while the d i g i t a l computer i s suitable for the l o g i c a l operations involved i n the optimization of the performance function. The results of the research undertaken show tl\at analogue computers can be used to synthesize the control variable for optimal con- t r o l once the correct i n i t i a l values are known. It i s well known that d i g i t a l computers can readily optimize a performance function P of several variables by some type of gradient method. The optimization of P i s used to determine the correct i n i t i a l values. It can therefore be concluded that i t i s possible to synthesize optimal con- t r o l l e r s for a variety of systems by hybrid computational means. REFERENCES 106 1. Kelley* H.J., "Gradient Theory of Optimal F l i g h t Paths", ARS Journal 30, 947-954, I960* 2. Bryson* A*E* , C a r r o l l , F.J., Mikami K., and. Denham, W.F., "Determination of the L i f t Drag Program that Minimizes Re-entry Heating with Acceleration or Range Constraints Using a Steepest Descent Computation Procedure", presented at IAS 29th Annual Meeting, New York, N#T*f Jan*, 23-25, 1961. 3. Bonn* E*V., "Solution of a Class of Optimal Control Problems by a Systematic Iterative Technique", Canadian IEEE Convention, Toronto, 1962* 4. Bonn, E*V*» "The P r a c t i c a l Realization of Optimal Control of Multivariable Dynamic Processes", Canadian Industrial Research Conference, Carleton University, Ottawa, 1964. 5. Bellman, R.E., "Adaptive Control Processes",: Princeton University Press, 1961. 6. Bellman, R.E. and Dreyfus, S.E., "Applied'Ifynami'Bo'PrQgramming'!, - Princeton University Press, 1962* 7. Leitmann, G*, "Optimization Techniques", Academic Press, 1962. 8. B l i s s , G.A., "Lectures on the Calculus of Variations", University of Chicago Press, 1946* 9. Pontryagin, L.S., Boltyansky, V.G., Gambrelidze, R.V. and Mishchenko, E.F., "The Mathematical Theory of Optimal Processes", Interscience Publishers, John V i l e y , 1962. 10. Miele^ A., "General Variational Theory of the F l i g h t Paths of Rocket-Power A i r c r a f t , Missile and S a t e l l i t e C a r r i e r s " , Astronautica ACTA 4, No* 4, 1958. 11. Tsien* H.S* and Evans, R.C., "Optimum Thrust Programming for a Sounding Rocket", ARS Journal 21* No. 5, 1951. 12. Leitmann, G», "Optimum Thrust Programming for High-Altitude Rockets", Aero/Space Eng., 16, No* 6, 1957* APPENDIX 107 1. The Euler-Lagrange Equations for Rocket Plight Problems Substituting the augmented function F of (2.10) into (2.11) y i e l d s the set of Euler-Lagrange equations X l = X 6 + X7 U , l ) : x 3 6D . . 64> . d¥ u 0s X2 = "^6h + X 6 6h + X7 5h ( A * 2 ) * X3 6D X3 = ~ X1 c o s ® ~ X2 s i n ^ + ~~in Q V L + V B sin <o + KA ( — 5 - ^ cos 0) + X 6 OV + X7 6V ( A* 3 ) X4 = X l ^ s^"n ® ~ X 2 ^ C 0 S 9 + X 3 g c o s 9 - X 4 f sin 0 + X 6 g f + X 7 (A.4) X 5 = ~2 ( V P c o s w - D ) + ~ 1 ~ ( L + V 0 sin to) m e m*T e + x 6 < > + x 7 ( > ( A ' 5 ) n  x 3 6D x4 ^ . M> x 6Y 0 = ^ 6 x " m V + X 6 0 L + X 7 O L ( A * 6 ) 0 = to-<-X3 h c o s - " X4 h S I N W + V X 6 0 T + X y (A.7) V V 6* 6Y 0 = X 3 -t s i l> * ~ X4 "mY c o s « + X 6 § ^ + X 7 3 ^ (A.8) 108 2. The V e r t i c a l Plight (The Sounding Rocket) Problem Assume that the thrust d i r e c t i o n i s v e r t i c a l and that the two additional constraints are $ = 0 - 1 = 0 (A*9) Y= " • = 0 (A,10) The equations of motion become <f2 = h - V = 0 (AU1) B - V p ^ 3 = V + g + = 0 (A.12) vj>5 = m + 0 = 0 (A.13) The Euler-Lagrange equations are * 2 = mSh ( A * 1 4 ) h = ~ X2 + ~ l o f (A.15) A 5 = ̂ | (V e0 - D) (A.16) m The f i r s t i ntegral i s n V X 2V - X 3 ( g + - 0(X 5 - A 3 = C (A.18) 3. The Horizontal F l i g h t Problem If the f l i g h t path i s assumed to be horizontal and i f the thrust d i r e c t i o n i s p a r a l l e l to V, the additional constraints are <£= © = 0 (A.19) Y = » = 0. (A. 20) 109 The equations of motion are ^ = x - V = 0 (A*21) D - V B ^ 3 = V + m e = 0 (A.22) vf 5 = m + p = 0 (A.23) The Euler-Lagrange equations are Aj = 0 (A*24) X 3 =~h + ^ O T (^?5) L = ~ f <V " D ) + X4 \ ( A * 2 6 ) m m v 0 - H - ^ (A-27> 0 = i f <*5 - S ^ <A-2«> The f i r s t integral i s V - * 3 i - K l j - » 3 ^ - ° ( A - 2 9 ) 4. The A r b i t r a r i l y Inclined Rectilinear Plight Problem If the f l i g h t path i s r e c t i l i n e a r at an arbitrary angle 0 with respect to a horizontal plane and i f the thrust d i r e c t i o n i s p a r a l l e l to the f l i g h t path, the additional constraints are <§> = 0 - constant =0 „ Y = «> = 0 (A*30) The equations of motion are ip 1 = x - V cos 0 = 0 (A.31) KO0 = h - V sin 0 = 0 (A.32) T Z . D - V p = V + g sin 0 + -^-2- = 0 (A.33) 110 ^ 5 = m + 0 = O (A.34) The Euler-Lagrange equations are x\ = 0 (A.35) A 2 = ^ O T ( A * 3 6 ) * X-j = -X-̂  cos © - X 2 sin 0 X + (A.37) 3 6D m A 5 = ~ l ( V " D ) + " I 7 ( A * 3 8 ) m m The f i r s t i n t e g r a l i s X-jV cos © + X 2 V sin © - X 3(g sin © + ^) - 8(X 5 V ~ A, — ) = C (A*41) -> m 5. The Z e r o - l i f t F l i g h t Problem If the thrust d i r e c t i o n i s tangent to the f l i g h t : path and i f the l i f t i s assumed to be zero, the additional constraints are <£> = L = 0 (A*42) ^ = <o = 0 (A*43) The equations of motion are f 1 = x - V cos © = 0 (A.44) l f 2 = h - V sin Q = 0 (A.45) D - V 8 V f 3 = V + g sin © + jjp^- = 0 (A.46) 4̂ 4 = 0 + f c o s Q = 0 (A.47) vf 5 = ra + 6 = 0 (A.48) The Euler-Lagrange equations are X x = 0 (A.49) * X 3 on A2=-5h (A.50) * X ^ On A- = -X, cos © - X„ sin © + m 6 T " X4 ^2 c o s 0 (A*51) XjV sin © - X2"V cos © + X-jg cos © - X 4 f sin © (A.52) X 5 = ~ i ( V e P " D ) ( A* 5 3> * X^ The f i r s t integral i s X,V cos © + X 0V sin © - X,(g sin © + -) - X, # cos © •L £ J m 4 V V - P(X 5- X -&) = C (A.55)

Cite

Citation Scheme:

    

Usage Statistics

Country Views Downloads
Japan 6 0
China 4 0
United States 2 0
City Views Downloads
Tokyo 6 0
Beijing 4 0
Mountain View 1 0
Ashburn 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}

Share

Share to:

Comment

Related Items