GEOMETRIC APPROACHTOMONOTONE STOCHASTIC CONTROLbyMaria ChiarollaLaiirea in Matematica, Università degli Studi di Ban, 1982A THESIS SUBMITTED IN PARTIAL FULFILMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYINTHE FACULTY OF GRADUATE STUDIESDEPARTMENT OF MATHEMATICSWe accept this thesis as conformingto the required standardTHE UNIVERSITY OF BRITISH COLUMBIAApril, 1992© Maria Chiarolla, 1992National Libraryof CanadaBibhothèque nationaledu CanadaCanadian Theses Service Service des thèes canadiennesOttawa. CanadaKIA 0N4The author has granted an irrevocable non-exclusive licence allowing the National libraryof Canada to reproduce, loan, dsbibute or sellcopies of his/her thesis by any means and inany form or format, maldng this thesis availableto interested persons.The author retains ownership of the copyrightin his/her thesis. Neither the thesis norsubstantial extracts from it may be printed orotherwise reproduced without hislher permission.L’auteur a accordé une licence irrevocable etnon exclusive pennettant a Ia Bibiothéquenatiönale du Canada de reproduire, préter,distribuer ou vendre des copies de sa thesede quelque manlére et sous quelque formeque ce soit pour mettre des exemplaires decette these a Ia disposition des personnesintéressées.L’auteur conseive (a propciété du droit d’auteurqui protege sa these. Ni (a these ni des extraitssubstantiels de celle-ci ne doivent êtreimprimés ou autrement reproduits sans sonautorisation.1IiuanaciaISBN 0-315-75398-6In presenting this thesis in partial fulfilment of the requirements for an advanceddegree at the University of British Columbia, I agree that the Library shall make itfreely available for reference and study. I further agree that permission for extensivecopying of this thesis for scholarly purposes may be granted by the head of mydepartment or by his or her representatives. It is understood that copying orpublication of this thesis for financial gain shall not be allowed without my writtenpermission.(Signature)Department of I’.A4TLE MAT( C SThe University of British ColumbiaVancouver, CanadaDate AP.(L 2’I-. ifDE-6 (2/88)AbstractThe two main questions arising in singular control problems are the characterizationof the boundary of the region of inaction A’ (i.e. the region where it is optimal totake no action) and the construction of an optimal control. Among the singular controlproblems the ones in which the class of admissible controls is restricted to the processeswith monotone non-decreasing components, and the payoff functional does not dependexplicitly on the control, are usually referred to as monotone follower, cheap controlproblems.We identify the free boundary DA’ of the two-dimensional monotone follower, cheapcontrol problem under very mild conditions. We prove that if the region of inaction isof locally finite perimeter (LFP), then such a region can be replaced by a new regionhaving a more regular boundary. In fact, we show that the new free boundary iscountably 1-rectifiable and it is also optimal to take no action in the larger set A’. Thenwe give conditions under which the hypothesis (LFP) holds; furthermore we obtain evenhigher regularity of the free boundary, namely C2”, except perhaps at a single cornerpoint. This result is easily extended to the n-dimensional case.Under the additional hypothesis that the free boundary of the new region of inactionA’ satisfies a Lipschitz condition (LIP) in a small neighbourhood of the corner point, weconstruct a control k which acts only when the process is not in A’ and then only tomove it instantaneously into A’. We show that k is the unique optimal control of thesingular control problem in question. Finally we give conditions under which (LIP) isverified. All of these results hold in the n-dimensional case.1’Table of ContentsAbstract iiList of Figures vAcknowledgments vi1 Introduction 11.1 Singular Control Problems 11.2 Objective 92 Characterization of the Value Function 112.1 Statement of the Problem 112.2 Some Properties of the Value Function 122.3 A Variational Inequality for the Value Function 273 The Free Boundary 343.1 Introduction 343.2 Identification of the Boundary . . 343.3 A New Approach to Regularity . 483.4 Regularity of the Free Boundary 573.5 Finite Perimeter of A’: a Verification of (LFP) 823.6 Higher Regularity of the Boundary 914 The Optimal Control 1021114.1 Introduction 1024.2 The Optimal Control : a Heuristic Description 1024.3 Construction of The Optimal Control Process 1064.4 Optimality 1324.6 A Verification of (LIP) and (NIN) 137Bibliography 144ivList of Figures3.1 Possible cases (a), (b) and (c) occurring in the proof of Proposition 3.5,taking &2 continuous 433.2 Sketch of the region of inaction A’ and its complement R0 U R1 U R2. . 46VAcknowledgmentsI find my words inadequate to express my gratitude to my supervisor, Prof. Ulrich Haussmann, for his invaluable contribution to my growth as a person and as a researcher. Overthe years Ulrich has been a continual source of inspiration, support and encouragement.I wish to thank him for his friendship, for his vigorous sense of guidance, for the numerous entire afternoons generously spent discussing mathematics, for his patience andunderstanding, and especially for being there when I felt helpless and lost.I wish to thank Prof. Larry Roberts for being supportive at a time when my lifetook a particularly difficult turn. My officemate Yunsun Nam endured me for threeyears and maintained me with her friendship. Ian Lisle efficiently typed Chapter 3(and much more!), but my most profound gratitude goes to Ian for his constant, tactfulcaring and for his many attentions. Chapter 3 was inspired by the work of Prof. RenatoCaccioppoli, who initiated the theory of sets of finite perimeter in 1952. Chapters 1, 2and 4 were courageously typed by Bernardo Hernandez-Morales; his tireless patience inmaking the continuous changes and corrections is gratefully acknowledged. Prof. KeeLam and Ms. Joan de Niverville always took personal interest in matters concerningthe graduate students; their cheerful and competent help is very much appreciated. Mywarmest thanks to Lily Crawford for allowing me to reshape the trees in her backyard ina Mediterranean fashion, and so release my tension.Above all my effort was continuously sustained by my family in Italy. This thesis isdedicated to my parents, Lisetta and Giovannino, with respect and gratitude.This thesis was typeset using LATEX.viChapter 1Introduction1.1 Singular Control ProblemsControl problems for diffusion processes with additive controllable input, when the payofffunctional does not depend explicitly on the control (i.e. the so-called cheap controlproblems) often lead to problems in which the displacement of the state caused by theoptimal control is singular with respect to Lebesgue measure as a function of time. Theoptimal control has usually to be found among controls of bounded variation on finitetime intervals, or, in particular, in the subset of controls whose components are increasingin the time variable. Thus, absolutely continuous controls are allowed as well as impulsecontrols. However, the value function will not be attained at any of these. The optimalcontrol k is singular, and may be characterized by two regions : the region of inactionA’, which is an open set, and its complement (A’)’, the region of action. If the processis in (A, then the control k will make the process jump to some point on the boundaryOA’, thereafter k acts only when the process is on ÜA’ and pushes it back into theregion of inaction A1. The optimal control forces the process to stay inside A’ with aninstantaneous action at the boundary OA’. Therefore k has the peculiarity of the “localtime” of the process at the boundary OA’.It should be noticed that such behaviour of the optimal control has really been shownonly for 1-dimensional problems, except for a few very special classes of 2-dimensionalproblems. The reason for this is that several conceptual and technical difficulties arise in1Chapter 1. Introduction 2more than one dimension as we will explain below.Singular control can be approached as the limit of impulse control. Menaldi andRofman [MRfJ study a cheap control problem for n-dimensional diffusion processes whereonly impulse controls are allowed. They obtain the optimal cost as a limit of impulsecontrol problems having a cost for each impulse. The existence of an optimal control isproved only after having restricted the problem to a very particular subset of impulsecontrols. This difficulty suggests already that an optimal control (if there is one) has tobe sought in a much larger set of admissible controls. In fact, Menaldi and Robin [MRbIprove that the infimum of the payoff functional E{f f(Xt)et dt} is continuous as afunction of the initial state, and is the same over the sets of continuous controls, impulsecontrols, or purely jumps controls as long as they are all of locally bounded variation.Menaldi and Robin allow the cost rate f to have polynomial growth, and the diffusionX to be n-dimensional. Moreover, in the 1-dimensional case, they prove the existence ofa singular optimal control k in the class of non-decreasing, non-negative controls, for anon-degenerate diffusion. The optimal process is characterized as the diffusion reflectedat the boundary A1. Moreover, the optimal cost is shown to be C2 and is obtained asthe maximum solution of(All—f)ll = 0 a.e. iniR;e W1’°(1R); All fii.0;(here A is the characteristic operator of the process), where the constraint ll, 0 isformally derived in the limit of impulse control problems when the cost of impulse tendsto zero. A generalization of this result to the 2-dimensional case leads to a variationalChapter 1. Introduction 3inequality of the type(Ai— f)’ñ,n2 = 0 a.e. inR2;i=1,2.Again f is supposed to be convex and the process is assumed to have constant drift anddiffusion coefficients (g and o respectively). Then ft is convex, but a new difficulty arises:identifying the region of inaction and proving that it has some regularity.Menaldi and Taksar [MT} study a multi-dimensional problem under the same hypotheses on f, g, u as above, but allow a cost for the control, which enters the equationof the motion additively and is of bounded variation. They approach the optimal costft as a limit of problems with absolutely continuous controls by means of penalization;so they prove that ft is a C’ solution of the Hamilton-Jacobi-Bellman equation of theproblem. The existence and uniqueness of the optimal policy k is also established, butnot in a constructive way. Menaldi and Taksar conjecture, as the reader might havealready guessed, that k should be singular and the optimal process should reflect at OA’in the direction of —Vu.It is now clear that in more than one dimension we have to face problems like thesmoothness of the boundary dA’, and the lack of knowledge of the direction of thereflection. Moreover, even if intuitively we may guess —Vft to be the optimal directionof reflection (since this is the direction of least increase of the cost ft), for some pointson the boundary the gradient Vft may be zero, and so leave the direction of the pushingindeterminate; besides, —Vft need not be among the admissible directions! These areessentially the main problems that have prevented researchers from constructing theoptimal directions and showing the singular feature of the optimal control. The main aimof this work is to overcome this limitation, cf. the statement of objective in Section 1.2.The question of the smoothness of the free boundary 0A1 is studied in a recent paperChapter 1. Introduction 4of Williams, Chow and Menaldi [WCMJ. The state is n-dimensional as in [MT], but thecontrols are monotone non-decreasing and non-negative (component-wise), f is of classC3, and a is non-degenerate. Under the extra hypothesis that and Vf1 never vanishsimultaneously, Williams et al. differentiate the Bellman equation and obtain obstacleproblems solved by the components of the gradient of the value function of theoriginal problem. To these new problems they intend to apply a result of Caffarelli [Cf]concerning the regularity of the free boundary arising in obstacle problems. Caffarelli’sresult guarantees C’-regularity in a neighbourhood of each point p e DA’ of positiveLebesgue density for the coincidence set {i = 0}. However, except for a very special 2-dimensional case (cf. (3.170)), Williams et al. are unable to prove that the points of OA’are points of positive Lebesgue density for the coincidence set. We solve this problem;we show that in their setting, all points of DA’ are of positive Lebesgue density; in fact,the boundary is Lipschitz, except perhaps at one single point (cf. Section 3.6).In some cases, the region of inaction A’ will be a wedge; this is the case for the2-dimensional portfolio model studied by Davis and Norman [DN]. Essential for thedetermination of A’ is the choice of the utility function f. In fact, they choose f(x)equal to x7/’-y or to log x, and from this they deduce the “homothetic property” and theconcavity of the optimal profit (note that in their paper i is a sup not an inf). The“homothetic property” allows them to find the form of ii, ñ c C2, and prove that A’ isa wedge in the positive orthant of 112. The unique optimal control k turns out to beequal to c,L + c2U where c1, c2 are constant directions, and the 1-dimensional processesL and U are the local times at the lower and upper boundaries respectively. Because ofthe particular shape of the region of inaction A’, the results of Varadhan and Williams[VW] or of Tanaka [Tkj apply and provide the optimal process at least up to the firsthitting time of the corner, which is not finite under the optimal policy.Chapter 1. Introduction 5Sometimes 1-dimensional singular control problems can be reduced to equivalent questions of optimal stopping. In fact, it can be shown that the space derivative of theoptimal cost ft coincides with the optimal risk th of an appropriate stopping problem,whose optimal continuation region is precisely the region of inaction of the control problem. Therefore, by solving the free boundary problem that characterizes the optimalrisk zij, one can obtain bounds on the continuation set of the stopping problem, and soon the inaction region of the singular control problem. This connection between singular stochastic control and optimal stopping was developed rigorously by Karatzas [Krl]mostly by analytical methods based on properties of solutions to free boundary problemsand variational inequalities. Subsequently, Karatzas and Shreve [KS 1] established theconnection between the two problems by using only direct probabilistic arguments. Inparticular, they proved the existence of an optimal control Ic for the control problemwith zero drift, in the setting of the monotone follower problem (i.e., over the set ofnon-decreasing controls), and they showed that the optimal stopping time & for the associated stopping problem is exactly the first time k acts (i.e., & = inf{t : > O}.) Theexistence of & is a straightforward consequence of the equivalence between control andstopping problems, and is obtained relatively easily from the existence of Ic. It shouldbe noticed that control processes are more easily topologized than stopping times, andtherefore Ic may sometimes be obtained by using compactness arguments, while a directproof of the existence of ô may require a number of more or less strong conditions (e.g.Friedman [Fri], van Moerbeke [vM]).The equivalence between control and stopping problems is used by Chow, Menaldi andRobin [CMR] to determine the free boundary 0A1 that separates the regions of action andinaction for a non-stationary control problem, whose 1-dimensional state is governed bya linear S.D.E. possibly degenerate, with time-dependent coefficients, and with controlsto be chosen among non-negative, non-decreasing processes. Chow et al. approach theChapter 1. Introduction 6control problem by a sequence of absolutely continuous control problems; this enablesthem to prove that the optimal cost ‘ü is the unique solution of a certain variationalinequality. Then they construct the optimal control, which is, in fact, Markovian andwhose input produces a reflected diffusion process as the optimal process. To constructsuch reflected diffusion, they assume some regularity of the free boundary.We point out that Chow, Menaldi and Robin also approach the control problem bya sequence of impulse control problems, after having shown that the infimum over theset of impulse controls and the one over the set of Lipschitz controls are both equalto the optimal cost il. However, under the hypothesis of convexity of the cost rate f,the value function 11e of each approximating Lipschitz control problem turns out to beconvex, while the value function ii of each approximating impulse control problem isonly e-convex (see Remark 2.7 for the definition). The convexity of j1e is then used todetermine a variational inequality for the optimal cost ñ. This is one of the reasons whyseveral authors prefer to approximate by absolutely continuous control problems insteadof impulse control problems. The reader may consult Baldursson [Bd] for conditionsunder which an approximation by absolutely continuous controls is possible, as well asHeinricher and Mizel [HM] for a counterexample. In Heinricher and Mizel’s model thevalue function obtained by minimizing over the set of bounded variation controls isstrictly smaller than the value function corresponding to absolutely continuous controls.The connection with optimal stopping often provides some information about the freeboundary OA’ for the singular control problem in the non-stationary case, as explainedabove, but only for 1-dimensional problems. In fact, in the multi-dimensional case suchconnection is no longer available since the gradient of the optimal cost i is a vector-valuedfunction; therefore, it cannot play the role of the optimal risk for a stopping problem.One might think of taking some directional derivative, i.e. a scalar valued function, butthe choice of a direction will very likely be related to the optimal control. Therefore, itChapter 1. Introduction 7remains an open question how to formulate an opportune stopping problem having someimplications for the control problem.A method somewhat similar to the one described above, in which space-derivativesare taken, is introduced by Soner and Shreve [SS] for a 2-dimensional problem, withconvex cost rate f, zero drift and constant diffusion coefficient, over the set of controls ofthe form j N8d8, where N3 is a unit vector and is a 1-dimensional, non-decreasingprocess. Their method uses the gradient flow of ‘I to change to a more convenientpair of coordinates, and to obtain a more standard free boundary problem. By usingthis ingenious device, Soner and Shreve characterize the optimal cost l as the uniqueC2—solution of the Hamilton-Jacobi-Bellman (HJB) equation. They also show the freeboundary OA’ to be of class C2” for any c E (0, 1); such smoothness of DA’ is essentialin their construction of the optimal process, which is a 2-dimensional Brownian motionreflected along A’ in the —Vi direction, and is obtained as the unique solution ofthe Skorokhod problem under the necessary conditions of Lions and Sznitman [LS]. Theproof of Soner and Shreve’s result makes critical use of the 2-dimensional nature of theproblem; therefore, it cannot be extended to higher dimensions.So far we have described only two 2-dimensional problems (i.e. [DN], [SS]), and theyboth admit a value function ii of class C2. Let us mention another class of 2-dimensionalproblems for which the optimal cost ‘IL is of class C2, and a connection with optimalstopping is still available. This is the class of finite-fuel problems; these are obtainedby studying a 1-dimensional Brownian motion under a constraint on the total variationof the control process, so that a reduction to a 2-dimensional problem is possible if onedefines the second state variable y to be the remaining fuel. Therefore, one ends updealing with a 2-dimensional problem in which, however, the second state variable isnot a diffusion, but stays constant until the time the control acts, and at that time itdecreases an amount equal to the displacement caused by the control. That is indeed aChapter 1. Introduction 8very special class of 2-dimensional problems.The finite fuel monotone follower problem is solved analytically by Chow, Menaldiand Robin [CMR] and probabilistically by Karatzas [Kr2]. The optimal policy is shownto behave as if one possessed an infinite amount of fuel, until the supply is exhausted,then no further control is exercised. This is justified by the fact that in the monotonefollower setting, fuel causes displacement only in one direction, and the displacementcaused by a unit of fuel is the same whenever it is used; hence there is no point in savingfuel. In particular, Karatzas proves that the directional derivative of the optimal costn(x, y) in the 45°-direction is the optimal risk zI of a suitable stopping problem, andif the control k(y) is optimal for the finite-fuel problem with available fuel y, then therandom times &(y) := inf{t > 0 : (y) > 0}, y > 0, are all optimal in the stoppingproblem. The specific selection of the 45°-direction can be understood by observing thata control k pushes the 2-dimensional state process (Xe, Y) in the (—1, —1) direction, sinceX = x + W — k, Y = y — kt; therefore the directional derivative of il in the oppositedirection may be associated with a stopping problem, by analogy with the 1-dimensionalcase.We conclude by recalling a different type of finite-fuel monotone follower problem, inthe class of cheap control problems with a finite horizon. This is considered by Sun andMenaldi [SM]; their model is characterized by a non-linear, degenerate, 2-dimensionalsystem of the formf dX = YdY = g(X,Y)dt + ydW +dkwhere g is non-linear, ‘y > 0, and the increasing control k is limited by a finite sourceof fuel (so none of the variables measures the remaining fuel, in contrast to the previousmodels). The optimal cost il is shown to be the maximum solution of a variationalinequality, in the a.e. sense and in the viscosity sense (see Lions [Lnj for the definitionChapter 1. Introduction 9of viscosity solution), and a solution of a sort of Hamilton-Jacobi-Bellman equation; theuniqueness is still an open problem. For the special case of a linear system (i.e. glinear) and a convex cost rate, and under the assumption that an optimal control for thelimit-free problem (i.e. the problem with unlimited fuel supply) exists, Sun and Menaldi[SM] obtain the same kind of optimal policy for the finite-fuel as we have seen for thestandard 1-dimensional finite-fuel monotone follower problem: behave as if you possessedan infinite amount of fuel until you run out of it, then use no further control.1.2 ObjectiveThe objective of this thesis is to derive, in a constructive way, the existence of the optimalcontrol for a 2-dimensional, monotone follower, cheap control problem. Our assumptionson the cost rate f are similar to the ones in [CMR], [MT].In Chapter 2 we state the control problem and we recall the main properties of thevalue function i. In Chapter 3 we identify the free boundary ÜÄ’ of the region of inactionand the three regions R0, R,, R2 into which the region of action splits. Then we assumethat the region of inaction A’ is of locally finite perimeter (i.e. (LFP)) and we showthat A’ can be replaced by a new region of inaction A’, without affecting the originalcontrol problem. We prove that the new free boundary DA’ is countably 1-rectifiable.All of these properties are deduced from the geometry of the problem. To justify theassumption of (LFP) we show that (LFP) is verified under the conditions of [WCM];moreover, in this case, we prove that the free boundary is of class C2’ and the valuefunction is of class C3 up to the boundary (except perhaps at one single point P0 ofIn Chapter 4 we construct a “potentially” optimal control k by making use of thegeometric properties of deduced in the previous chapter. Then, under the hypothesisChapter 1. Introduction 10that in a small neighbourhood of P0 the boundary satisfies a Lipschitz condition, we provethat k is the unique optimal control of the singular problem in question. We show thatk acts like the “local time” of the optimal process at the boundary OA’; k has constantdirection on each of the two branches O, 02 of 0A1, these directions being parallel to thesides of the wedge R0={i = 0, i = 1, 2}. Finally we give examples in which, near Fo,the required Lipschitz condition is satisfied, and hence k is optimal.Chapter 2Characterization of the Value Function2.1 Statement of the ProblemLet (, F, F) be a probability space, (We, t 0) a standard 2-dimensional Brownianmotion, and (Fe, t 0) a filtration satisfying the usual conditions with respect to W(i.e. F is an increasing, right-continuous family of completed u-subalgebras of F, andW is a martingale with respect to J).We now introduce some notation that will beused throughout this thesis.• x+ :=(xt,4) if x = (x1,2)e 112, with xt := max{0,x2}.• g is a constant vector in 112; o is a constant 2 x 2 matrix (possibly degenerate).• := {(x1,x2)E 112 : x1 0,x2 o}.• A function k with values in 112 is A*nondecreasing if k2— k1 E A* whenevertl<t2.• V is the set of controls k which are progressively measurable random processesfrom 11+ into 112, cadlag, a.s. A*nondecreasing, with k0 E A*.• W’°° := {v : v0, IvvIo, IID2v Io E L00(1R2)}, with i9(x) := (1 + IxI2) i) andp>1.• t(x) := (\ + (x(2)-P-2, 11(x) := ( + IxI2)’, x E 112, with 0 < < 1 and p> 1.11Chapter 2. Characterization of the Value Function 12• L is the set of functions v on JR2 such that ,uv2 is integrable.• W:={v:vEL, IVvIeL}.• G is the set of locally Lipschitz continuous functions v on JR2 such that v(x)IC(1 + xl)”, and IVv(x)l C(1 + IxlY’ a.e. x, for some constant C, with p> 1.The state of the system is described by the following Ito equation:(dX = gdt+ u• dW + dk(2.1)(X = x + k0where x E JR2 is the initial state of the control-free diffusion. Note that sometimes wewill write X(t) instead of X when we want to emphasize the dependence of X on theinitial point x. The cost associated with each initial position x E ]R2 and each controlk V is given by(2.2) J(k) = E {j°° f(X)etdt}where p> 0 is a discount factor, and the cost rate f is a strictly convex, non-negativefunction such thatf(x)—f +oo as lxi — +00.lxiThe value function is(2.3) ‘i%(x) = inf{J(k) : k v+}.Our goal is to construct a process k in V for which ñ(x) = J(k).2.2 Some Properties of the Value FunctionHere we recall some basic properties of the optimal cost ñ obtained already in [CMR]for the 1-dimensional case, and in [MT] for the n-dimensional case. For the cost rate fwe assume coercivity (see (2.4) below) besides hypotheses as in the mentioned papers,Chapter 2. Characterization of the Value Function 13i.e. we assume that there exist p> 1 and constants 0 < r Co, C1,C2 such that for anyA E (0,1), any x E 112 and any x’ such that Ix’I 1,(2.4) rxj’— Co f(x) Co (1+ IxI),(2.5) f(x) - f(x + x’)I c1(i + f(x) + f(x +(2.6) 0 < f(x + Ax’) + f(x — Ax’) — 2f(x) C2A(1 + f(x)),with q (1—2/p)+. (The first part of (2.4) is needed only in the case of cheap controlproblems). An example is f(x) = IxIP with p E IN, p even.Theorem 2.1 There exist constants C0, G1, G2, such that for each A E (0,1) and eachx’ with x’I < 1 the function ii(x) satisfies (2.7)—(2.9) below(2.7) 9xj”— Oo (x) ã0(1 +IxI)”,(2.8) Ii(x) — (x + x’)I O1(1 + xI + x + x’I)’Ix’I,(2.9) 0 i(x + Ax’) + (x — Ax’) — 2i(x) < 2A(1 + IxI)2.Proof: The proof is as in [MT]. Recall that i(x) inf{J(k) : k E V}, so in particularfor k 0 we get from (2.4)n(x) J(0) = E{f f(x + gt + u Wt)etdt}E{jCo(1 +x+gt+u. WtJP)e_Ptdt}+CoE{fIx+gt+u.WtI1)e_)tdt};but g and u are constant, so we have(2.9)’ E{j Ix + yt + a e1tdt}<M(g,a){lxPf e_ptdt + jtpe_Ptdt + j°°tP/2e_Ptdt}M(g,a,p)(Ix + l)P,Chapter 2. Characterization of the Value Function 14and hence(x) + CoM(g,u,p)(x + 1)with M(g, u), M(g, u, p) constants. Therefore, the second inequality in (2.7) follows ifwe put c:0 = max{1,M(g,,p)}.For the proof of the first inequality in (2.7) we follow [WCM]. From the first inequalityin (2.4) we haveJ(k) = E{f f(XiADe_1tdt}E{j (r (XI — c0) edt}for any control k. Also, since k is non-negative,I(X)i (X - k)j = I(XI = I(x + gt + a. W)I IxI - gt + a. WtI.Therefore, for some constants, , Go,u(x) rIxi — o.Now we observe that= inf J(k’)— inf J+(k)k EV+ kEV+= infsup{J(k’) — J+1(k)}k ksup{J(k) — J+1(k)},and similarlyi2(x + x’) — ‘i(x) < sup{J+1(/c) — J(k)}.Chapter 2. Characterization of the Value Function 15Recall that X(t) stands for X with X = x + k0. Hence we haveIi(x) — n(x + x’)Isup J(k)— J+’(k)Ik<sup E{f jf(X(t))- f(X+1(t)) etdt}SUPC1E{j 1 + f(X(t)) - f(X+, (t))’ 1IX(t) - X+x,(t)IePtdt}= SUPC1E{j Ix’I 1 + f(X(t))-f(X+x,(t))Ihletdt}.Now we apply Holder’s inequality to getjü(x) — (x + x’)Iciix’i (j°° e_Ptdt)’sup (E{f°° 1 + f(X(t)) — f(X+,(t)) e_Ptdt})I 00= Ci-j sup (E{f 1 + f(X(t)) - f(X+,(t)) etdt})c1 sup ( + E{j f(X(t))etd + f f(X, (t))e_Ptdt}) ‘‘1g.Because of (2.7) we may consider only those controls k for which(2.10) E{j f(X(t))etdt} O (1+ IxI).So from the last inequality above we have‘1 ‘—i/p- (x + x’)I C, (- + Ô0(1 + IxI) + O0(1 + Ix +C,M(p, 0) (1 + IxI + Ix +and (2.8) follows.Finally we observe that V is a convex set; also the strict convexity of f(x) and thelinearity of the dynamics imply that the payoff functional J(k) is simultaneously convexin (x,k). Therefore, (Ox + (1 — O)x’) OJ(k) + (1— O)J1(k’) for every x,x’,k,k’ andChapter 2. Characterization of the Value Function 160 e [0, 1]. So ñ is convex and the first part of inequality (2.9) follows. To prove thesecond part of (2.9) we start with‘ü(x + Ax’) + fz(x — Ax’) — 2ñ(x)= infinfsup{J+Ai(ki) + J_AI(k2)— 2J(k)},k1 k2 kso we have(x + Ax’) + ft(x — Ax’) — 2ñ(x)sup{J+AI(k) + J_A1(k) — 2J(k)}= SuP E{j (f(X+A,(t)) + f(X1(t)) - 2f(X(t)))etdt}sup E{j (f(Ax’ + X(t)) + f(-Ax’ + X(t)) - 2f(X(t)))etdt}sup E{j C2A(1 + f(X(t)))e’tdt},where again we have used the linearity of the dynamics and the growth properties off(x). If q = 0 then (2.9) follows immediately from the last inequality. If q> 0 then weapply Holder’s inequality to get12(x + Ax’) + i(x — Ax’) — 2ft(x)2/pC2A sup{ (f etdt) (E {f (1 + f(X(t)))et}) }C2Asup{(+ f f(X(t))eutdt)l2?h1}1 1-‘l—2/pC2A—--(_ + Co(1 + IxDjsince we may restrict ourselves only to controls k satisfying (2.10). Then, from the lastinequality above the second part of (2.9) follows. •Since ii is convex, i exists a.e. in ]R2. From the growth conditions (2.7)—(2.9) wecan actually obtain more regularity for Il. This is done by an application of Sobolev’sinequality.Chapter 2. Characterization of the Value Function 17Proposition 2.2 The optimal cost i% is inProof: From (2.7)—(2.9) we deduceo n(x) O0(1 + xI)P,(2.11) I(x)I O(1 + IxI)P1, a.e.,o D21t(x)x’ x’ C2(1 + IxI)(2 a.e., for Ix’I 1,(here Oi is slightly greater than the one in (2.8), as necessary.)We remark that the matrix D2(x) exists a.e. and is symmetric. In fact, since %is convex, the surface (x, n(x)) satisfies Euler’s Theorem a.e., and at any point (q, ñ(q))where Euler’s Theorem holds, one hasi(x) = (q) + (x- q) . V(q) + B(q)(x- q) (x- q) + o(Ix- q12)where B(q) is symmetric and equals D2ü(q) (cf. [Bs]).From (2.11)3 with x’= (e1,2) we have0 n11(x) +2i12(x) + i2(x)e C2(1 + IxI)(2)+, a.e..In particular, for j = 1, 2 = 0, and = 0, 2 = 1 we have0 ñ(x) C2(1 + IxI)(2)+, a.e..Also, for i = = 1, we get12(x) + 22(x) — C2(1 + x)2 2i12(x) 11(x) +ñ22(x), a.e.,that is2In12(x)I 2O(1 + IxI)(2)+, a.e.HenceI2(x)I = Iu21(x) C2(1 + IxI) ‘ ,2ooand u E W0 follows.Chapter 2. Characterization of the Value Function 18Theorem 2.3 There is a version of which is in C’(R2).Proof: From Proposition 2.2 we also have ñ30 E W2’°° since I(8)I (p + 1)3 andI(i)I (p + l)(p + 4)/3d. Now by Sobolev’s embedding theoremw2°°çiR) ‘—. C”(R2),and therefore there is a version of it/3 in C”(]R2). Thus a version of in C’(1R2) isobtained. •In order to characterize the optimal cost ii as the solution of a suitable variationalinequality, we introduce two different kinds of perturbed problems approximating Il byabsolutely continuous controls, and by impulse controls respectively. Only the first classof problems is needed to obtain a variational inequality for , but we will describe bothof them for completeness and also because the connection among these problems is interesting in its own right.Let V be the set of impulse controls k in V, i.e.= {k E V stopping times{8j}jEN with 0 O 0+i,j E IN,(2.12) .F9-measurable random variables {j}3 such thatk(t)=for every t >let Ve be the set of Lipschitz continuous controls k in V with Lipschitz constant boundedby 1/ ( > 0), i.e.(2.13) v = {k E V : k is a.s. Lipschitz continuous with Lipschitz constantalso, we set V0= U V. We now define the approximating problems(2.14) = inf{J(k) : k E V},(2.15) x) = illf{J(k) +E[e6i] : k v}.Chapter 2. Characterization of the Value Function 19Theorem 2.4 (x) = inf{J(k) : k e v0} = inf{J(k) : k e V}.Proof: The proof of this theorem is still based on the density of V0 in V as in [CMR],[MTI, or [MRb]; nevertheless, the arguments of the first and second paper do not apply toour problem because of our infinite horizon and cheap monotone control feature (resp.),while the reasoning of the third paper makes essential use of analytical methods. Here wegive a purely probabilistic proof by borrowing some of the ideas of the previous papers.(i) Fix x e 112. Let k€V be such that J(k) <oo, k = (k(1),k(2)), then we definecontrols k = (ks’), k2)) by(2.16) k(t)=nf (k(s)An)ds, tO,t—1/nwhere we have tacitly set k(1)(t) = k()(O) for all t <0. Then, k(t) is a continuous controlsuch that a.s. as n —* 00,(k(t) —* k(t—) := lim k(t) if t > 0,i sitk(O) = k(0) A n —+since k(t) is cadlag. Therefore, k(t) —* k(t) as n —* oc for all t except for the countableset of points where k(t) k(t—) (as guaranteed by the monotone property of k(t)).Moreover, we have a.s.I -k,(t) <2n2(2.17) dt —( k)(t) is non-decreasing in t,since k(t) is non-decreasing. Thus, k E V0 for each n.To simplify the notation we denote by X and X the trajectories starting at x andassociated with the controls k and k (respectively). Let R > 0, then we define thestopping times(TR = inf{t : IX(t)I R},(2.18)11=inf{t:IX(t)IR}, n€IN,Chapter 2. Characterization of the Value Function 20and we set(2.19) u(R, n) = ‘rR A TR.Now we modify the controls k as follows(k(t) ift<o(R,n),(2.20) k R(t) =I k,(o(R, n)) if t (R, n).Clearly k,ft E V0.Our goal is to show that(2.21) lii 1Iii J(k,R) J(k),R—oo fl—*OOwhich will implyinffr(x) li J(k,) 1Ii 11ii Jx(n,R) J(k),E>O R—*oo R—ooand therefore infñe(x) = n(x), since k E V is arbitrary. Let us denote by X” thetrajectory corresponding to k,a; then,J(k,R) = E {J°° f(flR(t)) etdt}(2.22)f(X(t)) e_Ptdt} + E{J) fR(t))e_Ptdt}Claimi. P{limrR<TR}=0.n—÷ryJAssume not, then let w correspond to a trajectory for which lim TR < T. So theren—*ooexists a sequence n1 —f oc, and a bounded sequence t1 <rj such that4R for i = 1 or 2.We may assume t1 —* t, t < TR by passing to a subsequence if necessary. Then, t1 < TRimplies(2.23) 3R— X)(t1)= urn k(t1)— k(t1)l—+ooChapter 2. Characterization of the Value Function 21for i as above; moreover,(2.24) urn k(t1)— k()(t1) k(t) —1—oosince for a subsequence (which we still denote by) t1, we haveI k()(t) if t1 t,urn k(t1)= ( k()(t—) if ti I t;hence, in general,(2.25) urn k(t1) k(t);l—oo(2.26) k(t—) IIi k((t1— —) = lli[k(t1— —) A ni] urn k(t1).l—oo I 1—oo l—ooTherefore, (2.23) and (2.24) imply3R k()(t) — k(t)(t_),butk()(t) — k(t)(t_) = .x)(t)— x1)(t_)I < 2Ron the set {t <TR}. Such contradiction proves Claim 1.( ço(R,n) ( fTtClaim 2. EjJ f(X(t))e°td J —f EjJ f(X(t))e1d J J(k).0 n—+oo 0From Claim 1 we have u(R, n) —* TR as n — oc, a.s.. Moreover,11[O,u(R,n)) (t)f(X (t))e’t lI[O,(R,n)) (t)C01 + 1X (t) I )Pe_Pt1[o,(R,fl))(t)Co(1 + 8R)Pe_Pt Ce_)t, a.s..Therefore, Claim 2 follows from the dominated convergence theorem since f is continuousand k(t) —f k(t) a.s..Claim 3. E{f f(XR(t))e_Ptdt} —* o as n —* oo, R —* oc.o(R,n)By the strong Markov property of X we haveE{f) f(:R(t))e_Ptdt} = JX(cT(R,n)) (O)e”’ };Chapter 2. Characterization of the Value Function 22now we recall that J(O) o(1 + IxI) for some Oo > 0 (see the proof of Theorem 2.1),thereforeE{Lfl) f(-:R(t))e_Ptdt}E{Co(l + Ix(u(R,n))I)e(’}Oo(1 + 8R)E {e_P0(Rn)}O0(1 + 8R)P{E {e_PTR a(R, n) = TR} P {a(R, n) = TR}+ E {e’) a(R, n) = T’R} . P {TR> r8R}]O0(i + 8R)T’[E {e_PT} + P {TR> TR}].Clearly, in (2.21) it suffices to consider oniy controls k such that J(k) <+00. Hence byinvoking again the strong Markov property and the dominated convergence theorem, weobtainE {e_PT(Xx(rR))} E {j f(X(t))edt} . 0 as R •‘ 00,since TR —* oo a.s.. Therefore it follows from the coercive property (2.7) of i thaturn E {i9X(rR)I1e_1T} = 0. Now we observe that k()(t) 0 (i = 1,2) impliesR—+ooX(t)I IX(t)I + IX(t)— k(t)I; therefore we will conclude thatE {IR)e_19T1} E { IXz(TR)1’ e_PTR} 0, as R cc,if we show(2.27) Urn E{jX(rR) — k(Tft)IPe_mR} = 0.R-*ooSince g, u are constant, (2.9)’ holds and so lim E{IX(t) — k(t)ePt} = 0; hence, byt—oopassing to a subsequence, we have lim E {Ix(t)— k(t)I” e”ti} = 0. [n order to show3—400(2.27) we setY(t) :=X(t)—k(t)=x+gt+o.W,Chapter 2. Characterization of the Value Function 23and we observe that for each j,urn E{IY(TR A tj)Ie_1\ui} = E{IY(tj)I)e_Iti}since a.s., TR —* co as R —* co and the dominated convergence theorem applies becauseof the estimateE{supIY(t)I} c(p,t,a,g)(1 + IxI)ttiwith c(p, t, a, g) constant. Now we apply the Ito formula to IY(tW)e_Pt from TR A t toTR and we havelim SUPE{IY(TR A — IY(TR)Ie_PTR}IR>O= lim supE{JR {_c(p,a)IY(t)I2 — c(p,g)Y(t)P_l +pIY(t)IP]etdt}2°° R>O TftAi3+ E{JR c_Pt Y(t)I2a dW} ,TRAtiwhere c(p,a) and c(p,g) are constants. But E{f (IY(t)V’ehluI)2dt} < c’o (bymeans of an estimate like (2.9)’), hence the expectation of the martingale above equalszero and solim supE{IY(rR A tj)IPe_TRti — IY(TR)IPe_PTh1}2°° R>Olim sup E{f 1[TRAtJ,TR](t)[c1Y(t)IP_2 +c2IY(t)’ +c3IY(t)Ije_Pt dt}3_00R>O 0.lirnE{f 1[tj,)(t)[c1Iy(t)IP2 +c2Y(t)’ +c3IY(t)IP]e_Ptdt}where c1, c2, c3 are positive constants and we have used the fact that TR —* 00 as R —* cc.Now an estimate like (2.9)’ allows us to apply the dominated convergence theorem, andhence (since tj —* cc as j —* cc) we obtainlim E{IY(TR A =3 —+00uniformly in R.Chapter 2. Characterization of the Value Function 2400. We definet< leN, n EN.sup k0() — k0(t), a.s.; so for any T > 0,lENsup Iv(t) — k0(t)I < sup sup k0(.L) — k0(t) —* 0O<t<T O<t<T lEN———— l/nt<(l+1)/nNow we apply Fatou’s lemmaurn E{IY(TR)IPe_PT}R-*oo= urn E{ lim (IYTR A tj)IPe_o)TR\ui)}R—÷oo j—oolim lim E{IY(TR A tj)Ie_PTui}R—*oo.3—oo= lim urn E{IY(TR A tj)I)e_1)Tti}JE:;) R-+oc= Jim E{IY(t,)I)e_)ti} 0,j—+oohence (2.27) is proved, and so we also haveE{Re_PTR}—* 0 as R —* cc.Finally, there exists R such that, for a fixed€> 0 and R R,(2.28) Co (1 + 8R)° E {e_PT1} <EAlso, because of Claim 1, for n sufficiently large, we have(2.29) C0(1 + 8R)”. P {TR > T8R} <;from (2.28), (2.29) we obtainE{J f(X1(t))e_Ptdt} <2E, n > n(R),o(R,n)but is arbitrarily small, so Claim 3 is proved. Now from Claims 1—3, (2.21) follows, andso does the first part of the theorem.(ii). Let k0 C V0 such that J(k0) <(2.30) v(t) = k0(--) ifClearly v C V and Iv(t) — k0(t)I(2.31)Chapter 2. Characterization of the Value Function 25as n —* 00, a.s., since k0 is continuous. Let X and Xr be the trajectories associatedwith the controls i/n and k0, respectively. Let R> 0 and defineTR = inf{t: X(t)I R}, T = inf{t : 1X(t)I R}.As in the proof of (i) we set(2.32) u(R, ii) = TR A r,and we define(v(t) if t<a(R,n),(2.33) in,(t) =I i,(u(R, n)) if t a(R, n),so is an impulse control (i.e., Vn,R e V). Very small changes make the arguments ofthe proof of (i) work even in this case. The analogue of Claim 1 holds, i.e. lim T’R <n—oo‘TR} = 0; the proof is the same with(2.34) k(t) = lim k,(t1— -) lim z4)(t1) ii v(ti) lim k(t1 + -) = k(t),1—*c,o l—+ooand(2.35) lim k(t1)= k,)(t), i = 1,2,1-+ocinstead of (2.26) and (2.25). Then, again we have u(R, n) —* TR as n —* 00, a.s.. So theanalogues of Claim 2 and Claim 3 hold, and as in the proof of (i) we may conclude that(2.36) IE J(i,R) J(k0),R—oo fland from this the second part of the theorem follows.We have implicitly shownCorollary 2.5 (x) = limüe(x),A similar result holds forChapter 2. Characterization of the Value Function 26Corollary 2.6 [[MRf], Theorem 3.1](x) = lim’ae(x), X 1R2Proof: From the definition (2.15) of ‘2 we have(2.37) < = (x) u(x) ü(x).Now let o> 0, then by the definition of infimum and Theorem 2.4 there exists a controlk E V. such thatJ(k) <i(x) + ,therefore(2.38) i(x) <J(k)+E E{e10i} ü(x)+c± E{e91}.So for a — 0 we obtain(2.39) e(X) n(x),since c is arbitrarily small and we can restrict ourselves only to those controls k EV such that E {z e} < oo. Then, (2.37) and (2.39) imply the assertion of theCorollary. •Remark 2.7 The estimates (2.7)—(2.9) hold for , uniformly in for E e (0, 1] (sameproof as for ii). So ‘& is convex. On the other hand, satisfies only (2.7) and (2.8),because of the cost EE{ e°i} associated with any impulse control k E V (i.e., becausedoes not correspond to a cheap control problem). In general, ü is not convex; however,it is possible to show that i is s-convex, i.e.VÜe(X) . (z — x) i(z)—‘ü(x) + e if (z — x) E A*.Remark 2.8 The convergence in Corollaries 2.5 and 2.6 is also uniform on compacta.Chapter 2. Characterization of the Value Function 272.3 A Variational Inequality for the Value FunctionIn this section we make essential use of the convergence —f ft, as—‘ 0, to obtain avariational inequality for the value function ii.We define the second order differential operator A by(2.40) Ày = —tr [*_]— g• Vv + pv,and the mapping M by(2.41) (Mv)(x) = inf{v(x + ) : A*}.Then we have the following standard resultTheorem 2.9 The optimal cost is the maximal solution of the quasi-variational inequality((Au—f)v(u—Mu)——0 a.e. in]R2,(2.42)(u G.Proof: The proof is similar to the one in [MRf], Theorem 3.1, although there f isbounded instead of having polynomial growth. The assertion follows from Corollary 2.6,since each is the maximal solution of((Au—f)V(u—Mu—E)=0 a.e. inlR2,(2.43)u E G,and any solution of (2.42) is also a solution of (2.43). •For fr we recall a classical penalized problem.Theorem 2.10 The optimal cost ñ is the unique solution of the problemlOu— lOu- 2Au +—(——) + —(—-—) = f a.e. in IR,(2.44) E Ox1 E Ox2u e G, E Lj(R2).Chapter 2. Characterization of the Value Function 28Proof: (Existence) The proof is the same as in [MT], Theorem 2, p.225. If a is non-degenerate, then the classical arguments (cf. [FR]) apply and provide a solution, andidentify it with i.On the other hand, if a is degenerate, then we regularize the differential operator Aby introducing a new operator1 026>0;2 Dxthis operator corresponds to a new problem involving a 2 x 4 matrix a3 [a, 61] (witha3 = aa* + 52J non-degenerate) and a 4-dimensional standard Brownian motion W6.If we call ñ the optimal cost of the new problem, then we have- 1/th4\— 1/th4\_(2.45) A3u +—ç——) + —iv———) = f, 6> 0;E0x1 E0X2also, satisfies conditions (2.7)—(2.9) (the proof is the same as in Theorem 2.1), andthe estimates hold uniformly in 6, for 6 small. Therefore, 14, V14 and A14 are locallybounded in x, uniformly in 6, and 14 E W(R2), which is reflexive. So there existsa subsequence 6k — 0 such that Iz14(x)I is locally bounded in x, and 14(x) — v(x),Vi4(x) —* Vv(x) locally uniformly in x, and also A14 —* Av in the sense of distributions.Then, for 6k —f 0 in (2.45) we obtain (2.44) for v. We still have to show that v = . Inorder to do that, we observe that for every control k e V(2.46) E {X(t) — X(t)} = E {[a, 61] . — a• w} C6’’2for some constant C > 0, if X3 and X are the diffusions associated with [a, 61] anda respectively. Now (2.46) and the estimates (2.4), (2.5) imply v = in the limit as* 0.(Uniqueness). The uniqueness of the solution to (2.44) follows from the weak maximumprinciple for degenerate elliptic equations (compare with the proof of Theorem 2 in [MT],p.226).Chapter 2. Characterization of the Value Function 29Assume uE and v are solutions of (2.44). SetIIII = max{i, 2I} for = (1,2) E R2Ao=A-pI;(x,A) = (A+ xI), p> 1,with A to be chosen later and p as in (2.7);W(x) u(x)— v(x);w(x) = W(x)ib(x, A).Then, since e and v satisfy (2.7) (cf. Remark 2.7), certainly urn w(x) = 0; hence ifIxl-*oowe suppose w(x) 0 and, for example, w(x) > 0 at some point x, then there exists x0such thatw(x0) = maxw(x) > 0.Since w Er12oo(1R) (by the same arguments as in Proposition 2.2), Aw is defined a.e..Let N be the set of zero measure where Aw is not defined; let {zn}neN be a sequencesuch that(Zn_*XO asn—*oo,(2.47)I Zn (N)’ for every n E IN;then we haveAow(z) =A0W(Zn)(Zn, A) + W(Zn)AO(Zn, A) — (uu*vW(z)) V(z, A)r 1 Ou — 1 due— 1 thI — 1 dye—1= [(Zn)) _(_-_()) +_(—(z)) -i-—(——(Z)) ]‘J)(zA)— pW(Zn)’IJ(Zn, A) + W(Zn)AOV)(Zn, A)+ (au*w(Zfl)-VW(Zn)). V,A).(2.48)Chapter 2. Characterization of the Value Function 30From the dynamic programming equation follows1 ö — 1 ö — 1 0? — 1 OVE —__Q__(z))— _(__(z)) + + _(—(z))= min{W(zn) - min{Vv(zn)= mm max {Vu(z) — Vve(z)2 49 IIII1/e II?7II1/emax{VW(zn)1[OW +0W += _[(—_(z)) (-.__(z))= 7(Zn) VW(z)with 7 = (71,72) given byIi if->0,7j(Z)= oxt—‘ 0 otherwise.Hence (2.48) and (2.49) implyA0w(z). VW(z)(z, A)— V(z, A)— pW(Zn)+ W(z)[A0(z,A) + V(z,A)](2.50)= 7(Zn) [_W(z)V(z, A) + Vw(zn)](zn, A)— pW(Zn)— gJ*Vw(z) + W(z)6(z, A)(z, A),‘(z, )since from(2.51) o asA+oofollowsAo(z,A) + . V(z,A) = 6(z,A)(z,A)b(z, )with(2.52) sup 15(x,A)I —*0 as A — +oc.Chapter 2. Characterization of the Value Function 31Then we haveAow(z) [_Ly(zn)v(zn,X)— p+ 6(z,)]w(z)(2.53) +[_aa*Z +since for n sufficiently large w(z) 0.But ‘y is bounded and = + is bounded as —* +00, uniformlyin n (since W(xo) 0 and W e C’(1R2)); hence we can choose ; > 0 large enough toguarantee(2.54) +6(zn,)_a*°3). <•Therefore, (2.53) and (2.54) implyAow(z) < —w(z),hence(2.55) Urn Aow(z) —w(x0)<0.n—*ooSince (2.55) holds true for any sequence satisfying (2.47), we conclude thatess-lim Aow(x) <0,but this contradicts the weak maximum principle (cf. [Bn], Theorem 1, p.334).We now take —* 0 and obtain a variational inequality for ii.Theorem 2.11 The optimal cost ii is the maximal solution ofAuf a.e. inlR2;2—0 a.e. inR;(2.56) Ox1 Ox2(Au— f)-.- = 0 a.e. in 1R2G02uL°°uE p’ E locChapter 2. Characterization of the Value Function 32Proof: We first observe that we have uniform boundedness of, IVI, andin L, L, and L respectively because (as we pointed out in Remark 2.7) the estimates(2.7)—(2.9) hold for , uniformly in €, e E (0, 1]. Therefore, there exist o E and asubsequence k —* 0 such that(2.57) —* u0 weakly in(2.58) A11 —* Auo weakly in1 Oñ- 1 Onek_(2.59) AIt— f= ——(——-—) — —(—--—-) is bounded in L2.k OX1 Ek Ox2Thus, we have alsoOf — Oik —(2.60) lim j + (—) j. = 0k—oo Ox1 Ox2i.e.(2.61) urn (—) = 0, i = 1,2.k—oo OxSince the convergence Vi2 —f Vu0 is locally uniform and (.) is a continuous function,we have10u-(2.62) I—} =0, z=1,2,Ox1i.e. (2.56)2 holds for u0. Also, from (2.44) follows A1 f, a.e. in 112; so we can passto the limit over a subsequence Ek to obtain(2.63) Au0 f a.e. in 112i.e. (2.56) holds for u0. Now we assume (xo) > 0 and (x0)> 0 at some x0 E 112then since u0 E we know Vu0 is continuous. Therefore, > 0 (i = 1,2) in aneighbourhood N of x0; again we invoke the locally uniform convergence of VÜ to Vu0and claim—->0 (i = 1,2) in N, fork sufficiently large. So, for such k’s, the equation(2.44) is in fact(2.64) Afi= f a.e in N,Chapter 2. Characterization of the Value Function 33and in the limit as k —* oc, we get(2.65) Aft0= f at x0,but x0 is arbitrary, so (2.56)3 holds for u0. But fr —* ii (cf. Corollary 2.5), hence n0 = ftand (2.56) holds for ft.Finally, we observe that any solution of (2.56) is also a solution of (2.42); therefore,it follows from Theorem 2.9 that ft is the maximal solution of (2.56). •Remark 2.12 For u e G, the inequality n Mu, a.e. in ]R2 (i.e. u(x) u(x + )for all E A*) is equivalent to Vu 0, a.e. in 112 and for all E A*, and thereforeto (2.56)2 (from the definition of A*). So Theorem 2.9 and Theorem 2.11 are essentiallyequivalent, which is not surprising if one realizes that they both characterize the samefunction ft, but by the two different approaches ft — ft and ftE ft, respectively.Chapter 3The Free Boundary3.1 IntroductionIn Section 3.2 we identify the free boundary DA’ of the region of inaction A1 and the threeregions into which the region of action (A splits. Then, in Section 3.3, we assume (LFP)and we study the measure theoretic boundary of A’, OMA1 by means of sets of finiteperimeter and their tangential properties in the measure theoretic sense. In Section 3.4we improve the previous results and we obtain the regularity of a new boundary DA’which is “equivalent” (in the measure theoretic sense) to the free boundary 0A1; in fact,we show that ÔA’ can be replaced by OA’. In Section 3.5 we give conditions under whichthe local finiteness of the perimeter (i.e. (LFP)) of A’ holds. Finally in Section 3.6 weupgrade the regularity of the boundary as well as that of the value function ü under theconditions of Section 3.5.3.2 Identification of the BoundaryIn this section a description of the boundary of the region of inaction is obtained as ageneralization of the fact that in the one dimensional case this boundary is exactly thefirst point of strict increase of the value function %. Such a result was already implicitin [MRbj, Theorem 4.1, although there the proof is quite confusing and seems to containseveral gaps. We reformulate and prove a similar result.34Chapter 3. The Free Boundary 35Theorem 3.1 There exist two functions ‘b2(xi) and ‘bi(x2) such thatI,2(xi,x)= 0 Vx2 &2(xi),(3.1) Vx1EIR: ‘(i2(xi,x) > 0 Vx2 >‘2(x1);I 1(ix) = 0 Vx1 I’i(x2),(3.2) Vx2EIR: c1(rix2) > 0 Vx1 > ‘i(x2);i.e. the functionsb2(x1) and ‘çb1(x2) are defined by() Jb2(xi) = inf{x2 :2(x1,)> 0},(1x2)= inf{xi :ñ1(xi,x2)> 0}.Proof: We recall that E C’(]R2) (cf. Theorem 2.3).Claim 1: Vx2, such that Vx1, i21(x,2)> 0.With i E II, and c> 0 (both to be fixed later) we define the following function,10 ifx1w(x1,x2)=( c(xi —i) if x1Since Aw(xi, x2) = —g1c+ pc(xi—for x1,we can choose c> 0, E IR suchthatAw(xi,x2) f(x1,x2) if x1(This can be done because the assumed polynomial growth condition (2.4) impliesr(xi,x2)jP— C0 f(xi,x2) with C0 and r independent of x2.) Since f 0 thenw is a solution of the quasi-variational inequality (2.42), whose maximal solution is (cf.Theorem 2.9); so we conclude that w i. Therefore, since 0, for every x2 thereexists some point ã i such that ‘i% (ii, x2) > 0; from this we now deduce Claim 1(since ñ is convex).Similarly, we have Vx1 3x2 such that Vx2 : u2(x1,x2) > 0.Claim 2: Vi : x2) remains bounded as x2 —* —oc, (i = 1,2).Chapter 3. The Free Boundary 36In fact, X2(1,•) is non-decreasing (by convexity) and 0 (by Theorem 2.11). Sou2(x1,)—* cc as x2 - —cc would implyi2(,•) +oo, which is impossible. Onthe other hand, since ii is convex and nonnegative we have‘ü(i+h,x2) ñ(i,x2)+h1(i,hIt1(i,x2) for every x2.Therefore,n1(,x2)—* +00 as x2 —* —cc would imply (for h>0) +h,x2)—* +ccas x2 —* —cc, so ü(1 + h,.) +cc (since ‘ü2 0). This is impossible because of thepolynomial growth of .Claim 3: Vx1 2 such that ‘ü2(xi,.) 0.In fact, if not, then we have that such that2(1,x)> 0 for all x2. So one ofthe following two cases occurs:case (a): X2,n —* —cc such that,1(1,x2,fl) > 0;orcase (b): x2 such thatf1(,x2)= 0 Vx2 x2.In case (a), by continuity of I and ña, we have ‘It > 0 and fL2 > 0 on some open setU containing(1,x2fl). Then, by Theorem 2.11, An= f a.e. in U, i.e.(3.4) — tr [*_]— g. V + pn = f a.e. in U.From (3.4) and tr 0 a.e. (since ü is convex) follows(3.5) püf+g•Vü inU,,so(3.6) pn(1,x2 with(1,x2)e U,and in particular(3.7) pn(1,x2,) f(i,x2,) + giii.1(ii,x2,) +92x(ii’x2,).Chapter 3. The Free Boundary 37Because we are assuming f(xi,x2)/I(xi,I —4 +00 as I(xi,x2)I —* +oo, we havef(1,x2) —* +00 as n —* +00, whilen1(,x2)and2(1,x2,fl) stay bounded as—* +00 (by Claim 2). Then, from (3.7) we havePU(X1,X2,n) —f +oo as n —> +00,but n(1,.) is non-decreasing, therefore it must be ñ(1,.) +oo and this contradictsthe polynomial growth of ñ. So case (a) cannot occur.Let us now assume that case (b) holds. Then, because of convexity and i2 0, wehave(3.8) 1(x,2)= 0 for every (x1,2)e (1,2) — A*.Now Claim 1 allows us to define, for x2‘&1(x2)= max{xi :1(xl,x2) = 0} = inf{xi :1(x,z2)> 0}.WedefinetheregionA = int({(xi,x2): x1 1(x2), x}). Fromthedefinitionoffollows that ii,,, 0 in A. Moreover, by assumption i2 0 on the line x1 = also, onthe left of b1, is constant along horizontal line segments and therefore 2 is constantalong horizontal line segments. In conclusion,2(1(x2),x2) =?2(1,x2) > 0 forx2 x2. So by continuity of ii2 we have ‘ix2 0 in U,1 fl A, U1 being a neighbourhoodof DA. Then, 0 and i2 0 in U1, fl A. Now Theorem 2.11 implies that thedynamic programming equation holds in U,1 fl A, i.e. —tr {uu*J— g V + pñ = fa.e. in U,,1 fl A. We know tr {aa*] 0 a.e. (since ñ is convex), so we havepi,x2) f(,x2) + gi1(, x2) +g2u(,x2) if (, x2) E U1 fl A.It follows from the continuity of ñ and the definition of that 1Iii b1(t) 1(x2),tZ2hence(1x2),x2) E OA for any x2 <.2. Therefore, we can take - urn and obtainz—+i,bj (z2)(z’,T2 )EAflU1Chapter 3. The Free Boundary 38(by continuity)(3.9) p2(j(x),x f(ibi(x2),x+g2i(ç&1x),, with x2 <2.Also, i(1,x2)= ui(’bi(x2),x (by the definition of, since (3.8) implies ‘bi(x2)for every x2 2); ‘iL2(i,2) n2(1,x2) (by convexity and x2 x2) finally,i2(i(x), =i2(1,x)(as we observed above). Then, from (3.9) followsp(1,x2) f(bi(x2),x+g22(1,x)f(’i(x2),x —Ig2u(x1,).Using the fact thatf(xi, x2)+00 as I(xi,x2)I — +00we obtain‘z2(1,x2)—* +00 as x2 — —00which is impossible since i(1,x2)decreases as x2 —f —00 (as i2 > 0). Therefore,case (b) cannot occur and Claim 3 is finally proved.Similarly we have Vx2 such that (ii, x2) = 0. Then, the functions in (3.3)are well defined. RWe remark that Menaldi and Robin (cf. [MRb], Theorem 4.1), claimed that thefunctions ‘/‘j are non-increasing. However, their proof is incorrect. We were able to provethis result only under some extra conditions, e.g. ñ2 0.We now define{(x1,x2): x1 b1(x2), b2(xi)},(3.10) R1 := {(x1,x2): x1 &(x),<x2},R2 := {(x1,x2): &1(x2)< xi,x2 ‘&2(xi)};(3.11) 0 := O(R0 U R1 U R2);Chapter 3. The Free Boundary 3900 := OR0 fl 0,(3.12) 1:=OR,nO,02 :=ORfl .We observe thatA’ := (R0 U R, U R2)’ = {(x,,x2): ‘,(x2) < x,,’2(x ) < x}(3.13) = {(x,,x2)21(x,,x)> O,ü2(x ,x)> O}is open since ‘ü, and u,2 are continuous; therefore (R0U R, UR2)’- is the region of inaction(i.e. the region where Aft = f a.e. holds), and 0 = OR0 fl OA’. Notice that A’ 0, asthis follows from Theorem 3.1 and equation (2.7). Also, since f, ft, il are continuousand the “dynamic programming equation” All = f holds a.e. in A’, this last equalitycan be interpreted to hold everywhere in A’ and tr [*] can be taken to be definedeverywhere in A’ by continuity.Lemma 3.2 Let = inf{ft(x,,x2): (x,,x2) E 1R2}. Then(i) R0 = {(x,,x2) : Vft(x,,x2)= O} {(x,,x2) : ll(x,,x2)= cro};(ii) VP€R0, P — A* C R0.Proof: (i) From the definition of follows = 0 in int(Ro). The result now followsby the continuity of ü1 and the convexity of ft.(ii) Let P e R0, then 0 (i = 1,2) impliesVQEPA*.Hence Q e R0 by (i). •Notice that from the definition of /‘j, the convexity of Ii, and the fact that 0follows1V(,,2)ER,: (—oo,E,] x {2} CR,,(3.14)e R2 : {‘} X (—OO,±2] C R2.Chapter 3. The Free Boundary 40Lemma 3.3(i) In R1 one has ê2 = const along horizontal line segments.(ii) In R2 one has i = const along vertical line segments.(iii) In int(Ri) one has i22 = const along a.e. horizontal line segment.(iv) In int(R2) one has = const along a.e. vertical line segment.Proof: Let P= (1,2) E R1, then il = const on (—oo,i] x {2} (by the definition ofso, for every fixed 6> 0, one has(3.15) ñ(•2+6)—(•, I in (—oo,i)( (.2)—(.,2—6) in (—oo,1),since i2,,1 0. It follows that X2(,2) is constant in (—oo,i]. Similarly (ii) follows.The same arguments prove (iii) and (iv), wherever Ii exists. •Lemma 3.4 The function is upper semicontinuous (u.s.c.), i = 1,2.Proof: Let us recall that 1(z) = inf{xi :21(xi,z) > 0} is defined for every z E JR andis finite (cf. Theorem 3.1). Let z e IR, e> 0, then&i(z)—b1(y) >—E—’&(y)for some such that (, z) > 0, and this holds for any y € JR. Now from the continuityof ñ follows (, y) > 0 if I — zI <6, for some 6 > 0. Therefore,if I — zI <6,i.e. is u.s.c.. •Notice that Lemma 3.4 implies that R0 U R is closed (i = 1, 2).Chapter 3. The Free Boundary 41For a function h 1R2 —* JR and a point Po E 1R2, we set(3.16) h(P0—) := urn h(P), h(Po+) := urn h(P),P—Po P—PoPER0 PEA’if these limits exist. Then we have the followingProposition 3.5(i) Do C {(xi,x2) : f(xi,x2)= Pco} fl {(xi,x2) (x1,2)=(ii) P’o = Ai(Fo—) = = f(P0) for every P0 E Do;(iii) OD(R0uR1)ncD.Proof: Let P0 E 0o, then P0 E R0 and so ui(Po) = a, according to Lemma 3.2. Moreover,since ft is constant in R0 and ft,,,, 2 are continuous, we havePao = pft(Po) = Aui(P0-) f(Po).On the other hand, since ft is convex, tr [u*4(Po+)] 0; therefore,f(P0) = Aft(P0+) = —tr [uu*-(Po+)] + pft(Po) pft(Po).Thus,Aft(Po+) = f(P0) = pft(Po)= Po =Now we show that D(Ro U R1) and D(Ro U R2) intersect. Suppose not, then (sinceis a function of x3, i j) there are essentially three cases (see Figure 3.1):(a) P1 = (xf,x) E O(Ro U Ri) such thatf P2 = (x,x) e D(Ro U R) fl ((—oo,xI) x {x}),> b2(z) for some z x.Chapter 3. The Free Boundary 42(b) Q, = (xl, z4) e O(R0 U R) such thatf = (x,.x) E O(Ro U R,) fl ({x} x (_oo,x)),> ,(z) for some z x.VP1 = (x,x) E O(R0 U R,) : O(Ro U J?2) n ((—cc,xi) x {x}) = 0,(c)lVQ,=(y,y)EO(RoUR2):O(RoURl)fl({y}x(_ o,y)) =0.Notice that (c) is the negation of (a) and (b) under the assumption that O(R0 U R,) flO(R0 U R2) = 0, since ‘b, and b2 are functions of different variables. Also, (c) occurs ifand only if O(R0 U R,) is to the left and above O(R0 U R2).Let us assume that (a) holds. Then, Vi(P2) = 0 by (3.10) (recall that R0 U R areclosed); so P2 e R0. Also, i(P2) = ü(P,) (by the definition of iJ,) since x <x ‘,(x).Therefore, we have (P,)=(cf. Lemma 3.2), i.e. P, e R0. Hence (z,a4) e R0 sincez x, and thus ñ(z, a4)= o• But this is impossible because i is no longer constantabove the graph of b2, by the definition of‘/‘2. So (a) cannot occur.Similarly we can show that (b) cannot occur. Therefore D(Ro U R,) must be to theleft and above O(R0 U R2), i.e. (c) must hold. So, since we are assuming 9(R0 U R1) flO(R0 U R) = 0, there must exist a sequence {(x, x) : n e ]N} of points of graph(&,)such that x — —00, —* —oc, and x >‘2(x); i.e. for each n E IN,=0,n2(x,x) >0,asn—*oo.From All= f a.e. in A’ and tr {* 0 follows—g.Vll+puif mA’(recall that ii, f ll, are continuous). Hence, by continuity, the same is true 011graph(,), and so we havef(x,x), for n E IN.Chapter 3. The Free Boundary 43Figure 3.1: Possible cases (a), (b) and (c) occurring in the proof of Proposition 3.5,taking 1, 2 continuous.Chapter 3. The Free Boundary 44Now we recall that f(x’,x2)—* +oo as I(x1,x2)I —* oo and so we get(3.17) —glts(x,x)+p11(x,x)—* +00 as n —* +00.However, 11(x, x) —* +00 as n —+ oo would imply ñ +oo since 12 is convex (and0, i%a2 0) and since x —* —oo. Then 12(x,x) —* +oo is impossible. Similarly,122(x,x) —* +00 as n —* oo cannot hold for that would imply ‘u2 +00 on graph(i&1);in fact, 122(x,.) I (by convexity) and ‘iL2 const along horizontal line segments tothe left of graph(i) (by Lemma 3.3(i)) imply ltx2(&1(.),.) I. Clearly 12x2 +00 ongraph(i&i) contradicts the polynomial growth of u2. Hence (3.17) is false and we mustconclude that 0(R U Ri) fl 0(R U R2) 0.Now let P = (xi,x2) 0(Ro U Ri) fl 0(1?o U R2), thus P E 0(R U R1 U R2), andP e 0(R). Hence P E 0o (ef. (3.12)). •Proposition 3.6 oo is a singleton.Proof: Let B = {P : f(P) <pao}, then B is a strictly convex set such that OB = {P:f(P)= poo}; R0 = {P : 12(P) = o} (cf. Lemma 3.2) is a convex set and so is int(Ro).Also, in int(Ro) one has Po = p11 = Alt f, which implies B fl int(Ro) = 0. Then, thereexists a hyperplane (straight line) separating B and int(Ro), hence OB fl Ro is at most asingleton (since B is strictly convex). Now the assertion of the Proposition follows fromProposition 3.5, (i) and (iii). •Proposition 3.7 If x? := lim ‘1(x2) and x := urn 2(x1), thenX2—+—OO(i) x? and x02 exist and are finite;(ii) 0 = {Po} := {(x?,x°2)} and R0 = 0 — A*.Chapter 3. The Free Boundary 45Proof: Let F= (1,2) E 01 \ 0o, then L’i(), x2 > b2(xi). Consider the linesegment S = {(x1,x2): x1 = x2}. Then ‘z2 is constant on S (since ñ = 0 on Sby the definition of b1). If SflR0 0, then fi = o on S (cf. Lemma 3.2) and so P E R0,which is impossible because 2 > &2(x1). Therefore, Sn R0 = 0, i.e. R0 has to lie belowany horizontal line through 01 \ 00. Similarly R0 must lie to the left of any vertical linethrough 02 \ 00. Since 01 and 02 meet at 00 we have R0 C 0o — A*. Now the assertions ofthe Proposition follow from the definition of R0, Lemma 3.2 and Proposition 3.6. •Lemma 3.8 If x, x° are defined as in Proposition 3.7, then/jis constant on (—oo,x9,i j, i,j = 1,2.Proof: Certainly (x) = ‘b(x) on (—oo, x], i j, since R0 = 0 — A* (see Proposition 3.7).Lemma 3.9 The function is continuous at x3° , i j (i,j = 1,2). Hence, in particular,0 graph(1)fl graph() = oo = {Po}.Proof: Let us fix i = 1, j = 2 for simplicity. By Lemma 3.4, bi is u.s.c.; therefore,if bi(x+) := lim 1(z), we have 1(x+) ‘çb(x). On the other hand, since x =z—xo= tb2(x1) for any x1 E (—oo,x?) (cf. Proposition 3.7 and Lemma 3.8),1(x+) <1(x) would imply x {x} C 0 (cf. (3.12) for the definition of 0w),but this is impossible since 0 is a singleton (cf. Proposition 3.6). So1(x+)=also, ‘‘1(x—) = 1(x) by Lemma 3.8. The last assertion of the Lemma follows fromPropositions 3.5—3.7. •Hence we have established that the functions j cause the plane to split into the fourregions R0, R1, R2 and A1, as shown in Figure 3.2.Chapter 3. The Free Boundary 46Figure 3.2: Sketch of the region of inaction A’ and its complement R0 U R, U R2.Chapter 3. The Free Boundary 47Remark 3.10 We can show that Vf(P0) points into _A*. This is the 2-dimensionalcounterpart of the fact that in the 1-dimensional case f(xo) 0 if {xo} = 0A1. Toprove this Remark we proceed by contradiction. So if Vf(P0) (A”) then, for example,fx (F0) > 0, and by continuity of f1 we may claim f(P) <f(Po) for P in some subset NofPo_A*. (Notice that N has positive measure since f is continuous.) Also, Po_A* = R0.Therefore, from Proposition 3.5 we getf(P) <f(P0 = pa0 = All(Po), P E N;but, at least in int(Ro), Alt is constant and equal to pa0, and we can always chooseN C int(Ro), hence we have Ali(P0—) = All(P) for all P e N; thus, f(P) <Alt(P) forevery P E N. That contradicts the variational inequality All f in a set of nonzeromeasure.- -Theorem 3.11 For the free boundary OA’ one has:(i) 00 = {(x,x)}; Vll(x?,x) = (0,0).(ii)01fl02=0o.(iii) 0A1 = 0 U 02.I (0,ü) on(iv) Vll = 2 therefore Vll 0 on 0A1 \ 0o•(ll,O) on 02\Oo;Proof: (i), (iii) and (iv) follow from the previous results.(ii) Clearly P E Oi fl 02 implies P E R0, and so P E Oo. On the other hand, it is obviousthatO0CO1fl2. •Remark 3.12 Clearly we also have= 0 =in int(Ri),=0 JChapter 3. The Free Boundary 48and= 0= 212 in int(R2).,ux2x = 0 JTherefore there exist functions c, c of x, i = 1, 2, such thatI —a22c(x)—g2c(x)+ p(x,, x2) a.e. in int(R,),(3.18) Au(x,,x2=t —a,,c,,(x,) —g,c,(x,) +pi(x,,x2 a.e. in int(R2),and ?2(x,,x) is constant in the variable x1 in int(R,), and in the variable x2 in int(R2).3.3 A New Approach to RegularityIn control problems some regularity of the free boundary is usually required in orderto construct an optimal policy. In particular, for the problem we are studying, it hasbeen the lack of information about the boundary that prevented several authors fromobtaining an optimal control in a constructive way. (Although Menaldi and Taksar [MT]proved that there is an optimal control and it is unique, the description of this control isstill an open question.)We believe that some regularity of the boundary OA’ is intrinsic to the control problemwhich we are studying, and is essentially a consequence of the polynomial growth propertyof ñ, ñ, which makes them functions of bounded variation, at least locally. Functionsof bounded variation are connected to sets of finite perimeter, and these sets possess anexterior normal, in the sense of metric density on some suitable subset of the topologicalboundary. From such a property one can deduce that this subset of the boundary of aset of locally finite perimeter is, locally, the graph of a Lipschitz function, at least up tolower dimensional sets. In this Section, under the assumption that the region of inaction,A’, is of locally finite perimeter, we define the subset DMA’ of DA’ and we obtain itsregularity. (cf. Section 3.5 for conditions ensuring the local finiteness of the perimeter ofA’.)Chapter 3. The Free Boundary 49This approach could probably be suitable for establishing the regularity of the boundary even when dealing with control problems which allow controls of bounded variation(instead of non-decreasing controls). This seems to be a new device to supply tangentialproperties of the boundary.We start with a few definitions (cf. [Zmj, §5).Definition 3.13 For C C 112 open, a function u E L1(O) whose partial derivatives inthe sense of distributions are measures with finite total variation in 0 is called a functionof bounded variation, i.e. ‘U BV(0). If u BV() for every bounded open set suchthat cl(1) C 0, then we say that u E BVi0(0).Thus, u E BV(0) if there exists a constant C > 0 such that(3.19) fu(x)°) dx <CIIIILo(Q) for i = 1,2 and all E C°(0).If ‘a E BV(0), then its generalized gradient Du is a vector valued measure whose totalvariation is finite and given by(3.20) IIDuW(0) = sup{ f u(x)ö dx e C00(0),(21, forxE0}.Thus we may extend the definition of IIDuII(A) to include cases in which A C 0 is notnecessarily open.Remark 3.14 Clearly, if ‘a E C’(O) then IIDuII(0) = fo IVul dx (this follows from(3.20) after an integration by parts). On the other hand, if ‘a E W”(O), then IIDuII(0) =fo I(u,u2)Idx where u is the weak derivative of ‘a w.r.t. x. Therefore, W”(O) CBV(0) but the two spaces are not equal (an example of a function in BV(0) \W1’(0)is given in Remark 3.17).Chapter 3. The Free Boundary 50Finally, we point out that if 0 C II, then u E BV(Q) (which is defined in a manneranalogous to the 2-dimensional case) has even a more appealing characterization, namely(c) of the following Remark.Remark 3.15 If &: R —* IR is in BV(IR) there are other definitions equivalent to (the1-dimensional analogue of) Definition 3.13 (cf. [Gs], p.26), i.e. the following statementsare equivalent:(a) the derivative of ‘& (in the distributions sense) is a finite measure;(b) i can be approximated in L’ by C°° functions with uniformly bounded variation;(c) there exists a function such that ‘ = a.e. and b has bounded variation (in theusual sense), i.e. V(’çb) = sup{V() : a < b, a,b E IR} < where the variation ofon [a,b], V), is defined as= sup{ (t) - (ti)I : m e N and(3.21)a=to<ti<...<tm=b}.Moreover, if satisfies any of the conditions (a)—(c) above, then the total variationIID’II(1R) in the sense of (the 1-dimensional analogue of) (3.20) is also given by(3.22) IID’II(R) = inf{V() : = a.e.}.Definition 3.16 A Borel set E C 112 is said to have locally finite perimeter if for everybounded open set fl C 112 the characteristic function of E, 11E, is a function of boundedvariation in . Then, the perimeter of E in is defined as(3.23) P(Ej) = IIDEII() <Do,(24) P(E,) := sup{J d(X) dx : E C°(Q),21, forxE1}.Chapter 3. The Free Boundary 51Remark 3.17 If E is a bounded open set with C2 boundary, then E is of finite perimeter and P(E, f) is the arc length of ! fl 9E in the classical sense. In fact, (cf. [Zmj,Remark 5.4.2) by the Gauss-Green theoremL (x)dx = j(1,2)(x) n(x)dH’(x)where n(x) is the outward unit normal to DE at x and H’ is the 1-dimensional Hausdorifmeasure in (It can be shown that H’ agrees with the usual arc length on every1-dimensional C’ submanifold of 1Ri+l, k 0.) Therefore,P(E,) H’(flOE) <00.On the other hand, since E has C2 boundary, n(x) is a C’ vector valued function suchthat In(x)I = 1; hence there exists an extension ñ of n such that ñ é C(1R2;2) andn(x)I 1 for all x e R2. Then, if = ñ with e C(cZ), 1, we haveP(E,c2) jdivdx= f8E’henceP(E, ) sup{f (x) dH’(x) : E C), I 1},i.e.P(E,12) H’(cnaE).SoP(E,) =H’(1flOE) <oc;therefore, 11E E BV(2) but 11E W1”()!Proposition 3.18 The value function and its partial derivatives (i = 1, 2) arefunctions of locally bounded variation, i.e.(3.25) ‘u,u1,2 E BVj0(1R2).Chapter 3. The Free Boundary 52Proof: From the estimates (2.11) in the proof of Proposition 2.2 it follows that(3.26) ñ E W1(R2),i.e.(3.27) zt,u1,u2E VV”(R2)Then, (3.25) follows from(3.28) W1(1R2)c B1’0(1R2),this last assertion being a consequence of jv(x)° dx = _j J&(x) dx and the factthat IIDvII(2)= LIVv(x)I dx if v E W”(1) with !Z open in 1R2. •We now make the following assumption(LFP): The sets {‘u1 = O}, i = 1, 2, are of locally finite perimeter.Remark 3.19 Since E B40(1R2), one can show that { > t} is of locally finiteperimeter for almost all t (cf. [Zm], Theorem 5.4.4, p.231). Then, there exists a countableset Q, dense in IR such that {n > t} is of locally finite perimeter for all t E Q. Let{tk}1 be a sequence in Q such that tk .t 0 as k — oo; so we have1{.>tk} E BV0(1R2)for all k E IN; moreover, 1I{%x.>tk} I {>O} as k —* cc. Therefore,i- Il{=o} as k —* ccand{tk} E BV10(IR2) for all k E INChapter 3. The Free Boundary 53imply= 0}, Q) = sup{J 11{.=o}(x) dx : E C°(Q),I(x)I2<1, forxEc}mSUP{J1{Itk}(x)0dx : E C),1, forxEc}= P({1 tk}JZ)k—.oofor all bounded open sets c 1R2. This shows thaturn P({n t,},f) <00 ‘ (LFP).k—*ooDefinition 3.20 If E C 112 is a Lebesgue measurable set, the measure-theoretic boundary of E is defined by(3.29) OME = {x : D(E,x) > 0} fl {x : D(Ec,x) > 0},where(3.30) D(E, x) = IEflB(x,r)II I being the Lebesgue measure in 112, and B(x, r) being the open ball with center x andradius r. (If llii = urn, we denote their common value by D(E, x).)Remark 3.21 If one says that E is open in the density topology if D(E, x) = 1 for allx E, then it is possible to show that these open sets produce a topology, and OMEis the boundary of E in this topology. (If the exterior of E is the set of all points xsuch that D(E, x) = 0, then OME is the set of all points x which are in neither themeasure-theoretic interior nor exterior of E) (cf. [Zm], Exercise 5.3). Clearly, OME is asubset of the topological boundary OE.Chapter 3. The Free Boundary 54Lemma 3.22 If E C 112 is a Lebesgue measurable set, then the subset of points of OEwhere a tangent exists is contained in the measure-theoretic boundary OME.Proof: Let P E DE be a point where the tangent exists and let n(P, E) denote theoutward unit normal to E at P. Then we have(a) lim 4B(P, r) fl {Q : (Q - P) n(P, E) <0, Q E}I =0r—*O r(b) r) fl {Q (Q - F) n(P, E) >0, Q e E}I =0.Now we define the half-spacesH-(P)= {Q : n(P, E) . (Q — F) <0},H(P)={Q:n(P,E).(Q—F) >0},then,(a)=-B(P, r) fl (E)C fl H(P) = 0,(b) = lim -B(P, r) fl E fl H+(P)I = 0.Therefore, fromB(P, r) fl H(P) = (B(P, r) fl (E)C fl H(P)) u (B(P, r) fl E fl H(F))followsIB(P,r)flEI IB(P,r) flEflH(P)Ilim > hmrr2 — r—O irr2— .IB(P, r) fl H(P)I —‘7rr2 ISimilarly, we obtainurnIB(P, r) fl (E)dI>1irr2and hence• IB(P, r) fl El • IB(P, r) fl (E)’l 1hm = urn = -.r—O irr2 r—*O irr2 2’thus, P E 0MB.Chapter 3. The Free Boundary 55Definition 3.23 A subset A of 1R2 is countably 1-rectifiable if(3.31) A c (0 f(A)) U Awhere H’(A0)= 0 and f3:A3 —* 112, A C 11, is a countable collection of Lipschitz maps(H’ being the 1-dimensional Hausdorif measure on 1R2).Remark 3.24 Notice that f can be assumed to be the restriction to A of a Lipschitzmap f3:]R 112, since a Lipschitz map defined on a subset of 11 can be extended to 11by an application of the Whitney extension theorem (cf. [Zm], Theorem 3.6.2).Remark 3.25 A definition of countably 1-rectifiability equivalent to the one above isobtained as a consequence of the C’-approximation theorem for Lipschitz maps (againbased on the- Whitney extension theorem) and Rademacher’s theorem; i.e. we haveA C 112 is countably 1-rectifiable if(3.32)AcUMUNi=1where H1(N) = 0 and each M is a 1-dimensional embedded C’ submanifold of 112 (H’being the 1-dimensional Hausdorif measure on 112). (See [Zm], Lemma 3.7.2, for theproof of this.)Remark 3.26 A fundamental result of De Giorgi shows that, if E is a set of locally finiteperimeter, then its measure-theoretic boundary OME is “equivalent” to 0E, a subset ofOE having tangential properties in the measure-theoretic sense. In fact, 0E is calledthe reduced boundary of E, and satisfies(a) H’(OME \ 0E) = 0,(b) VP e ThE, a unit vector n(P, E) such that (a) and (b) of Lemma 3.22 hold,(c) ThE is countably 1-rectifiable.Chapter 3. The Free Boundary 56(cf. [Zm], Corollary 5.6.8, Lemma 5.9.5 and Theorem 5.7.3 for the proof of (a)—(c).) Herewe just point out that the proof of part (b) consists in showing that for each x e 0—Eone can find a coneC(x,,v) = {P e R2: (P — x) v > IP — xI},with vertex x and major axis parallel to the unit vector v(x), such that C(x, , v) n(0—E) fl B(x, r) = 0 (for some r > 0), with the vector valued function v(x) uniformlycontinuous on compacta in (0E) fl B(x, r).We recall (cf. (3.10)—(3.12)) that(3.33) A’ = (R0 U R U R2)c = {(x,, x2) : x1 > ,(x2),x2 >2(x,)};(3.34) R0 = {(x,,x2) : x1 &,(x2),x ‘&(x,)} = {n = 0 =(3.35) R, = {(x,,x2) x1 ‘b,(x2),x > ‘b(x,)} = = 0iiX2 > 0};(3.36) R2 = {(x,,x2) : x1 > ‘bi(x2),x ‘b(x,)} = {ñ > 0,ii2 = 0}.Since the sets {I = 0} are assumed to be of locally finite perimeter, the remarks aboveallow us to claim regularity for H’-almost all of OM{ = 0}. Also, we observe that theunion of two sets of locally finite perimeter is a set of locally finite perimeter (cf. [Gs],Remark 1.7, pM). Then, since OA’ = 0, U O (see Theorem 3.11) where 0 is a connectedsubset of 0{i, = 0}, we obtain the followingTheorem 3.27 Assume (LFP). Then(i) the region of inaction A’ is of locally finite perimeter;(ii) DMA’ is countably 1-rectifiable, i.e. DMA’ C U N where• H’(N) = 0 (H’ being the 1-dimensional Hausdorff measure on 1R2),• each M1 is a 1-dimensional embedded C’ submanifold of 1R2.Chapter 3. The Free Boundary 57Now the question is, how “big” is DA’ \ DMA’? In the next section we will show thatit is possible to redefine A’ in order to obtain a new region of inaction A’ such that DA’equals DMA’ at least up to sets of lower dimension.3.4 Regularity of the Free BoundaryIn order to answer the question posed at the end of the previous section we must analyzethe free boundary DA’ in more detail and see how many “nasty” points are in it. In fact,since (as we have seen in Lemma 3.22) all points of DA1 where a tangent vector exists arealso points of the measure-theoretic boundary DMA’, it is natural to start the study ofDA’ \ DMA’ by examining those points of the boundary where the tangent fails to exist.Fortunately we have already obtained a parametric representation of A’ (cf. (3.33)) interms of the functions b,, &2 so that the problem is now reduced to the study of thedifferentiability of b, and I’2. We begin with some properties of ‘/-‘,, b2 which will lead usto the existence of two functions, and 2, differentiable almost everywhere and suchthat /‘ =/,ja.e., i = 1,2.Lemma 3.28 The function ‘j is locally bounded (i = 1,2).Proof: Fix i = 1. Recall that is u.s.c. (cf. Lemma 3.4), so b, is bounded aboveon compacta. Hence it suffices to show that b, is bounded below on any compact setK C [x, +oo) (cf. Lemma 3.8). Assume not, then e K such that urn ‘çb,(y) = —oc;v—*z1yE Kso there exists a sequence Yn — z1 such that y, E K and ‘hI’i(Yn) — —oc as n — oc. Also,since K c [x, +oc) we have{(‘!-‘i(Yn),Yn) : n E IN} cD1.By Theorem 2.11, Ai = f a.e. in A’, but tr [uu*4] 0 a.e. (since Ii is convex), hencepu f + g Vi2 a.e. in A’Chapter 3. The Free Boundary 58and by continuity this inequality can be interpreted to hold everywhere in cl(A’). Inparticular, one has (cf. Theorem 3.11)pii—g2il>f onö1.Therefore,fQtLi(y),y) pit(i/’i(y),y)—g2Ux(/)i(yn),yn),but then, in the limit as n —* 00, the LHS will diverge to +00 since—÷ +00 asI(xi,x2)I —+ +00, while the RHS will remain bounded since(‘i(yn),yn) (‘bi(Zi),Yn) —*i1x2(’I-’i(Yn),Yn) fLx2(’L’i(yn),Zi) =ñ2(bi(zi),zi) if y 1’ z1,‘22 (u/i(y), y,) is non-increasing with n if y J z1as follows from the fact that in R1 both ‘t and ñ2 are constant along horizontal linesegments (cf. Lemma 3.3), and ñ 0. Contradiction! (For‘‘2 the proof is thesame). •Remark 3.29 It is known that the set of points of discontinuity of the first kind of a realfunction iJ’: JR — JR is at most countable. Therefore, and /‘2 have at most a countablenumber of jumps, and to each one of them there corresponds a finite line segment onthe free boundary DA’. In order to guess the behaviour of the optimal process at theboundary of the region of inaction and then be able to construct the optimal control ofthe problem (2.3) we need to know what the boundary looks like. If the functions b,and‘2 have discontinuities of the second kind, then it becomes impossible to picture theshape of the boundary near those points and then discover the optimal strategy. That iswhy we would like to exclude the possibility of points of discontinuity of the second kindfor ‘L’, and 2; as we have been unable to do so, we try to circumvent the problem byChapter 3. The Free Boundary 59showing that the set of all such points has measure zero, hence the region of inaction canbe slightly modified by changing the boundary on a set of measure zero without affectingthe value function i and its properties at all.Throughout the remainder of this section we assume that (LFP) holds. Then, the setR0UR1 (i = 1,2) is a set of locally finite perimeter (cf. (3.34)—(3.36)), hence the gradientof‘ROUR is a vector valued measure whose total variation over any bounded open seto C JR2 is finite (cf. (3.20) and (3.24)) and given byIID1uRII(O) = sup{f ô) dx : j E C°(O),Ij(x)I21, forxEO}.j=1Now we extend this definition to sets of the form JR x ! and f x JR where is a boundedopen set in II, i.e.IIDhhi?ouRi II( x = sup{JR0U1Oi(x)dx : E C°(JR x1)12, forxeJRxc}IID1&uR2I( x R) = sup{J dx: j E C( x H),I(x)I21, forxE1.xJR}(3.37)where is a bounded open set in JR.In general, if E is a set of locally finite perimeter in H2, the total variation JIDIIEII(lRx1) need not be finite as JR x is not bounded. However, the set R0 U R is the closure ofthe subgraph of the function ‘çb (cf. (3.10)) and is locally bounded (cf. Lemma 3.28),so it is then clear that IID1JrouRi II(IR x ) < oc if is bounded and open (the sameconclusion holds for R0 U R2 m’utatis mutandis). This fact is here stated asProposition 3.30 If (LFP) holds and Q is a bounded open set in H, thenChapter 3. The Free Boundary 60(i) IID1IuR1I( R x ) <00;(ii) IID1IuR2I( x IR) <00.Proof: Let be a bounded open set in IR, then it follows from Lemma 3.28 that thereexists T> 0 such that(—oo,—T) x f C ((RouRi)n(lRx ci)) C (—oo,T) xci;therefore,ID1uRiII(R x = IID1&uRiII((—T,T) xci),and this is finite since R0 U R is a set of locally finite perimeter, hence (i) is proved. Theproof of (ii) is the same. •For the sake of completeness we report here some of the properties of the sets havingfinite perimeter in IR x ci.Remark 3.31 It was shown by Miranda that (cf. [MrJ, Teorema 2.1, p.526) if (9 is anopen connected subset of IR and E is a measurable subset of H x (9, such that(3.38) IDlE II(1R x ci) <00for every bounded open set ci, cl(ci) C 0, then E must necessarily satisfy the followingalternatives10 fora.e. x2EO(3.39) lim IIE(X1,X2) = orI.. 1 fora.e. x2O,10 fora.e. x2O(3.40) lim 1E(x1,X2) = or1 1 fora.e. x2EO.In particular, if E satisfies (3.38), (3.39a) and (3.40b), then (cf. [Mr], Teorema2.3, p.531)E L1’0(O) such that if L = {(x1,x2)e H x (9: x1Chapter 3. The Free Boundary 61one has(i) J11E(x1,x2)— 1L(xl,x2)I dx1 e L0(Q)and(ii) fdx2J[I1E(xl,x)— IIL(xl,x2)j dx1 = 0for every bounded open set , cl() C (9.That is, all sets E satisfying (3.38) and { (3.39a), (3.40b) } or { (3.39b), (3.40a) } are“essentially” “equivalent” to sets of the form L, i.e. subgraphs of functions ‘ç& E Lj0(O)(here “equivalent” is in the sense of (i), (ii) above, and “essentially” means that eitherE or its complement E’ is the subgraph of some e Lj0(O)).All the information so far collected allows us to obtain some regularity of the functionsi, ‘çb2 by applying a result of Miranda which connects functions b BV0(R) andsubgraphs L of locally finite perimeter.Definition 3.32 Let 0 be an open set in IR and E BV(0). Then the set {(x,y)x e 0, y = (x)} is called a generalized cartesian curve, and for every bounded open setC 0, cl() C 0, we say the length of the portion of the curve defined on Q is the totalvariation of the vector valued measure (Dç, )) over and we denote it by j/i + IDI2.Hence(3.41) J 1 + ID := suP{L(2(x) + (x)(x)) dx : E C), E II2 i}.(Here A is the Lebesgue measure in IR.)Remark 3.33 As in Remark 3.14, if ç e W”(12) we have (cf. [Gsj, p.160)j 1 + IDI2 = j 1 + II(x)I2 dx.Chapter 3. The Free Boundary 62Theorem 3.34 Under the assumption (LFP), the functions ‘‘i and 2 enjoy the following properties:(3.42) /‘, E BV0(1R), i = 1,2.I IID1JuRiII(1R x= J V/i + D&iI2,(3.43)_________[ !IDu1RouR2I( x II)= f V/i + ID2;(344) f IIDuil 11(11 x = IIDiII(I IIDiu2II( x H) = IID2();J lID’1iRuR, 11(11 x =I IID2&uR2lI( x IR) =for every Borel set C IR.(Here IID1EII(O) is the total variation of the i-th component D2IIE of the vectorvalued measure DuE over the set 0; )() is the Lebesgue measure of in H.)Proof: Because of (LFP) and Proposition 3.30 we can apply Teorema 1.10 in [Mr}, p.525,and obtain (3.42)—(3.45). •In particular, under the assumption (LFP), the previous Theorem provides us withE B40(1R), i = 1,2, but then (cf. Remark 3.15(c)) there exists a function iJ’j of locallybounded variation in the usual sense, such that/ij= a.e. We are going to show(I) the boundary O has 2-dimensional Lebesgue measure zero, i.e. IOZ = 0, i = 1,2;(II) the dynamic programming equation “Ai= f a.e. in A” can be interpreted to holdacross the non-dense part of the region of inaction, i.e. “Au= f a.e. in int(cl(A’))”;(III) the region of action A’ can be redefined as A’ = int(cl(A’)), and there is a uniquechoice of,b,, b2 such that A1 = {(x,,x2) : x1 > ,(x2), >Chapter 3. The Free Boundary 63(IV) H’(OA’ \ 9MA) = 0;(V) DA’ is countably 1-rectifiable (cf Definition 3.23).In order to prove (I) we need to show that the set of points of discontinuity of the 2ndkind of has zero Lebesgue measure, so that (I) follows from Fubini’s theorem andRemark 3.29.Theorem 3.35 If L E BV10(R) is upper semicontinuous and D is the set of all itspoints of discontinuity then= 0.Proof: It suffices to show that the set of points where ‘/‘l[a,b] is discontinuous has measurezero for all finite intervals [a, b]. So let us fix a < b and let b E BV([a, b]); we again set(3.46) D1, = {x e [a, bj : x is a point of discontinuity of }.Assume A(D,) > 0. Also, e BV([a, b]) and Remark 3.15, (c) imply the existenceof a function of bounded variation (i.e. V() < oc) such that & = a.e. in [a,b].Therefore, since b may have at most countably many discontinuities,on [a, b] \ N(3.47) N C [a, b] such that is continuous on [a, b] \ NA(N) = 0.Thus, A(D, \ N) > 0 and we set(3.48) D’ = D \ N;moreover, we may assume the elements of D1, to be points of discontinuity of the 2ndkind, since those of the 1st kind are at most countable (cf. Remark 3.29) and hence maybe assumed to be elements of N.Chapter 3. The Free Boundary 64Let y e D. Let 6 > 0 be fixed and such that 8 < ‘b(y) — lim’b(z). This is possiblesince b is u.s.c. and we are assuming that y is a point of discontinuity of the 2nd kind,so ‘(y) flm’b(z) > lim’/’(z). By continuity, since y e [a,b] \ N, there exists no E INysuch that(3.49) Iz— I <1/no, z e [a, bj \ N = kb(z) — ‘çt’(y) <.On the other hand, since ‘(y) — e> lim’b(z), it follows from (3.49) thatz—*y(3.50) xo E N fl B(y, 1/no) such that (x0) <b(y)— 6.Let m0 E IN be such that B(xo, 1/mo) C B(y, 1/no), then from )(N) = 0 follows(3.51) Vm mo E ([a, b] \ N) fl B(xo, 1/rn),thus {q,} C [a, b] \ N, q —* x1j as m —÷ cc, and one has(3.52) m m0 = q E B(y, 1/no).So (3.49) and (3.52) imply(3.53) m mo = I(qm) — b(y)I <;therefore,(3.54) (y) — € lim b(qm),m—oobut q —* xo as m — cc and ‘çb is u.s.c., so we must also have(3.55) urn b(qm) fim (z) (x).m—÷oo 0Then (3.54) and (3.55) contradict (3.50) and the Theorem is proved.Remark 3.36 The hypothesis that is u.s.c. is absolutely crucial in Theorem 3.35, asshown by the simple example ‘b= 11c, where Q is the set of all rational numbers. In fact,iI e BV(IR) since l is a.e. equal to the function of constant value zero, but D1Q = IR!Chapter 3. The Free Boundary 65Proposition 3.37 Under assumption (LFP), the free boundary OA’ has 2-dimensionalLebesgue measure zero, i.e.(3.56) IOA’l = 01 U 0 = 0.Proof: With the same notation as in Theorem 3.35, we have0 c {(x1,x2)E lix [x,+oo) : x2 E D1,x E [urn i(z),i(x2)j}U{(xi,x2)E lRx [x,+oo) x2 e (D1)c,xi =therefore, Fubini’s theorem implies IOi I = 0 sinceVx2 E (D1)cn [x,+) : ({1x2)}) = 0,andV compact set K C [x, +oo) : )(D,1 fl K) = 0by Lemma 3.4, Theorem 3.34 and Theorem 3.35.Similarly, 1021 = 0 and (3.56) follows. •Now that (I) has been proved, we proceed to explain point (II). We set(3.57) A1 = int(cl(A’)),(3.58) a = 0A1,= OR1 n(3.59)102 =ORn .Lemma 3.38 f 0\ã—_R2nA’, i=1,2,1% Do CChapter 3. The Free Boundary 66Proof: Let us fix i = 1 for simplicity. Then,0,\ã, =(OR,nOA’)\(OR,nDA’)=OR,n(OA’\OA’)= OR1 n ([OA’ fl (cl(A1))c] U [OA’ fl (cl((A1)c))c]),but A’ is open, so (cl((Al)c))c = ((Al)jc = A’ and we have0, \ Oi = [OR, fl OA’ fl (cl(Al))j u [OR, fl OA’ fl A’].Clearly, cl(A’) C cl(A’); on the other hand, since A’ is open, we have A’ = int(A’) Cint(cl(A’)) = A’ and so also cl(A’) C cl(A’). Therefore,cl(A1) = cl(A’),henceOA’ fl (cl(Al))c = OA’ fl (cl(Al))c = 0and we have(3.60) 0, \ ã = OR, n OA’ n A’ = R, fl A’since R, is closed.Finally we recall that Oo = {Po} and R0 = P0 — A* (cf. Proposition 3.7, (ii)), henceVr>0: B(Po,r)flRoI =that is,P0 int(cl(A’)) = A’,but {Po}=OoCOA’,soitmustbePo. •Proposition 3.39 Assume (LFP). Then R has zero Lebesgue density at every point ofO \O, i = 1,2; i.e.(3.61) VP e O \ 0 : D(R, F) = 0, i = 1,2.Chapter 3. The Free Boundary 67Proof: We recall that(3.62) D(R1,F) — lirn IRjflB(P,r)IFix i = 1 for simplicity and let P E O \ a,, then P € A’ (by Lemma 3.38) and A’ isopen, so B(P, r) C A’ for r sufficiently small, henceIR,flB(P,r)I IR,flA’I IO,I,but ‘,I = 0 by Proposition 3.37, thus (3.61) follows. •Corollary 3.40 Assume (LFP). Then the set ÔA’ \ DA’ has empty intersection with themeasure theoretic boundary of A’, OMA’, i.e.(a.6a) OMA C aA’.Proof: This follows immediately from Proposition 3.39 and the fact that all the elementsof 9MA’ are points of positive Lebesgue density for both A’ and its complement (cf.Definition 3.20). •We have shown that 0 \ 6 is a subset of int(cl(A’)) but its 2-dimensional measureis zero. It is then easy to convince ourselves that, after all, 0 \ 0 is a “false” region ofaction, since adding 0 \ 0 to A’ does not affect the dynamic programming equation; i.e.(II) must hold. In fact, from Lemma 3.38 we haveA’ =A’n(R1uR2’)= (OA’ \ aA’) U (A’ n A’),but A’ c int(cl(A’)) = A’, hence(3.64) A’ = A’ u (aA’ \ aA’).Chapter 3. The Free Boundary 68Therefore, (cf. (2.47), (3.56))(3.65) All= f a.e. in A1,(3.66)together with (3.64), imply (II), i.e.(3.67) All= f a.e. in A’.As we are planning to redefine the region of action and work with A’ instead of A’,we want to obtain a representation of A’ similar to the one provided by (3.33) for A’,i.e. we will show(3.68) A’ = {(x,,x2)e 1R2 : x > ,(x2), > (xi)}where ‘i, ‘‘2 can be selected to be u.s.c. (just as b,, ‘b2 were u.s.c.) and of locallybounded variation in the usual sense (instead, b,, ‘çb2 were only elements of BV0(1R)).This is what point (III) is about.We start by showing that,,‘çb2 can be uniquely chosen to be u.s.c. among all thefunctions provided by Remark 3.15, (c).We set(3.69) bv(b) = {b:1R — IR=a.e., V@) <oc Va,b E H, a <b}where the total variation V’@) is defined by (3.21), and i = 1, 2. Let(3.70) {u.s.c.} = {f: H —* H f is upper semicontinuous }.For any e bv() we define g: JR — II by(3.71) (x) = flrn(z), x E JR.Chapter 3. The Free Boundary 69Lemma 3.41 Assume (LFP). If q E bv(), then q E bv(b1)fl {u.s.c.}. Moreover,(3.72) (x)=llm(z), xEIR.Proof: Surely bv(i’1) $ 0 as i/’, E BV10(]R) (ef. Remark 3.15). Let E bv(), then qis continuous a.e. and so(x) = flrn q(z) = lim (z) = (x) for a.e. x e II,i.e.(x) = q(x) = (x) for a.e. x E IR.From (3.71) followsVE> 0 6o > 0 such that 0 < Iz — xI <b = (z) <q(x) + ;then, if z E B(x, 5) there exists 6 such that B(z, 6) C B(x, 6), and one has0<ly—zl < = b(y) (x)+,that is,sup 41(y) (x) + E,O<Iy—zI<hence(z) = llrn(y) (x) +E.So we haveV > 0 o> 0 such that 0 < Iz — xI <6 (z) b(x) + a,therefore(3.73) rn(z) (x),i.e. e {u.s.c.}.Chapter 3. The Free Boundary 70We now show that q is of locally bounded variation; in fact, if a, b E R are points ofcontinuity of such that a <b, then(3.74) V’(qf) V().Let a, b E JR be as above, let a = X0 <Xi <“ <Zn_i <Zn = b such that x—x_1 =i = 1,2,.. . , n. Also, let e > 0, then from= ] 7()follows(i) 6j > 0 such that 0 < I — x <6 = í) (x) +and with 6 < min{6i, 62,... , ö—i,(ii) z2, ..., z_1 such that 0 < Iz — Xj <6 and (z) (x) —So if we set z0 = a, z = b we obtain a new partition a = z0 < Zi < < Zn_i < Zn = bof [a, b] such that(xj)—(z1)I ifi= 1,2,...,n—1,andI(x) — cb(xj)I = I(x) — (x)1 = 0 if j = 0, nsince a, b are points of continuity of , hence ç equals q there. Then,Z I(x) - (x)I{I(x) - (Z)I + I(z) - (Z)I + I(x)-+ qi) — q(Zi_1)I +E + I(z) —i=iChapter 3. The Free Boundary 71therefore, V() + Vj’(), and since € > 0 is arbitrary (3.74) follows. Thus,for every a, b E IR, a< b(if a, b are points of discontinuity of q, then we can always find a’ < a and b’ > b suchthat a’, b’ are points of continuity of , and we have V’() ‘‘() V’() <oc).It remains to show that the reverse inequality of (3.73) holds too so that (3.72) isverified. We observe thatR(z) = (lim (z)) v (lim (z))z—*y—as well asllrn(z)= (fY) vsince q and are of locally bounded variation, hence they admit one-sided limits. Therefore,flrn(y)= (+1fY) v (‘())> (lim (lim cb(z))) V (lim (lim (z)))—y—*-l- z—*y-I- y—x— z—y—= (urn (z)) v (lirn (z))that is(3.75) (y) llrnq(z) =Hence (3.73) and (3.75) imply (3.72), and the Lemma is completely proved. •Proposition 3.42 Assume (LFP). Then, the set{ E bv(&) fl {u.s.c.} : (x) = flrn(z),x IR}is a singleton (i = 1,2).Chapter 3. The Free Boundary 72Proof: Assume not and let,qS E bv(b) fl [u.s.c.} such that (x) =j = 1,2, and suppose Yo E IR such that i(yo) <c2(yo). Since 4 e bv(1), qj admitsone-sided limits and so one has(3.76) = llrn(z) = (lirnj(z)) v (lirnj(z)).Then, there are “essentially” two cases:Case 1. i(Yo) = lim ç1(z), (I)2(yO) = lim 2(z).z—*yo+ Z—yO+Case 2. i(Yo) = urn q1(z), (Yo) = urn 2(z).z—iio+ z—*yo—In case 1, let > 0 be such that i(Yo) + 8 <‘2(yo) — , then1Ii(z)&(yo)I <36i, 62 > 0 such that 0 < z— Yo <6 A 62t I’2() — 4(yo)I <8,i.e.0 < z— Yo < 6 A 62 = 1(z) <i(yo) + 8 < (Yo) — a < (z),but this is impossible since = = /‘ a.e..In case 2 one has(z)=q(z) (z)= ci(Yo) <2(YO)=b2(z)i.e.lim g1(z) < lim 2(z).z—’yo—Then, the same arguments as for case 1 show that it is possible to find 8 > 0 and 6,62 > 0 such thatz<y0,and again we end up contradicting i = c2 = a.e..Proposition 3.42 justifies the following definition.Chapter 3. The Free Boundary 73Definition 3.43 For i = 1, 2 we define as the unique element of the set{ E bv(b) fl {u.s.c.} (x) = flrn(z), x e IR}.Lemma 3.44 Assume (LFP). For every x€IR one has (x) &(x), i = 1,2.Proof: Clearly L’ = a.e., so let Yo E IR be a point where and let us assumethat(3.77) 7i(YO) > &j(yO).Also, we may assume (for example) that(3.78) ?i(YO) = lim (z),z—*yo—since ‘z/ satisfies (3.76). Now let y > 0 be such thatlim i(z) > 7 > ‘j(yo),z—4yo—then6> 0 such that z < Yo, z— oI < = (z) > 7,hence(3.79) inf (z) ‘y > bj(y0).O<yo —z<6On the other hand, from the upper semicontinuity of ‘i/’j follows7>hrnb(Z),and so(3.80) > 0 such that sup ‘&(z) <7.0< Iz—Yo I <t5oThus, (3.79) and (3.80) implysup (z) < inf (z),O<yo—z<SAt5o O<yo—z<SAS0Chapter 3. The Free Boundary 74i.e.O<y0—z 6A=(z)< (z)and this contradicts‘/ = a.e.. Therefore, (3.77) must be false and the Lemma isproved. •Lemma 3.45 Assume (LFP). For every x e IR one has limib(z) x), i = 1,2.Proof: Assume not, then there exists E IR such that(3.81) 1im’,(z) >Z—4Xbut/‘jsatisfies (3.76), so there is no loss of generality if we assume (for example)(3.82)‘j() = 1im(z).Now let y > 0 be such that 1im’b(z) >7 > ‘j(), then from (3.82) follows6i>0suchthat0<z—<b=&(z)<7,hence(3.83) sup (z) <y.O<z—<5iOn the other hand, since lim ‘‘1(z) > 7 we have(3.84) ‘52 > 0 such that inf 1(z) >.0< IzxI <62Therefore, if 6 = 6 A 62, (3.83) and (3.84) implysup (z) < inf b(z),O<z—<6 0<z—x<6i.e.but this is impossible since ?/‘j = a.e. •Chapter 3. The Free Boundary 75Remark 3.46 From Lemma 3.44 and Lemma 3.45 one has(3.85) lim(z) x) = ffiiii(z) fl’b(z) x),z x zfor every x E IR, i = 1,2. In particular, Lemma 3.44 implies that the analogue ofProposition 3.6 holds for b1 and 2, i.e.(3.86) graph(’i) fl graph(2)is a singleton,since b2‘/‘, on (—oc, x] (by Lemma 3.8, Lemma 3.9 and the definition of 4’).Proposition 3.47 Assume (LFP). For the non-dense part of R in A’, Oj \ O, one has01 \ã’ c {(x,,x2) : x1 E (lim ,(z),,(x2)] x >(3.87)02 \ O c {(x,,x2) : x2 E (limb(z),bx, ],x, >Z—*21Proof: Let us recall that 0, \0, = R, flA1 (cf. (3.60)). Let P = (x,,x2) e 0, \O,, thenP E R, and hence (ef. (3.35))(3.88) x1 x2), x2 >2(x,).Also, P e A’ and A’ is open, so there is an open ball B(P, r) c A’, but then (cf.Proposition 3.37)(3.89) IB(P,r) fl R, IA’ fl Ru = IOu \ Oil 0i = 0.Claim: x1> lim ‘i(z).In fact, if not, then x1 < urn b,(z). Thus,Z—’X2(3.90) VE>08>0suchthat inf ‘i(z)>xi—.O<Iz—x21<6On the other hand, since b2 is u.s.c., from x2 > b2(x1) follows(3.91) < r/2 such that sup b2(t) <x2,O<It—xiI<2e0Chapter 3. The Free Boundary 76and from this we have(3.92) ‘I’2(xi —Eo) <x2 and (x1 —Eo,x2)E B(P,r).Now (3.90) with 6 = E implies(3.93) 6o > 0 such that 0 < Iz — x2 <6o ?,bi(z) > x1 —hence (3.92) and (3.93) implyI 1(z) > x1 —0 < z — x2 <6I b(xi — E) < Z,therefore (cf. (3.35))0< z —x2 <60 = (xi —Eo,z) ER1.Also, since (x1— 80, x2) e B(P, r),> 0 such that 0 < z — x2 < ö = (xi — Eo, z) E B(P, r);soforö=61A60we have(3.94) 0 < z — x2 <6 = (x1— 60, z) E R1 fl B(P, r),but then, also,{(t,z) E B(P,r):xi—r<tx1—60,x2<z<x+6} cR1,hence(3.95) IR1flB(F,r)I> (r—6o)6and this is impossible because of (3.89). So the Claim follows. The Claim and (3.88)prove (3.87). (The proof of (3.87)2 is the same.) •Chapter 3. The Free Boundary 77Proposition 3.48 Assume (LFF). If,7r(Oj \ 8) where 7r is the orthogonal projection on the x-axis, thenr(O \ ) n (lim = 0fori #j, i,j = 1,2.Proof: Assume not and take i = 1, j = 2 for simplicity. Let P = (i’, ) E Ui \ i withe (i ‘i(z),i(2)].Z—X2As in the proof of Proposition 3.47 (cf. (3.89)), from P e 0, \ O and 0 \ , = R, fl A’follows that, for some r > 0, B(P, r) C A’ and(3.96) IB(P, r) fl Ru =Thus, we can select ‘y (, — r, + r) such that(3.97) 1R(7,.) = 0 a.e. inB7(P,r),whereB7(P,r) = {z: (7,z) E B(P,r)}.In particular, we may fix‘yE (, —r,1 +r)fl (urn ,(z),i(2) ,z—+x2and we may assume (for example)(3.98) i(2) = lim ,(z),Z+X2since b, satisfies (3.76) and Proposition 3.42. Then, (3.98) and < 1,(2) imply(3.99)Chapter 3. The Free Boundary 78On the other hand, (3.97) implies(3.100) (-y, z) R, for a.e. z e B7(P, r);therefore it must be(3.101) (7,z) E A’ for a.e. z E B7(P,r),since locally R, is the complement of A’. (In fact, since R, fl (R2 \ Do) = 0 and R1 fl(R0 \ {x = x}) = 0, we can always assume B(P, r) fl (R2 U int(Ro)) = 0, with r smallerif necessary). So from (3.101) we deduce(3.102) > i(z), z > ‘2(7) for a.e. z E B7(P,r).But &, by Lemma 3.44, hence (3.99) and (3.102) imply(3.103) for a.e. z é (2 — Thx2)for i = min{r, 6}, and we have a contradiction. •Corollary 3.49 Assume (LFP). For the non-dense part of R, in A’, c9 \ 0, one has(3.104) 0 \ C {(x1,x2): x1 e (,(x2),x], >1 02 \ 2 c {(x1,x2): x2 (x,),,)],x >Proof: This follows from Proposition 3.47 and Proposition 3.48. •We can improve Corollary 3.49. In fact, we now show that the inclusions in (3.104)are equalities.Proposition 3.50 Assume (LFP). The non-dense part of R, in A’, oj \ 0, may becharacterized as follows(3.105) Di \ = {(x1,x2) : E(1x2),,(x2)],x2 >( 02 \ ã = {(x1,x2): x2 (ix),],x, >Chapter 3. The Free Boundary 79Proof: Let P= (1,2) E {(x1,x2) : x1 (i(x2),i(], > 2(xi)}, then >1(2), and let us assume that ‘1(2) = urn i(z) (this is possible because of (3.76)).So we havei > ‘i() = urn i(z) = T1ii i(z) lim i(z) = urn i(z);Z—*X2 Z+X2—therefore, if7 >0 is such that >7> 1(2), then6>0 such that Iz—<6=’l(z)<7<l.So it is possible to find a ball B(P, r) with r < Li—y, r < 6, such that(i) {(i(z),z) : z E (2 — 6,x2 +6)} fl B(P,r) = 0,(ii) B(P, r) fl (R2 U int(Ro)) = 0,(note that (ii) follows from the fact that P R1 and R1 fl (R2 U int(Ro)) = 0 as in theproof of (3.101)). Now (i) and > b1 only on a null set imply( — 6,2 + 6): bi(z) > }) = 0,hence from Fubini’s Theorem, (ii) and the definition of R1 followsB(P,r) fl Ri = 0,so that P is in the non-dense part of R1 in A’, 01 \ 0,. Hence (3.105), follows fromCorollary 3.49. (The proof of (3.105)2 is the same.) •Remark 3.51 We point out that there may be other points Q e Oj such that D(R, Q) =0; these are points where b1 has a cusp and they too have zero Lebesgue density w.r.t.R. The difference between such points and those in O \ 0 is that the latter ones verifya condition even stronger than D(R%, P) = 0, namely (3.89), i.e.(3.106) r > 0 s.t. IR fl B(P, r)I = 0.Chapter 3. The Free Boundary 80We are now ready to characterize the boundary of the new region of action A’, thatis we can finally show point (III).Theorem 3.52 Assume (LFP). The new region of action A’ = int(cl(A’)) is given by(3.107) A1 = {(x,,x2)e : x1 > ,(x2), >with ‘b, and b2 as in Definition 3.43.Proof: It suffices to recall (cf. (3.64)) that2A’=A’U(aA’\aA’)=A’UUa1\ã,1=150 (3.107) follows from (3.33), ?/‘j <,= /,, a.e., Proposition 3.50, and the fact that(3.108) {(x,,x2) : 1(x2) x1 > ,(x2),(x ) x2 >2(x,)} = 0(in fact, x1 b,(x2) and x2 2(x,) imply (x,,x2) E R0 = {(x,x)} — A*, i.e.x e (—oc, x?}, i = 1, 2, but there = ‘j (cf. Remark 3.46)). •We can finally show point (IV) as Theorem 3.52 enables us to identify OA’ precisely.In fact, OA’ is essentially obtained by adding all the finite line segments correspondingto the jumps of /‘j to the graph of , i = 1, 2, j = 1, 2, j i.[a3 ,ao)Definition 3.53 Let {C}1 and be the points of discontinuity of ‘, and ‘b2respectively. Then we set= lim ,(z),(3.109)[i] = iTi ,(z)and similarly‘b2[j= 11mb2(z),(3.110)iL2[j] = hni b2(z)=for every j E IN.Chapter 3. The Free Boundary 81Proposition 3.54 Assume (LFP). Then the new free boundary DA’ is given by= 51 U 52= (graph(lO )) U U [i[],i[i1) x {c})(3.111) 31u(graPh(&[O))uU{} x [2[],2[i])).Proof: This is obvious from Theorem 3.52. •As we saw in Lemma 3.22, all points on the boundary DA’ where a tangent vectorexists belong to the measure-theoretic boundary DMA’. Here we show that this is infact the case for almost every point of DA’. This result is an obvious consequence ofthe rectifiability of the boundaries 0, and 52 and proves point (IV). (Recall that U isrectifiable since R is the subgraph of a function of bounded variation.)Proposition 3.55 Assume (LFP). Then, the topological boundary DA’ and the measure-theoretic boundary DMA’ are the same except for a set of 1-dimensional Hausdorff measure zero, i.e.(3.112) H’(DA’ \ DMA’) = 0.Proof: It suffices to show that there exists a definite tangent to M’ almost everywherewith respect to the 1-dimensional Hausdorif measure H’ in 112 (cf. Lemma 3.22). Butand ‘L’2 are functions of locally bounded variation (in the usual sense), hence 0, and 52are locally rectifiable curves, and the measure H’ coincides with the arc-length s. Also,a result due to Tonelli (cf. [Tn]) guarantees that the classical formula(s’(t))2 = (x’(t))2 + (y’(t))2is valid for every rectifiable curve and a.e. with respect to the parameter t which isarbitrary (if, locally, x = x(t), y = y(t) is a parametric representation of the curve). Inparticular, if we choose the arc-length s as parameter, we obtain(x’(s))2 + (y’(s))2 = 1 s-a.e.Chapter 3. The Free Boundary 82and hence x’(s), y’(s) exist a.e. assuring the existence of a definite tangent almosteverywhere with respect to s.Finally, from Proposition 3.55 and Remarks 3.25 and 3.26 we obtain the regularity ofthe entire boundary of the new region of inaction, i.e. we show the validity of point (V).Theorem 3.56 Assume (LFP). Then the new region of inaction Al is of locally finiteperimeter and its boundary DA’ is countably 1-rectifiable, i.e.(3.113) 0)11 cUMUNwhere H’(N) = 0 and each M1 is a 1-dimensional embedded C1 submanifold of 1R2. •3.5 Finite Perimeter of A’: a Verification of (LFP)In the previous two sections we obtained the regularity of the free boundary arising inthe control problem defined by (2.1) under the assumption (LFP); that is, we assumedthe region of inaction A’ to be of locally finite perimeter. In order to complete thischapter we must show that such an assumption is, after all, reasonable and verifiable.We shall restrict ourselves to the case where the diffusion matrix a is nondegenerate.Such a condition naturally implies the coercivity (see below) of the bilinear form a(u, v)associated with the operator Au of (2.40), and this allows us to show (LFP) by means ofa localization of a result obtained by Brezis and Kinderlehrer [BK] in the framework ofvariational inequalities with obstacles relative to locally coercive vector fields.In addition to the assumptions stated in Section 2.1 and Section 2.2, we now assumethe following(3.114) uo is positive definite;(3.115) f e C2(1R)Chapter 3. The Free Boundary 83(3.116) f and V(f) never vanish simultaneously (i = 1,2).(It should be noticed that we already had f e C”1(1R2)as this follows from the growthconditions (2.4)—(2.6) by using the same arguments as in Proposition 2.2 and Theorem 2.3.) Let W’2() be the closure of C’°(f) in W”2(), for any open set c 1R2.Definition 3.57 [[Fr2], p.15] A bilinear form a(u, v) is said to be coercive on W’2(f) if(3.117) 3v>O such that a(u,u) vIIuIIi,2 for every u E W’2(),where II 111,2 is the norm in 1ITvl2()In particular, we will consider the bilinear form a(u, v) associated with the operatorAu, i.e.(3.118) a(u, v) := J { (uu*)jjuxvx — gjuv + dxi=1for u, v e W1’2(fZ), with fZ open in R2 (to be chosen later). Let us recall a few knownresults. We start with an obvious lemma.Lemma 3.58 [[WCM], Lemma 4.3]. Assume (3.114). Let 2 c 1R2 be an open ball andlet a(u, v) be defined by (3.118). Then, a(u, v) is coercive on W”2().Proof: This follows from p> 0, ou positive definite and f uua dx = 0 for u E W”2(!),i=1,2. •Definition 3.59 Let a be as in (3.118) and let(3.119) 1K(1) := {v e W”2() : v 0 a.e. in }.We say that w is a local solution of the variational inequality(3.120) a(w,v— w) (f,v — w) Vv E 1K(Z),Chapter 3. The Free Boundary 84if wE 1K() and we have(3.121) a(w, ij(v — w)) f fij(v — w) dx Vv E I((), e C(), 0.Theorem 3.60 [[WCM], Theorem 4.5)]. Assume (3.114) and (3.115). Let be an openball such that cl() C S where(3.122) S := {x e ]R2 : > 0,j $ i},then is a local solution of (3.120), i = 1,2.Proof: (sketch) This follows from the fact that i% can be approximated by the uniquesolution ye eC2!L(1R), for every i e (0, 1), of the penalized problem(3.123) Av+ = f a.e. in R2,E i=1where the uniqueness is accomplished among all continuous functions of at most polynomial growth. (Notice that (3.123) is similar to our (2.44) but with the smooth penaltyfunction j5’ replacing the non-smooth (.) of (2.44). This device yields more regularityfor the solution.) Then, the Schauder theory for classical solutions (which applies thanksto (3.114) and (3.115)) shows that yC e C3I4(1R2); hence (3.123) can be locally differentiated, and in the limit as —* 0 one obtains (3.120). (For the detailed proof see thereference above; there it is assumed f € C3, but, in fact, f E C2 is enough.) iTheorem 3.61 [[1+2], problem 5, p.30 and problem 1, p.44; [WCM], Theorem 4.6] Assume (3.114) and (3.115). Let 2 be an open ball such that c1() C S, then(i) il e W2’();(ii) A1 f, ‘Ii 0, (Ai—f)ii = 0 a.e. in .Chapter 3. The Free Boundary 85Proof: (sketch) (This result is proved in the second reference mentioned above.) Letbe an open ball such that cl() C Q C cl(O) C S. The fact that ñ2, is a local solution of(3.120) implies ñr e W1’() for every 1 <p < 00 (cf. [Fr2j, problem 5, p.30) and hencee W2”(l) for every 1 <p < 00(The fact that Lemma 3.58 ensures the coercivity of a(u, v) is used.) This regularity isthen improved by using Theorem 4.1 in [Fr2j, where again the hypothesis (3.114) playsa major role as usual arguments of elliptic regularity are applied (cf. [Fr2], problem 1,p.44). Then (ii), the “complementarity form” of (3.120), follows from Theorem 3.2 in[Fr2] by means of problem 5, p.30 in [Fr2].Clearly (cf. (3.33)—(3.36))S=RuA’, i=1,2,hence Theorem 3.61, (i) implies (for i = 1, 2)(3.124) il Eif is an open ball such that cl() C R U A’.Lemma 3.62 [[Fr2], problem 5, p.30] Assume (3.114) and (3.115). If w is a localsolution of (3.120), thena(7w, v — 7w) j fy(v — 7w) dx(3.125)—(*)..W + (aU*)ijwx.7x.}(v — 7w) dxi,j=1 i,j=1—— 7w) dxfor every v 1K0(!) and 7 E C00(), y = 1 on ‘, 0 1 in (cl(f’) C ), where(3.126) 1K0() := {v W0”2(Z) : v 0 a.e.}.Chapter 3. The Free Boundary 86Proof: Let f’ c c1(1’) c f be fixed; let ‘y e C0(Z) be as in the statement of theLemma. Then, (3.121) implies(3.127) a(w, — 7w)) f f( — w) dxwith ‘ij = 72 andwithvE1[<(),so that i3 E 1K0(). Now we calculatea(7w, — 7w)= J 1 (Ug*)ij(7w)x ( — 7w). dx—g1(w)( — 7w) dx+fP7W(V —7w)dx(3.128)= f (aa*)jj7wx( — 7w) dxi,j= 1— f 9j7W. ( — 7W) dx1=1+ j (au*)jj7Xw( — 7W) dxi,j= 1— f gj7w( — 7w) dx + f p7w( — 7w) dx.On the other hand,a(w,7(— 7w)) = j (JU*)jjWxj{7( — 7w) + 7( — 7w)}dx(3.129)— gjw— 7W) dx + j p7W( — 7W) dx;Chapter 3. The Free Boundary 87hence (3.128) and (3.129) implya(7w, — 7w) = a(w, — 7w))— f 1 (OU*)ijWx 7z ( — 7w) dx— f pw7( — 7w) dx + J — 7w) dxi,j=1—giw — 7w) dx + j p7w( — 7w) dx,i.e.a(7w, i3 — 7w) = a(w, y(i — 7w))—J 1 (gg*)W7( 7w) dxi,j=1— J (*){ + — 7w) dx•(3.130)— J — 7w) dx= a(w, ‘y(i — yw))—J{i (Jg*)7 w + (au*)ij7XwX1}(— 7w) dxi,j=1 i,j=1—f gi7x1w( — 7w)dx.Now from (3.127) and (3.130) we obtain (3.125) for i€1I<() of the form € = 7V withv e 1K(2). Finally, let v be any element in 1K0(), let ‘ suppv, then (3.125) holdsfor v since v = 7V and 1Ko(2) C 1K(). •Remark 3.63 We point out that, if w is a local solution of (3.120), then 7W is theunique solution of (3.125) since aQu, v) is coercive (cf. [Fr2j, Theorem 2.7, p.15).Let us define the bilinear form(3.131) ä(u,v) J (ua)uv dxChapter 3. The Free Boundary 88for ‘u, v E W”2(fl), open in 112; also, we set2Fr := fr7 — (oo*)i7i,j=1(3.132)— + g(Ilj-yi,j=1 i=1Pxr7 r=1,2.Then, if Q is an open ball such that cl(Q) C R U A1 (r = 1,2) and y e C’°() = 1on cl’, 0 1 in l, c1(’) C !, from Theorem 3.60 it follows that w = ii,. satisfies(3.125) which can be restated as(3.133) V—7Uxr) j Er(V — 7ir) dx, Vv €where Iyr e W2’°°(l) (by Theorem 3.61, (i)), and hence Fr e W”°°(1) (by (3.132)).Clearly, the bilinear form a(u, v) is coercive and 7itr E 1[<(1), hence 7’üXr is the uniquesolution of (3.133) (cf. Remark 3.63). Also, if we denote by(3.134) Aw = — (Ju*)1w..i,j= 1whenever the RHS makes sense (in the sense of distributions), then (3.133) may beformulated as(3.135) (A(7),V— 7Ur) f frr( — 7lLxr) dx, Vv E Ko();now we observe that 7u is a Lipschitz function (in fact, 7ñ,. e C1”() by Sobolevembedding theorem), hence we can restrict (3.135) to £(), the convex set of Lipschitzfunctions v satisfying vIOf = 0 and v 0 a.e. in ; i.e.(3.136) (A(7r)v—7r) f frr( — 7xr) dx, Vv E £().The variational inequality (3.136) is now in the setting of the problem studied by Brezisand Kinderlehrer [BK], except for the fact that our Fr e W”°°(), while theirs is inChapter 3. The Free Boundary 89C’(c1(f)). However, it is easy to see that all their estimates still hold in our case sincethey depend only on IFrIIi,oc, (where II is the norm in Wl0o()), hence Brezis andKinderlehrer’s Theorem 4 applies and provides us with(3.137) A(7ir) E BVi0(), r = 1,2.(For the proof of (3.137) the reader is referred to the paper mentioned above; here wejust remark that (3.137) is obtained by approximating y with the solution v of anappropriate penalized problem such that the variation IID(Ave)II is locally bounded by aconstant independent of 8, 0 < 1; hence (3.137) follows from the lower semicontinuityof the variation.)2Since A(7Ir) = A(7xr) +—P7Uxr and 7iXr E C”() (as we observedabove), from (3.137) follows(3.138) A(7ir) E BV0(), r = 1,2;but 1 on ‘, cl(’) C , hence we have(3.139) AILxr e BV10(), r = 1,2.Now (3.139) and fr,. e C’(1R2) (by (3.115)) imply (fr,. — Aitxr) e BV10(). Therefore,from Theorem 3.61, (ii) will followXr—(3.140) 11Rrfl= hra.e. inif we establish the following two facts: hr 0 in ! and Ail,. = 0 a.e. in Rr fl !.Then, (3.140) will imply 11Rrfl1 E BVi0(), i.e. we will obtain the local finiteness of theperimeter of Rr, r = 1,2, and hence that of A’. So we need to show that fe,. 0 in aneighbourhood of the free boundary 0r• This follows from a generalization of Lemma 7.3,p.195 of [Fr2], which makes essential use of the hypotheses (3.114)—(3.116). With ournotation such a result is stated asChapter 3. The Free Boundary 90Theorem 3.64 [[WCM], Theorem 4.8 and Corollary 4.9] Assume (3.114), (3.115) and(3.116). Then(3 141) hr <0 on thefree boundary determinedby Theorem 3.61, (ii), r = 1,2;hence(3.142) hr <0 on Or\O, r = 1,2. •Remark 3.65 Clearly Or \ 00 and the free boundary determined by (ii) of Theorem 3.61coincide; in fact they both are characterized by the function i/’r and the set Sr (cf.(3.122)).Lemma 3.66 Assume (3.114) and (3.115). Then(3.143) = 0 and V’üa,. = 0 on R,. , r = 1,2;(3.144) Ait=0 a.e. flRr, r=1,2.Proof: Since uI E C”1(Sr) (cf. (3.124)) and ii,. attains its minimum value in Rr, (3.143)follows. Now (3.144) is immediate if the boundary Or has zero 2-dimensional Lebesguemeasure; otherwise from the fact that the2XrXjXj exist a.e. and from the properties ofRr (the x-sections of Rr are half-lines, j r) it follows XrXrXr = Xr3r3j = XrZ’jXr = 0a.e. in Or \ Oo; then rjj(P) = 0 for a.e. P in O \ Oo follows by taking the limit of theNewton quotient along a sequence in {x,. = Xr(P)} fl (Or \ Oo) and by recalling that there0 (such a sequence exists for a.e. P if On > 0). •Remark 3.67 As a matter of fact, it is possible to show that On = 0 under the conditions (3.114), (3.115) and (3.116) (cf. [Fr2], Theorem 3.4 and Theorem 3.5, p.155).We are now ready to prove (LFP).Chapter 3. The Free Boundary 91Theorem 3.68 Assume (3.114), (3.115) and (3.116). Then the region of inaction A’is of locally finite perimeter, i.e. (LFP) is verified.Proof: Since f E C(1R2) and f3’r <0, we can cover Or with open balls such that(3.140) holds there, i.e.hr —11RrflfZ = . a.e. in ftJ XThen, 11RV E BVi0(), r = 1,2, (by Theorem 3.64, (3.115) and (3.139)); also OA’ =O U 02, hence the assertion of the Theorem follows. •Clearly all the results of Sections 3.3 and 3.4 hold in the present setting (since (LFP)holds for A’). However, under the assumptions (3.114)—(3.116), Proposition 3.54, canbe considerably improved thanks to the greater regularity of ftr as we show in the nextsection.3.6 Higher Regularity of the BoundaryIn this section we assume (3.114)—(3.116) and we show (with the notation of the previoussections) that the function which defines the boundary 0 of the region of inactionR, is Lipschitz continuous, and therefore = The proof is a generalization of aresult concerning the regularity of the free boundary of a filtration problem (cf. [Fr2],Theorem 6.1, p.l7’7). The arguments of the proof are based on PDE methods (and forthese we need to assume (3.114), (3.115) and (3.116)), and also on the geometry ofthe problem (that is, on the results of Section 3.2). As all the results of Section 3.2can be easily extended to the n-dimensional case, the regularity theorems which we aregoing to obtain hold for the n-dimensional monotone follower problem too. However, forsimplicity and consistency with the rest of this thesis, all the statements and the proofsbelow will be limited to the 2-dimensional case.Chapter 3. The Free Boundary 92We will need the following lemma which provides us with some basic properties ofLet us recall that‘üXr e C”(Sr) (cf. (3.124)).Lemma 3.69 Assume (3.114) and (3.115). Let ! be an open ball in Sr which intersectsOr\Oü, letP e A’fll be such thatdist(P,ôr\Oo) <6, dist(P,O) o >0, letM> 0be such thatI1)jj(r)I M in ,for all i, j. Then(3.145) iixr(P) M6 Gb2,(3.146) Vu(P)I C6,where C = Cfr0, IU2rIICI1()), r = 1, 2.Proof: The proof is essentially that of Lemma 3.2 and Corollary 3.3 in [Fr2], p.155.Recall that= 0, Vñ = 0 on ör \ 0o(by Lemma 3.66). Then (3.145) follows from the Taylor formula (assume 6 < o)(P—Q).e r0 r(P) = xr(Q) + Dj(r)(Q)(P— Q) e + f j Di()(Q +te)dtdr,with Q E Or \ O, e is the unit vector in the direction of QP, and D2 the directionalderivative in the direction e.Now another application of the Taylor formula yields0 r(P + /Ai6’e) = ftr(P) + Dj(i%xr)(P)/7i6’T+j j D(ui)(P+te)dtdTwith e3 in the direction for which D(ñj(P) = IVir(P)I and 6’ < 6 such that/M6’ <. Hence0 Mb2—IVür(P)I6/:+ M2(6’)Chapter 3. The Free Boundary 93i.e.Ixr (p)I 6(1 + M)/Ai/6’,i.e. (3.146) follows.Theorem 3.70 Assume (3.114), f C3(1R2) and (3.116). Then the functions ‘b, andare continuous everywhere and locally Lipschitz away from the corner point a0. Inparticular, the free boundary DA’ is given by(3.147) DA’ = 01 U 02 = (graph(,[0 ))) u (graph(2[0))).Proof: (Compare with [Fr2], Theorem 6.1, p.177.) Fix i = 1 for simplicity and recallthatE C”(c), i = 1,2,for every open ball C A’ (cf. (3.124)); hence(3.148) n e C”(Q),and so also(3.149)Then, from (3.148) and f E C2”() follows(3.150) E C4a(c) for every open ball 2 C A1,by elliptic regularity (cf (3.114) and [GT], Proposition 6.17, p.109). This enables us todifferentiate the Bellman equation once more; i.e. we getç Al—inand f 0 (by convexity) implies(3.151) Aii1, 0 in A’.Chapter 3. The Free Boundary 94Also, 0 (again by convexity) and = 0 on 0, \ 0o (by (3.143)). Now let bean open ball in S, = R, U A’ such that fl D 0, then we apply the strong maximumprinciple (cf. [GT], Theorem 3.5, p.35) toAit1, O infflA1,=O onflD1,0 in Q,E C2a(! n A’) n C°”(cz),and we conclude (p> 0 is used here)(3.152) > 0 in n A’,since if not, then the minimum (which is zero) would be achieved inside ! and thiscontradicts the maximum principle.We recall that f < 0 on D \ Do (by Theorem 3.64); hence, by continuity, f, <0 ina neighbourhood of any point of O \ 0o, and we assume(3.153) f, <0 in ftLet Q = (x?, x) E 9 fl and let R> 0 be such that there exists b> 0 for whichDR := {(x,,x2)E A’ : 1x2—41 <R, xi <b} c Q,cl(D2R) := cl{(x,,x2)E A’ : 1x2—41 <2R, x1 <b} c ftSetw := Ki1IX, + H1X,X — Fi1for K> 0, F> 0, and IHI 1 (with the constants K and F to be chosen later). Thenwe haveAw = Kf11 + Hf,2 — Ff1,Chapter 3. The Free Boundary 95and from this follows, for F sufficiently large (independently of K and H),(3.154) Aw > > 0 in D2R,since f11, fria are bounded in l and (3.153) holds. Also(3.155) w=0 onODRflO,,and (3.152) implies(3.156) w > 0 on ODRI fl {dist(.,Oi) > 6}, for all R < R’ 2R,if K K(6) with 0 < 6 < 1, a small number to be determined later. If we show(3.157) w> 0 on ODR fl {dist(., 0,) 6} fl A’,then by applying the maximum principle to (3.154)—(3.157) we will conclude(3.158) w > 0 in DR.We show (3.157) by contradiction. Assume (3.157) is false and let P E OD fl A’ besuch that(3.159) w(P) <0 and dist(P,O,) 6.Now set‘th(P) = w(P) + vP— P2, P E cl(D2R),with v> 0 small enough to guaranteeA(vIP — I2) > —EforE asin(3.154) (thiscanbedonebecauseof(3.114), i.e. ER2with > 0). Thus,(3.160) A3 > 0 in D2R,(3.161) 0 onOD2RflO,,Chapter 3. The Free Boundary 96and(3.162) Z’ > 0 on 9D2R fl {dist(., a) > 6}as this follows from (3.156). Therefore, if we show(3.163) > 0 on OD2R fl {dist(., 0,) 6} fl A’,then by aplying the strong maximum principle to (3.160)—(3.163) we will get>0 inD,which contradictsand hence we will have proved (3.157). So let us show that (3.163) holds. Let P EOD2R fl {dist(., a,) 5} fl A’, and recall thatC”(Q), = 0 on 0, Vi1 = 0 on 0 \ a,by (3.143) and (3.124); then, by Lemma 3.69 followsf&,1(P) C62 and IV1(P)I C6.Hence1Y(P) = Ki21(P) + Hf12(P) — FI1X1(P) + viP — P12Ki11(P) — HC6 — FC62 + viP — Fl2> K11X(P) — C5(1 + F) + yR2yR2since 6 < 1 and l — 1I R. Then, certainly z(P) > 0 if we choose 6 <c(1 + F) °(3.163) holds, and hence so does (3.157). Thus (3.158) holds true too, i.e.w>0 inDR,Chapter 3. The Free Boundary 97i.e.K11X + Hii12 — FI1X1 > 0 in D;but > 0 in A’, hence we have(3.164) KILX1X, + Hi12 > 0 in DR,which means that i% increases along lines of slope H/K in DR, for any H such thatI Hj 1. However, ñ = 0 on 0, therefore there exists a cone 7+(Q) with vertex Q, andangle 2/3 = 2 arctan(), and with axis parallel to thex1-axis such that7(Q) fl DR C A’.The same holds for any P e 0, fl VQ, VQ being a small neighbourhood of Q, i.e.(3.165) ‘y(P) fl VQ C A’, VP E VQ fl 0.But we do know that R, is a portion of the subgraph of /‘,(x2), hence it must also be(3.166) y(P) fl VQ C R,, VP E Vc fl 0,,if:= {(x,,x2): (—x1,x2)eThus is Lipschitz continuous inr2(VQ) (with r2 being the projection onto thex2-axis);in fact,(3.167) I1(x2) - &1(y2)I max{tan/3,tan(- /3)} = max{1/K,K}for every .x2, Y2 1r2(VQ). •Finally, from the Lipschitz continuity follows even greater regularity by the applicationof classical regularity results (compare with [Fr2], Theorem 6.2, p.179).Theorem 3.71 Assume (3.114), f e C3(1R2), and (3.116). Then, for i = 1, 2, jChapter 3. The Free Boundary 98(i) E C’((x?,+oo)) andñ e C2(A’ u(0\0o));(ii) f, Cm,a({f, <o}) (m € IN, 0 < < 1) = e Cm+’a((x, +oo)),(iii) f is analytic in {f < 0} = ‘j is analytic in (x, +oo),with {Q4,x°)} = Oo.Proof: (i) is a consequence of a fundamental result of Caffarelli, which guarantees C’-regularity of the boundary in a neighbourhood of any point of positive Lebesgue densityfor the coincidence set (cf. [Fr2], Theorem 3.10, p.1132). Our ‘/‘j being locally Lipschitz,all points on 0, \ oo are of positive density for R, hence there exists a C’ representationofb:(3.168) Xr=h(Xs), for{r,s}={1,2}.However, if r i, since is Lipschitz, then certainly= __0, and therefore wecan use the implicit function theorem to invert (3.168) and get(3.169) = j(x) with ‘j e C’ (j i).Finally, (ii) and (iii) follow from (i) by a classical result due to Kinderlehrer and Nirenberg(cf. [Fr2], Theorem 1.1, p.129). •Corollary 3.72 Assume (3.114), f E C3(1R2), and (3.116). Then, for i = 1, 2, j i,(i) e C°((—oo, +oc)) flC2a((x3Q, +oc));(ii) ‘ € C3(A’ U (OA’ \ 0)),with {(X?,X2°)} = Oo.Proof: This follows from f2 E C2, and (i), (ii) of Theorem 3.71, together with theestablished continuity of‘/‘ at the corner point Oo (ef. Lemma 3.9). •Chapter 3. The Free Boundary 99We conclude this chapter by remarking that the local finiteness of the perimeter ofthe region of inaction A’ provides us with a lot of information about its boundary. Thisis a property of geometric character, and has never been used before to study the regionof inaction, A’, of a singular control problem in the setting of the monotone followerproblem. As Sections 3.3 and 3.4 show, this approach turns out to be extremely powerfulin the 2-dimensional case; since, the functions /‘j being functions of a real variable andhaving obtained ‘b e B40(1R), the useful characterization of Remark 3.15,(c) is availablefor ‘‘j. This characterization allows the detailed analysis of the boundary that leads to theregularity. In more than two dimensions there are other characterizations ofwhich, however, are not intuitively easy to apply and work with, but they could still bea useful tool towards the regularity of DA’. This will be the aim of our future work.On the other hand, our Section 3.5 is a generalization of a resul-t of Brezis and Kinderlehrer [BK]; in fact, they were the first to show the finiteness of the perimeter of the “coincidence set” for an “obstacle problem” corresponding to a locally coercive vector field.A few years later, Caffarelli [Cf] studied the regularity of the free boundary of the coincidence set of Brezis and Kinderlehrer’s obstacle problem. Caffarelli proved that the freeboundary is a C’ surface in a neighbourhood of any point of positive Lebesgue densityfor the coincidence set. Caffarelli’s result was then used by Soner and Shreve [SSj to showthe regularity of the free boundary of their two-dimensional, non-degenerate, drift-free,bounded variation singular control problem. In their paper the key idea is to differentiate the Bellman equation in order to obtain an obstacle problem. The solution of thisobstacle problem is a scalar function of the gradient of the value function of the originalproblem, and whose coincidence set is the region of action of the original problem. Thissame idea was used by Williams, Chow and Menaldi [WCM] to study an n-dimensionalsingular problem in the setting of the monotone follower problem. They needed the extra conditions (3.114)—(3.116) on the data in order to differentiate the Bellman equationChapter 3. The Free Boundary 100and reduce the singular problem to an obstacle problem. However, they were unable toprove that the points of the free boundary are points of positive Lebesgue density for thecoincidence set. Perhaps this was due to the lack of knowledge of the shape of the regionsin which the region of action (A splits; in fact, Williams, Chow and Menaldi did notstudy the geometry of the problem. We do that in our Section 3.2 and, as we pointedout at the beginning of Section 3.6, all that was obtained in Section 3.2 generalizes tothe n-dimensional case (without assuming (3.114)—(3.116)).It should be noticed that in a very special 2-dimensional case, i.e. under (3.114),f e C3(1R2), and (3.116) plus the global conditions on f(3 170) JM1 > 0 such that f12 + M1f22 0 in 1R2(M2 > 0 such that f12 + M2f11 0 in 1R2Williams, Chow and Menaldi proved that the points in > 0} have positiveLebesgue density with respect to the coincidence set = 0}, and hence obtained theregularity by applying Caffarelli’s result. However, our Section 3.6 shows that it sufficesto assume (3.114), f e C3(1R2), and (3.116) in order to obtain a value function i soregular that its corresponding free boundary is, in fact, of class C2,. Again, this resultgeneralizes to the n-dimensional case (as does Section 3.2).In conclusion, there is essentially no problem once we assume (3.114), f e C3(1R2), and(3.116) (except perhaps at the corner point Oo); but under the more general hypothesesof Sections 2.1 and 2.2 the free boundary could be really “nasty”, and hence difficult tostudy (PDE methods no longer being applicable because of degeneracy and less regularityof the cost rate f). That is why (LFP) seems to be the right way to study 0A1 andanalyze its “bad” points (essentially those points which are not in DMA1). We believethat (LFP) is a property intrinsic to the kind of singular control problem studied in thisthesis (perhaps a consequence of the convexity and growth conditions), and although ourverification of (LFP) assumes (3.114)—(3.116) (cf. Section 3.5), we feel that it should beChapter 3. The Free Boundary 101possible to show (LFP) under the more general conditions of Sections 2.1 and 2.2. Weleave this for future work.Chapter 4The Optimal Control4.1 IntroductionThroughout this Chapter we assume (LFP). In Section 4.2 we deduce, from the geometryof the new region of inaction Al, the properties that a control k would be required topossess in order to be optimal. In Section 4.3 we construct a “potentially optimal” controlk by making an essential use of the characterization of A’ in terms of the functions /‘(cf. (3.107)). ic is obtained as the solution of a fixed point problem; the existence of thissolution is guaranteed if one assumes that the condition (LIP) below holds in a smallneighbourhood of the corner point Po (i.e., the only point of intersection of graph(b,)and graph(’2), and also the only point of cl(A’) where V = 0). In particular, k is alsoadapted and uniquely determined if (LIP) holds true. Then, in Section 4.4 we assume(LIP) and we show that k is the unique optimal control of the original problem (2.3);this is done by means of a generalization of Krylov’s proof of Ito’s formula for Sobolevfunctions. Finally in Section 4.5 we give conditions under which (LIP) holds.4.2 The Optimal Control: a Heuristic DescriptionWe recall that in Chapter 3 it was shown that the region of inaction A’ can be redefined(cf. (3.57)) asA’ = int(cl(A’)),102Chapter 4. The Optimal Control 103so that the dynamic programming equation holds a.e. in A’ (cf. (3.67)), i.e.= f a.e. in A’.Moreover,OA’=ã,uã2, ã,nã2=00withO c OR,(cf. (3.57)—(3.59) and Lemma 3.38), hence from Theorem 3.11,(iv) followsonO,\Oo,(4.1) V 0 onO0,on 02 \ Do,if e, := (1,0) and ë2 := (0, 1). Finally we recall that (cf. Proposition 3.54)= graph( q+) u [i[},i[r]) Xfoij, i,j=1,2,and(r ifil,TEN.1r ifi=2Therefore, since 0, C OR2, one has (by definition (3.3) of ‘/‘j and Definition 3.43 of ‘/‘j,i = 1,2)1(x1,X2) = ü(i[r],(r) V(x,,x2)e [,[r],,[r]) X(4.2)‘ii(xj,x2)= ir,’2[rJ) V(xi,x2)E {.} x [&2[L],’2[r]),for every r E IN.All the information so far collected suggests that optimality could be achieved in(2.3) by a control k that keeps the controlled process within cl(A’) after time zero; (ifXo e (Al)c, an instantaneous jump of the controlled process to a suitable point of theboundary would be required). More precisely a “potentially optimal” control couldbe a control k := (k, k?) E V such thatChapter 4. The Optimal Control 104(i) k =[2(xg) — X];(ii) k increases only at the boundary ôj with dk±V(X) for all t such that X E O;—- if ii e [i[t],i[r]) and X?_ 13r,(iii) Lk = for t > 0;0 otherwise,(iv) E {f°° f(t)e_Ptdt} <00,where zk = k — k_, r IN, {i,j} = {1, 2} and JC = (‘, -) is the controlledprocess starting from x E JR2 at time t = 0 and corresponding to the control k. In fact,heuristically, if Ii were C2 then by applying the Ito formula for semimartingales (cf. [My],p. 278) to (t)ePt we would obtain(x) = E{j Aii(X3)e8ds+(4.3)— j eVui(X8) (ëdk +ë2dk)- e [i3)- ü() - Vi3). (ëiLk +ë2Lk)] }.O<stThen, (iii) and (4.2) imply ‘(X8)—ñ(.) = 0; on the other hand, (iii) and (4.1) implyV’â(3) L (ë1zk +ë2/.k) at each jump time s; hence the summation above is zero.Also, k increases only at the boundary ô, and there it is orthogonal to Vñ (by (ii)),therefore the last integral above is zero. So (4.3) reduces to(4.4) i(x) = E {jt Aü(X3)e’8ds+Moreover, since g and a are constant, for the process with no control (i.e. X — k, t 0)we have (cf. (2.9)’)E {j°° — ktI1)e_1tdt} <cc,Chapter 4. The Optimal Control 105hencelimE{IXt — ktI’e = 0.t—*ooAlso, (iv) impliesurn E {f(it)e} = 0,t—*oohence from the coercive property (2.4) followslim E {rIIPe_1t} = 0.t-÷ooNow, some elementary computations show that (since k 0, i = 1, 2)IXtI jXj+jX—kj,so we havelim E = 0.t-ooThus, from the growth condition (2.7)2 followslim E {i(t)e_Pt} lim E {c(i + IXtI)e} = 0,t—*oo(0 < C = const), hence urn E )e_Pt } = 0 along a subsequence and from (4.4)we obtainn(x) = lirn E{jtnAfi(Xs)e_1’sds}.From the properties of the control k follows that X3 never leaves cl(A’); therefore,= limE {j f(Xs)e8ds}since i E C2 implies that Au= f can be taken to hold in cl(A’). Thus, (since f 0)n(x)=urn E{ft }Chapter 4. The Optimal Control 106hence the monotone convergence theorem implies(4.5) ñ(x) = E {f°° f(t)eds} = J(k),i.e., k is optimalIn the next section we will use these intuitive arguments to construct rigorously theoptimal control process.4.3 Construction of The Optimal Control Process.Motivated by the heuristic arguments of the previous section we now attempt the construction of a control process k which is a reasonably good candidate for optimality, asthe construction itself will show.Without loss of generality, from now on we will assumeoo = (0,0).Let P = (x1,x2) e 1R2 and let X = (Xi, X) be the (uncontrolled) diffusion processstarting from P at time t = 0. We set• := cl(int(R)), i = 1,2;• e := (1,0) andë2 :=(0,1).Let us start by observing that, at least when the functions ‘i/’j are non-increasing, a controlprocess k could, potentially, keep the “optimal” process inside the closure of the regionof inaction Al if, for every t > 0, k were defined to be the smallest positive number suchthat the entire path of X3 + kej up to time t is contained in cl(((Ro U E) — Ke)c), fori,j such that {i,j} = {1,2}. We set• X:=X+k’ëi+kë2 t>0,Chapter 4. The Optimal Control 107and we proceed to give an explicit representation of k. We introduce the followingstopping time(4.6) T’ := T’ A T,’ A T,where, for i = 1,2,1inf{t>0:Xt(w)Ei\Oo}, ifPEint(A’U),ji,(4.7) T (Lb’) :=0, if PER0UR;(4.8) T(w) inf{t >0: X(w) e o}, if P R0,10, ifPER0;with the usual convention that inf 0 = +oc.It is clear that k represents the least intensity of the pushing in the direction ëneeded in order to keep the process X inside cl(A’). Hence, at least up to time r, bothk and k? are well defined; in fact,k’ =0=k Vt<r’, ifPEA’;(4.9)= [‘i(x) —if pg A’k = [2X1 — X2]At time r1 the process X leaves For w e {r’ = T} it is not clear at all in whatdirection X should be pushed in order to force the “optimal” process X to stay insidecl(A’). However, when w e {T’= ‘}, i 0, then certainly (with {i,j} = {1,2})((w) = max[(X(w)) — X(w)] + Iq(w)(4.10) O<s<t(k(w) = k(),for all t T’(w),t < T2(LJ), with r2() defined by(4.11) T2(w) := r(w) A T(),(4.12) r,) := inf{t T’(W) : Xt(cü) + it(W) E ij \ Oo},Chapter 4. The Optimal Control 108(4.13) r(w) := inf{t r’(w) : X(w) + lt(W) e Oo}.Then, for ca.’€{T’ = r1} fl {r2 = r} and t r2(w) we have1(’) =(4.14)< +[ k(c) — k(i) = r2(,)<st [& ((X(w) +k2()(w)) — (x3i(w) + k(w))]for all t < r3(w), withr3(w) defined by(4.15) T3(w) := w) A r(w),(4.16) r(w) := inf{t r2(w) : X(w) + k(w) E Ij \ a},(4.17) r(w) := inf{t r2(w) : X(w) + t(W) EIt is then clear that we may proceed by induction and uniquely define k and k? at leastup to the first time X reaches the corner point (0,0). It is also clear that, up to thattime, (ks’, £) is a solution of the fixed point problem—= [ (x + — (x +(4.18) > t T, E IN,—=[2 (x + — (x + k2)] +with10 ifPéA1,k1_ = (k0 ifP’A’.To see this let = r (for example), then J( i for all t < T72, hence(x? + < X’ + k for all r’ t < r’1 and k = k. Thus, (4.18) holds, i.e.—= { (x + — (x + = 0 v r t T?1.Chapter 4. The Optimal Control 109On the other hand,. since r’— — = 2 (x + — (x + —)=2 [ r] - (x + if + k E {} x [2 [LI , [r I)( 0 otherwise;hence the process X + k is kept inside cl(A’) if satisfiesmax VT<t<r1T <s<ti.e., (4.18)2 holds (since k = k). (Notice that in (4.18) we have k instead of k totake into account the jumps of k, which occur in correspondence to the jumps of ‘& and‘2.) The oniy (but major!) question left is, “what happens when X reaches (0,0) ?“.Perhaps one should now expect Jc and k? to increase simultaneously, but how ? Byanalogy with the previous situations we are induced to think that if X = (0,0), thenk and k? could possibly be obtained as a solution of(4.19)=rax [, (x + — (x + )]+ +for i j and t E [ro,T], where= (x + k:0,x0 + = (0, 0)and T > 0 is any time. Notice that in (4.19) we have k instead of k_; in fact, k = 14since b1 and are continuous at zero (cf. Lemma 3.9 and the definition of çb). Then,the problem reduces to searching for a solution of= maxO<s<t(4.20) — -h = [2O’’ + h) — y2]+Chapter 4. The Optimal Control 110in the set of all processes (h1,h2) such that h 0, h non-decreasing a.s., h = 0 a.s.,with (Y’,2)being the diffusion starting at the corner point (0,0) at time t = 0. (Noticethat ‘4b(0) = 0 impliesF Tjvi 1i xri1—[Y1O ‘0) — 1] —and hence(4.21) ma [.(yi + h) — yi] + = [yi + h) —However, it seems convenient not to siipress “+“ in (4.20) as we are planning to unify(4.18) and (4.20) into a unique formula.)It is clear that solving (4.20) is not an easy task. Also, the existence of a solution isnot guaranteed, especially if and ‘4&2 behave very badly near the corner point. In thissection we will show that (4.20) admits a solution if we assume either of the followingconditions: for some a >0 and for i = 1,2,(LIP) : ‘/‘j is Lipschitz continuous in [—a, a] with Lipschitz constant £ip(1)< 1;(NIN) /‘j is non-increasing and continuous in [—a, a].Note that the non-increasing property of is the condition supposedly established in[MRb].Recall that, for simplicity, we are assuming Oo = {Po} = {(0, 0)}. Of course, both(LIP) and (NIN) are always satisfied in [—a, 0], since ‘/‘j is constant there (cf. Lemma 3.8)and hence jj = in [—a, 0].From now on we will always assume that either (LIP) or (NIN) holds true for Let1(x2)= maxi(y),2(x1)= max2(y),yx1Chapter 4. The Optimal Control 111then graph(coll(0))lies above graph(2l(O)) and both graphs lie within the firstquadrant. Moreover,A := {(x1,x2)E 0o + A* : x1 >1(x2), > Y2(Xl)} 0,as this follows from the definition of cpj and (3.86). The characterization of Al given by(3.107) implies A C A’. Now we choose•=(1,f2)e A° such that• R> 0 such that P±Rë, ±Rë2 E Afl[(O,c) x (0,)j;notice that this choice of P and R impliesfor {i,j} = {1,2}. Finally we set• L31= { (x1,2) : lxii <R, i =• t3p = {(x1,x2): lxii P, i = 1,2};•rR(w):=inf{t>O:Yt(w)l3R}, wEe;• mh(6) := sup jh(t’)— h(t”)i for h e C[0,T] and 6 > 0;It’—t”Ib-- ( 12) if t<TR;•=(‘2)iftTR.We recall P. Levy’s result about the modulus of continuity of the Brownian paths on[0, 1] (cf. [KS2I, p. 114); that is,i-—-- mw(6) 1Pihm___________= ii = 1.LL0 /1og(1/6) .1Chapter 4. The Optimal Control.112Therefore, for a.e. w, our diffusion Y gt + o- W has modulus of continuity given bymy()(6) <th(6), 6 <6(w) (some 6(w) > 0) where(4.22) ffi(6) := c + /6log(1/6)],where c is some constant. Let 6 > 0 be such that 6, . 0 as n —* cc, and set• r(w) := inf{t > 0: sup [mYI[0 ()(6) — 1h(6)] o}, for n E IN;• r(w) := TR(W) A r(w), for n E IN, w ENotice that r is a stopping time. Also, T TR A T a.s. because of P. Levy’s result. Let• i(n,w) := the process w) stopped at r;f m,. (2th(6)) + ii(6) if (NIN) holds for j,•‘2(6):=çI 2Cip(’i/’)th(6) + th(6) if (LIP) holds for ‘j,1 0 if (NIN) holds for j,:= £ip()th(6) ± th(6) if (LIP) holds for j,1 —for i=1,2,and6>O;• C(6) := max{Jj(6),.j(6) : i = 1,2} for 6> 0;• N : {(h1,h2) E (C[0,t])2: (i) h 0,14 = 0;(ii) h is non-decreasing;(iii) mhl(6) C(6);(iv) (h,h) E t3p Vs E [0,t];for i,j = 1,2 and for all 6 Efor n e IN, t> 0, and write N for N’ where T> 0 is fixed (but arbitrary);• ir(h’,h2):= h for (h’,h2) E (C[0,T])2 and i = 1,2;Chapter 4. The Optimal Control 113• I c is such that Y(w) is continuous for w 0 I.Proposition 4.1 Let (C[O, T])2 be endowed with the product topology induced by thesup-norm topology of C[O, T]. Then for each n E IN, N is a compact convex subset of(C[o, T])2.Proof: The set r(N) of elements of C[O, T] is equicontinuous by the definition ofN since C(6) < C(6) for 6 6,. Also, r(N) is bounded, as follows from (iv) of thedefinition of N.It remains only to check that ir(N) is closed in C[O,T]. Let hr E 7r(N) andh e C[O, T] be such that hr —* h in the sup-norm. Then, clearly h satisfies (i) and (ii) ofthe definition of N. Also, (iii) holds for h since mhr(ö) < C(6) for every 0 < 6 < 6, andr E IN, and since m.(t5) is a continuous function in C[0, Tj. Finally, (iv) follows triviallyfrom hr — h as r —> cc. Hence h e ir (Na). Thus ir(N) is compact in C[0, T], j = 1,2.Hence N is compact in (C[0,Tj)2,since N = ir1(N) x ir2(N).The convexity of N is clear from the definition of N. •Definition 4.2 For every n E IN, h e C[0, T] and w E we set(4.23) T(,t)(h) := max [ (2() + h8) —O<s<t(4.24) T(w,t)(h) := max [2 (‘(n,w) + h3)O<s<tfortE [0,T].Remark 4.3 If h E ir(N) for some i E {1, 2}, then h satisfies (iv) of the definition ofN. Therefore, from the definitions of Yt(n, w) and TR(w) followsN’1( c) + hIc[QTI V 72(n, w) + hc[oT] R + (P’ V P2)Chapter 4. The Optimal Control 114Hence if ‘/‘j is continuous in [—a, , then the functions E [O,T] I.’ [(;(n,) + h3)-(with {r,j} = {1,2}) is a.e. continuous since (1(n,w),i2(n,w)) is so. Therefore,(4.25) T(c4., .)(h) E C[O,T] for a.e. w eDefinition 4.4 For every n E IN, (h1,2)E (C[O,T])2 and w E ! we set(4.26) ()(h1,h2:= (i(w)(h2),i(w)(h1)).Proposition 4.5 For each i assume either (LIP) or (NIN). Then for every n E IN andw I one has(a) t(w) maps N into N;(b) t’(w) is continuous in N.Proof: (a) It suffices to show that Tü)(h)€ir(N,), for (h’,h2) e N,i j, i,j =1,2. Fix i = 1,j = 2 and suppress the w dependence for simplicity. Then,(i) (T (h2))0 = [i ((n) + h) - (n)j = [(o) - =0, and also (T (h2)) 0for t e [0,Tj (by (4.23));(ii) T’ (h2) is non-decreasing (by (4.23)) and continuous in [0,Tj since ‘i/it is continuousin [—,o] (cf. Remark 4.3).(iii) mTi(h2)(6 < C(6) for every 6 E (O,6]. In fact, let 6 e (0,6] and let t,s E [0,T] besuch that It — I < 6, s <t, then (cf. (ii), (4.21) and (4.23))0 (T(h2)) — (T(h2))= max [ (2(n) + h) — ‘(n)]—[ ((n) + h1) —Chapter 4. The Optimal Control 115Therefore, either(4.27) (T(h2)) — (T(h2)) = 0,if the first max is attained at some t’ s; or else(T(h2))— (T(h2))(4.28) (2() + h) - 1(n)]- [i(2() + h) - 1(n)]= [ (i?(n) + h) — — [ (72() + h) — 1(n)]if max [ (p2(n) + h) — 1(n)] = {i (i(n) + h) — with s <t’ t (noticeO<r<tthat max (,.2(n) + h) — ‘(n)] is attained since [‘ (.2() + h) — 1(n)j is aO<r<icontinuous function as was shown in Remark 4.3). In this case we have0 (T’(h2))— (T’(h2))3(4.29)i ((n) + h) - i(n)- i ((n) + h) + ‘(n).There are two cases.Case 1: n)+h2(n)+hthen,(4.30) i?(n)—j2()— (h— h) 0since h2 is non-decreasing. The continuity of and (4.30) imply0 (T(h2)) — (T(h2))i ((n) + h) - i (j2() + + I) -(I - 2(fl)I + - h) + m?I()(ö)m (2 (n) — j2()) + m1()(6) m, (2m(fl)(6)) + m()(),which becomes, if (LIP) holds,0 < (T(h2)) — (T(ii2)) 2ip(l)m(fl)(5) + m()(6),Chapter 4. The Optimal Control 116i.e., since 6(4.31) mTl(h2)(6) ‘i(6),and (iii) follows.Case 2: (n)+h1>?2(n)+hthen, if (NIN) holds, we have(4.32) (i(n) + h1) ‘i (2(n) + h)since p2(n) + h E [—,] for every r E [O,Tj (by Remark 4.3). Hence from (4.29) and(4.32) follows(4.33) 0 (T(h)) — (T,(h2)) m1(,)(6) < (6),and again we obtain (iii). On the other hand, if (LIP) holds, then (4.29) implies0 (T(h2)) — (T(h2))(4.34) £ip(i) [I(n) — 72(fl)I + Ih— hIJ + i(n)—£ip(l)m2()(6) + £ip(11)mh(6) + ffl1()(6),with mh(6) C(6). So, ifmh(6) <(6)[nw) +1—£ZP@i)then from (4.34) followsmTi(h2)(b) (6).If(4.35) (6) <mh(6) (6),then also(4.36) iñ(6) + £ip(1)th(6) <(1 —Chapter 4. The Optimal Control 117and hence (4.34)—(4.36) implymTi(h2)(5) £(6).Similarly ifmax{L(6),.i(o),2(6)}<mh(6) ‘,(ö)with {i,j} = {1,2}, then (4.34) and L(6) < ‘(6) implyo (T(h2)) — (T1(h2))£ip(’çbi) [th(5) + + iii(5)<Thus, we conclude that (iii) holds even in case 2.It only remains to show (iv) of the definition of N, i.e.(iv) IIT(h2)IIc[o,T] P’; this follows from the definitions of ‘(n) and 13p. In fact, wehaveo (T(h2)) = max [; (r2() + h) —i O<r<tcpi(P+R)+R< ’.(b) Let t e [O,Tj and let (h’,h2),(k’,k N,, then(r1(h1, h2))— (t’(k’, k2)) = ((T(h2)) — (T(k2)), (T(h’))— (T(k’)))and (cf. (4.23) and (4.21))(T(h2)) — (T(k2))fl [7 (2(fl) + h) — ‘(n)] — ma (72(n) + k) — 1(n)](4.37) fl ([ (j2() + h) — ‘(n)]— [ (j-2() + k) —< max(m- (1h2_k21—O(st iI’i S SChapter 4. The Optimal Control 118since 2(n) + h and 2(n) + k are in (—cii, cr) and Qi is continuous there (cf. Remark4.3). Hence(T1(h2)) — (T(k2)) m(6)if 11h2—k211c0,T} <6. Similarly, we obtain (by using (4.24))(T,(h’)) — (T,(k1)) m(5)whenever Ilk1 — k’llc[o,T] <6, and the Proposition is proved.Remark 4.6 Notice that if (LIP) holds true for both i = 1, 2, then the map t(w) is acontraction. In fact, from (4.37) followst(h’, h2) — t(k’, k2)J(c[oTl)2 < (ip(1)V £ip(2)) I(h’, h2) — (k’, k2)I(c[oT])if we set ll(h’,2)lI(c[o,T])2 = llh’llc[o,Tl + IIh2llc[o,T].Theorem 4.7 For each i assume either (LIP) or (NIN). Then, for every n E IN andI, the mapN —* N,has a fixed point; i.e., for a.e. E(4.38) FN(i) := {(h1,h2) e N i(w)(h’,h2)= (h1,h2)} 0.Proof: This follows from Schauder’s theorem, Proposition 4.1 and Proposition 4.5.Let (M,, ) be a measure space and let (X, d) be a metric space, let 2 be thefamily of all subsets of X. We now recall a few definitions (cf. [Iii]).Chapter 4. The Optimal Control 119Definition 4.8 A mapping F M —* is called measurable if for each open set G ofX one has(4.39) F’(G) := {w E M: F(w) fl G O} E .Remark 4.9 When F(w) E comp(X) := the set of all nonempty compact subsets of X,then F is measurable if F’(E) E for every closed subset E of X.Definition 4.10 A mapping T: M x X —* X is called a continuous random operator ifI T(. ,x) is measurable for every x E X;(4.40)( T(w,. ) is continuous for a.e. w M.Definition 4.11 A measurable function ‘-y: M —* X is said to be a random fixed pointof the random operator T: M x X —* X if y satisfies the condition(4.41) u ({w E M T(w,7(w)) = ‘y(w)}) =Remark 4.12 The map T : x N —* N defined by (4.26) is a continuous randomoperator if for each i either (LIP) or (NIN) holds (cf. Proposition 4.5 and Definition 4.2).Remark 4.13 In Theorem 4.7 we have shown that the map (w) has at least a fixedpoint in N for w E \ I, with P(I) = 0. It turns out that if either (LIP) or (NIN) holdsfor both i = 1, 2, then there are no fixed points of t(w) inSp := {(h1,h2) E (C[0,T})2 : (h,h) E 13p Vt [0,T];(4.42)h = 0; W is non-decreasing, i = 1, 2}other than those contained in N; i.e.,(4.43) FNQ.1i) = F(w) a.s.if FN(w) is defined by (4.38) and F() is the set of all the fixed points of the restrictionof Et(w) to Sp. In fact, FNJW) C F(w) since N C Sp.Chapter 4. The Optimal Control 120Conversely, if (NIN) holds for b1 and ‘L’2 then for every (h’, h2) e Sp we have(4.44) t(w)(h’,h2)E N,since t(w) maps Sp into Sp (cf. Proposition 4.5, proof (a), point (iv)); (T() (h’))0 =(o)] = 0 (i = 1,2); (T(w) (h)) 0 and (T(w) (h)) is non-decreasingby Definition 4.2 (i = 1,2); m(T()(hi))(ö) C(6) for every 6 E (0,6] as thisproperty depends only on (NIN) and IIh’IIc[o,Tl P’ <o (cf. Proposition 4.5, proof (a),point (iii)). Hence (4.44) implies F(w) C FN(w). On the other hand, if (LIP) holds for‘ib1, b2 then TQ) is a contraction (cf. Remark 4.6) and maps Sp into Sp, and we havethat F(c) FN(w) is a singleton.Finally, we observe that(4.45) FN,(w) comp(N), w I,where(4.46) comp(N) := {E E 2N : E 0, E compact},since, for all w \ I with P(I) = 0, FN(w) 0 (by Theorem 4.7), FN(w) C N,,FN () is closed and N is compact.Definition 4.14 We define the map(4.47) FN, ! —‘ comp(N)by setting(4.48) FN(w){(h1,h2) e N : (w)(h’,h2= (h1,h2)} if wE \I,(E ifwEl,where E e comp(Ni) is fixed (but arbitrary).Chapter 4. The Optimal Control 121Proposition 4.15 For each i = 1, 2 assume either (LIP) or (NIN). Then the map FNdefined by (.8) is measurable (in the sense of Definition J,.8).Proof: Since FN() E comp(N), it suffices to show that(449) F1’(E) E Ffor every closed subset E of N (where F’(E) is defined by (4.40) and we are applyingRemark 4.9). So let E be a closed subset of N, thenF(E) = {wE1:FN(w)flEO}(4.50) = IU{iE\I:FNjw)flEO} ifEflEO,I {üE12\I:FN(w)flEO} ifEflE=O.Also,{wQ\I : FN()flEO}(4.51) {w e \ I: t(w)(h1,h2)= (h1,2)for some (h’,h2) e E}= 0 {w e \ I: (w)(h, h) - (h, h)NN <u/r}r=1 p=lwhere {(h, h) : p E IN} is a dense sequence in E. (Notice that such a sequence existssince C[0, T] is separable, and hence so is E.) Therefore, since T : Q x N —* N is acontinuous random operator (cf. Remark 4.12), we have{c’ \ I: FN(w) n E O} e F.Hence F’(E) E F. •Together with the measurability of FN a selection theorem (see [SV], p. 289) playshere a crucial role. We restate such a theorem asChapter 4. The Optimal Control 122Proposition 4.16 Let S be a separable metric space and let F: M —* 2 be a measurablefunction such that F(w) is compact for each ‘€M. Then there exists a single valuedmeasurable function y: M —* S such that7(w)EF(Li)) VwéM.Now Proposition 4.15 and Proposition 4.16 allow us to select a fixed point in FN (c’.)in a measurable way; i.e. we obtain a measurable selector of the multivalued map FN.Proposition 4.17 For each i assume either (LIP) or (NIN). Then there exists a randomfixed point (n) : — N,, of the random operator ft,. : ! x N —* N.Proof: This follows from Proposition 4.16 with ! instead of M, and N instead of S,because of Proposition 4.15 and Definition 4.11.We would like to improve the previous result by showing that (n) is also adapted.This is immediate if T has a unique fixed point.Proposition 4.18 For each i assume either (LIP) or (NIN). If for every t E (O,T] therandom operator 1’ : Q x N —, N admits a unique random fixed point t(n) : —* N,,then ‘(n) is adapted to the filtration {Ft}tE[oT].Proof: It is clear that 5t(n) is F-measurable just as ‘‘(n) is F-measurable. It is easyto see that ‘5’(n)[ is a fixed point of it,. on [0, t]. By the uniqueness of this fixed point(4.52) t(n) = 5’(n).Therefore, for every t E [0, TJ(4.53) ‘‘t(W) is Ft-measurable..Chapter 4. The Optimal Control 123Now we would like to take n - 00 and obtain the existence of a solution of (4.55)below. Also, we would like ‘‘ to be adapted. So far we have not succeeded in obtainingthis result under the “mixed” conditions of Proposition 4.17. However, if we restrictourselves to the case in which (LIP) holds for both and ‘&2, then we can easily showthe existence of .At this point we fix T once and for all and from now on we will work only with suchT. From the definition of Y it follows that(4.54) (y’,2)=(’ VtTRAT.Proposition 4.19 For each i assume (LIP). Then there exists a unique, adapted, continuous and non-decreasing (component-wise) process ‘5’s= (‘5’’ ‘‘) such that ‘5’o = 0 a.s.,‘i takes values in l3p and2 1l+= max (Y, + — 1’. j(4.55) + 0 <t TR A T;= max [2 (Y81 + — y2]O<s<ttherefore, in particular, 5t solves (4.20) up to time Ta A T.Proof: If t(5’, 5/) is defined by the RHS of (4.55), then T is a contraction (seeRemark 4.6), hence it has a unique random fixed point in Sp (cf. [Ih], Theorem 2.1). Itis adapted by the same argument as in Proposition 4.18. •In this case we can see that ‘‘ = 1im(n) as follows. Let be defined by theRHS of (4.55). Since= (Th)t =(t’5’), t TAT,then ‘5 is a fixed point of(c) on [0, TAT]. By the contraction property of t’, ‘5’j =fort T AT. Since r AT TR AT a.s., then(4.56) 1im(n) = a.s., t TR AT.Chapter 4. The Optimal Control 124It is convenient to introduce a new notation for Ta A T in order to keep track of theprocess it refers to. Thus we set(4.57) 0(Y) := TR A T.Proposition 4.20 For each i assume (LIP) and let ‘5’ be the fixed point of (4.55). Then,the entire path of Y +‘5’ up to time 0(Y) is contained in cl(A’); i.e.,(4.58) {Y + : t E [0,0(Y)]} C c1(A) fl ([—a,a] x [—a,a]), a.s..Proof: We have three cases:(a) both ‘5’ and ‘5,2 increase at time t e [0,0(Y)]; that is,1(y22)_y1IHence(‘ + ,Y + e graph(1)fl graph(b2)=and since we are assuming Oo = (0,0) (see p. 102), it follows thatY + = (0,0) E cl(A’) fl ([-a, a] x [—a, a]).(b) Oniy one of ‘5” and increases at time t e [0,0(Y)]; i.e., for example,(4.60)i,(y22)_y1Hence(Ye’ + ‘5I 2 + ‘i’?) e graph(1)fl (.ñ U A’);that is,Yt+5’tE8i ccl(A’)Chapter 4. The Optimal Control 125and (4.58) follows from (4.54) and the fact that {‘ : t E [0, 0(Y)]} C Sp a.s..(c) Neither ‘5’ nor increases at time t [0,0(Y)], then(4.61)i>1(y2)_y1Hence(1+ , + e (ñi Ai) fl (ff2 U Al) = cl(A1)and again we obtain (4.58) from (4.54) and E Sp, a.s..Now we can finally answer the crucial question posed at the beginning of this section;that is, “how should k’ and k2 be defined after J reaches the corner point (0,0) ?“. Thefixed point of t (T is defined by the RHS of (4.55)) is actually a function of ‘(w), i.e.of Y(w) or of Y e )) where(4.62) 3) { e (C[0,T])2 : Y(0) = o},therefore we write it as ‘(Y); that is,(4.63) Y)=+ ‘(Y)) —Note that if r is a stopping time such that= X(w) + (w) = (0,0) =thenXs+r(W) + k(w) =X3+(w) — X(w) += X3+() — Xr(W) =: YT(w) E 3).Hence we set(4.64) t(T,W) :=Chapter 4. The Optimal Control 126For w E {r’ = r01} let(4.65) 0”°(ü) TA inf{t > 0 : YtTl(,) B}(4.66) := r’(w) + 0”°(w).For t E [T’(w),r1”(w)j we define(4.67) kt(w) := ‘(‘) +5’j_7i(T1,w).Notice that ‘‘ = on [0, O101 and ‘5’s is continuous by construction. Then, ifXTI,1(w) (0,0) we set(4.68) T2(W) :=otherwise we set(4.69) O”(w) := TA inf{t > 0 : YT’I(w)(4.70) r”2(w) := r”(w) + O”(w),and fort E [‘r”(w),r”2(w ] we define(4.71) j) := ki,i(i) + _i,i(r”,w),etc. It is clear that if (4.68) holds, then X2() cl(A’) \ 0o as this follows fromProposition 4.20. Hence we set2fr() if X2(w) E ñ1 \ 0o, i = 1,2,(4.72) -r (w) =:L. T(W) if X2(w) E A1,as well as(4.73) k72(w) :=since - is continuous and X2_ E (—o, o.Chapter 4. The Optimal Control 127Now define k inductively. Assume we have defined T’ and k on [0, Ta]. Then:=f w {r = T}It E(4.74) whereT1() := r(w) A rr1(w) Arr1(),T;+’(W) := inf{t T’(W) : X(w) +k(w) E E}, p = 1,2,0,E1:=Ei\Oo, B2:=R\Oo, B0:=8.(Notice that T’2 = T’ implies = Tn.)kt(w) :=+ w) + )}fwe{T=T}if(t e [T(W),T+’(W))where:=(4.75) Oni(,) T A inf{t > 0 : YT(w):= T’(w) +O’(w) = n(W) +‘(w),l(w) := min{l E IN : k,1(w) (0,0)},I T”1”(W) if Ern,in (w):= for p = 1, 2,3,(+00 otherwise,Tn+(W) := +i(w) A T+l(w) A 72+l(w),i:=ii\Oo, B2:=R\Oo, 3:=A1.Chapter 4. The Optimal Control 128(When (4.75) holds, kn+1() = i+i_(w) since ‘5’ is continuous as < cr.)Ic(w) :=Iii ‘ ._ i.intjW)“T(w)— W+ max (X(w) + fl()(w)) - (X(w) +1we{T=r}, j=1,2if(4.76) It e [r(oJ),T(w))where:= ‘(w) AT41(w), i j , i = 1, 2,r1(w) : inf{t : X(w) + £t(W) € ,p = i,O,E:=i\Oo, B0:=0.Theorem 4.21 For each i assume (LIP). Then the process ict= (, £) obtained by(4.74)—(’4. 76) is a solution of the fixed point problem (4.18); i.e.I k — = max [ (x + — (x + K])](4.77) 1 - = [2 (x + - (x +for every t E [r’,T1) and any n E IN.Proof: The proof of (4.18) shows that (4.77) holds at least when T = T,j = 1,2; i.e.,when Kt is defined by (4.76). On the other hand, when = we have (cf. (4.74))(4.78) = krn = Vt E [T’,T’’);also, by the definition of r (cf. (4.75)8 with n instead of n + 1 and p = 3) followsx + e A’,which together with the definition of ‘r’1 (cf. (4.74)4) implies(x + K, x + A’Chapter 4. The Optimal Control 129for every s e [T1, T’). Hence the RHS of (4.77) is zero; i.e., (4.77) holds.It remains only to show that (4.77) holds when T’2 = T’; that is, when k is definedby (4.75). In this case let i e IN, i = i(i), be such that (cf. (4.75)7)n+l =and let t T’1). First of all, we assume t < r’’1; then (cf. (4.67) with r1’ instead ofr’) k — = — n = Also,[ (x + — (x + ‘)J += max [ (x + + (k — )) — (x + i’)] +r <s<t= [ (x + + r(Tn)) - (x +=where we have used (4.63) and (4.64). Hence (4.77) holds. Now we proceed by inductionon p e IN, p < in. We assume that (4.77) holds for s E [r’’, T”P] with p + 1 i, andwe let t E [TnP,TnP4]. Then (4.75) implies— kra_ = — Icrn = I-’,p — k,. +Moreover,Xfl,P = (0, 0)since p + 1 in (cf. (4.75)7); thus, in (4.63) (with T’ instead of Y) we may ignore,i.e.= rt { (x + — (x + + 1.I. — IcL,,.Hence(4.79)—= r’’r<t [ (x + — (x +Chapter 4. The Optimal Control 130Also, T’ < T?2 implies thatmax [ (x + — (x + £‘)] +( )= [ (x + — (x + k])] +In fact, if not, then we would haveT<T’P [ (x + - (x1 + )] + > (x + - (x +for every s e [flP, t]; hence, because of the induction hypothesis,— > ‘ (x + — (x +for every s E tj. Hence, in particular,>Therefore,X,P int(A’ U 1?2)and hence (cf. (4.75)9)T’ <T’in contrast with the fact that p + 1 l and T’’ = Thus, (4.80) holds and henceso does (4.77), because of (4.79).Corollary 4.22 For each i assume (LIP). Then the process=(k, k) obtained by(4. 74)—(4. 76) is a solution of the fixed point problem= max [?4 (x + — x3’] +(4.81) Ost t 0,k?=max[çb2(X+k) _x]+.Chapter 4. The Optimal Control 131Proof: This is obtained by induction on n E IN. We already know (cf. (4.9)) that (4.81)holds for every t e [0, T’) if T1 > 0, and at t = 0 when r1 = 0. Now we assume that (4.81)holds for all t e [0,r) and then we show that (4.81) holds also for all t€[TZ,Tl).Let t e [,T1), then (4.77) implies(4.82)—= [ (x + — (x +If the max above is zero (this includes the cases = r and r’ = Ti’), then by inductionwe have== [ (x + — x];butVs€[r’,t] :(since the max above is zero), hence= ma [ (x + — X’];i.e., (4.81) holds for t. On the other hand, when = or then we can drop “+“in (4.82). Therefore, from (4.82) we getJ = [7 (x + — x8’] = [(x + —I It’In_.Hence the induction hypothesis implies=[k (x + — x’],i.e.,= max [ (x + k) — x]and we obtain (4.81). Similar arguments provide us with (4.81)2 and hence the proof ofthe Corollary is complete. •Chapter 4. The Optimal Control 132Corollary 4.23 Assume (LIP) for both i = 1, 2. Then the process= (, ) obtainedby (4. 74)—(4. 76) is uniquely defined, adapted and cadlag, with non-negative and non-decreasing components. In particular, k is progressively measurable.Proof: This follows from (4.74)—(4.76) and Proposition 4.19. The last assertion in thestatement above is true for any process which is adapted and right-continuous. •Remark 4.24 We believe that a unique solution of (4.55) exists even when (NIN) holds,or in the mixed case. However, since we have been unable to prove so, we assume (LIP)in order to have k adapted (cf. Proposition 4.19).It remains only to show that k as given by (4.74)—(4.76) is optimal for (x). We dothis by means of a generalized Ito formula for semimartingales and Sobolev functions.4.4 OptimalityLet the control k be defined by (4.74)—(4.76), then the corresponding controlled process:= X + k never leaves cl(A1) after time 0, i.e.(4.83) : t E (0,+oo)} C cl(A)as this follows from the construction of k (in fact, it suffices to argue as in the proof ofProposition 4.20 with the representation of k given by (4.81)). Notice that if b1 and ‘b2are continuous functions then the optimal process X is continuous too, except perhapsat time 0; in fact, the jumps of k (and hence of X) may only occur in correspondenceto the jumps ofRemark 4.25 It is clear that, because of (4.83), the optimality of k could be obtainedby means of the Ito formula for semimartingales and the dynamic programming equationChapter 4. The Optimal Control 133Alt = f (cf. Section 4.2). However, the classical formulation of the Ito formula requiresthe function ll (to which we would like to apply this formula) to be of class C2 at leastin a neighbourhood of cl(A’). But in the general setting of Chapter 2 we can only claimll e W°(R2), and even under the more generous conditions of [WCM] (i.e., those ofSections 3.5 and 3.6) we are still unable to extend ll to aC2-function near the cornerpoint 0o The problem is that the presence of this corner prevents us from applyingthe classical PDE results about extension operators. This does not mean that ñ cannotbe smooth near 00, but only that a different approach is required (in fact, due to theparticular class V of admissible controls, it is very likely that the corner remains evenunder stronger conditions on the data of the control problem).A more general Ito formula was proved by Krylov (cf. [Kry], p 122); his result appliesto a non-degenerate diffusion in a bounded region and to functions having only generalizedsecond derivatives. We adjust Krylov’s theorem to account for the fact that our processX is a semimartingale.Theorem 4.26 Assume (LFP), (LIP), a non-degenerate, and &j continuous (i = 1,2).Then the process k = (k, ) defined by (4. 7)—(4. 76) is optimal for the original controlproblem (2.3); i.e.,(4.84) ll(x) = J() for allx 112Proof: Let x E 112 and let Xj = X + k be the controlled process starting at timet=Ofromx. Letm>Obesuchthatx<mandk<m,i=1,2;wedefinerbyr(w):=inf{t>O:IXtImorkm,iE{1,2}}and we apply Krylov’s version of the ItO formula to ll(f()e_Pt from 0 to r.Weset Bm = B(O,m(1+v’)) andwerecallthat (cf. Proposition2.2) ft E W2’°°(Bm+i),here Bm+i = B(0,m(1 + /) + 1); All = f a.e. in A’ (cf. (3.67)); also OA’I = 0 (cf.Chapter 4. The Optimal Control 134Proposition 3.37 and recall that .9A’ C ÔA’). Then we redefine All on the free boundary0A1; i.e. we set(4.85) All := f on .9A’.We approximate ll by smooth functions. Let us set Bm Cl(Bm). Then there existfl E C2(Bm) such that for q < 00IIu— UIIC(m) ) 0,(4.86)— VllhIC(m) 0IID2u — D2ILIILQ(B ) 0;we now apply the Ito formula for semimartingales (cf. [My], p.278) to e_Ptu(f() from 0to r, hence we havee_PTu(Xr)—(4.87) = jm e (—All’)(t)ds + fm e 8Vuh1((s) dl5+ f eVu(X3)udW.Now we would like to pass to the limit in (4.87) and conclude an analogous result for ii.This is possible because of some very crucial estimates obtained by Krylov. Our setting ismuch simpler than his, since for us g and c are constant. Hence we trivially get estimateslike II K(det a)1!2 andE{Jrne8IgI ds} K(det a)”2j eds N(compare with {Kry], Lemma 2.5, p.54). However, in order to claim(4.88) E{Jrn e_P8Ih(s)Ids} Nhqfor any Borel function h(x) and with N = N(g, a, m, 2), we follow Krylov’s proof ofhis Lemma 2.8, p.56. All of Krylov’s arguments make use of “mollification” and theChapter 4. The Optimal Control 135application of the Ito formula to smooth functions. In these formulae we make theopportune changes to account for the presence of k in the process X; i.e. , we add theextra termtArj eVh)(X3)dk8.But from the definition of T follows ktArA I <mJ, hence at the bottom of p.57 in [Kry]we add to the right side the termtAr 2E{j e’z(T—y3,,x)(dk + dk)which is bounded below by—NIIfII4m/R, and we obtain his result, i.e. (4.88).Finally, by using (4.86) and (4.88), we pass to the limit in (4.87) and we have= frrneAü(3)ds + eT(rA)(4.89)-eVii3). (dk,dk)— f V(X)adW, a.s.We observe that(i) Aul(X) = f(X3) Vs e (O,+oo), a.s.,because of (4.83) and (4.85) (in fact, because of (4.85), An= f is extended to hold in(ii) E{Jrn e8Vn3) (d,d)ds} = 0since k increases only at the boundary O, but there i vanishes (cf. (4.1) and (4.83));(iii) E{jrn(e_IVn(ks)u)2ds}rAE{c(a, O1) (1 + I(sI)21)e_2129ds}E{c(u,Ci)j (1 + 1X8 + IksI)2_1)e_2P8ds}E{c(a, O) jrn(1 + m + m)2_1)e_2P8ds}c(u,Ci,m,p)f e2”ds < oc,Chapter 4. The Optimal Control 136where we have used the fact that Vll has at most polynomial growth (in fact, the constantC1 is the one given by (2.11)2), and the definition of r; here c(u, C1) is a constant whichdepends only on a and C1, and c(a, C1,m, p) is a constant which depends only on a, C1,m, and p. Finally we take expectations in (4.89), and by using (ii) and (iii) we obtainE{ll(x + o)} = E{fm All(k3)e”ds + ll(XrA)e_PT}E{f All(Xs)e_uisds}since ll 0. Also, we can replace All with f as this follows from (i); therefore, we haveE{ll(x + k0)} E{frn f(Js)e_P8ds};but ko = 0 a.s. if x E A’, and ll(x + Ico) = fz(x) a.s. if x e (A (cf. (4.2) and (4.9)),hence, as m —* +00, we obtainll(x) E {j°° fs)etdt} = J(k),i.e. (4.84) holds since ll(x) := infJ(k).Remark 4.27 There exists at most one optimal control k for the control problem (2.3).In fact, assume not, then let the controls k and Ii be such thatJ()=ll(x)=J().Now set := and denote X the controlled process starting from x at time 0 andcorresponding to the control ; thenxt =1 1=Chapter 4. The Optimal Control 137and hence (by strict convexity)f(X$) < f(Xj + + f(X +Thus,J () < Jx() + J(i) =which is impossible since i(x) := inf J(k).4.5 A Verification of (LIP) and (NIN)In order to give some examples in which (LIP) is satisfied, and hence k exists and isuniquely defined by (4.74)—(4.76), we resume the hypotheses of Section 3.6 (i.e., theconditions in [WCM]) and, in addition, we assumeI Vf •(M1,1)>0,(4.90) M1 E (0,1) such that 1 —( Vf, (IvIi,—1) 0,I Vf •(1,M2)>0,(4.91) M E (0,1) such that 2 —( Vf.(—1,M)0.Then we haveProposition 4.28 Assume (3.114), f e C3(1R2), (3.116), (4.90) and (4.91). Then jis a contraction (i = 1, 2); hence, in particular, (LIP) holds.Proof: This follows from [WCM], Theorem 4.12, and from the geometry of the regionof action. In fact, notice that (4.90) and (4.91) correspond to (3.170) (except for thefact that in (4.90) and in (4.91) we require M < 1 too). It is shown in [WCM] that,under (3.114), f e C3(R2), (3.116) and (3.170), one hasI Vv .(M1,1)>0,(4.92)—( Vv2 . (1,1v12) 0,Chapter 4. The Optimal Control 138where v is the solution of the penalized problem (3.123). (We point out that (4.92) isobtained by applying to A(Vv1 (Mi, 1)) and to A(Vv2 (1, M2)) a version of the maximum principle for elliptic operators andC2-functions bounded above by a polynomial;in fact, v6 e C4’(R2)if f E C3(R2).)Clearly (4.92)2, for example, means that v2 is non-decreasing along lines of slope M2;then, by using the fact that v2 —* uniformly on compacta as a — 0, we concludethat(4.93) T’1(P) C Ro U R2 for every P e 02,where F1(P) is the cone with vertex F, defined by= {Q = (x?, x) : x? xi’, x x, (P— Q) e2 P — QI cos(arctan(1/M2))}if F = (xf, x) and ë2 is the unit vector in thex2-direction. (For the detailed proof ofthis see [WCM], Theorem 4.12).Now we apply these same arguments to our (4.90)2 and we obtain(4.94) Fr(P) C R U R2 for every P e 02,with= {Q = (xv, x) x? xf, x <x, (P — Q) e2 IF — QI cos(arctan(1/M2))}.Thus, (4.93) and (4.94) imply(4.95) 1’(P) := P1(P) U Fr(P) C R0 U R2 for every P E 02.From this immediately follows the Lipschitz property of with £ip(2) M2 < 1. Infact, if not, then there would be a point P E 02 and a neighbourhood B,(P) of P suchthat(4.96) B,7(P) fl 02 fl int(I1(P)) 0Chapter 4. The Optimal Control 139or(4.97) B11(P) fl 0 fl int(Fr(P)) 0,11(P) and 11r(P) being the mirror images of the cones 1’(P) and 1’r(P) (respectively)in the horizontal line x2 = xc. Suppose, for example, that (4.97) holds; then there existsa point Q = (x?,x) E B,1(P) fl int(1’r(P)) such that Q E 02, and hence Q satisfies(4.93); so, in particular,{ (xi,x) : x1 x?,x2 x,x2—4 = M2(xi — x?)} c R0 U R2.Hence (xf, 4 + M2(xf — x?)) E 1?o U R2, i.e.b2(x1) 4+M2(xf—x?)=but Q int(1’r(P)) implies 4 — 4’> M2(x? — xfl, therefore we have2(X1) > Xwhich contradicts P E 02 (i.e., P e graph(i,’2)). Thus, the graph of 2 is constrainedto lie within the two cones with vertex F, with axis parallel to the x1-axis and angle2/ = 2 arctan(M). Moreover, since the same is true at every point of 02 (with the sameangle 2/3), we conclude that £ip(2) M2, and hence, in particular, (LIP) holds. •Remark 4.29 Notice that in Theorem 3.70 we had already shown that, if (3.114), f eC3 and (3.116) hold, then ‘i/’j is continuous everywhere and locally Lipschitz away fromthe corner point 0 (and hence i/ = Hence in Proposition 4.28, (4.90) and (4.91)allow us to improve Theorem 3.70 and obtain £ip(i) < 1.Remark 4.30 We point out that in [WCM] the two extra conditions (4.90) and (4.91),with M > 0, (i.e., (3.170)) were introduced only to show that the points of the freeChapter 4. The Optimal Control 140boundary are points of positive Lebesgue density for the region of inaction, i.e. (4.93).However, this follows already from (3.114), f e C3(112)and (3.116) (cf. Section 3.6).Corollary 4.31 Assume (3.114), f e C3(1R2), (3.116), and(4.98) f12 0 in It2Then b1 is non-increasing (i = 1,2); hence, in particular, (NIN) holds. If in addition(4.90)2 (resp. (4.91)2) holds, then ‘b1 (resp. 1’2) is also a contraction, and hence (LIP)holds for ‘/‘i (resp. ‘2).Proof: It is clear that (4.98) is a limiting case of (4.90), and hence the arguments ofthe proof of Proposition 4.28 apply and allow us to conclude that(4.99) P — A* C (A for every P Ewhere A* = {(x1,x2) E 112 : x1 0, x2 0}. This implies that ‘/j cannot have anypoint of strict increase; i.e. /‘j is non-increasing. So, in particular, (NIN) holds. The laststatement of the Corollary follows from Proposition 4.28. •Notice that (by convexity) the condition (4.98) implies that (4.90) and (4.91) holdfor every M1,2 E 11+.Examples: Let be fixed constant, then(x1—i)2 (x2 — x2)(a) f(xi,x2)= a2 + b2satisfies f12 0, hence the functions 4j are constant (in fact, b(x) ‘‘(x), i j, if(x, 4) is the corner point). Therefore, the region of inaction A1 is just a translation ofthe first quadrant. Obviously both (LIP) and (NIN) hold.— ‘) + ,i3(x2 — x2)) (—/3(xi— ‘) + c(x2 —(b) f(xi,x2)= a2 + b2Chapter 4. The Optimal Control 141satisfies (4.98) and (4.90)2 ifb2— 2()2 + (16a)2If this is the case, then the functions ‘i/ are non-increasing contractions. Both (LIP)and (NIN) hold. However, if K > 1 then the functions are only non-increasing and(NIN) holds. On the other hand, if K < 0 then and 2 are non-decreasing. Finally,if K e (—1,0) then b1 and 2 are non-decreasing contractions, and hence (LIP) holds.(c) f(xi,x2)= (a(xi — ,)2 +i3(x2 — X2)T, r E IN,r 2,satisfies (4.90) and (4.91) if__12cj2V/32 2r—1In particular, if o = ,6 = 1 then (4.101) is verified for r = 2,3, and (LIP) holds.We conclude this Section by showing that either of the two conditions (LIP) and(NIN) (together with the assumptions of Section 3.6) impliesC2-regularity of the valuefunctions ñ in the closure of A’, and so improves and completes Corollary 3.72.Proposition 4.32 Assume (3.114), f C3(1R2) and (3.116). For each i assume either(LIP) or (NIN). Then the free boundary is Lipschitz, i.e.(4.102) DA’ is of class C°”.Proof: Because of Theorem 3.71,(ii), it suffices to analyze the regularity of 0A1 nearthe corner point Oo = (0, 0). There are three cases(a) both , and ‘4&2 satisfy (NIN);(b) both , and 2 satisfy (LIP);(c) /‘ satisfies (NIN) and satisfies (LIP), i j.In any of these cases it is possible to find a small neighbourhood Be(Oü) of 0o suchChapter 4. The Optimal Control 142that B(D0) fl DA’ is the graph of a Lipschitz map, since /ij is a function of x, i j,graph(,) fl graph(2)= O and £ip(’/’jI[O,)) < 1. In fact, let lo be the straight linethrough and orthogonal to v = (+1, +i)/v; letp: B(ôo) fl Oil’ —*be the orthogonal projection of Be(Oü) fl OA’ on 10. Then P(Oo) = 0 and p is 1-to-i if Eis sufficiently small. Let ,u = (+1, —i)/i/ and define‘ir :p(B(Oo)flOA’)—*Rby setting7r(p(P)) = p(P)Clearly ir o p is 1-to-i; then let h denote the inverse function of ir op. It follows thath(z2) — h(z,) fi in case (a),— Z1— ( C(/3) in cases (b) and (c),where /3 := £p(?/’lI[O,a)) VCP(’b2I[O,a)) and C(/3) is a constant which depends on ‘3. HenceDA’ is Lipschitz. •Corollary 4.33 Assume (3.114), f e C3(1R2) and (3.116). For each i assume either(LIP) or (NIN). Then the value function i2 is of class C2” in A’ up to the boundary,and hence(4.103) C2(cl(A’)).Proof: This follows from Proposition 4.32 by the application of a classical Sobolevembedding theorem for Lipschitz domains (cf [GT], Theorem 7.26, p. 171). •Now, a posteriori, we can see how similar our result is to the one obtained by Davisand Norman [DN], in the sense that our optimal cost fz and their optimal profit show theChapter 4. The Optimal Control 143same behaviour outside of the region of inaction A1; moreover, although our A’ is notgenerally wedge-shaped, the direction of the reflection on the two branches 01, 0 of OA’turns out to be determined in a way analogous to the one in [DN]. This similarity of thetwo models is quite surprising if one observes that Davis and Norman allow the diffusion tohave state-dependent coefficients, and take as controls the processes of bounded variationof the formI—1 —1)Ut(here ). e [0, ), e [0, 1], and L, U are non-decreasing processes with L0 = 0 = U0).However, they restrict themselves to a very special form of the cost rate f in order todeduce their deus ex machina: the “homothetic property”. On the other hand, our diffusion has constant coefficients, and our controls are monotone non-decreasing (componentwise), but the cost rate f is quite general, although convex. Nevertheless, there is onething common to both models, that is the cheap control setting. Then, perhaps, whatwe have obtained is a characterization of the class of cheap control problems, monotoneor not.Bibliography{Bd} F.M. Baldursson, Singular Stochastic Control and Optimal Stopping, Stochastics,21(1987), pp. 1 — 40.[BK] H. Brezis and D. Kinderlehrer, The Smoothness of Solutions to Nonlinear Variational Inequalities, Indiana Univ. Math. J., 23, no. 9 (1974), pp. 831 — 844.[Bn] J. M. Bony, Principe du maximum dans les espaces de Sobolev, C. R. Acad. Sci.Paris, 265 (1967), pp. 333 — 336.[Bs] H. Busemann, Convex Surfaces, Intersci. Pubi., New York, 1958.[Cf] L.A. Caffarelli, The Regularity of Free Boundaries in Higher Dimensions, ActaMathematica, 139 (1977), pp. 155 — 184.[CMR] P.-L. Chow, J.-L. Menaldi and M. Robin, Additive Control of Stochastic LinearSystems with Finite Horizon, SIAM J. Control and Optim., 23 (1985), Pp. 858 —899.[DN] M.H.A. Davis and A.R. Norman, Portfolio Selection with Transaction Costs,preprint 1989.[Fri] A. Friedman, Regularity Theorems for Variational Inequalities in Unbounded Domains and Applications to Stopping Time Problems, Arch. Rational Mech. Anal.,52 (1973), pp. 134 — 160.[Fr2] A. Friedman, Variational Principles and Free Boundary Problems, John Wiley &Sons, New York, 1982.[FR] W.H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control,Springer-Verlag, New York, 1975.[Gs] E. Giusti, Minimal Surfaces and Functions of Bounded Variation, Birkhäuser,Boston, 1984.[GT] D. Gilbarg and N.S. Trudinger, Elliptic Partial Differential Equations of SecondOrder, Springer-Verlag, New York, 1983.[HM] A.C. Heinricher and V. J. Mizel, A Stochastic Control Problem with DifferentValue Functions for Singular and Absolutely Continuous Control, Proc. 25thConf. Decision and Control, Athens, Greece, 1986.144Bibliography 145[Ih] S. Itoh, Random Fixed Point Theorems with an Application to Random Differential Equations in Bariach Spaces, Jour. Math. Anal. and Appl. 67 (1979), PP. 261— 273.[KS1] I. Karatzas and S.E. Shreve, Connection Between Optimal Stopping and SingularStochastic Control I. Monotone Follower Problems, SIAM J. Control and Optim.,22 (1984), pp. 856 — 877.[KS2} I. Karatzas and S.E. Shreve, Brownian Motion and Stochastic Calculus, Springer-Verlag, New York, 1987.[Krl] I. Karatzas, A Class of Singular Stochastic Control Problems, Adv. Appl. Prob.,15 (1983), pp. 225 — 254.[Kr2] I. Karatzas, Probabilistic Aspects of Finite-Fuel Stochastic Control, Proc. Nat’l.Acad. Sci. USA, 82, (1985), pp. 5579 — 5581.[Kry] N.V. Krylov, Controlled Diffusion Processes, Springer-Verlag, New York, 1980.[Ln] P.L. Lions, Optimal Control of Diffusion Processes and HJB Equations: Part II,Comm. PDE, 8 (1983 b), pp. 1229 — 1276.[LS] P.L. Lions and A.S. Sznitman, Stochastic Differential Equations with ReflectingBoundary Conditions, Comm. Pure and Appi. Math, 37 (1986), pp. 511 — 537.[Mr] M. Miranda, Superfici Cartesiane Generalizzate ed Insiemi di Perimetro Localmente Finito sui Prodotti Cartesiani, Ann. Sc. Norm. Sup. Pisa, 18 (1964), pp.515 — 542.[MRb] J.-L. Menaldi and M. Robin, On Some Cheap Control Problems for DiffusionProcesses, Trans. Am. Math. Soc., 278 (1983), pp. 771 — 802.[MRf] J.-L. Menaldi and E. Rofman, On Stochastic Control Problems with ImpulseCost Vanishing, Proc. International Symposium on Semi-Infinite Programmingand Applications, Lecture Notes in Economics and Mathematical Systems 215,Springer-Verlag, New York, 1983, pp. 281 — 294.[MT] J.-L. Menaldi and M.I. Taksar, Optimal Correction Problem of a Multidimensional Stochastic System, Automatica, 25 (1989), pp. 223 — 232.[My] P.A. Meyer, Un Cours sur les Integrales Stochastiques, Séminaire de ProbabilitésX, Université de Strasbourg, Lecture Notes in Mathematics 511, Springer-Verlag,New York, 1976.Bibliography 146[SM] M. Sun and J.-L. Menaldi, Monotone Control of a Damped Oscillator UnderRandom Perturbations, IMA J. Math. Control and Infor., 5 (1988), PP. 169 —186.[SS] H.M. Soner and S.E. Shreve, Regularity of the Value Function for a Two-Dimensional Singular Stochastic Control Problem, SIAM J. Control and Optim.,27 (1989), pp. 876 — 907.[SV] D.W. Stroock and S.R.S. Varadhan, Multidimensional Diffusion Processes,Springer-Verlag, New York, 1979.[Tk] H. Tanaka, Stochastic Differential Equations with Reflecting Boundary Conditionsin Convex Regions, Hiroshima Math. J., 9 (1979), pp. 163 — 177.[Tn] L. Tonelli, Sul Differenziale dell’ Arco di Curve, Atti Accad. Lincei, 25 (1916),pp. 207 — 213.[vM] P. van Moerbeke, On Optimal Stopping and Free Boundary Problems, Arch. Rational Mech. Anal., 60 (1976), pp. 101 — 148.[VWJ S.R.S. Varadhan and R.J. Williams, Brownian Motion in a Wedge with ObliqueReflection, Comm. Pure Appi. Math., 38 (1985), pp. 405 — 443.[WCM] S.A. Williams, P.-L. Chow and J.-L. Menaldi, Regularity of the Free Boundaryin Singular Stochastic Control, preprint 1990.[Zm} W.P. Ziemer, Weakly Differentiable Functions, Springer-Verlag, New York, 1989.
- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Geometric approach to monotone stochastic control
Open Collections
UBC Theses and Dissertations
Featured Collection
UBC Theses and Dissertations
Geometric approach to monotone stochastic control Chiarolla, Maria 1992
pdf
Page Metadata
Item Metadata
Title | Geometric approach to monotone stochastic control |
Creator |
Chiarolla, Maria |
Date Issued | 1992 |
Description | The two main questions arising in singular control problems are the characterization of the boundary of the region of inaction A’ (i.e. the region where it is optimal to take no action) and the construction of an optimal control. Among the singular control problems the ones in which the class of admissible controls is restricted to the processes with monotone non-decreasing components, and the payoff functional does not depend explicitly on the control, are usually referred to as monotone follower, cheap control problems. We identify the free boundary ƌA1 of the two-dimensional monotone follower, cheap control problem under very mild conditions. We prove that if the region of inaction is of locally finite perimeter (LFP), then such a region can be replaced by a new region A1 having a more regular boundary. In fact, we show that the new free boundary is countably 1-rectifiable and it is also optimal to take no action in the larger set A1. Then we give conditions under which the hypothesis (LFP) holds; furthermore we obtain even higher regularity of the free boundary, namely C2α, except perhaps at a single corner point. This result is easily extended to the n-dimensional case. Under the additional hypothesis that the free boundary of the new region of inaction A1 satisfies a Lipschitz condition (LIP) in a small neighbourhood of the corner point, we construct a control k which acts only when the process is not in A1 and then only to move it instantaneously into A1. We show that k is the unique optimal control of the singular control problem in question. Finally we give conditions under which (LIP) is verified. All of these results hold in the n-dimensional case. |
Extent | 2242340 bytes |
Genre |
Thesis/Dissertation |
Type |
Text |
FileFormat | application/pdf |
Language | eng |
Date Available | 2008-12-19 |
Provider | Vancouver : University of British Columbia Library |
Rights | For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use. |
IsShownAt | 10.14288/1.0079907 |
URI | http://hdl.handle.net/2429/3236 |
Degree |
Doctor of Philosophy - PhD |
Program |
Mathematics |
Affiliation |
Science, Faculty of Mathematics, Department of |
Degree Grantor | University of British Columbia |
GraduationDate | 1992-05 |
Campus |
UBCV |
Scholarly Level | Graduate |
AggregatedSourceRepository | DSpace |
Download
- Media
- 831-ubc_1992_spring_chiarolla_maria.pdf [ 2.14MB ]
- Metadata
- JSON: 831-1.0079907.json
- JSON-LD: 831-1.0079907-ld.json
- RDF/XML (Pretty): 831-1.0079907-rdf.xml
- RDF/JSON: 831-1.0079907-rdf.json
- Turtle: 831-1.0079907-turtle.txt
- N-Triples: 831-1.0079907-rdf-ntriples.txt
- Original Record: 831-1.0079907-source.json
- Full Text
- 831-1.0079907-fulltext.txt
- Citation
- 831-1.0079907.ris
Full Text
Cite
Citation Scheme:
Usage Statistics
Share
Embed
Customize your widget with the following options, then copy and paste the code below into the HTML
of your page to embed this item in your website.
<div id="ubcOpenCollectionsWidgetDisplay">
<script id="ubcOpenCollectionsWidget"
src="{[{embed.src}]}"
data-item="{[{embed.item}]}"
data-collection="{[{embed.collection}]}"
data-metadata="{[{embed.showMetadata}]}"
data-width="{[{embed.width}]}"
async >
</script>
</div>
Our image viewer uses the IIIF 2.0 standard.
To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0079907/manifest