@prefix vivo: . @prefix edm: . @prefix ns0: . @prefix dcterms: . @prefix skos: . vivo:departmentOrSchool "Science, Faculty of"@en, "Mathematics, Department of"@en ; edm:dataProvider "DSpace"@en ; ns0:degreeCampus "UBCV"@en ; dcterms:creator "Jiaxing, Wang"@en ; dcterms:issued "2014-08-21T16:22:05Z"@en, "2014"@en ; vivo:relatedDegree "Master of Science - MSc"@en ; ns0:degreeGrantor "University of British Columbia"@en ; dcterms:description """In this thesis, we studied the act-and-wait control mechanism on the stabilization of the inverted pendulum in both piecewise constant feedback and continuously varying feedback models. We also extend the act-and-wait control mechanism with a more general model: the act-and-wait control with frequently varying feedback, which includes piecewise constant feedback and continuously varying feedback as two of its extreme cases. The frequently varying feedback model is valuable in approximating the continuously varying feedback system when the delay is large. The modeling error is discussed in the comparison of piecewise constant feedback and continuously varying feedback. The robustness of the three models are studied in the context of the parametric stability regions as well as for the interaction of delay and noise. We discovered that although act-and-wait control can stabilize the pendulum system even with large delay, the robustness is impaired by large delays. As a result, the piecewise constant feedback system is more sensitive to parametric noise than the frequently and continuously varying feedback models. The interplay of act-and-wait control and external noise leads the system to a periodically varying density for its state."""@en ; edm:aggregatedCHO "https://circle.library.ubc.ca/rest/handle/2429/50098?expand=metadata"@en ; skos:note "Dynamics of Balance withAct-and-Wait ControlbyWang, JiaxingB.Sc. Peking University, 2011A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF SCIENCEinThe Faculty of Graduate and Postdoctoral Studies(Mathematics)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)August 2014c© Wang, Jiaxing 2014AbstractIn this thesis, we studied the act-and-wait control mechanism on the stabi-lization of the inverted pendulum in both piecewise constant feedback andcontinuously varying feedback models. We also extend the act-and-wait con-trol mechanism with a more general model: the act-and-wait control withfrequently varying feedback, which includes piecewise constant feedback andcontinuously varying feedback as two of its extreme cases. The frequentlyvarying feedback model is valuable in approximating the continuously vary-ing feedback system when the delay is large. The modeling error is discussedin the comparison of piecewise constant feedback and continuously varyingfeedback. The robustness of the three models are studied in the context ofthe parametric stability regions as well as for the interaction of delay andnoise. We discovered that although act-and-wait control can stabilize thependulum system even with large delay, the robustness is impaired by largedelays. As a result, the piecewise constant feedback system is more sensitiveto parametric noise than the frequently and continuously varying feedbackmodels. The interplay of act-and-wait control and external noise leads thesystem to a periodically varying density for its state.iiPrefaceThis thesis is original, unpublished, independent work by the author, JiaxingWang.iiiTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . ixDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 The Model for act-and-wait control . . . . . . . . . . . . . . 82.1 Basic model with continuously varying control . . . . . . . . 82.2 Continuously varying vs piecewise constant act-and-wait con-trol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.3 Deadbeat control . . . . . . . . . . . . . . . . . . . . . . . . 133 Cases with frequently varying feedback . . . . . . . . . . . . 223.1 Formulation and comparison . . . . . . . . . . . . . . . . . . 223.2 Mathematical analysis of FV model . . . . . . . . . . . . . . 274 Noise Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . 354.1 Parametric noise . . . . . . . . . . . . . . . . . . . . . . . . . 354.2 External noise . . . . . . . . . . . . . . . . . . . . . . . . . . 43ivTable of Contents5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53AppendixA The mathematical calculation . . . . . . . . . . . . . . . . . . 56A.1 The calculation of matrices . . . . . . . . . . . . . . . . . . . 56A.2 Deadbeat control . . . . . . . . . . . . . . . . . . . . . . . . 59A.3 The probability density of the eigenvalues of Φm . . . . . . . 61A.4 The O-U type processes . . . . . . . . . . . . . . . . . . . . . 68vList of Tables2.1 Notation of variables . . . . . . . . . . . . . . . . . . . . . . . 123.1 Optimal control for different feedbacks . . . . . . . . . . . . . 24viList of Figures1.1 The evolution of the angular displacement θ(t) in system (1.1)with θ(0) = 1, with delay time τ = 0.2, ta = 0.04. . . . . . . . 31.2 The comparison of the time series for the angular displace-ment θ(t) in system (1.1) for different values of τ and ta. . . . 52.1 The stability region in the b− c plane. . . . . . . . . . . . . . 152.2 The comparison of stability regions using the normalized pa-rameters p = P × ta and d = D × ta. . . . . . . . . . . . . . . 182.3 The determinant of matrix Rm and the interior angles of sta-bility region vs τ . . . . . . . . . . . . . . . . . . . . . . . . . . 202.4 The evolution of the system (1.1) under the PWC control,τ = 2, ta = 0.4 for different decay rates and eigenvalues. . . 213.1 Simulations of the system (1.1) with different feedback cases. 254.1 The probability density function of the critical eigenvalue r. . 394.2 The probability density function of the decay rate ρ for dif-ferent values of τ . . . . . . . . . . . . . . . . . . . . . . . . . . 404.3 The probability density function of the decay rate ρ for dif-ferent values of ta. . . . . . . . . . . . . . . . . . . . . . . . . 424.4 Left: The probability density of θ(t) at different times. Mid-dle and Right: The distribution features: the mean, the stan-dard deviation of θ(t). . . . . . . . . . . . . . . . . . . . . . . 424.5 Comparison of the CV model and PWC model given samemagnitude of parametric noise. . . . . . . . . . . . . . . . . . 434.6 The probability density of the external noise driven system. 47viiList of Figures4.7 The amplification factor of the variance of θ(t) . . . . . . . . 48viiiAcknowledgementsThanks for my supervisor, Prof. Rachel Kuske, for her kind support. Ialso want to thank for MITACS and UBC FoGS for their contribution inMITACS Globalink Fellowship Award which supports my research.ixDedicationTo my parents.xChapter 1IntroductionDelays arise in feedback control systems when there is a significant timeinterval between when a variable is measured and when corrective forces areapplied. The presence of delay and its interplay with random perturbationsleads to instability or poor performance of the system. For example, in spaceapplications [9], there is an unavoidable two-way communication time delaybetween a ground control station and the telerobotic device working in lowEarth orbit. This round-trip communication time delay varies between fourto eight seconds and prevents any effective control interaction between theground operator and the telerobotic device in space. Moreover, for somerobotic applications such as the medical robots described in [11], the delayarises from time-consuming control force computations, which are essentialto ensure the reliable and safe interaction between robots and humans.The main difficulty of time-delayed systems is that the number of in-stabilities to be controlled is usually larger than the number of control pa-rameters. Thus, complete stabilization is impossible for these systems usingtraditional time-invariant feedback controls. Therefore, time periodic con-trol is the most common solution to deal with this difficulty. In [8], Khar-gonekar et al. showed that compared to time-invariant controls, periodiccontrols (time-varying controls) in many cases significantly improve the ro-bustness of the feedback system. Michiels et al. [13] improved the limitationof time-invariant output feedback in the stabilization by introducing delaysand time-varying gains. On the other hand, the stability can be improvedby state-dependent on-off controls. Asai et al. [2] introduced an intermit-tent controller in which the feedback is intermittently switched on and offaccording to a switching mechanism defined in the phase plane. This con-troller, introduced in [2], addressed the potential instability induced by large1Chapter 1. Introductiondelay, in contrast to the traditional time-invariant feedback controls. In thisthesis, we focus on a special case of time-periodic controls: the act-and-waitcontrol introduced in [6], [18].The concept of act-and-wait control states that the control is switchedon for a sampling period (acting period), then it is zero for a certain numberof periods (waiting periods), then it is switched on again, etc. Comparedto the traditional control with continuously applied delayed feedback (asin [6], [18], which we call continuous control), the act-and-wait control hasadvantages due to the larger stability region of control parameters and thepotential deadbeat control. Deadbeat control is the control under whichall of the eigenvalues of the system are zero, so that the system dampsrapidly to zero rather than decaying gradually, as it does for the traditionalcontinuous control. As we show in §4, for a two dimensional system, thesystem under deadbeat control converges to zero equilibrium from any initialperturbation within at most two on-off cycles. Thus, the deadbeat controlis desirable because the system converges to an equilibrium faster underdeadbeat control than continuous control. This contrast is shown in Figure1.1, which shows the angular displacement θ(t) of an inverted pendulumwith controller u(t) [17], as given by the linearized equationθ¨(t)− θ(t) = u(t), (1.1)which converges to equilibrium given the same initial condition θ(0) = 1.Here, θ is a dimensionless quantity representing the angular displacementof pendulum from vertical placement. Clearly, the trivial solution θ = 0 inequation (1.1) describes the equilibrium to be stabilized. The controller u(t)consists of a switching factor g(t) and the delayed feedback F (t− τ), whichdepends on θ(t − τ) and θ¨(t − τ) for the delay τ . Under the effect of theswitching factor g(t), the control is first switched off for a waiting period oflength tw and then is switched on with an acting period of length ta. Asdiscussed in detail in §2, the system is stabilized through an appropriatechoice of control parameters P and D which serve as part of the delayedfeedback F (t− τ) as defined in (2.3), (2.4).2Chapter 1. IntroductionWe compare the behavior of the traditional control with continuouslyapplied delayed feedback (tw = 0), which is not deadbeat control, withother cases under act-and-wait control (tw > 0). As shown in Figure 1.1,deadbeat control is achieved for those cases with act-and-wait control wherethe waiting period is longer than the delay time. If tw = τ , the systemconverges at the fastest rate. Hence, in this thesis, we always assume thattw = τ for act-and-wait control, unless otherwise noted.Another advantage of act-and-wait controller has been verified in [10],suggesting that stability can be achieved over a larger range of the feedbackdelay time for act-and-wait control. In [10], if the delay exceeds a certainvalue, the system is unstable under continuous control while it is still stableunder act-and-wait control. In [7], Insperger and Ste´pa´n showed that forcertain delays, the sampling period (∆t) has to be small enough to stabilizethe system under continuous control while with act-and-wait control, youcan always obtain a deadbeat control for any ∆t.0 0.5 1 1.5 2 2.5 3 3.5 4−0.200.20.40.60.811.2tθ(t)Figure 1.1: The evolution of the angular displacement θ(t) in system (1.1) withθ(0) = 1, with delay time τ = 0.2, ta = 0.04. The black dotted line representsthe system with continuous control. And others are systems with act-and-waitcontrol. Blue dash dotted line: tw = 0.08 < τ . Red dashed line: tw = 0.2 = τ .Magenta solid line: tw = 0.28 > τ . Deadbeat control is achieved for cases withtw ≥ τ , presented by the red dashed line and magenta solid line, and the strongestdeadbeat control occurs for tw = τ = 0.2.In this thesis, we use the classical inverted pendulum model or balance3Chapter 1. Introductionmodel to study the effect of delay and noise on this act-and-wait mechanism.Stabilizing the inverted pendulum with delayed feedback is a classical prob-lem in human balancing tasks [14] and mechanical systems [12]. We discusstwo types of the act-and-wait control model. The first is the piecewise con-stant act-and-wait control model (the PWC model), as shown in [7]. This ismotivated by digital control models where control is applied as a piecewiseconstant in the interval of a length of the sampling time. The second typeis the frequently varying act-and-wait control model (the FV model), wherethe control is updated once in each feedback loop, in contrast to the PWCmodel where the control is updated at particular sampling times. The FVmodel updates the control more than once during the time interval withinwhich the control is switched on. The FV model is applicable in biologicalmodels where the delayed feedbacks are more frequent or continuous. Thecontinuously varying act-and-wait control model (the CV model) serves asan extreme of the FV model as the frequency of updates approaches infinity.The continuously varying act-and-wait control (the CV model) is studied in[6]. From now on, the FV model refers to the act-and-wait model withfrequently varying feedback. The PWC model refers to the act-and-waitcontrol with piecewise constant feedback updated once in the active period.The CV model refers to the act-and-wait control with continuously varyingfeedback.In this thesis we focus on the FV model since it captures both the PWCmodel and the CV model as two of its extreme cases. It has been shown thatwhen the delay is small, there are no obvious differences between the PWCmodel and the CV model. For example, we can apply the deadbeat controlobtained from the PWC model to the CV model mechanism to stabilize thesystem. However, when the delay is large, it is inappropriate to apply theresults from the PWC model to the FV model or CV model since it may leadto unstable behavior. Here, the FV model is applied to approximate the CVmodel in the computational approach and analyze the difference betweenthe PWC model and the CV model. At the same time, the FV model isused to compare the PWC model and the CV model in the act-and-waitcontext in terms of stability, sensitivity and robustness. We also analyze the4Chapter 1. Introductioneffect of more frequent updates on robustness in the presence of noise in thecontext of the FV model.Beside the significant difference between the PWC model and the CVmodel, other interesting phenomena arises when the delay gets larger. Asshown in Figure 1.2 (a), we can see that the time series of θ(t) in (1.1) allexhibit deadbeat behavior, yet large delays reduce the rate of convergenceto the equilibrium given the same initial conditions. Most importantly, theinteraction of delay and noise results in oscillations, as shown in Figure 1.2(b), (c). In Figure 1.2 (b), with parametric noise, the system with largedelay exhibits damped oscillatory behavior instead of deadbeat damping.Figure 1.2 (c) shows the evolution of θ(t) driven by external noise, modeledas additive white noise in the equation of θ(t) (1.1), as discussed in detail in§4.2. We see from Figure 1.2 (c) that the external noise drives the systeminto a sustained oscillation the amplitude of which increases as τ increases,rather than converging to a steady state equilibrium.0 2 4 6 8 10−2−1012345 (a)tθ(t)0 2 4 6 8 10−2−1012345 (b)tθ(t)0 5 10 15 20 25 30−2−1012345 (c)tθ(t)Figure 1.2: The comparison of the time series for the angular displacement θ(t)in system (1.1) for different values of τ and ta. Blue solid line: τ = 0.2, ta = 0.04.Red dashed line: τ = 2, ta = 0.04. Black dash dotted line: τ = 2, ta = 0.4. Figure(a) shows the comparison of systems under deadbeat control P = P ∗, D = D∗.Figure (b) shows the comparison of systems with random control parameters P =P ∗ + .05ξ1, D = D∗ (ξ1 ∼ N(0, 1)). Figure (c) shows the comparison of systemswith external noise (white noise with factor δ = 0.02).The purpose of this thesis is to investigate the effect of act-and-wait con-trol in the context of the more general model, the FV model. Under the widescope of the FV model, we studied the influence of the delay, different choicesof the acting period length ta, the frequencies of varying feedbacks, and thenoise sensitivities through both analytical and numerical approaches. In §2,5Chapter 1. Introductionwe give the computational analysis of stability regions for different valuesof τ and ta. In §3, we introduce the FV model and compare the PWC, FVand CV models in the context of deadbeat controls and eigenvalues. In §4,we investigate the noise sensitivity and the interaction of noise and delayfor two different sources, parametric randomness and external fluctuations.The goal is to understand novel dynamics resulting from the combined ef-fects of time-delay, on-off control and noise sensitivity which are generic andtherefore expected to be prevalent in a wide range of balancing systems.The remainder of this thesis is organized as follows. The basic modelof an inverted pendulum with controller u(t) is presented in §2.1. We de-compose the controller u(t) into two parts: a switching factor g(t) and thedelayed feedback F (t− τ). The switching factor g(t) distinguishes the act-and-wait control from the traditional continuous control. We introduce twokinds of feedback: the continuously varying feedback and the piecewise con-stant feedback. These two different feedback mechanisms distinguish thePWC model from the CV model. In §2.2, we give the mathematical expres-sion of the CV and PWC models in piecewise differential equations and thecorresponding analytical solutions. In §2.3, we analyze the stability of theCV and PWC models through their stability regions and deadbeat controls.We discuss the effect of large delay by studying how it changes the size andshape of the stability regions. These changes in the stability regions makethe system sensitive to parametric and external noise. We compare dead-beat control parameters and stability regions for different choices of ta whenthe delay is large.In §3, we introduce the FV model, which captures the CV and PWCmodels as two of its extreme cases. In §3.1, we compare the differencesbetween the CV and PWC models, which are problematic when τ is large.This gives us the motivation to consider a more general model, the FV model.In §3.2, we give the mathematical analysis of the FV model and comparethe CV, PWC and FV models for small ta  1 and general ta < 1 for largeτ . We give the general analytical solutions for the deadbeat parametersP ∗, D∗ and discuss the asymptotic behavior of eigenvalues and deadbeatparameters.6Chapter 1. IntroductionIn §4, we consider the noise sensitivity and the interaction of noise anddelay. In §4.1, we explore the effect of parametric noise on the stabilizationof the act-and-wait model. We study the probability density of the criti-cal eigenvalues and the decay rate under parametric noise. We find thatparametric noise can reduce the convergence rate to the equilibrium, par-ticularly for larger values of τ . The interaction of delay and external noiseis studied in §4.2. We analyze the system with external noise via two al-ternating Ornstein-Uhlenbeck (O-U) type processes in the waiting periodand the acting period. We integrate the O-U type process and calculate theexpectation and variance of θ for the PWC model for specific times thatare integer multiples of the period. We also study how the delay inducesthe amplification of the input noise. Finally, a summary and discussion arepresented in §5.7Chapter 2The Model for act-and-waitcontrol2.1 Basic model with continuously varyingcontrolWe consider the task of vertically balancing an inverted pendulum. Upon anappropriate time scaling and linearization, the equations of motion aroundθ = 0 corresponding to the pendulum at vertical may be written as [17]θ¨(t)− θ(t) = u(t),u(t) = g(t)F (t− τ). (2.1)Here θ is a dimensionless quantity representing the angular displacement ofthe stick from vertical θ = 0. Clearly, the trivial solution θ = 0 in equations(1.1) describes the equilibrium to be stabilized. The controller u(t) consistsof a switching factor g(t) with values 0, 1 and the delayed feedback F (t−τ).In §2 and §3, we first consider the deterministic model before studying theeffects of noise in §4.This linearized equation (1.1) around θ = 0 describes the motion ofan inverted pendulum with an applied torque [14], [19], where F denotespivot control force. The model (1.1) also provides a simple model of humanpostural sway where F represents ankle torque [2]. In other contexts, (1.1)also captures the motion of a vertical rod controlled by a moving cart [12],[15] if the cart is much more massive than the pendulum.82.1. Basic model with continuously varying controlThe factor g(t) is an on-off switching factor which takes the values of 1or 0. When g(t) is 0, no control is applied to the system. If g(t) = 1, thecontrol is switched on and takes the value of F . In general, for act-and-waitcontrol, the control alternates between on and off. The length of periodwhen control is on is ta while the length of period when control is off is tw.The mathematical definition of g(t) isg(t) ={0, if lT < t ≤ tw + lT, l ∈ Z1, if tw + lT < t ≤ tw + ta = (l + 1)T , (2.2)where T = tw + ta denotes the period of act-and-wait control loop andl is an integer. In traditional continuous control [14], [16], the control iscontinuously applied to the system without a waiting period, correspondingto tw = 0 and g(t) = 1. Then the control force F is continuously appliedto the system. The controller u(t) = g(t)F (t) with g(t) = 1 is called thecontinuous controller [2], [7].The force F is often considered as an autonomous feedback with delaycomposed of a linear combination of the position θ(t− τ) and angular speedθ˙(t− τ) at time t− τ ,F (t− τ) = −Dθ˙(t− τ)− Pθ(t− τ), (2.3)where τ is delay. This kind of feedback in (2.3) is very common in biologicalapplications where the time delay in (2.3) is used to model the reaction timebetween perceiving a stimulus and initiating an control action, and is oftendue to the physical mechanism. For example, in the task of human balancingof a stick placed at the fingertip, by the time a person’s finger is trying tomove accordingly to control the stick, the finger’s position is being guidedby the information that is already 0.2s old [5], due to the time needed forthe electrical impulse to travel through nerve cells. At the time when thefinger moves to balance the stick, the stick may have slightly changed itsposition and angular speed. Thus the feedback given to balance the stick isalways delayed due to the reaction time modeled by τ in equation (2.3).Yet the feedback is not necessarily continuous as in (2.3) [15], [19]. In the92.2. Continuously varying vs piecewise constant act-and-wait controldigital force control system where digital computers act as system controllers[1], [4], [9], the feedback is only updated once at time tj over a fixed samplingperiod [tj , tj+1). In this case, we write the feedback asF (t− τ) = −Dθ˙(tj − τ)− Pθ(tj − τ) for t ∈ [tj , tj+1). (2.4)The delay τ in (2.4) models the time-consuming control force computations[9] that are essential before application of the control. For example, thesemay be necessary for the reliable and safe interaction between robots andhumans.2.2 Continuously varying vs piecewise constantact-and-wait controlIn [6] and [7], the act-and-wait control concept is introduced for systemswith continuously varying feedback (CV) and piecewise constant feedback(PWC) respectively. The point of the method is that the switching factorg(t) defined in (2.2) is periodically switched on (act) and off (wait). It isnatural for a delayed system to have a waiting period in order to wait longenough to see the effect of the feedback. It is shown that if the duration ofwaiting (when the control is turned off) is larger than the feedback delay,then the system can be better stabilized than those cases with tw < τ .Here we introduce the mathematical form of the CV controller gF whereg is defined in (2.2) and F as in (2.3) with ta > 0 and tw ≥ τ . Then thedelayed feedback term is switched off for a period of length tw (wait), and itis switched on for a period of length ta (act). If we denote x = (θ, θ˙)T , thensystem (1.1) with continuously varying feedback F (2.3) can be written inthe time periodic DDE form,x˙(t) = A˜x(t) + g(t)B˜Dx(t− τ), (2.5)102.2. Continuously varying vs piecewise constant act-and-wait controlwhereA˜ =(0 11 0), B˜ =(01), D = (−P,−D). (2.6)Similarly, for PWC feedback (2.4), we havex˙(t) = A˜x(t) + g(t)B˜Dx(tj− τ), t ∈ [tj , tj+1), tj = j×∆t, j ∈ Z, (2.7)with matrices defined as in (2.6). Notice here that the duration of theoff period tw and on period ta are chosen to be multiples of the samplinginterval ∆t in (2.7). For the digital control system where the PWC modelis generally applied, sampling is the reduction of a continuous signal toa discrete signal. Sampling is performed by measuring the value of thecontinuous function every ∆t seconds, which is called the sampling interval.It is natural to combine tw and ta into the sampling period of system (2.7).Also this enables us to convert system (2.7) into a discrete system asx(tj+1) = Ax(tj) + g(tj)BDx(tj − τ), (2.8)whereA = exp(A˜∆t) =(cosh(∆t) sinh(∆t)sinh(∆t) cosh(∆t)),B = (exp(A˜∆t)− I)A˜−1B˜ =(cosh(∆t)− 1sinh(∆t)),which we obtained by integrating (2.7) over the interval [tj , tj+1), tj =j ×∆t, j ∈ Z. The notation of the scheme is denoted in Table 2.1, which isused throughout the paper.In [6] and [7], it is shown that if tw ≥ τ , the system can be betterstabilized. So we assume tw ≥ τ and 0 < ta ≤ τ . In this case, (2.5) can beconsidered as an ordinary differential equation (ODE) in [lT, tw+ lT ) and asa DDE in [tw + lT, (l+ 1)T ). If t ∈ [lT, tw + lT ), then g(t) = 0 (the delayedfeedback in the control is turned off), and the solution of (2.5) associated112.2. Continuously varying vs piecewise constant act-and-wait controlTable 2.1: Notation of variablesVariable Descriptionh mesh size of simulations∆t the time length when feedback F is fixedτ time lagta the duration when control is ontw the duration when control is offT the duration of one control periodm = ta∆t the ratio of ta to ∆tn = tw∆t the ratio of tw to ∆tk = T∆t the ratio of T to ∆twith the initial state x(lT ), can be written asx(t) = eA˜tx(lT ), t ∈ [lT, tw + lT ). (2.9)If t ∈ [tw + lT, (l + 1)T ), then g(t) = 1 (the delayed feedback is switchedon). Since ta ≤ τ , and the solution over the interval [lT, lw + lT ) is alreadygiven by (2.9), system (2.5) can be written in the formx˙(t) = A˜x(t) + B˜DeA˜(t−τ)x(lT ), t ∈ [tw + lT, (l + 1)T ). (2.10)Solving (2.10) as an ODE over [tw+lT, (l+1)T ) with x(tw+lT ) = eA˜twx(lT )as an initial condition, we obtainx((l+1)T ) = ΦCV x(lT ), ΦCV = eA˜T+∫ TtweA˜(T−s)B˜DeA˜(s−τ)ds. (2.11)The placement of eigenvalues of ΦCV at zero is not ensured because theydepend nonlinearly on the components ofD as shown in (3.15). The stabilityis obtained by choosing the components of matrixD to give eigenvalues withmodulus ≤ 1.For the PWC control, the system (2.7) has the same solution as shown in(2.9) for t ∈ [lT, tw+ lT ). If t ∈ [tw+ lT, (l+1)T ), then g(t) = 1 (the delayed122.3. Deadbeat controlfeedback in the control is switched on). Since the feedback F depends ontw only, we can write (2.7) in the formx˙(t) = A˜x(t) + B˜DeA˜(tw−τ)x(lT ), t ∈ [tw + lT, (l + 1)T ). (2.12)Solving (2.12) as an ODE over [tw+lT, (l+1)T ) with x(tw+lT ) = eA˜twx(lT )as an initial condition, we obtainx((l + 1)T ) = ΦPWCx(lT ), ΦPWC = eA˜T +∫ TtweA˜(T−s)B˜DeA˜(tw−τ)ds.(2.13)As shown in Appendix A.1, we can write ΦCV and ΦPWC asΦCV = Q(eta+τ − (P+D)2 etata(P−D)2e−ta−eta2(P+D)2eta−e−ta2 e−(ta+τ) + (P−D)2 e−tata)Q′ = QΛ∞Q′,ΦPWC = Q(eta+τ + P+D2 (1− eta) P−D2 (1− eta)P+D2 (1− e−ta) e−(ta+τ) + P−D2 (1− e−ta))Q′ = QΛ1Q′,whereQ =1√2(1 11 −1). (2.14)2.3 Deadbeat controlAs a result from [6] and [7], it is possible to obtain deadbeat control by anoptimal choice of the constants P and D appearing in (2.3), (2.4) in act-and-wait control for tw ≥ τ . The system (2.5) or (2.7) is asymptotically stableif all of the eigenvalues of matrix ΦCV or ΦPWC , are inside the unit circleof the complex plane. The decay rate ρ is defined in terms of the criticaleigenvalue r,r = max{|λi|, i = 1, 2, · · · },ρ = r1T . (2.15)132.3. Deadbeat controlwhere λi(i = 1, 2) are the eigenvalues of matrix ΦCV or ΦPWC . The place-ment of the eigenvalues of ΦPWC at zero is determined using their lineardependence on the components of D as shown in (2.19). The optimal pa-rameters P ∗, D∗ at which the smallest decay rate is obtained are given byρ(P ∗, D∗) = min∀P,D{ρ(P,D)}. (2.16)If ρ(P ∗, D∗) = 0, the deadbeat control that is desired is obtained. We callP ∗, D∗ the deadbeat control parameter.For the sake of convenience, we introduce the intermediate variables band c as follows,b = Trace(ΦPWC) = λ1 + λ2, c = 4×Determinant(ΦPWC) = 4λ1λ2,(2.17)where λ1, λ2 (|λ1| ≥ |λ2|) are the two eigenvalues of ΦPWC . The stabilityregion of matrix ΦPWC is bounded by three straight lines: c = 4, c =4b − 4, c = −4b − 4, corresponding to ρ = 1 as shown in Figure 2.1. Ingeneral, the contour plots for ρ = r1T in the b− c plane are formed by threestraight lines: c = 4r2, c = 4r(b− r), c = −4r(b+ r). Because we focus onthe decay rate ρ rather than the real eigenvalues, we express ρ in terms ofb, c,ρ =(12(b+√b2 − c))1T , if b ≥ 0 and b2 ≥ c(12(−b+√b2 − c))1T , if b < 0 and b2 ≥ c(√c/4)1T , if b2 < c. (2.18)Also, b and c are linear functions of P and D for tw ≥ τ .(bc)=(λ1 + λ24λ1λ2)= R1(PD)+ S, (2.19)142.3. Deadbeat controlwhereR1 =(1− cosh(ta) − sinh(ta)4(cosh(T )− cosh(τ)) −4(sinh(T )− sinh(τ))), S =(2 cosh(τ + ta)4).−3 −2 −1 0 1 2 3−505 bcFigure 2.1: The stability region in the b − c plane. The red line represents thecontour curve of ρ = 1. The green line: ρ = 0.71T . The blue line: ρ = 0.41T . Theorigin is denoted by the black star indicating the deadbeat control where ρ = 0.In order to get the deadbeat control, we set λ1 = λ2 = 0 ⇐⇒ b = c = 0.For example, for the PWC model, we have the following relationship betweenP , D and the eigenvalues of ΦPWC ,(λ1 + λ24λ1λ2)=(−2 sinh( ta2 )[sinh(ta2 )P + cosh(ta2 )D] + 2 cosh(τ + ta)8 sinh( ta2 )[sinh(τ +ta2 )P − cosh(τ +ta2 )D] + 4).For λ1 = λ2 = 0, we solve for P ∗, D∗ as follows,(P ∗D∗)= −R1−1 · S, (2.20)152.3. Deadbeat controlwhich yieldsP ∗ =cosh(2τ + 3ta2 )2 sinh( ta2 ) sinh(ta + τ),D∗ =sinh(2τ + 3ta2 )2 sinh( ta2 ) sinh(ta + τ). (2.21)Figure 2.2 shows the contour plots for the decay rates in the rescaledP − D plane with different τ and ta. We compare the stability regions ofcontinuous control (g = 1) with act-and-wait control as discussed in [7]. Forthe case with the control on for all time (g = 1) for small τ = 0.2 withminimized decay rate ρ∗ = 0.1944 > 0, we notice that the stability areais a D-shaped area, similar to the original model described in [19]. Thesubplot Figure 2.2 (b) shows the stability region of the act-and-wait PWCmodel (2.7). We get a larger stability region and deadbeat control ρ∗ = 0indicating that the act-and-wait control improves the stability. The blackstar in subplot (b), (c), (d) corresponds to deadbeat control parameters P ∗,D∗ for the act-and-wait control.Also as shown in the bottom row of Figure 2.2, the stability regionsare quite different for different values of ta = ∆t. First of all, when ta =∆t decreases, the deadbeat control parameters P ∗, D∗ grow approximatelylinearly with 1/ta because the control is turned on for a shorter length oftime. In order to damp the system to the equilibrium, the control force gF(2.4) has to be larger in the shorter active period. This behavior can be seenfrom the asymptotic behavior of (2.21) for 0 < ta  1,P ∗ ≈12 sinh( ta2 )cosh(2τ)sinh(τ)1 + tanh(2τ)3ta2 +O(t2a)1 + coth(τ)ta +O(t2a)≈1tacosh(2τ)sinh(τ)(1 + C1(τ)ta)D∗ ≈12 sinh( ta2 )sinh(2τ)sinh(τ)1 + coth(2τ)3ta2 +O(t2a)1 + coth(τ)ta +O(t2a)≈1tasinh(2τ)sinh(τ)(1 + C2(τ)ta).(2.22)If τ > 1, C1(τ) ≈ C2(τ) ≈ 0.5. In general, (1+C1(τ)ta), (1+C2(τ)ta) is a162.3. Deadbeat controlcorrection term near 1 for small ta. As we can see from (2.22), the deadbeatcontrol parameters P ∗, D∗ depend on the factor1tawhich becomes large forsmall ta. So we use the scaled parameters p = P×ta ≈cosh(2τ)sinh(τ)(1+C1(τ)ta)and d = D× ta ≈sinh(2τ)sinh(τ)(1 +C2(τ)ta) in Figure 2.2 to depict the stabilityregions in order to compare different cases.For the case when the delay is large τ = 2, the stability regions in Figure2.2 (c) and (d) are both thin. The rescaled stability region moves upwardand to the right when ta gets larger because there is a correction factor(1 + .5ta) in (2.22) which increases with ta. However, for this τ , it is hardto tell which choice of ta is better simply based on the size and the locationof the stability region. In the later section §3.1, we explore the behavior ofcertain performance measures for different models as a function of ta.After the rescaling, the stability region shown in Figure 2.2 is largelydecided by the delay τ . As τ increases, the stability regions become smalland thin in the p− d plane as shown in Figure 2.2 (c) and (d). The changeof the shape of the stability region can be explained in (2.19) by recognizingthat R1−1 is the transformation of the generic stability region in the b-cplane (Figure 2.1) to the stability region in the P-D plane (Figure 2.2),affecting both the overall area and the interior angles of the stability region.First of all, the compression factor det(R1) represents the ratio of thearea of the stability region in the b− c plane to the one in the P −D plane,withdet(R1) = 16 sinh2(ta2) sinh(τ + ta). (2.23)The larger det(R1) is, the smaller the stability region in P and D is. Aswe can see from (2.23), det(R1) grows exponentially with τ , also shown inFigure 2.3. Thus the area of stability region in the P − D plane becomessmaller overall as τ increases. Then for larger values of τ , there is a greatervariation in the decay rate ρ for small variations in control parameters devi-ating from P ∗, D∗. This change in det(R1) adds to the sensitivity to bothparametric and external noise when τ is large, as can be seen by comparingFigure 2.2 (b) and (c). Note that the general form of Rm shown in Figure172.3. Deadbeat controlpd(a)0 1 2 300.10.20.30.40.5pd(b)0 10 2001234ψ2ψ1ψ3pd(c)6 7 8 9 10 1167891011pd(d)6 7 8 9 10 1167891011Figure 2.2: The comparison of stability regions using the normalized parametersp = P × ta and d = D × ta. The brown line shows the stability boundary wherethe decay rate is ρ = 1. The green line shows the contour curve where ρ = 0.7.The blue line shows the contour curve where ρ = 0.4. The black star shows wherethe deadbeat control parameters lie (P ∗, D∗ corresponds to ρ = 0). Top and Left:The system with continuously applied PD controller τ = 0.2, minimized decay rateρ∗ = 0.0805. Top and right: The PWC model with small delay, τ = 0.2, ta =0.04, ρ∗ = 0. Bottom and left: The PWC model with large delay τ = 2, ta = 0.04,ρ∗ = 0. Bottom and right: The PWC model with large delay τ = 2, ta = 0.4,ρ∗ = 0.182.3. Deadbeat control2.3 is discussed in §3 for which R1 is a special case.Also, as we can see from (2.23), for small ta and τ  ta, the determinantdet(R1) can be approximated by det(R1) ≈ 4 sinh(τ)t2a given a fixed value ofτ . This explains why the stability region in Figure 2.2 (d) (τ = 2, ta = 0.4)is smaller than the stability region in Figure 2.2 (c) (τ = 2, ta = 0.04). Thisalso implies that we can not choose large values for the acting period lengthta, since otherwise we would get a smaller stability region.Second, the transformation (2.19) also changes the interior angles of thestability regions. The interior angles are the angles between the vectorsV1, V2, V3 which form the stability boundary in P and D as shown inFigure 2.2. The vectors Vi are given by (2.19) and the vectorsv1 = (1, 4)′, v2 = (−1, 4)′, v3 = (−4, 0)′,which are the three vectors lying on the stability boundary in the b−c planeas indicated by the red triangle in Figure 2.1,Vi = R1−1vi =−18 sinh(ta/2) sinh(τ + ta)(4 cosh(τ + ta/2) − cosh(ta/2)4 sinh(τ + ta/2) sinh(ta/2))vi.The three interior angles of the stability region ψ1, ψ2, ψ3 are then deter-mined bycos(ψ1) =(V1, V2)|V1| × |V2|= − tanh(τ)cos(ψ2) =(V2, V3)|V3| × |V3|=√cosh(3τ + ta) + 1cosh(3τ + ta) + cosh(τ + ta)cos(ψ3) =(V1,−V3)|V2| × |V3|=√cosh(3τ + ta)− 1cosh(3τ + ta) + cosh(τ + ta). (2.24)Then the largest interior angle of the stability region (ψ1 in Figure 2.2(b)) depends only on τ regardless of the value of ta. As τ increases, ψ1increases to pi while others decrease (ψ2, ψ3 → 0 in Figure 2.2 (b)) as shownin Figure 2.3. Then there is a thin triangular stability region with a large192.3. Deadbeat controlobtuse angle and two small acute angles when τ is large, with the contourcurves in Figure 2.2 squeezed together. As a result the decay rate ρ increasesrapidly with variation of P , D away from P ∗, D∗. Hence, the system issensitive to both parametric and external noise.0 0.5 1 1.5 2 2.5 3051015202530τdet(Rm)0 0.5 1 1.5 2 2.5 300.511.522.533.5τψ iFigure 2.3: Left: The determinant of matrix Rm vs τ . ta = τ/5 varying alongwith τ . Note that the general form of Rm is discussed in §3 in which R1 is a specialcase. The red marked line with circle: m = 1, that is, the PWC case. The blackmarked line with star: m = 10. The blue marked line with square: m = ∞, thatis, the CV case. Right: The interior angles of the stability region shown in Figure2.2 (b), (c), (d). The red marked line with triangle: the obtuse interior angle ψ1(2.24). The black marked line with square: the acute interior angle ψ2 (2.24). Theblue marked line with star: the acute interior angle ψ3 (2.24).In reality, the control force may not be executed exactly at the deadbeatcontrol parameters P = P ∗ and D = D∗, but may include some errorsbetween P and D and the deadbeat control parameters P ∗, D∗. Based onthe above analysis, we conclude that for the case with small τ , not only isthe rescaled stability region larger, but also the contour curves around thedeadbeat control are less close together. Then the system is less sensitive toparametric noise or controlling force error.For small τ , despite of the errors in P , D, the decay rate remains small inthe neighborhood of P ∗, D∗, and the system has nearly deadbeat dampingbehavior. However for large τ , a small error in P or D can lead to a largeincrease in decay rate. Instead of deadbeat damping behavior, the systemdecays slowly to the equilibrium which gives the system potential sensitivityto noise (Figure 1.2). This point is studied in detain in §4.202.3. Deadbeat controlSimilarly we can see from Figure 2.4 that for ρ near the critical valueρ = 1, the system can oscillate strongly before converging to zero. If ρgets close to 1, the transient of the system to zero can be very different fordifferent eigenvalues given the same decay rate. From Figure 2.4, we cansee that those with zero traces, that is, λ1 + λ2 = 0, oscillate less and decayfaster than the ones with nonnegative traces (shown by the magenta dashdotted line and the red marked line with +’s) even though they have thesame value of decay rate ρ = 0.7.0 10 20 30 40−6−4−20246 tθ(t)Figure 2.4: The evolution of the system (1.1) under the PWC control, τ = 2, ta =0.4 for different decay rates and eigenvalues. The blue dashed line shows the casewith deadbeat control with zero decay rate and zero eigenvalues, ρ = λ1 = λ2 = 0.Other lines have the same decay rate ρ = 0.7 but different values of eigenvalues.The black marked line with x’s: λ1 = 0.7i, λ2 = −0.7i. The green solid line:λ1 = 0.7, λ2 = −0.7. The magenta dash dotted line λ1 = 0.7, λ2 = 0.7. The redmarked line with +’s: λ1 = −0.7, λ2 = −0.7.21Chapter 3Cases with frequentlyvarying feedback3.1 Formulation and comparisonIn §2.3, we notice that for large values of τ , the stability region of the PWCmodel shrinks and becomes thin (Figure 2.2). Thus the system is much moresensitive to errors in control parameters. Moreover, though very simple, thePWC model is limited when delay is large. If we use the PWC model, thenwe have a choice of ta. However, as we can see from Figure 2.2, our choiceof ta may not necessarily yield a preferred change in stability. Althoughzero decay rate is always obtained by choosing the right P ∗ and D∗, thereare rapid changes in the decay rate ρ in the neighborhood of P ∗ and D∗ forlarger values of τ .Furthermore, the PWC model is only applicable to the system withdigital control. Yet in some biological applications, the feedback may becontinuous, as shown in (2.3), or updated very frequently within the actingperiod of length ta.For small delay, when we compute the deadbeat control parameters P ∗,D∗ (2.16) for two different models, the PWC model (2.7) and the CV model(2.5), we see that they are close in value, for reasonable values of actingperiod length ta. Then, if we interchange deadbeat control parameters P ∗,D∗, we observe nearly deadbeat damping behavior. This is because, aswe see from the stability region (Figure 2.2), the contour lines for ρ areevenly spread out. Any slight variation in P , D near the deadbeat controlparameters P ∗, D∗ still corresponds to a small value of decay rate, which223.1. Formulation and comparisongives near deadbeat damping behavior.However, if the delay τ is large, the comparison of the PWC model(2.7) and the CV model (2.5) depends on a number of factors. First, if weconsider the CV model that varies continuously in time, we would expectthat any approximation to it should include small ∆t since ∆t denotes thetime between observations in signal processing. Hence, if we take n =τ∆tlarge (as defined in Table 2.1), ta = ∆t is small in the PWC model (2.7) andthe deadbeat control parameters and the stability region of the PWC model(2.7) are almost the same as the CV model (2.5) with the same ta. We cansee from the matrices ΦPWC (2.13) and ΦCV (2.11), that the eigenvalues ofthese two matrices are almost the same when ta is small (shown in (3.12)).However, as we discussed before in §2.3, for small values of ta, the deadbeatcontrol parameters P ∗ and D∗ grow linearly with 1/ta in order to ensurethe stability of the system (2.7), (2.5). This makes the control force F in(2.3) and (2.4) larger. Nevertheless, in reality, the control force cannot betoo large due to physical limitations. Moreover, for small ta, the system isvery sensitive to parametric noise, since the control is turned on for a shorttime, as shown in Figure 1.2.Instead, if we consider small to moderate values of n =τ∆t, there isa significant difference between matrices ΦPWC (2.13) and ΦCV (2.11) forthe PWC model (2.7) and the CV model (2.5), which results in differentvalues of P ∗, D∗, as shown in Table 3.1. Therefore, we would not want toapproximate the deadbeat control parameters for the CV model by the P ∗,D∗ from the PWC model, since this difference leads to instability when weapply P ∗, D∗ of (2.5) to (2.7) and vice versa as shown in Figure 3.1. Inthe left column of Figure 3.1, as we apply the deadbeat control parametersP ∗ = 22.6011 and D∗ = 22.5965 of the PWC model (2.7) to the CV model(2.5), θ(t) oscillates and diverges instead of converging to zero equilibrium.We observe similar results in the right figure of Figure 3.1. As we apply thedeadbeat control P ∗ = 18.6285 and D∗ = 18.6232 of the CV model (2.5) tothe PWC model (2.7) , the PWC model (2.7) diverges too.To summarize, for systems with large delay τ , we have to make ta largerin order to have a realistic control force. This means n (Table 2.1) cannot be233.1. Formulation and comparisontoo large for the PWC model. Yet for small to moderate values of n (Table2.1), the PWC model cannot be used to model the CV model because itleads to instability.Additionally, according to the Nyquist criterion [3], the sampling periodlength ∆t has to fall in a reasonable range to capture the information ofthe underlying system. Usually, the sampling frequency is no greater thanone-half of the natural frequency, which means we want ta ≥ 2∆t.Based on these observations, a more flexible feedback model is neededwhen considering cases with larger values of τ . We also want to modelsystems where the control varies continuously or frequently in time (ta∆t→∞). In that case we would not tie sampling size ∆t directly to active periodlength ta.Table 3.1: Optimal control for different feedbacksParameter τ = 0.2 τ = 2CV P ∗ = 113.09,D∗ = 46.88 P ∗ = 18.63,D∗ = 18.62PWC P ∗ = 114.28,D∗ = 49.14 P ∗ = 22.60,D∗ = 22.60Now we introduce a new act-and-wait model called the Frequent Varying(FV) model to allow the feedback to vary more frequently within the actingperiod (ta ≥ 2∆t). Hence, instead of the control being piecewise constantover ta, we subdivide ta into m pieces and let the feedback change m timeswithin time ta. More specifically, ta = m ·∆t, where ∆t is the sampling size.The control is fixed within time ∆t. As in the PWC model (2.7), we havethe differential equation of the formx˙(t) = A˜x(t) + g(t)B˜Dx(tj− τ), t ∈ [tj , tj+1), tj = j×∆t, j ∈ Z. (3.1)The only difference here is that ta = m · ∆t, m ∈ N while for the PWCmodel, ta = ∆t, that is, m = 1. This change gives a more general model,where the feedback is updated more frequently than the PWC model . As243.1. Formulation and comparison0 0.2 0.4 0.6 0.8 1−0.500.511.5 tθ(t)0 0.2 0.4 0.6 0.8 1−0.500.511.5 tθ(t)0 2 4 6 8 10−505 tθ(t)0 2 4 6 8 10012345 tθ(t)Figure 3.1: Simulations of the system (1.1) with different feedback cases. The toprow: The delay is τ = 0.2 and the acting period length is ta = 0.04 for all the cases.The blue dash dotted line: the PWC model (2.7), that is, m = 1 in (3.1). Theblack marked line with +’s: the CV model (2.5), that is, m = ∞ in (3.1). Left:P = 114.28 and D = 49.14 (shown in Table 3.1), that is, the deadbeat control ofthe PWC model (2.7) is applied to all the cases represented by different lines inthe figure. Right: P = 113.09 and D = 46.88 (shown in Table 3.1), that is, thedeadbeat control of the CV model (2.5) is applied to all the cases in the figure.The bottom row: The delay is τ = 2 and the acting period length is ta = 0.4 forall the cases. The red dashed line: The PWC model (2.7), that is, m = 1 in (3.1).The green solid line: The FV model (3.1) with m = 2. The blue dash dotted line:The FV model (3.1) with m = 10. The black marked line with +’s: The CV model(2.5), that is m = ∞ in (3.1). Left: P = 22.60 and D = 22.60 (shown in Table3.1), that is, the deadbeat control of the PWC model (2.7) is applied to all thecases represented by different lines in the figure. Right: P = 18.63 and D = 18.62(shown in Table 3.1), that is, the deadbeat control of the CV model (2.5) is appliedto all the cases in the figure.253.1. Formulation and comparisonmentioned before in (2.8), we can rewrite (3.1) in a discretized systemx(j + 1) = Ax(j) + g(j)BDx(j − n). (3.2)For the sake of simplicity of further discussion, we denote the time systemby {x(j), j ∈ Z} where x(j) = x(tj), tj = j ×∆t, j ∈ Z, and g(j) = g(tj),where g(t) is the same as defined in (2.2). Then m =ta∆tand n =τ∆t, asdefined in Table 2.1.The frequency of feedback in the FV model (3.2) increases asm increases.When m −→ ∞, the system resembles the CV model (2.5). For the PWCmodel (2.7) , ta = ∆t, that is m = 1. Thus the PWC model (2.7) is oneparticular example of the FV model when m = 1. The CV model (2.5) isapproximated by the FV model with m −→ ∞. Thus, the FV model (3.2)is a more general model, which includes the PWC model (2.7) and the CVmodel (2.5) as two of its special cases. From now on, we can use the FVmodel to discuss the stability of general act-and-wait control.When the delay is small, there is no significant difference between caseswith different values of m. As two extremes, the CV model and the PWCmodel behave similarly, since they have similar deadbeat control parametersP ∗, D∗. If we apply the deadbeat control of the CV model to the PWCmodel or vice versa, robustness still is ensured, as shown in Figure 3.1.Yet for large delay, given different values of m, that is, depending on thefrequency with which the feedback is updated, the system (3.2) has distinctdeadbeat control parameters P ∗, D∗ corresponding to different values of m.This difference in P ∗, D∗ leads to instability if we exterchange the controlparameter values, as shown in Figure 3.1. If we apply the deadbeat controlP ∗ and D∗ of the PWC model (m = 1 in (3.2)) to the FV model (m ≥ 2 in(3.2)), the system (3.2) blows up, as shown in the left column of Figure 3.1.On the other hand, if we apply the deadbeat control parameters P ∗ and D∗of the CV model (m =∞ in (3.2)) to cases with different m, including theFV model and the PWC model in which m <∞ in (3.2), the system wouldoscillate more for smaller values of m, so that at some m, the system woulddiverge instead of converge to zero. This behavior can be seen in the PWC263.2. Mathematical analysis of FV modelmodel (m = 1) in (3.2), as shown in the right column of Figure 3.1. Next,we compare the FV, CV and PWC models through mathematical analysisin §3.2.3.2 Mathematical analysis of FV modelAs we observed in Figure 3.1, there are instabilities if we exchange thedeadbeat control parameters for the cases with different m. As an extreme,there is a significant difference of the deadbeat control parameters for thePWC model (m = 1) and the CV model (m = ∞) as shown in Table 3.1.To illustrate where the difference comes from, we look at the mathematicalexpression of the time variation of x over one feedback loop. First, weconsider one step in (3.2),x(n+m) = Ax(n+m− 1) + g(n+m− 1)BDx(m− 1),where n and m are defined in Table 2.1. We again apply (3.2) to x(n+m−1)to get,x(n+m) = A2x(n+m−2)+g(n+m−1)BDx(m−1)+g(n+m−2)ABDx(m−2).We then iterate (n+m) times until we get x(n+m) in terms of x(0),x(n+m) = An+mx(0) + g(n+m− 1)BDx(m− 1)+ g(n+m− 2)ABDx(m− 2) + · · ·+ g(n)Am−1BDx(0)+ g(n− 1)AmBDx(−1) + · · ·+ g(0)Am+n−1BDx(−n).Since g(n− 1) = · · · = g(0) = 0 according to (2.2), we havex(n+m) = An+mx(0) + g(n+m− 1)BDx(m− 1)+ g(n+m− 2)ABDx(m− 2) + · · ·+ g(n)Am−1BDx(0).273.2. Mathematical analysis of FV modelSubsequently, we combine terms above together with g(n+m− 1) = · · · =g(n) = 1 and x(m− i) = Am−ix(0) to getx(n+m) = [An+m +BDAm−1 +ABDAm−2 + · · ·+Am−1BD]x(0)= Φmx(0). (3.3)whereΦm = QΛmQ−1, Q =1√(2)(1 11 −1), (3.4)andΛm =(e(τ+ta) − P+D2 meta(1− e−∆t) P−D2 (1− e∆t) sinh(ta)sinh(∆t)P+D2 (1− e−∆t) sinh(ta)sinh(∆t) e−(τ+ta) − P−D2 me−ta(1− e∆t)).(3.5)For ∆t → 0 ( m → ∞), the form of (3.3) has a limit that correspondsto the CV model (2.5) withΦCV = Φ∞ = limm→∞Φm = QΛ∞Q−1, (3.6)where the corresponding matrix Λ∞ = limm→∞Λm isΛ∞ =(eτ+ta − P+D2 · ta · eta −P−D2 · sinh(ta)P+D2 · sinh(ta) e−(τ+ta) + P−D2 · tae−ta). (3.7)This is because, as shown in Appendix A.2, the Taylor expansion for largem isΛm ≈ Λ∞ +∆t2(P+D2 taeta −P−D2 sinh(ta)−P+D2 sinh(ta)P−D2 tae−ta). (3.8)Thus Λ∞ = limm→∞Λm = lim∆t→0 Λm since ∆t =tamgiven fixed ta.On the other hand, for m = 1, (3.3) refers to the PWC model (2.7) whereta = ∆t. We have shown in Appendix A.1 that matrix ΦPWC in (2.13) isequal to Φ1 in (3.3) when m = 1. Under the same decomposition as in (3.4),we haveΦPWC = Φ1 = QΛ1Q−1, (3.9)283.2. Mathematical analysis of FV modelwhere the corresponding matrix Λ1 isΛ1 =(e(τ+ta) + P+D2 (1− eta) P−D2 (1− eta)P+D2 (1− e−ta) e−(τ+ta) + P−D2 (1− e−ta)). (3.10)Hence, now we can see from the matrix formulation that the FV model,as given in (3.3), is a more general model for act-and-wait control. It cap-tures the PWC model (2.7) (m = 1) and the CV model (2.5) (m = ∞) astwo of its extreme cases.For ta  1, we compare Λm for two cases: m = 1 and m = ∞. First,we consider the asymptotic behavior of Λ1 defined in (3.10) by expandingthe term eta by Taylor series,Λ1 ∼(e(τ+ta) + P+D2 (1− 1− ta +O(t2a))P−D2 (1− 1− ta +O(t2a))P+D2 (1− 1 + ta +O(t2a)) e−(τ+ta) + P−D2 (1− 1 + ta +O(t2a)))=(e(τ+ta) − P+D2 (ta +O(t2a))P−D2 (−ta +O(t2a))P+D2 (ta +O(t2a)) e−(τ+ta) + P−D2 (ta +O(t2a)))=(e(τ+ta) − P+D2 (ta +O(t2a))P−D2 (−ta +O(t2a))P+D2 (ta +O(t2a)) e−(τ+ta) + P−D2 (ta +O(t2a)))=(e(τ+ta) − P+D2 ta −P−D2 taP+D2 ta e−(τ+ta) + P−D2 ta)+O(t2a). (3.11)Similarly, for the other extreme case, the CV model, we have the asymptoticbehavior of Λ∞ in (3.7) by expanding the term eta and sinh(ta) for ta ≈ 0as follows,Λ∞ =(eτ+ta − P+D2 · ta · eta −P−D2 · sinh(ta)P+D2 · sinh(ta) e−(τ+ta) + P−D2 · tae−ta)∼(eτ+ta − P+D2 · ta · (1 + ta +O(t2a)) −P−D2 · (ta +O(t3a))P+D2 · (ta +O(t3a)) e−(τ+ta) + P−D2 · ta(1− ta +O(t2a)))=(e(τ+ta) − P+D2 ta −P−D2 taP+D2 ta e−(τ+ta) + P−D2 ta)+O(t2a) ≈ Λ1. (3.12)293.2. Mathematical analysis of FV modelAs a result,Φ∞ = QΛ∞Q−1 ≈ QΛ1Q−1 = Φ1 for ta  1.Thus, the eigenvalues of the PWC model (2.7) (m = 1) and the CV model(2.5) (m = ∞) are almost the same for ta  1. If we interchange theirdeadbeat control parameters, the robustness is still assured, as shown in thetop row of Figure 3.1 when ta = 0.04 is small.However, for general ta < 1, although Λm → Λ∞ with a correction termof order O(1m) as shown in (3.8), the structural difference between Λm andΛ∞ is significant for small m. For the first entry of Λm and Λ∞,|Λm(1, 1)−Λ∞(1, 1)| =P +D2taeta(1 +e−∆t − 1∆t)=P +D2taeta(1 +e−tam − 1ta/m)> 0. (3.13)This difference (3.13) is large for ta < 1 and small m. Taking the casepresented in Figure 3.1(d) as an example, ta = 0.4, the difference |Λm(1, 1)−Λ∞(1, 1)| is 1.9539 for m = 1 and 1.0409 for m = 2. This is why we observethe significant difference between the CV model and the PWC model indeadbeat control parameters P ∗, D∗ in Table 3.1. As m increases, thedifference between Λm and Λ∞ is O( 1m), as shown in (3.8).Next, let us explore the value of m for which the FV model approximatesthe CV model. Assume that ||Λm−Λ∞||1 ≤ η so that the stability is assuredwhen we apply the deadbeat control parameters of the CV model to the FVmodel as shown in Figure 3.1(d). Then from (3.8), we have||Λm −Λ∞||1 ≈∆t2[P +D2taeta +P +D2sinh(ta)]=P +D4∆t[taeta + sinh(ta)]≤ η. (3.14)For example, for the case in Figure 3.1(d), if we choose η = 0.5 in (3.14),then we find that m ≥ 8 to meet the constraint in (3.14). As we increase m,303.2. Mathematical analysis of FV modelthe error in the eigenvalues becomes smaller and reduces the oscillation inthe time series. As a result, the FV model converges faster to equilibrium,as shown in Figure 3.1.Let us now examine the eigenvalues of the FV model. Recalling thatλm1 and λm2 are the eigenvalues of matrix Φm, then from (3.4) and (3.5), wehave(λm1 + λm24λm1 · λm2)= Rm(PmDm)+ S +Nm ·(0P 2m −D2m), (3.15)whereRm =(−m[cosh(ta)− cosh(ta −∆t)] −m[sinh(ta)− sinh(ta −∆t)]4m[cosh(τ + ∆t)− cosh(τ)] −4m[sinh(τ + ∆t)− sinh(τ)]),S =(2cosh(τ + ta)4), Nm = [2− 2cosh(∆t)][m2 −sinh2(ta)sinh2(∆t)].(3.16)Similar to the cases m = 1 (the PWC model) and m =∞ (the CV model),the deadbeat control is obtained at λm1 = λm2 = 0. Thus the deadbeatcontrol parameters P ∗m, D∗m of the FV model satisfyingRm(PD)+ S +Nm ·(0P 2 −D2)= 0. (3.17)As we noticed from equation (3.17), there is a nonlinear term which addscomplication to solving for P ∗, D∗. However, it is shown in Appendix A.2that the real roots of (3.17) exist for all m ≥ 1, including m = ∞ (theCV model). There are two sets of roots of (3.17), since it is a second ordersystem. We can rule out one set of roots of (3.17) (the larger one) becauseit is not physical (shown in Appendix A.2). The good news is that the otherset of the roots of (3.17) can be well approximated by the linear part of313.2. Mathematical analysis of FV model(3.17) by dropping the nonlinear term Nm · (0, P 2 −D2)′,(P ∗mD∗m)≈ −Rm−1 · S. (3.18)This is because the nonlinear term Nm is negligible. We consider the be-havior of Nm for ta < 1. First,N∞ = sinh2(ta)− t2a = (ta +O(t3a))2 − t2a = O(t4a). (3.19)Then for any fixed m, we claim that Nm = O(t4a). Since m∆t = ta, that is,∆t =tam≤ ta < 1, we have,Nm = [2− 2cosh(∆t)][m2 −sinh2(ta)sinh2(∆t)]= [2− 2cosh(tam)][m2 −sinh2(ta)sinh2( tam)]= −[(tam)2 +O((tam)4)][m2(( tam) +O((tam)3))2 − sinh2(ta)(( tam) +O((tam)3))2]= −[(tam)2 +O(t4a)][t2a +O(t4a)− sinh2(ta)( tam)2 +O(( tam)4)]=[( tam)2 +O(t4a)]( tam)2 +O(t4a)(sinh2(ta)− t2a +O(t4a))= [1 +O(t2a)][N∞ +O(t4a)] = O(t4a). (3.20)We get this result becausetam≤ ta and N∞ = O(t4a), as shown in (3.19).Since we assume 0 < ta < 1, t4a is very small for moderate ta, as compared tothe linear part in (3.17). Thus, from (3.19) and (3.20), we conclude that thenonlinear part of N∞ and Nm is negligible. Accordingly, the linear approx-imation (3.18) also holds for (P,D) around the deadbeat control (P ∗m, D∗m).Generally, around (P ∗m, D∗m), we have(λm1 + λm24λm1 λm2)≈ Rm(PD)+ S. (3.21)323.2. Mathematical analysis of FV modelTo find λm1 , λm2 , we must then analyze the behavior of Rm. Specifically, thePWC model corresponds to m = 1, as shown in (2.20), and the CV modelcorresponds to m =∞ withR∞ =(−ta sinh(ta) −ta cosh(ta)4ta sinh(τ) −4ta cosh(τ)). (3.22)We show here that,Rm ≈ R∞ +O(ta/m) = R∞ +O(∆t).For example, for the first entry of Rm(1, 1) in (3.16), taking ta fixed andm large with ∆t =tam→ 0 as m → ∞, we can conduct the asymptoticanalysis. For large m, ∆t 1, Taylor expansion isRm(1, 1) = −m(cosh(ta)− cosh(ta −∆t)) = −m∆t(cosh(ta)− cosh(ta −∆t))∆t= −m∆tcosh(ta)− [cosh(ta)− sinh(ta)∆t+ cosh(ta)∆t22 +O(∆t3)]∆t= −m∆tsinh(ta)∆t− cosh(ta)∆t22 +O(∆t3)∆t.Recall that ta = m∆t as shown in Table 3.1, we can rewrite the aboveasymptotic expansion of entry Rm(1, 1) for ∆t→ 0, by dropping the secondorder error term O(∆t2) as follows,Rm(1, 1) = −ta(sinh(ta)−cosh(ta)2∆t+O(∆t2)) ∼ R∞(1, 1)+tacosh(ta)2∆t.Generally, given fixed τ and ta, we have the asymptotic behavior ofRm whenm is large by dropping the second order error terms O(∆t2) ∼ O(1m2),Rm ∼(−ta sinh(ta) + tacosh(ta)2 ∆t −ta cosh(ta) + tasinh(ta)2 ∆t4ta sinh(τ) + 2tacosh(τ)∆t −4ta cosh(τ)− 2tasinh(τ)∆t)= R∞ +O(∆t).333.2. Mathematical analysis of FV modelCombining the results of (3.23) and (3.21), we have the asymptotic be-havior of the eigenvalues of the FV model as m→∞,(λm1 + λm24λm1 λm2)= Rm(PD)+ S = (R∞ +O(∆t))(PD)+ S= R∞(PD)+ S +O(∆t)(PD)=(λ∞1 + λ∞24λ∞1 λ∞2)+O(∆t). (3.23)Given (3.23), we know that the eigenvalues of the FV model λm1 , λm2 convergewith correction terms of the order of O(∆t) = O(tam) to the eigenvalues ofthe CV model λ∞1 , λ∞2 as m→∞. Therefore, we can use (3.18) to explainthe difference in deadbeat control parameters,Rm−1 =1det(Rm)(Rm(2, 2) −Rm(1, 2)−Rm(2, 1) Rm(1, 1))=1det(R∞) +O(∆t)(R∞(2, 2) +O(∆t) −R∞(1, 2) +O(∆t)−R∞(2, 1) +O(∆t) R∞(1, 1) +O(∆t))= R∞−1 +O(∆t),(P ∗mD∗m)∼ −Rm−1 ·S = (R∞−1 +O(∆t)) ·S =(P ∗∞D∗∞)+O(∆t). (3.24)As a result, as m increases, the deadbeat control P ∗m, D∗m → P∗∞, D∗∞ witherror O(∆t), and ∆t→ 0, as shown in (3.24).34Chapter 4Noise Sensitivity4.1 Parametric noiseAs shown in §1, parametric noise can reduce the convergence rate to theequilibrium, particularly for larger values of τ . In this section, we study theFV model with random fluctuations of the control parameters around thedeadbeat control parameters P ∗, D∗ as follows,P = P ∗ + u1P∗1 = P∗ + ξ1, D = D∗ + u2D∗2 = D∗ + ξ2. (4.1)where (1, 2) ∼ N(0, I2). The error in control parameters P and D cancause additional oscillations, as shown in Figure 1.2 (b) for large τ , not seenfor small values of τ . The system exhibits damped oscillations and slowerdecay to zero rather than abrupt damping, as shown in Figure 1.2 (a).This observation suggests that the decay rate ρ is larger for large valuesof τ when there are random fluctuations in the control parameters P , Daround the neighborhood of P ∗, D∗. We analyze this noise sensitivity bycalculating the probability density for the decay rate ρ. First, we have tocalculate the corresponding probability density for the critical eigenvaluer = max{|λm1 |, |λm2 |} where λm1 , λm2 are eigenvalues of the matrix Φm for thegeneral FV model (including the PWC model (m = 1) and CV model (m =∞)). Note that r = ρT indicates how strongly the system is decaying overone period of length T . As a two dimensional system, given the deadbeatcontrol without noise, so that r = 0, and the time series of θ(t) converges tozero within a 2T time interval from any initial condition, as proved belowin (4.26) and observed in Figure 1.2. However, if there is any noise, r 6= 0,it takes the system more than two on-off cycles (2T ) to converge to zero.354.1. Parametric noiseThus, the values of r are directly related to additional oscillations causedby noise and the number of periods the system takes to converge to zero.We consider random fluctuations of the control parameters around thedeadbeat control parameters P ∗, D∗, as shown in (4.1). Thus the ran-dom noise (ξ1, ξ2) has a joint two dimensional normal distribution (ξ1, ξ2) ∼N(0,Σ0), whereΣ0 =((u1P ∗)2 00 (u2D∗)2). (4.2)It is convenient to use the quantities: b = λm1 +λm2 , c = 4λm1 ·λm2 , similar tothe definition in (2.17). Then (b, c) has the following distribution,(bc)= Rm(P ∗ + ξ1D∗ + ξ2)+ S = Rm(ξ1ξ2)∼ N(0,RmΣ0RmT ). (4.3)The probability density function of (b, c) isP (b, c) =12pi√|RmΣ0RmT |exp(−(b, c)(RmΣ0RmT )−1(b, c)′/2). (4.4)where Σ0 is defined in (4.2).We then look at the distribution of the critical eigenvalue r = max{|λm1 |, |λm2 |},where λm1 , λm2 are eigenvalues of the matrix Φm. Using the following rela-tionship between r and (b, c),r =12(b+√b2 − c), if b ≥ 0 and b2 ≥ c12(−b+√b2 − c), if b < 0 and b2 ≥ c√c/4, if b2 < c, (4.5)we determine the probability density function of r by combining (4.4) and(4.5) as shown in Appendix A.3. Then the probability density p(r) is given364.1. Parametric noisebyp(r) =∫ 2r0(−4b+ 8r)P (b, 4r(b− r))db+∫ 0−2r(4b+ 8r)P (b,−4r(b+ r))db+∫ 2r−2r8rP (b, 4r2)db, (4.6)where P (b, c) is the probability density of (b, c) as shown in (4.4).We point out here that (4.6) and (4.4) do not work for the case withparametric noise only in one parameter, that is, u1 = 0 or u2 = 0 in (4.2).This is because if u1 = 0 or u2 = 0, the matrix Σ0 is singular. For example,for parametric noise in P , P = P ∗+ ξ1 and D = D∗, the probability densityfunction of r isp(r) =16r2 − 8amr(4r − am)2·Πb(f−11 (r)), r = 016r2 − 8amr(4r − am)2·Πb(f−11 (r)) +8ram·Πb(f−13 (r)), r <12|am|16r2 − 8amr(4r − am)2·Πb(f−11 (r)) +−16r2 − 8amr(4r + am)2·Πb(f−12 (r)), r ≥12|am|,(4.7)wheref−11 (r) =4r24r − am, f−12 (r) =4r2−4r − am, f−13 (r) =4r2am, am =Rm(2, 1)Rm(1, 1),(4.8)and Πb is the probability density function of b,Πb(b) =1√2pi1Rm(1, 1)exp(−b22Rm2(1, 1)), b ∼ N(0,Rm2(1, 1)).The detail of (4.7) is shown is shown in Appendix A.3. Similarly, if thenoise is only in D, P = P ∗ and D = D∗ + ξ2. Then the probability density374.1. Parametric noisefunction of r isp(r) =−16r2 − 8amr(4r + am)2·Πb(f−12 (r)), r = 0−16r2 − 8amr(4r + am)2·Πb(f−12 (r)) +8ram·Πb(f−13 (r)), r <12|am|−16r2 − 8amr(4r + am)2·Πb(f−12 (r)) +16r2 − 8amr(4r − am)2·Πb(f−11 (r)), r ≥12am.(4.9)The functions f1, f2, f3 in (4.9) are the same as shown in (4.8) except thatam =Rm(2, 2)Rm(1, 2)for this case and the probability density function of b isΠb(b) =1√2pi1Rm(1, 2)exp(−b22Rm2(1, 2)), b ∼ N(0,Rm2(1, 2)),as shown in Appendix (4.7).From (4.4), we can see that the probability density function P (b, c) de-pends on Rm, and from Figure 2.3, det(Rm) increases exponentially as τincreases, so that the covariance matrix of (b, c) increases with τ . Then (b, c)have wider distributions as τ grows. The variance of r also increases withτ . As shown in Figure 4.1, the system has limited sensitivity to parametricnoise for small τ . When the delay is small, the probability density of thecritical eigenvalue r is concentrated near zero. For large τ , the distributionof r could be problematic. This is because, as shown in Figure 2.2, thestability region narrows with increasing τ and the density of r spreads outover values above and below 1. Then, a small fluctuation in P and D resultsin a large change in the eigenvalues shown in Figure 4.1. Recalling the phe-nomena observed in Figure 2.4, we know that when eigenvalue approaches1, there are more fluctuations in the system. This explains the additionaloscillations caused by parametric noise for large τ , as shown in Figure 1.2(b).While r gives the relative convergence rate over one period of length T ,we also want to know the distribution of ρ, the absolute decay rate over one384.1. Parametric noise0 0.5 1 1.5 2 2.5012345r=|λ|maxp(r)0 0.5 1 1.5 2 2.500.20.40.60.81r=|λ|maxp(r)Figure 4.1: The probability density function of the critical eigenvalue r. The para-metric noise magnitude: u1 = 0.025, u2 = 0.025. Number of numerical simulations:N = 50000. The blue solid line: The probability density given by (4.6). The redstar: The numerically simulated density. The left panel: The case with small delayτ = 0.2, ta = 0.04. The right panel: The case with large delay τ = 2, ta = 0.4.unit of time. We compare different cases with different values of T . Forρ = r1/T , the probability density function of ρ is given byq(ρ) = p(ρT )TρT−1 (4.10)As shown in Figure 4.2, for small τ , the probability density of ρ is concen-trated close to 0 and decays with increasing ρ, like an exponential distribu-tion. This behavior for T < 1 is due to the term TρT−1, which decreasesexponentially with ρ. Because ρ is near zero with a large probability, the sys-tem with small τ exhibits nearly deadbeat behavior despite the parametricnoise, as shown in Figure 1.2 (b). However for large τ , given the same per-turbation in the control parameters P, D, the decay rate ρ is concentratedaround 1. Subsequently, the system shown in Figure 1.2 (b) converges moreslowly, exhibiting additional oscillations when compared to the system withsmall τ .As we observe from Figure 1.2 (b), the smaller the ta is, the more sensi-tive the system is to parametric noise. As shown in Figure 4.3, the densityof ρ shifts to smaller values for smaller ta. Yet this small difference alone isnot able to explain the noise sensitivity displayed in Figure 1.2 for differentvalues of ta. To consider the effect of ta in detail, the mean and the stan-394.1. Parametric noise0 0.01 0.02 0.03 0.04 0.050100200300400ρq(ρ)0 0.5 1 1.5 2 2.500.511.522.5ρq(ρ)0 0.01 0.02 0.03 0.04 0.05050100150200250ρq(ρ)0 0.5 1 1.5 2 2.500.511.522.5ρq(ρ)Figure 4.2: The probability density function of the decay rate ρ for differentvalues of τ . Number of numerical simulations: N = 50000. The blue solid line:The probability density given in (4.10). The red star: The numerically simulateddensity. The left panel: The case with small delay τ = 0.2, ta = 0.04. The right:The case with big delay τ = 2, ta = 0.4. Top Row: The parametric noise magnitude:u1 = 0.025, u2 = 0.025. Bottom Row: The parametric noise magnitude: u1 = 0.05,u2 = 0.404.1. Parametric noisedard deviation of θ(t) are illustrated in Figure 4.4. Due to the parametricnoise, the system oscillates beyond a period of length 2T before convergingto zero instead of abruptly damped behavior as in the deterministic sys-tem. Although the mean of θ(t) converges to zero in an interval of length2T , the standard deviation converges to zero slowly suggesting additionaloscillations. This is because the control pushes the system back to the zeroequilibrium in the acting period, yet it is not accurate enough due to theparametric noise, allowing the system to escape from zero in the waitingperiod. Although q(ρ) has slightly smaller values for the case with smallerta = 0.04, the control comes into effect only for a very short time, and thesystem escapes further in the waiting period. In contrast, q(ρ) has largervalues for ta = 0.4, yet the control is turned on longer to push the systemback to zero and reduce its chance to escape during the following waitingperiod. As a result, the case with a longer waiting period ta = 0.4 convergesto the zero equilibrium faster and oscillates less. That is why we observea larger standard deviation for the case with smaller ta, given the samewaiting period length tw = τ , as shown in Figure 4.4.Under the more general form of the FV system, we consider the effectof m on the parametric noise sensitivity. We find that the system is lesssensitive to the parametric noise for larger m. In Figure 4.5, we comparetwo extreme cases: the CV model (m→∞) and the PWC model (m = 1).In the left panel of Figure 4.5, the standard deviation of the time series θ(t)decreases with m. This indicates that the FV model with larger m oscillatesless and decays faster to the equilibrium compared to cases with smallervalues of m. In the right panel of Figure 4.5, we see that the probabilitydensity p(r) of the critical eigenvalue is concentrated at lower values of r forlarger m. We then conclude that given the same magnitude of the pertur-bation of the control parameters, we typically sample smaller eigenvalues ifwe update the feedback more frequently.414.1. Parametric noise0 0.5 1 1.5 2 2.500.511.522.5ρq(ρ)Figure 4.3: The probability density function of the decay rate ρ for different valuesof ta. The parametric noise magnitude: u1 = 0.025 and u2 = 0.025. Number ofnumerical simulations: N = 50000. The blue solid line: The probability densitygiven in (4.10) for the case τ = 2, ta = 0.4. The red star: The numerically simulateddensity for the case τ = 2, ta = 0.4. The magenta dashed line: The probabilitydensity given in (4.10) for the case τ = 2, ta = 0.04. The black triangle: Thenumerically simulated density for the case τ = 2, ta = 0.04.−2 −1 0 1 201234θ(t)Q(θ,t)0 5 10 15 2001234tθ(t)0 5 10 15 2001234tθ(t)Figure 4.4: Left: The probability density of θ(t) at different times. The parametersfor the cases presented: τ = 2, ta = 0.4, T = 2.4. The red dashed line: Theprobability density of θ(t) at t = 2T = 4.8. The blue marked line with +’s: Theprobability density of θ(t) at t = 3T = 7.2. The magenta solid line: The probabilitydensity of θ(t) at t = 4T = 9.6. Middle and Right: The blue solid line: The meanof θ(t). The red dash dotted line: The standard deviation of θ(t). The strip area:The period when control is switched on. The parameters for the cases presented:τ = 2, u1 = 0.025, u2 = 0.025. The middle panel: ta = 0.4. The right panel:ta = 0.04.424.2. External noise0 5 10 15 2000.20.40.60.81tθ(t)0 0.5 1 1.5 2 2.500.20.40.60.811.21.4r=|λ|maxp(r)Figure 4.5: Comparison of the CV model and PWC model for τ = 2, ta = τ/5 =0.4, u1 = 0.025, u2 = 0.025. Number of numerical simulations: N = 50000. Theleft panel: Comparison of the standard deviation of θ(t) for the CV and the PWCmodels. The red dashed line: The CV case. The blue solid line: The PWC case.The right panel: The density of the corresponding relative decay ratio r for the CVand PWC models. The blue solid line: The probability density given in (4.6) forthe CV model. The red star: The numerically simulated density for the CV model.The black dashed line: The probability density given in (4.6) for the PWC model.The magenta triangle: The numerically simulated density for the PWC model.4.2 External noiseThe interaction of delay and external noise results in sustained oscillationsinstead of decaying oscillations as seen for parametric noise in Figure 1.2(b). In this section, we study the system (1.1) with external noise describedasθ¨(t)− θ(t) = u(t) + δζ, (4.11)where δ is a constant and ζ is white noise. We use two alternating Ornstein-Uhlenbeck (O-U) type processes to analyze the act-and-wait model withexternal noise as follows,dx(t) = A˜x(t)dt+ δdwt, lT ≤ t < lT + tw, l ∈ Z, (4.12)dx(t) = A˜x(t)dt+ B˜Dx(t− τ)dt+ δdwt, lT + tw ≤ t < (l + 1)T.(4.13)Note wt = (0, Bt)′ in (4.12) and (4.13) where Bt denotes a Wiener process(Standard Brownian motion). The first entry of wt is zero because the434.2. External noiseexternal noise is added to the equation for the inverted pendulum (1.1) andthus appears only in the second entry of x˙(t) = (θ˙(t), θ¨(t))′ as shown in(4.11). For external noise, the noise drives the system to escape furtherfrom the zero equilibrium in the waiting period while the control pushesit back in the active period. By this interaction of control and noise, thesystem exhibits attracting periodically varying oscillations. These dynamicsfollow a normal distribution with zero mean and the standard deviationthat varies periodically in time, as shown in Figure 4.6. For large τ , weobserve the amplification of noise, which means that a small input noiseleads to significant oscillations in θ(t) and θ′(t), measured via the standarddeviation.We analyze the system with external noise via two alternating O-U typeprocesses in the waiting period and the acting period, as shown in (4.12)and (4.13). First, in the waiting period when the control is off, we havethe O-U process expressed in the differential form, as in (4.12). Integratingboth sides of (4.12), we write the time series x(t) in the waiting period asx(t) = exp(A˜t)x(t−lT )+δ∫ t−lT0exp(A˜(t−lT−ξ))dw(ξ), lT ≤ t < lT+tw.(4.14)From (4.14), we calculate the expectation and variance of x(t) asE(x(t)) = exp(A˜(t− lT ))Ex(lT ), lT ≤ t < lT + tw, (4.15)Var(x(t)) = exp(A˜(t−lT ))Var(x(lT )) exp(A˜(t−lT ))+δ24R(t−lT ), (4.16)whereexp(A˜t) =(cosh(t) sinh(t)sinh(t) cosh(t)), R(t) =(sinh(2t)− 2t cosh(2t)− 1cosh(2t)− 1 sinh(2t) + 2t).Second, in the acting period when control is on, we have the O-U type444.2. External noiseprocess expressed in differential form asdx(t) = A˜x(t)dt+ B˜Dx(t− τ)dt+ δdwt, lT + tw ≤ t < (l+ 1)T, (4.17)for the CV model anddx(t) = A˜x(t)dt+ B˜D exp(A˜(tw − τ))x(0)dt+ δdwt, lT + tw ≤ t < (l + 1)T= A˜x(t)dt+ B˜Dx(0)dt+ δdwt, (4.18)for the PWC model. The second line in (4.18) is obtained because we taketw = τ . We consider the PWC model, as it is solvable in an analytical form.Integrating both sides of (4.18), for the PWC model, we have the analyticalform of x(t) in the acting period lT + tw ≤ t < (l + 1)T ,x(t) = Φ(t−lT )x(lT )+δ∫ t−lT0exp(A˜(t−lT−ξ))dw(ξ), lT+tw ≤ t < (l+1)T.(4.19)The expectation and variance of x(t), lT + tw ≤ t < (l + 1)T in the actingperiod isE(x(t)) = Φ(t− lT )Ex(lT ), lT + tw ≤ t < (l + 1)T, (4.20)Var(x(t)) = Φ(t−lT )Var(x(lT ))Φ(t−lT )′+δ24R(t−lT ), lT+tw ≤ t < (l+1)T,(4.21)whereΦ(t) =(cosh(t) + (1− cosh(t− τ))P sinh(t) + (1− cosh(t− τ))Dsinh(t)− sinh(t− τ)P cosh(t)− sinh(t− τ)D).Specifically, for the times at integer multiples of the period, that is, t = lT ,l ∈ Z, we have the recurse form of the expectation and variance of x(lT ),l ∈ Z asE(x((l + 1)T )) = ΦPWC(Ex(lT )), (4.22)454.2. External noiseVar(x((l + 1)T )) = ΦPWCVar(x(lT ))Φ′PWC +δ24R(T ). (4.23)Note that for the parameters P = P ∗, D = D∗, the eigenvalues of ΦPWCare zeros. Additionally, since ΦPWC is a 2× 2 matrix, according to JordanDecomposition, Φ2PWC = 0. Thus, by multiplying ΦPWC on both the leftand the right hand of (4.23), we haveΦPWCVar(x((l + 1)T ))Φ′PWC = Φ2PWCVar(x(lT ))Φ′2PWC +δ24ΦPWCR(T )Φ′PWC=δ24ΦPWCR(T )Φ′PWC . (4.24)Combining (4.23) and (4.24), we getVar(x((l + 2)T )) = ΦPWCVar(x((l + 1)T ))Φ′PWC +δ24R(T )=δ24ΦPWCR(T )Φ′PWC +δ24R(T ). (4.25)Similarly,E(x((l + 2)T )) = ΦPWC(Ex((l + 1)T )) = Φ2PWC(Ex(lT )) = 0. (4.26)In an interval of length 2T , the system converges to periodically varyingoscillations with zero expectation given in (4.26) and variance given in (4.25).Note that (4.26) indicates that the system converges to equilibrium in a timeinterval of the length of 2T at most in the absence of noise, as shown for themean of θ(t) in the right panel of Figure 4.6. The standard deviation attractsto a periodically sustained oscillation due to the interaction of on-off controland noise. The time series θ(t) follows alternating normal distributions, asfollows from (4.14) and (4.18) and as shown in the left panel of Figure 4.6.Additionally, as we noted in the Figure of 1.2(c), the system with largeτ is more sensitive to external noise compared to the system with small τ .Using the analysis of the O-U type process above, we can determine howthe standard deviation is changing along with τ . Considering (4.25), the464.2. External noise−5 0 500.20.40.60.81θ(t)Q(θ,t)0 2 4 6 80123456tFigure 4.6: The probability density of the external noise driven system. Theparameters are: τ = 2, ta = 0.4, m = 1, δ = 0.1. Number of numerical simulationsN = 10000. Left: The periodically varying probability density of θ(t). The redmarked line with star: The probability density of θ(t) at t = nT . The blue markedline with square: The probability density of θ(t) at t = nT + T/3. The magentamarked line with triangle: The probability density of θ(t) at t = (n + 1)T . Right:The evolution of the distribution features of θ(t). The strip area shows where thecontrol is on. The triangle: The standard deviation of θ(t) given by numericalsimulations. The star: The mean of θ(t) given by numerical simulations. Theyellow solid line: The theoretical prediction of the mean of θ(t) given by (4.15) and(4.20). The blue solid line: The theoretical prediction of the standard deviation ofθ(t) given by (4.16) and (4.21).474.2. External noisevariance of θ(nT ) (n ≥ 2) isVar(θ(nT )) = δ2 ×M(τ, ta),with the amplification factor M(τ, ta) given byM(τ, ta) =116 sinh(ta + τ)2[sinh(6τ + 4ta)− 2 sinh(4τ + 3ta)+ sinh(2τ + 2ta)− sinh(4τ + 2ta) + 2 sinh(2τ + ta)+4T (1− cosh(2τ + ta))] +14[sinh(2τ + 2ta)− 2(τ + ta)].(4.27)This amplification factor M(τ, ta) grows exponentially with τ , as shownin the left panel of Figure 4.7. This explains why the system becomessensitive to external noise when the delay is large. Additionally, we plotthe amplification factor M(τ, ta) vs ta for the large delay case with τ = 2,and find that M(τ, ta) increases with ta. This dependence explains thephenomena, as shown in Figure 1.2 (c) that for larger acting periods, thesystem is more sensitive to external noise.0 0.5 1 1.5 2 2.5 310−2100102104106τlog(M(τ,t a))0 0.1 0.2 0.3 0.4 0.510001500200025003000350040004500taM(τ,t a)Figure 4.7: The amplification factor of the variance of θ(t): M(τ, ta) =Var(θ(t))δ2(4.27). The left: Semi-log plot of M(τ, ta) vs τ . The right: Plot of M(τ, ta) vs ta(τ = 2).48Chapter 5SummaryDelays in feedback control systems are generic in applications in biology [2],[14] mechanics [4], [5] and robotics [9], [11]. The presence of delay and itsinterplay with random perturbations often leads to instability or poor per-formance of the system. The main difficulty of time-delayed systems is thatthe number of instabilities to be controlled is usually larger than the num-ber of control parameters. Thus, complete stabilization is not possible forthese systems using traditional time-invariant feedback controls. Recently,it has been recognized that the realistic or ideal control may be one that isswitched on and off, proposed to procure simplicity in mechanical systems,or to provide a stabilizing mechanism particularly when the time delay islong.In this thesis, we focus on one special case of on and off controls: the act-and-wait control introduced in [6] and [18], which present case studies forsmall values of τ , in contrast to the traditional continuous control. The act-and-wait control has advantages due to the larger stability region of controlparameters and the potential deadbeat control. However, the combinationof delayed feedback, on and off control, and nonlinearities, naturally makesthese systems difficult to analyze. In addition, these elements can also makethe system sensitive to random effects. To better understand the effectsof the act-and-wait control, the new challenge is to develop and analyzemathematical models that incorporate these observations as well as get theinsight on the influence of the delay and noise. We studied the canonicalmodel of balancing an inverted pendulum [17] with act-and-wait control.We implemented the mathematical analysis of the PWC model and the CVmodel as well as the more general FV model. Under the wide scope of theFV model, we studied the influence of the delay, different choices of the49Chapter 5. Summaryacting period length ta, the frequencies of varying feedbacks, and the noisesensitivities through both analytical and numerical approaches.We built the basic model of an inverted pendulum with controller u(t)in §2.1. We compared two different kinds of feedback: the continuouslyvarying feedback and the piecewise constant feedback. These two differentfeedback mechanisms distinguish the PWC model from the CV model. Laterin §2.2, we analyzed the CV and PWC models by finding the analyticalexpressions and corresponding characteristic matrices: ΦCV and ΦPWC ,respectively. By analyzing the stability regions and deadbeat controls ofthe PWC and CV models in §2.3, we found that although the act-and-waitcontrol improves the stability, the effectiveness is impaired by large values ofτ . The stability region becomes thinner and smaller as τ increases. Althoughthe deadbeat control can be obtained, the decay rate is much more sensitiveto the variations of P and D away from P ∗, D∗. For small τ , inspite of theerror in P , D, the decay rate remains small in the neighborhood of P ∗, D∗,and the system has nearly deadbeat damping behavior. However, for largeτ , a small error in P and D can lead to a large increase in decay rate. Insteadof deadbeat damping behavior, the system decays slowly to the equilibrium,which gives the system potential sensitivity to noise. We also compareddeadbeat control parameters and stability regions for different choices of tawhen the delay is large. We found that our choice of ta for the PWC modelmay not necessarily give a preferred change in stability.In §3, we introduced the FV model, which captures the CV and PWCmodels as two extreme cases. In §3.1, we compared the difference betweenthe CV and PWC models which are problematic for large τ . In addition,we discussed the appropriate choices of ta for large τ . Because of theseobservations, a more flexible feedback model is needed to consider caseswith larger values of τ . Therefore, we introduced the FV model to allow thefeedback to vary more frequently within the acting period. In §3.2, we didthe mathematical analysis of the FV model and compared the CV, PWCand FV models for small ta  1 and general ta < 1. For ta  1, these threemodels did not differ much. However, for large ta < 1, significant differencesemerged between the CV model (m =∞) and the FV model with small m50Chapter 5. Summaryincluding the PWC model (m = 1). We gave general analytical solutionsfor the deadbeat parameters P ∗, D∗ and discussed the asymptotic behaviorof the FV model as m → ∞. The eigenvalues of the FV model convergeto those of the CV model with a correction term of order O(1m) for fixedτ and ta. Therefore, the FV model serves as a good approximation to theCV model with moderate value of m. It is simple to implement in practicalapplications and efficient in numerical simulations.In §4, we considered the noise sensitivity and the interaction of noiseand delay. We contrasted the effects of noise in system (1.1) for two sources,parametric randomness and external fluctuations. In §4.1, we gained in-sight on how parametric noise, which appears in the control parameters Pand D as random variables, influences the stability of the balanced statewith act-and-wait control. We found that parametric noise could reduce theconvergence rate to the equilibrium and causes additional fluctuations, par-ticularly for larger values of τ . Underlying this fluctuation is the variationof the eigenvalues for the system. The calculation of the distribution of thecritical eigenvalue for the system with random P and D shows significantprobability for eigenvalues to move away from zero, particularly for largervalues of τ . However, even though the act-and-wait system is sensitive toparametric noise for large τ , the stabilization is assured while the systemis unstable under the traditional continuous control. In §4.2, we also com-pared the act-and-wait system with the additive (external) noise, in whichcase larger excursions from the balanced state and sustained oscillationscan be observed. We modeled the external noise driven system with twoalternating Ornstein-Uhlenbeck (O-U) type processes, active in the wait-ing period and the acting period. We integrated the O-U type processesand calculated the expectations and variances of θ for the times at integermultiples of the period for the PWC model. Within two periods of time(2T ), the system converges to an attracting periodically varying oscillationwith zero expectation and periodically varying variance. For large τ , weobserved the amplification of noise, which means that a small input noiseleads to dramatic oscillations in θ(t) and θ′(t), measured via the standarddeviation.51Chapter 5. SummaryIn this thesis, we considered major components that affect the stabilityof the system with act-and-wait control, especially the delay and differentsources of random fluctuations. Most observations were verified throughboth numerical and analytical approaches, yet in some cases, the analyticalanalysis is omitted due to the complexity of the problem. First, we foundthe analytical solution of O-U type processes for the PWC model whilenot considering more complicated FV and CV models. Through numericaltesting, we found similar behavior of the FV and CV models given externalnoise as the PWC model, yet some detailed differences can be capturedthrough future work. Moreover, we conducted the numerical tests to provethat the deadbeat control parameters P ∗ and D∗ can always be found, thatis, the determinant of the second order equation is always no less than zerofor the range of τ and ta of our interest, yet the solid analytical proof isomitted. It would be interesting to explore the nonlinear equations (3.17)of the deadbeat control parameters P ∗ and D∗.52Bibliography[1] JC Allwright, Alessandro Astolfi, and HP Wong. A note on asymptoticstabilization of linear systems by periodic, piecewise constant, outputfeedback. Automatica, 41(2):339–344, 2005.[2] Yoshiyuki Asai, Yuichi Tasaka, Kunihiko Nomura, Taishin Nomura,Maura Casadio, and Pietro Morasso. A model of postural control inquiet standing: robust compensation of delay-induced instability usingintermittent activation of feedback control. PLoS One, 4(7):e6169, 2009.[3] Jonathan M Blackledge. Digital signal processing: mathematical andcomputational methods, software development and applications. Else-vier, 2006.[4] Eniko Enikov and Gabor Stepan. Microchaotic motion of digitally con-trolled machines. Journal of Vibration and control, 4(4):427–443, 1998.[5] Thomas Erneux. Applied delay differential equations, volume 3.Springer, 2009.[6] Tama´s Insperger. Act-and-wait concept for continuous-time control sys-tems with feedback delay. Control Systems Technology, IEEE Transac-tions on, 14(5):974–977, 2006.[7] Tama´s Insperger and Ga´bor Ste´pa´n. Act-and-wait control concept fordiscrete-time systems with feedback delay. Control Theory & Applica-tions, IET, 1(3):553–557, 2007.[8] Poolla Khargonekar, Kameshwar Poolla, and Allen Tannenbaum. Ro-bust control of linear time-invariant plants using periodic compensation.Automatic Control, IEEE Transactions on, 30(11):1088–1096, 1985.53Bibliography[9] Won S Kim and Antal K Bejczy. Demonstration of a high-fidelitypredictive/preview display technique for telerobotic servicing in space.Robotics and Automation, IEEE Transactions on, 9(5):698–702, 1993.[10] Keiji Konishi, Hideki Kokame, and Naoyuki Hara. Delayed feedbackcontrol based on the act-and-wait concept. Nonlinear Dynamics, 63(3):513–519, 2011.[11] La´szlo´ L Kova´cs, Tama´s Insperger, and Ga´bor Ste´pa´n. Teaching-inforce control of industrial robots used in medical applications. In Pro-ceedings of 15th CISM-IFToMM symposium on robot design, dynamicsand control, pages 1–4, 2004.[12] Maria Landry, Sue Ann Campbell, Kirsten Morris, and Cesar OAguilar. Dynamics of an inverted pendulum with delayed feedbackcontrol. SIAM Journal on Applied Dynamical Systems, 4(2):333–351,2005.[13] Wim Michiels, Silviu-Iulian Niculescu, and Luc Moreau. Using delaysand time-varying gains to improve the static output feedback stabiliz-ability of linear systems: a comparison. IMA Journal of MathematicalControl and Information, 21(4):393–418, 2004.[14] John Milton, Juan Luis Cabrera, Toru Ohira, Shigeru Tajima, Yuki-nori Tonosaki, Christian W Eurich, and Sue Ann Campbell. Thetime-delayed inverted pendulum: implications for human balance con-trol. Chaos: An Interdisciplinary Journal of Nonlinear Science, 19(2):026110–026110, 2009.[15] J Sieber and B Krauskopf. Complex balancing motions of an invertedpendulum subject to delayed feedback control. Physica D: NonlinearPhenomena, 197(3):332–345, 2004.[16] Jan Sieber and Bernd Krauskopf. Bifurcation analysis of an invertedpendulum with delayed feedback control near a triple-zero eigenvaluesingularity. Nonlinearity, 17(1):85, 2004.54[17] David JW Simpson, Rachel Kuske, and Yue-Xian Li. Dynamics ofsimple balancing models with state dependent switching control. arXivpreprint arXiv:1104.1446, 2011.[18] G Ste´pa´n and T Insperger. Robust time-periodic control of time-delayedsystems. In Iutam Symposium on Dynamics and Control of NonlinearSystems with Uncertainty, pages 343–352. Springer, 2007.[19] G Ste´pa´n and L Kolla´r. Balancing with reflex delay. Mathematical andComputer Modelling, 31(4):199–205, 2000.55Appendix AThe mathematicalcalculationA.1 The calculation of matricesWe show the calculation of A and B in (2.8).A = exp(A˜∆t),B = (exp(A˜∆t)− I)A˜−1B˜,whereA˜ =(0 11 0), B˜ =(01), A˜−1 =(0 11 0).The decomposition of A˜ isA˜ = QΛQ′, Q =1√2(1 11 −1), Λ =(1 00 −1), Q = Q′.So thatA = exp(A˜∆t) = QeΛ∆tQ′ = Q(e∆t 00 e−∆t)Q′ (A.1)=(e∆t+e−∆t2e∆t−e−∆t2e∆t−e−∆t2e∆t+e−∆t2)=(cosh(∆t) sinh(∆t)sinh(∆t) cosh(∆t)), (A.2)56A.1. The calculation of matricesandB = (exp(A˜∆t)− I)A˜−1B˜=(cosh(∆t)− 1 sinh(∆t)sinh(∆t) cosh(∆t)− 1)(0 11 0)(01)=(cosh(∆t)− 1sinh(∆t)).Next, we show that ΦPWC = Φ1 (2.13), ΦCU = Φ∞ (2.11).ΦPWC = eA˜T +∫ TtweA˜(T−s)B˜DeA˜(tw−τ)ds= eA˜T + eA˜T∫ Ttwe−A˜sdsB˜DeA˜(tw−τ)= QeΛ∆TQ′[I +∫ TtwQe−ΛsQ′dsB˜D]= QeΛ∆TQ′[I +Q∫ Ttwe−ΛsdsQ′B˜D]= QeΛ∆TQ′[I +Q(e−tw − e−T 00 eT − etw)Q′B˜D]= Q[eΛ∆T + eΛ∆T(e−tw − e−T 00 eT − etw)Q′B˜DQ]Q′= Q[(eT 00 e−T)+(eT 00 e−T)(e−tw − e−T 00 eT − etw)Q′B˜DQ]Q′= Q[(eT 00 e−T)+(eT−tw − 1 00 1− e−T+tw)12(−(P +D) −(P −D)P +D P −D)]Q′= Q(eT + P+D2 (1− eT−tw) P−D2 (1− eT−tw)P+D2 (1− e−(T−tw)) e−T + P−D2 (1− e−(T−tw)))Q′= Q(eT + P+D2 (1− eta) P−D2 (1− eta)P+D2 (1− e−ta) e−T + P−D2 (1− e−ta))Q′ = Φ1,whereQ′B˜DQ =12(−(P +D) −(P −D)P +D P −D).57A.1. The calculation of matricesΦCU = eA˜T +∫ TtweA˜(T−s)B˜DeA˜(s−τ)ds= eA˜T +Q12∫ Ttw(eT−s 00 e−(T−s))(−(P +D) −(P −D)P +D P −D)(es−τ 00 e−(s−τ))dsQ′= Q(eT 00 e−T)Q′ +Q12∫ Ttw(−eT−τ (P +D) −eT−2s+τ (P −D)e−T+2s−τ (P +D) e−T+τ (P −D))dsQ′= Q(eT 00 e−T)Q′ +Q(− (P+D)2 eT−τ (T − τ) − (P−D)2e−T+τ−eT−τ−2(P+D)2eT−τ−e−T+τ2(P−D)2 e−T+τ (T − τ))Q′= Q(eT − (P+D)2 eta(ta) −(P−D)2e−ta−eta−2(P+D)2eta−e−ta2 e−T + (P−D)2 e−ta(ta))Q′= Q(eT − (P+D)2 etata −(P−D)2 sinh(ta)(P+D)2 sinh(ta) e−T + (P−D)2 e−tata)Q′ = Φ∞.We show the calculation of matrix Φm (3.4).Φm = An+m +BDAm−1 +ABDAm−2 + · · ·+Am−1BD,usingBDAm−1 +ABDAm−2 + · · ·+Am−1BD=m−1∑i=0Q(ei∆t 00 e−i∆t)Q′BDQ(e(m−1−i)∆t 00 e−(m−1−i)∆t)Q′=m−1∑i=012Q(−e(m−1)∆t(e∆t − 1)(P +D) −e−(m−2i−1)∆t(e∆t − 1)(P −D)−e(m−2i−1)∆t(e−∆t − 1)(P +D) −e−(m−1)∆t(e−∆t − 1)(P −D))Q′=12Q(−me(m−1)∆t(e∆t − 1)(P +D) −∑m−1i=0 e−(m−2i−1)∆t(e∆t − 1)(P −D)−∑m−1i=0 e(m−2i−1)∆t(e−∆t − 1)(P +D) −me−(m−1)∆t(e−∆t − 1)(P −D))Q′=12Q(−m(em∆t − e(m−1)∆t)(P +D) − e−(m−1)∆t−e(m+1)∆t1−e2∆t (e∆t − 1)(P −D)− e(m−1)∆t−e−(m+1)∆t1−e−2∆t (e−∆t − 1)(P +D) −m(e−m∆t − e−(m−1)∆t)(P −D))Q′.58A.2. Deadbeat controlSoΦm = Q(eT 00 e−T)Q′ +Q(−meta(1− e−1∆t)P+D2 (1− e∆t) sinh(ta)sinh(∆t)(P−D)2(1− e−∆t) (P+D)2sinh(ta)sinh(∆t) −me−ta(1− e∆t) (P−D)2)Q′= Q(eT −meta(1− e−1∆t)P+D2 (1− e∆t) sinh(ta)sinh(∆t)(P−D)2(1− e−∆t) (P+D)2sinh(ta)sinh(∆t) e−T −me−ta(1− e∆t) (P−D)2)Q′,whereQ′BDQ =12(−(e∆t − 1)(P +D) −(e∆t − 1)(P −D)−(e−∆t − 1)(P +D) −(e−∆t − 1)(P −D)).A.2 Deadbeat controlWe discuss the solution of the nonlinear equations (3.17) of the deadbeatcontrol parameters P ∗ and D∗,Rm(PD)+ S +Nm ·(0P 2 −D2)= 0. (A.3)For m ≥ 2, Nm 6= 0. The resulting second order system is solvable,P ∗max = H1D +H2, D∗max =H3 +√H42Nm(H21 − 1), (A.4)P ∗min = H1D +H2, D∗min =H3 −√H42Nm(H21 − 1), (A.5)whereH1 = − coth(ta −∆t2), H2 =cosh(τ + ta)m sinh(ta − ∆t2 ) sinh(∆t2 ),H3 = Rm(2, 1)H1 + 2H1H2Nm +Rm(2, 2),H4 = (Rm(2, 1)H1+2H1H2Nm+Rm(2, 2))2−4Nm(H21−1)(Rm(2, 1)H2+4+NmH22 ).59A.2. Deadbeat controlThe existence of real roots of equation (A.3) is not assured and for theapplication we require the control parameters P and D to be real. Howeverthrough a numerical test for τ ranging from 0.1 to 8s and ta ∈ [0.004, 1), thedeterminant of the reduced single variable quadratic equation, denoted byH4, is always greater than zero, increasing with τ . Thus for any 0.01 < τ < 8and ta < 1, the roots in (A.4), (A.5) are real, that is, the deadbeat control(P ∗m, D∗m) exists.There are two roots for (A.3), a larger one (A.4) and a smaller one (A.5),yielding two stability regions. Through numerical tests for 0.01 < τ < 8 andta < 1, we see that the stability region around smaller values of (P ∗, D∗)is wider while the region around larger (P ∗, D∗) is a long and thin strip.Also, the absolute value of larger root of (P ∗, D∗) is extremely large. As wediscussed above, the values of (P ∗, D∗) should be physically realistic. So wealways choose the smaller root of (P ∗, D∗) as shown in (A.5).Next, we show that Λ∞ = limm→∞Λm for which Λm and Λ∞ are definedin (3.5) and (3.7).Proof of Λ∞ = limm→∞Λm given fixed τ and ta.Assume m is large so that ∆t =tamis small, allowing the asymptoticanalysis through Taylor expansions as shown for one entry in Λm,Λm(1, 1) = e(τ+ta) −P +D2meta(1− e−∆t)= e(τ+ta) −P +D2m∆teta(1− e−∆t)∆t= e(τ+ta) −P +D2taeta1− (1−∆t+ ∆t22 + · · · )∆t= e(τ+ta) −P +D2taeta(1−∆t2+O(∆t2))∼ e(τ+ta) −P +D2taeta +P +D2taeta ∆t2∼ Λ∞(1, 1) +P +D2taeta ∆t2. (A.6)We get this approximation by dropping the second order error term O(∆t2)and using the fact that the quantity m∆t = ta is fixed. As in (A.6), we get60A.3. The probability density of the eigenvalues of Φmthe asymptotic behavior of Λm(2, 2),Λm(2, 2) = e−(τ+ta)−P −D2me−ta(1− e∆t) ∼ Λ∞(2, 2) +P −D2tae−ta ∆t2.Next,Λm(1, 2) =P −D2(1− e∆t)sinh(ta)sinh(∆t)=P −D2sinh(ta)2(1− e∆t)e∆t − e−∆t=P −D2sinh(ta)2(e−∆t − 1)1− e−2∆t=P −D2sinh(ta)−2(e−∆t − 1)(e−∆t − 1)(e−∆t + 1)=P −D2sinh(ta)−2e−∆t + 1= −P −D2sinh(ta)22−∆t+ ∆t22 + · · ·= −P −D2sinh(ta)[1 +∆t2+O(∆t2)]∼ Λ∞(1, 2)−P −D2sinh(ta)∆t2. (A.7)We get this approximation by dropping the second order error term O(∆t2).As in (A.7), we get the asymptotic behavior of Λm(2, 1),Λm(2, 1) =P +D2(1− e−∆t)sinh(ta)sinh(∆t)≈ Λ∞(2, 1)−P +D2sinh(ta)∆t2.Combining all of the above approximations, yieldsΛm = Λ∞ +∆t2(P+D2 taeta −P−D2 sinh(ta)−P+D2 sinh(ta)P−D2 tae−ta).Thus Λ∞ = limm→∞Λm = lim∆t→0 Λm.A.3 The probability density of the eigenvalues ofΦmWe calculate the probability density of the critical eigenvalue r = max{|λm1 |, |λm2 |}where λm1 , λm2 are eigenvalues of matrix Φm as shown in§4.1. Recallingb = λm1 + λm2 , c = 4λm1 λm2 , r is a piecewise function of (b, c) as in (4.5),61A.3. The probability density of the eigenvalues of Φmr =f1(b, c) =12(b+√b2 − c), (b, c) ∈ I = {b ≥ 0, b2 ≥ c}f2(b, c) =12(−b+√b2 − c), (b, c) ∈ II = {b < 0, b2 ≥ c}f3(b, c) =√c/4, (b, c) ∈ III = {b2 < c}. (A.8)First of all, we consider the case with parametric noise only in P aroundthe deadbeat control P ∗, that is, P = P ∗ + ξ1, D = D∗. Then we have thefollowing map(bc)= Rm(P ∗ + ξ1D∗)+ S = Rm(ξ10)=(Rm(1, 1)ξ1Rm(2, 1)ξ1). (A.9)Then we write c in terms of b as c = amb where am =Rm(2, 1)Rm(1, 1)< 0. Thuswe can write r as a single variable function in b asr =12(b+√b2 − amb) = f1(b), b ≥ 0, ⇒ r ∈ [0,∞)12(−b+√b2 − amb) = f2(b), b ≤ am, ⇒ r ∈ [12|am|,∞)12√amb = f3(b), am < b < 0, ⇒ r ∈ (0,12|am|). (A.10)So the probability density function of r is as shown in (4.7),p(r) =16r2 − 8amr(4r − am)2·Πb(f−11 (r)), r = 016r2 − 8amr(4r − am)2·Πb(f−11 (r)) +8ram·Πb(f−13 (r)), r <12|am|16r2 − 8amr(4r − am)2·Πb(f−11 (r)) +−16r2 − 8amr(4r + am)2·Πb(f−12 (r)), r ≥12|am|,(A.11)where f1, f2, f3 are defined in (4.8),f−11 (r) =4r24r − am, f−12 (r) =4r2−4r − am, f−13 (r) =4r2am. (A.12)62A.3. The probability density of the eigenvalues of ΦmAnd Πb is the probability density function of b,Πb(b) =1√2pi1Rm(1, 1)exp(−b22Rm2(1, 1)), b ∼ N(0,Rm2(1, 1)).Note that f1, f2, f3 are not one-to-one maps ∀r ∈ [0,∞). We confirm thatfor each branch in (A.10), the map f1(r), f2(r), f3(r) are one-to-one mapsfor the corresponding range of r in (A.10), so that the inverse functionsshown in (4.7) are valid.First, for f1(b), the derivative isddbf1(b) =12[1 +122b− am√b2 − amb] > 0, for b ≥ 0.Thus r = f1(b) ≥ f1(0) = 0 and r = f1(b) is a one-to-one map fromb ∈ [0,∞) to r ∈ [0,∞).Second, for f2(b), the derivative isddbf2(b) =12[−1 +122b− am√b2 − amb]< 0, for b ≤ am.Thus r = f2(b) ≥12|am| and r = f2(b) is a one-to-one map from b ∈(−∞, am] to r ∈ [12|am|,∞).Third, f3(b) =12√amb is a one-to-one map from b ∈ (am, 0] to r ∈(0,12|am|).Next, we consider the case with parametric noise only in D around dead-beat control D∗, that is, P = P ∗, D = D∗+ ξ2. Then we have the followingmap(bc)= Rm(P ∗D∗ + ξ2)+S = Rm(0ξ2)=(Rm(1, 2)ξ2Rm(2, 2)ξ2). (A.13)Then we write c in terms of b as c = amb where am =Rm(2, 2)Rm(1, 2)> 0. So we63A.3. The probability density of the eigenvalues of Φmreduce r to a single variable function in b asr =12(b+√b2 − amb) = f1(b), b ≥ am, ⇒ r ∈ [am2,∞)12(−b+√b2 − amb) = f2(b), b ≤ 0, ⇒ r ∈ [0,∞)12√amb = f3(b), 0 < b < am, ⇒ r ∈ (0,12|am|). (A.14)So the probability density function of r isp(r) =−16r2 − 8amr(4r + am)2·Πb(f−12 (r)), r = 0−16r2 − 8amr(4r + am)2·Πb(f−12 (r)) +8ram·Πb(f−13 (r)), r <12|am|−16r2 − 8amr(4r + am)2·Πb(f−12 (r)) +16r2 − 8amr(4r − am)2·Πb(f−11 (r)), r ≥12am,(A.15)with f1, f2, f3 the same as shown in (4.8) except that the parametersam =Rm(2, 2)Rm(1, 2)for this case. The probability density function of b isΠb(b) =1√2pi1Rm(1, 2)exp(−b22Rm2(1, 2)), b ∼ N(0,Rm2(1, 2)).We have to make sure that for each branch in (A.14), the map f1(r),f2(r), f3(r) are one-to-one maps for the corresponding range of r in (A.14),so that the inverse functions shown in (4.9) are valid.First, for f1(b), the derivative isddbf1(b) =12[1 +122b− am√b2 − amb]> 0, for b ≥ am.Thus r = f1(b) ≥ f1(am) =12am and r = f1(b) is a one-to-one map fromb ∈ [am,∞) to r ∈ [12am,∞).64A.3. The probability density of the eigenvalues of ΦmSecond, for f2(b), the derivative isddbf2(b) =12[−1 +122b− am√b2 − amb]< 0, for b ≤ 0.Thus r = f2(b) ≤ f2(0) = 0 and r = f2(b) is a one-to-one map fromb ∈ (∞, 0] to r ∈ [0,∞).Third, f3(b) =12√amb is a one-to-one map from b ∈ (0, am) to r ∈(0,12am).Last, we consider the case with parametric noise in both P and D arounddeadbeat control P ∗, D∗, that is P = P ∗ + ξ1, D = D∗ + ξ2. First ofall, we need to prove that the region {r(b, c) ≤ a} is the triangular region{4a(b− a) ≤ c ≤ 4a2, b ≥ 0} ∪ {−4a(b+ a) ≤ c ≤ 4a2, b < 0} as shown bythe contour plots in Figure 2.1. To prove this, we use (A.8). First for regionI,12(b+√b2 − c) ≤ a⇒ b ≤ 2a, 4a(b− a) ≤ c ≤ b2,because12(b+√b2 − c) ≤ a⇒√b2 − c ≤ 2a− b⇒ b2 − c ≤ 4a2 + b2 − 4ab.So{r(b, c) ≤ a} ∩ I ⊆ A = {4a(b− a) ≤ c ≤ b2, 0 ≤ b ≤ 2a}.Next we prove that A ⊆ {r(b, c) ≤ a} ∩ I. For (b, c) ∈ A,∂f1∂b=12(1 +b√b2 − c) > 0,∂f1∂c=12(−1√b2 − c) < 0 on I.So for any b, f1(b, c) attains its maxium at (b, 4a(b− a)) wheref1(b, 4a(b− a)) =12(b+ |b− 2a|) =12(b+ 2a− b) = a.So f1(b, c) ≤ a for any (b, c) ∈ A which means A ⊆ {r(b, c) ≤ a} ∩ I.Thus {r(b, c) ≤ a} ∩ I = A.65A.3. The probability density of the eigenvalues of ΦmSecond for region II,12(−b+√b2 − c) ≤ a⇒ b ≥ −2a, −4a(b+ a) ≤ c ≤ b2,because12(−b+√b2 − c) ≤ a⇒√b2 − c ≤ 2a+ b⇒ b2 − c ≤ 4a2 + b2 + 4ab.So{r(b, c) ≤ a} ∩ II ⊆ B = {−4a(b+ a) ≤ c ≤ b2, −2a ≤ b ≤ 0}.Next we prove that B ⊆ {r(b, c) ≤ a} ∩ II. For (b, c) ∈ B,∂f2∂b=12(−1 +b√b2 − c) < 0,∂f2∂c=12(−1√b2 − c) < 0 on I.So for any b, f2(b, c) attains its maxium at (b,−4a(b+ a)) wheref2(b,−4a(b+ a)) =12(−b+ |b+ 2a|) =12(−b+ b+ 2a) = a.Thus f2(b, c) ≤ a for any (b, c) ∈ B which means B ⊆ {r(b, c) ≤ a} ∩ II.Thus {r(b, c) ≤ a} ∩ II = B.Third for region III, {r(b, c) ≤ a} ∩ III ⊆ C = {b2 < c ≤ 4a2}, andC ⊆ {r(b, c) ≤ a} ∩ III. Thus C = {r(b, c) ≤ a} ∩ III. Combining the aboveresults,{r(b, c) ≤ a} = A ∪B ∪ C= {4a(b− a) ≤ c ≤ 4a2, b ≥ 0} ∪ {−4a(b+ a) ≤ c ≤ 4a2, b < 0}.confirming that the region {r(b, c) ≤ a} is the triangular region {4a(b−a) ≤c ≤ 4a2, b ≥ 0} ∪ {−4a(b+ a) ≤ c ≤ 4a2, b < 0}.66A.3. The probability density of the eigenvalues of ΦmThus the cumulative distribution function of r isP (r ≤ a) =∫∫AP (b, c)dcdb+∫∫BP (b, c)dcdb+∫∫CP (b, c)dcdb=∫ 2a0∫ b24a(b−a)P (b, c)dcdb+∫ 0−2a∫ b2−4a(b+a)P (b, c)dcdb+∫ 2a−2a∫ 4a2b2P (b, c)dcdb.(A.16)And the probability density function of r isp(r) =∫ 2r0(−4b+ 8r)P (b, 4r(b− r))db+∫ 0−2r(4b+ 8r)P (b,−4r(b+ r))db+∫ 2r−2r8rP (b, 4r2)db, (A.17)where P (b, c) is the probability density of (b, c) as shown in (4.4). Next, weshow the calculation details of (A.17). Because p(r) =dP (r)dr, we calculatethe derivatives of the three integrals in (A.16) consisting P (r).dda∫ 2a0∫ b24a(b−a)P (b, c)dcdb=∫ 2a0∂∂a∫ b24a(b−a)P (b, c)dcdb+ 2∫ 4a24a2P (b, c)dcdb=∫ 2a0(−4y + 8a)P (b, 4a(b− a))db.dda∫ 0−2a∫ b2−4a(b+a)P (b, c)dcdb=∫ 0−2a∂∂a∫ b2−4a(b+a)P (b, c)dcdb− (−2)∫ 4a24a2P (b, c)dcdb=∫ 0−2a(4y + 8a)P (b,−4a(b+ a))db.67A.4. The O-U type processesdda∫ 2a−2a∫ 4a2b2P (b, c)dcdb=∫ 2a−2a∂∂a∫ 4a2b2P (b, c)dcdb+ 2∫ 4a2(2a)2P (b, c)dcdb− (−2)∫ 4a2(2a)2P (b, c)dcdb=∫ 2a−2a8aP (b, 4a2)db.Combing the above results and the fact that p(r) =dP (r)dr, we get theexpression of the probability density in (A.17).A.4 The O-U type processesIn this section, we give some calculation details of equations (4.14) - (4.21).For the sake of simplicity, we assume 0 ≤ t < T . Then the the general resultscan be obtained for lT ≤ t < (l + 1)T , l ∈ Z by making the substitutiont′ = t− lT , l ∈ Z.At first, for 0 ≤ t < tw, the formula (4.16) is obtained through thefollowing equationsVar((x(t)) = Var(exp(A˜t)x(0)) + Var(δ∫ t0exp(A˜(t− ξ))dw(ξ))= exp(A˜t)Var(x(0)) exp(A˜t) +∫ t0exp(A˜t′)(0 00 δ2)exp(A˜t′)dt′.(A.18)Notice in the second line of (A.18), because the external noise only comesin the second entry of x(t) as shown in (4.11), the variance of dw(ξ) at timet = ξ isVar(dw(ξ)) =(0 00 δ2).68A.4. The O-U type processesNext,∫ t0exp(A˜t′)(0 00 δ2)exp(A˜t′)dt′=∫ t0Q(et′00 e−t′)Q′(0 00 δ2)Q(et′00 e−t′)Q′dt′=δ22Q∫ t0(e2t′−1−1 e−2t′)dt′Q′=δ22Q(e2t−12 −t−t 1−e−2t2)Q′=δ24(sinh(2t)− 2t cosh(2t)− 1cosh(2t)− 1 sinh(2t) + 2t), (A.19)for 0 ≤ t < tw where matrix Q is defined in (2.14). Combining (A.18)and (A.19), we get the result in (4.16).69A.4. The O-U type processesSecond, we show the calculation details of (4.19) for tw ≤ t < tw+ta = T .x(t) = exp(A˜(t− tw))x(tw) +∫ ttwexp[A˜(t− ξ)]B˜Dx(0)dξ+ δ∫ ttwexp(A˜(t− ξ))dw(ξ)= exp(A˜(t− tw))(exp(A˜tw)x(0) + δ∫ tw0exp(A˜(tw − ξ))dw(ξ))+∫ ttwexp[A˜(t− ξ)]B˜Dx(0)dξ + δ∫ ttwexp(A˜(t− ξ))dw(ξ)= exp(A˜t)x(0) + δ∫ tw0exp(A˜(t− ξ))dw(ξ)+∫ ttwexp[A˜(t− ξ)]dξB˜Dx(0) + δ∫ ttwexp(A˜(t− ξ))dw(ξ)= exp(A˜t)x(0) +∫ ttwexp[A˜(t− ξ)]dξB˜Dx(0)+ δ∫ t0exp(A˜(t− ξ))dw(ξ)=(cosh(t) + (1− cosh(t− τ))P sinh(t) + (1− cosh(t− τ))Dsinh(t)− sinh(t− τ)P cosh(t)− sinh(t− τ)D)x(0)+ δ∫ t0exp(A˜(t− ξ))dw(ξ)= Φ(t)x(0) + δ∫ t0exp(A˜(t− ξ))dw(ξ), (A.20)for tw ≤ t < tw + ta = T , whereΦ(t) =(cosh(t) + (1− cosh(t− τ))P sinh(t) + (1− cosh(t− τ))Dsinh(t)− sinh(t− τ)P cosh(t)− sinh(t− τ)D),and Φ(T ) = ΦPWC = Φ1 = An+1 +BD.70"@en ; edm:hasType "Thesis/Dissertation"@en ; vivo:dateIssued "2014-09"@en ; edm:isShownAt "10.14288/1.0166014"@en ; dcterms:language "eng"@en ; ns0:degreeDiscipline "Mathematics"@en ; edm:provider "Vancouver : University of British Columbia Library"@en ; dcterms:publisher "University of British Columbia"@en ; dcterms:rights "Attribution-NonCommercial-NoDerivs 2.5 Canada"@en ; ns0:rightsURI "http://creativecommons.org/licenses/by-nc-nd/2.5/ca/"@en ; ns0:scholarLevel "Graduate"@en ; dcterms:title "Dynamics of balance with act-and-wait control"@en ; dcterms:type "Text"@en ; ns0:identifierURI "http://hdl.handle.net/2429/50098"@en .