International Conference on Applications of Statistics and Probability in Civil Engineering (ICASP) (12th : 2015)

A constrained nonlinear stochastic optimal control for dynamic systems El-Khoury, Omar; Shafieezadeh, Abdollah 2015-07

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata


53032-paper_437_ElKhoury.pdf [ 587.01kB ]
JSON: 53032-1.0076207.json
JSON-LD: 53032-1.0076207-ld.json
RDF/XML (Pretty): 53032-1.0076207-rdf.xml
RDF/JSON: 53032-1.0076207-rdf.json
Turtle: 53032-1.0076207-turtle.txt
N-Triples: 53032-1.0076207-rdf-ntriples.txt
Original Record: 53032-1.0076207-source.json
Full Text

Full Text

12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015  1  A Constrained Nonlinear Stochastic Optimal Control for Dynamic Systems  Omar El-Khoury Graduate Student, Dept. of Civil, Environmental, & Geodetic Engineering, Ohio State University, Columbus, USA Abdollah Shafieezadeh Assistant Professor, Dept. of Civil, Environmental, & Geodetic Engineering, Ohio State University, Columbus, USA  ABSRTACT: An ideal controller assumes that the system is unconstrained and the control force in unbounded. However, in reality, control devices are restricted by their force capacity. Traditionally, the clipping strategy has been used extensively, where an ideal actuator is assumed in the control design, and then the inequality constraints are enforced through saturation. This approach may not provide optimal solutions since constraints are not considered in the control optimization. To overcome this limitation, this paper presents a constrained nonlinear stochastic optimal control algorithm for dynamic systems subjected to Gaussian white noise excitations. In this control algorithm, stochasticity and nonlinearity of a Hamiltonian dynamic system is considered based on stochastic averaging of energy envelope using Markovian approximation. An Ito equation of energy envelope is derived and represented by diffusion and drift components. For the control design, a prescribed cost function, the diffusion and drift components together with the force constraints are considered in solving the Hamilton Jacobian Bellman (HJB) equation. This proposed control approach is called here Constrained Stochastic Control (CSC). The performance of the CSC algorithm is demonstrated for a hysteretic column and the results are compared to simulation results for Clipped Stochastic Control (Cl-SC) and uncontrolled cases. Noticeable improvements in peak and root mean square values of displacement in the CSC case are observed over the Cl-SC algorithm.    1. INTRODUCTION In control engineering, active devices ideally can apply any force of any scale. However, an active device such as an actuator is restricted by a maximum force which cannot be exceeded. In conventional control methods e.g. clipped linear quadratic regulator (LQR), the controller is first designed assuming unboundedness, and then the constraints are enforced through saturation. This approach has been shown to yield good results in civil engineering structures such as multi-story buildings or multi-span bridges subjected to severe winds or earthquakes (Fan et al., 2009). But clipped-based methods may result in suboptimal and sometimes far from optimal solutions especially in highly nonlinear systems where the control design is based on an ideal linear controller (Zapateiro et al., 2010; Alavinasab et al., 2006). To overcome the shortcomings of clipped optimal approaches, a Constrained Stochastic Control (CSC) algorithm is proposed here for nonlinear dynamic systems. This method is based on stochastic averaging of the total energy of the system (Zhu et al., 1997; Jia et al., 2013). The energy 12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015  2  component is assumed to be a slow varying process as compared to the displacement and velocity. Consideration of energy allows for a better characterization of nonlinearity and stochasticity in the controlled system (Zhu, 2006; Wang et al., 2009; Gu et al., 2012). In this approach, the control design, based on stochastic averaging, is derived as the solution of a nonlinear dynamic equation, called Hamilton Jacobian Bellman (HJB) equation (Gu et al., 2012). To solve this equation, an iterative strategy based on Newton’s method developed by Miranda and Fackler (2002) is used. The method has been extensively investigated for solving economics and finance problems (Fackler, 2005; Ortigueira, 2006; Daigneault et al., 2010; Balikcioglu et al., 2011). Applying this method to engineering, the constraint of the actuator is embedded in the iterative process to solve the dynamic equation and compute the optimal control force. In this paper, a hysteretic column subjected to Gaussian white noises is analyzed to assess the performance of CSC algorithm and compare it with both the corresponding uncontrolled system and the system equipped with Clipped Stochastic Control (Cl-SC) algorithm.  2. METHODOLOGY In commonly used control algorithms, the control design is based on a deterministic excitation and on a state space linearization of nonlinear systems (Zapateiro et al., 2010; Kim et al., 2013). However, in civil engineering applications, earthquake or wind excitations are considered stochastic with characteristics that are not fully known in prior. But these hazards can be modelled using techniques such as filters that are applied to the Gaussian white noise. In this paper, stochastic averaging of energy envelope is used for control design in order to characterize stochasticity and nonlinearity in the system.  Focusing on system behavior, a dynamic system can be described by Hamilton energy (𝑬) which consists of kinetic energy (𝑲) and potential energy (𝑷) 𝑬 = 𝑲 + 𝑷 (1) The differential equation of the energy component is defined as 𝒅𝑬 = π’Ž(𝑬, 𝒖)𝑑𝑑 + 𝝈(𝑬)𝒅𝑩 (2) where π’Ž(𝑬, 𝒖), 𝝈(𝑬), 𝒖, and 𝑩 are the drift and diffusive components, control vector, and a Gaussian white noise process, respectively. For an affine controlled dynamic system, the drift component is represented as π’Ž(𝑬, 𝒖) = (π’Žπ’–(𝑬) +π’Žπ’„(𝑬) βˆ— 𝒖)                                     |𝒖| < π’–π’Žπ’‚π’™ (3) where π’Žπ’– is the drift component for the uncontrolled case, π’Žπ’„ is the additional drift component for the controlled case, and π’–π’Žπ’‚π’™ is the maximum control force vector.  For control design, an infinite horizon performance index for a stationary ergodic system is considered 𝐽 = limπ‘‘πΉβ†’βˆž1π‘‘πΉπΈβˆ« 𝐿(𝑬, 𝒖)𝑑𝐹0 (4) where 𝐿(𝑬, 𝒖) denotes the cost function defined as 𝐿(𝑬, 𝒖) = 𝐻(𝑬) + 𝒖𝑻𝑹𝒖 (5) where 𝐻 is a cost function of energy components and 𝑹 is the positive definite covariance matrix for the control force vector. According to Fleming and Soner (2006), the corresponding dynamic programming equation for finite time horizon is derived as 12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015  3  πœ† = min(𝐿(𝑬, 𝒖) +πœ•π‘‰πœ•π‘¬π’Ž(𝑬, 𝒖)+12πœ•2π‘‰πœ•π‘¬2𝝈𝟐(𝑬)) (6) where the parameter, πœ†, is the optimal averaged cost function. In this equation, the value function, 𝑉, plays the role of a Lagrange multiplier to enforce the equality constraint in Equation (2). Initially, the unbounded control force is derived by differentiating Equation (6) and equating it to zero, πœ•πœ†/πœ•π’– = 𝟎. The resulting control force  𝒖 =12π‘Ήβˆ’1πœ•π‘‰πœ•π‘¬π’Žπ’„(𝑬) (7) In order to incorporate the boundedness of the control force, the control vector is modified to |𝒖|= min (|12π‘Ήβˆ’1πœ•π‘‰πœ•π‘¬π’Žπ’„(𝑬)| , π’–π’Žπ’‚π’™)= 𝑓 (𝑬,πœ•π‘‰πœ•π‘¬, π’–π’Žπ’‚π’™) (8) Substituting the drift and diffusion component in Equation (2) and the control force in Equation (6), the dynamic equation becomes a function of the energy and value function  πœ†= min(𝐿 (𝑬, 𝑓 (𝑬,πœ•π‘‰πœ•π‘¬))+πœ•π‘‰πœ•π‘¬π’Ž(𝑬, 𝑓 (𝑬,πœ•π‘‰πœ•π‘¬))+12πœ•2π‘‰πœ•π‘¬2𝝈𝟐(𝑬)) (9) Equation (9) is a nonlinear differential equation that can be solved iteratively using Newton’s method (Miranda and Fackler, 2002). In this method, the value function, 𝑉, is discretized to   𝑉 =βˆ‘π‘π‘—βˆ…π‘—(𝑬)𝑛𝑗=1 (10) where βˆ…1,… βˆ…π‘› are Gaussian Quadrature basis functions and 𝑐1,… 𝑐𝑛 are the basis function coefficients. Then, Equation (10) is substituted into Equation (9), and the basis coefficients are computed iteratively with respect to an error tolerance criterion. Finally, the value function is determined as a function of energy and the control force vector in Equation (8) is calculated. The resulting control force is then applied to the  nonlinear system.  3. NUMERICAL EXAMPLE To demonstrate the proposed methodology in section 2, a 1D hysteretic column subjected to horizontal Gaussian white noises is considered. Such a scenario can be a representation of a bridge structure subjected to lateral excitations such as winds or earthquakes, as shown in Figure (1). The equation of motion of the controlled hysteretic column is  ?̈? + 2πœ‰?Μ‡? + (𝛼 βˆ’ π‘˜1)π‘₯= πœ‡(𝑑) βˆ’ (1 βˆ’ 𝛼)𝑧+ 𝑒  (11) where π‘₯ and 𝑧 are the linear and hysteretic displacements, respectively. 𝑒 is the control force. The parameter, 𝛼, is the ratio of the post-yield to pre-yield stiffness of the column; for 𝛼 equals 1, the column is perfectly elastic, while for 𝛼 equals 0, the column is perfectly plastic. πœ‰ and π‘˜1 are the damping and buckling constants, respectively. The external excitation, πœ‡ (=√2𝐷1𝑑𝐡/𝑑𝑑 ), denotes the white noise with covariance, 2𝐷1. The displacement, 𝑧, is introduced to characterize hysteretic behavior of the column using Bouc-Wen model (Ismail et al., 2009). The corresponding nonlinear first order differential equation is 12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015  4  zΜ‡ = A?Μ‡? βˆ’ Ξ²|?Μ‡?|𝑧|𝑧|nβˆ’1 βˆ’ Ξ³?Μ‡?|𝑧|n (12) where A (= 1), Ξ³ (= 0.5),  Ξ² (= 0.5),  and n (= 1) are fitting parameters that control the shape of the hysteresis loop. The smoothness of the curve is controlled by n, the general slope is controlled by (Ξ³ + Ξ²), and the slenderness is controlled by Ξ². For n and A equal 1 and Ξ² equals Ξ³, Equation (12) reduces to 𝑑𝑧𝑑π‘₯= 1 βˆ’ γ𝑧 βˆ’ Ξ³|𝑧| (13) Solving Equation (13), the hysteretic displacement is derived as 𝑧(π‘₯)= {π‘₯ + π‘₯0 βˆ’π‘Ž ≀ π‘₯ ≀ βˆ’π‘₯012Ξ³[1 βˆ’ 𝑒2Ξ³(π‘₯+π‘₯0)] βˆ’π‘₯0 ≀ π‘₯ ≀ π‘Ž (14) where the parameter, π‘Ž, is the amplitude of the displacement, and is computed at zero kinetic energy. The residual displacement, π‘₯0, is determined by solving for the roots of 𝑧(π‘Ž) = βˆ’π‘§(βˆ’π‘Ž).  (a)  (b)  (c) Figure 1: Cantilever hysteretic column subjected to a lateral load (a) uncontrolled (b) equipped with actuator, and (c) Bridge equipped with actuator between span and abutment.   The equivalent nonlinear stochastic system of Equation (11) can now be represented as ?̈? + (2𝛿(𝐸) + 2πœ‰)?Μ‡? +π‘‘π‘ˆ(π‘₯)𝑑π‘₯+ 𝑒= πœ‡ (15) where the Hamilton energy component, 𝐸, is 𝐸 =?Μ‡?22+ (𝛼 βˆ’ π‘˜1)π‘₯22+ (1 βˆ’ 𝛼)𝑧22 (16) where the first component is the kinetic energy, 𝑉(π‘₯), and the second and third components denote the potential energy, π‘ˆ(π‘₯). The nonlinear damping, 2𝛿(𝐸), is defined as 2𝛿(𝐸) =π΄π‘Ÿ2∫ √2𝐸 βˆ’ 2π‘ˆ(π‘₯)π‘Žβˆ’π‘Žπ‘‘π‘₯ (17) In this equation, π΄π‘Ÿ, is the area of hysteresis loop π΄π‘Ÿ = ∫[(1 βˆ’ 𝛼)π‘§π‘Žβˆ’π‘Ž+ (𝛼 βˆ’ π‘˜1)π‘₯]?Μ‡?<0+ [(1 βˆ’ 𝛼)𝑧+ (𝛼 βˆ’ π‘˜1)π‘₯]?Μ‡?>0𝑑π‘₯ (18) Substituting Equation (14) in Equation (18),  π΄π‘Ÿ is π΄π‘Ÿ = (1 βˆ’ 𝛼) [π‘₯0Ξ³βˆ’ (π‘Ž βˆ’ π‘₯0)2] (19) Applying Ito’s rule to the energy component in Equation (16) and stochastic averaging of energy envelope of Equation (15), the resultant averaged Ito equation is derived as 𝑑𝐸 = π‘š(𝐸, 𝑒)π‘‘πœ + 𝜎(𝐸)𝑑𝐡(𝑑) (20) where the drift component, π‘š(𝐸, 𝑒), and the diffusion component, 𝜎(𝐸), are defined as π‘š(𝐸, 𝑒)=1𝑇(𝐸)[βˆ’π΄π‘Ÿβˆ’ 4πœ‰ ∫√2𝐸 βˆ’ 2π‘ˆ(π‘₯)π‘Žβˆ’π‘Žπ‘‘π‘₯] + 𝐷1+πœ•πΈπœ•?Μ‡?𝑒 (21) 12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015  5  𝜎2(𝐸) =2𝐷1 ∫ √2𝐸 βˆ’ 2π‘ˆ(π‘₯)π‘Žβˆ’π‘Žπ‘‘π‘₯𝑇(𝐸) (22) In these equations, the averaged time,  𝑇(𝐸), is  𝑇(𝐸) = 2 ∫1√2𝐸 βˆ’ 2π‘ˆ(π‘₯)π‘Žβˆ’π‘Žπ‘‘π‘₯ (23) For the control design, the cost function, 𝐿(𝐸, 𝑒), in Equation (5) is defined as  𝐿(𝐸, 𝑒) = 𝑠 Γ— 𝐻 + 𝑅𝑒2 (24) where the parameters, 𝑠 and R, are positive constants representing the gain for energy and control force, respectively. The HJB equation is  πœ† = min(𝐿(𝐸, 𝑒) +πœ•π‘‰πœ•πΈπ‘š(𝐸, 𝑒)+12πœ•2π‘‰πœ•πΈ2𝜎2(𝐸)) (25) and the control force of the CSC algorithm is  |𝑒|= min (|12π‘…βˆ’1πœ•π‘‰πœ•πΈπ‘šπ‘(𝐸)| , π‘’π‘šπ‘Žπ‘₯) (26) The HJB equation in Equation (25) is solved iteratively to derive the value function, 𝑉, as discussed earlier in section 2. Then, the control force is computed from Equation (26) and applied to the system in Equation (11).  4. RESULTS The stationary transition probability density (STPD) of the controlled system can be determined analytically by solving Fokker-Planck-Kolmogorrov differential equation as follows 0=πœ•[π‘š(𝐸, 𝑒) Γ— STPD]πœ•πΈ+12πœ•2[𝜎2(𝐸) Γ— STPD]πœ•πΈ2 (27) Solving Equation (27) yields STPD=𝐢𝜎2(𝐸)exp (∫2 Γ—π‘š(𝐸, 𝑒)𝜎2(𝐸)𝐸0) (28) where the constant 𝐢 can be obtained using the fact that the integration of STPD over the entire domain is equal to 1: 𝐢 =𝜎2(𝐸)exp (∫2 Γ— π‘š(𝐸, 𝑒)𝜎2(𝐸)∞0) (29) The parameter values of the hysteretic column are set to πœ‰ = 0.025, 𝛼 = 0.5, and π‘˜1 = 0.04. For this dynamic system, STPD is calculated using Equation (28), for both the uncontrolled (𝑒 = 0) and controlled systems, and the results are shown in Figure (2). The STPD plot of the controlled system (Figure (2b)) is skewed further to the left compared to that of the uncontrolled system indicating a significant reduction in the response of the system. In addition, the mean energy of the controlled system is 0.25, while the mean energy of the uncontrolled structure is 2.57. For the evaluation of the control design, Monte Carlo simulations of the hysteretic column under Gaussian noise of different intensities (𝐷1 = 0.5 & 𝐷1 = 1) are conducted for the uncontrolled, Cl-SC and CSC strategies (Table 1). The same cost function in Equation (24) is considered for both controllers. The force capacity of the actuator is considered to be a percentage of the inertial force (π‘’π‘šπ‘Žπ‘₯ = Ω βˆ— max?̈?𝑒𝑛𝑐), where Ω is a variable ranging from 0 to 1 and max?̈?𝑒𝑛𝑐  is the maximum inertial force for the uncontrolled case at 𝐷1 = 0.5. In comparison to Cl-SC, CSC (𝑠 = 5, 𝑅 = 1) provides more reduction in the root mean square (rmsπ‘₯) and peak (maxπ‘₯) values of displacement: for 𝐷1 = 1, 7.4% reduction in maxπ‘₯ and 8.6% reduction in rmsπ‘₯ are achieved as shown in Table 1 and Figure 3. Similar results are observed for different excitations in Table 1.  12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015  6  5. CONCLUSION In this paper, a bounded control algorithm is suggested based on stochastic optimal control. This algorithm is designed through stochastic averaging of Hamiltonian systems, which reduces the dimension of the HJB equation. In addition, it considers the constraints of active devices, which has a maximum force embedded in the HJB equation in order to provide better optimal solutions. The control algorithm is compared to the clipped optimal strategy and it is shown to provide better control performance at different excitation intensities considered in the analysis. This design may be promising in solving constrained nonlinear problems.   Table 1: Response results for uncontrolled (UNC), Clipped Stochastic Control (Cl-SC), and Bounded Stochastic Control (CSC) 𝐷1 = 0.5 ; 𝑠 = 5; π‘’π‘šπ‘Žπ‘₯ = 3; 𝑅 = 1 Response maxπ‘₯    rmsπ‘₯ max?̈?   UNC  8.34 2.26 4.36 Cl-SC  2.03 0.62 3.95 CSC 1.86 0.56 3.99 𝐷1 = 1 ; 𝑠 = 5; π‘’π‘šπ‘Žπ‘₯ = 3; 𝑅 = 1 Response maxπ‘₯    π‘Ÿπ‘šπ‘ π‘₯ max?̈?   UNC  12.98 3.55 6.50 Cl-SC  3.11 0.93 4.69 CSC 2.88 0.85 4.67 𝐷1 = 0.5 ; 𝑠 = 6; π‘’π‘šπ‘Žπ‘₯ = 3; 𝑅 = 1 Response maxπ‘₯    π‘Ÿπ‘šπ‘ π‘₯ max?̈?   UNC  8.34 2.26 4.36 Cl-SC  2.03 0.60 4.21 CSC 1.77 0.53 4.23 𝐷1 = 1 ; 𝑠 = 6; π‘’π‘šπ‘Žπ‘₯ = 3; 𝑅 = 1 Response maxπ‘₯    π‘Ÿπ‘šπ‘ π‘₯ max?̈?   UNC  12.98 3.55 6.50 Cl-SC  3.14 0.90 4.89 CSC 2.84 0.80 4.79   (a)        (b) Figure 2: Stationary Transition Probability Density (STPD) versus Energy of the hysteretic column (a) UNC and (b) Unbounded Stochastic Control (USC) - 𝐷1 = 0.5 ; 𝑠 = 5.    (a)  (b) Figure 3: Time History Simulation of displacement of hysteretic column for (a) UNC and (b) Stochastic Control approaches - 𝐷1 =1, 𝑠 = 5.    12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015  7   6. REFERENCES Alavinasab, A., Moharrami, H., & Khajepour, A. (2006). β€œActive Control of Structures Using Energy‐Based LQR Method.” Computer‐Aided Civil and Infrastructure Engineering, 21(8), 605-611. Balikcioglu, M., Fackler, P. L., & Pindyck, R. S. (2011). β€œSolving optimal timing problems in environmental economics.” Resource and Energy Economics,33(3), 761-768. Daigneault, A. J., Miranda, M. J., & Sohngen, B. (2010). Optimal forest management with carbon sequestration credits and endogenous fire risk. Land Economics, 86(1), 155-172. Fackler, P. L. (2005). β€œA MATLAB solver for nonlinear rational expectations models.” Computational Economics, 26(2), 173-181. Fan, Y. C., Loh, C. H., Yang, J. N., & Lin, P. Y. (2009). β€œExperimental performance evaluation of an equipment isolation using MR dampers.” Earthquake Engineering & Structural Dynamics, 38(3), 285-305. Fleming, W.H. and Soner, H.M. (2006), β€œControlled Markov Processes and Viscosity Solutions”,  Springer, 2nd Edition. Gu, X. D., Zhu, W. Q., & Xu, W. (2012). β€œStochastic optimal control of quasi non-integrable Hamiltonian systems with stochastic maximum principle.” Nonlinear Dynamics, 70(1), 779-787. Ismail, M., Ikhouane, F., & Rodellar, J. (2009). β€œThe hysteresis Bouc-Wen model, a survey.” Archives of Computational Methods in Engineering, 16(2), 161-188. Jia, W., Zhu, W., & Xu, Y. (2013). β€œStochastic averaging of quasi-non-integrable Hamiltonian systems under combined Gaussian and Poisson white noise excitations.” International Journal of Non-Linear Mechanics, 51, 45-53. Kim, Y. M., You, K. P., You, J. Y., Paek, S. Y., & Nam, B. H. (2013). β€œLQR Control of Along-Wind Response of a Tall Building.” Applied Mechanics and Materials, 421, 767-771. Lin, Y. K., & Cai, G. Q. (1995). β€œProbabilistic structural dynamics: advanced theory and applications.” New York: McGraw-Hill. Ortigueira, S. (2006). β€œMarkov-perfect optimal taxation.” Review of Economic Dynamics, 9(1), 153-178. Wang, Y., Ying, Z. G., & Zhu, W. Q. (2009). β€œRobustness of non-linear stochastic optimal control for quasi-Hamiltonian systems with parametric uncertainty.” International Journal of Systems Science, 40(12), 1217-1227. Zapateiro, M., Karimi, H. R., Luo, N., & Spencer, B. F. (2010). β€œReal‐time hybrid testing of semiactive control strategies for vibration reduction in a structure with MR damper.” Structural Control and Health Monitoring, 17(4), 427-451. Zhu, W. Q. (2006). β€œNonlinear stochastic dynamics and control in Hamiltonian formulation.” Applied Mechanics Reviews, 59(4), 230-248. Zhu, W. Q., Huang, Z. L., & Yang, Y. Q. (1997). β€œStochastic averaging of quasi-integrable Hamiltonian systems.” Journal of applied mechanics, 64(4), 975-984. Zhu, W. Q., Ying, Z. G., Ni, Y. Q., & Ko, J. M. (2000). β€œOptimal nonlinear stochastic control of hysteretic systems.” Journal of engineering mechanics,126(10), 1027-1032.  


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items