12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015 1 A Constrained Nonlinear Stochastic Optimal Control for Dynamic Systems Omar El-Khoury Graduate Student, Dept. of Civil, Environmental, & Geodetic Engineering, Ohio State University, Columbus, USA Abdollah Shafieezadeh Assistant Professor, Dept. of Civil, Environmental, & Geodetic Engineering, Ohio State University, Columbus, USA ABSRTACT: An ideal controller assumes that the system is unconstrained and the control force in unbounded. However, in reality, control devices are restricted by their force capacity. Traditionally, the clipping strategy has been used extensively, where an ideal actuator is assumed in the control design, and then the inequality constraints are enforced through saturation. This approach may not provide optimal solutions since constraints are not considered in the control optimization. To overcome this limitation, this paper presents a constrained nonlinear stochastic optimal control algorithm for dynamic systems subjected to Gaussian white noise excitations. In this control algorithm, stochasticity and nonlinearity of a Hamiltonian dynamic system is considered based on stochastic averaging of energy envelope using Markovian approximation. An Ito equation of energy envelope is derived and represented by diffusion and drift components. For the control design, a prescribed cost function, the diffusion and drift components together with the force constraints are considered in solving the Hamilton Jacobian Bellman (HJB) equation. This proposed control approach is called here Constrained Stochastic Control (CSC). The performance of the CSC algorithm is demonstrated for a hysteretic column and the results are compared to simulation results for Clipped Stochastic Control (Cl-SC) and uncontrolled cases. Noticeable improvements in peak and root mean square values of displacement in the CSC case are observed over the Cl-SC algorithm. 1. INTRODUCTION In control engineering, active devices ideally can apply any force of any scale. However, an active device such as an actuator is restricted by a maximum force which cannot be exceeded. In conventional control methods e.g. clipped linear quadratic regulator (LQR), the controller is first designed assuming unboundedness, and then the constraints are enforced through saturation. This approach has been shown to yield good results in civil engineering structures such as multi-story buildings or multi-span bridges subjected to severe winds or earthquakes (Fan et al., 2009). But clipped-based methods may result in suboptimal and sometimes far from optimal solutions especially in highly nonlinear systems where the control design is based on an ideal linear controller (Zapateiro et al., 2010; Alavinasab et al., 2006). To overcome the shortcomings of clipped optimal approaches, a Constrained Stochastic Control (CSC) algorithm is proposed here for nonlinear dynamic systems. This method is based on stochastic averaging of the total energy of the system (Zhu et al., 1997; Jia et al., 2013). The energy 12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015 2 component is assumed to be a slow varying process as compared to the displacement and velocity. Consideration of energy allows for a better characterization of nonlinearity and stochasticity in the controlled system (Zhu, 2006; Wang et al., 2009; Gu et al., 2012). In this approach, the control design, based on stochastic averaging, is derived as the solution of a nonlinear dynamic equation, called Hamilton Jacobian Bellman (HJB) equation (Gu et al., 2012). To solve this equation, an iterative strategy based on Newtonβs method developed by Miranda and Fackler (2002) is used. The method has been extensively investigated for solving economics and finance problems (Fackler, 2005; Ortigueira, 2006; Daigneault et al., 2010; Balikcioglu et al., 2011). Applying this method to engineering, the constraint of the actuator is embedded in the iterative process to solve the dynamic equation and compute the optimal control force. In this paper, a hysteretic column subjected to Gaussian white noises is analyzed to assess the performance of CSC algorithm and compare it with both the corresponding uncontrolled system and the system equipped with Clipped Stochastic Control (Cl-SC) algorithm. 2. METHODOLOGY In commonly used control algorithms, the control design is based on a deterministic excitation and on a state space linearization of nonlinear systems (Zapateiro et al., 2010; Kim et al., 2013). However, in civil engineering applications, earthquake or wind excitations are considered stochastic with characteristics that are not fully known in prior. But these hazards can be modelled using techniques such as filters that are applied to the Gaussian white noise. In this paper, stochastic averaging of energy envelope is used for control design in order to characterize stochasticity and nonlinearity in the system. Focusing on system behavior, a dynamic system can be described by Hamilton energy (π¬) which consists of kinetic energy (π²) and potential energy (π·) π¬ = π² + π· (1) The differential equation of the energy component is defined as π π¬ = π(π¬, π)ππ‘ + π(π¬)π π© (2) where π(π¬, π), π(π¬), π, and π© are the drift and diffusive components, control vector, and a Gaussian white noise process, respectively. For an affine controlled dynamic system, the drift component is represented as π(π¬, π) = (ππ(π¬) +ππ(π¬) β π) |π| < ππππ (3) where ππ is the drift component for the uncontrolled case, ππ is the additional drift component for the controlled case, and ππππ is the maximum control force vector. For control design, an infinite horizon performance index for a stationary ergodic system is considered π½ = limπ‘πΉββ1π‘πΉπΈβ« πΏ(π¬, π)π‘πΉ0 (4) where πΏ(π¬, π) denotes the cost function defined as πΏ(π¬, π) = π»(π¬) + ππ»πΉπ (5) where π» is a cost function of energy components and πΉ is the positive definite covariance matrix for the control force vector. According to Fleming and Soner (2006), the corresponding dynamic programming equation for finite time horizon is derived as 12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015 3 π = min(πΏ(π¬, π) +ππππ¬π(π¬, π)+12π2πππ¬2ππ(π¬)) (6) where the parameter, π, is the optimal averaged cost function. In this equation, the value function, π, plays the role of a Lagrange multiplier to enforce the equality constraint in Equation (2). Initially, the unbounded control force is derived by differentiating Equation (6) and equating it to zero, ππ/ππ = π. The resulting control force π =12πΉβ1ππππ¬ππ(π¬) (7) In order to incorporate the boundedness of the control force, the control vector is modified to |π|= min (|12πΉβ1ππππ¬ππ(π¬)| , ππππ)= π (π¬,ππππ¬, ππππ) (8) Substituting the drift and diffusion component in Equation (2) and the control force in Equation (6), the dynamic equation becomes a function of the energy and value function π= min(πΏ (π¬, π (π¬,ππππ¬))+ππππ¬π(π¬, π (π¬,ππππ¬))+12π2πππ¬2ππ(π¬)) (9) Equation (9) is a nonlinear differential equation that can be solved iteratively using Newtonβs method (Miranda and Fackler, 2002). In this method, the value function, π, is discretized to π =βππβ π(π¬)ππ=1 (10) where β 1,β¦ β π are Gaussian Quadrature basis functions and π1,β¦ ππ are the basis function coefficients. Then, Equation (10) is substituted into Equation (9), and the basis coefficients are computed iteratively with respect to an error tolerance criterion. Finally, the value function is determined as a function of energy and the control force vector in Equation (8) is calculated. The resulting control force is then applied to the nonlinear system. 3. NUMERICAL EXAMPLE To demonstrate the proposed methodology in section 2, a 1D hysteretic column subjected to horizontal Gaussian white noises is considered. Such a scenario can be a representation of a bridge structure subjected to lateral excitations such as winds or earthquakes, as shown in Figure (1). The equation of motion of the controlled hysteretic column is ?Μ? + 2π?Μ? + (πΌ β π1)π₯= π(π‘) β (1 β πΌ)π§+ π’ (11) where π₯ and π§ are the linear and hysteretic displacements, respectively. π’ is the control force. The parameter, πΌ, is the ratio of the post-yield to pre-yield stiffness of the column; for πΌ equals 1, the column is perfectly elastic, while for πΌ equals 0, the column is perfectly plastic. π and π1 are the damping and buckling constants, respectively. The external excitation, π (=β2π·1ππ΅/ππ‘ ), denotes the white noise with covariance, 2π·1. The displacement, π§, is introduced to characterize hysteretic behavior of the column using Bouc-Wen model (Ismail et al., 2009). The corresponding nonlinear first order differential equation is 12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015 4 zΜ = A?Μ? β Ξ²|?Μ?|π§|π§|nβ1 β Ξ³?Μ?|π§|n (12) where A (= 1), Ξ³ (= 0.5), Ξ² (= 0.5), and n (= 1) are fitting parameters that control the shape of the hysteresis loop. The smoothness of the curve is controlled by n, the general slope is controlled by (Ξ³ + Ξ²), and the slenderness is controlled by Ξ². For n and A equal 1 and Ξ² equals Ξ³, Equation (12) reduces to ππ§ππ₯= 1 β Ξ³π§ β Ξ³|π§| (13) Solving Equation (13), the hysteretic displacement is derived as π§(π₯)= {π₯ + π₯0 βπ β€ π₯ β€ βπ₯012Ξ³[1 β π2Ξ³(π₯+π₯0)] βπ₯0 β€ π₯ β€ π (14) where the parameter, π, is the amplitude of the displacement, and is computed at zero kinetic energy. The residual displacement, π₯0, is determined by solving for the roots of π§(π) = βπ§(βπ). (a) (b) (c) Figure 1: Cantilever hysteretic column subjected to a lateral load (a) uncontrolled (b) equipped with actuator, and (c) Bridge equipped with actuator between span and abutment. The equivalent nonlinear stochastic system of Equation (11) can now be represented as ?Μ? + (2πΏ(πΈ) + 2π)?Μ? +ππ(π₯)ππ₯+ π’= π (15) where the Hamilton energy component, πΈ, is πΈ =?Μ?22+ (πΌ β π1)π₯22+ (1 β πΌ)π§22 (16) where the first component is the kinetic energy, π(π₯), and the second and third components denote the potential energy, π(π₯). The nonlinear damping, 2πΏ(πΈ), is defined as 2πΏ(πΈ) =π΄π2β« β2πΈ β 2π(π₯)πβπππ₯ (17) In this equation, π΄π, is the area of hysteresis loop π΄π = β«[(1 β πΌ)π§πβπ+ (πΌ β π1)π₯]?Μ?<0+ [(1 β πΌ)π§+ (πΌ β π1)π₯]?Μ?>0ππ₯ (18) Substituting Equation (14) in Equation (18), π΄π is π΄π = (1 β πΌ) [π₯0Ξ³β (π β π₯0)2] (19) Applying Itoβs rule to the energy component in Equation (16) and stochastic averaging of energy envelope of Equation (15), the resultant averaged Ito equation is derived as ππΈ = π(πΈ, π’)ππ + π(πΈ)ππ΅(π‘) (20) where the drift component, π(πΈ, π’), and the diffusion component, π(πΈ), are defined as π(πΈ, π’)=1π(πΈ)[βπ΄πβ 4π β«β2πΈ β 2π(π₯)πβπππ₯] + π·1+ππΈπ?Μ?π’ (21) 12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015 5 π2(πΈ) =2π·1 β« β2πΈ β 2π(π₯)πβπππ₯π(πΈ) (22) In these equations, the averaged time, π(πΈ), is π(πΈ) = 2 β«1β2πΈ β 2π(π₯)πβπππ₯ (23) For the control design, the cost function, πΏ(πΈ, π’), in Equation (5) is defined as πΏ(πΈ, π’) = π Γ π» + π π’2 (24) where the parameters, π and R, are positive constants representing the gain for energy and control force, respectively. The HJB equation is π = min(πΏ(πΈ, π’) +ππππΈπ(πΈ, π’)+12π2πππΈ2π2(πΈ)) (25) and the control force of the CSC algorithm is |π’|= min (|12π β1ππππΈππ(πΈ)| , π’πππ₯) (26) The HJB equation in Equation (25) is solved iteratively to derive the value function, π, as discussed earlier in section 2. Then, the control force is computed from Equation (26) and applied to the system in Equation (11). 4. RESULTS The stationary transition probability density (STPD) of the controlled system can be determined analytically by solving Fokker-Planck-Kolmogorrov differential equation as follows 0=π[π(πΈ, π’) Γ STPD]ππΈ+12π2[π2(πΈ) Γ STPD]ππΈ2 (27) Solving Equation (27) yields STPD=πΆπ2(πΈ)exp (β«2 Γπ(πΈ, π’)π2(πΈ)πΈ0) (28) where the constant πΆ can be obtained using the fact that the integration of STPD over the entire domain is equal to 1: πΆ =π2(πΈ)exp (β«2 Γ π(πΈ, π’)π2(πΈ)β0) (29) The parameter values of the hysteretic column are set to π = 0.025, πΌ = 0.5, and π1 = 0.04. For this dynamic system, STPD is calculated using Equation (28), for both the uncontrolled (π’ = 0) and controlled systems, and the results are shown in Figure (2). The STPD plot of the controlled system (Figure (2b)) is skewed further to the left compared to that of the uncontrolled system indicating a significant reduction in the response of the system. In addition, the mean energy of the controlled system is 0.25, while the mean energy of the uncontrolled structure is 2.57. For the evaluation of the control design, Monte Carlo simulations of the hysteretic column under Gaussian noise of different intensities (π·1 = 0.5 & π·1 = 1) are conducted for the uncontrolled, Cl-SC and CSC strategies (Table 1). The same cost function in Equation (24) is considered for both controllers. The force capacity of the actuator is considered to be a percentage of the inertial force (π’πππ₯ = β¦ β max?Μ?π’ππ), where β¦ is a variable ranging from 0 to 1 and max?Μ?π’ππ is the maximum inertial force for the uncontrolled case at π·1 = 0.5. In comparison to Cl-SC, CSC (π = 5, π = 1) provides more reduction in the root mean square (rmsπ₯) and peak (maxπ₯) values of displacement: for π·1 = 1, 7.4% reduction in maxπ₯ and 8.6% reduction in rmsπ₯ are achieved as shown in Table 1 and Figure 3. Similar results are observed for different excitations in Table 1. 12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015 6 5. CONCLUSION In this paper, a bounded control algorithm is suggested based on stochastic optimal control. This algorithm is designed through stochastic averaging of Hamiltonian systems, which reduces the dimension of the HJB equation. In addition, it considers the constraints of active devices, which has a maximum force embedded in the HJB equation in order to provide better optimal solutions. The control algorithm is compared to the clipped optimal strategy and it is shown to provide better control performance at different excitation intensities considered in the analysis. This design may be promising in solving constrained nonlinear problems. Table 1: Response results for uncontrolled (UNC), Clipped Stochastic Control (Cl-SC), and Bounded Stochastic Control (CSC) π·1 = 0.5 ; π = 5; π’πππ₯ = 3; π = 1 Response maxπ₯ rmsπ₯ max?Μ? UNC 8.34 2.26 4.36 Cl-SC 2.03 0.62 3.95 CSC 1.86 0.56 3.99 π·1 = 1 ; π = 5; π’πππ₯ = 3; π = 1 Response maxπ₯ πππ π₯ max?Μ? UNC 12.98 3.55 6.50 Cl-SC 3.11 0.93 4.69 CSC 2.88 0.85 4.67 π·1 = 0.5 ; π = 6; π’πππ₯ = 3; π = 1 Response maxπ₯ πππ π₯ max?Μ? UNC 8.34 2.26 4.36 Cl-SC 2.03 0.60 4.21 CSC 1.77 0.53 4.23 π·1 = 1 ; π = 6; π’πππ₯ = 3; π = 1 Response maxπ₯ πππ π₯ max?Μ? UNC 12.98 3.55 6.50 Cl-SC 3.14 0.90 4.89 CSC 2.84 0.80 4.79 (a) (b) Figure 2: Stationary Transition Probability Density (STPD) versus Energy of the hysteretic column (a) UNC and (b) Unbounded Stochastic Control (USC) - π·1 = 0.5 ; π = 5. (a) (b) Figure 3: Time History Simulation of displacement of hysteretic column for (a) UNC and (b) Stochastic Control approaches - π·1 =1, π = 5. 12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12 Vancouver, Canada, July 12-15, 2015 7 6. REFERENCES Alavinasab, A., Moharrami, H., & Khajepour, A. (2006). βActive Control of Structures Using EnergyβBased LQR Method.β ComputerβAided Civil and Infrastructure Engineering, 21(8), 605-611. Balikcioglu, M., Fackler, P. L., & Pindyck, R. S. (2011). βSolving optimal timing problems in environmental economics.β Resource and Energy Economics,33(3), 761-768. Daigneault, A. J., Miranda, M. J., & Sohngen, B. (2010). Optimal forest management with carbon sequestration credits and endogenous fire risk. Land Economics, 86(1), 155-172. Fackler, P. L. (2005). βA MATLAB solver for nonlinear rational expectations models.β Computational Economics, 26(2), 173-181. Fan, Y. C., Loh, C. H., Yang, J. N., & Lin, P. Y. (2009). βExperimental performance evaluation of an equipment isolation using MR dampers.β Earthquake Engineering & Structural Dynamics, 38(3), 285-305. Fleming, W.H. and Soner, H.M. (2006), βControlled Markov Processes and Viscosity Solutionsβ, Springer, 2nd Edition. Gu, X. D., Zhu, W. Q., & Xu, W. (2012). βStochastic optimal control of quasi non-integrable Hamiltonian systems with stochastic maximum principle.β Nonlinear Dynamics, 70(1), 779-787. Ismail, M., Ikhouane, F., & Rodellar, J. (2009). βThe hysteresis Bouc-Wen model, a survey.β Archives of Computational Methods in Engineering, 16(2), 161-188. Jia, W., Zhu, W., & Xu, Y. (2013). βStochastic averaging of quasi-non-integrable Hamiltonian systems under combined Gaussian and Poisson white noise excitations.β International Journal of Non-Linear Mechanics, 51, 45-53. Kim, Y. M., You, K. P., You, J. Y., Paek, S. Y., & Nam, B. H. (2013). βLQR Control of Along-Wind Response of a Tall Building.β Applied Mechanics and Materials, 421, 767-771. Lin, Y. K., & Cai, G. Q. (1995). βProbabilistic structural dynamics: advanced theory and applications.β New York: McGraw-Hill. Ortigueira, S. (2006). βMarkov-perfect optimal taxation.β Review of Economic Dynamics, 9(1), 153-178. Wang, Y., Ying, Z. G., & Zhu, W. Q. (2009). βRobustness of non-linear stochastic optimal control for quasi-Hamiltonian systems with parametric uncertainty.β International Journal of Systems Science, 40(12), 1217-1227. Zapateiro, M., Karimi, H. R., Luo, N., & Spencer, B. F. (2010). βRealβtime hybrid testing of semiactive control strategies for vibration reduction in a structure with MR damper.β Structural Control and Health Monitoring, 17(4), 427-451. Zhu, W. Q. (2006). βNonlinear stochastic dynamics and control in Hamiltonian formulation.β Applied Mechanics Reviews, 59(4), 230-248. Zhu, W. Q., Huang, Z. L., & Yang, Y. Q. (1997). βStochastic averaging of quasi-integrable Hamiltonian systems.β Journal of applied mechanics, 64(4), 975-984. Zhu, W. Q., Ying, Z. G., Ni, Y. Q., & Ko, J. M. (2000). βOptimal nonlinear stochastic control of hysteretic systems.β Journal of engineering mechanics,126(10), 1027-1032.
- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- International Conference on Applications of Statistics and Probability in Civil Engineering (ICASP) (12th : 2015) /
- A constrained nonlinear stochastic optimal control...
Open Collections
International Conference on Applications of Statistics and Probability in Civil Engineering (ICASP) (12th : 2015)
A constrained nonlinear stochastic optimal control for dynamic systems El-Khoury, Omar; Shafieezadeh, Abdollah 2015-07
pdf
Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.
Page Metadata
Item Metadata
Title | A constrained nonlinear stochastic optimal control for dynamic systems |
Creator |
El-Khoury, Omar Shafieezadeh, Abdollah |
Contributor |
International Conference on Applications of Statistics and Probability (12th : 2015 : Vancouver, B.C.) |
Date Issued | 2015-07 |
Description | An ideal controller assumes that the system is unconstrained and the control force in unbounded. However, in reality, control devices are restricted by their force capacity. Traditionally, the clipping strategy has been used extensively, where an ideal actuator is assumed in the control design, and then the inequality constraints are enforced through saturation. This approach may not provide optimal solutions since constraints are not considered in the control optimization. To overcome this limitation, this paper presents a constrained nonlinear stochastic optimal control algorithm for dynamic systems subjected to Gaussian white noise excitations. In this control algorithm, stochasticity and nonlinearity of a Hamiltonian dynamic system is considered based on stochastic averaging of energy envelope using Markovian approximation. An Ito equation of energy envelope is derived and represented by diffusion and drift components. For the control design, a prescribed cost function, the diffusion and drift components together with the force constraints are considered in solving the Hamilton Jacobian Bellman (HJB) equation. This proposed control approach is called here Constrained Stochastic Control (CSC). The performance of the CSC algorithm is demonstrated for a hysteretic column and the results are compared to simulation results for Clipped Stochastic Control (Cl-SC) and uncontrolled cases. Noticeable improvements in peak and root mean square values of displacement in the CSC case are observed over the Cl-SC algorithm. |
Genre |
Conference Paper |
Type |
Text |
Language | eng |
Notes | This collection contains the proceedings of ICASP12, the 12th International Conference on Applications of Statistics and Probability in Civil Engineering held in Vancouver, Canada on July 12-15, 2015. Abstracts were peer-reviewed and authors of accepted abstracts were invited to submit full papers. Also full papers were peer reviewed. The editor for this collection is Professor Terje Haukaas, Department of Civil Engineering, UBC Vancouver. |
Date Available | 2015-05-21 |
Provider | Vancouver : University of British Columbia Library |
Rights | Attribution-NonCommercial-NoDerivs 2.5 Canada |
DOI | 10.14288/1.0076207 |
URI | http://hdl.handle.net/2429/53342 |
Affiliation |
Non UBC |
Citation | Haukaas, T. (Ed.) (2015). Proceedings of the 12th International Conference on Applications of Statistics and Probability in Civil Engineering (ICASP12), Vancouver, Canada, July 12-15. |
Peer Review Status | Unreviewed |
Scholarly Level | Faculty Researcher |
Rights URI | http://creativecommons.org/licenses/by-nc-nd/2.5/ca/ |
AggregatedSourceRepository | DSpace |
Download
- Media
- 53032-paper_437_ElKhoury.pdf [ 587.01kB ]
- Metadata
- JSON: 53032-1.0076207.json
- JSON-LD: 53032-1.0076207-ld.json
- RDF/XML (Pretty): 53032-1.0076207-rdf.xml
- RDF/JSON: 53032-1.0076207-rdf.json
- Turtle: 53032-1.0076207-turtle.txt
- N-Triples: 53032-1.0076207-rdf-ntriples.txt
- Original Record: 53032-1.0076207-source.json
- Full Text
- 53032-1.0076207-fulltext.txt
- Citation
- 53032-1.0076207.ris
Full Text
Cite
Citation Scheme:
Usage Statistics
Share
Embed
Customize your widget with the following options, then copy and paste the code below into the HTML
of your page to embed this item in your website.
<div id="ubcOpenCollectionsWidgetDisplay">
<script id="ubcOpenCollectionsWidget"
src="{[{embed.src}]}"
data-item="{[{embed.item}]}"
data-collection="{[{embed.collection}]}"
data-metadata="{[{embed.showMetadata}]}"
data-width="{[{embed.width}]}"
data-media="{[{embed.selectedMedia}]}"
async >
</script>
</div>
Our image viewer uses the IIIF 2.0 standard.
To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.53032.1-0076207/manifest